Chapter 5&6(6.1 6.2) application

Today, I read Chapter 5&6(6.1 6.2) to find something useful which could be used in my research.

Summary

Chapter 5 Application

Quantitative measures of performance

RMSE: the square root of MSE so that it is in the same units as the original data

: the proportion of the information in the data that is explained by the model. It is a measure of correlation, not accuracy.

The variance-bias trade-off: high variance or high bias, that is a question. Increasing bias can reduce model variance.

Chapter 6 Application

Ordinary linear regression finds parameter estimates that have minimum bias, whereas ridge regression, the lasso, and the elastic net find estimates that have lower variance.

Linear regression:

SSE: sum-of-squared errors.

When there are highly correlated predictors, we should remove one of the offending predictors or find models which can tolerate collinearity.

Tomorrow, I will read more of the book to frame questions.

Chapter 4 application

Today, I read Chapter 4 to find something which could be used in my research.

Summary

Over-fitting

Model tuning: tune an appropriate value of a tuning parameter

Data splitting: maximum dissimilarity sampling

Common steps in model building:

1.      Pre-processing the predictor data

2.      Estimating model parameters

3.      Selecting predictors for the model

4.      Evaluating model performance

5.      Fine tuning class prediction rules (via ROC curves, etc)

Resampling techniques:

1.      K-fold cross validation

2.      Generalized cross validation (approximate the leave-one-out error rate)

For example, in the last leave-one-out cross validation, df=1 and n=12.

3.      Repeated training/testing splits

4.      Bootstrap

Choose final tuning parameters with different considerations of various factors

Choosing between models:
1. Start with several models that are the least interpretable and most flexible.

2. investigate simpler models that are less opaque

3. consider using simplest model that reasonably approximates the performance of the more complex methods

Chapter 3 application 2

Today, I finished reading chapter 3 and found something which could be applied to my research. In addition, I looked into IP to try to know what do all the data mean. I have known some of them basically.

Summary

Tomorrow, I will read chapter 4 to find more information which may be useful for my research.

Chapter 3 application

Today, I browsed the rest of the paper and started to look into the data. There are many ways of data pre-processing in chapter 3 and they can be applied into the log data.

Summary

paper:

Chapter 3:

1. Individual predictors:

Centering: the predictor has a zero mean

Scaling: each value of the predictor variable is divided by its standard deviation. Scaling the data coerce the values to have a common standard deviation of one.

Removing skewness: Box-Cox transformation

After holiday, I will start to frame questions of each chapter and try to apply them into the data.

Preparing for using data

Today, I started to look into data in IP and tried to be familiar with IP software. Also, Yifu gave me a paper which is written by the people from Hess Corporation. I read a little of it.

Summary

SPWLA paper

TOC: total organic carbon

According the paper, the lithology is very complicated. I think if I need to predict NMR permeability with R method, the lithology of the data and the lithology of where I need to predict should be similar.

Tomorrow, I will continue to read the paper and look into the data.

Chapter 10 (Computing finished)

Today, I fixed some problems of computing of Chapter 10.

Codes:
pp2=preProcess(age28data[, 1:7], "pca")
pca1=predict(pp2, age28data[, 1:7])
head(pca1)
pca1$Data="Training Set"
pca1$Data[startpoints]="Starting Values"
pca3=predict(pp2, cbresults[, 1:7])
pca3$Data="Cubist"
head(pca3)
pca4=predict(pp2, nnetresults[, 1:7])
pca4$Data="Neural Network"
head(pca4)
pcadata=rbind(pca1, pca3, pca4)
dim(pcadata)
pcadata$Data=factor(pcadata$Data,
                    levels = c("Training Set","Starting Values",
                    "Cubist","Neural Network"))
lim=extendrange(pcadata[, 1:2])

# convert a vector of strings into a vector of numbers
m=with(pcadata, as.numeric(levels(factor(as.numeric(factor(pcadata$Data))))))
m=with(pcadata, levels(factor(Data)))
m
colm=c()
for(i in 1:length(pcadata$Data))
{
  colm[i]=which(pcadata$Data[i]==m)
}
colm

xyplot(PC2 ~ PC1, data = pcadata, groups = Data,
       auto.key = list(columns = 2),
       xlim = lim, ylim = lim,
       col=colm,
       type = c("g", "p")
      )

xyplot(PC2 ~ PC1, data = pcadata, groups = Data,
       auto.key = list(columns = 2),
       xlim = lim, ylim = lim,
       col=c("blue", "pink", "red", "green"),
       type = c("g", "p")
)

Next week, I will start to look into the NMR data.

Chapter 10 computing

Today, I did computing of Chapter 10.

Codes:
modelPrediction=function(x, mod)
{
  if(x[1] < 0 | x[1] > 1) return(10^38)
  if(x[2] < 0 | x[2] > 1) return(10^38)
  if(x[3] < 0 | x[3] > 1) return(10^38)
  if(x[4] < 0 | x[4] > 1) return(10^38)
  if(x[5] < 0 | x[5] > 1) return(10^38)
  if(x[6] < 0 | x[6] > 1) return(10^38)
  x=c(x, 1 - sum(x))
  if(x[7] < 0.05) return(10^38)
  tmp <- as.data.frame(t(x))
  names(tmp)=c('Cement','BlastFurnaceSlag','FlyAsh',
               'Superplasticizer','CoarseAggregate',
               'FineAggregate', 'Water')
  tmp$Age=28
  -predict(mod, tmp)
}

# Cubist mdoel
cbresults=startingvalues
cbresults$Water=NA
cbresults$Prediction=NA
cbresults
for(i in 1:nrow(cbresults))
{
  results=optim(unlist(cbresults[i,1:6]),
                     modelPrediction,
                     method = "Nelder-Mead",
                    control=list(maxit=5000),
                      mod = cbmodel)
  cbresults$Prediction[i]=-results$value
  # save the final mixture values
  cbresults[i,1:6]=results$par
}
#col.sums <- apply(x, 2, sum)  row.sums <- apply(x, 1, sum)  rbind  cbind
cbresults$Water=1 - apply(cbresults[,1:6], 1, sum)
# Keep the top three mixtures, -: decreasing order
cbresults=cbresults[order(-cbresults$Prediction),][1:3,]
cbresults$Model="Cubist"
cbresults

       Cement BlastFurnaceSlag      FlyAsh Superplasticizer CoarseAggregate FineAggregate      Water Prediction  Model
523 0.1747206       0.17969922 0.012880890     0.0027363548       0.3200175     0.2532112 0.05673427  102.44370 Cubist
495 0.2098331       0.05295034 0.006751067     0.0003692619       0.3538669     0.3238973 0.05233208   86.01767 Cubist
477 0.2110388       0.04514574 0.062017524     0.0023323792       0.3178446     0.3077021 0.05391885   85.51025 Cubist

# neural network model
nnetresults=startingvalues
nnetresults$Water=NA
nnetresults$Prediction=NA
nnetresults
for(i in 1:nrow(nnetresults))
{
  results=optim(unlist(nnetresults[i,1:6]),
                modelPrediction,
                method = "Nelder-Mead",
                control=list(maxit=5000),
                mod = nnetmodel)
  nnetresults$Prediction[i]=-results$value
  nnetresults[i,1:6]=results$par
}
nnetresults$Water=1 - apply(nnetresults[,1:6], 1, sum)
nnetresults=nnetresults[order(-nnetresults$Prediction),][1:3,]
nnetresults$Model="NNet"
nnetresults

       Cement BlastFurnaceSlag       FlyAsh Superplasticizer CoarseAggregate FineAggregate      Water Prediction Model
69  0.1617514       0.13291706 5.509921e-10      0.007644831       0.3905959     0.2564540 0.05063673   81.92797  NNet
357 0.1366392       0.04989215 2.395829e-08      0.002012726       0.4513948     0.3012054 0.05885566   80.16362  NNet
477 0.2145572       0.04306283 4.614330e-02      0.001330241       0.2993219     0.3299018 0.06568276   79.59500  NNet
(I am faced with some problems of drawing graphs. Tomorrow, I will try to fix them.)

Tomorrow, I will fix some problems of the codes.