2/22/2017

Start to apply logging data and parameter matrix in R

Today, I started to apply logging data and parameter matrix in R.

Summary:

I try to select 6 most possible models to predict:
ANN, Boosted tree, Cubist, Random forest, SVM and Elastic Net.
As what I read last semester, these six are the best ones used to predict other kinds of data.
At this evening, I code for ANN model. Now the laptop is calculating and it has not finished yet. So I do not know if it is good or not for now. I will leave it calculating by itself. Tomorrow, I may get some results.

Codes:
## T2 Prediction
head(betapredict)
head(logmatch)
betaorigin=betapredict
logorigin=logmatch
logchange1=logorigin
betachange1=betaorigin
library(caret)
set.seed(1)
trainingrows=createDataPartition((1:3055), p=0.8, list=FALSE)
head(trainingrows)
trainlog=logchange1[trainingrows, ]
trainbeta=betachange1[trainingrows,]
testlog=logchange1[-trainingrows, ]
testbeta=betachange1[-trainingrows,]

library(earth)
library(kernlab)
library(nnet)
library(MASS)
parameter1=trainbeta$P1
parameter2=trainbeta$P2
parameter3=trainbeta$P3
parameter4=trainbeta$P4
parameter5=trainbeta$P5
parameter6=trainbeta$P6

ctr1=trainControl(method = "cv", number = 10)
nnetgrid=expand.grid(.decay=c(0, 0.01, 0.1), .size=c(1:10), .bag=FALSE)
set.seed(100)
nnettune=train(trainlog, parameter1, method="avNNet", tuneGrid = nnetgrid, trControl = ctr1,
               preProcess = c("center", "scale"), linout = TRUE, trace = FALSE, 
               MaxNWts = 10*(ncol(trainlog)+1)+10+1, maxit=200)
nnettune

Tomorrow, I will continue to build models and compare them.

Results:

Model Averaged Neural Network 

2447 samples
  16 predictor

Pre-processing: centered (16), scaled (16) 
Resampling: Cross-Validated (10 fold) 
Summary of sample sizes: 2202, 2202, 2202, 2201, 2202, 2203, ... 
Resampling results across tuning parameters:

  decay  size  RMSE         Rsquared 
  0.00    1    0.001182810  0.2659956
  0.00    2    0.001178855  0.2652559
  0.00    3    0.001176240  0.2700417
  0.00    4    0.001171193  0.2758881
  0.00    5    0.001171282  0.2757166
  0.00    6    0.001175032  0.2719288
  0.00    7    0.001169831  0.2771893
  0.00    8    0.001170633  0.2768607
  0.00    9    0.001173294  0.2730265
  0.00   10    0.001174102  0.2723126
  0.01    1    0.001202662  0.2380567
  0.01    2    0.001202690  0.2380329
  0.01    3    0.001202191  0.2388848
  0.01    4    0.001201014  0.2403814
  0.01    5    0.001208686  0.2342891
  0.01    6    0.001192577  0.2520885
  0.01    7    0.001187736  0.2564749
  0.01    8    0.001188651  0.2550046
  0.01    9    0.001189708  0.2548316
  0.01   10    0.001190979  0.2544052
  0.10    1    0.001212622  0.2285682
  0.10    2    0.001211344  0.2293782
  0.10    3    0.001209365  0.2291870
  0.10    4    0.001211336  0.2291943
  0.10    5    0.001211175  0.2295812
  0.10    6    0.001204673  0.2362646
  0.10    7    0.001203994  0.2369177
  0.10    8    0.001202633  0.2369342
  0.10    9    0.001203141  0.2373625
  0.10   10    0.001204911  0.2364219

Tuning parameter 'bag' was held constant at a value of FALSE
RMSE was used to select the optimal model using  the smallest value.


The final values used for the model were size = 7, decay = 0 and bag = FALSE. 

The results come out now but R2 of training model is not very good. The largest is smaller than 0.3. It should be at least 0.7. I will try to figure it out today.

2/21/2017

Change x-axis from bins to t2 in ms and recalculate R2

Today, I changed x-axis from bins to t2 in ms and recalculated R2.

Summary:
After ignoring the first several or last several points with large residuals, I improved most of the low R2 depths.

There are only 11 depths with R2 lower than 0.7 and 3 depths with R2 lower than 0.4. The lowest is 0.293 (the middle one of the second figure). I think the reason is not only for the first or last peaks but also for the bad fitting results in middle parts, which is unavoidable for this fitting functions if I need the accuracy for other depths.


The following is the new R2 distributions for all used 3055 depths.

Tomorrow, I will do depth matching for logging data and parameters data and start applying them in R.

2/20/2017

Depth shifting and find poor fitting results

Today, I drew figures for logging data of depth shifting and found the poor fitting results.

Summary:

First, I search online and find the explanation for R2 to be negative.
R2 compares the fit of the chosen model with that of a horizontal straight line (the null hypothesis). If the chosen model fits worse than a horizontal line, then R2 is negative. Note that R2 is not always the square of anything, so it can have a negative value without violating any rules of math. R2 is negative only when the chosen model does not follow the trend of the data, so fits worse than a horizontal line.
A negative R2 is not a mathematical impossibility or the sign of a computer bug. It simply means that the chosen model (with its constraints) fits the data really poorly.

Second, I make sure that all the logging data I use are depth shifted.

Third, I found the fitting depths whose R2 is lower than 0.7 and 0.
There are 154 depths whose R2 are lower than 0.7. There are 26 depths whose R2 are lower than 0.
The first plot is for the first 16 dpeths lower than 0.7 and the second and third are for lower than 0.

I think although the values of 0.7 and 0 are very different, the fitting plots are similar. They all cannot fit the first declining curves or the last ascending curves.

For the last 5th small picture, it does not fit any peaks, which shows bugs in my code. After fixing bugs, There are 151 depths whose R2 are lower than 0.7. There are 25 depths whose R2 are lower than 0.
The first plot is for the first 25 depths lower than 0.7 and the second is for lower than 0.

Now the smallest R2 (about -0.4) depth is deleted automatically.

After adjusting for several times, the percentages of each category are shown as follows (3 means 3 or more).

Tomorrow, I will start to apply these data into R.

2/17/2017

fix one problem

Today, I fixed one problem, which was that my codes could not identify peaks if they have two or more largest values of one peak. They are now solved.

Summary:

The above are the new first 32 depths used to be fitted.
As a result, their fitting results are better than yesterday's.
The following is the new R2 distribution.


Next week, I will start to predict the parameters of these distributions with some models.


2/16/2017

fit T2 distribution

Today, I fitted T2 distribution with adjusted guassian distributions.

Summary:

Most of the fitting results are very good shown in the figures.
The R^2 distribution figure shows that most of them are close to 1, which means that  the fitting results are very good. However, there are some very small R^2. It is obvious that these fitting has problems.

These are the first 16 depths to be fitted. Most of them are really good. Most of their R2 is larger than 0.9. However, some of them are not good.
For example, the 16th figure showed a bad fitting result. The reason is that: the largest peak of the 16th figure has three largest values, which are equal to each other. As a result, My code cannot identify that this is a peak. Also, maybe there are some other problems before fitting.

I will try to solve the problem tomorrow and see if there are any other questions.

2/15/2017

adjust all the depth

Today, I discussed with you, Dr. Misra, and made an agreement about how to deal with the imperfect data. Finally, I have keep all depths following the agreement.

Summary:

I conclude all depths' conditions of t2 distribution and made a sheet (3 means 3 or more).

Codes changed:

%% whole depth
% find peaks
for i=1:4256
    for j=2:63
        if T2distri(i,j-1)<T2distri(i,j) && T2distri(i,j)>T2distri(i,j+1)
            p(i,j)=T2distri(i,j);
        end
    end
end

[row,col,v]=find(p);
col1=col+1;
p2=[row,col1,v]; % ascending it by row
% if there are more than two peaks at one depth, delete some of them

p3=p2;

for i=1:9674
    if p3(i,1)==p3(i+1,1) && p3(i+1,1)==p3(i+2,1)
        if p3(i,3)>=0.1*p3(i+1,3) && p3(i,3)>=0.1*p3(i+2,3) && p3(i+1,3)>=0.1*p3(i,3) && p3(i+1,3)>=0.1*p3(i+2,3) &&  p3(i+2,3)>=0.1*p3(i,3) && p3(i+2,3)>=0.1*p3(i+1,3)                  
        p3(i,3)=0;
        p3(i+1,3)=0;
        p3(i+2,3)=0;
        end
    end
end

for i=1:9675
    if p3(i,3)==0
        if p3(i,1)==p3(i+1,1)
           p3(i+1,3)=0;
        end
    end
end

for i=1:6046 % change till the end
    if p3(i,3)==0
        p3(i,:)=[];
    end
end
% for every depth with more than 2 peaks and other peaks cannot be ignored,
% i have deleted them.

p4=p3;

%% for other depths with more than 2 peaks, i can choose the largest 2 peaks
for i=1:5651 % change till the end
    if p4(i,1)==p4(i+1,1) && p4(i+1,1)==p4(i+2,1)
        if p4(i,3)<p4(i+1,3) && p4(i,3)<p4(i+2,3)
            p4(i,:)=[];
        else if p4(i+1,3)<p4(i,3) && p4(i+1,3)<p4(i+2,3)
            p4(i+1,:)=[];
            else
               p4(i+2,:)=[];
            end
        end
    end
end
% now there is only one or two peaks at every depth


%% adjust two peaks at every depth
p5=p4;

for i=2:6175 % change the number till the end
    if p5(i,1)~=p5(i-1,1) && p5(i,1)~=p5(i+1,1)
        if p5(i,2)<=32          
        p5=[p5(1:i,:);[p5(i,1),0,0];p5(i+1:end,:)];
        else
        p5=[p5(1:i-1,:);[p5(i,1),0,0];p5(i:end,:)];
        end
    end
end

%% set initial values for iteration
for i=1:6176  
    if p5(i,3)==0
        beta0(i,:)=[0;0;0];
    else
        beta0(i,:)=[0.01;1;p5(i,3)];
    end      
end

I have set the initial values for iteration for every depth.

Tomorrow, I will do the fitting step.

2/14/2017

fit T2 distribution with gaussian distributions at every depth 1

Today, I tried to fit T2 distribution with gaussian distributions at every depth.

Summary:
1. I  find peaks at every depth. But there are 1 or 2 or 3 or more peaks at some depth. Some of them are really small though.
2. If there are more than 2 peaks at one depth, I decide to delete some of them and keep two peaks at most.
3. If there are only 1 peak at one depth, I copy it so that every depth has two peaks.
4. I also find that there are no peaks at some depth. I finally find that it is because the shape of T2 distribution at some depth are like this:
So I add the first and last point value to data matrix as the peaks at some depth.
5. Finally there are two peaks at every depth.
6. I set initial values for iteration when fitting. Because there may cause calculating problems when fitting in Matlab if the initial values are not similar to the results. So setting initial values for iteration is very important.

The following are codes I write today.

%% whole depth
% find peaks
for i=1:4256
    for j=2:63
        if T2distri(i,j-1)<T2distri(i,j) && T2distri(i,j)>T2distri(i,j+1)
            p(i,j)=T2distri(i,j);
        end
    end
end

[row,col,v]=find(p);
col1=col+1;
p2=[row,col1,v]; % ascending it by row
% if there are more than two peaks at one depth, delete some of them
p3=p2;

for i=1:7977 % change the number till the end
    if p3(i,1)==p3(i+1,1) && p3(i+1,1)==p3(i+2,1)
        if p3(i,3)<p3(i+1,3) && p3(i,3)<p3(i+2,3)
            p3(i,:)=[];
        else if p3(i+1,3)<p3(i,3) && p3(i+1,3)<p3(i+2,3)
                p3(i+1,:)=[];
            else
                 p3(i+2,:)=[];
            end
        end
    end
end
% now there is only one or two peaks at every depth

%% adjust two peaks at every depth
p4=p3;
for i=2:8483 % change the number till the end
    if p4(i,1)~=p4(i-1,1) && p4(i,1)~=p4(i+1,1)
        p4=[p4(1:i,:);p4(i,:);p4(i+1:end,:)];
    end
end
% but there are some missing depth

%% add missing depth
p5=p4;
for i=2:8512
    if p5(i,1)>=p5(i-1,1)+2
        p5=[p5(1:i-1,:);[p5(i-1,1)+1,1,T2distri(p5(i-1,1)+1,1)];
            [p5(i-1,1)+1,64,T2distri(p5(i-1,1)+1,64)];p5(i:end,:)];
    end
end

%% set initial values for iteration
for i=1:8512,
    beta0(i,:)=[0.01;1;p5(i,3)];
end

Tomorrow, I will try to finish the following part of fitting T2 distribution at every depth.