Today, I realized parallel computing in MATLAB.
Summary:
I checked that Your laptop is 4 core CPU. So theoretically, it will save 3/4 time overall.
I ran 40 times to get the results for Vp and Vs prediction:
Average values: 0.8555 and 0.8295 in terms of R2; 0.0652 and 0.0684 in terms of NRMSE
Best Values: 0.8722 and 0.8492 in terms of R2 and; 0.0599 and 0.0603 in terms of NRMSE
Running time now is 348 seconds (about 900 seconds initially).
Next week, I will finish other tasks discussed today.
9/28/2017
try some changes of the parameters
Today, I tried some changes of the parameters of ANN models.
Summary:
1. randomly select initial weights and biases (combine parallel computing to decrease training time)
and record the best result after training ANN models for several times (10 or 20 or 30 ......) I found that the best result among them is a little better than the best result in the first day.
I think it is one effective method to avoid local minimum and obtain global optimization.
2. compare prediction results with and without some data preprocessing steps (such as pca, reciprocal transformation, remove constant values). They can be done after the first step since they may lead to little improvement.
3. another to methods may be useful: Nguyen-Windrow initialization; evaluate effective number of parameters.
Tomorrow, I will validate all of the above methods.
Summary:
1. randomly select initial weights and biases (combine parallel computing to decrease training time)
and record the best result after training ANN models for several times (10 or 20 or 30 ......) I found that the best result among them is a little better than the best result in the first day.
I think it is one effective method to avoid local minimum and obtain global optimization.
2. compare prediction results with and without some data preprocessing steps (such as pca, reciprocal transformation, remove constant values). They can be done after the first step since they may lead to little improvement.
3. another to methods may be useful: Nguyen-Windrow initialization; evaluate effective number of parameters.
Tomorrow, I will validate all of the above methods.
9/27/2017
compare three training functions and predict Vp and Vs seperately
Today, I compared three training functions and predicted Vp and Vs seperately.
Summary:
1.
I validate that BR and SCG is equally effective to predict geomechanical data, which are better than LM. So, if we write the paper, we can include them as one of our sections.
I validate that BR and SCG is equally effective to predict geomechanical data, which are better than LM. So, if we write the paper, we can include them as one of our sections.
2.
You mentioned that I can try to predict Vp first and Vs next. I did and I found that prediction accuracy did not improve.
As I mentioned before, accuracy of the ANN model without deleting outliers is 0.8611 and 0.8252 in terms of R2 and 0.0622 and 0.0660 in terms of NRMSE.
However, when I predict Vp in the first model and predict Vs in the second model, accuracy of ANN models without deleting outliers is 0.8500, 0.8178 in terms of R2 and 0.0710 and 0.0733 in terms of NRMSE.
Tomorrow, I will think about other ways to improve the accuracy.
You mentioned that I can try to predict Vp first and Vs next. I did and I found that prediction accuracy did not improve.
As I mentioned before, accuracy of the ANN model without deleting outliers is 0.8611 and 0.8252 in terms of R2 and 0.0622 and 0.0660 in terms of NRMSE.
However, when I predict Vp in the first model and predict Vs in the second model, accuracy of ANN models without deleting outliers is 0.8500, 0.8178 in terms of R2 and 0.0710 and 0.0733 in terms of NRMSE.
Tomorrow, I will think about other ways to improve the accuracy.
9/26/2017
look for deep learning applications and compare training functions
Today, I looked for deep learning applications and compared training functions.
Summary:
1. I have not found deep learning applications related to our research.
2. I think that training functions may affect the prediction accuracy. I am now comparing LM, SCG and BR. I will select the best one after comparison.
Tomorrow, I will continue above work.
Summary:
1. I have not found deep learning applications related to our research.
2. I think that training functions may affect the prediction accuracy. I am now comparing LM, SCG and BR. I will select the best one after comparison.
Tomorrow, I will continue above work.
9/25/2017
look for applications of deep learning
Today, I looked for applications of deep learning.
Summary:
Deep learning is a type of machine learning, which is well-suited to identification applications such as face recognition, text translation, voice recognition and advanced driver assistance systems.
Usually, there are hundreds of hidden layers of NN models, which needs millions of images and videos to train them. That is why deep learning algorithms can outperform human at classifying images, win against the world's best GO player, or enable a voice-controlled assistant.
However, I have not found applications for function approximation problems, which is like our research problem.
Tomorrow, I will continue to look for deep learning application similar to our research.
Summary:
Deep learning is a type of machine learning, which is well-suited to identification applications such as face recognition, text translation, voice recognition and advanced driver assistance systems.
Usually, there are hundreds of hidden layers of NN models, which needs millions of images and videos to train them. That is why deep learning algorithms can outperform human at classifying images, win against the world's best GO player, or enable a voice-controlled assistant.
However, I have not found applications for function approximation problems, which is like our research problem.
Tomorrow, I will continue to look for deep learning application similar to our research.
9/22/2017
finish paper 2
Today, I finished paper 2.
Summary:
I sent you by an email.
Next week, I will improve paper 1 for Fuel.
Summary:
I sent you by an email.
Next week, I will improve paper 1 for Fuel.
9/21/2017
check all results possible in the basic model.
Today, I checked all results possible in the basic model.
Summary:
I compare 4 conditions.
1. do not delete outliers, ANN model with just one layer
R2: 0.8564 0.8187 NRMSE: 0.0632 0.0672
2. do not delete outliers, ANN model with two layers
R2: 0.8611 0.8252 NRMSE: 0.0622 0.0660
3. delete outliers, ANN model with two layers
R2: 0.8494 0.8265 NRMSE: 0.0695 0.0689
4. delete outliers, ANN model with two layers, predict Vp and Vp/Vs together
R2: 0.8299 0.7964 NRMSE: 0.0712 0.0807
The second performs the best.
Tomorrow, I will continue to look for methods to improve the model such as deep ANN.
Summary:
I compare 4 conditions.
1. do not delete outliers, ANN model with just one layer
R2: 0.8564 0.8187 NRMSE: 0.0632 0.0672
2. do not delete outliers, ANN model with two layers
R2: 0.8611 0.8252 NRMSE: 0.0622 0.0660
3. delete outliers, ANN model with two layers
R2: 0.8494 0.8265 NRMSE: 0.0695 0.0689
4. delete outliers, ANN model with two layers, predict Vp and Vp/Vs together
R2: 0.8299 0.7964 NRMSE: 0.0712 0.0807
The second performs the best.
Tomorrow, I will continue to look for methods to improve the model such as deep ANN.
Subscribe to:
Posts (Atom)