2/10/2017

Fit T2 distribution with 2 gaussian distributions

Today, I successfully fitted T2 distribution with 2 gaussian distributions.

Summary:

%% fit T2 with gaussian distribution
% b(1) is scale, b(2) is sigma, b(3) is miu.
modelfun1=@(b,x)(b(1)*(1/(b(2)*sqrt(2 * pi))) * exp(-(x - b(3)).^2 ./ (2 * b(2)^2)));
x1=(1:40);
y1=T2row1(:,(1:40));
x2=(40:64);
y2=T2row1(:,(40:64));
beta10=[0.01;1;10];
beta20=[0.01;1;50];
beta1=nlinfit(x1,y1,modelfun1,beta10);
beta2=nlinfit(x2,y2,modelfun1,beta20);
[beta1,R1,J1,CovB1,MSE1,ErrorModelInfo1]=nlinfit(x1,y1,modelfun1,beta10);
[beta2,R2,J2,CovB2,MSE2,ErrorModelInfo2]=nlinfit(x2,y2,modelfun1,beta20);
fun1=beta1(1,:)*(1/(beta1(2,:)*sqrt(2 * pi))) * exp(-(x1 - beta1(3,:)).^2 ./ (2 * beta1(2,:)^2));
fun2=beta2(1,:)*(1/(beta2(2,:)*sqrt(2 * pi))) * exp(-(x2 - beta2(3,:)).^2 ./ (2 * beta2(2,:)^2));

%% plot
hold off;
plot(T2row1)
hold on;
plot(x1,fun1, 'r-');
plot(x2,fun2, 'r-');
title('T2 distribution fitting');
rsq1=1-sum(R1.^2)/sum((y1-mean(y1)).^2)
rsq2=1-sum(R2.^2)/sum((y2-mean(y2)).^2)


Result:

beta1 =

    0.0426
    6.5562
   19.3092

beta2 =

    0.0042
    2.6393
   49.9026

The first is the scale to adjust gaussian distribution, the second is standard deviation, the third is mean value. I calculated that the r square for the first gaussian distribution is 0.9695 and  the r square for the second gaussian distribution is 0.9907. Both of them are good enough for fitting.

Next week, I will try to apply the method to all the depth of T2 distribution.

2/09/2017

Fitting T2 distribution may be a problem.

Today, I found that the prediction of T2 distribution may be unrealistic.

Summary:
The x-axis of T2 distribution is location such as 1, 2, 3, 4…… the y-axis of T2 distribution is the value that we want to predict.
If we want to fit T2 distribution with some other distributions, we should first know what these distributions are. Usually, we use the probability density function of distributions. The x-axis of some distribution is the values and the y-axis of it is the probability. So they are different from T2 distribution.

From Hess data, we know that there are 64 values in one specific depth. So the x-axis of T2 distribution is 1 to 64. The y-axis of T2 distribution is the value we want. Assuming that the 5th and the 20th value are the same, so they have the same y-value in T2 distribution figure. However, when we use the 64 data to do distribution fitting, the two values are regarded as the same one. They would both have the same x-axis value in the distribution and they contribute to the value’s probability as y-axis. As a result, the distribution we predict is totally different as the T2 distribution.

Tomorrow, I want to have a meeting with you if you have time.

2/08/2017

GMM example

Today, I continued to look for FMM codes. Finally, I found one of GMM example. I think it is really helpful for fitting T2 distribution.

Summary:

First, I want to point out one problem:
In distribution, the pdf shows that y is the probability and x is the value of the variable. But in T2 distribution, the y is the value of T2 and x is just the time.
I will try to solve it.

 % $Author: ChrisMcCormick $    $Date: 2014/05/19 22:00:00 $    $Revision: 1.3 $

%%======================================================
%% STEP 1a: Generate data from two 1D distributions.

mu1 = 10;      % Mean
sigma1 = 1;    % Sigma
m1 = 100;      % Number of points

mu2 = 20;
sigma2 = 3;
m2 = 200;

% Generate the data.
X1 = (randn(m1, 1) * sigma1) + mu1;
X2 = (randn(m2, 1) * sigma2) + mu2;

X = [X1; X2];

%%=====================================================
%% STEP 1b: Plot the data points and their pdfs.

x = [0:0.1:30];
y1 = gaussian1D(x, mu1, sigma1);
y2 = gaussian1D(x, mu2, sigma2);

hold off;
plot(x, y1, 'b-');
hold on;
plot(x, y2, 'r-');
plot(X1, zeros(size(X1)), 'bx', 'markersize', 10);
plot(X2, zeros(size(X2)), 'rx', 'markersize', 10);

set(gcf,'color','white') % White background for the figure.

%%====================================================
%% STEP 2: Choose initial values for the parameters.

% Set 'm' to the number of data points.
m = size(X, 1);

% Set 'k' to the number of clusters to find.
k = 2;

% Randomly select k data points to serve as the means.
indeces = randperm(m);
mu = zeros(1, k);
for (i = 1 : k)
    mu(i) = X(indeces(i));
end

% Use the overal variance of the dataset as the initial variance for each cluster.
sigma = ones(1, k) * sqrt(var(X));

% Assign equal prior probabilities to each cluster.
phi = ones(1, k) * (1 / k)

%%===================================================
%% STEP 3: Run Expectation Maximization

% Matrix to hold the probability that each data point belongs to each cluster.
% One row per data point, one column per cluster.
W = zeros(m, k);

% Loop until convergence.
for (iter = 1:1000)
    
    fprintf('  EM Iteration %d\n', iter);

    %%===============================================
    %% STEP 3a: Expectation
    %
    % Calculate the probability for each data point for each distribution.
    
    % Matrix to hold the pdf value for each every data point for every cluster.
    % One row per data point, one column per cluster.
    pdf = zeros(m, k);
    
    % For each cluster...
    for (j = 1 : k)
        
        % Evaluate the Gaussian for all data points for cluster 'j'.
        pdf(:, j) = gaussian1D(X, mu(j), sigma(j));
    end
    
    % Multiply each pdf value by the prior probability for each cluster.
    %    pdf  [m  x  k]
    %    phi  [1  x  k]   
    %  pdf_w  [m  x  k]
    pdf_w = bsxfun(@times, pdf, phi);
    
    % Divide the weighted probabilities by the sum of weighted probabilities for each cluster.
    %   sum(pdf_w, 2) -- sum over the clusters.
    W = bsxfun(@rdivide, pdf_w, sum(pdf_w, 2));
    
    %%===============================================
    %% STEP 3b: Maximization
    %%
    %% Calculate the probability for each data point for each distribution.

    % Store the previous means so we can check for convergence.
    prevMu = mu;    
    
    % For each of the clusters...
    for (j = 1 : k)
    
        % Calculate the prior probability for cluster 'j'.
        phi(j) = mean(W(:, j));
        
        % Calculate the new mean for cluster 'j' by taking the weighted
        % average of *all* data points.
        mu(j) = weightedAverage(W(:, j), X);
    
        % Calculate the variance for cluster 'j' by taking the weighted
        % average of the squared differences from the mean for all data
        % points.
        variance = weightedAverage(W(:, j), (X - mu(j)).^2);
        
        % Calculate sigma by taking the square root of the variance.
        sigma(j) = sqrt(variance);
    end
    
    % Check for convergence.
    % Comparing floating point values for equality is generally a bad idea, but
    % it seems to be working fine.
    if (mu == prevMu)
        break
    end

% End of Expectation Maximization loop.    
end

%%=====================================================
%% STEP 4: Plot the data points and their estimated pdfs.

x = [0:0.1:30];
y1 = gaussian1D(x, mu(1), sigma(1));
y2 = gaussian1D(x, mu(2), sigma(2));

% Plot over the existing figure, using black lines for the estimated pdfs.
plot(x, y1, 'k-');
plot(x, y2, 'k-');
I think it is the most suitable example close to my research.

Tomorrow, I will continue to look into it.

2/07/2017

EM algorithm for GMM applied in Matlab

Today, I finished reading the paper and  found codes for fitting a Gaussian Mixture Model to data.

Summary:

Models in the paper:
Weibull distribution
Log-normal distribution
Generalized gamma distribution
Generalized F-distribution
Use a mixture of parametric distributions

A mixture of two gamma distributions

Codes:
options = statset('Display','final');
gm = fitgmdist((T2row1)',2,'Options',options);

ComponentMeans = gm.mu
ComponentCovariances = gm.Sigma
MixtureProportions = gm.PComponents

AIC = zeros(1,4);
gm = cell(1,4);
for k = 1:4
    gm{k} = fitgmdist(X,k);
    AIC(k)= gm{k}.AIC;
end

[minAIC,numComponents] = min(AIC);
numComponents

gm2 = gm{numComponents}

I applied the codes to the first row of NMR T2 distribution data and get the following results:

30 iterations, log-likelihood = 391.441

ComponentMeans =

    0.0002
    0.0018


ComponentCovariances(:,:,1) =

   4.7465e-08


ComponentCovariances(:,:,2) =

   3.3443e-07


MixtureProportions =

    0.6682    0.3318


numComponents =

     2


gm2 =

Gaussian mixture distribution with 2 components in 2 dimensions
Component 1:
Mixing proportion: 0.500000
Mean:   -3.0377   -4.9859

Component 2:
Mixing proportion: 0.500000
Mean:    0.9812    2.0563

There are only codes for plotting the contour figures instead of fitting codes.

Tomorrow, I will continue to learn how to plot fitting figures and improve the fitting results.

2/06/2017

Estimating the cure fraction in population-based cancer studies by using FMM

Today, I read the paper 'Estimating the cure fraction in population-based cancer studies by using FMM'. The estimating result shown in the paper was good. I will try to understand all of them and see if models in this paper can be applied in my research.

Summary:
The cure fraction (the proportion of patients who are cured of disease) is of interest
to both patients and clinicians and is a useful measure to monitor trends in survival of curable
disease. The paper extends the non-mixture and mixture cure fraction models to estimate the
proportion cured of disease in population-based cancer studies by incorporating a finite mixture
of two Weibull distributions to provide more flexibility in the shape of the estimated relative survival or excess mortality functions. The methods are illustrated by using public use data from
England and Wales on survival following diagnosis of cancer of the colon where interest lies
in differences between age and deprivation groups. We show that the finite mixture approach
leads to improved model fit and estimates of the cure fraction that are closer to the empirical
estimates.This is particularly so in the oldest age group where the cure fraction is notably lower.
The cure fraction is broadly similar in each deprivation group, but the median survival of the
‘uncured’ is lower in the more deprived groups. The finite mixture approach overcomes some of
the limitations of the more simplistic cure models and has the potential to model the complex
excess hazard functions that are seen in real data.
 

The estimating result is shown below, which is good.

Tomorrow, I will look more into the paper.

2/02/2017

Looking for best functions for fitting T2 distribution

Today, I continue to looking for the best functions for fitting T2 distribution.

Summary:
T2distribution(:,1)=[];
T2distri=T2distribution;
T2row1=T2distri(1,:);
T2row2=T2distri(2,:);
T2row3=T2distri(3,:);
T2row4=T2distri(4,:);

subplot(2,2,1)
area(T2row1)
subplot(2,2,2)
area(T2row2)
subplot(2,2,3)
area(T2row3)
subplot(2,2,4)
area(T2row4)

[phat1,pci1]=mle(T2row1,'distribution','gam')
[phat2,pci2]=mle(T2row2,'distribution','gam')
[phat3,pci3]=mle(T2row3,'distribution','gam')
[phat4,pci4]=mle(T2row4,'distribution','gam')

%% same as above
[phat1,pci1]=gamfit(T2row1)
[phat2,pci2]=gamfit(T2row2)
[phat3,pci3]=gamfit(T2row3)
[phat4,pci4]=gamfit(T2row4)
phat11=phat1(:,1);
phat12=phat1(:,2);
y1=gampdf(T2row1,phat11,phat12);
plot(y1)

[phat5,pci5]=mle(T2row1,'distribution','norm')
[phat6,pci6]=mle(T2row2,'distribution','norm')
[phat7,pci7]=mle(T2row3,'distribution','norm')
[phat8,pci8]=mle(T2row4,'distribution','norm')
phat51=phat5(:,1);
phat52=phat5(:,2);
y5=normpdf(T2row1,phat51,phat52);
plot(y5)

%% example
x = gaminv((0.005:0.01:0.995),100,10);
y = gampdf(x,100,10);
y1 = normpdf(x,1000,100);
figure;
plot(x,y,'-',x,y1,'-.')

After fitting, the results are not good. They are not even on the same order of magnitudes. I will try to solve the problem tomorrow.

2/01/2017

Maximum Likelihood Estimates

Today, I looked into the application of MLE.

Summary:

function [theta] = LG_RML(u,y,oa,ob,od)
na = oa - 1;nb = ob - 1;nc = od - 1;d = 1;
L = length(u);
nn = max(na,nc);
uk = zeros(d + nb,1);
yk = zeros(na,1);
vk = zeros(nc,1);
yfk = zeros(nn,1);ufk = zeros(nn,1);vfk = zeros(nc,1);
thetae_1 = zeros(na+nb+1+nc,1);
thetae = zeros(na+nb+1+nc,L);
P = eye(na+nb+1+nc);
for k = 1:L
    hk = [-yk;uk(d:d+nb);vk];
    v = y(k) - hk'*thetae_1;
    hkf=[-yfk(1:na);ufk(d:d+nb);vfk];
    K=P*hkf/(1+hkf'*P*hkf);
    thetae(:,k)=thetae_1+K*v;
    P=( eye(na+nb+1+nc)-K*hkf' )*P;
    yf=y(k)-thetae(na+nb+2:na+nb+1+nc,k)*yfk(1:nc);%yf(k)
    uf=u(k)-thetae(na+nb+2:na+nb+1+nc,k)*ufk(1:nc);%uf(k)
    vf=v-thetae(na+nb+2:na+nb+1+nc,k)*vfk(1:nc);%vf(k)
    thetae_1=thetae(:,k);      
    for i=d+nb:-1:2
        uk(i)=uk(i-1);
    end
    uk(1)=u(k);    
    for i=na:-1:2
        yk(i)=yk(i-1);
    end
    yk(1)=y(k);
    for i=nc:-1:2
        vk(i)=vk(i-1);
        vfk(i)=vfk(i-1);
    end
    vk(1)=v;
    vfk(1)=vf;
    for i=nn:-1:2
        yfk(i)=yfk(i-1);
        ufk(i)=ufk(i-1);
    end
    yfk(1)=yf;
    ufk(1)=uf;
end
theta = thetae;
end

a=[1 -1.5 0.7];b=[1 0.5];c=[1 -0.5];d=1;
L=1000;
na=length(a)-1;nb=length(b)-1;nc=length(c)-1;
uk=zeros(d+nb,1);
yk=zeros(na,1);
u=randn(L,1);
xi=randn(L,1);
xik=zeros(nc,1);
for k=1:L
    y(k)=-a(2:na+1)*yk+b *uk(d:d+nb)+c *[xi(k);xik];
    for i=d+nb:-1:2
        uk(i)=uk(i-1);
    end
    uk(1)=u(k);
    for i=na:-1:2
        yk(i)=yk(i-1);
    end
    yk(1)=y(k);
    xik(1)=xi(k);
end
thetae = LG_RML(u,y,3,2,2);
figure(1)
plot([1:L],thetae(1:na,:),[1:L],thetae(na+nb+2:na+nb+1+nc,:));
xlabel(k);ylabel('parameter estimation a&d');
legend('a_1' ,'a_2' ,'d_1' );axis([0 L -2 2]);
figure(2)
plot([1:L],thetae(na+1:na+nb+1,:));
xlabel(k);ylabel('parameter estimation b' );
legend('b_0' ,'b_1' );axis([0 L 0 1.5]);



There are many applications of MLE. I will try to find one that is the most similar to my research.

Tomorrow, I will look more into MLE.