Chapter 7 exercises

Today, I finished exercises in Chapter 7 and started reading Chapter 8.

Exercises in Chapter 7
#7.1
set.seed(100)
x=runif(100, min=2, max=10)
y=sin(x)+rnorm(length(x))*0.25
sindata=data.frame(x=x, y=y)
plot(x, y)
datagrid=data.frame(x=seq(2, 10, length=100))
library(kernlab)
rbfsvm=ksvm(x=x, y=y, data=sindata, kernel="rbfdot", kpar="automatic", C=1, epsilon=0.1)
#C: cost of constraints violation; epsilon: epsilon in the insensitive-loss function
#kpar=list(sigma=n), when n gets smaller, the line gets flatter. It can be kpar="automatic".
modelprediction=predict(rbfsvm, newdata=datagrid)
points(x=datagrid$x, y=modelprediction[,1], type="l", col="blue")
#type="p, l, b, c, o, s, h, n"

#7.2
library(mlbench)
set.seed(200)
#inputs are 10 independent variables uniformly distributed on the interval [0,1],
#only 5 out of 10 are actually used in the formula
trainingdata2=mlbench.friedman1(200, sd=1)
#convert 'x' data from a matrix to a data frame
trainingdata2$x=data.frame(trainingdata2$x)
featurePlot(trainingdata2$x, trainingdata2$y)
library(caret)
knnmodel=train(trainingdata2$x, trainingdata2$y, method = "knn", preProcess = c("center", "scale"), tuneLength = 10)
knnmodel
knnpred=predict(knnmodel, newdata=trainingdata2$x)
postResample(pred = knnpred, obs = trainingdata2$y)

#7.3
#build SVM, neural network, MARS and KNN mdoels

# 7.4
#if nonlinear models outperform the optimal linear model, it shows that there is a nonlinear relationship between
#predictors and outcomes. if there is mo significant outperformance, linear model will be recommended.

#7.5
#top important predictors of optimal linear model and optimal nonlinear model are different.
#top predictors which are unique to the optimal nonlinear model is proved to own definitely a nonlinear
#relationship with outcomes

Chapter 8
Regression Trees and Rule-Based Models
Weaknesses: 1. Model instability; 2. Less-than-optimal predictive performance
If the relationship between predictors and the response cannot be adequately defined by rectangular subspaces of the predictors, then tree-based or rule-based models will have larger prediction error than other kinds of models.

8.1
Basic Regression Trees
The predictor to split on and value of the split
The depth or complexity of the tree
The prediction equation in the terminal nodes
CART: classification and regression tree methodology
SSE finds the optimal split-point for every predictor
SSE=∑_(i∈S_1)▒〖(y_i-y ̅_1)〗^2 +∑_(i∈S_2)▒〖(y_i-y ̅_2)〗^2
Cost-complexity tuning
SSEC_p=SSE+Cp×(#Terminal Nodes)
C_p is the complexity parameter
Smaller penalties tend to produce more complex models, which result in larger trees.

Tomorrow, I will continue to read Chapter 8.