Chapter 4 Introduction {mod_intro}

Some significant applications are demonstrated in this chapter.

4.1 Example one

4.2 Generalized additive models

• splines

  • A cubic spline is a piecewise polynomial with the property of being continuously differentiable until second order.
  • The number of knots act as an smoothing parameter in unpenalized splines.
  • Knots defined based on the quantiles of the data make the spline flexible in dense areas and less flexible in sparse areas, which is desirable.
  • In general, it is more appropiate to select more knots than expected and use a penalty term to control for smothness avoiding the need to select number of knots.
  • For $l$ knots and degree $r$, the space of polynomial splines is a vector space with dimension equals to the number of free parameters. $l-1$ polynomial functions of degree $r$ have $(r+1)(l-1)$ parameters. The condition of $r-1$ times continuously differentiable generate $r$ constrains for all the $l-2$ inside knots. Then, the number of free parameters is $(r+1)(l-1) - r(l-2) = r+l-1$.
  • Natural splines assumes that the curvature, the second derivative, at the first and last knot is zero. Then, a natural cubic spline will have $l$ free parameters.

  Simon Wood 2016

  B-splines, whose construction from polynomial pieces gives them many attractive computational properties, as described by de Boor (1978). 2016 donnell

4.3 GAM

• smoothing bases
  • natural cubic splines are smoothest interpolators
  • cubic smoothing splines
  • cubic regression splines
  • cyclic cubic regression spline
  • p-splines
  • thin plate regression splines
  • tensor products smooths
• polynomial spline
• cubic spline
  • on each data
• regression spline
  • on knots evenly spaced or with quantiles
  • penalizing to avoid overfitting
  • for $\lambda$ known, it is still a augmented linear model
  • select $\lambda$ with ordinary cross validation
  • computational and invariante advante og generalized cross validation
  • Implementation
  • initialize lambda
  • given lambda, obtain beta
  • compute gcv, and interate with previous step
  • Identifiability, constrain one the rest of intercept parameters to zero.

4.4 GAM

• penalized likelihood maximization
  • no simple trick to produce an unpenalized glm whose likelihood is equivalent to the penalized likelihood of the GAM
  • penalized iteratively re-weighted least squares
  • The suggestion of representing GAMs using spline like penalized regression smoothers was made in section 9.3.6 of Hastie and Tibshirani (1990)

4.5 Practice MGCV

4.5.1 Basics of gam model

• When the relationship is almost linear $df = 1$, the confidence interval are zero when the estimates is zero due to the identifiability constrain. This restriction sets the mean value of $f$ to zero, such as there is no uncertainty when $f = 0$.
• The points on the smoothed effects are just the partial residual, which simple are the Pearson residuals plus the smooth function for the corresponding covariate being plotted.
• Considering an initial model with $k_1$ knots and $df<k_1 -1$; then, increasing the number of knot to $k_2$, can modified the number of effective degrees of freedom. It happens because different subspace of functions are obtained when $k=k_1$ or $k+k_2$ for a particular $df$.
• Smoother functions can be obtained introducing and additional parameter to the GCV score $\gamma$. For example, $\gamma = 1.4$ is suggested to avoid overfitting without compromising model fit.

4.5.2 Smoothing several variables

• We can use thin plate regression spline or tensor products.