The Living Thing / Notebooks :

Cross Validation

On substituting simulation for analysis in model selection, in e.g. choosing the “right” regularisation parameter for sparse regression.

Asymptotically equivalent to generalised Akaike information criteria. (e.g. Ston77) Related to bootstrap in various ways.

The computationally expensive default option when your model doesn’t have any obvious short cuts for complexity regularization. For example, I think that the AIC for penalised reqgression requires penalties twice-differentiable at the optimum. I’m not sure they couldn’t be made to work, however. should investigate.

Present alternatives, especially outside n-fold cross-validation, especially computationally tractable ones. Methods based on statistical learning theory or concentration inequalities win gratitude.

Basic Cross Validation


Generalised Cross Validation

Why the name? It’s specialised cross-validation, AFAICT.

TBD. Hat matrix, smoother matrix. Note comparative computational efficiency. Define hat matrix.