The Living Thing / Notebooks :

Model averaging

On keeping many incorrect hypotheses and using them all as one goodish one

A mere placeholder to remind me to create a model averaging notebook, since I’ve seen the idea pop up in disconnected areas recently, specifically a Bayesian heuristic for dropout in neural nets, AIC for frequentist model averaging, and in a statistical learning context for optimal time series prediction.

Relationship to Bayesian posterior predictive distributions?

This seems to not be quite the same thing as bagging – or is it?

Model weights are often in terms of degrees-of-freedom penalties. It would probably be an instructive exercise for me to work out why for myself.

Refs