A mere placeholder to remind me to create a model averaging notebook, since I’ve seen the idea pop up in disconnected areas recently, specifically a Bayesian heuristic for dropout in neural nets, AIC for frequentist model averaging, and in a statistical learning context for optimal time series prediction.

Relationship to Bayesian posterior distributions?

This seems to not be quite the same thing as
bagging - or *is* it?

Model weights are often in terms of degrees-of-freedom penalties. It would probably be an instructive exercise for me to work out why for myself.

## Refs

- BaGr69
- Bates, J. M., & Granger, C. W. J.(1969) The Combination of Forecasts.
*Journal of the Operational Research Society*, 20(4), 451–468. DOI. - BuBA97
- Buckland, S. T., Burnham, K. P., & Augustin, N. H.(1997) Model Selection: An Integral Part of Inference.
*Biometrics*, 53(2), 603–618. DOI. - ClHj08
- Claeskens, G., & Hjort, N. L.(2008) Model selection and model averaging. . Cambridge ; New York: Cambridge University Press
- ClGe04
- Clyde, M., & George, E. I.(2004) Model Uncertainty.
*Statistical Science*, 19(1), 81–94. DOI. - FrNe15
- Fragoso, T. M., & Neto, F. L.(2015) Bayesian model averaging: A systematic review and conceptual classification.
*ArXiv:1509.08864 [Stat]*. - Hans07
- Hansen, B. E.(2007) Least Squares Model Averaging.
*Econometrica*, 75(4), 1175–1189. DOI. - HjCl03
- Hjort, N. L., & Claeskens, G. (2003) Frequentist Model Average Estimators.
*Journal of the American Statistical Association*, 98(464), 879–899. DOI. - HMRV99
- Hoeting, J. A., Madigan, D., Raftery, A. E., & Volinsky, C. T.(1999) Bayesian model averaging: a tutorial.
*Statistical Science*, 14(4), 382–417. DOI. - LeBa06
- Leung, G., & Barron, A. R.(2006) Information Theory and Mixing Least-Squares Regressions.
*IEEE Transactions on Information Theory*, 52(8), 3396–3410. DOI. - Phil87
- Phillips, R. F.(1987) Composite Forecasting: An Integrated Approach and Optimality Reconsidered.
*Journal of Business & Economic Statistics*, 5(3), 389–395. DOI. - ShHu06
- Shen, X., & Huang, H.-C. (2006) Optimal Model Assessment, Selection, and Combination.
*Journal of the American Statistical Association*, 101(474), 554–568. DOI. - WaZZ09
- Wang, H., Zhang, X., & Zou, G. (2009) Frequentist model averaging estimation: a review.
*Journal of Systems Science and Complexity*, 22(4), 732. DOI. - ZhLi11
- Zhang, X., & Liang, H. (2011) Focused information criterion and model averaging for generalized additive partial linear models.
*The Annals of Statistics*, 39(1), 174–200. DOI.