A mere placeholder to remind me to create a model averaging notebook, since I’ve seen the idea pop up in disconnected areas recently, specifically a Bayesian heuristic for dropout in neural nets, AIC for frequentist model averaging, and in a statistical learning context for optimal time series prediction.
Relationship to Bayesian posterior predictive distributions?
This seems to not be quite the same thing as bagging – or is it?
Model weights are often in terms of degrees-of-freedom penalties. It would probably be an instructive exercise for me to work out why for myself.
- FrNe15: Tiago M. Fragoso, Francisco Louzada Neto (2015) Bayesian model averaging: A systematic review and conceptual classification. ArXiv:1509.08864 [Stat].
- HMRV99: Jennifer A. Hoeting, David Madigan, Adrian E. Raftery, Chris T. Volinsky (1999) Bayesian model averaging: a tutorial. Statistical Science, 14(4), 382–417. DOI
- PiVe17: Juho Piironen, Aki Vehtari (2017) Comparison of Bayesian predictive methods for model selection. Statistics and Computing, 27(3), 711–735. DOI
- Phil87: Robert F. Phillips (1987) Composite Forecasting: An Integrated Approach and Optimality Reconsidered. Journal of Business & Economic Statistics, 5(3), 389–395. DOI
- ZhLi11: Xinyu Zhang, Hua Liang (2011) Focused information criterion and model averaging for generalized additive partial linear models. The Annals of Statistics, 39(1), 174–200. DOI
- HjCl03: Nils Lid Hjort, Gerda Claeskens (2003) Frequentist Model Average Estimators. Journal of the American Statistical Association, 98(464), 879–899. DOI
- WaZZ09: Haiying Wang, Xinyu Zhang, Guohua Zou (2009) Frequentist model averaging estimation: a review. Journal of Systems Science and Complexity, 22(4), 732. DOI
- LaFr05: J. F. Lawless, Marc Fredette (2005) Frequentist prediction intervals and predictive distributions. Biometrika, 92(3), 529–542. DOI
- LeBa06: G. Leung, A.R. Barron (2006) Information Theory and Mixing Least-Squares Regressions. IEEE Transactions on Information Theory, 52(8), 3396–3410. DOI
- Hans07: Bruce E. Hansen (2007) Least Squares Model Averaging. Econometrica, 75(4), 1175–1189. DOI
- BuBA97: S. T. Buckland, K. P. Burnham, N. H. Augustin (1997) Model Selection: An Integral Part of Inference. Biometrics, 53(2), 603–618. DOI
- ClHj08: Gerda Claeskens, Nils Lid Hjort (2008) Model selection and model averaging. Cambridge ; New York: Cambridge University Press
- ClGe04: Merlise Clyde, Edward I. George (2004) Model Uncertainty. Statistical Science, 19(1), 81–94. DOI
- ShHu06: Xiaotong Shen, Hsin-Cheng Huang (2006) Optimal Model Assessment, Selection, and Combination. Journal of the American Statistical Association, 101(474), 554–568. DOI
- BaGr69: J. M. Bates, C. W. J. Granger (1969) The Combination of Forecasts. Journal of the Operational Research Society, 20(4), 451–468. DOI