Exponential families! The secret magic at the heart of traditional statistics.
Exponential families are (sorta) probability distributions that just work, in the sense that and the things you would hope you can do with them, you can.
Noted here so I have somewhere to dump notes, but not something I am going to go into right now.
Most interesting models are not exponential families. There are curved exponential families which generalise exponential families in some way that I have not looke into.
- ShHY04: (2004) Adaptive Model Selection and Assessment for Exponential Family Distributions. Technometrics, 46(3), 306–317. DOI
- AlSH04: (2004) Exponential Families for Conditional Random Fields. In Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence (pp. 2–9). Arlington, Virginia, United States: AUAI Press
- Brow86: (1986) Fundamentals of statistical exponential families: with applications in statistical decision theory. Hayward, Calif: Institute of Mathematical Statistics
- WaJo08: (2008) Graphical models, exponential families, and variational inference (Vol. 1).
- BrCZ10: (2010) Nonparametric regression in exponential families. The Annals of Statistics, 38(4), 2005–2046. DOI
- JeMø91: (1991) Pseudolikelihood for Exponential Family Models of Spatial Point Processes. The Annals of Applied Probability, 1(3), 445–461. DOI
- Efro78: (1978) The Geometry of Exponential Families. The Annals of Statistics, 6(2), 362–376. DOI
- JuSH12: (2012) The RKHS Approach to Minimum Variance Estimation Revisited: Variance Bounds, Sufficient Statistics, and Exponential Families. ArXiv:1210.6516 [Math, Stat].
- TPSR15: (2015) Vector-Space Markov Random Fields via Exponential Families. In Journal of Machine Learning Research (pp. 684–692).