The Living Thing / Notebooks :

Probability on manifolds

Information Geometry

The unholy offspring of Fisher information and differential geometry, about which I know little except that it sounds like it should be intuitive. See also information criteria. I also know that even though this sounds intuitive, it is not mainstream and it has also not been especially useful to me even in places where it seemed that it should, at least not beyond the basic delta method.

Homogeneous probability

Albert Tarantola’s baby, from his maybe forthcoming manuscript. How does it relate to information geometry? I don’t know yet. Haven’t had time to read. Also not very common.

Hamiltonian Monte Carlo

Are we talking about a similar thing with Hamiltonian Monte Carlo? That is very mainstream.

To Read

Natural gradient

Should work out what this is. I suspect it’s a synonym for one of the above things. Especially comon in the form of Natural Policy Gradients in reinforcement learning/control research. A brutally short explanation here, and a longer informal one here, and a lengthy one here

Refs

Amar98
Amari, Shun-ichi. (1998) Natural Gradient Works Efficiently in Learning. Neural Computation, 10(2), 251–276. DOI.
Amar87
Amari, Shunʼichi. (1987) Differential geometrical theory of statistics. In Differential geometry in statistical inference (pp. 19–94).
Amar01
Amari, Shunʼichi. (2001) Information geometry on hierarchy of probability distributions. IEEE Transactions on Information Theory, 47, 1701–1711. DOI.
BBLG17
Betancourt, M., Byrne, S., Livingstone, S., & Girolami, M. (2017) The geometric foundations of Hamiltonian Monte Carlo. Bernoulli, 23(4A), 2257–2298. DOI.
FFPP13
Fernández-Martínez, J. L., Fernández-Muñiz, Z., Pallero, J. L. G., & Pedruelo-González, L. M.(2013) From Bayes to Tarantola: New insights to understand uncertainty in inverse problems. Journal of Applied Geophysics, 98, 62–72. DOI.
GiCa11
Girolami, M., & Calderhead, B. (2011) Riemann manifold Langevin and Hamiltonian Monte Carlo methods. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 73(2), 123–214. DOI.
MoTa95
Mosegaard, K., & Tarantola, A. (1995) Monte Carlo sampling of solutions to inverse problems. Journal of Geophysical Research, 100(B7), 12431.
XSLB14
Xifara, T., Sherlock, C., Livingstone, S., Byrne, S., & Girolami, M. (2014) Langevin diffusions and the Metropolis-adjusted Langevin algorithm. Statistics & Probability Letters, 91(Supplement C), 14–19. DOI.