The Living Thing / Notebooks :

Probability on manifolds

Information Geometry

The unholy offspring of Fisher information and differential geometry, about which I know little except that it sounds like it should be intuitive. See also information criteria. I also know that even though this sounds intuitive, it is not mainstream and it has also not been especially useful to me even in places where it seemed that it should, at least not beyond the basic delta method.

Homogeneous probability

Albert Tarantola’s baby, from his maybe forthcoming manuscript. How does it relate to information geometry? I don’t know yet. Haven’t had time to read. Also not very common.

Hamiltonian Monte Carlo

Are we talking about a similar thing with Hamiltonian Monte Carlo? That is very mainstream.

To Read

Natural gradient

Should work out what this is. I suspect it’s a synonym for one of the above things. Especially comon in the form of Natural Policy Gradients in reinforcement learning/control research. A brutally short explanation here, and a longer informal one here, and a lengthy one here


Amari, Shun-ichi. (1998) Natural Gradient Works Efficiently in Learning. Neural Computation, 10(2), 251–276. DOI.
Amari, Shunʼichi. (1987) Differential geometrical theory of statistics. In Differential geometry in statistical inference (pp. 19–94).
Amari, Shunʼichi. (2001) Information geometry on hierarchy of probability distributions. IEEE Transactions on Information Theory, 47, 1701–1711. DOI.
Betancourt, M., Byrne, S., Livingstone, S., & Girolami, M. (2017) The geometric foundations of Hamiltonian Monte Carlo. Bernoulli, 23(4A), 2257–2298. DOI.
Fernández-Martínez, J. L., Fernández-Muñiz, Z., Pallero, J. L. G., & Pedruelo-González, L. M.(2013) From Bayes to Tarantola: New insights to understand uncertainty in inverse problems. Journal of Applied Geophysics, 98, 62–72. DOI.
Girolami, M., & Calderhead, B. (2011) Riemann manifold Langevin and Hamiltonian Monte Carlo methods. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 73(2), 123–214. DOI.
Mosegaard, K., & Tarantola, A. (1995) Monte Carlo sampling of solutions to inverse problems. Journal of Geophysical Research, 100(B7), 12431.
Xifara, T., Sherlock, C., Livingstone, S., Byrne, S., & Girolami, M. (2014) Langevin diffusions and the Metropolis-adjusted Langevin algorithm. Statistics & Probability Letters, 91(Supplement C), 14–19. DOI.