Learning on manifolds

Finding the lowest bit of a krazy straw, from the inside

Usefulness: đź”§
Novelty: đź’ˇ
Uncertainty: đź¤Ş đź¤Ş đź¤Ş
Incompleteness: đźš§ đźš§ đźš§

A placeholder for learning on curved spaces. Not discussed: learning OF curved spaces.

Also: learning where there is an a priori manifold seems to also be a usage here? See the work of, e.g.Â Nina Miolane and collaborators on the Geomstats project.

Girolami et al discuss Langevin Monte Carlo in this context.

The below headings may one day be filled in.

Information Geometry

The unholy offspring of Fisher information and differential geometry, about which I know little except that it sounds like it should be intuitive. See also information criteria. I also know that even though this sounds intuitive, it is not mainstream and it has also not been especially useful to me even in places where it seemed that it should, at least not beyond the basic delta method.

Hamiltonian Monte Carlo

You can also discuss Hamiltonian Monte Carlo in this setting. I will not.

Homogeneous probability

Albert Tarantolaâ€™s framing, from his maybe forthcoming manuscript. How does it relate to information geometry? I donâ€™t know yet. Havenâ€™t had time to read. Also not a very common phrasing, which is a danger sign.

Refs

Absil, P.-A, R Mahony, and R Sepulchre. 2008. Optimization Algorithms on Matrix Manifolds. Princeton, N.J.; Woodstock: Princeton University Press. http://public.eblib.com/choice/publicfullrecord.aspx?p=457711.

Amari, Shun-ichi. 1998. â€śNatural Gradient Works Efficiently in Learning.â€ť Neural Computation 10 (2): 251â€“76. https://doi.org/10.1162/089976698300017746.

Amari, ShunĘĽichi. 1987. â€śDifferential Geometrical Theory of Statistics.â€ť In Differential Geometry in Statistical Inference, 19â€“94.

â€”â€”â€”. 2001. â€śInformation Geometry on Hierarchy of Probability Distributions.â€ť IEEE Transactions on Information Theory 47: 1701â€“11. https://doi.org/10.1109/18.930911.

Aswani, Anil, Peter Bickel, and Claire Tomlin. 2011. â€śRegression on Manifolds: Estimation of the Exterior Derivative.â€ť The Annals of Statistics 39 (1): 48â€“81. https://doi.org/10.1214/10-AOS823.

Barndorff-Nielsen, O E. 1987. â€śDifferential and Integral Geometry in Statistical Inference.â€ť In Differential Geometry in Statistical Inference. Sn Aarhus.

Betancourt, Michael, Simon Byrne, Sam Livingstone, and Mark Girolami. 2017. â€śThe Geometric Foundations of Hamiltonian Monte Carlo.â€ť Bernoulli 23 (4A): 2257â€“98. https://doi.org/10.3150/16-BEJ810.

Boumal, Nicolas. 2013. â€śOn Intrinsic CramĂ©r-Rao Bounds for Riemannian Submanifolds and Quotient Manifolds.â€ť IEEE Transactions on Signal Processing 61 (7): 1809â€“21. https://doi.org/10.1109/TSP.2013.2242068.

Boumal, Nicolas, Bamdev Mishra, P.-A. Absil, and Rodolphe Sepulchre. 2014. â€śManopt, a Matlab Toolbox for Optimization on Manifolds.â€ť Journal of Machine Learning Research 15: 1455â€“9. http://jmlr.org/papers/v15/boumal14a.html.

Boumal, Nicolas, Amit Singer, P.-A. Absil, and Vincent D. Blondel. 2014. â€śCramĂ©r-Rao Bounds for Synchronization of Rotations.â€ť Information and Inference 3 (1): 1â€“39. https://doi.org/10.1093/imaiai/iat006.

Carlsson, Gunnar, Tigran Ishkhanov, Vin de Silva, and Afra Zomorodian. 2008. â€śOn the Local Behavior of Spaces of Natural Images.â€ť International Journal of Computer Vision 76 (1): 1â€“12. https://doi.org/10.1007/s11263-007-0056-x.

Chen, Minhua, J. Silva, J. Paisley, Chunping Wang, D. Dunson, and L. Carin. 2010. â€śCompressive Sensing on Manifolds Using a Nonparametric Mixture of Factor Analyzers: Algorithm and Performance Bounds.â€ť IEEE Transactions on Signal Processing 58 (12): 6140â€“55. https://doi.org/10.1109/TSP.2010.2070796.

FernĂˇndez-MartĂ­nez, J. L., Z. FernĂˇndez-MuĂ±iz, J. L. G. Pallero, and L. M. Pedruelo-GonzĂˇlez. 2013. â€śFrom Bayes to Tarantola: New Insights to Understand Uncertainty in Inverse Problems.â€ť Journal of Applied Geophysics 98 (November): 62â€“72. https://doi.org/10.1016/j.jappgeo.2013.07.005.

Ge, Rong, and Tengyu Ma. 2017. â€śOn the Optimization Landscape of Tensor Decompositions.â€ť In Advances in Neural Information Processing Systems. http://arxiv.org/abs/1706.05598.

Girolami, Mark, and Ben Calderhead. 2011. â€śRiemann Manifold Langevin and Hamiltonian Monte Carlo Methods.â€ť Journal of the Royal Statistical Society: Series B (Statistical Methodology) 73 (2): 123â€“214. https://doi.org/10.1111/j.1467-9868.2010.00765.x.

Hosseini, Reshad, and Suvrit Sra. 2015. â€śManifold Optimization for Gaussian Mixture Models.â€ť arXiv Preprint arXiv:1506.07677. http://arxiv.org/abs/1506.07677.

Lauritzen, S L. 1987. â€śStatistical Manifolds.â€ť In Differential Geometry in Statistical Inference, 164. JSTOR.

Miolane, Nina, Johan Mathe, Claire Donnat, Mikael Jorda, and Xavier Pennec. 2018. â€śGeomstats: A Python Package for Riemannian Geometry in Machine Learning,â€ť May. http://arxiv.org/abs/1805.08308.

Mosegaard, Klaus, and Albert Tarantola. 1995. â€śMonte Carlo Sampling of Solutions to Inverse Problems.â€ť Journal of Geophysical Research 100 (B7): 12431. http://www.gfy.ku.dk/~klaus/ip/MT-1995.pdf.

Mukherjee, Sayan, Qiang Wu, and Ding-Xuan Zhou. 2010. â€śLearning Gradients on Manifolds.â€ť Bernoulli 16 (1): 181â€“207. https://doi.org/10.3150/09-BEJ206.

Peters, Jan. 2010. â€śPolicy Gradient Methods.â€ť Scholarpedia 5 (11): 3698. https://doi.org/10.4249/scholarpedia.3698.

Steinke, Florian, and Matthias Hein. 2009. â€śNon-Parametric Regression Between Manifolds.â€ť In Advances in Neural Information Processing Systems 21, 1561â€“8. Curran Associates, Inc. http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2008_0692.pdf.

Townsend, James, Niklas Koep, and Sebastian Weichwald. 2016. â€śPymanopt: A Python Toolbox for Optimization on Manifolds Using Automatic Differentiation.â€ť Journal of Machine Learning Research 17 (137): 1â€“5. http://jmlr.org/papers/v17/16-177.html.

Wang, Yu Guang, and Xiaosheng Zhuang. 2016. â€śTight Framelets and Fast Framelet Transforms on Manifolds,â€ť August. http://arxiv.org/abs/1608.04026.

Xifara, T., C. Sherlock, S. Livingstone, S. Byrne, and M. Girolami. 2014. â€śLangevin Diffusions and the Metropolis-Adjusted Langevin Algorithm.â€ť Statistics & Probability Letters 91 (Supplement C): 14â€“19. https://doi.org/10.1016/j.spl.2014.04.002.