The Living Thing / Notebooks :

Learning graphical models from data

what is independent of what

Usefulness: 🔧
Novelty: 💡
Uncertainty: 🤪 🤪 🤪
Incompleteness: 🚧 🚧 🚧

Learning the graph structure, not just the clique potentials.

Much more work.

Learning these models turns out to need a conditional independence test, an awareness of multiple testing and graphs.


Bayati, M., and A. Montanari. 2012. “The LASSO Risk for Gaussian Matrices.” IEEE Transactions on Information Theory 58 (4): 1997–2017.

Buntine, W. L. 1996. “A Guide to the Literature on Learning Probabilistic Networks from Data.” IEEE Transactions on Knowledge and Data Engineering 8 (2): 195–210.

Bühlmann, Peter, and Sara van de Geer. 2011. Statistics for High-Dimensional Data: Methods, Theory and Applications. 2011 edition. Heidelberg ; New York: Springer.

Cai, T. Tony. 2017. “Global Testing and Large-Scale Multiple Testing for High-Dimensional Covariance Structures.” Annual Review of Statistics and Its Application 4 (1): 423–46.

Colombo, Diego, Marloes H. Maathuis, Markus Kalisch, and Thomas S. Richardson. 2012. “Learning High-Dimensional Directed Acyclic Graphs with Latent and Selection Variables.” The Annals of Statistics 40 (1): 294–321.

Cox, D. R., and H. S. Battey. 2017. “Large Numbers of Explanatory Variables, a Semi-Descriptive Analysis.” Proceedings of the National Academy of Sciences 114 (32): 8592–5.

Drton, Mathias, and Marloes H. Maathuis. 2017. “Structure Learning in Graphical Modeling.” Annual Review of Statistics and Its Application 4 (1): 365–93.

Friedman, Jerome, Trevor Hastie, and Robert Tibshirani. 2008. “Sparse Inverse Covariance Estimation with the Graphical Lasso.” Biostatistics 9 (3): 432–41.

Fu, Fei, and Qing Zhou. 2013. “Learning Sparse Causal Gaussian Networks with Experimental Intervention: Regularization and Coordinate Descent.” Journal of the American Statistical Association 108 (501): 288–300.

Geer, Sara van de. 2014. “Worst Possible Sub-Directions in High-Dimensional Models.” In. Vol. 131.

Geng, Zhi, Yue Liu, Chunchen Liu, and Wang Miao. 2019. “Evaluation of Causal Effects and Local Structure Learning of Causal Networks.” Annual Review of Statistics and Its Application 6 (1): 103–24.

Gogate, Vibhav, William Webb, and Pedro Domingos. 2010. “Learning Efficient Markov Networks.” In Advances in Neural Information Processing Systems, 748–56.

Hallac, David, Jure Leskovec, and Stephen Boyd. 2015. “Network Lasso: Clustering and Optimization in Large Graphs,” July.

Harris, Naftali, and Mathias Drton. 2013. “PC Algorithm for Nonparanormal Graphical Models.” Journal of Machine Learning Research 14 (1): 3365–83.

Hinton, Geoffrey E., Simon Osindero, and Kejie Bao. 2005. “Learning Causally Linked Markov Random Fields.” In Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics, 128–35. Citeseer.

Jung, Alexander, Nguyen Tran Quang, and Alexandru Mara. 2017. “When Is Network Lasso Accurate?” April.

Khoshgnauz, Ehsan. 2012. “Learning Markov Network Structure Using Brownian Distance Covariance,” June.

Kocaoglu, Murat, Alex Dimakis, and Sriram Vishwanath. 2017. “Cost-Optimal Learning of Causal Graphs.” In PMLR, 1875–84.

Krämer, Nicole, Juliane Schäfer, and Anne-Laure Boulesteix. 2009. “Regularized Estimation of Large-Scale Gene Association Networks Using Graphical Gaussian Models.” BMC Bioinformatics 10 (1): 384.

Lederer, Johannes. 2016. “Graphical Models for Discrete and Continuous Data,” September.

Lee, Su-In, Varun Ganapathi, and Daphne Koller. 2006. “Efficient Structure Learning of Markov Networks Using $ L_1 $-Regularization.” In Advances in Neural Information Processing Systems, 817–24. MIT Press.

Liu, Han, Fang Han, Ming Yuan, John Lafferty, and Larry Wasserman. 2012. “The Nonparanormal SKEPTIC,” June.

Liu, Han, John Lafferty, and Larry Wasserman. 2009. “The Nonparanormal: Semiparametric Estimation of High Dimensional Undirected Graphs.” Journal of Machine Learning Research 10 (December): 2295–2328.

Mansinghka, Vikash, Charles Kemp, Thomas Griffiths, and Joshua Tenenbaum. 2012. “Structured Priors for Structure Learning,” June.

Mazumder, Rahul, and Trevor Hastie. 2012. “The Graphical Lasso: New Insights and Alternatives.” Electronic Journal of Statistics 6 (November): 2125–49.

Montanari, Andrea. 2012. “Graphical Models Concepts in Compressed Sensing.” Compressed Sensing: Theory and Applications, 394–438.

Neapolitan, Richard E., and others. 2004. Learning Bayesian Networks. Vol. 38. Prentice Hall Upper Saddle River.

Peters, Jonas, Joris Mooij, Dominik Janzing, and Bernhard Schoelkopf. 2012. “Identifiability of Causal Graphs Using Functional Models,” February.

Ramsey, Joseph, Madelyn Glymour, Ruben Sanchez-Romero, and Clark Glymour. 2017. “A Million Variables and More: The Fast Greedy Equivalence Search Algorithm for Learning High-Dimensional Graphical Causal Models, with an Application to Functional Magnetic Resonance Images.” International Journal of Data Science and Analytics 3 (2): 121–29.

Schelldorfer, Jürg, Lukas Meier, and Peter Bühlmann. 2014. “GLMMLasso: An Algorithm for High-Dimensional Generalized Linear Mixed Models Using ℓ1-Penalization.” Journal of Computational and Graphical Statistics 23 (2): 460–77.

Textor, Johannes, Alexander Idelberger, and Maciej Liśkiewicz. 2015. “Learning from Pairwise Marginal Independencies,” August.

Wu, Rui, R. Srikant, and Jian Ni. 2012. “Learning Graph Structures in Discrete Markov Random Fields.” In INFOCOM Workshops, 214–19.

Zhao, Tuo, Han Liu, Kathryn Roeder, John Lafferty, and Larry Wasserman. 2012. “The Huge Package for High-Dimensional Undirected Graph Estimation in R.” Journal of Machine Learning Research : JMLR 13 (April): 1059–62.

Zhou, Mingyuan, Yulai Cong, and Bo Chen. 2017. “Augmentable Gamma Belief Networks,” 44.