Learning the graph structure, not just the clique potentials.
Much more work.
Learning these models turns out to need a conditional independence test, an awareness of multiple testing and graphs.
bnlearn learns belief networks
A new R package for learning sparse Bayesian networks and other graphical models from high-dimensional data via sparse regularization. Designed from the ground up to handle:
- Experimental data with interventions
- Mixed observational / experimental data
- High-dimensional data with p >> n
- Datasets with thousands of variables (tested up to p=8000)
- Continuous and discrete data
The emphasis of this package is scalability and statistical consistency on high-dimensional datasets. […] For more details on this package, including worked examples and the methodological background, please see our new preprint.
The main methods for learning graphical models are:
- estimate.dag for directed acyclic graphs (Bayesian networks).
- estimate.precision for undirected graphs (Markov random fields).
- estimate.covariance for covariance matrices.
Currently, estimation of precision and covariances matrices is limited to Gaussian data.
Nonparanormal skeptic (🚧)
Bayati, M., and A. Montanari. 2012. “The LASSO Risk for Gaussian Matrices.” IEEE Transactions on Information Theory 58 (4): 1997–2017. https://doi.org/10.1109/TIT.2011.2174612.
Buntine, W. L. 1996. “A Guide to the Literature on Learning Probabilistic Networks from Data.” IEEE Transactions on Knowledge and Data Engineering 8 (2): 195–210. https://doi.org/10.1109/69.494161.
Bühlmann, Peter, and Sara van de Geer. 2011. Statistics for High-Dimensional Data: Methods, Theory and Applications. 2011 edition. Heidelberg ; New York: Springer.
Cai, T. Tony. 2017. “Global Testing and Large-Scale Multiple Testing for High-Dimensional Covariance Structures.” Annual Review of Statistics and Its Application 4 (1): 423–46. https://doi.org/10.1146/annurev-statistics-060116-053754.
Colombo, Diego, Marloes H. Maathuis, Markus Kalisch, and Thomas S. Richardson. 2012. “Learning High-Dimensional Directed Acyclic Graphs with Latent and Selection Variables.” The Annals of Statistics 40 (1): 294–321. http://projecteuclid.org/euclid.aos/1333567191.
Cox, D. R., and H. S. Battey. 2017. “Large Numbers of Explanatory Variables, a Semi-Descriptive Analysis.” Proceedings of the National Academy of Sciences 114 (32): 8592–5. https://doi.org/10.1073/pnas.1703764114.
Drton, Mathias, and Marloes H. Maathuis. 2017. “Structure Learning in Graphical Modeling.” Annual Review of Statistics and Its Application 4 (1): 365–93. https://doi.org/10.1146/annurev-statistics-060116-053803.
Friedman, Jerome, Trevor Hastie, and Robert Tibshirani. 2008. “Sparse Inverse Covariance Estimation with the Graphical Lasso.” Biostatistics 9 (3): 432–41. https://doi.org/10.1093/biostatistics/kxm045.
Fu, Fei, and Qing Zhou. 2013. “Learning Sparse Causal Gaussian Networks with Experimental Intervention: Regularization and Coordinate Descent.” Journal of the American Statistical Association 108 (501): 288–300. https://doi.org/10.1080/01621459.2012.754359.
Geer, Sara van de. 2014. “Worst Possible Sub-Directions in High-Dimensional Models.” In. Vol. 131. http://arxiv.org/abs/1403.7023.
Geng, Zhi, Yue Liu, Chunchen Liu, and Wang Miao. 2019. “Evaluation of Causal Effects and Local Structure Learning of Causal Networks.” Annual Review of Statistics and Its Application 6 (1): 103–24. https://doi.org/10.1146/annurev-statistics-030718-105312.
Gogate, Vibhav, William Webb, and Pedro Domingos. 2010. “Learning Efficient Markov Networks.” In Advances in Neural Information Processing Systems, 748–56. http://papers.nips.cc/paper/4010-learning-efficient-markov-networks.
Hallac, David, Jure Leskovec, and Stephen Boyd. 2015. “Network Lasso: Clustering and Optimization in Large Graphs,” July. https://doi.org/10.1145/2783258.2783313.
Harris, Naftali, and Mathias Drton. 2013. “PC Algorithm for Nonparanormal Graphical Models.” Journal of Machine Learning Research 14 (1): 3365–83. http://jmlr.org/papers/v14/harris13a.html.
Hinton, Geoffrey E., Simon Osindero, and Kejie Bao. 2005. “Learning Causally Linked Markov Random Fields.” In Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics, 128–35. Citeseer. http://www.cs.toronto.edu/~osindero/PUBLICATIONS/HintonOsinderoBao05_CLMRF.pdf.
Jung, Alexander, Nguyen Tran Quang, and Alexandru Mara. 2017. “When Is Network Lasso Accurate?” April. http://arxiv.org/abs/1704.02107.
Khoshgnauz, Ehsan. 2012. “Learning Markov Network Structure Using Brownian Distance Covariance,” June. http://arxiv.org/abs/1206.6361.
Kocaoglu, Murat, Alex Dimakis, and Sriram Vishwanath. 2017. “Cost-Optimal Learning of Causal Graphs.” In PMLR, 1875–84. http://proceedings.mlr.press/v70/kocaoglu17a.html.
Krämer, Nicole, Juliane Schäfer, and Anne-Laure Boulesteix. 2009. “Regularized Estimation of Large-Scale Gene Association Networks Using Graphical Gaussian Models.” BMC Bioinformatics 10 (1): 384. https://doi.org/10.1186/1471-2105-10-384.
Lederer, Johannes. 2016. “Graphical Models for Discrete and Continuous Data,” September. http://arxiv.org/abs/1609.05551.
Lee, Su-In, Varun Ganapathi, and Daphne Koller. 2006. “Efficient Structure Learning of Markov Networks Using $ L_1 $-Regularization.” In Advances in Neural Information Processing Systems, 817–24. MIT Press. http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2006_849.pdf.
Liu, Han, Fang Han, Ming Yuan, John Lafferty, and Larry Wasserman. 2012. “The Nonparanormal SKEPTIC,” June. http://arxiv.org/abs/1206.6488.
Liu, Han, John Lafferty, and Larry Wasserman. 2009. “The Nonparanormal: Semiparametric Estimation of High Dimensional Undirected Graphs.” Journal of Machine Learning Research 10 (December): 2295–2328. http://jmlr.org/papers/v10/liu09a.html.
Mansinghka, Vikash, Charles Kemp, Thomas Griffiths, and Joshua Tenenbaum. 2012. “Structured Priors for Structure Learning,” June. http://arxiv.org/abs/1206.6852.
Mazumder, Rahul, and Trevor Hastie. 2012. “The Graphical Lasso: New Insights and Alternatives.” Electronic Journal of Statistics 6 (November): 2125–49. https://doi.org/10.1214/12-EJS740.
Montanari, Andrea. 2012. “Graphical Models Concepts in Compressed Sensing.” Compressed Sensing: Theory and Applications, 394–438. http://arxiv.org/abs/1011.4328.
Neapolitan, Richard E., and others. 2004. Learning Bayesian Networks. Vol. 38. Prentice Hall Upper Saddle River. https://books.secure-services.me/Gentoomen%20Library/Artificial%20Intelligence/Bayesian%20networks/Learning%20Bayesian%20Networks%20-%20Neapolitan%20R.%20E..pdf.
Peters, Jonas, Joris Mooij, Dominik Janzing, and Bernhard Schoelkopf. 2012. “Identifiability of Causal Graphs Using Functional Models,” February. http://arxiv.org/abs/1202.3757.
Ramsey, Joseph, Madelyn Glymour, Ruben Sanchez-Romero, and Clark Glymour. 2017. “A Million Variables and More: The Fast Greedy Equivalence Search Algorithm for Learning High-Dimensional Graphical Causal Models, with an Application to Functional Magnetic Resonance Images.” International Journal of Data Science and Analytics 3 (2): 121–29. https://doi.org/10.1007/s41060-016-0032-z.
Schelldorfer, Jürg, Lukas Meier, and Peter Bühlmann. 2014. “GLMMLasso: An Algorithm for High-Dimensional Generalized Linear Mixed Models Using ℓ1-Penalization.” Journal of Computational and Graphical Statistics 23 (2): 460–77. https://doi.org/10.1080/10618600.2013.773239.
Textor, Johannes, Alexander Idelberger, and Maciej Liśkiewicz. 2015. “Learning from Pairwise Marginal Independencies,” August. http://arxiv.org/abs/1508.00280.
Wu, Rui, R. Srikant, and Jian Ni. 2012. “Learning Graph Structures in Discrete Markov Random Fields.” In INFOCOM Workshops, 214–19. http://www.ifp.illinois.edu/~srikant/WorkingPapers/wusrini11.pdf.
Zhao, Tuo, Han Liu, Kathryn Roeder, John Lafferty, and Larry Wasserman. 2012. “The Huge Package for High-Dimensional Undirected Graph Estimation in R.” Journal of Machine Learning Research : JMLR 13 (April): 1059–62. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4729207/.
Zhou, Mingyuan, Yulai Cong, and Bo Chen. 2017. “Augmentable Gamma Belief Networks,” 44.