The Living Thing / Notebooks : Statistical learning theory for dependent data

Statistical learning theory for dependent data such as time series and other dpendency structures.

Non-stationary, non-asymptotic bounds please. Keywords: Ergodic, α-, β-mixing.

Mohri and Kuznetsov have done lots of work here; See, e.g. their NIPS2016 tutorial, or KuMo16.


Alquier, P., Li, X., & Wintenberger, O. (2013) Prediction of time series by statistical learning: general losses and fast rates. Dependence Modeling, 1, 65–93. DOI.
Alquier, P., & Wintenberger, O. (2012) Model selection for weakly dependent time series forecasting. Bernoulli.
Cortes, C., Kuznetsov, V., Mohri, M., & Yang, S. (2016) Structured Prediction Theory Based on Factor Graph Complexity. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, & R. Garnett (Eds.), Advances in Neural Information Processing Systems 29 (pp. 2514–2522). Curran Associates, Inc.
Kontorovich, L. (Aryeh), Cortes, C., & Mohri, M. (2008) Kernel methods for learning languages. Theoretical Computer Science, 405(3), 223–236. DOI.
Kontorovich, L., Cortes, C., & Mohri, M. (2006) Learning Linearly Separable Languages. In J. L. Balcázar, P. M. Long, & F. Stephan (Eds.), Algorithmic Learning Theory (pp. 288–303). Springer Berlin Heidelberg
Kuznetsov, V., & Mohri, M. (2014) Forecasting Non-Stationary Time Series: From Theory to Algorithms.
Kuznetsov, V., & Mohri, M. (2015) Learning Theory and Algorithms for Forecasting Non-Stationary Time Series. In Advances in Neural Information Processing Systems (pp. 541–549). Curran Associates, Inc.
Kuznetsov, V., & Mohri, M. (2016) Generalization Bounds for Non-stationary Mixing Processes. In Machine Learning Journal.
McDonald, D. J., Shalizi, C. R., & Schervish, M. (2011a) Generalization error bounds for stationary autoregressive models. arXiv:1103.0942 [Cs, Stat].
McDonald, D. J., Shalizi, C. R., & Schervish, M. (2011b) Risk bounds for time series without strong mixing. arXiv:1106.0730 [Cs, Stat].
Mohri, M., & Rostamizadeh, A. (2009) Rademacher complexity bounds for non-iid processes. In Advances in Neural Information Processing Systems (pp. 1097–1104).
Rakhlin, A., Sridharan, K., & Tewari, A. (2014) Sequential complexities and uniform martingale laws of large numbers. Probability Theory and Related Fields, 161(1–2), 111–153. DOI.
van de Geer, S. (2002) On Hoeffdoing’s inequality for dependent random variables. In Empirical Process Techniques for Dependent Data. Birkhhäuser