The Living Thing / Notebooks :

Statistics for long memory processes

Stochastic processes where we know that ancient history is still relevant for the future predictions, even if we know the recent history; how do we analyse such things?

I haven’t said anything about the types of generating process here. Maybe there is an explicit long-time dependency in your process which is not mediated through a hidden state; can we statistically distinguish these cases? Sometimes, e.g. Küns86, but generally you probably want a model, or how will you do it?

There are some obvious process model which have a long-memory in this sense, such as stack automata, or Hawkes processes with non-exponential kernels.

Note that “long memory” can work with not only time series but any random field, e.g. spatial, or random fields indexed by any number of dimensions, over whatever topology, causal or non-causal. I guess you’d need a notion of distance to make this meaningful, so let’s presume we are on a metric space at least.

Questions for me:

  1. Can we use the “memory length” of a system to infer the number of hidden states for some class of interesting systems, or vice versa?
  1. If so, when?
  2. Is the length even computable? Under what assumptions?
  1. Can we infer the memory length alone as a parameter of interest in some model classes? (need making precise).

For now, see learning theory for dependent data, which has some concrete results for predictive time series.


Beran, J. (1992) Statistical Methods for Data with Long-Range Dependence. Statistical Science, 7(4), 404–416.
Beran, J. (1994) Statistics for Long-Memory Processes. . CRC Press
Beran, J. (2010) Long-range dependence. Wiley Interdisciplinary Reviews: Computational Statistics, 2(1), 26–35. DOI.
Beran, J., & Terrin, N. (1996) Testing for a change of the long-memory parameter. Biometrika, 83(3), 627–638. DOI.
Berkes, I., Horváth, L., Kokoszka, P., & Shao, Q.-M. (2006) On discriminating between long-range dependence and changes in mean. The Annals of Statistics, 34(3), 1140–1165. DOI.
Csörgö, S., & Mielniczuk, J. (1999) Random-design regression under long-range dependent errors. Bernoulli, 5(2), 209–224. DOI.
Doukhan, P., Oppenheim, G., & Taqqu, M. S.(2003) Theory and applications of long-range dependence. . Birkhauser
Giraitis, L., & Surgailis, D. (1999) Central limit theorem for the empirical process of a linear sequence with long memory. Journal of Statistical Planning and Inference, 80(1–2), 81–93.
Granger, C. W. J., & Joyeux, R. (1980) An Introduction to Long-Memory Time Series Models and Fractional Differencing. Journal of Time Series Analysis, 1(1), 15–29. DOI.
Hurvich, C. M.(2002) Multistep forecasting of long memory series using fractional exponential models. International Journal of Forecasting, 18(2), 167–179. DOI.
Künsch, H. R.(1986) Discrimination between monotonic trends and long-range dependence. Journal of Applied Probability, 23(4), 1025–1030.
Lahiri, S. N.(1993) On the moving block bootstrap under long range dependence. Statistics & Probability Letters, 18(5), 405–413. DOI.
McLeod, A. I.(1998) Hyperbolic decay time series. Journal of Time Series Analysis, 19(4), 473–483. DOI.
Pipiras, V., & Taqqu, M. S.(2017) Long-range dependence and self-similarity. . Cambridge, United Kingdom ; New York, NY, USA: Cambridge University Press
Saichev, A. I., & Sornette, D. (2010) Generation-by-generation dissection of the response function in long memory epidemic processes. The European Physical Journal B, 75(3), 343–355. DOI.
Schmitt, F. G., & Huang, Y. (2016) Stochastic analysis of scaling time series: from turbulence theory to applications. . Cambridge: Cambridge University Press