The Living Thing / Notebooks :

Long memory models

Stochastic ` where we know that ancient history is still relevent for the future predictions, even if we know the recent history.

In my own mental map this is near-synonymous with stateful models but where we ignore the state, where we ignore the state, which I suppose is a kind of coarse graining. In this formulation, we have a Markov process but because we do not observe the whole state it looks non-Markov. This is reasonably consistent with reality, where we believe the current state of reality determines the future, but we don’t know the whole current state.

(Sorta related: hidden variable quantum mechanics.)

But sure, maybe there is an explit long-time dependency in your process which is not nediated through a hidden state; can we statistically distingish these cases? Sometimes, e.g. Küns86, but generally you probably want a model, or how will you do it?

There are some obvious process model which have a long-memeory in this sense, such as stack automata, or Hawkes processes with non-exponential kernels.

Note “long memory” can work with not only time series by but any random field, e.g. spatial, or random fields indexed by any number of dimensions, causal or non-causal.


  1. Can we use the “memory length” of a system to infer the number of hidden states for some class of interesting systems, or vice versa?
  1. When?
  1. Can we infer the memory length alone as a parameter of interest in some model classes? (need making precise). Information criteria don’t do this model order selection consistently; what does?


Beran, J. (1992) Statistical Methods for Data with Long-Range Dependence. Statistical Science, 7(4), 404–416.
Beran, J. (2010) Long-range dependence. Wiley Interdisciplinary Reviews: Computational Statistics, 2(1), 26–35. DOI.
Berkes, I., Horváth, L., Kokoszka, P., & Shao, Q.-M. (2006) On discriminating between long-range dependence and changes in mean. The Annals of Statistics, 34(3), 1140–1165. DOI.
Breidt, F. J., Crato, N., & de Lima, P. (1998) The detection and estimation of long memory in stochastic volatility. Journal of Econometrics, 83(1–2), 325–348.
Csörgö, S., & Mielniczuk, J. (1999) Random-design regression under long-range dependent errors. Bernoulli, 5(2), 209–224. DOI.
Diebold, F. X., & Inoue, A. (2001) Long memory and regime switching. Journal of Econometrics, 105(1), 131–159. DOI.
Doukhan, P., Oppenheim, G., & Taqqu, M. S.(2003) Theory and applications of long-range dependence. . Birkhauser
Farmer, J. D., Gerig, A., Lillo, F., & Mike, S. (2007) Market efficiency and the long-memory of supply and demand: is price impact variable and permanent or fixed and temporary?. Quantitative Finance, 6(2), 107–112. DOI.
Giraitis, L., & Surgailis, D. (1999) Central limit theorem for the empirical process of a linear sequence with long memory. Journal of Statistical Planning and Inference, 80(1–2), 81–93.
Gneiting, T. (2000) Power-law correlations, related models for long-range dependence and their simulation. Journal of Applied Probability, 37(4), 1104–1109.
Granger, C. W. J., & Joyeux, R. (1980) An Introduction to Long-Memory Time Series Models and Fractional Differencing. Journal of Time Series Analysis, 1(1), 15–29. DOI.
Horváth, L. (2001) Change-point detection in long-memory processes. Journal of Multivariate Analysis, 78(2), 218–234.
Künsch, H. R.(1986) Discrimination between monotonic trends and long-range dependence. Journal of Applied Probability, 23(4), 1025–1030.
Lahiri, S. N.(1993) On the moving block bootstrap under long range dependence. Statistics & Probability Letters, 18(5), 405–413. DOI.
Robinson, P. M.(2003) Time series with long memory. . Oxford Univ Pr
Saichev, A. I., & Sornette, D. (2010) Generation-by-generation dissection of the response function in long memory epidemic processes. The European Physical Journal B, 75(3), 343–355. DOI.