The Living Thing / Notebooks :

Feedback system identification, not necessarily linear

After all, if you have a system whose future evolution is important to predict, why not try to infer a plausible model instead of a convenient one?

I am in the process of taxonomising here. Stuff which fits the particular (likelihood) model of recursive estimation and so on will be kept there. Miscellaneous other approaches here.

A compact overview is inserted incidentally in Cosma’s review of Fan and Yao – FaYa03 — (wherein he also recommends Bosq98, TaKa00 and BoBl07.)

To reconstruct the state, as opposed to the parameters, you do state filtering. There can be interplay between these steps, if you are doing simulation-based online parameter inference, as in recursive estimation.

Anyway, for what kind of systems can you infer parameters? Mutually exciting point processes? Yep, EFBS04 do that.

From an engineering/control perspective, we have BrPK16, who give a sparse regression version. Generally it seems it can be done by indirect inference, or recursive hierarchical generalised linear models, generalising the process for linear time series.

There are many highly general formulations; Kita96 gives a Bayesian “smooth” one.

See e.g. the HeDG15 paper:

We address […] these problems with a new view of predictive state methods for dynamical system learning. In this view, a dynamical system learning problem is reduced to a sequence of supervised learning problems. So, we can directly apply the rich literature on supervised learning methods to incorporate many types of prior knowledge about problem structure. We give a general convergence rate analysis that allows a high degree of flexibility in designing estimators. And finally, implementing a new estimator becomes as simple as rearranging our data and calling the appropriate supervised learning subroutines.

[…] More specifically, our contribution is to show that we can use much-more- general supervised learning algorithms in place of linear regression, and still get a meaningful theoretical analysis. In more detail:

Also, sparsely or unevenly observed series are tricky. I’m looking at those at the moment.

Awaiting filing