# Functional regression, spline bases etc

Statistics where the samples are not just data but whole curves and manifolds, or subsamples from them. Function approximation meets statistics.

Functional data analysis, […] is about the analysis of information on curves or functions. For example, these twenty traces of the writing of “fda” are curves in two ways: first, as static traces on the page that you see after the writing is finished, and second, as two sets functions of time, one for the horizontal “X” coordinate, and the other for the vertical “Y” coordinate.

FDA is a collection statistical techniques for answering questions like, “What are the main ways in which the curves vary from one writing to another?” In fact, most of the questions and problems associated with the usual multivariate data analyzed by statistical packages like SAS and SPSS have their functional counterparts.

But what is unique about functional data is the possibility of also using information on the rates of change or derivatives of the curves. We use slopes, curvatures, and other characteristics made available because these curves are intrinsically smooth, and we can use this information in many useful ways. For example, our high school physics tells us that force = mass times acceleration, and that suggests that we look at the acceleration or second derivative of the pen’s position as a function of time. What we see in the plot of the magnitudes of the acceleration vector is that acceleration hits nearly ten meters/second/second. That’s a lot of energy! Equally remarkable is the stability of these acceleration records from one trial to the next. Also, note that where the acceleration magnitudes are near zero, both the X and Y accelerations must simultaneously be zero. The brain seems to know what it’s doing!

Regression upon the shapes of curves entire. A very nonparametric thing to do. Can be simpler than you’d think - just doing typical statistics on a functional basis, Hilbert-space-style. You can try to infer the differential operator that defines continuous dynamics. Apropos that, see the kernel trick. Many other nonparametric methods of function approximation, such as spline bases and density estimation, mixture models, and so on are generalised by functional data analysis representation.

See Wahb90 for the foundational spline-smoothing work, and RaSi05 for the name-check “Functional data” business.

An interesting sub-problem here is how you align the curves that are your objects of study; That is a problem of warping.

## Refs

BFLL15
Battey, H., Fan, J., Liu, H., Lu, J., & Zhu, Z. (2015) Distributed Estimation and Inference with Statistical Guarantees. arXiv:1509.05457 [Math, Stat].
BaLi14
Battey, H., & Linton, O. (2014) Nonparametric estimation of multivariate elliptic densities via finite mixture sieves. Journal of Multivariate Analysis, 123, 43–67. DOI.
BaLi13
Battey, H., & Liu, H. (2013) Smooth projected density estimation. arXiv:1308.3968 [Stat].
BaLi16
Battey, H., & Liu, H. (2016) Nonparametrically filtered parametric density estimation.
BaSa13
Battey, H., & Sancetta, A. (2013) Conditional estimation for dependent functional data. Journal of Multivariate Analysis, 120, 1–17. DOI.
HaSh00
Han, K., & Shin, H. (n.d.) Functional Linear Regression for Functional Response via Sparse Basis Selection.
HeBu14
Heinonen, M., & d’Alché-Buc, F. (2014) Learning nonparametric differential equations with operator-valued kernels and gradient matching. arXiv:1411.5172 [Cs, Stat].
HoKo12
Horváth, L., & Kokoszka, P. (2012) Inference for functional data with applications. (Vol. 200). New York: Springer
HsEu15
Hsing, T., & Eubank, R. L.(2015) Theoretical foundations of functional data analysis, with an introduction to linear operators. . Chichester, West Sussex: John Wiley and Sons, Inc
LiRH14
Liu, C., Ray, S., & Hooker, G. (2014) Functional Principal Components Analysis of Spatially Correlated Data. arXiv:1411.4681 [Math, Stat].
Morr15
Morris, J. S.(2015) Functional Regression. Annual Review of Statistics and Its Application, 2(1), 321–359. DOI.
PaSa14
Paparoditis, E., & Sapatinas, T. (2014) Bootstrap-based testing for functional data. arXiv:1409.4317 [Math, Stat].
RaHG09
Ramsay, J. O., Hooker, G., & Graves, S. (2009) Functional Data Analysis with R and MATLAB. (2009 edition.). Dordrecht ; New York: Springer
RaSi05
Ramsay, J. O., & Silverman, B. . (2005) Functional Data Analysis. . New York: Springer-Verlag
TaPa16
Tavakoli, S., & Panaretos, V. M.(2016) Detecting and Localizing Differences in Functional Time Series Dynamics: A Case Study in Molecular Biophysics. Journal of the American Statistical Association, 1–31. DOI.
Wahb90
Wahba, G. (1990) Spline Models for Observational Data. . SIAM