Transforming something (let us assume, discrete experimental data rather than abstract mathematical objects) into a new basis, as in the Fourier or (no longer orthogonal, Laplace or z-transforms), usually with a goal of handling some inconveniently curvy data as linear data plus a non-local curvy transformation. This is big in, e.g. signal processing and time series analysis.

Other names: Spectral methods, Hilbert space methods, basis expansions, orthogonal projection methods.

An *orthogonal* basis requires you to have a domain and inner product specified.
If you choose a new inner product, you need a different base to be orthogonal in general.
If you use the machinery of an implicit Hilbert space for a given inner product
without troubling to define an explicit form for the “feature space”, you are
doing the kernel trick.

Todo: discuss orthogonal measurements in terms of isometry properties.

## General setting

Here I am especially interested in the *discrete* case, by which I mean
that we are finding orthogonal bases
for signal vectors of countable, often finite, length,
e.g. if we are looking at spaces of 1 second of audio at 44kHz,
then we consider spaces \(\mathbb{R}^{44100}\).
If we are looking at RGB webcam images we might consider \(\mathbb{R}_+^{640\times 480 \times 3}\).
Apparently this case can be related to the continuous case by considering general transforms on Locally Compact Abelian groups, via
Pontryagin_duality.

See the following course notes

## Orthogonal basis zoo

- Discrete Fourier transform and cousins.
Mostly a list of things to look up later,
since I can’t remember what any of them are right now, and I ran across them
when reading about something else. Idk.
- Discrete Hartley transform
- Hadamard transform
- Spherical harmonic transform (What is this again?)
- MDCT (TODO - understand how this works for audio resynthesis and psychoacoustic purposes)

- Legendre transform (into Legendre polynomials)
- Bases of disjoint support; the orthogonal bases for non-negative/non-positive data
- Practical: Welch periodograms, and Tukey DTFT

For related ideas, see also

- functional regression,
- sparse basis dictionaries (which are not necc orthogonal),
- Hilbert space miscellanea.

## Refs

- BoBl07
- Bosq, D., & Blanke, D. (2007) Inference and prediction in large dimensions. . Chichester, England ; Hoboken, NJ: John Wiley/Dunod
- Gaba00
- Gabardo, J.-P. (2000) Hilbert spaces of distributions having an orthogonal basis of exponentials.
*Journal of Fourier Analysis and Applications*, 6(3), 277–298. DOI. - Jone92
- Jones, L. K.(1992) A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training.
*The Annals of Statistics*, 20(1), 608–613. - MGVB11
- Mailhé, B., Gribonval, R., Vandergheynst, P., & Bimbot, F. (2011) Fast orthogonal sparse approximation algorithms over local dictionaries.
*Signal Processing*, 91(12), 2822–2835. DOI. - ReSt00
- Reiter, H., & Stegeman, J. D.(2000) Classical harmonic analysis and locally compact groups. . Courier Corporation
- Rudi17
- Rudin, W. (2017) Fourier analysis on groups. . Courier Dover Publications
- UnTa14
- Unser, M. A., & Tafti, P. (2014) An introduction to sparse stochastic processes. . New York: Cambridge University Press
- WaST14
- Wang, Y.-X., Smola, A., & Tibshirani, R. J.(2014) The Falling Factorial Basis and Its Statistical Applications.
*ArXiv:1405.0558 [Stat]*. - WiBö15
- Wiatowski, T., & Bölcskei, H. (2015) A Mathematical Theory of Deep Convolutional Neural Networks for Feature Extraction.
*ArXiv:1512.06293 [Cs, Math, Stat]*.