The Living Thing / Notebooks :

Linear algebra

If the thing is twice as big, the transformed version of the thing is also twice as big. {End}

Usefulness: 🔧
Novelty: 💡
Uncertainty: 🤪 🤪 🤪
Incompleteness: 🚧 🚧 🚧

Oh! the hours I put in to studying the taxonomy and husbandry of matrices. Time has passed. I have forgotten much. Jacobians have begun to seem downright Old Testament.

And when you put the various operations of matrix calculus into the mix (derivative of trace of a skew-hermitian heffalump painted with a camel-hair brush) the combinatorial explosions of theorems and identities is intimidating.

Things I need:

Basic linear algebra intros

Linear algebra and calculus

The multidimensional statistics/control theory workhorse.

See matrix calculus.

Multilinear Algebra

Oooh you are playing with tensors? I don’t have a bunch to say here but here is a compact explanation of Einstein summation, which turns out to be as simple as it needs to be, but no simpler.

Refs

Alexander Graham. 1981. Kronecker Products and Matrix Calculus: With Applications. Horwood.

Axler, Sheldon. 1995. “Down with Determinants!” The American Mathematical Monthly 102 (2): 139–54. https://doi.org/10.2307/2975348.

———. 2014. Linear Algebra Done Right. New York: Springer. http://dx.doi.org/10.1007/978-3-319-11080-6.

Boyd, Stephen P., and Lieven Vandenberghe. 2018. Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares. Cambridge, UK ; New York, NY: Cambridge University Press.

Darrell A. Turkington. 2001. Matrix Calculus Zero-One Matrices. Cambridge University Press.

Dwyer, Paul S. 1967. “Some Applications of Matrix Derivatives in Multivariate Analysis.” Journal of the American Statistical Association 62 (318): 607. https://doi.org/10.2307/2283988.

Gene H. Golub, and Charles F. van Loan. 1983. Matrix Computations. JHU Press.

George A. F. Seber. 2007. A Matrix Handbook for Statisticians. Wiley.

Giles, M. 2008. “An Extended Collection of Matrix Derivative Results for Forward and Reverse Mode Automatic Differentiation.” Http://Eprints.maths.ox.ac.uk/1079, January. http://www2.maths.ox.ac.uk/~gilesm/files/NA-08-01.pdf.

Giles, Mike B. 2008. “Collected Matrix Derivative Results for Forward and Reverse Mode Algorithmic Differentiation.” In Advances in Automatic Differentiation, edited by Christian H. Bischof, H. Martin Bücker, Paul Hovland, Uwe Naumann, and Jean Utke, 64:35–44. Berlin, Heidelberg: Springer Berlin Heidelberg. http://eprints.maths.ox.ac.uk/1079/.

Laue, Soeren, Matthias Mitterreiter, and Joachim Giesen. 2018. “Computing Higher Order Derivatives of Matrix and Tensor Expressions.” In Advances in Neural Information Processing Systems 31, edited by S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, 2750–9. Curran Associates, Inc. http://papers.nips.cc/paper/7540-computing-higher-order-derivatives-of-matrix-and-tensor-expressions.pdf.

Magnus, Jan R., and Heinz Neudecker. 1999. Matrix Differential Calculus with Applications in Statistics and Econometrics. Rev. ed. New York: John Wiley. http://www.janmagnus.nl/misc/mdc2007-3rdedition.

Minka, Thomas P. 2000. “Old and New Matrix Algebra Useful for Statistics.” http://msr-waypoint.com/en-us/um/people/minka/papers/matrix/minka-matrix.pdf.

Parlett, Beresford N. 2000. “The QR Algorithm.” Computing in Science & Engineering 2 (1): 38–42. https://doi.org/10.1109/5992.814656.

Petersen, Kaare Brandt, and Michael Syskind Pedersen. 2012. “The Matrix Cookbook.” http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=3274.

Willi-Hans Steeb. 2006. Problems and Solutions in Introductory and Advanced Matrix Calculus. World Scientific.