The Living Thing / Notebooks :

Dimensionality reduction

Wherein I teach myself, amongst other things, how a *sparse* PCA works, and decide where to file multidimensional scaling.

Related: “clustering”? Also with the notion of similarity as seen in kernel tricks. Inducing a differential metric. Matrix factorisations and random features, high-dimensional statistics and discuss random projections and their role in compressed sensing etc.


PCA and cousins

Kernel PCA, linear algebra and probabilistic formulations.

Linear algebra version of PCA puts us in the world of matrix factorisationa.

Non-linear versions

Autoencoder and word2vec

“The nonlinear PCA” interpretation, I just heard from Junbin Gao.

\[L(x, x') = \|x-x\|^2=\|x-\sigma(U*sigma*W^Tx+b)) + b')\|^2\]

Locality Preserving projections

Try to preserve the nearness of points if they are connected on some (weight) graph.

\[\sum_{i,j}(y_i-y_j)^2 w_{i,j}\]

So we seen an optimal projection vector.

(requirement for sparse similarity matrix?)

Multidimensional scaling


Topological data analysis

Start with the distances between points and try to find a lower dimensional manifold which preserves their distances. Local MDS? TDB.

Random projection

see random embeddings

Stochastic neighbour embedding

Probabilisitically preserving closeness.