The Living Thing / Notebooks :

RKHS distribution embedding

The intersection of reproducing kernel methods, dependence tests <{filename}independence.rst>`_and `probability metrics; where you use a clever RKHS embedding to measure probability distributions.

A mere placeholder for now.

This abstract by Zoltán Szabó might serve to highlight some keywords.

Maximum mean discrepancy (MMD) and Hilbert-Schmidt independence criterion (HSIC) are among the most popular and successful approaches in applied mathematics to measure the difference and the independence of random variables, respectively. Thanks to their kernel-based foundations, MMD and HSIC are applicable on a large variety of domains such as documents, images, trees, graphs, time series, dynamical systems, sets or permutations. Despite their tremendous practical success, quite little is known about when HSIC characterizes independence and MMD with tensor kernel can discriminate probability distributions, in terms of the contributing kernel components. In this talk, I am going to provide a complete answer to this question, with conditions which are often easy to verify in practice. [Joint work with Bharath K. Sriperumbudur (PSU).

ITE toolbox (estimators)

Refs

GFTS08
Gretton, A., Fukumizu, K., Teo, C. H., Song, L., Schölkopf, B., & Smola, A. J.(2008) A Kernel Statistical Test of Independence. In Advances in Neural Information Processing Systems 20: Proceedings of the 2007 Conference. Cambridge, MA: MIT Press
Muan00
Muandet et al. - 2016 - Kernel Mean Embedding of Distributions A Review a.pdf. (n.d.) http://www.arxiv.org/pdf/1605.09522.pdf.
MFSG14
Muandet, K., Fukumizu, K., Sriperumbudur, B., Gretton, A., & Schölkopf, B. (2014) Kernel Mean Shrinkage Estimators. ArXiv:1405.5505 [Cs, Stat].
MFSS17
Muandet, K., Fukumizu, K., Sriperumbudur, B., & Schölkopf, B. (2017) Kernel Mean Embedding of Distributions: A Review and Beyond. Foundations and Trends® in Machine Learning, 10(1–2), 1–141. DOI.
ReWi09
Reid, M. D., & Williamson, R. C.(2009) Generalised Pinsker Inequalities. In arXiv:0906.1244 [cs, math].
ReWi11
Reid, M. D., & Williamson, R. C.(2011) Information, Divergence and Risk for Binary Experiments. Journal of Machine Learning Research, 12(Mar), 731–817.
SMFP15
Schölkopf, B., Muandet, K., Fukumizu, K., & Peters, J. (2015) Computing Functions of Random Variables via Reproducing Kernel Hilbert Space Representations. ArXiv:1501.06794 [Cs, Stat].
SSGF12
Sejdinovic, D., Sriperumbudur, B., Gretton, A., & Fukumizu, K. (2012) Equivalence of distance-based and RKHS-based statistics in hypothesis testing. The Annals of Statistics, 41(5), 2263–2291. DOI.
SGSS07
Smola, A., Gretton, A., Song, L., & Schölkopf, B. (2007) A Hilbert Space Embedding for Distributions. In M. Hutter, R. A. Servedio, & E. Takimoto (Eds.), Algorithmic Learning Theory (pp. 13–31). Springer Berlin Heidelberg
SHSF09
Song, L., Huang, J., Smola, A., & Fukumizu, K. (2009) Hilbert Space Embeddings of Conditional Distributions with Applications to Dynamical Systems. In Proceedings of the 26th Annual International Conference on Machine Learning (pp. 961–968). New York, NY, USA: ACM DOI.
SFGS12
Sriperumbudur, B. K., Fukumizu, K., Gretton, A., Schölkopf, B., & Lanckriet, G. R. G.(2012) On the empirical estimation of integral probability metrics. Electronic Journal of Statistics, 6, 1550–1599. DOI.
SGFL08
Sriperumbudur, B. K., Gretton, A., Fukumizu, K., Lanckriet, G., & Schölkopf, B. (2008) Injective Hilbert Space Embeddings of Probability Measures. In Proceedings of the 21st Annual Conference on Learning Theory (COLT 2008).
SGFS10
Sriperumbudur, B. K., Gretton, A., Fukumizu, K., Schölkopf, B., & Lanckriet, G. R. G.(2010) Hilbert Space Embeddings and Metrics on Probability Measures. Journal of Machine Learning Research, 11, 1517−1561.
StZV17
Strobl, E. V., Zhang, K., & Visweswaran, S. (2017) Approximate Kernel-based Conditional Independence Tests for Fast Non-Parametric Causal Discovery. ArXiv:1702.03877 [Stat].
SzSr17
Szabo, Z., & Sriperumbudur, B. K.(2017) Characteristic and Universal Tensor Product Kernels. ArXiv:1708.08157 [Cs, Math, Stat].
ZPJS12
Zhang, K., Peters, J., Janzing, D., & Schölkopf, B. (2012) Kernel-based Conditional Independence Test and Application in Causal Discovery. ArXiv:1202.3775 [Cs, Stat].
ZFGS16
Zhang, Q., Filippi, S., Gretton, A., & Sejdinovic, D. (2016) Large-Scale Kernel Methods for Independence Testing. ArXiv:1606.07892 [Stat].