Spun off from a reading group I ran to the theme what even is is the Wasserstein GAN? It’s a particular type of generative adversarial network.
The Wasserstein GAN paper (Arjovsky, Chintala, and Bottou 2017) made enough of a splash that it’s worth considering separately from the other GAN stuff. Is it even “adversarial”? That looks marginal to me.
Today I’m leading a reading group GANs are famous for generating images, but I am interested in their use in simulating from difficult distributions in general.
I will not summarize WGANs better than the following handy sources, so these are the basis for the tutorial until such time as I find myself actually using this stuff in my own work.
- Alexi Pan reads the WGAN paper.
- Mindcodec discusses Wasserstein-type metrics, i.e. optimal transport ones, with an eye to WGAN.
- Here is a deep learning course that culminates in WGAN with some involvement by the authors of the WGAN paper.
- Vincent Hermann presents the Kantorovich-Rubinstein duality trick intuitively.
Connection to other types of regularisation? (Gulrajani et al. 2017; Miyato et al. 2018)
Arjovsky, Martin, Soumith Chintala, and Léon Bottou. 2017. “Wasserstein Generative Adversarial Networks.” In International Conference on Machine Learning, 214–23. http://proceedings.mlr.press/v70/arjovsky17a.html.
Goodfellow, Ian, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. “Generative Adversarial Nets.” In Advances in Neural Information Processing Systems 27, edited by Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, 2672–80. NIPS’14. Cambridge, MA, USA: Curran Associates, Inc. http://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf.
Gulrajani, Ishaan, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. 2017. “Improved Training of Wasserstein GANs,” March. http://arxiv.org/abs/1704.00028.
Lee, Hung-yi, and Yu Tsao. n.d. “Generative Adversarial Network.” In, 222.
Miyato, Takeru, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. 2018. “Spectral Normalization for Generative Adversarial Networks.” In ICLR 2018. http://arxiv.org/abs/1802.05957.
Panaretos, Victor M., and Yoav Zemel. 2019. “Statistical Aspects of Wasserstein Distances.” Annual Review of Statistics and Its Application 6 (1): 405–31. https://doi.org/10.1146/annurev-statistics-030718-104938.