The Living Thing / Notebooks :

Generative deep learning models

automatic fractal Trump

Many neural networks, — e.g. adversarial ones and variational ones — are generative - they simulate examples of the phenomenon you wish to analyse. This is close to the simulation-based inference we know about from many Bayesian methods; but substantially broader and weirder.

This is interesting to me for a bunch of reasons. I mostly care about it for making generative art with neural networks, but there is some theory here that isn’t about fractal Donald Trumps or whatever.

TBC.

Refs

AGLM17
Arora, S., Ge, R., Liang, Y., Ma, T., & Zhang, Y. (2017) Generalization and Equilibrium in Generative Adversarial Nets (GANs). ArXiv:1703.00573 [Cs].
BCCC17
Bahadori, M. T., Chalupka, K., Choi, E., Chen, R., Stewart, W. F., & Sun, J. (2017) Neural Causal Regularization under the Independence of Mechanisms Assumption. ArXiv:1702.02604 [Cs, Stat].
BlBo17
Blaauw, M., & Bonada, J. (2017) A Neural Parametric Singing Synthesizer. ArXiv:1704.03809 [Cs].
BVVD15
Bowman, S. R., Vilnis, L., Vinyals, O., Dai, A. M., Jozefowicz, R., & Bengio, S. (2015) Generating Sentences from a Continuous Space. ArXiv:1511.06349 [Cs].
CCDH16
Chen, X., Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., & Abbeel, P. (2016) InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, & R. Garnett (Eds.), Advances in Neural Information Processing Systems 29 (pp. 2172–2180). Curran Associates, Inc.
Coll08
Collins, N. (2008) The Analysis of Generative Music Programs. Organised Sound, 13(03), 237–248. DOI.
DCSF15
Denton, E., Chintala, S., Szlam, A., & Fergus, R. (2015) Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks. ArXiv:1506.05751 [Cs].
DSTB14
Dosovitskiy, A., Springenberg, J. T., Tatarchenko, M., & Brox, T. (2014) Learning to Generate Chairs, Tables and Cars with Convolutional Networks. ArXiv:1411.5928 [Cs].
ERRD17
Engel, J., Resnick, C., Roberts, A., Dieleman, S., Eck, D., Simonyan, K., & Norouzi, M. (2017) Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders. ArXiv:1704.01279 [Cs].
FSPW16
Fraccaro, M., Sø nderby, S. ren K., Paquet, U., & Winther, O. (2016) Sequential Neural Models with Stochastic Layers. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, & R. Garnett (Eds.), Advances in Neural Information Processing Systems 29 (pp. 2199–2207). Curran Associates, Inc.
GaGh15
Gal, Y., & Ghahramani, Z. (2015) On Modern Deep Learning and Variational Inference. In Advances in Approximate Bayesian Inference workshop, NIPS.
GaGh16
Gal, Y., & Ghahramani, Z. (2016) Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference. In 4th International Conference on Learning Representations (ICLR) workshop track.
GoSS14
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) Explaining and Harnessing Adversarial Examples. ArXiv:1412.6572 [Cs, Stat].
Grav13
Graves, A. (2013) Generating Sequences With Recurrent Neural Networks. ArXiv:1308.0850 [Cs].
GDGR15
Gregor, K., Danihelka, I., Graves, A., Rezende, D. J., & Wierstra, D. (2015) DRAW: A Recurrent Neural Network For Image Generation. ArXiv:1502.04623 [Cs].
HeWH16
He, K., Wang, Y., & Hopcroft, J. (2016) A Powerful Generative Model Using Random Weights for the Deep Image Representation. ArXiv:1606.04801 [Cs].
Hint07
Hinton, G. E.(2007) Learning multiple layers of representation. Trends in Cognitive Sciences, 11(10), 428–434. DOI.
HiOT06
Hinton, G., Osindero, S., & Teh, Y. (2006) A Fast Learning Algorithm for Deep Belief Nets. Neural Computation, 18(7), 1527–1554. DOI.
JeBV16
Jetchev, N., Bergmann, U., & Vollgraf, R. (2016) Texture Synthesis with Spatial Generative Adversarial Networks. In Advances in Neural Information Processing Systems 29.
KaFe14
Karpathy, A., & Fei-Fei, L. (2014) Deep Visual-Semantic Alignments for Generating Image Descriptions. ArXiv:1412.2306 [Cs].
KrSS16
Krishnan, R. G., Shalit, U., & Sontag, D. (2016) Structured Inference Networks for Nonlinear State Space Models. ArXiv:1609.09869 [Cs, Stat].
KWKT15
Kulkarni, T. D., Whitney, W., Kohli, P., & Tenenbaum, J. B.(2015) Deep Convolutional Inverse Graphics Network. ArXiv:1503.03167 [Cs].
LGRN00
Lee, H., Grosse, R., Ranganath, R., & Ng, A. Y.(n.d.) Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations. . Presented at the Proceedings of the 26th International Confer- ence on Machine Learning, 2009
LCKF10
Leskovec, J., Chakrabarti, D., Kleinberg, J., Faloutsos, C., & Ghahramani, Z. (2010) Kronecker Graphs: An Approach to Modeling Networks. J. Mach. Learn. Res., 11, 985–1042.
LoWe16
Louizos, C., & Welling, M. (2016) Structured and Efficient Variational Deep Learning with Matrix Gaussian Posteriors. ArXiv Preprint ArXiv:1603.04733.
MnGr14
Mnih, A., & Gregor, K. (2014) Neural Variational Inference and Learning in Belief Networks. In Proceedings of The 31st International Conference on Machine Learning.
MoDH12
Mohamed, A. r, Dahl, G. E., & Hinton, G. (2012) Acoustic Modeling Using Deep Belief Networks. IEEE Transactions on Audio, Speech, and Language Processing, 20(1), 14–22. DOI.
PaVe14
Paul, A., & Venkatasubramanian, S. (2014) Why does Deep Learning work? - A perspective from Group Theory. ArXiv:1412.6621 [Cs, Stat].
PASA16
Poole, B., Alemi, A. A., Sohl-Dickstein, J., & Angelova, A. (2016) Improved generator objectives for GANs. In Advances in Neural Information Processing Systems 29.
RaMC15
Radford, A., Metz, L., & Chintala, S. (2015) Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. ArXiv:1511.06434 [Cs].
ReMW14
Rezende, D. J., Mohamed, S., & Wierstra, D. (2014) Stochastic Backpropagation and Approximate Inference in Deep Generative Models. ArXiv:1401.4082 [Cs, Stat].
Sala15
Salakhutdinov, R. (2015) Learning Deep Generative Models. Annual Review of Statistics and Its Application, 2(1), 361–385. DOI.
SLZC16
Sun, Z., Liu, J., Zhang, Z., Chen, J., Huo, Z., Lee, C. H., & Zhang, X. (2016) Composing Music with Grammar Argumented Neural Networks and Note-Level Encoding. ArXiv:1611.05416 [Cs].
ThBe15
Theis, L., & Bethge, M. (2015) Generative Image Modeling Using Spatial LSTMs. ArXiv:1506.03478 [Cs, Stat].
THSB17
Tran, D., Hoffman, M. D., Saurous, R. A., Brevdo, E., Murphy, K., & Blei, D. M.(2017) Deep Probabilistic Programming. ArXiv:1701.03757 [Cs, Stat].
OoKK16
van den Oord, A., Kalchbrenner, N., & Kavukcuoglu, K. (2016) Pixel Recurrent Neural Networks. ArXiv:1601.06759 [Cs].
ZKSE16
Zhu, J.-Y., Krähenbühl, P., Shechtman, E., & Efros, A. A.(2016) Generative Visual Manipulation on the Natural Image Manifold. ArXiv:1609.03552 [Cs].