The Living Thing / Notebooks :

Differentiable learning of automata

Learning stack machines, random access machines, nested hierarchical parsing machines, Turing machines and whatever other automata-with-memory that you wish, from data. In other words, teaching computers to program themselves, via a deep learning formalism.

The border between these and recurrent neural networks is porous.

Google branded: Differentiable neural computers

Christopher Olah’s Characteristically pedagogic intro

Adrian Colyer’s introduction to neural Turing machines.

Andrej Karpathy’s memory machine list has some good starting point.s

Refs

Bott11
Bottou, L. (2011) From Machine Learning to Machine Reasoning. arXiv:1102.1808 [Cs].
ElST16
Ellis, K., Solar-Lezama, A., & Tenenbaum, J. (2016) Sampling for Bayesian Program Learning. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, & R. Garnett (Eds.), Advances in Neural Information Processing Systems 29 (pp. 1289–1297). Curran Associates, Inc.
GrWD14
Graves, A., Wayne, G., & Danihelka, I. (2014) Neural Turing Machines. arXiv:1410.5401 [Cs].
GWRH16
Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwińska, A., … Hassabis, D. (2016) Hybrid computing using a neural network with dynamic external memory. Nature, advance online publication. DOI.
GHSB15
Grefenstette, E., Hermann, K. M., Suleyman, M., & Blunsom, P. (2015) Learning to Transduce with Unbounded Memory. arXiv:1506.02516 [Cs].
GCCB16
Gulcehre, C., Chandar, S., Cho, K., & Bengio, Y. (2016) Dynamic Neural Turing Machine with Soft and Hard Addressing Schemes. arXiv:1607.00036 [Cs].
KaSu15
Kaiser, Ł., & Sutskever, I. (2015) Neural GPUs Learn Algorithms. arXiv:1511.08228 [Cs].
PeLi16
Perez, J., & Liu, F. (2016) Gated End-to-End Memory Networks. arXiv:1610.04211 [Cs, Stat].
RHDH16
Rae, J., Hunt, J. J., Danihelka, I., Harley, T., Senior, A. W., Wayne, G., … Lillicrap, T. (2016) Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, & R. Garnett (Eds.), Advances in Neural Information Processing Systems 29 (pp. 3621–3629). Curran Associates, Inc.
WeCB14
Weston, J., Chopra, S., & Bordes, A. (2014) Memory Networks. arXiv:1410.3916 [Cs, Stat].