Learning stack machines, random access machines, nested hierarchical parsing machines, Turing machines and whatever other automata-with-memory that you wish, from data. In other words, teaching computers to program themselves, via a deep learning formalism.
The border between these and recurrent neural networks is porous.
Google branded: Differentiable neural computers
Christopher Olah’s Characteristically pedagogic intro
Adrian Colyer’s introduction to neural Turing machines.
Andrej Karpathy’s memory machine list has some good starting point.
- WFCH17: (2017) An inner-loop free solution to inverse problems using deep neural networks. ArXiv:1709.01841 [Cs].
- LHHN17: (2017) Deep Learning with Dynamic Computation Graphs. In Proceedings of ICLR.
- GCCB16: (2016) Dynamic Neural Turing Machine with Soft and Hard Addressing Schemes. ArXiv:1607.00036 [Cs].
- Bott11: (2011) From Machine Learning to Machine Reasoning. ArXiv:1102.1808 [Cs].
- PeLi16: (2016) Gated End-to-End Memory Networks. ArXiv:1610.04211 [Cs, Stat].
- GWRH16: (2016) Hybrid computing using a neural network with dynamic external memory. Nature, advance online publication. DOI
- GHSB15: (2015) Learning to Transduce with Unbounded Memory. ArXiv:1506.02516 [Cs].
- WeCB14: (2014) Memory Networks. ArXiv:1410.3916 [Cs, Stat].
- KaSu15: (2015) Neural GPUs Learn Algorithms. ArXiv:1511.08228 [Cs].
- GrWD14: (2014) Neural Turing Machines. ArXiv:1410.5401 [Cs].
- PuWe17: (2017) Recurrent Inference Machines for Solving Inverse Problems. ArXiv:1706.04008 [Cs].
- ElST16: (2016) Sampling for Bayesian Program Learning. In Advances in Neural Information Processing Systems 29 (pp. 1289–1297). Curran Associates, Inc.
- RHDH16: (2016) Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes. In Advances in Neural Information Processing Systems 29 (pp. 3621–3629). Curran Associates, Inc.