Learning stack machines, random access machines, nested hierarchical parsing machines, Turing machines and whatever other automata-with-memory that you wish, from data. In other words, teaching computers to program themselves, via a deep learning formalism. Obviously Artificial general intelligence would be good at handling these; It’s not the absolute hippest research area right now though, on account of being hard in general, just like we alwasy thought. Some progress has been made.
The border between these and recurrent neural networks is porous.
Google branded: Differentiable neural computers
Christopher Olah’s Characteristically pedagogic intro
Adrian Colyer’s introduction to neural Turing machines.
Andrej Karpathy’s memory machine list has some good starting point.
- WFCH17: (2017) An inner-loop free solution to inverse problems using deep neural networks. ArXiv:1709.01841 [Cs].
- LHHN17: (2017) Deep Learning with Dynamic Computation Graphs. In Proceedings of ICLR.
- GCCB16: (2016) Dynamic Neural Turing Machine with Soft and Hard Addressing Schemes. ArXiv:1607.00036 [Cs].
- Bott11: (2011) From Machine Learning to Machine Reasoning. ArXiv:1102.1808 [Cs].
- PeLi16: (2016) Gated End-to-End Memory Networks. ArXiv:1610.04211 [Cs, Stat].
- GWRH16: (2016) Hybrid computing using a neural network with dynamic external memory. Nature, advance online publication. DOI
- GHSB15: (2015) Learning to Transduce with Unbounded Memory. ArXiv:1506.02516 [Cs].
- WeCB14: (2014) Memory Networks. ArXiv:1410.3916 [Cs, Stat].
- KaSu15: (2015) Neural GPUs Learn Algorithms. ArXiv:1511.08228 [Cs].
- GrWD14: (2014) Neural Turing Machines. ArXiv:1410.5401 [Cs].
- PuWe17: (2017) Recurrent Inference Machines for Solving Inverse Problems. ArXiv:1706.04008 [Cs].
- ElST16: (2016) Sampling for Bayesian Program Learning. In Advances in Neural Information Processing Systems 29 (pp. 1289–1297). Curran Associates, Inc.
- RHDH16: (2016) Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes. In Advances in Neural Information Processing Systems 29 (pp. 3621–3629). Curran Associates, Inc.