What’s so special about speech anyway?
Sam Kriss calls the spamularity the language of god. See also Feral, Thomas Urquhart, natural language processing.
“They’re using phrase-structure grammar, long-distance dependencies. FLN recursion, at least four levels deep and I see no reason why it won’t go deeper with continued contact. […] It doesn’t have a clue what I’m saying.”
“It doesn’t even have a clue what it’s saying back,” she added.
Peter Watts, Blindsight
Dan Stowell summarises a neural basis for recursive syntax:
For decades, Noam Chomsky and colleagues have famously been developing and advocating a “minimalist” (BTCB14) idea about the machinery our brain uses to process language. […] They propose that not much machinery is needed, and one of the key components is a “merge” operation that the brain uses in composing and decomposing grammatical structures.
Then yesterday I was reading this introduction to embeddings in artificial neural network and NLP, and I read the following:
“Models like [this] are powerful, but they have an unfortunate limitation: they can only have a fixed number of inputs. We can overcome this by adding an association module, A, which will take two word or phrase representations and merge them.” (Bott11)
- JiKo11: Dezhe Z Jin, Alexay A Kozhevnikov (2011) A compact statistical model of the song syntax in Bengalese finch. PLoS Comput Biol, 7(3), 1001108. DOI
- Grei69: Sheila A. Greibach (1969) An Infinite Hierarchy of Context-Free Languages. J. ACM, 16(1), 91–106. DOI
- AuBB97: Jean-Michel Autebert, Jean Berstel, Luc Boasson (1997) Context-free languages and pushdown automata. In Handbook of formal languages, vol. 1 (pp. 111–174). New York, NY, USA: Springer-Verlag New York, Inc.
- SmKi08: Kenny Smith, Simon Kirby (2008) Cultural evolution: implications for understanding the human language faculty and its evolution. Philosophical Transactions of the Royal Society B: Biological Sciences, 363, 3591–3603. DOI
- Elma91: Jeffrey L Elman (1991) Distributed representations, simple recurrent networks, and grammatical structure. Machine Learning, 7, 195–225. DOI
- Shie87: Stuart M. Shieber (1987) Evidence Against the Context-Freeness of Natural Language. In The Formal Complexity of Natural Language (pp. 320–334). Springer Netherlands
- Bott11: Leon Bottou (2011) From Machine Learning to Machine Reasoning. ArXiv:1102.1808 [Cs].
- Jin09: Dezhe Z Jin (2009) Generating variable birdsong syllable sequences with branching chain networks in avian premotor nucleus HVC. Physical Review E, 80(5), 051902. DOI
- BTCB14: Johan J. Bolhuis, Ian Tattersall, Noam Chomsky, Robert C. Berwick (2014) How Could Language Have Evolved? PLoS Biol, 12(8), e1001934. DOI
- KoMT97: Takeshi Koshiba, Erkki Mäkinen, Yuji Takada (1997) Inferring Pure Context-Free Languages from Positive Data. ACTA CYBERNETICA, 14, 469–477.
- Elma95: Jeffrey L Elman (1995) Language as a dynamical system. , 195.
- ChCh08: Morten H Christiansen, Nick Chater (2008) Language as shaped by the brain. Behavioral and Brain Sciences, 31, 489–509. DOI
- Kirb03: Simon Kirby (2003) Language Evolution. Oxford University Press, USA
- Elma93: Jeffrey L Elman (1993) Learning and development in neural networks: the importance of starting small. Cognition, 48, 71–99. DOI
- Kirb98: Simon Kirby (1998) Learning, Bottlenecks and the Evolution of Recursive Syntax.
- CaSo03: Ramon Ferrer i Cancho, Ricard V. Solé (2003) Least effort and the origins of scaling in human language. Proceedings of the National Academy of Sciences, 100(3), 788–791. DOI
- MBNP10: James L Mcclelland, Matthew M Botvinick, David C Noelle, David C Plaut, Timothy T Rogers, Mark S Seidenberg, Linda B Smith (2010) Letting structure emerge: connectionist and dynamical systems approaches to cognition. Trends in Cognitive Sciences, 14(8), 348–356. DOI
- PuGa82: Geoffrey K Pullum, Gerald Gazdar (1982) Natural Languages and Context-Free Languages. Linguistics and Philosophy, 4(4), 471–504.
- Mann02: Christopher D Manning (2002) Probabilistic syntax. In Probabilistic linguistics (pp. 289–341). Cambridge, MA: MIT Press
- EBJK97: Jeffrey L Elman, Elizabeth A Bates, Mark H Johnson, Annette Karmiloff-Smith, Domenico Parisi, Kim Plunkett (1997) Rethinking Innateness: A Connectionist Perspective on Development (Neural Networks and Connectionist Modeling) . The MIT Press
- BOBB11: Robert C. Berwick, Kazuo Okanoya, Gabriel J.L. Beckers, Johan J. Bolhuis (2011) Songs to syntax: the linguistics of birdsong. Trends in Cognitive Sciences, 15(3), 113–121. DOI
- Wolf00: J Gerard Wolff (2000) Syntax, parsing and production of natural language in a framework of information compression by multiple alignment, unification and search. Journal of Universal Computer Science, 6(8), 781–829.
- John59: John W Backus (1959) The syntax and semantics of the proposed international algebraic language of the Zurich ACM-GAMM conference. In Proceedings of the International Conference on Information Processing. Zürich: UNESCO
- Grei66: Sheila A. Greibach (1966) The Unsolvability of the Recognition of Linear Context-Free Languages. J. ACM, 13(4), 582–587. DOI
- BeBo90: Jean Berstel, Luc Boasson (1990) Transductions and context-free languages. In Handbook of Theoretical Computer Science, Vol. A: Algorithms and Complexity (pp. 1–278).
- PeFH12: Karl-Magnus Petersson, Vasiliki Folia, Peter Hagoort (2012) What artificial grammar learning reveals about the neurobiology of syntax. Brain and Language, 120(2), 83–95. DOI