The Living Thing / Notebooks :

Syntax

What’s so special about speech anyway?

Sam Kriss calls the spamularity the language of god. See also Feral, Thomas Urquhart.

“They’re using phrase-structure grammar, long-distance dependencies. FLN recursion, at least four levels deep and I see no reason why it won’t go deeper with continued contact. […] It doesn’t have a clue what I’m saying.”

“What?”

“It doesn’t even have a clue what it’s saying back,” she added.

—Peter Watts

Dan Stowell summarises a neural basis for recursive syntax:

For decades, Noam Chomsky and colleagues have famously been developing and advocating a “minimalist” (BTCB14) idea about the machinery our brain uses to process language. […] They propose that not much machinery is needed, and one of the key components is a “merge” operation that the brain uses in composing and decomposing grammatical structures.

Then yesterday I was reading this introduction to embeddings in artificial neural network and NLP, and I read the following:

“Models like [this] are powerful, but they have an unfortunate limitation: they can only have a fixed number of inputs. We can overcome this by adding an association module, A, which will take two word or phrase representations and merge them.” (Bott11)

Refs

ArSO94
Arimura, H., Shinohara, T., & Otsuki, S. (1994) Finding Minimal Generalizations for Unions of Pattern Languages and Its Application to Inductive Inference from Positive Data. In In Proc. the 11th STACS, LNCS 775 (pp. 649–660). Springer-Verlag
AuBB97
Autebert, J.-M., Berstel, J., & Boasson, L. (1997) Context-free languages and pushdown automata. In G. Rozenberg & A. Salomaa (Eds.), Handbook of formal languages, vol. 1 (pp. 111–174). New York, NY, USA: Springer-Verlag New York, Inc.
BeBo90
Berstel, J., & Boasson, L. (1990) Transductions and context-free languages. In J. van Leeuwen, A. R. Meyer, M. Nivat, M. Paterson, & D. Perrin (Eds.), Handbook of Theoretical Computer Science, Vol. A: Algorithms and Complexity (pp. 1–278).
BOBB11
Berwick, R. C., Okanoya, K., Beckers, G. J. L., & Bolhuis, J. J.(2011) Songs to syntax: the linguistics of birdsong. Trends in Cognitive Sciences, 15(3), 113–121. DOI.
BTCB14
Bolhuis, J. J., Tattersall, I., Chomsky, N., & Berwick, R. C.(2014) How Could Language Have Evolved?. PLoS Biol, 12(8), e1001934. DOI.
Bott11
Bottou, L. (2011) From Machine Learning to Machine Reasoning. arXiv:1102.1808 [cs].
BrKe01
Bressler, S. L., & Kelso, J. A. S.(2001) Cortical coordination dynamics and cognition. Trends in Cognitive Sciences, 5(1), 26–36. DOI.
CaSo03
Cancho, R. F. i, & Solé, R. V.(2003) Least effort and the origins of scaling in human language. Proceedings of the National Academy of Sciences, 100(3), 788–791. DOI.
Chom69
Chomsky, N. (1969) Aspects of the Theory of Syntax. . The MIT Press
ChCh08
Christiansen, M. H., & Chater, N. (2008) Language as shaped by the brain. Behavioral and Brain Sciences, 31, 489–509. DOI.
CDEC02
Christiansen, M. H., Dale, R. A. C., Ellefson, M. R., & Conway, C. M.(2002) The role of sequential learning in language evolution: computational and experimental studies. In A. Cangelosi & D. Parisi (Eds.), Simulating the Evolution of Language. Springer-Verlag New York, Inc.
Deac10
Deacon, T. W.(2010) A role for relaxed selection in the evolution of the language capacity. Proceedings of the National Academy of Sciences, 107, 9000–9006. DOI.
Domi05
Dominey, P. F.(2005) From Sensorimotor Sequence to Grammatical Construction: Evidence from Simulation and Neurophysiology. Adaptive Behavior, 13, 347–361. DOI.
Elma91
Elman, J. L.(1991) Distributed representations, simple recurrent networks, and grammatical structure. Machine Learning, 7, 195–225. DOI.
Elma93
Elman, J. L.(1993) Learning and development in neural networks: the importance of starting small. Cognition, 48, 71–99. DOI.
Elma95
Elman, J. L.(1995) Language as a dynamical system. , 195.
Elma03
Elman, J. L.(2003) Generalization from Sparse Input. In Proceedings of the 38th Annual Meeting of the Chicago Linguistic Society. Citeseer
EBJK97
Elman, J. L., Bates, E. A., Johnson, M. H., Karmiloff-Smith, A., Parisi, D., & Plunkett, K. (1997) Rethinking Innateness: A Connectionist Perspective on Development (Neural Networks and Connectionist Modeling). . The MIT Press
Gibs98
Gibson, E. (1998) Linguistic complexity: Locality of syntactic dependencies. Cognition, 68(1), 1–76.
Grei66
Greibach, S. A.(1966) The Unsolvability of the Recognition of Linear Context-Free Languages. J. ACM, 13(4), 582–587. DOI.
Grei69
Greibach, S. A.(1969) An Infinite Hierarchy of Context-Free Languages. J. ACM, 16(1), 91–106. DOI.
Jin09
Jin, D. Z.(2009) Generating variable birdsong syllable sequences with branching chain networks in avian premotor nucleus HVC. Physical Review E, 80(5), 051902. DOI.
JiKo11
Jin, D. Z., & Kozhevnikov, A. A.(2011) A compact statistical model of the song syntax in Bengalese finch. PLoS Comput Biol, 7(3), -1001108. DOI.
John59
John W Backus. (1959) The syntax and semantics of the proposed international algebraic language of the zurich acmgamm conference. In Proceedings of the International Conference on Information Processing. Zürich: UNESCO
KaPe09
Katz, J., & Pesetsky, D. (2009) The recursive syntax and prosody of tonal music. Ms., Massachusetts Institute of Technology.
Kirb98
Kirby, S. (1998) Learning, Bottlenecks and the Evolution of Recursive Syntax. . Presented at the In E. Briscoe (Ed.), Linguistic
Kirb03
Kirby, S. (2003) Language Evolution. . Oxford University Press, USA
KoMT97
Koshiba, T., Mäkinen, E., & Takada, Y. (1997) Inferring Pure Context-Free Languages from Positive Data. ACTA CYBERNETICA, 14, 469–477.
Kuic70
Kuich, W. (1970) On the entropy of context-free languages. Information and Control, 16(2), 173–200. DOI.
Mann02
Manning, C. D.(2002) Probabilistic syntax. In Probabilistic linguistics (pp. 289–341). Cambridge, MA: MIT Press
MBNP10
Mcclelland, J. L., Botvinick, M. M., Noelle, D. C., Plaut, D. C., Rogers, T. T., Seidenberg, M. S., & Smith, L. B.(2010) Letting structure emerge: connectionist and dynamical systems approaches to cognition. Trends Cogn Sci, 14(8), 348–356. DOI.
Pere00
Pereira, F. (2000) Formal grammar and information theory: together again?. Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences, 358(1769), 1239–1253. DOI.
PeFH12
Petersson, K.-M., Folia, V., & Hagoort, P. (2012) What artificial grammar learning reveals about the neurobiology of syntax. Brain and Language, 120(2), 83–95. DOI.
PuGa82
Pullum, G. K., & Gazdar, G. (1982) Natural Languages and Context-Free Languages. Linguistics and Philosophy, 4(4), 471–504.
Shie87
Shieber, S. M.(1987) Evidence Against the Context-Freeness of Natural Language. In W. J. Savitch, E. Bach, W. Marsh, & G. Safran-Naveh (Eds.), The Formal Complexity of Natural Language (pp. 320–334). Springer Netherlands
Smit03
Smith, K. (2003) The Transmission of Language: models of biological and cultural evolution.
SmKi08
Smith, K., & Kirby, S. (2008) Cultural evolution: implications for understanding the human language faculty and its evolution. Philosophical Transactions of the Royal Society B: Biological Sciences, 363, 3591–3603. DOI.
Wolf00
Wolff, J. G.(2000) Syntax, parsing and production of natural language in a framework of information compression by multiple alignment, unification and search. Journal of Universal Computer Science, 6(8), 781–829.