How do brains work?

Dr. Greg Dunn and Dr. Brian Edwards, *Self Reflected*:

I mean, how do brains work at the level slightly higher than a synapse, but much lower than, e.g. psychology. “How is thought done?” etc.

Notes pertaining to *large, artificial* networks are filed under
artificial neural networks.
The messy, biological end of the stick is here.
Since brains seem to be the seat of the most flashy and important bit of the
computing taking place in our bodies, we
understandably want to know how they works, in order to

- fix Alzheimers disease
- steal cool learning tricks
- endow the children of elites with superhuman mental prowess to cement their places as Übermenschen fit to rule the thousand year Reich
- …or whatever.

Real brains are different to the “neuron-inspired” computation of the simulacrum in very many ways, not just the usual difference between model and reality. The similitude between “neural networks” and neurons is intentionally weak for reasons of convenience.

For one example, most simulated neural networks are based on a continuous activation potential and discrete time, unlike spiking biological ones which are driven by discrete events in continuous time.

Also real brains support heterogeneous types of neuron, have messier layer organisation, use less power, don’t have well-defined backpropagation (or not in the same way), and many other things that I as a non-specialist do not know.

To learn more about:

- Just saw a talk by Dan Cireșan in which he mentioned “Foveation” - blurring the edge of an image when training classifiers on it. The allure of ignoring stuff to learn it. What is, rigorously speaking, happening there? Nice actual crossover with deep learning.
- Algorithmic statistics of neurons sounds interesting.

## Fun data

Allen Institute Brain Observatory.

## To read

- JiKo11: Dezhe Z Jin, Alexay A Kozhevnikov (2011) A compact statistical model of the song syntax in Bengalese finch.
*PLoS Comput Biol*, 7(3), 1001108. DOI - KSSP15a: Anna Kutschireiter, Simone Carlo Surace, Henning Sprekeler, Jean-Pascal Pfister (2015a) A Neural Implementation for Nonlinear Filtering.
*ArXiv Preprint ArXiv:1508.06818*. - KSSP15b: Anna Kutschireiter, Simone C Surace, Henning Sprekeler, Jean-Pascal Pfister (2015b) Approximate nonlinear filtering with a recurrent neural network.
*BMC Neuroscience*, 16(Suppl 1), P196. DOI - SNVV14: Arjen Stolk, Matthijs L. Noordzij, Lennart Verhagen, Inge Volman, Jan-Mathijs Schoffelen, Robert Oostenveld, … Ivan Toni (2014) Cerebral coherence between communicators marks the emergence of meaning.
*Proceedings of the National Academy of Sciences*, 111(51), 18183–18188. DOI - KSOO11: Kentaro Katahira, Kenta Suzuki, Kazuo Okanoya, Masato Okada (2011) Complex sequencing rules of birdsong can be explained by simple hidden Markov processes.
*PLoS ONE*, 6(9), 24516. DOI - KAAB18: Robert E. Kass, Shun-Ichi Amari, Kensuke Arai, Emery N. Brown, Casey O. Diekman, Markus Diesmann, … Mark A. Kramer (2018) Computational Neuroscience: Mathematical and Statistical Perspectives.
*Annual Review of Statistics and Its Application*, 5(1), 183–214. DOI - Bret12: Romain Brette (2012) Computing with Neural Synchrony.
*PLoS Comput Biol*, 8(6), e1002561. DOI - BQFW01: Riccardo Barbieri, Michael C Quirk, Loren M Frank, Matthew A Wilson, Emery N Brown (2001) Construction and analysis of non-Poisson stimulus-response models of neural spiking activity.
*Journal of Neuroscience Methods*, 105(1), 25–37. DOI - Cadi14: C. F. Cadieu (2014) Deep neural networks rival the representation of primate it cortex for core visual object recognition.
*PLoS Comp. Biol.*, 10, e1003963. DOI - HaPB10: Robert Haslinger, Gordon Pipa, Emery Brown (2010) Discrete Time Rescaling Theorem: Determining Goodness of Fit for Discrete Time Statistical Models of Neural Spiking.
*Neural Computation*, 22(10), 2477–2506. DOI - EFBS04: U Eden, L Frank, R Barbieri, V Solo, E Brown (2004) Dynamic Analysis of Neural Encoding by Point Process Adaptive Filtering.
*Neural Computation*, 16(5), 971–998. DOI - SmLe06: Evan C. Smith, Michael S. Lewicki (2006) Efficient auditory coding.
*Nature*, 439(7079), 978–982. DOI - SmLe05: Evan Smith, Michael S. Lewicki (2005) Efficient Coding of Time-Relative Structure Using Spikes.
*Neural Computation*, 17(1), 19–45. DOI - LBRN07: Honglak Lee, Alexis Battle, Rajat Raina, Andrew Y. Ng (2007) Efficient sparse coding algorithms.
*Advances in Neural Information Processing Systems*, 19, 801. - OlFi96: Bruno A. Olshausen, David J. Field (1996) Emergence of simple-cell receptive field properties by learning a sparse code for natural images.
*Nature*, 381(6583), 607–609. DOI - SKRB98: Steven P Strong, Roland Koberle, Rob R de Ruyter van Steveninck, William Bialek (1998) Entropy and Information in Neural Spike Trains.
*Phys. Rev. Lett.*, 80(1), 197–200. DOI - SmBr03: A Smith, E Brown (2003) Estimating a state-space model from point process observations.
*Neural Computation*, 15(5), 965–991. DOI - ASWS04: José M Amigó, Janusz Szczepański, Elek Wajnryb, Maria V Sanchez-Vives (2004) Estimating the Entropy Rate of Spike Trains via Lempel-Ziv Complexity.
*Neural Computation*, 16, 717–736. DOI - Elma90: Jeffrey L Elman (1990) Finding structure in time.
*Cognitive Science*, 14, 179–211. DOI - Jin09: Dezhe Z Jin (2009) Generating variable birdsong syllable sequences with branching chain networks in avian premotor nucleus HVC.
*Physical Review E*, 80(5), 051902. DOI - Bret08: Romain Brette (2008) Generation of Correlated Spike Trains.
*Neural Computation*, 0(0), 080804143617793–28. DOI - SaEb94: J. Sandkühler, A. A. Eblen-Zajjur (1994) Identification and characterization of rhythmic nociceptive and non-nociceptive spinal dorsal horn neurons in the rat.
*Neuroscience*, 61(4), 991–1006. DOI - VoIS15: Maxim Volgushev, Vladimir Ilin, Ian H. Stevenson (2015) Identifying and Tracking Simulated Synaptic Inputs from Neuronal Firing: Insights from In Vitro Experiments.
*PLoS Computational Biology*, 11(3). DOI - Elma93: Jeffrey L Elman (1993) Learning and development in neural networks: the importance of starting small.
*Cognition*, 48, 71–99. DOI - SmLe04: Evan C. Smith, Michael S. Lewicki (2004) Learning efficient auditory codes using spikes predicts cochlear filters. In Advances in Neural Information Processing Systems (pp. 1289–1296).
- SiOl01: Eero P Simoncelli, Bruno A Olshausen (2001) Natural Image Statistics and Neural Representation.
*Annual Review of Neuroscience*, 24(1), 1193–1216. DOI - FeKH04: Michale S Fee, Alexay A Kozhevnikov, Richard H Hahnloser (2004) Neural mechanisms of vocal sequence generation in the songbird.
*Annals of the New York Academy of Sciences*, 1016, 153–170. DOI - FeSo07: Pau Fernández, Ricard V Solé (2007) Neutral fitness landscapes in signalling networks.
*Journal of The Royal Society Interface*, 4(12), 41. DOI - ShBG16: Yanning Shen, Brian Baingana, Georgios B. Giannakis (2016) Nonlinear Structural Vector Autoregressive Models for Inferring Effective Brain Network Connectivity.
*ArXiv:1610.06551 [Stat]*. - Para86: (1986)
*Parallel Distributed Processing: Explorations in the Microstructure of Cognition*. MIT Press - OrRK17: Josue Orellana, Jordan Rodu, Robert E. Kass (2017) Population Vectors Can Provide Near Optimal Integration of Information.
*Neural Computation*, 29(8), 2021–2029. DOI - BOBB11: Robert C. Berwick, Kazuo Okanoya, Gabriel J.L. Beckers, Johan J. Bolhuis (2011) Songs to syntax: the linguistics of birdsong.
*Trends in Cognitive Sciences*, 15(3), 113–121. DOI - OlFi04: Bruno A Olshausen, David J Field (2004) Sparse coding of sensory inputs.
*Current Opinion in Neurobiology*, 14(4), 481–487. DOI - VBZD15: Carlos E. Vargas-Irwin, David M. Brandman, Jonas B. Zimmermann, John P. Donoghue, Michael J. Black (2015) Spike Train SIMilarity Space (SSIMS): A Framework for Single Neuron and Ensemble Data Analysis.
*Neural Computation*, 27(1), 1–31. DOI - SCCT12: Kazutoshi Sasahara, Martin L. Cody, David Cohen, Charles E. Taylor (2012) Structural Design Principles of Complex Bird Songs: A Network-Based Approach.
*PLoS ONE*, 7(9), 44436. DOI - MaMD14: Gary Marcus, Adam Marblestone, Thomas Dean (2014) The atoms of neural computation.
*Science*, 346(6209), 551–552. DOI - ZRBA14: Semir Zeki, John Paul Romaya, Dionigi M. T. Benincasa, Michael F. Atiyah (2014) The experience of mathematical beauty and its neural correlates.
*Frontiers in Human Neuroscience*, 8. DOI - BBVK02: ERVRL Brown, Riccardo Barbieri, Valérie Ventura, R. Kass, L. Frank (2002) The time-rescaling theorem and its application to neural spike train data analysis.
*Neural Computation*, 14(2), 325–346. DOI - BuMe05: Catalin V. Buhusi, Warren H. Meck (2005) What makes us tick? Functional and neural mechanisms of interval timing.
*Nature Reviews Neuroscience*, 6(10), 755–765. DOI