The Living Thing / Notebooks : Neural networks (made of real neurons, in actual brains)

How do brains work?

Brain, slightly stylized.

Dr. Greg Dunn and Dr. Brian Edwards, Self Reflected

Note

I mean, how do brains work at the level slightly higher than a synapse, but much lower than, e.g. psychology. “How is thought done?” etc.

Notes pertaining to large, artificial networks are filed under artificial neural networks. The messy, biological end of the stick is here. Since brains seem to be the seat of the most flashy and important bit of the computing taking place in our bodies, we understandably want to know how they works, in order to

Real brains are different to the “neuron-inspired” computation of the simulacrum in very many ways, not just the usual difference between model and reality. The similitude between “neural networks” and neurons is intentionally weak for reasons of convenience.

For one example, most simulated neural networks are based on a continuous activation potential and discrete time, unlike spiking biological ones which are driven by discrete events in continuous time.

Also real brains support heterogeneous types of neuron, have messier layer organisation, use less power, don’t have well-defined backpropagation (or not in the same way), and many other things that I as a non-specialist do not know.

To learn more about:

To read

ASWS04
Amigó, J. M., Szczepański, J., Wajnryb, E., & Sanchez-Vives, M. V.(2004) Estimating the Entropy Rate of Spike Trains via Lempel-Ziv Complexity. Neural Computation, 16, 717–736. DOI.
BQFW01
Barbieri, R., Quirk, M. C., Frank, L. M., Wilson, M. A., & Brown, E. N.(2001) Construction and analysis of non-Poisson stimulus-response models of neural spiking activity. Journal of Neuroscience Methods, 105(1), 25–37. DOI.
Bret08
Brette, R. (2008) Generation of Correlated Spike Trains. Neural Computation, 0(0), 80804143617793–28. DOI.
Bret12
Brette, R. (2012) Computing with Neural Synchrony. PLoS Comput Biol, 8(6), e1002561. DOI.
BBVK02
Brown, E., Barbieri, R., Ventura, V., Kass, R., & Frank, L. (2002) The time-rescaling theorem and its application to neural spike train data analysis. Neural Computation, 14(2), 325–346. DOI.
BuMe05
Buhusi, C. V., & Meck, W. H.(2005) What makes us tick? Functional and neural mechanisms of interval timing. Nature Reviews Neuroscience, 6(10), 755–765. DOI.
EFBS04
Eden, U., Frank, L., Barbieri, R., Solo, V., & Brown, E. (2004) Dynamic Analysis of Neural Encoding by Point Process Adaptive Filtering. Neural Computation, 16(5), 971–998. DOI.
Elma90
Elman, J. L.(1990) Finding structure in time. Cognitive Science, 14, 179–211. DOI.
Elma93
Elman, J. L.(1993) Learning and development in neural networks: the importance of starting small. Cognition, 48, 71–99. DOI.
FeKH04
Fee, M. S., Kozhevnikov, A. A., & Hahnloser, R. H.(2004) Neural mechanisms of vocal sequence generation in the songbird. Annals of the New York Academy of Sciences, 1016, 153–170. DOI.
FeSo07
Fernández, P., & Solé, R. V.(2007) Neutral fitness landscapes in signalling networks. Journal of The Royal Society Interface, 4(12), 41. DOI.
Gibs98
Gibson, E. (1998) Linguistic complexity: Locality of syntactic dependencies. Cognition, 68(1), 1–76.
HaGe09
Harrison, M., & Geman, S. (2009) A Rate and History-Preserving Resampling Algorithm for Neural Spike Trains. Neural Computation, 21(5), 1244–1258. DOI.
HaAK13
Harrison, M. T., Amarasingham, A., & Kass, R. E.(2013) Statistical Identification of Synchronous Spiking. In P. M. DiLorenzo & J. D. Victor (Eds.), Spike Timing: Mechanisms and Function. CRC Press
HaPB10
Haslinger, R., Pipa, G., & Brown, E. (2010) Discrete Time Rescaling Theorem: Determining Goodness of Fit for Discrete Time Statistical Models of Neural Spiking. Neural Computation, 22(10), 2477–2506. DOI.
HeLV11
Hemmen, J. L. van, Longtin, A., & Vollmayr, A. N.(2011) Testing resonating vector strength: Auditory system, electric fish, and noise. Chaos: An Interdisciplinary Journal of Nonlinear Science, 21(4), 47508. DOI.
Jin09
Jin, D. Z.(2009) Generating variable birdsong syllable sequences with branching chain networks in avian premotor nucleus HVC. Physical Review E, 80(5), 51902. DOI.
KSAC05
Kennel, M. B., Shlens, J., Abarbanel, H. D. I., & Chichilnisky, E. J.(2005) Estimating Entropy Rates with Bayesian Confidence Intervals. Neural Computation, 17(7). DOI.
LBRN07
Lee, H., Battle, A., Raina, R., & Ng, A. Y.(2007) Efficient sparse coding algorithms. Advances in Neural Information Processing Systems, 19, 801.
NeBR04
Nemenman, I., Bialek, W., & de Ruyter Van Steveninck, R. (2004) Entropy and information in neural spike trains: Progress on the sampling problem. Physical Review E, 69(5), 56111.
PSMP07
Panzeri, S., Senatore, R., Montemurro, M. A., & Petersen, R. S.(2007) Correcting for the sampling bias problem in spike train information measures. Journal of Neurophysiology, 98, 1064–1072. DOI.
SeCB13
Seth, A. K., Chorley, P., & Barnett, L. C.(2013) Granger causality analysis of fMRI BOLD signals is invariant to hemodynamic convolution but not downsampling. NeuroImage, 65, 540–555. DOI.
SmBr03
Smith, A., & Brown, E. (2003) Estimating a state-space model from point process observations. Neural Computation, 15(5), 965–991. DOI.
SmLe04
Smith, E. C., & Lewicki, M. S.(2004) Learning efficient auditory codes using spikes predicts cochlear filters. In Advances in Neural Information Processing Systems (pp. 1289–1296).
SmLe06
Smith, E. C., & Lewicki, M. S.(2006) Efficient auditory coding. Nature, 439(7079), 978–982. DOI.
SmLe05
Smith, E., & Lewicki, M. S.(2005) Efficient Coding of Time-Relative Structure Using Spikes. Neural Computation, 17(1), 19–45. DOI.
SKRB98
Strong, S. P., Koberle, R., de Ruyter van Steveninck, R. R., & Bialek, W. (1998) Entropy and Information in Neural Spike Trains. Phys. Rev. Lett., 80(1), 197–200. DOI.
VBZD15
Vargas-Irwin, C. E., Brandman, D. M., Zimmermann, J. B., Donoghue, J. P., & Black, M. J.(2015) Spike Train SIMilarity Space (SSIMS): A Framework for Single Neuron and Ensemble Data Analysis. Neural Computation, 27(1), 1–31. DOI.
Vict02
Victor, J. D.(2002) Binless strategies for estimation of information from neural data. Physical Review E, 66, 51903. DOI.
WiLP15
Wibral, M., Lizier, J. T., & Priesemann, V. (2015) Bits from brains for biologically inspired computing. Computational Intelligence, 2, 5. DOI.