Biological NN vs. common computer models for machine learning
From Goodfellow, Bengio, Courville (2016) - Deep Learning:
on which common computer models learn: 10^4 - 10^9 (e.g. images). // Humans: when we are awake: full-time video, audio, tactile, smell, etc.
Connections per neuron
Computer model: 100 - 10^3 (there are outliers, but not commonly used) // Human: >10^4
Computer model: 10^6 - 10^7, also, neuron models are simplified a lot. // Human: 10^10
There is a lot we cannot efficiently model in silicon and more we don't know about individual neurons and their local interaction. One problem: we want to be able to transfer learning (topology, weights) between computers, so we have to use digital circuits.
Size is limited by computation speed, and well-parallelizable. I hope that Intel releases Stratix 10 MX soon, and not only to national interest buddies in the US. I think the work that can be done by current neural networks is impressive.