How Complex Is a Real Neuron?

If you are familiar with deep neural networks and how they work, and a little bit of history behind it, you have heard how deep neural networks …

March 9, 2022 5 minute
How Complex Is a Real Neuron?,Artificial Neural Network (ANN)

Description :

If you are familiar with deep neural networks and how they work, and a little bit of history behind it, you have heard how deep neural networks are inspired by real brain neurons and the reason behind the name of these networks.

But recently a study has shown that this traditional way of complexity comparison between a single real brain neuron and an artificial one is not correct.

A Beginner's Guide to Neural Networks and Deep Learning | Pathmind

Scientists have attempted to mimic a real neuron’s function, using a deep neural network.  And the result was surprising even to scientists conducting the research. It needed between 5 to 8 hidden layers each consisting of roughly 256 artificial neurons, just to simulate a single brain neuron’s function.

The scientists also cautioned that the relation between a network's complexity and the number of layers and neurons of a network is not clear yet. In other words, it is not clear if a network with 2 layers does a more complex task compared to a single layer one, nor if a 1000-neuron network is a thousand times more complex than a single neuron.

Speech enhancement from fused features based on deep neural network and  gated recurrent unit network | EURASIP Journal on Advances in Signal  Processing | Full Text

 

 

Models for image classification tasks often require up to 50 layers of neurons. If each biological neuron is like a five-layer artificial neural network, then perhaps an image classification network with 50 layers is equivalent to 10 real neurons in a biological network.

The authors have shared their code to encourage other researchers to find a clever solution with fewer layers. But, letting others know how difficult it was to find a deep neural network that could imitate the neuron with 99 percent accuracy, the authors are confident that their result does provide a meaningful comparison for further research.

These researchers hope that their study can change how we think of ANN, and eventually help improve the state-of-the-art models of DNN.