Artificial dendrite boosts the neural network performance

A memristor-based neural network with artificial dendrites shows the capability of performing high-accuracy computation tasks with significantly reduced power consumption.

Like Comment
Read the paper

In the biological nerve system, a typical neuron is composed of one soma, many dendrites to receive synaptic inputs, and a single axon to send out outputs. The dendrites, especially those in human brain, have a very complex topological structure and rich dynamics. More and more recent studies in neuroscience reveal that dendrites play an important role in processing spatial-temporal information in biological neural networks [1], [2], which is one of the main reasons why our brain can handle very sophisticated tasks yet consuming extremely low power. However, such important function of dendrite is usually neglected in most state-of-the-art neural networks, where the neuron function is simplified as an “integrate-and-fire” (IF) or “leaky-integrate-and-fire” (LIF) model. Such simplification has limited the performance of artificial neural networks, especially in terms of power and flexibility in handling complex tasks when compared to biological neural networks. Therefore, incorporating dendritic computing units into artificial neural networks is highly desired to boost their performance.

Fig. 1 Implementation of artificial dendrite. (a) An image of a brain neuron of a mouse. (b-c) Schematic of the membrane of a biological dendrite (b) and a metal oxide-based dynamic memristor as artificial dendrite (c). (d-e) Measured nonlinear current response to the applied voltage on the fabricated artificial dendrite implementing the key functions of filtering and nonlinear integration.

For the hardware implementation of artificial dendrites, there have been several attempts to implement the dendritic functions using electronic circuits; however, this approach usually comes with a high hardware cost, e.g., more than 40 silicon transistors are required to just mimic the N-methyl-D-aspartate (NMDA) channel in the dendrite. For future high-density integration, a more scalable and bio-plausible artificial dendrite using a single device, similar to the memristive synapse and neuron in the field of neuromorphic computing, is highly desired.

 In this work, we have developed a two-terminal dynamic memristor, which consists of a Pt/TaOx/AlOδ/Al material stack, to implement the artificial dendrite. Driven by the electrical and chemical potentials inside the memristor, the oxygen ion migration modulates its electrical properties. This mimics the electrochemical dynamic process in biological dendrites, and hence implements the key dendritic functions of filtering and nonlinear spatial-temporal integration.

 To validate the dendritic computing functions, we have constructed a complete neural network by integrating the developed artificial dendrite device with filamentary memristor-based artificial synapse and Mott memristor-based artificial soma. This hardware-implemented neural network is used to recognize digits in the Street View House Numbers (SVHN) dataset, and our results show that the incorporation of artificial dendrite helps largely reduce the soma dynamic power by more than 30 times and in the meanwhile significantly increase the accuracy from 80.1% to 88.5%. Further system evaluation reveals that our memristor-based system has more than three orders of magnitude advantage in power consumption than CPU. As we can see, the neural network equipped with functional dendrites shows the potential of substantial overall performance improvement in terms of power and accuracy by extracting critical information from a noisy background.

Fig. 2 Hardware implementation and performance of a complete neural network. (a) Examples of SVHN images. (b) Comparison of the recognition accuracies for the simplified SVHN dataset with the two neural networks. (c) Comparison of the power consumption of the key components in the two neural networks. (d) Images of the hardware-implemented neural network with artificial dendrites.

We shall point out that the filtering and integration function implemented in this work is just part of many important functions of biological dendrites. Also, there is still plenty of room for future improvement in both device and system performances, such as the operation voltage and speed of the dendrite device. We hope our work could inspire new designs of artificial dendrite and attract more research efforts on dendritic computation, which are crucial to make the memristor-based neuromorphic computing more accurate and more efficient in the future. Co-optimizations in device, algorithm, circuit, and architecture are needed to build more intelligent neural network that can handle more sophisticated tasks.

 This work was recently published in Nature Nanotechnology:

 "Power-efficient Neural Network with Artificial dendrites", Nature Nanotechnology (2020).

 https://www.nature.com/articles/s41565-020-0722-5

Reference:

[1]       P. Poirazi and A. Papoutsi, “Illuminating dendritic function with computational models,” Nat. Rev. Neurosci., 21, 303–321, 2020.

[2]       A. Gidon et al., “Dendritic action potentials and computation in human layer 2/3 cortical neurons,” Science, 367, 83–87, 2020.

Jianshi Tang

Assistant Professor, Tsinghua University

No comments yet.