Fulfilling Brain-inspired Hyperdimensional Computing with In-memory Computing
In this work, we present a scalable methodology to implement hyperdimensional computing paradigm employing in-memory computing techniques on a phase change computational memory array.
The most efficient computer possible already exists. And no, it’s not a Mac or a PC—it’s the human brain. When computers were originally invented, they were designed around a brain model. At one time, many even referred to them as electronic brains. Indeed, it is certainly impressive how today’s computers can emulate single brain functions, such as learning and identifying visual objects or recognizing text and speech patterns. However, the evolution of computers has a long way to go to match the remarkable capacity of the human brain, which can learn and adapt without needing to be programmed or updated, has intricately connected memory, doesn’t easily crash, and works in real time.
Nonetheless, scientists around the world continue to be inspired by the brain and strive to mimic its abilities in the development of technology. Neuroscientists now know that the mind operates by evaluating the state of thousands of synaptic connections at a time and computes with patterns of neural activity that are not readily associated with numbers. Drawing inspiration from this cerebral functionality, our research team at IBM Research Europe in Zurich set out to explore ways to move computing away from the conventional von Neumann digital paradigm that we are used to, and looked to the cerebral attributes of neuronal circuits to create a novel in-memory hyperdimensional computing system.
So, what’s hyperdimensional computing (HDC)? HDC is an emerging computational paradigm that aims to mimic attributes of the human brain’s neuronal circuits such as hyperdimensionality, fully distributed holographic representation, and (pseudo)randomness. What we discovered is that an HDC framework functions exceptionally well within an in-memory computing architecture. In fact, based on the results of our experiments in training and classifying datasets, HDC is a killer application for in-memory computing in many respects.
While our brain operates by evaluating the state of thousands of synaptic connections at a time and computes with seemingly arbitrary patterns of neural activity, HDC models these neural activity patterns of the brain’s circuits in the form of high-dimensional vectors called hypervectors that operate on a rich form of algebra that defines rules to build, bind, and bundle them. Essentially, HDC represents data using bits in the order of thousands and distributes the related information across all bits evenly.
There are many advantages to computing with hypervectors. For one, training algorithms in an HDC architecture is transparent, quicker, and more efficient as object categories are learned in one pass of training from the available data. This beats other brain-inspired approaches such as neural networks, which require a large number of iterations for training. Moreover, this computing paradigm is memory-centric with parallel operations and is extremely robust against noise and variations or faulty components in a computer platform.
Taking all of this into consideration, we build the first-of-its-kind in-memory hyperdimensional computing prototype using 760,000 phase-change memory devices that can achieve over 600% energy savings compared to most optimized digital systems based on CMOS technology, and can perform high-level cognitive tasks such as language and new classification and gesture recognition at comparable accuracy levels.
To date, this is the most scalable hardware solution science has seen mimicking the brain’s neural activity with hyperdimensional computing. And we believe our research, which is now being featured in the peer-reviewed journal Nature Electronics, will play an essential role in the advancement of next-generation AI hardware.