Among various sensing functions of human, vision is the primary way to collect information. The human visual system enables the recognition of various objects and visual information sensing in a complicated environment, which inspires the development of biomimicry visual system through electronic devices for future machine vision. Today’s artificial visual system mainly relies on the existing digital technologies, which consists of photoreceptors to perceive the visual inputs as digital images, a memory unit to store visual information, and a processing unit to conduct complex image processing tasks. This design presents complicated circuitry, generates a lot of redundant data through digital image sensors, occupies large amount storage space, and causes high power consumption.
In a sharp contrast, the sensory neurons in retina of human visual system can directly respond to optical stimuli, and perform the first stage image processing. The pre-processed information is then passed through optic nerve to the visual cortex of the human brain for more complicated visual signal processing, exhibiting much higher efficiency than that of existing artificial visual system. Therefore, it motivates us to mimic human vision sensor. We design an optoelectronic resistive random-access memory (ORRAM), which enables to sense optical signals in a neuromorphic way, and integrates sensing, memory and processing functions for a more efficient artificial visual system.
The two-terminal Pd/MoOx/ITO ORRAM synaptic device exhibits ultraviolet (UV) light sensing, optical triggered non-volatile and volatile resistance switching, and light tuneable synaptic behaviours (Fig.1). Unlike conventional image sensors that compute a linear response of light intensity (image brightness) at each pixel, ORRAM synaptic devices sense information in ways like neural signals with light-tunable and time-dependent plasticity.
Fig. 1 (a) Schematic of the ORRAM device. (b) Non-volatile optical SET and electrical RESET. (c) Comparison of conventional image sensor and ORRAM. (d) Experimental demonstration of light-tuneable plasticity in ORRAM.
The spike current and retention time of ORRAM devices increase faster under higher light intensity, according the characteristics of light-tunable plasticity. Correspondingly, the pixel with higher brightness shows more enhanced accumulation effect, enabling the functions of first-stage image processing, such as, contrast enhancement and noise reduction. We constructed an artificial visual system consisting of pre-processing based on ORRAM arrays and image recognition based on artificial neural network (Fig. 2a). Fig. 2b shows experimental results on the image contrast enhancement through a 3×5 ORRAM array, and Fig. 2c demonstrates background noise reduction through simulation. After the image pre-processing conducted through ORRAMs, the pre-processed image database is inputted into the artificial network to conduct image training and recognition. The image recognition efficiency (processing speed and energy consumption) is proved to be largely improved with the image pre-processing at the front-end (Fig. 2d). The proof-of-concept device provides the potential to simplify the circuitry of neuromorphic visual system and contribute to the development of applications in edge computing.
Fig. 2 (a) Schematic of an artificial visual system with ORRAM for neuromorphic pre-processing and artificial neural network for image recognition. (b) Experimental demonstration of image contrast enhancement. (c) Demonstration of background noise reduction by simulation. (d) Simulation results of image recognition with and without ORRAM for pre-processing.