In recent years, due to advances in the immense processing capacity and parallelism of modern graphics processing units (GPUs), deep learning based on convolutional neural networks (CNNs) has developed rapidly, leading to effective solutions for a variety of artificial intelligence problems. apps. However, the massive amounts of data involved in vision processing limit the application of CNNs to those portable, power-efficient, and computationally-efficient hardware for on-site data processing.
Several studies have been conducted in the field of optical computing to address the challenges of electrical neural networks. Optical computing has many attractive advantages, such as optical parallelism, which can significantly improve computing speed, and optical passivity can reduce power costs and minimize latency. Optical neural networks (ONNs) offer a way to increase computational speed and overcome the bandwidth bottlenecks of electrical units. However, ONNs require a coherent laser as the light source for computation and can hardly be combined with a mature machine vision system in natural light scenes. Thus, hybrid opto-electronic neural networks, in which the front-end is optical and the back-end electrical, have been proposed. These lens-based systems increase the difficulty of use in peripheral devices, such as autonomous vehicles.
In a new article published in Light sciences and applications, a team of researchers, led by Professor Hongwei Chen from Beijing National Information Science and Technology Research Center (BNRist), Department of Electronic Engineering, Tsinghua University, China, has developed an opto neural network architecture -lensless electronics (LOEN) for computer vision tasks that use a passive mask inserted into the imaging light path to perform convolution operations in the optical domain and have overcome the challenge of processing incoherent light signals and broadband in natural scenes. In addition, the optical link, image signal processing and back-end network are smoothly combined to achieve joint optimization for specific tasks to reduce computational effort and power consumption in the whole pipeline.
Compared to the hardware architecture in conventional industrial vision, an optical mask closing on the imaging sensor is proposed to replace the lenses in this article. According to the theory of geometric optics that light travels in a straight line, scenes can be viewed as sets of point light sources, and the optical signal is spatially modulated by the mask to achieve the shift convolution operation and overlay on the image sensor. It has been verified that optical masks can replace convolutional layers of neural networks for feature extraction in the optical domain.
For object classification tasks such as handwritten digit recognition, a lightweight real-time recognition network is built to verify the performance of optical convolution in the architecture. When using a single convolution kernel, the recognition accuracy can reach 93.47%. When the multi-channel convolution operation is implemented by arranging multiple nuclei in parallel on the mask, the classification accuracy can be improved to 97.21%. Compared with traditional machine vision links, it can save about 50% power consumption.
Moreover, by enlarging the dimension of the optical mask, the image is convolved in the optical domain and the sensor captures a crenellated image which is unrecognizable to the human eye, which can naturally encrypt private information without computational consumption. The performance of optical encryption was verified on the face recognition task. Compared with the random MLS pattern, the mask recognition accuracy jointly optimized by an end-to-end network has been improved by more than 6%. Along with privacy protection encryption, it basically achieved the same recognition accuracy performance as non-encryption methods.
This work proposes an extremely simplified system for machine vision tasks, which not only realizes optoelectronic neural network computation in natural scenes, but also opens the whole optoelectronic link to complete joint optimization to achieve the best results for a specific vision task. Combined with the nonlinear materials, the natural light neural network will be realized. The new architecture will have many potential applications in many real-world scenarios, such as autonomous driving, smart homes, and smart security.
Light sciences and applications
Warning: AAAS and EurekAlert! are not responsible for the accuracy of press releases posted on EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.