Non-line-of-sight reconstruction with signal-object collaborative regularization
Non-line-of-sight (NLOS) imaging aims at recovering obscured objects from multiple scattered light. Our recent work provides a signal-object collaborative regularization (SOCR) framework for high-quality and noise-robust NLOS reconstructions.
Non-line-of-sight technology enables us to detect objects out of the direct line of sight. In real applications, lasers or other light sources are used to illuminate a visible wall, the scattered light from which reaches the hidden object and is scattered back again. The photons collected by detectors are used to recover the location, shape, albedo, and normal of the target. Potential applications of this technology include autonomous driving, survivor rescuing, and remote sensing.
The measured data is inevitably corrupted with ambient noise. The signal-noise ratio is usually low due to multiple reflections of the photons before they are detected. Reconstructions obtained with existing methods are usually blurry with heavy background noise, especially in the case of short exposure time. A broad class of approaches introduces regularization of the hidden object to enhance the reconstruction quality. Commonly used priors include L1, L2 and total variation. In practice, these priors are not enough to obtain clear reconstructions due to heavy measurement noise and the complex nature of real-world objects.
In our recent work, we propose a signal-object collaborative regularization (SOCR) framework for NLOS reconstructions. In this framework, signal and reconstructed target are optimized simultaneously. The proposed collaborative regularization incorporates sparseness and non-local self-similarity of the hidden object as well as smoothness of the signal. The novelty of our work includes
- The albedo and surface normal can be reconstructed under both confocal and non-confocal settings.
- An estimation of the oracle signal corresponding to the reconstructed target is obtained, which gives insights to noise estimation of the raw measurements.
- Two dictionaries are used to capture the local spatial structures and non-local correlations of the hidden targets, which can be beneficial for further tasks, such as recognition and classification.
- The proposed framework is robust in practical applications. Even in cases where the measurement noise is high, our algorithm still reconstructs the hidden objects faithfully with clear local structures and sharp boundaries.
The designed collaborative regularization is a combination of three priors of the signal and the object. It is necessary to introduce them all for high-quality reconstructions of the unseen target. The figure below shows reconstruction results of the instance of the dragon from the Stanford dataset in extremely low exposure time with different combinations of the priors.
The proposed SOCR reconstruction algorithm is solved using the alternative iteration method with two stages. In the initializing stage, a basic reconstruction is obtained by solving the least-squares problem. A sparse reconstruction is then obtained by solving a L1-regularized problem. This reconstruction is used to initialize the dictionaries. In the second stage, the estimated signal, reconstructed target, and dictionaries are updated iteratively to obtain the final reconstruction.
In summary, we have proposed a signal-object collaborative regularization (SOCR) framework for high-quality reconstruction of the albedo and surface normal of the unseen target. The method is robust to noise and applicable in both confocal and non-confocal settings. It can be used as a plug-in module in different physical models.
More details can be found in our article published in Light: Science & Applications (https://www.nature.com/articles/s41377-021-00633-3).