If you had checked my website (https://person.zju.edu.cn/nanoscopy), you might discover that I was mainly focusing on microscopy instead of spectroscopy. Yet, I have been interested in it for a whole. It is my luck to have a colleague around whose research interest involves optical coating. Prof. Weidong Shen’s lab is especially helpful when we need a custom filter or dichroic mirror to upgrade our microscopes. However, even for an expert like him, Weidong cannot always tell what exactly happens inside the vacuum chamber of the coating machine. When the parameters went wrong during the filter fabrication, we might only obtain the filters with random transmissions. These cases, though quite uncommon, are very expensive in terms of time, money, and human resources. Is there any possibility to restore these filters from the trash bin?

This question triggered the whole project. We decided to detect the spectrum using these ‘random’ filters, similar to employing speckle patterns as structured illumination in microscopy for better resolution – a concept that we were quite familiar with. We also abandoned the compressive sensing algorithm from the very beginning, as we had witnessed the power of deep neural network (DNN) since we applied it for de-noising, acquisition acceleration, and other advanced image processing tasks. In a word, although microscopy and spectroscopy sound unrelated, our experiences on microscopy were very beneficial and guided us towards the story we are now talking about in the paper.

As we targeted to achieve better spectral resolution with fewer filters, we were soon aware that the spectral responses of the filters cannot be completely random. Further analysis shows there always lies a contradiction that a mathematically optimal spectral response converges to a white-noise-like one, whereas the optical design tends to create sufficiently smooth spectral curves to simplify the production. To resolve this dilemma, we expanded our DNN to cover the task of the filter design besides the spectrum reconstruction. This strategy eventually allows 7,000–11,000 times faster signal processing and ~10 times improvement regarding noise tolerance. These improvements enabled us to precisely and dynamically reconstruct the spectra of the entire field of view, previously unreachable with compact computational spectral cameras.

Is our DNN framework the best option for spectrum detection? Possibly not, but I believe we are going in the right direction. More importantly, our lab is open to any potential collaboration to pursue high-quality but cost-effective spectral imaging solutions together. If you are interested in our work, please refer to the paper “Deeply learned broadband encoding stochastic hyperspectral imaging” published in Light: Science & Applications following the link: https://www.nature.com/articles/s41377-021-00545-2. Alternatively, you may reach me via my email address: haox@zju.edu.cn.