Camera-Based Localization Microscopy Optimized with Calibrated Structured Illumination

When multiple phase-shifted exposures of field-wide structured illumination replace the standard flat illumination in localization microscopy and are analyzed appropriately, one records effectively four times as many photons, which doubles resolution or, in principle, quadruples imaging speed.

Like Comment
Read the paper

Fluorescently labeled molecules and particles are used extensively to probe the structures and dynamics of life on its smallest length-scale.  Wide-field microscopy with uniform illumination allows super-localization of such point-sources with high through-put and with a precision that is bounded only by the number of photons used (Fig. 1a).  A decade ago, we demonstrated how to reach the theoretical limit of precision, the so-called Cramér-Rao bound, in practice with an optimal data analysis (Mortensen et al., Nat. Methods 7, 2010).  We took pleasure in knowing that our analysis has optimal photon economy (Larson, Nat Methods 7, 2010); it is mathematically impossible to supersede it.  “Game over!” it seemed. 

Figure 1. Schematic illustration of C-MELM. a: Conventional analysis determines the position xm of a fluorescent molecule by illuminating the sample with uniform light (left) and finding the centroid using a model fit to the resulting spot (right). The resolution depends on the number of photons I in the spot and the used method. b: Camera-based MINFLUX determines the position of the molecule by the relative number of photons I1/I0 obtained under exposure to phase-shifted (field-wide) structured illuminations. If the illumination is harmonic and appropriately shifted, then the total photon number in the two images combined is also I. c: C-MELM combines the information components in a and b. A joint model fit to spots imaged under structured illuminations determines xm optimally using information from the (common) centroid of the spots and the relative number of photons in the two spots.

Then in 2017 Kim Mortensen was contacted by a science reporter for a comment on the just-published MINFLUX method from the lab of Nobel laureate Stefan Hell (Balzarotti et al., Science 355, 2017) and realized that the game had changed.  MINFLUX does not use uniform illumination but a beam of light with doughnut-shaped intensity profile.  It refrains from imaging but instead just records the number of photons from a point source in several exposures with the doughnut in different positions.  The ratios between these photon counts reveal the position of the point source with improved photon economy.  The gain was a whooping order-of-magnitude.  The drawback of MINFLUX is its low throughput: It is limited to tracking a single particle at a time because the doughnut must go where the particle goes.

We immediately realized that the low throughput of MINFLUX is overcome by using structured illumination in camera-based wide-field microscopy (Fig. 1b).  A point-source then images as a diffraction-limited spot, and the total intensity of a spot gives the photon count central to MINFLUX.  This is a trivial insight, knowing MINFLUX.  Maybe less trivial, MINFLUX uses only the intensity detected from a point-source. This information is independent of the centroid of the spot.  Thus, our localization analysis from 2010 provides information that is independent of the localization information provided by structured illumination analyzed a la MINFLUX – from the same photons! 

Combination of the two methods is trivial, in theory, and achieves high throughput with optimal precision of localization (Fig. 1c).  Two talented graduate students, a theorist and an experimenter, were given the task to bringing this idea to fruition with proven experimental and data-analytical protocols.  They did.

The devil is in the detail: Structured illumination is, ideally, a known analytical function.  As a starting point, one can assume an optimal function (a harmonic function), calculate away, and prove everything works.  The function, however, must be created in the lab, with maximal stability and control.  To this end, we modified the illumination path of an existing wide-field microscope.  With a so-called Digital Micromirror Device – and genuine engineering involving scissors, glue, and cardboard – we created structured illumination at the sample.  

Since the structured illumination may vary across the imaged area, it is calibrated, locally, everywhere across that area, before and after it is used, to know it and that it is stable, in order to obtain the theoretically possible localization precision.  This may sound grueling but does have advantages: The illumination needs not resemble a harmonic function all that well, it can vary – as it will – across the imaged area, and it actually simplifies the data analysis (see our paper).

We call the method C-MELM, short for Calibrated Modulation-Enhanced Localization Microscopy (Fig. 1c).  With it, we have again achieved maximal photon economy in the sense that we in practice again saturate the Cramér-Rao bound for the illumination that we use. Thus, all photons were used optimally, and only the unavoidable shot-noise of these photons was responsible for the localization errors in our results.

C-MELM yields a precision that can be obtained with uniform illumination only by using at least four times as many photons. This translates into twice the resolution or, in principle, four times higher speed of imaging than with uniform illumination.

The resolution, accessibility, and flexibility of our approach makes the future appear bright for structured illumination approaches to nanoscale localization microscopy. On one hand, it will be very interesting to see how our method will be adapted in various labs for applications to biological and/or artificial systems and in conjunction with the various super-resolution microscopy modalities.  On the other hand, a number of intriguing possibilities exist for further development of the concept to other types of microscopy and other sources of illumination.  Certainly, we are looking forward to applying this method and extensions of it in the Novo Nordisk Foundation Challenge Center for Intestinal Absorption and Transport of Biopharmaceuticals that funded this research.

This game is still on.

Read more about our work in Communications Physics (

Kim I. Mortensen

Senior Researcher, Technical University of Denmark