We introduce the Holographic – Single Scatterer Localization Microscopy in which we combine dynamical laser speckle illumination with centroid localization of backscattered light spots in order to localize isolated scattering particles. The reconstructed centroid images show very accurate particle localization, with precision much better than the width of diffraction-limited image of the particles recorded by the CCD. Furthermore, the method provides an improved resolution in distinguishing two very close scattering objects compared to the standard laser scanning techniques and can be assimilated to a confocal technique in the ability of light background rejection in three-dimensional disposition of scattering objects. The illumination is controlled via a digital holography setup based on the use of a spatial light modulator. This allows not only a high level of versatility in the illumination patterns, but also the remarkable characteristics of absence of moving mechanical parts, typical of the laser scanning techniques, and the possibility of strongly miniaturizing the setup.
© 2017 Optical Society of America
Optical microscopy is a fundamental means not only to visualize biological samples and investigate the dynamics at cellular level, but it is also a useful tool to analyze scattering and reflective samples. Much research in optical microscopy is oriented towards the overcoming of the performances of the standard bright-field microscopy and many techniques were proposed over the years in order to respond to requests on contrast, resolution and sectioning ability. For example, Scanning Near-Field Optical Microscopy (SNOM) [1–6] breaks Abbe’s resolution limit probing the optical near field close to the sample. However, the majority of the new proposed methods is intrinsically related to fluorescence microscopy and it is based on the modification of the effective Point-Spread Function (PSF)  of a fluorescence microscope. In the Stimulated Emission Depletion microscopy (STED) , the fluorescence of photo-switchable molecules is restricted to a narrow spatial volume by a spatially modulated excitation light which results into an effective narrower PSF, providing a super-resolved image of the sample in a scanning process. The Structured Illumination Microscopy (SIM)  is able to double the resolution of a conventional fluorescence microscope in three dimensions  shifting the information about high spatial frequencies of the sample, normally lost in the imaging of the far-field, in the frequency passband of the microscope. This is achieved by elaborating a sequence of images obtained from a properly spatially structured illumination of the sample. The resolution of the SIM can be pushed even further if non-linear absorption of the fluorophores is involved . The reconstruction of super-resolution SIM images requires well known and controlled illumination patterns, but new methods using random illumination speckle patterns were successfully realized [12–14], with advantages related to set-up simplification and insensitivity to aberration-induced illumination deformations. Photo-activated Localization Microscopy (PALM) , Stochastic Optical Reconstruction Microscopy (STORM)  and related methods [17,18] surpass Abbe’s resolution limit  by exploiting high precision localization of the center of fluorescence emission from single photo-switchable chromophores, activated randomly and imaged one by one in a time sequence by a pixelated detector (like a CCD) [7,20]. A superresolution method based on spatial localization of spectrally discriminated quantum-dots was also recently proposed . Improved axial and optical sectioning ability is also a need in the study of biological samples, which are typically thick. Confocal microscopy [22–24] and multiphoton microscopy [25–27] belong to the class of Laser Scanning Microscopy (LSM) techniques in which the image of a well-defined plane of the sample is collected from a point-by-point acquisition of the signal coming from the excitation of a focused laser spot on the sample, requiring a raster scanning to have a complete image of the sample.
In the last years many attempts to extend the resolution enhancement also to non-fluorescent samples have been made. Using the nonlinear dependence of the transient absorption on the illumination intensity and pump-probe illumination schemes, typical of STED microscopy, super-resolution methods are successfully reported [28–31]. However, these methods require high intensity and complex optical setups, as well as, a raster scanning of the sample to effectively image the sample. Recent works [32–34] tried even to generalize the concept of super-resolution of SIM fluorescence microscopy to the scattering non-fluorescent samples. However, because of the coherence of the light in the scattering or reflectance imaging modality of this class of samples, the direct generalization of the SIM technique is not straightforward and its performance is still under debate [35,36]. On the other hand, only recently the possibility of the extension to fluorescence-free samples of the localization microscopy underlying the basic idea of the PALM and STORM was explored in the works of Zhang et al. [37,38]. Analogously to the fluorescence emitted by an isolated chromophore, the back-scattered light coming from an isolated point-like particle gives rise to a diffraction-limited image on the CCD, which can be fitted with the PSF of the microscope in order to localize the scattering emission center. As in the PALM, in which the localized emission center is the most probable position of the emitting fluorophore, in the scattering analogous the localized center of emission is the most probable position of the scattering center. Obviously, differently from what happens with the photo-switchable chromophores in the PALM, there is no possibility of time switching the scattered signal of two close emitting particles simultaneously excited by the illumination beam. In this situation, indeed, the two close scattering objects cannot be individually distinguished because of the light diffraction limit. A possibility to overcome this limit is to spectrally separate the emission from close adjacent scattering particles. Based on this principle, in the references [37,38] the authors used a wavelength-based discrimination of the scattered light coming from a mixture of 25plasmonic nanoparticles and demonstrated a three-dimensional sub-resolution reconstruction of the relative scatterer positions for the different species of plasmonic particles by fitting the center of the PSF for each filtered wavelength. However, this resolution enhancement happens only for the sample regions were particles of different species are closely packed and, at same time, are sufficiently separated from the particles of the same species. Such wavelength dependence can be avoided if the localization approach is realized with a spatially structured time-varying illumination and if the discrimination criterion is based on the shape variation of the apparent PSF occurring when multiple, instead of single, objects are emitting scattered light simultaneously. A similar approach requires a very dense illumination mapping of the sample, where for example, a focused illumination spot is accurately moved in time across the sample. Furthermore, instead of focused scanning spots, even a dynamical changing speckle intensity pattern can be used to realize the dense illumination mapping of the sample, provided that a sufficient number of different speckle patterns can be generated [39,40]. A speckle pattern is characterized by a granular intensity [41,42] which arises from the interference of de-phased wavelets in a coherent laser beam. Such random de-phase originates from the roughness at the wavelength scale in any reflective surfaces, but can also be controlled on the propagating beam through an optical modulator. Dynamical Speckle patterns are obtained in a very simple way using a Spatial Light Modulator (SLM), which represents an interesting optical device for microscopy  and for laser scanning microscopy setups [25,44]. It can be coupled with a mechanical scanning system  to obtain an array of focalized spots scanning the sample simultaneously, or even used to holographically scanning the sample, without further mechanical movement of any component in the setup [46,47].
In the present work, we report about the Holographic - Single Scatterer Localization Microscopy (H-SSLoM), in which we use a SLM to control the random speckle intensity pattern of the illuminating laser beam in the sample plane. The back-scattered light, coming from the serial illumination mapping of the sample, is collected by a CCD in a time sequence. Each frame of this image stack is then elaborated in order to localize the centroid positions of the scattering objects and a new image of the sample showing the whole recorded information is reconstructed. We investigate the minimal discernible gap between two distinct scattering objects in a plane and we test also the sectioning ability of the method on a controlled 3D disreibution of the scattering centers. The imaging capability of the method on reflective surfaces is also tested. The method shows accuracy in localization of sparse scattering object in a plane well below the Abbe’s diffraction limit of the microscope. Moreover the sectioning ability on 3D objects together with the resolution and the contrast showed on reflective samples can be assimilated to the performances of a confocal microscope operating in the same wavelength range. Furthermore, the use of the SLM for the holographic control of the illumination ensures a high level of versatility in the illumination patterns, as well as the absence of mechanical movement in the sample scanning together with the possibility of highly reducing the dimensions of the scanning setup.
2. Optical setup and working principle
In this work, the illumination of the sample is controlled through a Computer Generated Holography (CGH) setup [48–50], whose schematic representation is shown in Fig. 1(a). The laser beam at λ = 532 nm (from Nd:YVO4 continuous-wave frequency-doubled laser), after a beam expander (lenses L1 and L2) is reflected onto a computer controlled SLM (phase-only, Holoeye Pluto) which is programmed for visualizing a hologram (kinoform) to reconstruct the desired illumination distribution in the microscope object plane. The diffracted beam is transmitted by the lens L3 and spatially filtered via an iris located at its second focal plane. After re-collimation (lens L4), the beam is finally focused onto the sample plane by means of the external lens L5 and the internal lens system (including tube lens LT and the microscope 100X oil-immersion objective, NA = 1.4). The light signal back-scattered by the sample is collected by the same objective, and imaged onto a CCD (Hamamatsu Photonics ORCA-Flash2.8) that records a 16-bit grayscale image of the microscope object plane. The time sequence of different illumination patterns, necessary when a complete mapping of the sample plane is needed, is achieved by properly refreshing the kinoform displayed on the spatial lightmodulator (typical refresh rate used is 25 Hz).
The working principle of the proposed Holographic - Single Scatterer Localization Microscopy is based on the precise localization of the centroids of the imaged light spots of sub-diffraction sparse scattering objects. The centroid localization in the CCD collected raw images is carried out with the open source ImageJ plugin QuickPALM , which analyzes each frame of a collected stack of images in order to identify the centroids of the imaged light spots. The crucial parameters of the centroid localization are the minimum Signal to Noise Ratio (SNR) and the maximum FWHM of the light spots appearing in the analyzed raw images. These parameters have to be chosen according to the noise level visible in the acquired images and to the width of the lateral PSF of the system. Only those recorded spots whose intensity is enough to overcome the minimum specified SNR and whose width matches the width of the lateral PSF are properly processed and the relative centroid is calculated and recorded, while all the other spots in the analyzed image are discarded. In particular, the FWHM control operates as a spot shape control and is able to discriminate the spots scattered from multiple simultaneously scattering particles based on the apparent width of the imaged spot. This role of the FWHM parameter as filtering control is further clarified in Appendix A. When all the frames in the acquired stack are successfully processed, a new image is reconstructed and each properly localized centroid is rendered as single bright pixel, similar to PALM microscopy images [15,51]. A complete description on the operating features of the elaboration software can be found in the Henriques et al. work .
In the localization microscopy, the precision of the localization procedure used to reconstruct the super-resolved image has to be experimentally determined. From the theory [37,51], in the case of low background level, localization precision associated to the centroid position determination of a sub-diffraction particle experiment can be approximated as , where s is the standard deviation of the system PSF and N number of collected photons. In this situation, a reasonable estimator of the experimental localization precision can be given by measuring the width of the distribution of multiple localization experiments conducted on the same isolated emitting particle [37,38,52,53]. Accordingly, the localization precision of our localization setup is determined by a multiple localization experiment on a single isolated scattering object [Fig. 1]. The sub-diffraction scatterer in our experiment is a silver disk (nominally 100 nm in diameter) fabricated on a coverslip glass substrate by electron beam lithography. The SEM image of the disk is shown in Fig. 1(b) together with its normalized intensity plot profile measured along the horizontal direction in Fig. 1(c) (mediated along the vertical direction). The sample is covered with the index-matching objective oil to avoid light contributions from reflection at the air-glass substrate interface and it is illuminated with a diffraction-limited focused light spot generated in the sample plane through the CGH setup. A series of 1000 kinoforms, generating the light spot in the sample plane in correspondence of the disk position, is displayed on the SLM in a time sequence and the light backscattered by the sample for each kinoform is recorded by the CCD. The image resulting from the summation of all the acquired images in the stack is shown in Fig. 1(d) (we refer to this image as summed image), with its horizontal intensity profile in Fig. 1(e). The image in Fig. 1(f) is reconstructed by plotting the positions of the centroids localized in the elaboration of the 1000 images of the stack. Each localized centroid is rendered as a bright pixel whose intensity is the mean intensity of the relative spot intensity measured in the CCD recorded frame . The reconstructed distribution has a Gaussian profile [Fig. 1(g)], clearly seen in the image of localized centroids by reconstructing the localized positions in an image whose pixel linear dimension is smaller than the pixel dimension in the original CCD acquired image. In particular, the image of Fig. 1(f) is obtained by setting the image pixel dimension at 1\5 of the pixel dimension in the original CCD acquired image. The centroid distribution shows a FWHM of about 25 nm, while the FWHM of the summed image is about 250 nm, according to the lateral dimension of the diffraction-limited PSF of our imaging system. We want to point out that an accurate determination of the width of centroid distribution, and so of the localization precision, should require, in principle, to perform the multiple localization experiment on a point-like scattering object. The scattering disk used in our experiment cannot be considered really a point scatter (diameter 100 nm), but this affects the localization precision by enlarging the measured distribution width and the precision of the centroid localization of our setup be considered at least of 25 nm.
Despite this high accuracy in localizing single isolated objects, two very close scattering objects, illuminated with a static focused light spot and lying in the distance less than the width of the diffraction-limited illumination spots [Fig. 1(d)], cannot be distinguished as single entities. However, the localization accuracy proven in Fig. 1 can be turned in an improved localization resolution if a very dense scanning of an extended area of the sample is realized and the time sequence of the collected back-scattered light is elaborated through the centroid localization procedure coupled with the rejection of spots whose width does not match the system PSF (See Appendix A). A very dense illumination mapping of the sample plane can be realized using dynamically changing speckle intensity patterns. In the CGH setup, the speckle light grains are easily produced in the hologram reconstruction plane by imposing a random spatially varying phase profile in the SLM plane. This phase profile in our experiment is a 8-bit grayscale image digitally built through the Matlab random number generator, in which the pixel values are randomly chosen in the [0-255] gray-level interval [inset Fig. 2(a)]. In Fig. 2(a), the image of a typical light pattern, generated by one of these random kinoforms on a reflective homogeneous silver slab situated in the microscope focal plane, is shown. This light pattern is characterized by a granular intensity pattern, where high intensity light spots are randomly distributed over a lower intensity background. From speckle theory, it is known that a speckle pattern fulfills the condition of fully development when the phases of interfering wavelets are uniformly distributed, resulting in a negative exponential probability distribution of the speckle intensity pattern [42,54,55]. This condition is met in the speckle patterns generated with the random kinoforms in the CGH setup. In Fig. 2(b) it is shown (using a Log scale) the intensity probability P(I) of a kinoform-generated speckle pattern as function of the dimensionless parameter , where is the mean value of the intensity in the speckle image. The linear decreasing trend confirms that the pattern shows an exponentially decreasing intensity probability and responds to the request on fully developed condition. Furthermore, the relation among different randomly-generated speckle patterns is investigated by means of image correlation analysis. The scatter plot between two speckle patterns reported in Fig. 2(c) clearly shows that the considered patterns are not correlated, and the estimation of correlation coefficient is always close to zero for every analyzed couple of randomly chosen speckle patterns. This confirms that our setup is able to generate independent speckle patterns imposing different random kinoforms in the SLM plane.
The high intensity and circularly symmetric spots randomly appearing in the speckle patterns can be used as sample mapping spots for our proposed method. A typical spot under consideration is pointed out by the yellow circle in the Fig. 2(a) with its zoomed image in the left panel of Fig. 2(d).
Choosing the control parameters in QuickPALM elaboration software in order to match the PSF of the system and to collect sufficiently intense spots of the intensity speckle patterns, it is possible to filter out the background (with lower intensity and non-symmetric shape in the intensity distribution) and apply the localization procedure only to the light signals coming from the bright and symmetric diffraction limited spots, whose centroid can thus be localized with high precision (right panel of Fig. 2(d)). We refer to the image of localized centroids obtained with the sequence of random illumination speckle pattern as H-SSLoM image hereafter. As the position of these high intensity symmetric spots in the objective focal plane is randomly changed when a new random kinoform is used to modulate the illuminating laser beam, a dense mapping of the whole objective field of view with these spots can be realized using a sufficiently high number of random kinoforms, sequentially displayed onto the SLM in a time sequence.
The number of properly localized spots in H-SSLoM images depends on the stochastic generation of symmetric and contrasted spots in the sample plane, on the nature of the sample and on the background noise level in the raw collected images. For this reason, a large number of independent speckle illumination patterns is required to reconstruct properly the image of the sample. In the next sections, the presented images are reconstructed using a number of independent illumination patterns chosen based on the analysis conducted in the appendix B about the dependence of the reconstruction on the number of frames included in the elaboration.
3. Localization resolution
To test the ability of the method in localizing multiple scattering objects with dynamical speckle illumination and to characterize the minimum discernible distance between the scattering features that the method can resolve, the H-SSLoM procedure is applied to a sample constituted by a series of 100 nm in diameter silver disks, whose edge-to-edge distance is varied from 30 to 600 nm. The SEM images of the array are shown in Fig. 3(a).
The sample is fabricated by electron beam lithography on a glass. In the resolution test, the sample is placed on the microscope stage and it is covered with the index matching microscope oil. The sample is illuminated with a series of 3600 independent speckle patterns of the type as in Fig. 2(b) and, for each of these illumination patterns, the image of the backscattered light spots is collected by the CCD. The acquired stack of 3600 images is then elaborated by the localization software and the image of localized centroid is reconstructed. The FWHM parameter of the elaboration software is set also in this experiment at the FWHM width of the system PSF. Figure 3(b) and Fig. 3(c) show, respectively, the reconstructed H-SSLoM imageand the relative summed image of the sample. Comparing the SEM image with the H-SSLoM image, it is clear that the random speckle illumination provides a proper way to densely map an extended area of the sample. All the disk positions are indeed correctly localized with high accuracy. The FWHM in the reconstructed centroid distributions relative to the isolated disks is about 25 nm, in accordance with the results in Fig. 2. Furthermore, it is evident that the H-SSLoM method is also able to improve the resolution. Two distinct scattering objects are evident in the H-SSLoM image [Fig. 3(b)] even for nominal edge-to-edge distance of 50 nm. This is not the case of the summed image where all the pairs of disks separated less than 250 nm are imaged as single blurred spots [Fig. 3(c)]. In Fig. 4 it is shown a zoomed view of the images relative to the disks separated by 90 nm with the relative normalized intensity profiles in the horizontal direction. In the H-SSLoM image, the two single disks are distinguished and a clear deep in the intensity plot between the two distributions is also visible [Fig. 4(c)-4(d)]. In the summed image and in the relative intensity profile [Fig. 4(e)-4(f)] instead, no information on the two distinct single objects is observable, according to the diffraction limit in ordinary optical microscope images.
In this resolution enhancement, the key role is played by the discrimination of imaged scattered spots based on their width. The spots having a FWHM larger than the PSF of the system are not processed in the centroid localization process. This spots are mainly associated with the illuminating spots localized in the gap between two scattering objects. Otherwise, when a speckle spot is produced randomly in a position such that only one of the scatterer is illuminated, the relative scattered spot is properly localized.
The localization of many of these spots permits the reconstruction of the centroid distribution for each of the disks of the sub-diffraction scattering couple. It should be noted that the reconstructed centroid distribution for small gap distances appears slightly asymmetric toward the adjacent spot. This could be ascribed to some proximity effect, which is not taken in account in the elaboration. However, the presented results clearly show that the proposed H-SSLoM method is able to distinguish light scattering objects in a flat two-dimensional sample with a resolution (50 nm in our experiment) well below the diffraction limit of our setup.
4. Sectioning ability and reflective samples
The spatial filtering operated by the elaboration software on the light spots which do not satisfy the elaboration parameters can be used to discard background light coming from a 3D distribution on scattering objects. This provides the technique with 3D sectioning capability. To test this ability, we realize the H-SSLoM method on a sample of silver nanoparticles of 50 nm mean radius dispersed onto a PDMS substrate.
The substrate has a grooved profile obtained by molding the PDMS on a sinusoidal silver-coated diffraction grating with dimensions of 3,2 μm in pitch and about 700 nm in height. The sample is fixed to a coverslip and mounted on a piezo stage able to translate in the axial direction. The position of the sample is varied in the range of 1.5 µm with a pitch of 100 nm. For each z-position, the image sequence for H-SSLoM is acquired illuminating the sample with7000 independent random speckle patterns. In Fig. 5, the H-SSLoM images are compared with the relative summed images for two axial positions, separated by 500 nm. From Fig. 5(a) it is evident that in the H-SSLoM image the background light, coming from particles in different planes with respect to the objective focal plane, is significantly reduced compared to the summed image in Fig. 5(b). This produces a visible contrast enhancement for the only particles lying in the objective focal plane. Indeed, repeating the experiment for a new axial position of the sample, many of the particles, blurred in the previous image, are clearly visible [Fig. 5(c)] with an improved contrast respect to the summed image in Fig. 5(d). This feature of the H-SSLoM images makes the method similar to a confocal microscope in which the spatial filtering of collected light, as well as rejection of the not-in-focus light, are realized by the check of spots shape and intensity in the raw images, imposed through the control parameters of the elaboration software.
A typical sample where the spot shape discrimination and the localization of spot centroid should not provide any spatial resolution improvement is constituted by high reflective homogeneous samples on which the thin features to be analyzed are present as non-reflective regions. In this case there are no discrete emitting particles to be localized and the centroid of each reflected spot does not contain super-resolution information on the non-reflective features of sample (see also Appendix A). This behavior should be common to all the localization method on not fluorescent samples where the spot discrimination is based on imaged spot width. We test this aspect applying the H-SSLoM method to a reflective sample constituted of a series of non-reflective slits engraved on a flat homogeneous silver surface by means of focused ion beam milling. The slits are 100 nm wide and 10 to 15 µm long. A series of 24000 different random kinoforms is used to illuminate densely the sample and the light backscattered by the sample surface is recorded by the CCD at each step. Figure 6 shows the resulting images with the summed image in Fig. 6(a) and the relative H-SSLoM image in Fig. 6(b), together with a zoomed view of two crossing lines in the insets.
As expected, the apparent width of the scratches in the H-SSLoM images is about 250 nm, confirming that no super-resolved features are reconstructed in the image. However, the H-SSLoM image in Fig. 6(b) shows an improved visibility of the slits with respect to summed image in Fig. 6(a), with sharper edges and enhanced contrast (see Appendix B for the dependence of the contrast on the number of indent speckle illumination patterns enclosed in the elaboration). This characteristic is even highlighted in the magnified views in insets. Due to the reduced size of the reconstructed pixel (1/10 of the CCD pixel size in these images), the H-SSLoM image preserves the same quality when zoomed in, while the zooming operation deteriorates the summed image.
5. Discussion and conclusions
In this work we propose a new microscopy method able to extent to sparse back-scattering samples the capability of a pixelated detector in localizing sub-diffraction emitters, with accuracy much better than the width of the recorded diffraction-limited image. The possibility to apply any localization elaboration on the images collected from back-scattering samples without any spectral discrimination of the emission from closely embedded scatterers, requires necessarily the use of a spatially structured illumination, able to make accessible information about the presence of individual scatterers by exciting each of them at different times. This illumination requirement is here efficiently realized using a dynamical random speckle intensity pattern [Fig. 2] to map in time the sample. Coupled with a software spatial filtering which retains and elaborates the information coming only from light spots with enough intensity and whose width matches the lateral PSF of the imaging system, the dynamical changing position of the speckle grains realizes a very dense scanning of the surface. Furthermore, the mapped region is always the largest sample area accessible at once in the objective field of view. The recorded back-scattered light from the sample is elaborated in a statistical procedure that localizes the positions of the light spot centroids. The reconstructed centroid images show a very accurate ability to well localize isolated scattering objects [Fig. 1] but, above all, also an improved resolution in distinguishing two very close point-like scattering objects. The resolution test in Fig. 3 and Fig. 4 shows that the method retains information on the presence of individual scattering objects in a well-defined sample plane for object separations up to 50 nm, while the corresponding conventional laser scanning images lose this information accordingly to the light diffraction limit. Furthermore, the software spatial filtering operated in the image elaboration procedure is able to improve the contrast in three-dimensional distributions of scattering objects [Fig. 5] producing images with z-sectioning ability comparable to the confocal microscope images. The imaging capability on nanostructures engraved on reflective surfaces is also tested. On this kind of samples, the image contrast is produced by the proper localization of the reflected spots (fitting the PSF of the system), respect to the dark absorbing/highly scattering nanostructures. In this configuration, the resolution is obviously limited by the diffraction-limited size of the illumination speckle grains. However, the centroids reconstruction procedure is still able to improve the structure visibility respect to the summed images [Fig. 6]. The samples analyzed in the present work are essentially binary, but we believe that the method can be easily extended also to sample having different scattering properties, with the possibility to reconstruct the gray-scale images according with the scattering efficiency.
The illumination patterns in H-SSLoM are controlled via a computer generated holography setup, based on the laser phase modulation through a LC spatial light modulator (SLM). Even if the speckle illumination could be in principle realized also through a rotating diffuser, with the advantages related to the shortening of the acquisition time, the use of the SLM allows the realization of the sample mapping without any optical component in mechanical movement, making the method immediately appealing in the practical situations where vibrations must be avoided and mechanical stability is needed. Moreover, the setup could be significantly miniaturized, making it usable also in space-demanding situations, even for Space applications. The use of a SLM to control the illumination could also permit the realization of the method with holographic controlled spot array in a mechanical motion-less scanning of the sample, suitable also to image samples that require specific illumination pattern, i.e. degradable samples.
Appendix A Simulation and spot discrimination criterion
In order to give a deeper sight at the working principle and the performances of the discrimination criterion operated in the elaboration of the H-SSLoM method, here it is proposed a phenomenological simulation about the illumination and the elaboration process of the method for a simple situation, similar to the our experimental framework. In the simulation, the optical setup consists of a microscope objective, operating in epi-illumination condition, and of a CCD collecting the image of the scattered light of the objective focal plane. The simulated sample is made up of two binary scattering squares and the analysis is conducted for a gap width between the two objects larger and smaller than the diffraction-limited width of the lateral Point Spread Function (assumed to have a Gaussian intensity profile with FWHMos=250 nm, where the subscript specifies that this value of the FWHM is due to optical system diffraction limit) of the simulated setup. In order to keep the situation as simple as possible, the illumination of the sample is obtained with a single diffraction-limited Gaussian spot moving across the sample in a time sequence to completely map the simulated field of view (FOV), and no light polarization effects are considered. The center and the peak intensity of this illumination spot are randomly chosen in each frame of the time sequence, in order to simulate one ideal circularly symmetric and well-contrasted speckle spot which can arise in a random position in the typical illuminating speckle pattern of the H-SSLoM [Fig. 2]. All the simulation in this section are obtained generating a Gaussian spots whose centers randomly span a simulated sample area of .
When the illuminating spot is generated in a position which partially overlaps with a scattering object, a simulated scattered light signal arises from the sample. This scattered light spot assumed to be a new Gaussian spot, whose center is the center of mass of the intensity of the illuminating spot, calculated on the overlapping pixel region between the illuminating spot and the scattering object, and with peak intensity proportional to mean intensity of the illuminating spot, calculated on the same region of pixels. The FWHM of the scattered spot is imposed to be diffraction-limited and, because this spot is imaged through the same objective used to illuminate the sample, it is fixed at the same value of the FWHMos of the illumination spots. The sequence of simulated images of the scattered light spots for each illumination frame is recorded in an stack of images, which is then analyzed through the elaboration software (QuickPALM ) for the centroid localization procedure. As no noise is present in the simulated images, the only elaboration parameter to be chosen is the maximum FWHMel (“elaboration”) of analyzed spot in order to be accepted as valid spot for centroid localization. Because the system is diffraction-limited, the smallest possible FWHMsc of the spots originating from sub-resolution scattering objects in the simulated images is fixed at the system FWHMos. Choosing FWHMel = FWHMos in the elaboration parameter ensures that the elaboration software localizes only the centroid of spots whose width matches the lateral PSF of the system. A typical example of the simulated situation in which the analyzed spot passes the width discrimination control is shown in Fig. 7(a). The dotted red line represents the FWHMel dimension, which is fixed at the value of FWHMos=250 nm. From the intensity plot profile in Fig. 7(a), it is clear that this spot fulfills the threshold condition on maximum FWHM imposed by the elaboration software. In the present simulation, this spot is the typical scattered spot arising from the illumination of an isolated scattering object or from a multi-scatterer system in which the illumination spot partially overlaps only with one scatterer. On the other side, when the illumination spot occurs in a position able to excite more than one scatter at time, the simulated recorded scattered light spot has an asymmetric and larger intensity profile [Fig. 7(b)] and does not pass the width control in the elaboration. Based on this discrimination criterion, the H-SSLoM method should be able to distinguish two close discrete scattering objects by localizing and retaining the information only on centroids of scattered spots which originate from the illumination of only one scatterer at time. In order to simulate the distribution of the localized centroids for scattering objects separated by a gap bigger than the FWHM of the optical system, the simulation is performed on a couple of scattering squares having a side width of 150 nm and a gap distance of 350 nm [Fig. 8(a)]. Fig. 8(b) - 8(c) show, respectively, the summed image of simulated raw sequence images of the scattered spots and the relative elaborated H-SSLoM image. In this section the simulated H-SSLoM images are rendered in green colors in order to clarly distinguish them from the experimental images in the other sections of the text. As the two squares are separated more than the system resolution limit, they are distinguishable as individual scattering objects in both the summed and H-SSLoM images. The appearing width of the objects in the summed image is equal to the simulated diffraction-limited PSF of the illumination and collection system, as expected from diffraction-limited imaging [Fig. 8(b)].
The localized spot centroids give rise to a Gaussian distribution, visible in H-SSLoM simulated image in Fig. 8(c), whose peaks are separated by a distance matching the center-to-center distance of the scattering objects. From the intensity plot, we can also determine the FWHM of the reconstructed centroid distribution, which results to be broadened because of finite size of the simulated scatterers (FWHMos/sidesquare = 1.67). However, the centroid distribution FWHM results narrower than the width of the scattering objects. This is due to the fact that in the simulation, the intensity associated to the scattered spot is proportional to the intensity of the illumination spot overlapping with the scatter area. For the spots whose centroids are located at the edges of the squares, the scattered amplitude is systematically small and this is rendered as pixels with low grayscale values in the centroid distribution (bottom panel Fig. 8(c)). These centroid positions are even cut from the valid localized spot if the elaboration procedure is performed with a threshold on the minimum signal to noise attributable to an analyzed spot to be considered valid. For this reason the information on the scatterer inner is mostly lost when also the noise is taken into account and only the position of the scatterer center can be properly reconstructed. A similar effect should be present also in the images experimental images of Fig. 1 and Fig. 3, where FWHMos/size = 2.5 and the elaboration is performed with non-zero noise level.
The simulation about the improved resolution ability of the H-SSLoM in localizing discrete scatterers is presented in Fig. 8(e) and Fig. 8(f), where are shown the simulated images of two scattering objects separated by a gap of 50 nm [Fig. 8(f)]. The summed image in Fig. 8(e) presents an enlarged intensity profile, which does not contain information about the two separated scatterers. These are instead distinguishable in the H-SSLoM image in Fig. 8(f), where two centroids distribution are clearly present in the H-SSLoM intensity profile. Even if the appearing gap does not correspond to real gap width, the distance between the peaks of the two reconstructed centroid distributions is equal to the center-to-center distance between the simulated scatterers (150 nm). The centroid distributions appear slightly asymmetric toward the gap. This is due to the spot shape filtering which essentially cut all the spots originated in that region because they are associated with bigger scattered FWHM [Fig. 7(b)]. This characteristic is not observed in the experimental images of Fig. 3 and Fig. 4, suggesting that in the experimental situation the coupling between the coherent scattering from the two objectss can play a role in the centroid reconstruction.
The presented simulation framework can be applied also to describe the performances in terms of resolution of the technique on a simulated homogeneous reflective sample on which the sub-resolution features are constituted of non-reflective regions. In this case, the simulation is run on a simulating absorbing slit, having a lateral width of 150 nm, engraved on a reflective surface. The simulated summed and the H-SSLoM images are shown in Fig. 9(a) and Fig. 9(b), respectively. Even in the H-SSLoM image, the appearing slit width in the is comparable with the FWHM of the simulate optical system, confirming that the centroid localization does not give super-resolution information on this class of samples. However, as in the experimental image in Fig. 6(b), the visible contrast is enhanced in the H-SSLoM image compared with the wide-field summed image.
Even if the presented simulation is very simple and disregards many aspects of the real experiment (like the proper simulation of the speckle pattern, the simulation of actual scattering mechanism, no effects due to the coherence of the scattering imaging are included), the key points of the principle (based on the centroid localization coupled with the FWHM discrimination criterion of the scattered spot in a dense illumination mapping of the sample) oh the method proposed in this paper and its resolution performances are properly described. A more accurate model should include all the neglected contributions for a proper description.
Appendix B Dependence of reconstruction performances on number of speckle illumination patterns
The illumination process is based on the generation of random speckle grains with high intensity, symmetric shape and with good contrast respect to the surrounding speckles, which all happen with a probability distribution in each speckle pattern. Furthermore, the number of properly localized spots in the H-SSLoM depends also on the sample nature and on the noise level present in the raw images. All these factors contribute in the determination of the minimum number of independent of independent speckle illumination patterns required for the correct reconstruction of a given sample, and any statistical analysis about the estimation of this number should include the probability of all the mentioned processes.
However, here we want to show that all these conditions can be met in the real experiment requiring a reasonable acquisition time. In this section it is conducted an analysis about the characteristics of H-SSLoM images elaborated including different independent illumination patterns for image reconstruction. In Fig. 10 are presented the H-SSLoM images of two silver disks separated by a gap of 150 nm and reconstructed for six different values of illumination pattern number N included in the elaboration. The images are rendered using a constant brightness/contrast level in the 16-bit grayscale images and for each image the number N, together with the intensity plot profile, is reported in the same panel. Even for N=500, the two separated distribution can be recognized in the H-SSLoM images, but the peak-to peak distance does not accurately measure the center-to center distance between the disks (250 nm from SEM image in Fig. 3(a)) because of the very few points used to reconstruct the distribution. As the number of included frames increases, the intensity distributions are better reconstructed and, starting from N=3000, the center-to-center distance is correctly reproduced from the peak-to-peak distance of the localized centroid distributions. Using ta SLM refresh rate of 25 Hz (typical refresh rate in our experiment), this reconstruction is accomplished in 150 s. The accumulation of a bigger number of frames can provide a better reconstruction of the centroid distributions and permits to localize the emission center with improved precision.
Increasing the number of illumination patterns should be necessary if the scattering objects are not perfectly identical and the experiment aims also to reconstruct the scattering properties in a grayscale image. In this case a larger number of illuminating speckles would be required in order to mediate on the random intensity of the speckle grains and ascribe the intensity differences in the reconstructed distribution to the differences in the scattering properties of the objects. In this case, the dependence of the granularity degree of the reconstructed H-SSLoM images on a homogeneous reflective sample on the number included frames N in the elaboration have be analyzed. For this purpose, the granularity of H-SSLoM images is characterized by the speckle severity (with standard deviation and mean intensity in the image). In Fig. 11 it is reported the dependence of the speckle severity S on the number of independent patterns N illuminating a region of an homogenous silver surface. In Fig. 11(a) the summed (top) and the H-SSLoM (bottom) images for three values of N are rendered at fixed brightness/contrast values. As the N increases, the mean value of the intensity in the elaborated images increases, resulting in a brighter reconstructed images. Also the absolute value of the intensity standard deviation increases with increasing N. However, the speckle severity S decreases with a saturating trend as the number of illumination patterns increases. This is shown in Fig. 11(b), plotting the value of the measured speckle severity in the H-SSLoM images at different values of N.
This behavior shows that a minimum number of patterns N is required in order to ascribe intensity variation in the H-SSLoM images to the scattering properties of the surface above the intensity fluctuations of the illuminating speckle grains. An empirical criterion on the minimum N can be established from the value where the saturation trend is recognizable (for example N > 5000 from the graph in Fig. 11(b)).
Reducing granularity in the H-SSLoM images is important also in order to reconstruct the images of non-reflective features inscribed in a reflective surface (as in the case of Fig. 6), where speckle severity can hinder structure visualization if a not sufficient number of illumination patterns are included. In Fig. 12(a) are presented four H-SSLoM images of the non-reflective slit inscribed in the homogeneous reflective surface is reconstructed using a different number N of speckle illuminating patterns. The relative intensity plot profiles are plotted in Fig. 12(b). For N=1000, the contrast visible in the image is low and the presence of the line in the image is barely visible from the intensity profile, where the grayscale image value in the region of the slit is only slightly lower than the mean value outside the line and is almost confused in the speckle fluctuations. Increasing N, the visible contrast in the reconstructed images also increases and the groove profile become defined in the intensity plots. In order to characterize the visibility in the different images, we use the parameter, where is the mean speckle intensity and is the speckle standard deviation measured outside of the region of the groove, and is the average intensity evaluated in the five neighbor pixels around the minimum intensity pixel in the slit. The value of the visibility parameter V for the four analyzed images is plotted in the graph of Fig. 12(c), showing that there is more visibility as the number of frames included in the reconstructed image increases. This is in accordance with the considerations reported above on the speckle severity reduction as the number N increases, which is necessary to reconstruct a properly contrasted H-SSLoM image on this class of samples. This also justify the large number (N=24000) of illumination patterns used to reconstruct the image of the structures presented in Fig. 6(b).
Fabrication was performed at the Harvard University Center for Nanoscale Systems (CNS), a member of the National Nanotechnology Coordinated Infrastructure (NNCI), which is supported by the National Science Foundation under NSF award no. 1541959. CNS is a part of Harvard University.
We gratefully thank Dr. Emanuele Orabona for many useful discussions and Mr. Giovanni De Gregorio for the support in the simulation.
References and links
1. E. Betzig, A. Lewis, A. Harootunian, M. Isaacson, and E. Kratschmer, “Near Field Scanning Optical Microscopy (NSOM): Development and Biophysical Applications,” Biophys. J. 49(1), 269–279 (1986). [CrossRef] [PubMed]
2. D. W. Pohl, W. Denk, and M. Lanz, “Optical stethoscopy: Image recording with resolution λ/20,” Appl. Phys. Lett. 44(7), 651–653 (1984). [CrossRef]
3. A. Harootunian, E. Betzig, M. Isaacson, and A. Lewis, “Super‐resolution fluorescence near‐field scanning optical microscopy,” Appl. Phys. Lett. 49(11), 674–676 (1986). [CrossRef]
4. A. Ambrosio, M. Allegrini, G. Latini, and F. Cacialli, “Thermal processes in metal-coated fiber probes for near-field experiments,” Appl. Phys. Lett. 87(3), 033109 (2005). [CrossRef]
5. A. Ambrosio, A. Camposeo, A. Carella, F. Borbone, D. Pisignano, A. Roviello, and P. Maddalena, “Realization of submicrometer structures by a confocal system on azopolymer films containing photoluminescent chromophores,” J. Appl. Phys. 107(8), 083110 (2010). [CrossRef]
6. A. Ambrosio, M. Alderighi, M. Labardi, L. Pardi, F. Fuso, M. Allegrini, S. Nannizzi, A. Pucci, and G. Ruggeri, “Near-field optical microscopy of polymer-based films with dispersed terthiophene chromophores for polarizer applications,” Nanotechnology 15(4), S270–S275 (2004). [CrossRef]
7. L. Novotny and B. Hecht, Principles of Nano-Optics (Cambridge University Press, 2006).
8. S. W. Hell and J. Wichmann, “Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy,” Opt. Lett. 19(11), 780–782 (1994). [CrossRef] [PubMed]
10. M. G. L. Gustafsson, L. Shao, P. M. Carlton, C. J. R. Wang, I. N. Golubovskaya, W. Z. Cande, D. A. Agard, and J. W. Sedat, “Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination,” Biophys. J. 94(12), 4957–4970 (2008). [CrossRef] [PubMed]
11. M. G. L. Gustafsson, “Nonlinear structured-illumination microscopy: wide-field fluorescence imaging with theoretically unlimited resolution,” Proc. Natl. Acad. Sci. U.S.A. 102(37), 13081–13086 (2005). [CrossRef] [PubMed]
12. E. Mudry, K. Belkebir, J. Girard, J. Savatier, E. Le Moal, C. Nicoletti, M. Allain, and A. Sentenac, “Structured illumination microscopy using unknown speckle patterns,” Nat. Photonics 6(5), 312–315 (2012). [CrossRef]
13. H. Yilmaz, E. G. van Putten, J. Bertolotti, A. Lagendijk, W. L. Vos, and A. P. Mosk, “Speckle correlation resolution enhancement of wide-field fluorescence imaging,” Optica 2(5), 424 (2015). [CrossRef]
14. J. Min, J. Jang, D. Keum, S.-W. Ryu, C. Choi, K.-H. Jeong, and J. C. Ye, “Fluorescent microscopy beyond diffraction limits using speckle illumination and joint support recovery,” Sci. Rep. 3, 2075 (2013). [CrossRef] [PubMed]
15. E. Betzig, G. H. Patterson, R. Sougrat, O. W. Lindwasser, S. Olenych, J. S. Bonifacino, M. W. Davidson, J. Lippincott-Schwartz, and H. F. Hess, “Imaging intracellular fluorescent proteins at nanometer resolution,” Science 313(5793), 1642–1645 (2006). [CrossRef] [PubMed]
18. M. Bates, B. Huang, and X. Zhuang, “Super-resolution microscopy by nanoscale localization of photo-switchable fluorescent probes,” Curr. Opin. Chem. Biol. 12(5), 505–514 (2008). [CrossRef] [PubMed]
19. M. Born and E. Wolf, Principles of Optics: Electromagnetic Theory of Propagation, Interference and Diffraction of Light (Cambridge University Press, 1997).
20. N. Bobroff, “Position measurement with a resolution and noise‐limited instrument,” Rev. Sci. Instrum. 57(6), 1152–1157 (1986). [CrossRef]
21. M. Song, A. Karatutlu, I. Ali, O. Ersoy, Y. Zhou, Y. Yang, Y. Zhang, W. R. Little, A. P. Wheeler, and A. V. Sapelkin, “Spectroscopic super-resolution fluorescence cell imaging using ultra-small Ge quantum dots,” Opt. Express 25(4), 4240–4253 (2017). [CrossRef] [PubMed]
23. R. H. Webb, “Confocal optical microscopy,” Rep. Prog. Phys. 59(3), 427–471 (1996). [CrossRef]
25. J. Bewersdorf, A. Egner, and S. W. Hell, “Multifocal Multi-Photon Microscopy,” in Handbook Of Biological Confocal Microscopy, J. B. Pawley, ed. (Springer US, 2006), pp. 550–560.
27. A. Ambrosio, P. Maddalena, A. Carella, F. Borbone, A. Roviello, M. Polo, A. A. R. Neves, A. Camposeo, and D. Pisignano, “Two-Photon Induced Self-Structuring of Polymeric Films Based on Y-Shape Azobenzene Chromophore,” J. Phys. Chem. C 115(28), 13566–13570 (2011). [CrossRef]
28. P. Wang, M. N. Slipchenko, J. Mitchell, C. Yang, E. O. Potma, X. Xu, and J.-X. Cheng, “Far-field imaging of non-fluorescent species with subdiffraction resolution,” Nat. Photonics 7(6), 449–453 (2013). [CrossRef] [PubMed]
29. R. Heintzmann, “Super-resolution imaging: Beyond the realm of fluorescence,” Nat. Photonics 7(6), 426–428 (2013). [CrossRef]
31. E. S. Massaro, A. H. Hill, and E. M. Grumstrup, “Super-Resolution Structured Pump–Probe Microscopy,” ACS Photonics 3(4), 501–506 (2016). [CrossRef]
32. S. Chowdhury, A.-H. Dhalla, and J. Izatt, “Structured oblique illumination microscopy for enhanced resolution imaging of non-fluorescent, coherently scattering samples,” Biomed. Opt. Express 3(8), 1841–1854 (2012). [CrossRef] [PubMed]
33. B.-J. Chang, S. H. Lin, L.-J. Chou, and S.-Y. Chiang, “Subdiffraction scattered light imaging of gold nanoparticles using structured illumination,” Opt. Lett. 36(24), 4773–4775 (2011). [CrossRef] [PubMed]
34. D. Dan, M. Lei, B. Yao, W. Wang, M. Winterhalder, A. Zumbusch, Y. Qi, L. Xia, S. Yan, Y. Yang, P. Gao, T. Ye, and W. Zhao, “DMD-based LED-illumination super-resolution and optical sectioning microscopy,” Sci. Rep. 3, 1116 (2013). [CrossRef] [PubMed]
35. K. Wicker and R. Heintzmann, “Resolving a misconception about structured illumination,” Nat. Photonics 8(5), 342–344 (2014). [CrossRef]
36. R. Horstmeyer, R. Heintzmann, G. Popescu, L. Waller, and C. Yang, “Standardizing the resolution claims for coherent microscopy,” Nat. Photonics 10(2), 68–71 (2016). [CrossRef]
37. P. Zhang, S. Lee, H. Yu, N. Fang, and S. H. Kang, “Super-resolution of fluorescence-free plasmonic nanoparticles using enhanced dark-field illumination based on wavelength-modulation,” Sci. Rep. 5(1), 11447 (2015). [CrossRef] [PubMed]
38. P. Zhang, K. Kim, S. Lee, S. K. Chakkarapani, N. Fang, and S. H. Kang, “Augmented 3D super-resolution of fluorescence-free nanoparticles using enhanced dark-field illumination based on wavelength-modulation and a least-cubic algorithm,” Sci. Rep. 6(1), 32863 (2016). [CrossRef] [PubMed]
41. J. W. Goodman, “Some fundamental properties of speckle,” J. Opt. Soc. Am. 66(11), 1145 (1976). [CrossRef]
42. J. W. Goodman, “Statistical properties of laser speckle patterns,” 9–75 (1975).
43. C. Maurer, A. Jesacher, S. Bernet, and M. Ritsch-Marte, “What spatial light modulators can do for optical microscopy,” Laser Photonics Rev. 5(1), 81–101 (2011). [CrossRef]
44. N. Matsumoto, S. Okazaki, Y. Fukushi, H. Takamoto, T. Inoue, and S. Terakawa, “An adaptive approach for uniform scanning in multifocal multiphoton microscopy with a spatial light modulator,” Opt. Express 22(1), 633–645 (2014). [CrossRef] [PubMed]
45. Y. Shao, W. Qin, H. Liu, J. Qu, X. Peng, H. Niu, and B. Z. Gao, “Multifocal multiphoton microscopy based on a spatial light modulator,” Appl. Phys. B 107(3), 653–657 (2013). [CrossRef]
46. V. Nikolenko, B. O. Watson, R. Araya, A. Woodruff, D. S. Peterka, and R. Yuste, “SLM microscopy: scanless two-photon imaging and photostimulation with spatial light modulators,” Front. Neural Circuits 2, 5 (2008). [CrossRef] [PubMed]
48. J. W. Goodman, Introduction to Fourier Optics (Roberts and Company Publishers, 2005).
49. A. Ambrosio, L. Marrucci, F. Borbone, A. Roviello, and P. Maddalena, “Light-induced spiral mass transport in azo-polymer films under vortex-beam illumination,” Nat. Commun. 3, 989 (2012). [CrossRef] [PubMed]
50. E. Orabona, A. Ambrosio, A. Longo, G. Carotenuto, L. Nicolais, and P. Maddalena, “Holographic patterning of graphene-oxide films by light-driven reduction,” Opt. Lett. 39(14), 4263–4266 (2014). [CrossRef] [PubMed]
51. R. Henriques, M. Lelek, E. F. Fornasiero, F. Valtorta, C. Zimmer, and M. M. Mhlanga, “QuickPALM: 3D real-time photoactivation nanoscopy image processing in ImageJ,” Nat. Methods 7(5), 339–340 (2010). [CrossRef] [PubMed]
53. H. Deschout, F. Cella Zanacchi, M. Mlodzianoski, A. Diaspro, J. Bewersdorf, S. T. Hess, and K. Braeckmans, “Precisely and accurately localizing single emitters in fluorescence microscopy,” Nat. Methods 11(3), 253–266 (2014). [CrossRef] [PubMed]
55. D. C. Ong, S. Solanki, X. Liang, and X. Xu, “Analysis of laser speckle severity, granularity, and anisotropy using the power spectral density in polar-coordinate representation,” Opt. Eng. 51(5), 054301 (2012). [CrossRef]