1. Introduction

By encoding the normally inaccessible high-resolution information into the measured images through moiré effect, structured illumination microscopy (SIM) can break through the spatial resolution barrier of wide-field microscopy. Due to the low photodamage, low photobleaching and high compatibility for various fluorescent labels, SIM has been widely applied in the super-resolution imaging of fine structures in or between cells [1], including cytoskeleton [2], mitochondria [3], lysosomes [4], endoplasmic reticulum [5], Golgi apparatus [6], nuclear pore complex [7], and plasmodesmata [8]. Compared with other super-resolution imaging techniques, such as single molecule location microscopy (SMLM) [9,10] and stimulated emission depletion microscopy (STED) [11,12], which require either repetitive acquirement of sparse excited samples or point-by-point scanning, SIM has the unique advantage of high imaging speed, because only a few images are needed to extract a super-resolution image. Therefore, SIM has become an irreplaceable tool in observing the dynamics of fine structures in live cells. However, limited by the linear structured illumination mode, the resolution of SIM can only be improved by two times over the wide-field microscopy, which is about 100 nm. Such a resolution of SIM is much lower than that of SMLM or STED, which can reach almost 20 nm. SIM has the characteristics of high-speed and super-resolution, but the limited resolution greatly hampers its applications in exploring the dynamics of finer subcellular structures. To improve the resolution of SIM, several schemes have been proposed. For example, Gustafsson achieved the lateral resolution of 50 nm with a saturated structured-illumination microscopy (SSIM) by utilizing a nonlinear dependence of the fluorescence emission rate on the illumination intensity [13]. SSIM obtains the resolution enhancement, but it comes at the cost of severe photodamage and reduces the imaging speed. The higher laser power is required to induce the fluorescence saturation in SSIM, and so the sample will be more susceptible to photodamage and photobleaching. Furthermore, more spatial frequency components are mixed in SSIM, and thus more structured illumination patterns with different phase shift steps and directions are required to separate and recombine these frequency components, which results in the low imaging speed. Alternatively, a photoswitchable fluorescent protein can be used to provide the nonlinear structured illumination instead of the saturation to reduce the laser power [14], but the imaging speed is still reduced. Recently, Zhao et. al. proposed a sparse deconvolution method to further improve the resolution of SIM by employing the sparsity and continuity priors of biological structures (Sparse-SIM) [15]. As an image postprocessing method, the Sparse-SIM has no modification to the instrument structure or acquirement mode of conventional SIM, thus the advantages of SIM are preserved.

Recently, deep image prior (DIP) has attracted lots of attention in computational vision, which is based on the structure of a generator network that is sufficient to capture a great deal of low-level image statistics priors to any learning [16]. In this work, we introduce the concept of DIP into the image postprocessing of SIM to further improve the lateral resolution, that is, deep resolution-enhanced SIM (DRE-SIM). An untrained convolution network with encoder-decoder structure is designed to recover the high frequency information that is lost in SIM, and the optical diffraction limit and SIM image reconstruction are modeled to provide the criterion for the parameter updating of the network. DRE-SIM can realize the resolution enhancement in a self-supervised mode without the training datasets, which can greatly mitigate the burden of massive data acquirement. Our theoretical simulation and experimental measurements show the further super-resolution capability of DRE-SIM compared with conventional SIM, and a resolution enhancement by a factor of about 1.4 is experimentally obtained.

2. Principle

The resolution of the image acquired in wide-field microscopy is restricted by optical diffraction limit due to the finite aperture of objective. The light from each point source in the object plane is interfered into a pattern in the image plane with a special distribution after passing through a microscopy system, which is defined as point spread function (PSF). Modeled with a linear space-invariant system, the imaging process of a wide-field microscopy can be descripted by the convolution of PSF and the object image with additional noise, and is given by

For the ideal case of an aberration-free objective, the PSF is equal to the well-known Airy disk, which is calculated by the diffraction limit with a finite aperture. According to this model, the lateral resolution limit of wide-field microscopy can be described by the Rayleigh criterion, and is written as r = 0.61λ/NA, where λ is the emission wavelength, and NA is the numerical aperture of the objective.

To break through the diffraction limit of wide-field microscopy, a structured illumination is introduced to encode the inaccessible high-frequency information into the observed images through moiré effect [17,18]. The fluorescence-labeled sample is illustrated with a sinusoidal pattern to generate a frequency shift in SIM. The recorded image in SIM is expressed in the frequency domain as

To exceed the resolution limitation of SIM while preserve its superiority in the imaging speed, the postprocessing of SIM image is a preferred option. From the perspective of spatial frequency, the prior within the natural image bridges the lower frequency components preserved in SIM and the higher frequency components being inaccessible in SIM, thus it can provide the feasibility to further improve the resolution of SIM. Besides, for the SIM imaging model that contains the finite aperture diffraction, the structured illumination and image reconstruction are indispensable to achieve this task. In some extent, it is similar to a deconvolution microscopy [19], which improves the contrast and resolution of wide-field images. Employing the sparsity and continuity of biological structures, Sparse-SIM has improved the lateral resolution of SIM by iterative deconvolution [15], which has opened a pathway that a super-resolution image can be further improved in the resolution. However, there is still lots of room in the resolution enhancement. On the one side, only two handcrafted priors are utilized in the Sparse-SIM, one is Hessian matrix continuity that is used to reduce artifacts and increase robustness, and the other one is L1 norm sparsity that is used to balance the extraction of high-frequency information. The two handcrafted priors do show good constrain effect during the iteration, but they are far from fully exploiting the prior of natural images. The prior with more implicit characteristics would be preferred for better resolution enhancement. On the other side, multiple algorithm parameters in the Sparse-SIM need to be artificially modified to guarantee the best resolution enhancement, which is quite time-consuming and unstable. Here, DRE-SIM is proposed to further extract the higher spatial frequency from SIM by adopting the concept of DIP. The powerful ability of DIP has been verified by the applications in denoising [20], inpainting [21] and pixel super-resolution [22]. Inspired by this concept, various works have been reported to handle different computational imaging tasks by untrained neural networks associated with corresponding physical models, including coherent phase imaging [23], digital in-line holography [24,25], and ghost imaging [26]. Taking the advantage of the untrained neural network, DRE-SIM can extract much more implicit image priors than the handcrafted priors, and get rid of the fussy algorithm parameter optimization. As has been proved, a neural network itself is sufficient to capture a great deal of low-level image statistics prior for various tasks [16], thus it can also be used to extend the spatial frequency bound of SIM. Taking the SIM imaging model into consideration, the transformation from the original image to the SIM image is used to physically ensure the resolution enhancement in DRE-SIM.

The framework of DRE-SIM is shown in Fig. 1 (a), which contains a neural network with the updatable parameters to achieve the image resolution enhancement, a physical model to describe the SIM imaging and reconstruction process, and a loss function to provide the criterion for updating the parameters of the neural network. The neural network is an hourglass-shape encoder-decoder network with skip blocks between the corresponding downsampling and upsampling blocks, which has been verified to extract the priors for various tasks [27,28]. The structures inside these blocks are shown in Fig. 1 (b). The input SIM image x₀ is first processed by the neural network with randomly initialized parameters θ, the output image f_θ(x₀) is then calculated with equivalent OTF of SIM to acquire the corresponding SIM image O(f_θ(x₀)). Here, the equivalent OTF of SIM is estimated from the optical parameters of SIM system, where the cut-off frequency bound of the microscope and the frequency shift of structured illumination patterns are considered, as shown in Fig. 1(c). The details are provided in Appendix A. 4. The loss function is sequentially calculated to update the parameters θ of the neural network, and is given by

 

Fig. 1. Schematic diagram of DRE-SIM. (a) The flowchart of DRE-SIM. (b) The detailed block structures in the neural network. (c) Estimation of the equivalent OTF of SIM system.

Download Full Size | PDF

It is worth noting that the neural network tends to first extract natural super-resolution information and then exactly fit towards reducing loss with the input SIM images during the parameter optimization. Here, DRE-SIM can optimize the network parameters in a self-supervised mode without the training datasets.

3. Results and discussion

To demonstrate the further super-resolution capability of DRE-SIM, a theoretical simulation is conducted. A simulated image containing some curves with a width of 32.5 nm is utilized as the ground truth, which is similar to some biological structures. The wide-field image is calculated by the convolution of the ground truth and the PSF of a microscope with numerical aperture NA = 1.2 and fluorescence wavelength of 580 nm. The SIM image is reconstructed with HiFi-SIM algorithm [29] based on the modulated images acquired by sequentially structured illumination and diffraction limitation. Then, the SIM image is processed by the DRE-SIM algorithm in Fig. 1 to further improve the resolution. These simulation images, including ground truth, wide-field, SIM and DRE-SIM images, are shown in Figs. 2(a)-(d), together with the enlarged area for better observing the structure details. Compared with the SIM image, the DRE-SIM image has the higher contrast, and the outline of these curves is clearer. Furthermore, the intensity distribution along the labeled line in the enlarged area is also extracted, as shown in the insets of Figs. 2(a)-(d). Obviously, these curves in the DRE-SIM image are more distinguishable than those in the SIM image, and the intensity distribution of these curves in the DRE-SIM image shows a sharper profile. In order to intuitively demonstrate the resolution enhancement of DRE-SIM, the frequency spectra in Figs. 2(a)-(d) are calculated by Fourier transform, and the calculated results are shown in Figs. 2(e)-(h). Generally, the more high-frequency components are extracted, the higher resolution is obtained. Only the low frequency components are preserved in the wide-field image due to the limited OTF of the microscope, and these frequency components are extended by twice in the SIM image, while can be further extended in the DRE-SIM image, which are corresponding to the higher resolution.

 

Fig. 2. Theoretical simulation of DRE-SIM. The ground truth, wide-field, SIM and DRE-SIM images of simulated biological structure, together with the extracted intensity distribution along the labeled line in the enlarged area (a-d), and the corresponding distribution in the frequency domain by Fourier transform (e-h).

Download Full Size | PDF

The noise is inevitable in the image acquirement, and the signal-noise-ratio (SNR) plays an important role in the super-resolution imaging [30]. The noise may be amplified in super-resolution recovery, which leads to the calculation error and even vague artifacts, thus the SNR determines the super-resolution image quality and resolution limit in some extent. Therefore, it is necessary to analyze the effects of the noise on SIM and DRE-SIM, here Gaussian noise and Gaussian-Poisson mixed noise are added to the recorded images in SIM. The corresponding wide-field, SIM and DRE-SIM images are shown for comparison, as presented in Fig. 3. With the increase of noise level, the artifacts in the background of SIM image increase, and the distortion of the curves becomes serious, which are attributed to the calculation error, because the phase and frequency estimation in the SIM image reconstruction is vulnerable to the noise. However, DRE-SIM shows good performance in the noise elimination and artifact depression, and so it still can obtain the resolution enhancement under the noise condition. Obviously, DRE-SIM has the high resistance to the noise and artifacts in the background of SIM image, which make it suitable for processing the experimental images.

 

Fig. 3. Effects of the noise on SIM and DRE-SIM. The wide-field, SIM and DRE-SIM images of simulated biological structure under the conditions of 40%, 60%, 80% Gaussian noise and 80% Gaussian + Poisson noise (a-d, e-h, i-l).

Download Full Size | PDF

To further verify the resolution enhancement of DRE-SIM, various biological structures inside the cells are experimentally imaged. Firstly, the tubulin microtubules in mouse embryonic fibroblast (MEF) cells are measured. Tubulin microtubules, as an important part of the cytoskeleton, have the functions to maintain the structure of cells, which can provide a platform for the intracellular transport and participate in cell division. The distribution of tubulin microtubules is quite similar to the simulated biological structure with the curves in Fig. 2. In the experiment, a commercial SIM system is used to acquire the raw SIM images of tubulin microtubules in MEF cells. Then, the SIM image is further processed by DRE-SIM to acquire the resolution-enhanced image. The wide-field, SIM and DRE-SIM images of tubulin microtubules in MEF cells are shown in Figs. 4 (a)-(c), together with the extracted intensity distribution along the labeled line in the enlarged area. DRE-SIM provides a sharper intensity profile than SIM, and the intersection of multi microtubules can be clearly distinguished in DRE-SIM, which is still blurry in SIM. Additionally, Fourier ring correlation (FRC) is calculated to quantitatively represent the lateral resolution [31,32], as shown in Fig. 4(d). The resolution is increased to 91 nm in DRE-SIM from 139 nm in SIM.

 

Fig. 4. Experimental results of DRE-SIM. The wide-field, SIM and DRE-SIM images of tubulins in MEF cells (top row) and actins in NIH/3T3 cells (bottom row), together with the extracted intensity distribution along the labeled line in the enlarged area (a-c, e-g), and the corresponding Fourier ring correlations for the SIM and DRE-SIM images (d, h).

Download Full Size | PDF

Secondly, the actin microfilaments in NIH/3T3 cells, as a more complex fine structure, are measured to show the versatility of DRE-SIM in the resolution enhancement. Actin microfilaments are responsible for cell shape, cell mobility and cyclosis, which have various forms for different functions, such as stress fibers and bundles. Similarly, the actin microfilaments in NIH/3T3 cells are imaged by the SIM system, and then the SIM image is processed by DRE-SIM, as shown in Figs. 4(e)-(g). The substructure in the actin bundle is undistinguished in the wide-field image, and only a fuzzy profile can be observed in the SIM image, but the two intersected actin fibers in the bundle are clearly resolved in the DRE-SIM image. The FRC calculation shows that DRE-SIM improve the resolution of SIM from 121 nm to 87 nm, as shown in Fig. 4(h). In general, DRE-SIM can further improve the resolution of SIM by a factor of about 1.4 based on our experimental observations.

To demonstrate the wide applicability of DRE-SIM in diverse biological structures, the reconstructed SIM images of clathrin-coated pits (CCPs) and endoplasmic reticulum (ER) in COS-7 cells [33] are also further processed by DRE-SIM. As can be seen in Fig. 5 (c), the CCPs in DRE-SIM image show clearer ring shape than that in SIM image. Similarly, the ER in DRE-SIM image provides sharper profile, as shown in Fig. 5 (f). Overall, DRE-SIM is able to further enhance the resolution of SIM images and provide finer details for various biological structures.

 

Fig. 5. Resolution enhancement of CCPs and ER in COS-7 cells by DRE-SIM. (a)-(c) The wide-field, SIM and DRE-SIM images of CCPs. (d)-(f) The wide-field, SIM and DRE-SIM images of ER. The insets are enlarged view of the areas enclosed by the yellow square.

Download Full Size | PDF

The performance of DRE-SIM is further compared with that of Richardson-Lucy deconvolution (RL) [34,35], Richardson-Lucy deconvolution with TV regularization (RLTV) [36], and Sparse-SIM. The SIM image of tubulins with serious background [33] is processed with these methods. In fact, a background removal operation is utilized in Sparse-SIM, which can greatly suppress the background and facilitate the resolution enhancement. For a fair comparison, the background removal algorithm is also embedded into DRE-SIM. As shown in Fig. 6, DRE-SIM shows less artifacts and better resolution enhancement compared with RL and RLTV, and it is close to that of Sparse-SIM in visual effects. Therefore, DRE-SIM is able to achieve the resolution enhancement comparable with Sparse-SIM, while has the advantages of less parameter optimization and more flexible implementation.

 

Fig. 6. Comparison of DRE-SIM, RL, RLTV and Sparse-SIM in resolution enhancement of tubulins. (a-f) The SIM, RL, RLTV, SIM with background removal, Sparse-SIM and DRE-SIM with background removal images of tubulins. The insets are enlarged view of the areas enclosed by the yellow square.

Download Full Size | PDF

According to the experimental results, DRE-SIM provides a powerful tool to further improve the resolution of SIM. However, there are still some limitations in DRE-SIM. On the one hand, the resolution is still lower compared with other super-resolution techniques, i.e., SMLM and STED, which greatly depends on the SNR of the measured image, because the balance between the resolution enhancement and artifact depression needs to be considered. On the other hand, the space-variance of PSF in the microscope will enlarge the bias between the OTFs in the physical model and actual one, which brings the calculation error and degrades the performance of DRE-SIM. Since DRE-SIM is a postprocessing method, it has the advantages of improving the lateral resolution while keeping the imaging speed, which will provide a flexible super-resolution tool to investigate the dynamics of finer biological structures. Meanwhile, DRE-SIM will greatly promote the development in many fields of biology, such as cellular metabolism, organelle structure and dynamics, and so on. In addition, the applications of DRE-SIM are not limited to the resolution enhancement of SIM images. The microscopic images acquired by other imaging techniques can also be further processed to get higher resolution, including spinning-disc confocal SIM (SD-SIM) [37], STED [11], two-photon microscopy [38], and expansion microscopy [39]. Moreover, taking the image formation model of 3D SIM [40] into the framework of DRE-SIM, the 3D resolution enhancement could also be achieved.

4. Conclusion

In summary, we have presented an image postprocessing method DRE-SIM to surpass the resolution limitation of SIM by an untrained neural network. Utilizing the implicit priors and self-supervising framework in neural network, DRE-SIM has further achieved the resolution enhancement based on the reconstructed SIM image. A lateral resolution enhancement by a factor of about 1.4 was experimentally demonstrated by imaging the fine biological structures in cells. Compared with the Sparse-SIM, DRE-SIM utilized the implicit priors by the neural network instead of the hand-crafted priors, and therefore it eliminated the requirement for fussy artificial parameter optimization. Besides, no training datasets were needed to update the neural network in DRE-SIM, which makes it easier to be used in practical applications and avoid network generalization problem. Considering the comprehensive advantages of DRE-SIM in the lateral resolution and imaging speed, it will have a wide application prospect in the area of biomedical imaging.

Appendix A: Methods

1. Sample preparation

Mouse embryonic fibroblast (MEF) cells were fixed for 15 minutes at room temperature in a buffer containing 3.7% formaldehyde, 0.3% Triton X-100, 0.1% Glutaraldehyde, 80 mM PIPES pH 6.8, 1 mM EGTA, and 1 mM MgCl The fixed cells were rinsed three times with PBS, incubated for 7 minutes with freshly prepared sodium borohydride at a concentration of 1 mg/ml to reduce background fluorescence, and washed three times with Mili-Q water. After blocking with 2% BSA-PBS for 30 minutes, cells were incubated with mouse anti-tubulin antibody (sigma) in 2% BSA-PBS for 1 to 2 hours at room temperature, followed by 1 hour with Alexa Fluor 647-conjugated anti-mouse secondary antibody. Following that, cell samples were treated with or without 4% paraformaldehyde and washed three times with PBS before imaging.

NIH/3T3 cells were cultured in DMEM supplemented with 10% FBS and grown on the confocal dish of 170 ± 5 nm thickness. Cells were fixed with 4% PFA for 15 minutes, followed by PBS washes three times, and permeabilization with 0.5% Trtion-X100 for 10 minutes. Another two rounds of PBS washes were done before blocking with 1% BSA. For the immunolabeling, the microtubule cells were stained with Alexa Fluor 488 phallodin 1:500 for 30 minutes at 37°C. Cells were then briefly washed with PBS for imaging.

2. Structured illumination microscope

SIM imaging in Fig. 4 was performed on the Nikon N-SIM system equipped with 488 nm (70 mW), 561 nm (70 mW), and 647 nm (125 mW) laser lines, two Back-thinned CCD cameras (ORCA-Fusion BT) and a 100×, NA 1.49 objective (Nikon, CFI Apochromat TIRF 100XC Oil). For tubulin microtubules, we used 647 nm laser with intensity of 20.7 mW and 200 ms exposure time to acquire SIM images, and the corresponding emission light wavelength was 690 nm. For actin microfilaments, we used 488 nm laser with intensity of 22.9 mW and 200 ms exposure time to acquire SIM images, and the corresponding emission light wavelength was 605 nm. Different SIM images were acquired by moving the observation field. The SIM images in Figs. 5 and 6 were obtained by a home-built SIM system [33].

3. DRE-SIM processing

All the processing of simulated and experimental images was performed on a server with Intel i9-10920X, 192 GB of RAM and NVIDIA GeForce RTX3090. For the processing of SIM images with size 512 × 512, the computation time is about 30 seconds for about 700 iterations. The code of DRE-SIM is available at [41].

4. Estimation of the equivalent OTF of SIM system

The equivalent PSF in SIM is first estimated according to the parameters of the SIM system, which can be expressed as

(6)$${H_\textrm{e}} = {\left[ {\frac{{2{J_1}(\frac{{2\pi {m_f}NA \cdot r}}{\lambda })}}{{\frac{{2\pi {m_f}NA \cdot r}}{\lambda }}}} \right]^2}, $$

where J₁ is the first order Bessel function, λ is the fluorescence wavelength, NA is the numerical aperture of the objective lens, m_f is the modulation factor used to modulate the equivalent NA after SIM reconstruction, m_f = (1 + k_f /k_c), k_f is the structured illumination frequency, k_c is cut-off frequency for acquisition, and r is the radial space coordinate of object surface. The equivalent OTF of SIM system is then acquired by the Fourier transform of equivalent PSF.

5. Resolution enhancement in SD-SIM and STED by DRE-SIM

To demonstrate the widespread applicability of DRE-SIM in various super-resolution microscopy modalities, the images of tubulin microtubules in COS-7 cells acquired by SD-SIM [15] and STED are processed by DRE-SIM, as shown in Fig. 7. Obviously, the resolution enhancement for the two images can also be achieved.

 

Fig. 7. Resolution enhancement of SD-SIM and STED by DRE-SIM. (a-b) The SD-SIM image of tubulins before and after processing of DRE-SIM. (c-d) The STED image of tubulins before and after processing of DRE-SIM.

Download Full Size | PDF

Funding

National Natural Science Foundation of China (11727810, 12034008, 12074121, 12274129, 12274139, 62105101, 62175066, 91850202, 92150301); Science and Technology Commission of Shanghai Municipality (20ZR1417100, 21JM0010700, 21XD1400900).

Acknowledgment

We thank Prof. Kebin Shi for fruitful discussions.

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are available in Refs. [33] and [41].

References

1. R. Heintzmann and T. Huser, “Super-resolution structured illumination microscopy,” Chem. Rev. 117(23), 13890–13908 (2017). [CrossRef]  

2. D. Li, L. Shao, B. Chen, X. Zhang, M. Zhang, B. Moses, D. E. Milkie, J. R. Beach, J. A. Hammer, M. Pasham, T. Kirchhausen, M. A. Baird, M. W. Davidson, P. Xu, and E. Betzig, “Extended-resolution structured illumination imaging of endocytic and cytoskeletal dynamics,” Science 349(6251), aab3500 (2015). [CrossRef]  

3. X. Huang, J. Fan, L. Li, H. Liu, R. Wu, Y. Wu, L. Wei, H. Mao, A. Lal, P. Xi, L. Tang, Y. Zhang, Y. Liu, S. Tan, and L. Chen, “Fast, long-term, super-resolution imaging with Hessian structured illumination microscopy,” Nat. Biotechnol. 36(5), 451–459 (2018). [CrossRef]  

4. Q. Chen, X. Shao, M. Hao, H. Fang, R. Guan, Z. Tian, M. Li, C. Wang, L. Ji, H. Chao, J. Guan, and J. Diao, “Quantitative analysis of interactive behavior of mitochondria and lysosomes using structured illumination microscopy,” Biomaterials 250, 120059 (2020). [CrossRef]  

5. A. Markwirth, M. Lachetta, V. Mönkemöller, R. Heintzmann, W. Hübner, T. Huser, and M. Müller, “Video-rate multi-color structured illumination microscopy with simultaneous real-time reconstruction,” Nat. Commun. 10(1), 4315 (2019). [CrossRef]  

6. A. V. Weigel, C. Chang, G. Shtengel, C. S. Xu, D. P. Hoffman, M. Freeman, N. Iyer, J. Aaron, S. Khuon, J. Bogovic, W. Qiu, H. F. Hess, and J. Lippincott-Schwartz, “ER-to-Golgi protein delivery through an interwoven, tubular network extending from ER,” Cell 184(9), 2412–2429 (2021). [CrossRef]  

7. L. Schermelleh, P. M. Carlton, S. Haase, L. Shao, L. Winoto, P. Kner, B. Burke, M. C. Cardoso, D. A. Agard, M. G. L. Gustafsson, H. Leonhardt, and J. W. Sedat, “Subdiffraction multicolor imaging of the nuclear periphery with 3D structured illumination microscopy,” Science 320(5881), 1332–1336 (2008). [CrossRef]  

8. J. Fitzgibbon, K. Bell, E. King, and K. Oparka, “Super-resolution imaging of plasmodesmata using three-dimensional structured illumination microscopy,” Plant Physiol. 153(4), 1453–1463 (2010). [CrossRef]  

9. M. J. Rust, M. Bates, and X. Zhuang, “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM),” Nat. Methods 3(10), 793–796 (2006). [CrossRef]  

10. S. Manley, J. M. Gillette, G. H. Patterson, H. Shroff, H. F. Hess, E. Betzig, and J. Lippincott-Schwartz, “High-density mapping of single-molecule trajectories with photoactivated localization microscopy,” Nat. Methods 5(2), 155–157 (2008). [CrossRef]  

11. S. W. Hell and J. Wichmann, “Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy,” Opt. Lett. 19(11), 780–782 (1994). [CrossRef]  

12. H. Blom and J. Widengren, “Stimulated emission depletion microscopy,” Chem. Rev. 117(11), 7377–7427 (2017). [CrossRef]  

13. M. G. Gustafsson, “Nonlinear structured-illumination microscopy: wide-field fluorescence imaging with theoretically unlimited resolution,” Proc. Natl. Acad. Sci. U.S.A. 102(37), 13081–13086 (2005). [CrossRef]  

14. E. H. Rego, L. Shao, J. J. Macklin, L. Winoto, G. A. Johansson, N. Kamps-Hughes, M. W. Davidson, and M. G. Gustafsson, “Nonlinear structured-illumination microscopy with a photoswitchable protein reveals cellular structures at 50-nm resolution,” Proc. Natl. Acad. Sci. U.S.A. 109(3), E135–E143 (2012). [CrossRef]  

15. W. Zhao, S. Zhao, L. Li, X. Huang, S. Xing, Y. Zhang, G. Qiu, Z. Han, Y. Shang, D. Sun, C. Shan, R. Wu, L. Gu, S. Zhang, R. Chen, J. Xiao, Y. Mo, J. Wang, W. Ji, X. Chen, B. Ding, Y. Liu, H. Mao, B. Song, J. Tan, J. Liu, H. Li, and L. Chen, “Sparse deconvolution improves the resolution of live-cell super-resolution fluorescence microscopy,” Nat. Biotechnol. 40(4), 606–617 (2022). [CrossRef]  

16. D. Ulyanov, A. Vedaldi, and V. Lempitsky, “Deep image prior,” in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, (IEEE, 2018), pp. 9446–9454

17. M. G. Gustafsson, “Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy,” J. Microsc. (Oxford, U. K.) 198(2), 82–87 (2000). [CrossRef]  

18. A. Lal, C. Shan, and P. Xi, “Structured illumination microscopy image reconstruction algorithm,” IEEE J. Select. Topics Quantum Electron. 22(4), 50–63 (2016). [CrossRef]  

19. J. Sibarita, “Deconvolution microscopy,” in Microscopy Techniques (Springer, Berlin, 2005), pp. 201–243.

20. Y. Jo, S. Y. Chun, and J. Choi, “Rethinking deep image prior for denoising,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (IEEE/CVF, 2021), pp. 5087–5096.

21. H. Zhang, L. Mai, N. Xu, Z. Wang, J. Collomosse, and H. Jin, “An internal learning approach to video inpainting,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (IEEE/CVF, 2019), pp. 2720–2729.

22. S. Han, T. B. Lee, and Y. S. Heo, “Deep image prior for super resolution of noisy image,” Electronics 10(16), 2014 (2021). [CrossRef]  

23. F. Wang, Y. Bian, H. Wang, M. Lyu, G. Pedrini, W. Osten, G. Barbastathis, and G. Situ, “Phase imaging with an untrained neural network,” Light: Sci. Appl. 9(1), 77 (2020). [CrossRef]  

24. H. Li, X. Chen, Z. Chi, C. Mann, and A. Razi, “Deep DIH: single-shot digital in-line holography reconstruction by deep learning,” IEEE Access 8, 202648–202659 (2020). [CrossRef]  

25. C. Bai, T. Peng, J. Min, R. Li, Y. Zhou, and B. Yao, “Dual-wavelength in-line digital holography with untrained deep neural networks,” Photonics Res. 9(12), 2501–2510 (2021). [CrossRef]  

26. S. Liu, X. Meng, Y. Yin, H. Wu, and W. Jiang, “Computational ghost imaging based on an untrained neural network,” Opt. Laser. Eng. 147, 106744 (2021). [CrossRef]  

27. O. Ronneberger, P. Fischer, and T. Brox, “U-net: convolutional networks for biomedical image segmentation,” in International Conference on Medical Image Computing and Computer-assisted Intervention (Springer, 2015), pp. 234–241.

28. T. Falk, D. Mai, R. Bensch, Ö. Çiçek, A. Abdulkadir, Y. Marrakchi, A. Böhm, J. Deubner, Z. Jäckel, and K. Seiwald, “U-Net: deep learning for cell counting, detection, and morphometry,” Nat. Methods 16(1), 67–70 (2019). [CrossRef]  

29. G. Wen, S. Li, L. Wang, X. Chen, Z. Sun, Y. Liang, X. Jin, Y. Xing, Y. Jiu, and Y. Tang, “High-fidelity structured illumination microscopy by point-spread-function engineering,” Light: Sci. Appl. 10(1), 70 (2021). [CrossRef]  

30. E. A. Ingerman, R. A. London, R. Heintzmann, and M. G. L. Gustafsson, “Signal, noise and resolution in linear and nonlinear structured-illumination microscopy,” Light: Sci. Appl. 273(1), 3–25 (2019). [CrossRef]  

31. N. Banterle, K. H. Bui, E. A. Lemke, and M. Beck, “Fourier ring correlation as a resolution criterion for super-resolution microscopy,” J. Struct. Biol. 183(3), 363–367 (2013). [CrossRef]  

32. S. Koho, G. Tortarolo, M. Castello, T. Deguchi, A. Diaspro, and G. Vicidomini, “Fourier ring correlation simplifies image restoration in fluorescence microscopy,” Nat. Commun. 10(1), 3103 (2019). [CrossRef]  

33. C. Qiao, D. Li, Y. Guo, C. Liu, T. Jiang, Q. Dai, and D. Li, “Evaluation and development of deep neural networks for image super-resolution in optical microscopy,” Nat. Methods 18(2), 194–202 (2021). [CrossRef]  

34. W. H. Richardson, “Bayesian-based iterative method of image restoration,” J. Opt. Soc. Am. 62(1), 55–59 (1972). [CrossRef]  

35. L. B. Lucy, “An iterative technique for the rectification of observed distributions,” Astron. J. 79, 745–754 (1974). [CrossRef]  

36. N. Dey, L. Blanc Feraud, C. Zimmer, P. Roux, Z. Kam, J. C. Olivo Marin, and J. Zerubia, “Richardson-Lucy algorithm with total variation regularization for 3D confocal microscope deconvolution,” Microsc. Res. Tech. 69(4), 260–266 (2006). [CrossRef]  

37. O. Schulz, C. Pieper, M. Clever, J. Pfaff, A. Ruhlandt, R. H. Kehlenbach, F. S. Wouters, J. Großhans, G. Bunt, and J. Enderlein, “Resolution doubling in fluorescence microscopy with confocal spinning-disk image scanning microscopy,” Proc. Natl. Acad. Sci. U. S. A. 110(52), 21000–21005 (2013). [CrossRef]  

38. W. Zong, R. Wu, M. Li, Y. Hu, Y. Li, J. Li, H. Rong, H. Wu, Y. Xu, Y. Lu, H. Jia, M. Fan, Z. Zhou, Y. Zhang, A. Wang, L. Chen, and H. Cheng, “Fast high-resolution miniature two-photon microscopy for brain imaging in freely behaving mice,” Nat. Methods 14(7), 713–719 (2017). [CrossRef]  

39. D. Sun, X. Fan, Y. Shi, H. Zhang, Z. Huang, B. Cheng, Q. Tang, W. Li, Y. Zhu, J. Bai, W. Liu, Y. Li, X. Wang, X. Lei, and X. Chen, “Click-ExM enables expansion microscopy for all biomolecules,” Nat. Methods 18(1), 107–113 (2021). [CrossRef]  

40. L. Shao, P. Kner, E. H. Rego, and M. G. L. Gustafsson, “Super-resolution 3D microscopy of live whole cells using structured illumination,” Nat. Methods 8(12), 1044–1046 (2011). [CrossRef]  

41. Y. He, Y. Yao, Y. He, Z. Huang, D. Qi, T. Jia, Z. Sun, and S. Zhang, “Code for DRE-SIM,” Github, (2022), https://github.com/evenhe/DRE_SIM