Abstract
In lensfree on-chip microscopy, the iterative phase retrieval with defocused images easily enables a high-resolution and whole field reconstruction. However, on the reconstruction of the dense sample, conventional methods suffer from the stagnation problem and noise affection under two intensity measurements, which gives rise to a remarkable loss of the image contrast and resolution. Here we propose a novel dual-plane phase retrieval algorithm to perform a stable and versatile lensless reconstruction. A weighted feedback constraint was utilized to speed up the convergence. Then, a gradient descent minimization based on total variation metric was proposed to suppress the noise affection. With these two object constraints, a smoothed but resolution-preserving result can be achieved. Numerical simulations of Gaussian and Poisson noise were given to prove the noise-robustness of our method. The experiments of USAF resolution target, H&E stained pathological slide, and label-free microglia cell demonstrated the superior performance of our approach compared to other state-of-the-art methods.
© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
In lensfree holographic microscopy, quasi-monochromatic light wave passes through sample and then captured by digital camera. However, the current sensor chip, such as CCD or CMOS, merely measures an averaged intensity of light field, and the corresponding phase information is lost. If the phase information of diffraction pattern is retrieved, the wavefronts at any planes can be recovered by using numerical diffraction propagation. Based on this principle, lensfree on-chip microscope with iterative phase retrieval algorithm achieves a non-invasive, wide field-of-view (FOV) and high-resolution image reconstruction, which has been exploited in the application of living cell observation [1,2], optical sectioning [3], point-of-care device [4], and whole pathological slide imaging [5,6].
To recover the sample image, different iterative phase retrieval algorithms were investigated in lensfree imaging, including single-shot [2,7–10] and multi-image phase retrieval [11–28]. The former requires a prior acknowledge of the object scene, such as support region or background image (captured after object removal), to escape from the local minimum in the iterative procedure. These single-shot methods have the ability of dynamic imaging but have to satisfy the sparse sample assumption. The latter can perform a high fidelity and stable image reconstruction at the cost of measuring many observation planes. Among multi-image phase retrieval methods, axial scanning strategy [17–28] captures a series of defocused patterns and utilizes multi-distance (or height) phase retrieval (MDPR) for image recovery, which possesses the advantage of simple operation and high stability. However, the sequential measurement of MDPR decreases the acquisition rate for lensfree system. Also, the retrieved accuracy of MDPR is highly dependent on the number of observation planes. For example, as shown in [18], the visually acceptable reconstruction of dense sample requires at least three intensity measurements. If one can achieve the reconstruction with two-shot measurements, the weakness of MDPR on the acquisition rate can be overcome by using a dual-camera lensfree system. Till now, much approaches composed of modified Gerchberg-Saxton (GS) algorithm [24–26] and non-iterative method [27,28] have been proposed to decrease the measurement times to 2, but they merely accomplished the reconstruction of binary or isolated object, and dense sample reconstruction, such as pathological slide and label-free living cell, is not considered.
Compared to previously reported methods, we propose a dual-plane phase retrieval algorithm to achieve a stable and versatile lensless reconstruction. Two object constraints were designed to solve the stagnation problem and noise affection. Firstly, a weighted feedback term was used to speed up the convergence. Secondly, a gradient descent minimization based on total variation (TV) metric was proposed to remove noisy information so that a smoothed but resolution-preserving result can be obtained. Numerical simulations with the addition of Gaussian and Poisson noise proved the noise robustness of our method. Experimental results of USAF target, pathological slide, and label-free microglia cell were given to validate the superior performance compared to other state-of-the-art methods.
2. Theory
2.1 The overview of MDPR
The geometrical configuration of lensfree on-chip system was shown in Fig. 1(a). A fiber laser directly incidents on the sample and recorded by a tightly-placed CMOS sensor chip, which contributes a unit magnification for multi-distance observation planes. Thus, the magnification effect of the spherical wave is ignored, and angular spectrum diffraction model with a pixel-to-pixel projection can be used for numerical reconstruction. If the complex amplitude of the object is O and the sample-to-sensor distances are denoted as Zn, the n-th diffracted image on the sensor is expressed as
where ${A_{Z_{n}}}({\cdot} )$ is the forward propagation operator over the distance of Zn and $\varepsilon$ is the additive noise function. In the following context, the operator ${A_{Z_{n}}}^{- 1}$ denotes the inverse diffraction propagation, which can be taken by inputting $- {Z_n}$ into angular spectrum kernel. The goal of MDPR is to retrieve O with the diffracted intensity images ${I_n}$.MDPR is composed of three types: successive mode, parallel mode, and regularization mode. Assuming that the k-th object estimation is ${O^k}$, the successive mode [17–19], also called as single-beam multiple-intensity phase reconstruction (SBMIR) algorithm, is generalized as follows.
- (2) The combined complex-valued image is successively propagated to the next plane until all observation planes are reached, and this process can be denoted as$$C_{n + 1}^k = \sqrt {{I_{n + 1}}} \frac{{{A_{{Z_{n + 1}} - {Z_n}}}(C_n^k)}}{{|{{A_{{Z_{n + 1}} - {Z_n}}}(C_n^k)} |}}, n \in [1,N - 1],$$where the subscript ${Z_{n + 1}} - {Z_n}$ denotes the distance interval between the (n + 1)-th and n-th observation plane.
2.2 Our proposal
For conventional MDPR methods, the bad performance under N = 2 is attributed to two aspects: stagnation problem and noise affection. To solve the two issues, we design two object constraints on APR algorithm. Firstly, a weighted feedback constraint is used to speed up the convergence. In our previous work [22], the weighted feedback constraint was proved to perform a threefold convergence acceleration, but it easily magnifies the background noise for fewer measurements. Thus, a smoothness operation based on TV minimization is designed to constrain the background noise.
The L1-norm TV metric of a complex-valued image S is denoted as
With these two object constraints, the flowchart of our method is shown in Fig. 1(b) and its detail is generalized as: (1) the k-th object estimation ${O^k}$, after the forward propagation and amplitude replacement, is transferred to a set of object guesses as follows
3. Numerical simulation
Numerical simulations with N = 2 are given to compare our method with state-of-the-art algorithms: SBMIR [17–19,24,25], APR [20,21], APRDF [22] and AL [23]. The simulations with Gaussian and Poisson noise are provided in Figs. 3–5. The image of Fig. 2(a) is used as the ground truth. The simulated parameters are listed as: (1) the ground truth image has 600×600 pixels with a pixel size of 1.34µm; (2) the diffraction distances are selected at Z1=0.75mm and Z2=1.5mm; (3) the wavelength is 532nm; (4) the parameters a, b and t in our method are set as 0.7, 0.5 and 0.05, which are used for all simulations and experiments. The normalized correlation coefficient (NCC) is selected to indicate the retrieved accuracy.
After 100 iterations, the NCC curves of different methods are plotted in Figs. 3(a) and 3(b) with the Gaussian noise variance of 0.05 and 0.08. As shown in Figs. 3(a) and 3(b), the curves of conventional methods are divergent, which implies that the conventional MDPR methods with N = 2 are susceptible to the noise turbulence. As a comparison, our method reaches to be convergent merely using ∼20 iterations. The noise susceptibility of MDPR is obviously improved by our method. The corresponding retrieved amplitudes of different methods are shown in Figs. 3(c)–3(g), and the corresponding running time is 26.7s, 126.6s, 28.5s, 29.1s, and 79.6s. It is noted that our method has the best imaging contrast in Fig. 3(g), and the retrieved images of other methods are contaminated by noise in Figs. 3(c)–3(f). A similar simulation with the addition of Poisson noise is illustrated in Fig. 4, where the photon count level is set as $\textrm{1}{\textrm{0}^7}$ photons/pixel. Figure 4(a) is the convergence curve and Figs. 4(b)–4(f) are retrieved by SBMIR, AL, APR, APRDF, and our method with 100 iterations. It can be observed that our method still holds the best resistance for Poisson noise, and other methods are incapable of stable recovery. Especially for the AL method, its NCC curve and retrieved image show a terrible imaging quality, because the noise modeling of AL is based on the assumption of Gaussian distribution rather than Poisson distribution.
In MDPR, the distance selection strategy affects the convergence speed. To further show this issue, the results with different intervals between two observation planes are reconstructed, and the corresponding NCC values are pictured in Fig. 5. Here Figs. 5(a) and 5(b) represent the results with the addition of Gaussian and Poisson noise, respectively. As shown in Fig. 5, the NCC values of conventional methods vary at different intervals. Without this limitation, our method has the highest but nearly-identical NCC values for each interval. Numerical simulations prove the noise-robustness and convergence stability of our method.
4. Experimental results
4.1 USAF resolution target
In the experiment, a fiber laser with a wavelength of 532nm is chosen for illumination. A bare CMOS sensor chip (Sony, IMX206, 1.34 µm×1.34 µm) is held by a linear translation stage (M-403, resolution 0.012 µm, Physik Instrumente Inc.) for the multi-plane measurements. Three types of samples, including USAF resolution target, H&E stained pathological slide, and label-free microglia cell, are used to validate the effectiveness. In the real scene, the tilt illumination of the incident plane wave usually triggers an aligning problem for multi-distance data, which is difficult to be offset by mechanical adjustment. Thus the image registration algorithm [34] is applied to align all axial patterns. The reconstruction of positive USAF resolution target (Ready Optics) is presented in Fig. 6, where the two sample-to-sensor distances are quantified as 0.745mm and 0.975mm by the auto-focusing algorithm [30,31]. After 40 iterations, the retrieved amplitudes of the resolution target are displayed in Figs. 6(a1)–6(e1). The high-resolution regions labeled with green and blue boxes are displayed in Figs. 6(a2)–6(e2) and Figs. 6(a3)–6(e3). For a fair comparison, the regularized parameter of AL is finely tuned, and $\mu = 0.4$ is employed for the reconstruction. Even so, AL cannot accomplish the recovery with N = 2, and its retrieved images are blurred in Figs. 6(b1)–6(b3). In Figs. 6(a), 6(c), and 6(d), although the recovery of low-resolution parts (Groups 4, 5 and 6) can be performed by SBMIR, APR, and APRDF, their reconstructed Groups 7 and 8 are filled with noisy background, which makes it difficult for visual observation. Different from the conventional algorithms, our method outputs the contrast-enhanced images in Figs. 6(e1)–6(e3), where the artifact is remarkably removed, and a half-pitch lateral resolution of 1.096µm is obtained.
To show the data-efficiency, we provide the reconstructed results of our method with N = 2 and APR with N = 11 in Fig. 7. As APR is run using 11 intensity images with 40 iterations, the background noise in Figs. 6(c2) and 6(c3) is wiped out, and the imaging contrast is strengthed in Figs. 7(a1) and 7(a2). However, the comparison of Fig. 6(c3) and Fig. 7(a2) indicates that the imaging resolution of APR is slightly undermined from N = 2 to N = 11, which originates from the subpixel drift accumulation of the aligned multi-distance data. It is deduced from the above results that increasing observation planes actually functions well in the suppression of background noise, but the resulting aligning problems could lead to the resolution loss. After similar iterations, the results of our method with N = 2 are displayed in Figs. 7(b1) and 7(b2). Our method offers a comparable imaging quality by contrast with APR with 11 measurements, which demonstrates the data-efficiency of our method. To quantitatively show the improvement, the plotlines between blue-dash and red lines of Figs. 7(a2) and 7(b2) are pictured in Fig. 7(c). It can be seen that the image contrast of our method is superior to that of APR for all elements of group 8. Also, the element 6 of group 8 can be resolved by our method. This resolution-holding property benefits from the edge-preserving effect of TV minimization. The experiment of the resolution target proves that our method accomplishes a data-efficient reconstruction without the loss of resolution.
4.2 Pathological slide
The experiments of H&E stained rat lung and intestine are given in Fig. 8 and Fig. 9. With the use of two intensity images, the full-FOV recovery can be achieved by our method. Due to the layout limitation, the reconstructed full-FOV image is not exhibited. The selected regions with a size of 134µm×134µm are shown in Figs. 8(a1)–8(e1) and Figs. 8(a2)–8(e2) for lung tissue. The intestine images are retrieved in Figs. 8(a3)–8(e3) and Figs. 8(a4)–8(e4). Similar to Fig. 6, the reconstruction result using two intensity patterns is still impossible for AL. The retrieved images of SBMIR, APR, and APRDF are wrapped with noise artifacts. Especially for the intestine tissue, the detailed information, such as texture and cell nucleus, cannot be clearly distinguished from the noisy background for the conventional methods. In contrast, the lung and intestine pathological slides are both clearly recovered by our method in Figs. 8(e1)–8(e4).
As a comparison, the typical single-shot phase retrieval methods, based on support constraint [7] and compressive sensing [10], are used for the reconstruction of the intestine. It should be pointed out that these two single-shot methods also need to record two intensity images. The two images are composed of the diffraction pattern ${I_1}$ in Z1 and background image ${R_1}$ that recorded by removing the sample. As done in [7], a binary support mask is obtained by the image segmentation and displayed in the red box of Fig. 9(a1). Then, a modified GS algorithm is employed for phase retrieval, and is expressed as
Repetitively calculating Eqs. (13) and (14), the support-based method reconstructs the full-FOV intestine image in Fig. 9(a1), and the selected regions from the blue and green boxes are zoomed in Figs. 9(a2)–9(a3). Intestine tissue includes rich detail, and the support mask cannot be precisely sketched, which leads to a bad convergence. Compressive sensing approach [10] constructs the lensless deconvolution as an optimization problem and combines an iterative shrinkage/thresholding solver with a total variation penalty to output an artifact-free sample image. The retrieved images of compressive sensing approach are displayed in Figs. 9(b1)–9(b3), where the twin-image and border artifact are eliminated, but the imaging resolution is damaged. Compared to the single-shot methods, the retrieved images of our method in Figs. 9(c1)–9(c3) show the remarkable enhancement of the image contrast and resolution.
4.3 Phase imaging
The experiments of the resolution target and pathological slide merely focus on the amplitude reconstruction. To further prove the capability of phase recovery, we provide the experiment of the label-free microglia cell line (BV-2) in Fig. 10 and Fig. 11. The used cells were purchased from Cell Culture Center (Institute of Basic Medicine, Chinese Academy of Medical Sciences). The cells were uniformly seeded in 24-well plates, and placed in a 37°C cell culture incubator. Cell growth was terminated when the cell density reached 40% of the bottom wall area. With the preparation, we fixed the living cells on the clean glass slide for convenient usage and observation. Merely capturing two defocused images, the full-FOV phase image of microglia cells is retrieved by our method in Fig. 10, where the FOV is quantified as 28.6 mm2. The yellow, black and red regions in Fig. 10 are digitally cropped from the retrieved images, and shown in Fig. 11 for different methods. As a comparison, the phase images in Fig. 11 were processed by phase unwrapping algorithm. In Figs. 11(b1)–11(b3), the AL is incompetent for the phase reconstruction under N = 2. The retrieved images of SBMIR, APR and APRDF suffer from the noise affection in Figs. 11(a), 11(c) and 11(d), in which the cell phase images are covered with the nonuniform and noisy background. Different from the conventional methods, our method effectively eliminates the noise artifact and accomplishes a contrast-enhanced phase reconstruction in Figs. 11(e1)–11(e3).
5. Conclusions
We proposed a dual-plane phase retrieval to achieve a stable and versatile image reconstruction for lensfree on-chip microscopy. In our method, two object constraints, including weighted feedback and TV minimization, were proposed to eliminate the stagnation convergence and noisy background under two-shot measurements. In numerical simulations, we analyzed the robustness effect of our method on the noise type, noise level, and distance selection. In the experiment, the results of the positive USAF resolution target proved that our method is data-efficient and resolution-preserving in comparison with SBMIR, AL, APR, and APRDF. Also, our method outperformed the conventional methods on the reconstruction of the pathological slide and label-free microglia cell line (BV-2). Without the extra prior knowledge of object scene, our method can achieve a high fidelity reconstruction by contrast with the support-based and compressive sensing method.
This work effectively decreases the redundancy of axial scanning measurement, and its advantages can be generalized in the following. Firstly, our method could achieve a data-reduction for lensfree pixel super-resolution (PSR) imaging. As shown in [5,6], the PSR needs a 3D-stack (6×6×8) data to accomplish a twin-image-free and subpixel reconstruction. Equipped with our method, the data requirement of PSR will be obviously decreased. Secondly, the acquisition rate for axial scanning system is increased to provide a promising dynamic living cell imaging tool for lensfree dual-sensor system. Thirdly, the proposed method could be applied for other lensfree systems, such as wavelength scanning [12–13] and multi-angle illumination [14], to perform a data-efficient reconstruction.
Funding
National Natural Science Foundation of China (61575055, 61575053, 11874132, 61975044).
Disclosures
The authors declare no conflicts of interest.
References
1. L. Herve, O. Cioni, P. Blandin, F. Navarro, M. Menneteau, T. Bordy, S. Morales, and C. Allier, “Multispectral total-variation reconstruction applied to lens-free microscopy,” Biomed. Opt. Express 9(11), 5828–5836 (2018). [CrossRef]
2. D. Ryu, Z. Wang, K. He, G. Zheng, R. Horstmeyer, and O. Cossairt, “Subsampled phase retrieval for temporal resolution enhancement in lensless on-chip holographic video,” Biomed. Opt. Express 8(3), 1981–1995 (2017). [CrossRef]
3. Y. Zhang, Y. Shin, K. Sung, S. Yang, H. Chen, H. Wang, D. Teng, Y. Rivenson, R. P. Kulkarni, and A. Ozcan, “3D imaging of optically cleared tissue using a simplified CLARITY method and on-chip microscopy,” Sci. Adv. 3(8), e1700553 (2017). [CrossRef]
4. D. Tseng, O. Mudanyali, C. Oztoprak, S. O. Isikman, I. Sencan, O. Yagliderea, and A. Ozcan, “Lensfree microscopy on a cellphone,” Lab Chip 10(14), 1787–1792 (2010). [CrossRef]
5. A. Greenbaum, W. Luo, T. Su, Z. Göröcs, L. Xue, S. O. Isikman, A. F. Coskun, O. Mudanyali, and A. Ozcan, “Imaging without lenses: achievements and remaining challenges of wide-field on-chip microscopy,” Nat. Methods 9(9), 889–895 (2012). [CrossRef]
6. A. Greenbaum, Y. Zhang, A. Feizi, P. Chung, W. Luo, S. R. Kandukuri, and A. Ozcan, “Wide-field computational imaging of pathology slides using lensfree on-chip microscopy,” Sci. Transl. Med. 6(267), 267ra175 (2014). [CrossRef]
7. O. Mudanyali, D. Tseng, C. Oh, S. O. Isikman, I. Sencan, W. Bishara, C. Oztoprak, S. Seo, B. Khademhosseini, and A. Ozcan, “Compact, light-weight and cost-effective microscope based on lensless incoherent holography for telemedicine applications,” Lab Chip 10(11), 1417–1428 (2010). [CrossRef]
8. S. M. F. Raupach, “Cascaded adaptive-mask algorithm for twin-image removal and its application to digital holograms of ice crystals,” Appl. Opt. 48(2), 287–301 (2009). [CrossRef]
9. C. Gaur, B. Mohan, and K. Khare, “Sparsity-assisted solution to the twin image problem in phase retrieval,” J. Opt. Soc. Am. A 32(11), 1922–1927 (2015). [CrossRef]
10. W. Zhang, L. Cao, D. J. Brady, H. Zhang, J. Cang, H. Zhang, and G. Jin, “Twin-image-free holography: a compressive sensing approach,” Phys. Rev. Lett. 121(9), 093902 (2018). [CrossRef]
11. D. W. E. Noom, K. S. E. Eikema, and S. Witte, “Lensless phase contrast microscopy based on multiwavelength Fresnel diffraction,” Opt. Lett. 39(2), 193–196 (2014). [CrossRef]
12. S. Witte, V. T. Tenner, D. W. E. Noom, and K. S. E. Eikema, “Lensless diffractive imaging with ultra-broadband table-top sources: from infrared to extreme-ultraviolet wavelengths,” Light: Sci. Appl. 3(3), e163 (2014). [CrossRef]
13. W. Luo, Y. Zhang, A. Feizi, Z. Gorocs, and A. Ozcan, “Pixel super-resolution using wavelength scanning,” Light: Sci. Appl. 5(4), e16060 (2016). [CrossRef]
14. W. Luo, A. Greenbaum, Y. Zhang, and A. Ozcan, “Synthetic aperture-based on-chip microscopy,” Light: Sci. Appl. 4(3), e261 (2015). [CrossRef]
15. P. Li and A. Maiden, “Lensless LED matrix ptychographic microscope: problems and solutions,” Appl. Opt. 57(8), 1800–1806 (2018). [CrossRef]
16. D. Claus and J. M. Rodenburg, “Diffraction-limited superresolution ptychography in the Rayleigh-Sommerfeld regime,” J. Opt. Soc. Am. A 36(2), A12–A19 (2019). [CrossRef]
17. G. Pedrini, W. Osten, and Y. Zhang, “Wave-front reconstruction from a sequence of interferograms recorded at different planes,” Opt. Lett. 30(8), 833–835 (2005). [CrossRef]
18. A. Greenbaum and A. Ozcan, “Maskless imaging of dense samples using pixel super-resolution based multi-height lensfree on-chip microscopy,” Opt. Express 20(3), 3129–3143 (2012). [CrossRef]
19. L. J. Allen and M. P. Oxley, “Phase retrieval from series of images obtained by defocus variation,” Opt. Commun. 199(1-4), 65–75 (2001). [CrossRef]
20. L. J. Allen, W. McBride, N. L. O. Leary, and M. P. Oxley, “Exit wave reconstruction at atomic resolution,” Ultramicroscopy 100(1-2), 91–104 (2004). [CrossRef]
21. Z. Liu, C. Guo, J. Tan, Q. Wu, L. Pan, and S. Liu, “Iterative phase-amplitude retrieval with multiple intensity images at output plane of gyrator transforms,” J. Opt. 17(2), 025701 (2015). [CrossRef]
22. C. Guo, C. Shen, Q. Li, J. Tan, S. Liu, X. Kan, and Z. Liu, “A fast-converging iterative method based on weighted feedback for multi-distance phase retrieval,” Sci. Rep. 8(1), 6436 (2018). [CrossRef]
23. A. Migukin, V. Katkovnik, and J. Astola, “Wave field reconstruction from multiple plane intensity-only data: augmented lagrangian algorithm,” J. Opt. Soc. Am. A 28(6), 993–1002 (2011). [CrossRef]
24. Y. Zhang, G. Pedrini, W. Osten, and H. J. Tiziani, “Whole optical wave field reconstruction from double or multi in-line holograms by phase retrieval algorithm,” Opt. Express 11(24), 3234–3241 (2003). [CrossRef]
25. Z. Li, L. Li, Y. Qin, G. Li, D. Wang, and X. Zhou, “Resolution and quality enhancement in terahertz in-line holography by sub-pixel sampling with double-distance reconstruction,” Opt. Express 24(18), 21134–21146 (2016). [CrossRef]
26. C. Guo, Y. Zhao, J. Tan, S. Liu, and Z. Liu, “Multi-distance phase retrieval with a weighted shrink-wrap constraint,” Opt. Lasers Eng. 113, 1–5 (2019). [CrossRef]
27. Y. Zhang, G. Pedrini, W. Osten, and H. J. Tiziani, “Reconstruction of in-line digital holograms from two intensity measurements,” Opt. Lett. 29(15), 1787–1789 (2004). [CrossRef]
28. B. Das and C. S. Yelleswarapu, “Dual plane in-line digital holographic microscopy,” Opt. Lett. 35(20), 3426–3428 (2010). [CrossRef]
29. R. W. Gerchberg and W. O. Saxton, “A practical algorithm for the determination of phase from image and diffraction plane pictures,” Optik 35, 237–246 (1972).
30. Y. S. Choi and S. J. Lee, “Three-dimensional volumetric measurement of red blood cell motion using digital holographic microscopy,” Appl. Opt. 48(16), 2983–2990 (2009). [CrossRef]
31. E. Krotkov, “Focusing,” Int. J. Comp. Vis. 1(3), 223–237 (1988). [CrossRef]
32. I. M. Gelfand and V. A. Fomin, translated and edited by R. A. Silverman, Calculus of Variations (Dover, 2000).
33. D. H. Brandwood, “A complex gradient operator and its application in adaptive array theory,” IEE Proc., Part H: Microwaves, Opt. Antennas 130(1), 11–16 (1983). [CrossRef]
34. M. Guizar-Sicairos, S. T. Thurman, and J. R. Fienup, “Efficient subpixel image alignment algorithms,” Opt. Lett. 33(2), 156–158 (2008). [CrossRef]