We propose a method for acquiring color images with a lensless in-line holographic microscope (LIHM) using sunlight illumination, which is suitable for weakly scattered amplitude objects. In the color LIHM, the sample is illuminated by sunlight, and the in-line hologram is recorded by a color CMOS imaging sensor located behind the sample. For weakly-scattered amplitude object, we show that the hologram can be described as the convolution of object transmission and a point spread function (PSF) that depends on the spectral distribution of the light. The captured color hologram is first separated into the red, green, and blue components, and then the sub-holograms of each color are used to reconstruct the corresponding color components of the sample by a deconvolution process. We proved that the deconvolution process was able to improve the imaging resolution, which was deteriorated because of insufficient temporal coherence of the light. The resolution enhancement capability of our color LIHM was demonstrated by numerical simulations and imaging experiments with the U. S. Air Force target as the sample. We also imaged a stained root of the herb Saposhnikovia divaricata to further demonstrate the capability of our method for color imaging applications. Our proposed color LIHM method provides a way to realize color holographic imaging with white light sources and thus reduces the cost and complexity of the lensless color microscope.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
Compared with conventional microscope, lensless in-line holographic microscope (LIHM) provides a promising alternative microscopic imaging solution because of its advantages of low cost, compactness, wide field-of-view, multilayer imaging capability, and flexible imaging distances [1–5]. In many application areas, e.g., biology and medicine, multispectral or color microscopic images are very useful because of they provide richer information compared with monochromatic images [6,7]. Intuitively, color imaging in LIHM can be achieved by either using multiple laser sources or light-emitting diodes (LEDs) with different color [8–10] or white light sources with different spectral filters . However, the use of multiple light sources or different spectral filters increases the system complexity and cost as well as the image acquisition time because multiple holograms in different color have to be acquired sequentially to obtain the full color image.
To simplify the system design and reduce the cost, we propose a method that can acquire color images in LIHM by using white light illumination without any spectral filters and acquiring only one in-line hologram with a color CMOS imaging sensor. In this case, the in-line color hologram can be decomposed into red (R), green (G), and blue (B) sub-holograms which can then be used to reconstruct the corresponding color components of the sample image. However, because of the coherence requirements in holographic reconstruction, the issues of insufficient spatial and temporal coherence must be resolved so that the sample images of different colors can be reasonably reconstructed. For the issue of insufficient spatial coherence, the existing methods include using a pinhole  or a deconvolution process . In our experiment, we chose to use sunlight illumination and the spatial coherence was good enough because the sun is very far away from the earth and can be considered as a point source. For the issue of insufficient temporal coherence, spectral filters can be used with the trade-off of increasing system cost and reducing imaging speed in color imaging. Alternatively, a computational spectral demultiplexing method can be applied  with the cost of additional computation time and possibly unreliable convergence by using the iterative phase retrieval algorithm. Previously we also proposed a differential holographic reconstruction method to solve the insufficient temporal coherence issue . However, this method is more useful for ultra-broadband light sources and not so effective for the narrower bandwidths of different color components in the color imaging case. In this paper, we propose to use a deconvolution method to solve the insufficient temporal coherence issue with a synthetic point spread function (PSF) computed from the spectral distribution of each color component. We will show that the deconvolution method is applicable for weakly-scattered amplitude objects.
Compared with traditional color holographic imaging method, our method can significantly reduce the system cost and complexity by using sunlight illumination instead of multiple light source illuminations or white light illumination with different spectral filters.
The schematic of our color LIHM system setup is shown in Fig. 1(a), which consists of a weakly-scattered amplitude object illuminated by sunlight and a color CMOS imaging sensor with pixel size of 2.2 μm (DFK72AUC02, the Imaging Sources Europe GmbH) behind the object to record the in-line hologram. It should be observe that the in-line hologram will be blurry and look more like “shadow” instead of “hologram” because of the insufficient temporal coherence. On the other hand, the distance between the object and the imaging sensor is short (5 ~20 mm) and the “shadow” still contains visible information of the object. As we know, the color CMOS imaging sensor consists of a Bayer pattern of RGB filter array as shown in Fig. 1(b). The spectral responses SX(λ) (here X = R, G, or B and represents the colors of red, green, or blue) of the R, G, and B components of the sensor are also shown in the figure, where the full-width-at-half-maximum (FWHM) spectral widths are around 95 nm, 85 nm, and 90 nm for the R, G, and B components, respectively. The in-line hologram captured by the color CMOS imaging sensor can thus be decomposed into three channels of sub-holograms in red, green, and blue, as shown in Fig. 1(c). Color sample images can then be achieved by combining the reconstruction results of the sub-holograms in RGB channels. However, conventional holographic reconstruction of the sub-holograms by directly propagating the holograms back to the object plane with scalar diffraction (will be called “direct reconstruction” hereafter) will not be able to achieve satisfactory sample images because of insufficient temporal coherence resulted from the broadband spectral response of the RGB channels. To solve this problem, we propose to perform the reconstruction by a deconvolution method with a synthetic PSF computed from the spectral responses of the color channels. The details of our method are explained below and reconstructed color images with better contrast and resolution are also demonstrated comparing with those acquired by direct reconstructions.
For the derivation below, we will use the Cartesian coordinates where the z-direction is along the sunlight propagation direction and the xy-plane is the transverse plane, as shown in Fig. 1(a). And we set the sample plane as z = 0. For weakly-scattered amplitude object, the optical field after the sample plane can be written as 1 + o(x, y; λ), where o(x, y; λ) << 1 is the amplitude transmission coefficient of the sample for light wavelength λ and is real. Here the z coordinate is omitted for simplification. Then the in-line hologram intensity under monochromatic light illumination with wavelength λ can be written as , knowing that o(x, y; λ) is real,16].
For each color channel with spectral response SX(λ), the sub-hologram can be regarded as the incoherent superposition of holograms with monochromatic light illuminations within the spectrum. In addition, we assume that o(x, y; λ) = o(x, y; X) (X = R, G, or B) is independent of λ within the spectrum SX(λ) because of the relatively narrow bandwidth for each color channel. Thus, the intensities of the sub-holograms in our color LIHM can be written as:Eq. (3) is independent of the spatial coordinates (x, y) and can be easily removed with a high-pass spatial filter. Thus the sub-hologram intensity can be considered as a convolution of o(x, y; X) and hX(x, y) after removing the first term in Eq. (3). And o(x, y; X) can then be recovered by a deconvolution process with the computed hX(x, y) according to Eq. (4). In the reconstruction, we used a simple Wiener filter provided by Matlab to compute the deconvolution. Finally the color image of the sample o(x, y) can be acquired by combining the three color components o(x, y; R), o(x, y; G), and o(x, y; B).
Numerical simulation was first performed to verify the resolution enhancement capability of our deconvolution method with a U. S. Air Force (USAF) target as the sample. In the simulation, the illumination light spectrum was the solar radiation spectrum at sea level, as shown in Fig. 1(b). The pixel size we used in numerical simulation was 2.2 μm, which was identical to that of the imaging sensor used in our experiment. The distance between the sample and the sensor was set as 1.6 mm. The synthetic PSF of each color channel was numerically computed according to Eq. (4), knowing the illumination light spectrum and the spectral response of the Bayer pattern color filter array provided in the datasheet of the imaging sensor.
The simulation result is shown in Fig. 2. Figures 2(a)(b)(c) show the simulated sub-holograms along with the spectral distribution of the colors red, blue, and green, which are numerical computed by scalar diffraction . The direct reconstruction results with center wavelengths of 604 nm, 537nm and 455 nm are shown in Figs. 2(d)(e)(f), respectively. Obviously, the reconstruction results are blurry because of the insufficient temporal coherence caused by broad spectral distributions. The reconstruction results of using the deconvolution method are shown in Figs. 2(g)(h)(i). We can clearly see the improvement in resolution and contrast compared to the results with direct reconstruction. The reconstructed RGB components can be recombined to generate the color sample image, as shown in Figs. 2(j) and 2(k), which are the recombined color images of Figs. 2(d)(e)(f) and Figs. 2(g)(h)(i), respectively. Here we observe that the recombined color images are not colorful simply because the USAF target is not a color sample. Notice that the USAF target is deviated from the assumption of weakly-scattered object. And we chose to use it as the sample because we were also using it in our experiment to calibrate the imaging resolution. Nevertheless, the results in Fig. 2 obvious show the resolution enhancement capability of our method even when the approximation of weakly-scattered object is not strictly satisfied.
We then did experiment with a USAF target (R1DS1P, Thorlabs Co., USA) as the sample to further verify the effectiveness of our proposed method. To avoid saturation of the sensor because of strong sunlight during the experiment, a neutral-density filter with optical density (OD) of 2 was introduced. Notice that the exact OD of the filter is not critical as long as the imaging sensor is not saturated. And it’s possible to remove the filter if other white light sources with adjustable intensity are used. The synthetic PSF used in the experiment was the same as in the numerical simulation. The distance between the sample and the sensor was around 1.6 mm. The exposure time of the imaging sensor in our experiment was 0.2 ms.
The experimental result is shown in Fig. 3. Figures 3(a)(b)(c) show the red, green, and blue components of the captured hologram using the color imaging sensor. The direct reconstruction results with center wavelengths of 604 nm, 537 nm and 455 nm are shown in Figs. 3(d)(e)(f), respectively. As expected, the direct reconstruction results are not satisfactory. In contrast, the reconstruction results by using our deconvolution method show better imaging resolution and contrast, which can be observed in Figs. 3(g)(h)(i). The section curves of element 3, group 6 in the USAF target (bar width = 6.20 μm) were plotted in the corresponding reconstructed images of Figs. 3(d)-(i), which clearly show the resolution improvements after using our deconvolution method with the synthetic PSF. The RGB color components in Figs. 3(d)(e)(f) and Figs. 3(g)(h)(i) can be recombined to generate the color sample image, as shown in Figs. 3(j) and 3(k), respectively. Again, the recombined color images are not colorful because the USAF target is not a color sample. One may find out that the experimental results show worse imaging resolution and contrast compared to the simulation results, which mainly due to the additional noise in the experiments caused by ambient light illuminations including the scattering of sunlight by the edge of the aperture and dusts.
Since the USAF target is not a color sample, we did imaging experiment using a color sample, a stained root of the herb Saposhnikovia divaricata to demonstrate the color imaging capability of our color LIHM system. The imaging results are shown in Fig. 4.
Figure 4(a) shows the captured color in-line hologram under sunlight illumination using the color CMOS imaging sensor. In comparison, Fig. 4(b) shows the in-line hologram using laser illumination with wavelength of 473 nm (MBL-III-473, Changchun New Industries Optoelectronics Technology Co.). We can see that Fig. 4(a) is much more blurry because of the insufficient temporal coherence. As baseline sample images for comparison, Figs. 4(c) and 4(d) show the holographic reconstruction result of Fig. 4(b) and a microscope image obtained with 4X objective, respectively. The direct reconstruction results of the sub-hologram components of red, blue, and green with center wavelengths of 604 nm, 537 nm, and 455 nm are shown in Figs. 4(e)(f)(g), respectively. And Fig. 4(h) is the RGB color image obtained by combing the color components in Figs. 4(e)(f)(g). Because of the broad spectral distribution and thus insufficient temporal coherence, we can see that the reconstructions look blurry compared with laser illumination result in Fig. 4(c) and the conventional microscope image in Fig. 4(d). In contrast, Figs. 4(i)(j)(k) show the reconstruction results for red, green, and blue components using our proposed deconvolution method with the abovementioned synthetic PSF. And Fig. 4(l) is the RGB color image obtained by combing the color components in Figs. 4(i)(j)(k). We can easily observe the resolution enhancement compared with the direct reconstruction. By comparing Fig. 4(d), 4(h), and 4(l), we can see that the color holographic reconstruction shows similar color distribution as the conventional microscope image. Nevertheless, the resolution and contrast of the color LIHM image are still worse than that of the microscope image in Fig. 4(d). This is mainly because of the approximation of weakly scattered amplitude object in our method.
4. Discussions and conclusions
We should notice that in our system, the short distance between the sample and the imaging sensor is critical. Because of the insufficient temporal coherence, large sample-to-sensor distance will lead to very blurry “shadow” where the high spatial frequency components of the hologram will be lost, and thus cannot be used for holographic reconstruction. This is a distinct difference compared with conventional holographic reconstructions where sufficient temporal coherence is guaranteed .
Another issue that affects our image reconstruction is the deconvolution process. We used a simple Wiener filter for deconvolution where the PSF is computed from the spectral responses of the RGB components. However, the spectral distribution might not be accurate because the sunlight spectrum might change in different weathers . Considering this issue, better reconstructions should be possible if accurate spectral distribution is known for computing the synthetic PSF. Furthermore, it is well known that the RGB channels in the imaging sensors have crosstalk which would affect the accuracy of the spectral response used in our PSF. O. Flasseur et al. has proposed a parametric inverse problem approach to compensate for it in lensless color microscopy . Z. E. Phillips et al.  and W. Lee et al.  also proposed a linear model to correct the crosstalk problem for quantitative phase microscopy with color LED array. Although we cannot directly use these approaches because of the difference in setups and illuminations, it might be possible to solve the problem and further improve our image reconstruction quality with similar methods in the future.
We also observe that one of the major limitations of our method is the assumption that the sample is amplitude object. And only in this case, the captured sub-hologram intensities can be expressed as a convolution process with the analytic expression shown in Eq. (3). Further studies are needed to evaluate whether our assumptions are too restrictive and it might be possible to loosen the approximations. Nevertheless, for samples that exhibits phase modulation, we can approximately treat it as amplitude object if the phase modulation is small. And our reconstruction can still obtain better reconstruction results, as can be seen in Fig. 4, where the sample was not a pure amplitude object.
In summary, we propose a novel color lensless in-line holographic imaging method that can be used to acquire color images of weakly scattered amplitude objects under white light illumination including sunlight. In our method, only one color in-line hologram is required which can be obtained with a color CMOS imaging sensor. The basic principle of our color LIHM is to describe the RGB components of the acquired in-line hologram as a convolution process. And the color sample image can be thus obtained by recombining the RGB components reconstructed with a deconvolution process. We demonstrated the resolution enhancement capability of the deconvolution process by performing numerical simulations and experiments with the USAF target as the sample. The color sample of stained root of Saposhnikovia divaricata was also imaged to demonstrate the color imaging capability of our system. Because of the simple setup and one-shot color imaging capability, our color LIHM method is suitable for low-cost and compact color imaging applications.
Shanghai Pujiang Program (12PJ1405100); National Natural Science Foundation of China (61205192).
1. A. Greenbaum, W. Luo, T. W. Su, Z. Göröcs, L. Xue, S. O. Isikman, A. F. Coskun, O. Mudanyali, and A. Ozcan, “Imaging without lenses: achievements and remaining challenges of wide-field on-chip microscopy,” Nat. Methods 9(9), 889–895 (2012). [CrossRef] [PubMed]
2. A. C. Sobieranski, F. Inci, H. C. Tekin, M. Yuksekkaya, E. Comunello, D. Cobra, A. von Wangenheim, and U. Demirci, “Portable lensless wide-field microscopy imaging platform based on digital inline holography and multi-frame pixel super-resolution,” Light Sci. Appl. 4(10), e346 (2015). [CrossRef] [PubMed]
4. E. Serabyn, K. Liewer, C. Lindensmith, K. Wallace, and J. Nadeau, “Compact, lensless digital holographic microscope for remote microbiology,” Opt. Express 24(25), 28540–28548 (2016). [CrossRef] [PubMed]
5. W. Bishara, U. Sikora, O. Mudanyali, T. W. Su, O. Yaglidere, S. Luckhart, and A. Ozcan, “Holographic pixel super-resolution in portable lensless on-chip microscopy using a fiber-optic array,” Lab Chip 11(7), 1276–1279 (2011). [CrossRef] [PubMed]
7. D. Roblyer, R. Richards-Kortum, K. Sokolov, A. K. El-Naggar, M. D. Williams, C. Kurachi, and A. M. Gillenwater, “Multispectral optical imaging device for in vivo detection of oral neoplasia,” J. Biomed. Opt. 13(2), 024019 (2008). [CrossRef] [PubMed]
15. S. Feng and J. Wu, “Differential holographic reconstruction for lensless in-line holographic microscope with ultra-broadband light source illumination,” Opt. Commun. 430, 9–13 (2019). [CrossRef]
17. J. W. Goodman, Introduction to Fourier Optics (Roberts & Company Publishers, 2004), Chap. 3.
18. S. Nann and C. Riordan, “Solar Spectral Irradiance under Clear and Cloudy Skies: Measurements and a Semiempirical Model,” J. Appl. Meteorol. 30(4), 447–462 (1991). [CrossRef]