We propose a signal enhanced guide-star reconstruction method for holographic fluorescence microscopy. In the late 00’s, incoherent digital holography started to be vigorously studied by several groups to overcome the limitations of conventional digital holography. The basic concept of incoherent digital holography is to acquire the complex hologram from incoherent light by utilizing temporal coherency of a spatially incoherent light source. The advent of incoherent digital holography opened new possibility of holographic fluorescence microscopy (HFM), which was difficult to achieve with conventional digital holography. However there has been an important issue of low and noisy signal in HFM which slows down the system speed and degrades the imaging quality. When guide-star reconstruction is adopted, the image reconstruction gives an improved result compared to the conventional propagation reconstruction method. The guide-star reconstruction method gives higher imaging signal-to-noise ratio since the acquired complex point spread function provides optimal system-adaptive information and can restore the signal buried in the noise more efficiently. We present theoretical explanation and simulation as well as experimental results.
© 2016 Optical Society of America
Digital holography records the digitally sampled optical interference pattern generated by the object light wave emanating from a three-dimensional scene and a reference light wave using imaging devices such as a charge coupled device (CCD) [1–13]. Since the three-dimensional complex optical field can be obtained, digital holography has many distinct advantages. One of the well-known features of digital holography is three-dimensional imaging by digital refocusing or sectioning [5–7]. Direct access to the phase information enables various image processing techniques such as edge enhancement or wavefront compensation [8–10]. Also, by adopting a dynamic phase diffraction grating, super-resolution can be realized [11,12]. These features make digital holography a powerful tool for optical microscopy . However, the need of a coherent light source limited the versatility of digital holography. It was difficult to apply digital holography techniques to imaging applications using incoherent light sources such as astronomy or natural light holography. Especially, fluorescence microscopy [14,15], which is an important technique in the microscopy field, was not included in the area of digital holography since fluorescence light is naturally incoherent. There have been efforts of adopting digital holography to fluorescence microscopy by adopting mechanical scanning, however they usually need complicated additional systems and are very time consuming [16–18].
In late 00’s, incoherent digital holography (IDH) started to be vigorously studied by several groups to overcome the limitations of conventional digital holography. The basic concept of IDH is to utilize temporal coherency of a spatially incoherent light source. Every point source in object is considered to be spatially uncorrelated to each other. Self-interference incoherent digital holography (SIDH) is one kind of IDH which uses a variation of the Michelson interferometer to generate the self-interference of a signal wavefront. Various studies have shown that IDH is capable of opening new exciting possibilities in fields such as natural light holography, astronomy, microscopy, and especially fluorescence microscopy, which is described above [19–29]. It is noteworthy that a distinct characteristic of IDH enables super-resolution microscopy due to Lagrange invariant violation . Also it is possible to compensate the distortions introduced by aberrant media by adopting incoherent digital holographic adaptive optics (IDHAO) technique [27–29].
Those progresses in holographic fluorescence microscopy (HFM) enable promising new applications. However, in the typical imaging situation of HFM, signal strength is an important issue to be overcome [25,30]. Unlike conventional fluorescence microscopy, the image is captured at a defocused plane in HFM. In contrast to conventional imaging methods, in HFM, the sensor plane is usually located at a different position from the focused plane of the image. Light from an infinitesimal point in a fluorescence object does not form the focusing point at the sensor plane, but is spread to cover a certain area at the sensor plane. When the signal is weak, which means photon number is decreased, the ratio of noise intensity increases since photon capturing of the sensor is a statistical process. If the excitation light is too strong, the sample could be damaged or severe photo-bleaching could be a problem. In order to reduce the noise, multiple shots should be taken or exposure time should be increased (Usually near one second of exposure time is needed for one shot). However, this type of approach slows down the system speed. Therefore it is an important issue to enhance the signal-to-noise ratio (SNR) in holographic fluorescence microscopy. In this study, we find that the guide-star reconstruction method of IDHAO can be used to enhance the SNR in holographic fluorescence microscopy. This can be distinguished from conventional IDHAO technique since it is first to report that guide-star reconstruction can be used in HFM for the purpose of enhancing the signal. We present the theoretical description of the proposed scheme with the simulation and experimental results.
Incoherent digital holographic adaptive optics (IDHAO) is a distinct technique that can restore the distorted wavefront from the object through aberrant media [27,28]. In IDHAO, a guide-star hologram is acquired from the imaging system and is used to restore the object wavefront. This reconstruction process is called guide-star reconstruction. Since the phase information of the wavefront is acquired using an incoherent light source, it can be utilized in various applications such as astronomy or natural light hologram acquisition. The idea of IDHAO can be applied to microscopy as well. It has been reported that it is possible to image the fluorescence microscopic object through aberrant media in tissue or cell .
In this paper, we report that this guide-star reconstruction also has the capability of enhancing the SNR compared to the conventional numerical propagation method in digital holography when the acquired signal is weak and noisy. It is a noteworthy feature especially in an imaging environment such as holographic fluorescence microscopy. When the signal from the object is weak, even the subtle aberration and noise shades the signal from the fluorescence object, so that the image cannot be reconstructed from the acquired hologram by the conventional numerical propagation reconstruction method. However, as shown in the following sections, guide-star reconstruction is able to restore the signal from the same hologram better than the conventional reconstruction method.
2.1. Description of hologram acquisition in HFM system
In SIDH, the wavefront emanating from the object is captured by a CCD with phase shifting procedure. The complex hologram is acquired which can be expressed as a convolution of the object intensity with the complex point spread function (CPSF). This procedure can be described in a straightforward way. Figure 1 shows the basic configuration of the HFM system. The spherical wave emanated from the object propagates through the objective lens and the tube lens which have the focal lengths of fo and ft, respectively. The objective lens should be located at the distance of fo from the object and has the ideal thin lens profile. zo and zt are set as twice the focal length of the tube lens (zo = zt = 2ft). We assume that there is an aberration Ψ on the objective lens which describes the imperfection of the optical element in the imaging system. Then, the wavefront enters into a variation of the Michelson interferometer system consisting of two mirrors (MA, MB) and a beam splitter (BS). The optical field generated by the mirror A which has the focal length of fA is expressed as follows:Eq. (1) only with the different curvature and spatial shift. Since the spherical wave from the point source has the temporal coherency, the wave fronts from two mirrors interfere with each other at the CCD plane when path length is matched and the captured intensity has a form of29]. Especially, when the point source is located at the center, we define the complex hologram as a guide-star hologram as follows:
2.2. Signal enhancing of HFM using guide-star reconstruction
We can set the guide-star hologram as Q-function multiplied with φ term which indicates the variation from the ideal Q-function:
We should also consider the noise in the capturing process of the hologram acquisition. When a CCD camera captures an image, the noise will always appear on the acquired signal. The source of noise could be various types including background ambient photoflux from outside of the sample object scene. Even if the image of the fluorescence object is captured in a perfect dark room, the CCD itself generates noises such as dark current, readout noise and shot noise [31,32]. Dark current is caused by thermally generated electrons within the device and readout noise originates from voltage measuring process. Shot noise is caused by inherent statistical detection processes of the CCD whose deviation follows Poisson distribution. In the low-light-level condition, shot noise becomes more dominant. Therefore long exposure enhances the SNR of the acquired image. We define the SNR of this acquired image of the CCD as a term “acquisition SNR” that has the form of:32]. All of these noise sources are added together in the captured intensity in Eq. (3). Since this noise is randomly distributed, it is not canceled out in the phase shifting process and becomes meaningless complex noise that adds to the hologram. This noise on the acquired complex hologram is indicated as εH as follows:
Conventionally, the image reconstruction of the acquired hologram is performed with the numerical propagation method such as Fresnel propagation, which can be interpreted as a cross correlation with the Q-function. In this case, zr is the optimum propagation distance for the full-field hologram. The noise term εH is also cross correlated with the Q-function and yields the noise on the reconstructed image which is indicated as εR as follows:Equations (13) and (15) are simplified modeling of the conventional reconstruction method and guide-star reconstruction according to the aberration and the noise level without considering the exit pupil of the system. In conventional reconstruction, the Q-function is used since there is an assumption of the lens and the ideal phase propagation. When the aberration is not severe, which can be modeled as a normal imaging environment, the phase variance of φ is very subtle and the cross correlation of Qzrφ and Qzr gives a similar form of a delta function. In this case, the image can be reconstructed using both methods and there is no obvious difference when visually recognized. However, we focus on the case of severe noise corrupting the full field hologram. When the signal is very weak, the noise term becomes relatively large and the image signal is corrupted by the noise. In this case, the degree of imaging SNR of the reconstructed result yields apparent differences in image quality. As Eqs. (13) and (15) indicate, the guide-star hologram can give the better imaging SNR than the conventional reconstruction method in that, it gives the signal as a form of image convolved with a delta function theoretically (In practice, the pupil function should be also considered therefore exact delta function cannot be acquired). This can be understood as guide-star hologram giving an optimized solution for image reconstruction adaptive to each specific optical system, while the conventional method is constrained by certain approximations and assumptions of optical propagation. This imaging SNR enhancement shows a meaningful difference in the weak-signal imaging environment of HFM. We have found that in some cases, the object image which cannot be detected using the conventional numerical propagation method could be reconstructed using the guide-star reconstruction method. We present this signal enhancing effect of HFM using the guide-star reconstruction by both simulations and experiments.
We performed the simulation in order to compare reconstruction results of the conventional reconstruction method (numerical propagation) with the guide-star reconstruction according to the noise intensity. The optical capturing procedure of the HFM setup is simulated based on diffractive optics theory assuming a spatially incoherent light source. MATLAB is used as a simulation tool. The configuration of the simulation is modeled in Fig. 1 based on the experimental setup. The objective lens has the focal length of 20 mm, and the tube lens has the focal length of 70 mm. Pupil size of the objective lens and tube lens is 1.5 mm. Pupil size of the camera is same as the simulation grid size. Pitch size of the simulation grid is 4 um and the total pixel number is 800 × 800.
In the simulation environment, there are no correction factors representing imperfections of the system such as alignment mismatching or aberration of lens which distort the wavefront of the signal at the CCD plane since an ideal situation is assumed in the simulation. Instead, we impose the artificial weak aberration layer in order to simulate the slight distortion of the wavefront, which models the experimental situation. The phase aberration layer is located at the objective lens in the HFM system and is assumed to have a subtle variance of phase varying from −0.4π to 0.4π. Figure 2 shows the object used at simulations and phase images of used aberration which is generated by Zernike’s polynomials according to the m, n values. In the presented result, the aberration pattern shown in Fig. 2(b) is applied.
Since spatial incoherency is assumed, light that propagates from each point source arrives at the CCD plane and is added as intensity, not complex amplitude. After the light propagation simulation, an additive Gaussian noise is added to the captured intensity profile in order to simulate the various noise sources at once. As Eq. (11) indicates, the noise level at the camera is a function of various factors including signal intensity and exposure time of the CCD. Figure 3(a) shows the acquired intensity pattern without noise and Fig. 3(b) shows the intensity of the wavefront after the Gaussian noise is imposed. Finally the signal intensity is captured by the virtual CCD and the whole simulation procedure is repeated 3 times changing the phase according to a three-step phase shifting process. Figure 3(c) shows the amplitude of the acquired complex hologram. The guide-star hologram acquisition is simulated with a point source located at the center position in the object plane. Figure 4 shows the unwrapped phase difference (phase of φ) between the simulated guide-star hologram and the corresponding Q-function which is used in the reconstruction stage as discussed in Sec. 2.2. The phase map is acquired using the Goldstein 2D phase unwrapping algorithm . To find the precise corresponding Q-function, we iteratively change the center position and propagation distance of Q-function until the integration value of phase difference over the plane is minimized. After the minimization process, Fig. 4 shows that there exists the phase difference between the complex guide-star hologram and the ideal Q-function because of the distortion induced by the aberration.
The reconstruction simulation was performed with varying acquisition SNR of the added Gaussian noise and Fig. 5 shows the reconstruction results according to these acquisition SNR values of the virtually captured full-field holograms. It is observed that when the noise level is low (when acquisition SNR is large enough), the image is well reconstructed using both reconstruction methods. Figure 6 shows the sectional view of reconstructed images at Fig. 5 sliced along red line. When the SNR value is 30, both methods clearly show six peaks which represent horizontal bars in the presence of the aberration. As the acquisition SNR value decreases, which means a low-signal situation, the noise gets severe in both cases. However we can find that guide-star reconstruction shows better contrast when acquisition SNR of the signal is decreased. Figure 7 plots the acquisition SNR values and following imaging SNR values of the reconstructed image using conventional and guide-star reconstruction methods. It should be noted that acquisition SNR on the x-axis refers to the SNR value of the capturing simulation process. Otherwise, imaging SNR on the y-axis means the SNR value of reconstructed image which is defined as the average value of the image intensity of the reconstructed resolution target over the average value of background area which does not contain the reconstructed image. For the fairness of the comparison, we selected same four bars for all reconstructed images (two vertical lines and two horizontal lines of element 2 in group −2) to measure the average intensity. By this simulation result, it can be shown that guide-star reconstruction could enhance and restore the buried signal under a noisy environment or weak signal in HFM.
4. Experiment and discussion
We performed the experiment to verify the signal enhancing capability of the guide-star reconstruction method in HFM. The setup is comprised of the microscopy system and the SIDH system as shown in Fig. 8. The microscopy system has basically the same configuration of the conventional microscopy setup. The object is magnified by the objective lens and refocused after passing the tube lens. Mercury lamp is used for excitation of the object and Olympus U-FBW filter set is adopted as an excitation filter. The emission filter has a narrow bandwidth (10 nm) with a center frequency of 530 nm.
The SIDH system includes a beam splitter and two mirrors with different curvatures. A planar mirror and a concave mirror, which has a focal length of 500 mm, are aligned to have same optical path length in order to generate the interference between two duplicated wavefronts with different curvatures. A piezo actuator is attached to the backside of the planar mirror in order to conduct the phase shifting process on the interference pattern and three shots of images are captured while changing the phases to acquire one complex hologram. A CCD camera is located at the end of the holographic fluorescence microscope setup to capture the interference pattern. It should be noted that in the experimental setup, there is no explicit (artificial) aberration layer added. In order to change the acquisition SNR, the holograms are acquired varying the exposure time from 0.5 ms to 5 ms. The exact acquisition SNR values are not determined, however as Eq. (11) shows, the acquisition SNR value is believed to vary according to the exposure time of the CCD. In this experiment, it should be guaranteed that the signal strength of captured image is only dependent on the exposure time to make sure the comparison is valid. Therefore the biological objects that show severe photobleaching effect are not appropriate for the comparison because the luminescence intensity is reduced during the repetitive image acquisition. For the sample object, a USAF resolution target with a fluorescent back plate is used since it shows no noticeable photobleaching effect.
The guide-star hologram acquisition is performed using a pinhole which has the aperture size of 15-μm with the fluorescent back plate as an object. Practically, the guide-star can be acquired using various methods such as a focused laser or embedded fluorescent bead in the sample object [29,34]. Since the guide-star is considered as reference point source in the reconstruction process, the guide-star size can be selected as sub-resolution limit to prevent the possible resolution loss . Figure 9 shows the amplitude/phase of the acquired guide-star hologram. As discussed above, in the conventional reconstruction method, this complex point spread function is assumed to be an ideal quadratic phase function. However in practice, the complex point spread function has a slightly distorted phase profile from the assumptions due to the conditions of the optical system. Figure 10 shows the extracted phase difference between the guide-star and the ideal Q-function, which is corresponding to Fig. 4 in the simulation part, using the same method. It can be shown that there is a slight distortion in the guide-star hologram although there is no explicit aberration layer in the experimental setup. Since φ shows the extracted phase error at the CCD plane, we can directly calculate the Strehl ratio if we consider the CCD plane as the aperture plane of the classical imaging system. Although Strehl ratio is not widely used or consistently defined in digital holographic imaging systems, it can provide the general information of describing the inherent phase distortion of the system. Using the Mahajan formula , the calculated Strehl ratio is 0.509.
Since those differences are not very significant, when the signal is strong enough, the image is well reconstructed by the conventional propagation method using the quadratic phase function. However as described in Sec. 2, when the signal is weak, the guide-star reconstruction method can provide a higher SNR since the complex point spread function provides system adaptive information and can restore the signal buried in the noise more efficiently.
Figure 11 shows experimental results of guide-star reconstruction of HFM with the acquired complex hologram when exposure time is 5 ms. It can be verified that guide-star reconstruction gives an improved imaging SNR over the conventional reconstruction method comparing Fig. 11(a) and 11(b). In Fig. 11(c), which shows the cross-sectional view of the conventional reconstruction result, the image suffers from severe noise, so peaks are difficult to recognize. In contrast, in Fig. 11(d), peaks are clearly distinguishable from the surrounding noise. We present the reconstruction result set according to the exposure time of the captured image in Fig. 12. Since acquisition SNR decreases with exposure time of the CCD, imaging SNR also decreases and the imaging results become noisier. Figure 12 (a) shows the phase of the acquired hologram varying the exposure time while Figs. 12(b) and 12(c) show reconstruction results of the hologram according to the conventional and guide-star reconstruction method, respectively. Figure 13 plots the imaging SNR values of the reconstructed images using conventional and guide-star reconstruction methods according to the exposure time. Two vertical lines and two horizontal lines of element 1 in group 2 are selected to measure the average intensity. It can be shown that the guide-star reconstruction provides better results than conventional reconstruction over all regions. We can conclude that the experimental result supports theoretical analysis and simulation result as well.
We presented the signal enhancing effect of the guide-star reconstruction method in holographic fluorescence microscopy. This is a noteworthy feature in HFM since the low signal environment is a critical issue which slows down the system speed and degrades the imaging quality. We compared the images of numerical propagation reconstruction method and guide-star reconstruction considering the practical imaging situation including aberration and noise. The guide-star reconstruction method gives higher imaging SNR since the complex point spread function provides optimal system-adaptive information and can restore the signal buried in the noise more efficiently. We also have extracted the phase difference term which shows the error between theoretical assumptions and practical imaging situations. We have presented a theoretical explanation of the guide-star reconstruction method and simulation results as well as experimental results. In order to show the feasibility, the resolution target with a photobleaching-free fluorescence plate is used as an object since it is appropriate for the reliable comparison. We expect to adapt proposed configuration to the practical application of cellular imaging in future work. We believe that the presented guide-star reconstruction method can broaden the potential of HFM.
This research was supported by “The Cross-Ministry Giga KOREA Project” of The Ministry of Science, ICT and Future Planning, Korea [GK15D0200, Development of Super MultiView (SMV) Display Providing Real-Time Interaction].
References and links
2. E. N. Leith and J. Upatniek, “Reconstructed wavefronts and communication theory,” J. Opt. Soc. Am. 52(10), 1123–1130 (1962). [CrossRef]
3. E. N. Leith and J. Upatnieks, “Wavefront reconstruction with diffused illumination and three-dimensional objects,” J. Opt. Soc. Am. 54(11), 1295–1301 (1964). [CrossRef]
9. P. Ferraro, S. De Nicola, A. Finizio, G. Coppola, S. Grilli, C. Magro, and G. Pierattini, “Compensation of the inherent wave front curvature in digital holographic coherent microscopy for quantitative phase-contrast imaging,” Appl. Opt. 42(11), 1938–1946 (2003). [CrossRef] [PubMed]
11. C. Liu, Z. Liu, F. Bo, Y. Wang, and J. Zhu, “Super-resolution digital holographic imaging method,” Appl. Phys. Lett. 81(17), 3143 (2002). [CrossRef]
12. M. Paturzo, F. Merola, S. Grilli, S. De Nicola, A. Finizio, and P. Ferraro, “Super-resolution in digital holography by a two-dimensional dynamic phase grating,” Opt. Express 16(21), 17107–17118 (2008). [CrossRef] [PubMed]
13. M. K. Kim, “Principles and techniques of digital holographic microscopy,” SPIE Reviews 1(1), 018005 (2010).
14. Y. Engelborghs and A. J. W. G. Visser, Fluorescence Spectroscopy and Microscopy (Springer Protocols 2014).
17. T.-C. Poon, “Scanning holography and two-dimensional image processing by acousto-optic two-pupil synthesis,” J. Opt. Soc. Am. 2(4), 521–527 (1985). [CrossRef]
18. T.-C. Poon, K. B. Doh, B. W. Schilling, M. H. Wu, K. K. Shinoda, and Y. Suzuki, “Three-dimensional microscopy by optical scanning holography,” Opt. Eng. 34(5), 1338–1344 (1995). [CrossRef]
19. A. W. Lohmann, “Wavefront reconstruction for incoherent objects,” J. Opt. Soc. Am. 55(11), 1555–1556 (1965). [CrossRef]
20. G. Cochran, “New method of making Fresnel transforms with incoherent light,” J. Opt. Soc. Am. 56(11), 1513–1517 (1966). [CrossRef]
21. S.-G. Kim, B. Lee, and E.-S. Kim, “Removal of bias and the conjugate image in incoherent on-axis triangular holography and real-time reconstruction of the complex hologram,” Appl. Opt. 36(20), 4784–4791 (1997). [CrossRef] [PubMed]
24. O. Cossairt, N. Matsuda, and M. Gupta, “Digital refocusing with incoherent holography,” Computational Photography (ICCP 2014), 1-9 (2014).
25. J. Rosen and G. Brooker, “Non-Scanning Motionless Fluorescence Three-Dimensional Holographic Microscopy,” Nat. Photonics 2(3), 190–195 (2008). [CrossRef]
26. J. Rosen, N. Siegel, and G. Brooker, “Theoretical and experimental demonstration of resolution beyond the Rayleigh limit by FINCH fluorescence microscopic imaging,” Opt. Express 19(27), 26249–26268 (2011). [CrossRef] [PubMed]
29. C. Jang, J. Kim, D. C. Clark, S. Lee, B. Lee, and M. K. Kim, “Holographic fluorescence microscopy with incoherent digital holographic adaptive optics,” J. Biomed. Opt. 20(11), 111204 (2015). [CrossRef] [PubMed]
31. M. H. White, D. R. Lampe, F. C. Blaha, and I. A. Mack, “Characterization of surface channel CCD image arrays at low light levels,” IEEE J. Solid-State Circuits 9(1), 1–12 (1974). [CrossRef]
32. Technical note, http://www.photometrics.com/
33. R. M. Goldstein, H. A. Zebken, and C. L. Werner, “Satellite radar interferometry: two-dimensional phase unwrapping,” Radio Sci. 23(4), 713 (1988). [CrossRef]
34. R. Horstmeyer, H. Ruan, and C. Yang, “Guidestar-assisted wavefront-shaping methods for focusing light into biological tissue,” Nat. Photonics 9(9), 563–571 (2015). [CrossRef]
36. V. Mahajan, “Strehl ratio for primary aberrations in terms of their aberration variance,” J. Opt. Soc. Am. 73(6), 860–861 (1983). [CrossRef]