Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Versatile compressive microscope for hyperspectral transmission and fluorescence lifetime imaging

Open Access Open Access

Abstract

Increasing demand for multimodal characterization and imaging of new materials entails the combination of various methods in a single microscopic setup. Hyperspectral imaging of transmission spectra or photoluminescence (PL) decay imaging count among the most used methods. Nevertheless, these methods require very different working conditions and instrumentation. Therefore, combining the methods into a single microscopic system is seldom implemented. Here we demonstrate a novel versatile microscope based on single-pixel imaging, where we use a simple optical configuration to measure the hyperspectral information, as well as fluorescence lifetime imaging (FLIM). The maps are inherently spatially matched and can be taken with spectral resolution limited by the resolution of the used spectrometer (3 nm) or temporal resolution set by PL decay measurement (120 ps). We verify the system's performance by its comparison to the standard FLIM and non-imaging transmission spectroscopy. Our approach enabled us to switch between a broad field-of-view and micrometer resolution without changing the optical configuration. At the same time, the used design opens the possibility to add a variety of other characterization methods. This article demonstrates a simple, affordable way of complex material studies with huge versatility for the imaging parameters.

© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

Spectroscopy is a general term comprising many different methods ranging from simple absorption spectroscopy to complex time-resolved methods [1]. Depending on the used technique, it is possible to identify changes in chemistry [2], concentration [3], defects [4] in structure, and many more. The spectroscopic techniques are often combined to attain complementary pieces of information and a reliable model of the material energy states and ongoing processes.

Absorption spectroscopy and photoluminescence (PL) decay measurement count among the most used methods. They provide us with information about the optically active energy states and the timescales of the light-induced processes, respectively. Moreover, PL decay often serves as a highly reliable way to identify specific processes in biological samples or PL markers [5]. Therefore, an extension of these techniques into the imaging mode – hyperspectral imaging (HSI) and fluorescence lifetime imaging (FLIM) – is immensely useful and commonly used. [5]

HSI makes it possible to create a 3D datacube, where the third dimension corresponds to the wavelength. The sample transmittance in various wavelengths provides a significantly larger amount of information than the monochromatic or a standard three-channel camera. Apart from the transmission, HS information can also be studied for reflectance or sample emission. [6] There are several ways of acquiring HS information, usually based on scanning [7]. However, these systems suffer from the necessity to use a very high light intensity due to a narrow spectrometer slit and the light being dispersed across the detector. In combination with a massive amount of data acquired during the measurement, there has been a continuous effort to develop new imaging systems. This has also triggered the interest in the acquisition of the HS datacube via compressive imaging [8,9].

FLIM focuses on emission dynamics inside the sample [10]. PL decay can take place on a broad range of timescales, starting from microseconds [11,12], up to femtosecond timescale, where unique measuring systems such as streak camera, up-conversion, or time-correlated single-pixel counting (TCSPC) are used. [1316] Among those, TCSPC is the most common FLIM method, as its optimum performance matches the decay of many color centers, biological samples, and other PL sources. It works on the principle of exciting a sample using a laser pulse and measuring the delay between the laser pulse and capturing one photon emitted from the sample. With this simple measurement repeated multiple times, it is possible to acquire a histogram of measured delays, which corresponds to the PL decay dynamics. For the system to work properly, it is essential to exclude the possibility that two or more photons will arrive at the detector for the same excitation pulse. Therefore, the light levels are typically very low, and one emitted photon per 50-100 excitation pulses should be captured.

The working conditions of HSI and TCSPC-based FLIM are very different and require a different kind of instrumentation – low-light conditions and laser beam scanning for TCSPC, while HSI is commonly attained by line image scanning (push-broom technique) or a variable color filter (staring configuration) acquired for a high-light-level illumination. Therefore, combining such different experiments in a single setup is a complex task which is seldom addressed in the literature.

In some cases, FLIM is acquired for a set of wavelengths, which is denoted as Hyperspectral Macroscopic Fluorescent Lifetime Imaging (HMFLI) [1719]. This type of setup does not study the absorption spectra and indirectly measures the emission spectra as it focuses on the dependency of emission dynamics on wavelength. Another design presented in [20] shows a sample illuminated through a digital micromirror device (DMD), where the emitted light is separated to FLIM and HS detectors giving an emission dynamic and spectrum. Here the DMD is used to illuminate selected sections of the sample and measure in a raster-scanning pattern making the measurement relatively long. As these lead to 4D output, the amount of raw data created in many such systems is significant.

This requires the use of compressive algorithms, deep learning (as seen in [17,18]), or compressed sensing. If the line of field detectors for such acquisitions is used, such setup's price becomes significant. This is a considerable advantage of compressed sensing-based systems, where only a single point detector is required. Therefore, compressive imaging is used in microscopy to create multispectral images of a sample. [21] A compressive fluorescence microscope [22] has been created with the option of using a 128-channel spectrometer to depict spectrally dependent fluorescence. In another article [23], an advanced avalanche diode module is used to create a multidimensional dataset, in which each pixel contains information about fluorescence dynamics in multiple wavelengths. Similar to [20], both of these setups [22,23] use DMD to modulate illumination of the sample but, instead of raster-scanning, employ pseudo-random Hadamard patterns.

In this article, we combined transmission HSI with FLIM by using a single setup based on compressed sensing, namely on the so-called single-pixel camera (SPC). This method makes it possible to reconstruct an image by using only a fraction of the measurements necessary compared to the number of pixels [24,25]. Moreover, SPC only requires measuring a total integrated signal modulated by a random pattern for image retrieval. Therefore, we can easily attain the HSI information by measuring a spectrum and analogously extending the PL decay measurement into FLIM. [26,27]

We used these benefits to create a simple modular microscope able to trace both HS information of the transmitted light in the visible spectral range and FLIM based on a standard non-imaging TCSPC instrumentation. Because the HSI and FLIM images are modulated by the same random patterns, we created a multidimensional map of a measured sample, where the HS and FLIM maps are automatically matched. Moreover, the attainable spectral and temporal resolution follows the resolution of the used spectrometer and TCSPC setup.

Finally, we demonstrate a huge versatility of our system concerning the image magnification and field of view without the necessity to carry out any modification of the setup, such as using multiple objectives. The patterns used for SPC measurements are imposed by a large DMD chip, which can be used over its whole area – enabling measurements over millimeter-sized spots on the sample. At the same time, by scaling down the pattern on the DMD, we can carry out imaging with micrometer resolution, where the resolution is limited by the microscope objective lens.

We demonstrate the functionality of the setup on the testing samples of fluorescent dyes (Rhodamine, Fluorescein), luminophore (LuAG:CE), and a real-life sample – the cells of Convallaria stained by a PL marker. The setup performance was evaluated by comparing the PL of laser dyes to those measured by a standard FLIM setup using TCSPC and excitation scanning. Transmittance measurements are compared against the standard spectrometer results.

2. Methodology

The tested experimental setup is depicted in Fig. 1. It has been designed so that switching between the two configurations can be done simply by turning on the respective light source and redirecting the optical fiber to the designated detector.

 figure: Fig. 1.

Fig. 1. The hyperdimensional single-pixel camera setup. A high-intensity LED (1) is used as a source for hyperspectral imaging, a pulse laser diode (2) for lifetime measurement. A sample (3) is illuminated by one of the sources and through microscope objective (4) projected on the DMD (9). For PL lifetime measurement, a filter (5) is used to block the reflected laser light. A camera (6) can be used to see the positioning of the sample. To correctly focus the setup, a flip mirror (7) can be used to reflect the light to a screen (8) positioned at the same distance as the DMD. Light modulated on the DMD is then reflected towards a collimator with a diffuser (10) and focused into an optical fiber. The fiber then brings it either into a spectrometer or a photomultiplier and a TCSPC module (11).

Download Full Size | PDF

The light source for the HSI configuration was a high-intensity white LED (ThorLabs MCWHL5), which we also used in our previous article [26]. A pulsed laser diode PicoQuant LDH-P-C-405M at 405 nm has been used for FLIM configuration, producing pulses with a repetition rate of up to 40 MHz. The repetition rate was set to the highest possible value, given by the sample PL decay dynamics.

The sample was put into a focal plane of a microscope objective (Olympus PLN 4x, NA = 0.1) which created a magnified image of the sample on the surface of a digital micromirror device (DMD) DLP7000BFLP. The scaling of the image was calibrated based on the USAF 1951 resolution target.

A GG435 Schott filter blocking all wavelengths shorter than 415 nm was added to prevent the reflected and scattered excitation laser light from reaching the detector in the FLIM configuration.

The DMD was used to impose a random mask so that tilted mirrors reflected the light away or towards the collimator (Thorlabs RC08SMA-P01) with an engineered diffuser. The collimator then focused the light into a multimode fiber (1 mm core), which guided it to a detector.

The detector in the HSI configuration was the Ocean Optics Flame-S spectrometer (3 nm resolution), and in the case of the FLIM configuration, we used a PicoQuant PMA Hybrid photomultiplier. The signal from the photomultiplier was timed with the laser pulses using a standard TCSPC module, PicoQuant PicoHarp 300. The setup features 120 ps temporal resolution given by the detector response time.

If not stated otherwise, all measurements were done with DMD patterns 64-by-64 pixels. Since the native resolution of the DMD is significantly higher (1024-by-768), micromirrors were binned into groups, which acted as one pixel in the final image. For the large field of view, we used the entire DMD chip as a 64-by-64-pixel pattern. On the contrary, a single DMD mirror served as a single pixel for the highest attainable resolution.

Depending on the scene's complexity, a compression factor, i.e., the ratio between the number of measurements M and the number of image pixels N, was set to M/N = 0.15-0.35.

The acquired dataset was a set of spectra or decay curves measured for each random mask. By using the SPC image retrieval, we reconstructed the images using the total variation minimization as the regularizing term [28]. We point the reader to extensive literature describing the SPC theory and retrieval algorithms for more details. [25,2931] We employed the TVAL3 algorithm for our data [32]. TVAL3 reconstruction parameters have been selected based on the complexity of the scene and the noise level. The role of the parameters on the image properties was discussed in our previous work [33].

We subtracted a long-term drift present in the measured signals to eliminate the effect of varying illumination levels, dark detector currents, etc. The subtraction is possible owing to the fact that the sums of measured random pixels have to vary randomly around a mean value. Subtracting a low-frequency change, therefore, does not influence the signal and improves data quality. This step eliminates the need for highly stable laboratory conditions or precise detector cooling.

To improve the signal-to-the-noise ratio and decrease the total reconstruction time, we used binning of 5 spectral channels for HSI. This was justified by the fact that spectral channels in the used spectrometer featured spectral spacing well below the actual spectrometer resolution. In the case of FLIM, TCSPC setup carried out binning inherently during the measurement, where the single temporal point was set to 512 ps.

To reduce the noise present in the reconstructed decays, different types of smoothing were employed. In spectral reconstruction, the measured spectral range was split into a set number of reconstructed frames, where the intensities within each frame were averaged, i.e., more frames for reconstruction led to narrowed bands. It is worth stressing that the averaging did not necessarily decrease the spectral resolution. For instance, the used spectrometer with 0.25 nm/channel sampling featured the actual spectral resolution of 3 nm.

In the temporal configuration, a moving mean over 5-10 reconstructed images was used. The span of temporal averaging used was dependent on the PL decay curve lifetime, meaning shorter decays had fewer frames averaged so that no useful information about the dynamics was lost.

Testing samples have been characterized for both HSI and FLIM configurations to evaluate the quality of measurement and reconstruction. Spectral reference was carried out by independent measurement of a transmission and reflection spectrometer Photon RT.

FLIM was verified using our setup in the non-imaging mode and by using a standard FLIM setup at the University of Bergen (UiB). Here, two-photon excitation via femtosecond pulses (140 fs, wavelength-tunable between 690 and 1040 nm, set to 810 nm for our experiments, repetition rate 80 MHz) was used to trigger PL decay. The intensity was controlled using an electro-optical modulator. The light was brought through a set of mirrors into an inverted Leica TCS SP5 microscope to illuminate the sample. PL was detected using a photomultiplier and combined with a clocking signal in TCSPC module SPC-830, which was mapped by a continuous line scanning. The temporal binning of the system for the presented measurements was 64 ps. Due to a limitation imposed by the laser repetition rate, samples with a microsecond PL component could not have been measured. Therefore, the standard FLIM setup was used to measure and compare fast-decaying laser dyes [34,35].

3. Results and discussion

As the initial tests, we carried out HSI and FLIM of a homogeneous sample, where the local transmittance and PL decay had to agree with the values measured by a standard characterization method. As testing samples, we used a monocrystalline luminophore LuAG:Ce and a solution of laser dye Rhodamine B in Ethanol, 10−3 M concentration. In Fig. 2(A), we compare transmission spectra for LuAG:Ce doped monocrystal and Rhodamine B in a quartz cuvette with a 1 mm optical path. The spectrum gathered from a Photon RT system in both cases is in good agreement with our reconstructed ones with respect to the position of the absorption band and their shape. Nevertheless, we can spot that the values differ by about 5-10%, which we ascribe to the noise present in the reconstruction and selected light source.

 figure: Fig. 2.

Fig. 2. Calibration tests for homogeneous samples. (A-B) Comparison of the reconstructed and measured absorption spectrum of luminophore crystal LuAG:Ce (panel A) and laser dye Rhodamine B (panel B). The blue line – transmission measured using Photon RT, the red line – curve extracted from a group of pixels in the reconstructed image, compression factor M/N = 0.25. (C) Decays reconstructed for two randomly selected pixels of PL image of Fluorescein laser dye (scattered points), and their respective monoexponential fits (solid lines) used to determine PL lifetime. (D) Comparison of histograms of lifetime distribution in images of Fluorescein dye from compressive-FLIM setup (orange, compression ratio M/N = 0.25) and standard FLIM setup (blue) (E) Correlation between fitted intensity and lifetime for Fluorescein dye measurement. With increasing pixel intensity, the lifetime values converge to the mean value.

Download Full Size | PDF

For the initial FLIM tests, we used a solution of Fluorescein laser dye in Ethanol, 10−3 M concentration, which was measured both at UiB FLIM and our setup. In both cases, we fitted each pixel with a single-exponential decay with an offset considering the background signal. In Fig. 2(C), we display decay curves reconstructed from the compressed imaging measurement of the tested dye and the fitted curve. We picked two random pixels (different colors in Fig. 2(C)) from the reconstructed datacube to illustrate the quality of the data used for lifetime analysis. The agreement between the fit and the curve justified using the simple single-exponential function.

Based on the fitted lifetimes, we compare in Fig. 2(D) the histograms constructed from lifetimes fitted to the compressive imaging (orange histogram) and UiB FLIM setup (blue histogram). The methodology follows the steps described in the previous publications [30,31]. Histograms provide an insight into the mean values and the actual precision of the method, which was (3.9 ± 0.3) ns compared to (3.54 ± 0.13) ns measured using the UiB FLIM setup. We confirmed that, within the measurement uncertainty, the mean values agree. A minor shift of lifetimes within the experimental error can be ascribed to the fact that the two experiments – compressive FLIM and standard FLIM – used a different sample thickness (1 mm thick cuvette and less than 0.1 mm layer of dye, respectively). Reabsorption of the light in dyes leads to seemingly longer PL lifetimes [32] and can cause, in agreement with our results, an increase in the measured PL lifetime for the thicker cuvette (compressive FLIM).

In the measurement presented in Fig. 2(D), the compressive FLIM setup suffers from a 2-times higher lifetime error compared to the standard FLIM. Nevertheless, it is worth stressing that the lifetimes are retrieved from an underdetermined dataset (25%). At the same time, we observed that the PL lifetime could be very precisely reconstructed for simple scenes, even for a very low number of measurements. Namely, we attained for the same measured sample a lifetime of (3.87 ± 0.15) ns for only 5% of the measured data and the same reconstruction parameters. This is caused by the fact that the TVAL3 algorithm tends for a lower compression ratio, i.e., a lower number of measurements, to smooth the image, which in turn smooths the PL decay curves. This results in a significantly lower standard deviation for a low number of measurements (0.15 ns for 5% compared to 0.27 ns for 25%) as the picture is smoothened substantially. Therefore, the compression ratio can be well optimized based on the level of details required in the image or the accuracy with which we need to determine the PL lifetime of the sample.

While the PL lifetime of the studied laser dye does not depend on the intensity for the used excitation conditions, we cannot a-priori exclude the possibility that the reconstruction algorithm might distort the PL lifetime value. Therefore, we tested whether the reconstructed PL lifetime remains constant for all excitation intensities. We utilized the fact that we measured a highly homogeneous sample with an uneven excitation spot. Therefore, we could extract the dependence of the PL lifetime in each pixel depending on the local PL intensity. The resulting dependence – see Fig. 2 E – demonstrates that the mean value across the whole range remains the same, while the PL lifetime error decreases with the increasing PL intensity, as it would be expected for the standard measurement.

After the initial tests of the spectral and PL lifetime reconstruction, we created a simple composite sample to demonstrate the system's imaging capabilities in both configurations. The sample schematically depicted in Fig. 3(A) consisted of a quartz cuvette containing Rhodamine B (the same solution as in initial spectral testing) being positioned so that half of the measured field of view was covered by the laser dye while the other half was a quartz cuvette wall. LuAG:Ce monocrystal has been placed at the front side of the cuvette so that the light coming to the lower half of the image plane was transmitted through the crystal. As a result, we created four sectors, each behaving differently in spectral and PL measurement. For instance, the light in the upper right quadrant only interacted with the quartz wall of the cuvette, which has practically no absorption in the visible spectrum and is not fluorescent. Therefore, this corner should be the brightest in spectral and darkest in PL measurement.

 figure: Fig. 3.

Fig. 3. Characterization of a combined sample of Rhodamine B laser dye and LuAG:Ce luminophore in both HSI and FLIM configurations. (A) Schematic image of the sample – a quartz cuvette filled with Rhodamine B placed so that the dye covers the left half of the image plane with LuAG:Ce covering the lower half creating four distinctive sections. Note the edge of the crystal being depicted as it behaved differently from the rest of the sections. (B) Four different reconstructions of relative transmission (from 0 to 1) from the hyperspectral cube in selected wavelength. Notice the crystal edge being reconstructed as having minimal transmission in all panels due to the light being refracted on it and diverted away from the detection path. (C) Four different reconstructions of PL dynamics in various delays after excitation. Each panel is depicted in its own relative intensity scale (from 0 to 1), so the luminophore's longer-lasting, less intensive decay can be seen in the later frames despite the intensity being fractional compared to the fluorescent dye in the first frame. (D) A false-color RGB image of the sample, where the red channel represents absorption at 540 nm (peak absorption of Rhodamine B), the green channel at 460 nm (peak absorption of LuAG:Ce), and the blue channel represents the overall PL intensity of the sample. All sample images (panels B-D) feature a resolution of 64×64 pixels; a 3.5 mm × 2.6 mm field of view.

Download Full Size | PDF

HS image retrieval follows the absorption bands of the monocrystalline luminophore (absorption band peaking at 460 nm) and Rhodamine B laser dye (absorption band at 500-580 nm) – see Fig. 3(B). The dark line present for all transmission spectral images is the edge of the crystal, partly scattering the transmitted light and therefore appearing dark in all transmission measurements. The PL snapshot (Fig. 3(C)) for various delays after excitation, on the other hand, is dominated by the laser dye. Both the dye and luminophore feature very high PL emission internal quantum efficiency. Nevertheless, they differ highly in their PL lifetimes. Since the dye PL is emitted within a short temporal window (tens of nanoseconds), it is extremely bright. At the same time, the luminophore features a PL microsecond timescale, i.e., it emits a lower intensity over a more extended period. For the crystalline luminophore, we can also observe the effect of PL outcoupling, which is the most efficient on the crystal edge. In Fig. 3(D), we have created an overall false-color image combining the information about the transmission and PL lifetime of the sample in one picture. This image demonstrates the ability of our system to provide perfectly matching PL and absorption maps, which is documented by the sharp transitions for each quadrant, which is the same for all three color channels.

Figure 3 was captured in the regime where the entire DMD chip was used to impose the random pattern. Therefore, for the microscopy values, the attained field of view was large – namely 3.5 mm × 2.6 mm. A significant versatility in the system is gained owing to the implementation of the DMD. As the resolution of captured images is much lower than that of the DMD itself, the same resolution can be achieved while only using a fraction of the DMD area (using fewer micromirrors per pixel of the random mask). This effectively created a zoom effect enabling focus on specific parts of the sample. In Fig. 4(A), a USAF 1951 test pattern has been measured using different zoom factors. The zoom factor number in our case corresponds to the number of rows and columns binned into a single mask pixel, i.e., zoom factor 2 equals using 2-by-2 pixels in one mask pixel.

 figure: Fig. 4.

Fig. 4. Demonstration of system spatial resolution and compression ratios. All pictures were captured in 64-by-64 resolution. (A) USAF 1951 test pattern captured using different DMD-created zooms without changing the optical setup at 550 nm. From left to right, the DMD is used fully (16-by-12 micromirrors per pixel), in zoom factor 2 (2-by-2 micromirrors per pixel) and in zoom factor 1 (1 micromirror per pixel). The last panel shows group 6 element 2 of USAF 1951. Red rectangles show the position of the next zoom in the previous image. All three panels use a compression ratio M/N = 0.5. (B) Decreasing compression ratio for the image made with zoom factor 1.

Download Full Size | PDF

In Fig. 4(A), the left panel depicts the USAF target measured by the entire DMD chip. Note that since the image uses 64×64 pixels only, some lines are not resolved. However, when we use a zoom 2 (middle panel) or zoom 1 (right panel), we can clearly distinguish lines in group 6 element 2, where the line width corresponds to 6.96 µm.

While the system is able to show even smaller elements, as evidenced by the middle panel in 4A, we have selected this USAF element to further illustrate the effect of compression ratios on the reconstructed image – see Fig. 4(B). A lower compression ratio causes a loss in image quality but also allows for shorter measurement times and data volumes. With M/N being only 5%, USAF pattern stripes of the studied element are already notable, even though such measurement was using ten times fewer data than in Fig. 4(A). With very simple scenes or samples, such a compression ratio might be used to gather the necessary information.

To test the performance of our system in the high magnification regime, we used real-life biological samples. We selected a sample of Convallaria Majalis rhizome prepared by Leica company as a test target for fluorescence microscopes. The thin cut allowed us to observe individual cells, providing an interesting structure for the compressed imaging to reconstruct. With the resolution still being 64-by-64, only 128-by-128 micromirrors (zoom factor 2) have been used compared to the full DMD micromirror array (1024-by-768), creating a field of view of approximately 430 µm × 430 µm. In Fig. 5(A), the rhizome image for selected wavelengths from the hyperspectral measurement is depicted. While the cell volume itself absorbs a relatively low amount of light, the cell walls act similarly to the edge of the crystal in the previous example (Fig. 3(B)), as they refract light from the optical path and appear dark to the detection.

 figure: Fig. 5.

Fig. 5. Characterization of Convallaria Majalis rhizome cut stained by a fluorescent dye. The sample was positioned so that the image's upper left corner was uncovered. A smaller area of the DMD was used, effectively acting as a zoom. All sample images feature a resolution of 64×64 pixels. (A) Relative transmission in selected wavelengths. Notice the darker appearance of cell walls due to dispersion, similar to the crystal edge in Fig. 3(B). (B) FLIM measurement, relative intensity throughout the sample in the upper panel, and results of two-exponential fit in lower panels – lifetime components and their weight factors. (C) Correlation between reconstructed intensity and lifetime components for each pixel. Values for both components converge to their respective mean values with increasing intensity. This can also be seen in section 5B, where the highest lifetime values are in pixels with the lowest relative intensities. Section 5D shows a false RGB image, where red and green channels represent transmissions at 440 and 640 nm, respectively. The blue channel represents fluorescence intensity from FLIM measurement.

Download Full Size | PDF

In Fig. 5(B), the PL decay analysis of the reconstructed sample image is shown. As the rhizome contained several emitting species, we fitted the decay as a two-exponential decay, which can describe reasonably well with respect to the fit residuals of the PL dynamics. On the left side, the percentage weights of both components throughout the image are depicted, a1 being around 85% and a2 around 15%. On the right side, values of these components for each pixel are shown, τ1 being about 0.6 ns and τ2 around 3 ns. The lifetime τ1 was relatively close to the binning of decay dynamics (512 ps). Nevertheless, since the first component is dominating the PL decay, the dynamics covered a delay range significantly exceeding the binning interval. Therefore, it was possible to reliably extract a consistent value for high-intensity spots across the figure – see Fig. 5(C). Figure 5(C) shows the correlation between lifetime components and intensity for each pixel. The values for both components can be seen converging to their respective values as the intensity increases. As expected, the more light we could gather from a spot of the sample, the better the estimated decay dynamics are, which shows the importance of a high signal-to-noise ratio for a reliable lifetime retrieval

The data collected from the same sample using a standard FLIM microscope (τ1= 0.5 ± 0.3 ns, a1 = 72 ± 15%, τ2 = 1.7 ± 0.5 ns, a2 = 28 ± 13%), are in accord with the compressive FLIM within the confidence intervals. We ascribe the higher mean value of τ2 for the compressive setup to the noise caused by the lower overall intensity of the respective measurement.

This is caused by the fact that we effectively created a zoomed image by using only a fraction of the DMD area. The light intensity is therefore reduced compared to the previous cases (almost 50 times) and induces an increased noise level in the reconstruction. Nevertheless, we can clearly identify the cells – see Fig. 5(D) – and study the connected PL dynamics.

4. Conclusion

We built a simple and versatile system capable of measuring both the hyperspectral information and fluorescence decay lifetimes based on the standard non-imaging spectrometer and TCSPC instrumentation. With a few simple steps, it is possible to switch between the HS and FLIM configurations and acquire maps, which are inherently spatially matched. At the same time, our experiment allows, without any change in the setup, switching imaging parameters from the large field of view (3.5 mm × 2.6 mm) up to the magnification limited only by the microscope lens (resolution of 4.8 microns at 400 nm wavelength).

The setup was tested in both hyperspectral and FLIM configurations, where we observed good agreement with the standard methods, while the compressive signal has an increased noise level compared to the standard techniques. Nevertheless, owing to the use of compressive sampling, such results were also attained by using a reduced number of acquisitions compared to the standard scanning of pixels. In our measurement, we were able to reduce the number of measurements compared to the number of pixels to 5% for very simple scenes up to 25% for complex samples. We demonstrated that the frame-by-frame reconstruction of the PL dynamics has no significant effect on the retrieved PL lifetime, which was in agreement with the standard FLIM setup.

The single-pixel configuration opens the door for a simple implementation of arbitrary optical characterization methods, such as imaging in the infrared region up to 2500 nm wavelength requiring only a suitable light source and spectrometer with no further changes to the setup. [23]

Funding

Ministerstvo Školství, Mládeže a Tělovýchovy (CZ.02.1.01/0.0/0.0/16_026/0008390); Grantová Agentura České Republiky (22-09296S).

Acknowledgments

We gratefully acknowledge the Mechanics laboratory of TOPTEC (Institute of Plasma Physics, Czech Academy of Sciences) for providing us with unique optomechanics components for the compressive imaging experiments.

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. D. L. Pavia, G. M. Lampman, G. S. Kriz, and J. A. Vyvyan, Introduction to spectroscopy, 4th ed. (Cengage Learning, 2008).

2. X. Li, R. Li, M. Wang, Y. Liu, B. Zhang, and J. Zhou, “Hyperspectral Imaging and Their Applications in the Nondestructive Quality Assessment of Fruits and Vegetables,” in Hyperspectral Imaging in Agriculture, Food and Environment, InTech, 2018.

3. S. Gutiérrez, J. Tardaguila, J. Fernández-Novales, and M. P. Diago, “On-the-go hyperspectral imaging for the in-field estimation of grape berry soluble solids and anthocyanin concentration,” Aust. J. Grape Wine Res. 25(1), 127–133 (2019). [CrossRef]  

4. P. Rizk, R. Younes, A. Ilinca, and J. Khoder, “Wind turbine blade defect detection using hyperspectral imaging,” Remote Sensing Applications: Society and Environment 22, 100522 (2021). [CrossRef]  

5. A. K. Estandarte, S. Botchway, C. Lynch, M. Yusuf, and I. Robinson, “The use of DAPI fluorescence lifetime imaging for investigating chromatin condensation in human chromosomes,” Sci. Rep. 6(1), 31417 (2016). [CrossRef]  

6. Z. Xu, Y. Jiang, and S. He, “Multi-mode Microscopic Hyperspectral Imager for the Sensing of Biological Samples,” Appl. Sci. 10(14), 4876 (2020). [CrossRef]  

7. Y. W. Wang, N. P. Reder, S. Kang, A. K. Glaser, and J. T. C. Liu, “Multiplexed optical imaging of tumor-directed nanoparticles: A review of imaging systems and approaches,” Nanotheranostics 1(4), 369–388 (2017). [CrossRef]  

8. D. Guzzi, C. Lastri, P. Marcoionni, I. Pippi, A. Barducci, and V. Nardino, “Compressive sensing and hyperspectral imaging,” in International Conference on Space Optics — ICSO 2012, Nov. 2017, p. 92.

9. G. Tsagkatakis and P. Tsakalides, “Compressed hyperspectral sensing,” Mar. 2015, p. 940307.

10. J. R. Lakowicz, Principles of Fluorescence Spectroscopy, Springer US: Boston, MA, 2006. [CrossRef]  

11. J. Junek, L. Ondič, and K. Žídek, “Random temporal laser speckles for the robust measurement of sub-microsecond photoluminescence decay,” Opt. Express 28(8), 12363 (2020). [CrossRef]  

12. J. Junek and K. Žídek, “Fluorescence lifetime imaging via spatio-temporal speckle patterns in a single-pixel camera configuration,” Opt. Express 29(4), 5538 (2021). [CrossRef]  

13. Hamamatsu Photonics K.K., “Guide to Streak Cameras,” Hamamatsu Photonics K.K., Apr. 2008. Accessed: Apr. 12, 2021. [Online]. Available: https://www.hamamatsu.com/resources/pdf/sys/SHSS0006E_STREAK.pdf

14. “Introduction to time-resolved spectroscopy with applications in biophysics and physical chemistry,” 2015. Accessed: Apr. 12, 2021. [Online]. Available: http://web.vu.lt/ff/m.vengris/images/TR_spectroscopy02.pdf

15. D. Phillips, R. C. Drake, D. V. O’Connor, and R. L. Christensen, “Time Correlated Single-Photon Counting (Tcspc) Using Laser Excitation,” Instrum. Sci. Technol. 14(3-4), 267–292 (1985). [CrossRef]  

16. L. Wei and D. Ho, “Recent Advances in Fluorescence Lifetime Analytical Microsystems: Contact Optics and CMOS Time-Resolved Electronics,” Sensors 17(12), 2800 (2017). [CrossRef]  

17. J. T. Smith, M. Ochoa, and X. Intes, “UNMIX-ME: spectral and lifetime fluorescence unmixing via deep learning,” Biomed. Opt. Express 11(7), 3857 (2020). [CrossRef]  

18. M. Ochoa, A. Rudkouskaya, R. Yao, P. Yan, M. Barroso, and X. Intes, “High compression deep learning based single-pixel hyperspectral macroscopic fluorescence lifetime imaging in vivo,” Biomed. Opt. Express 11(10), 5401 (2020). [CrossRef]  

19. P. de Beule, D. M. Owen, H. B. Manning, C. B. Talbot, J. Requejo-Isidro, C. Dunsby, J. Mcginty, R. K. P. Benninger, D. S. Elson, I. Munro, M. John Lever, P. Anand, M. A. A. Neil, and P. M. W. French, “Rapid hyperspectral fluorescence lifetime imaging,” Microsc. Res. Tech. 70(5), 481–484 (2007). [CrossRef]  

20. A. Bednarkiewicz and M. P. Whelan, “Microscopic fluorescence lifetime and hyperspectral imaging with digital micromirror illuminator,” in https://doi.org/10.1117/12.728422, Jul. 2007 6630), 66300A. doi: 10.1117/12.728422.

21. N. Radwell, K. J. Mitchell, G. M. Gibson, M. P. Edgar, R. Bowman, and M. J. Padgett, “Single-pixel infrared and visible microscope,” Optica 1(5), 285–289 (2014). [CrossRef]  

22. V. Studera, J. Bobin, M. Chahida, H. S. Mousavia, E. Candes, and M. Dahane, “Compressive fluorescence microscopy for biological and hyperspectral imaging,” Proc Natl Acad Sci U S A 109(26, Jun. 2012).

23. A. Ghezzi, A. Farina, A. Bassi, G. Valentini, I. Labanca, G. Acconcia, I. Rech, and C. D’Andrea, “Multispectral compressive fluorescence lifetime imaging microscopy with a SPAD array detector,” Opt. Lett. 46(6), 1353–1356 (2021). [CrossRef]  

24. E. J. Candes and M. B. Wakin, “An Introduction To Compressive Sampling,” IEEE Signal Process. Mag. 25(2), 21–30 (2008). [CrossRef]  

25. R. Baraniuk, “Compressive Sensing [Lecture Notes],” IEEE Signal Process. Mag. 24(4), 118–121 (2007). [CrossRef]  

26. L. Klein and K. Žídek, “Collection of micromirror-modulated light in the single-pixel broadband hyperspectral microscope,” Rev. Sci. Instrum. 91(6), 063701 (2020). [CrossRef]  

27. K. Žídek, O. Denk, and J. Hlubuček, “Lensless Photoluminescence Hyperspectral Camera Employing Random Speckle Patterns,” Sci. Rep. 7, 1 (2017). [CrossRef]  

28. A. Stern, Optical Compressive Imaging. CRC Press: Boca Raton, 2016.

29. D. Mackenzie, “Compressed Sensing Makes Every Pixel Count,” in What’s happening in the mathematical sciences 7, Providence, RI: American Mathematical Society, (2009), 114–127.

30. M. P. Edgar, G. M. Gibson, and M. J. Padgett, “Principles and prospects for single-pixel imaging,” Nat. Photonics 13(1), 13–20 (2019). [CrossRef]  

31. M. F. Duarte, M. A. Davenport, D. Takhar, J. N. Laska, T. Sun, K. F. Kelly, and R. G. Baraniuk, “Single-pixel imaging via compressive sampling,” IEEE Signal Process. Mag. 25(2), 83–91 (2008). [CrossRef]  

32. C. Li, W. Yin, and Y. Zhang, “User’s Guide for TVAL3: TV Minimization by Augmented Lagrangian and Alternating Direction Algorithms,” 2009. Accessed: Apr. 27, 2019. [Online]. Available: http://www.caam.rice.edu/

33. A. Musiienko, K. Žídek, and O. Denk, “Differential single-pixel camera enabling low-cost microscopy in near-infrared spectral region,” Opt. Express 27(4), 4562–4571 (2019). [CrossRef]  

34. A. S. Kristoffersen, S. R. Erga, B. Hamre, and Ø. Frette, “Testing fluorescence lifetime standards using two-photon excitation and time-domain instrumentation: Rhodamine B, coumarin 6 and lucifer yellow,” J. Fluoresc. 24(4), 1015–1024 (2014). [CrossRef]  

35. A. S. Kristoffersen, S. R. Erga, B. Hamre, and Ø. Frette, “Testing Fluorescence Lifetime Standards using Two-Photon Excitation and Time-Domain Instrumentation: Fluorescein, Quinine Sulfate and Green Fluorescent Protein,” J. Fluoresc. 28(5), 1065–1073 (2018). [CrossRef]  

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (5)

Fig. 1.
Fig. 1. The hyperdimensional single-pixel camera setup. A high-intensity LED (1) is used as a source for hyperspectral imaging, a pulse laser diode (2) for lifetime measurement. A sample (3) is illuminated by one of the sources and through microscope objective (4) projected on the DMD (9). For PL lifetime measurement, a filter (5) is used to block the reflected laser light. A camera (6) can be used to see the positioning of the sample. To correctly focus the setup, a flip mirror (7) can be used to reflect the light to a screen (8) positioned at the same distance as the DMD. Light modulated on the DMD is then reflected towards a collimator with a diffuser (10) and focused into an optical fiber. The fiber then brings it either into a spectrometer or a photomultiplier and a TCSPC module (11).
Fig. 2.
Fig. 2. Calibration tests for homogeneous samples. (A-B) Comparison of the reconstructed and measured absorption spectrum of luminophore crystal LuAG:Ce (panel A) and laser dye Rhodamine B (panel B). The blue line – transmission measured using Photon RT, the red line – curve extracted from a group of pixels in the reconstructed image, compression factor M/N = 0.25. (C) Decays reconstructed for two randomly selected pixels of PL image of Fluorescein laser dye (scattered points), and their respective monoexponential fits (solid lines) used to determine PL lifetime. (D) Comparison of histograms of lifetime distribution in images of Fluorescein dye from compressive-FLIM setup (orange, compression ratio M/N = 0.25) and standard FLIM setup (blue) (E) Correlation between fitted intensity and lifetime for Fluorescein dye measurement. With increasing pixel intensity, the lifetime values converge to the mean value.
Fig. 3.
Fig. 3. Characterization of a combined sample of Rhodamine B laser dye and LuAG:Ce luminophore in both HSI and FLIM configurations. (A) Schematic image of the sample – a quartz cuvette filled with Rhodamine B placed so that the dye covers the left half of the image plane with LuAG:Ce covering the lower half creating four distinctive sections. Note the edge of the crystal being depicted as it behaved differently from the rest of the sections. (B) Four different reconstructions of relative transmission (from 0 to 1) from the hyperspectral cube in selected wavelength. Notice the crystal edge being reconstructed as having minimal transmission in all panels due to the light being refracted on it and diverted away from the detection path. (C) Four different reconstructions of PL dynamics in various delays after excitation. Each panel is depicted in its own relative intensity scale (from 0 to 1), so the luminophore's longer-lasting, less intensive decay can be seen in the later frames despite the intensity being fractional compared to the fluorescent dye in the first frame. (D) A false-color RGB image of the sample, where the red channel represents absorption at 540 nm (peak absorption of Rhodamine B), the green channel at 460 nm (peak absorption of LuAG:Ce), and the blue channel represents the overall PL intensity of the sample. All sample images (panels B-D) feature a resolution of 64×64 pixels; a 3.5 mm × 2.6 mm field of view.
Fig. 4.
Fig. 4. Demonstration of system spatial resolution and compression ratios. All pictures were captured in 64-by-64 resolution. (A) USAF 1951 test pattern captured using different DMD-created zooms without changing the optical setup at 550 nm. From left to right, the DMD is used fully (16-by-12 micromirrors per pixel), in zoom factor 2 (2-by-2 micromirrors per pixel) and in zoom factor 1 (1 micromirror per pixel). The last panel shows group 6 element 2 of USAF 1951. Red rectangles show the position of the next zoom in the previous image. All three panels use a compression ratio M/N = 0.5. (B) Decreasing compression ratio for the image made with zoom factor 1.
Fig. 5.
Fig. 5. Characterization of Convallaria Majalis rhizome cut stained by a fluorescent dye. The sample was positioned so that the image's upper left corner was uncovered. A smaller area of the DMD was used, effectively acting as a zoom. All sample images feature a resolution of 64×64 pixels. (A) Relative transmission in selected wavelengths. Notice the darker appearance of cell walls due to dispersion, similar to the crystal edge in Fig. 3(B). (B) FLIM measurement, relative intensity throughout the sample in the upper panel, and results of two-exponential fit in lower panels – lifetime components and their weight factors. (C) Correlation between reconstructed intensity and lifetime components for each pixel. Values for both components converge to their respective mean values with increasing intensity. This can also be seen in section 5B, where the highest lifetime values are in pixels with the lowest relative intensities. Section 5D shows a false RGB image, where red and green channels represent transmissions at 440 and 640 nm, respectively. The blue channel represents fluorescence intensity from FLIM measurement.
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.