1. Introduction

Fluorescence lifetime imaging microscopy (FLIM) is an optical imaging technique that characterizes the fluorescence profile of each pixel within an image by the average time between excitation and emission, termed the fluorescence lifetime [1]. Fluorescence lifetime is sensitive to the local nanoscale environment and depends on factors such as fluorophore-protein interactions, pH, ion concentration, and temperature, thus providing much more detailed information about a sample than fluorescence intensity, which is often reported in arbitrary units [2–4]. FLIM has been used in diverse applications [2,3], and notably has shown promising results for label-free characterization of reduced nicotinamide adenine dinucleotide and reduced nicotinamide adenine dinucleotide phosphate (NAD(P)H) in a wide variety of biological specimens and diseases [5–10].

An assortment of FLIM methods have been developed, primarily categorized by fluorescence detection in either the frequency- or time-domain. Frequency-domain FLIM uses a modulated light source for excitation and calculates the fluorescence lifetime of a sample from the modulation and phase shift of the emitted fluorescence [1], whereas time-domain FLIM uses a pulsed laser source for excitation and recreates the intensity vs. time histogram of photon emission, from which the fluorescence lifetime is calculated. This intensity vs. time histogram takes the shape of an exponential decay that is representative of the probability distribution of photon emission at a given time after excitation. Time-domain FLIM acquisition is generally slower than frequency-domain FLIM acquisition due to its photon counting electronics, but it is favorable for more weakly fluorescent samples and samples with multiple lifetime components [11]. Time-domain FLIM requires a pulsed laser source for excitation, such as those compatible with two-photon excitation. Two-photon excitation provides the benefits of a smaller excitation volume, improved depth sectioning, low photobleaching, and deeper tissue imaging capabilities [12], making it a robust platform for nondestructive imaging of biological specimens with sub-cellular spatial resolution. For these reasons, two-photon laser scanning time-domain FLIM systems are often used for studying a variety of biological and biomedical topics. However, two-photon FLIM applications and clinical translatability are limited by traditionally slow image acquisition and processing [2,5,11,13]. Additionally, slow processing inhibits FLIM users from getting real-time feedback on the quality of their images until after datasets have been acquired, which is costly in time, data storage, dollars, and effort.

The most common detection method for two-photon FLIM is time-correlated single photon counting (TCSPC), which generally uses a photomultiplier tube (PMT) for fluorescence detection. The signal from the PMT is sent through analog electronics that time-tag each incident photon using a constant fraction discriminator, and either a time-amplitude converter and an analog-to-digital converter or a time-digital converter [11]. TCSPC provides favorable fluorescence lifetime accuracy and time resolution (generally 20–200 ps, determined by the timing jitter of photon counting electronics) [14,15]. However, TCSPC requires a long acquisition time because photon counting electronics experience a “dead time” after photon arrival in which no other incident photon can be counted. Dead time is often on the scale of 10–100 ns [11,14,16], however recent photon counting FLIM systems can achieve dead times of less than 1 ns [17], 1.5 ns [18], and 7 ns [16] with various tradeoffs, such as sacrificing the number of time bins in each reconstructed fluorescence decay. The dead time of most TCSPC systems generally limits the maximum acceptable photon rate to about 5–10% of the excitation repetition rate of the laser, thus greatly restricting the maximum possible TCSPC image acquisition rate under ideal conditions to around 4–10 fps, depending on the image size [11,13,15,17]. Furthermore, additional frame averaging is often needed to achieve a proper photon count for accurate fluorescence lifetime estimation.

While TCSPC is a well-developed and accurate method for FLIM, the slow scanning, detection, and analysis have driven many new innovations [13,14]. Improvements in laser scanning include the use of 2D acousto-optic deflectors (AODs) to manually select areas of interest and omit scanning over background pixels [19], and the slightly more widely used technique of using a high-speed resonant galvo (RG) that provides line rates up to 8 kHz [20–24]. Innovations for FLIM detectors have recently focused on single-photon avalanche diode (SPAD) arrays, which are large arrays of solid-state detectors that have an increased maximum photon rate capability relative to a single-PMT TCSPC setup. However, compared to PMTs, SPAD arrays suffer from a relatively higher dark count rate and lower quantum efficiency (QE) over the visible spectrum, in part due to their low fill factor, which often creates the need for extensive frame averaging [25]. For example, one SPAD array-enabled wide-field FLIM system was able to achieve 10 µs exposure time but needed to average 10,000 exposures for an effective frame rate of 10 fps [26]. Another way to overcome limited photon detection rates while maintaining a favorable QE and dark count rate is to use an analog output PMT that is directly sampled with a high-speed (100 MHz – 4 GHz) digitizer, which eliminates the issues with dead time associated with photon counting electronics [3,13]. This has been demonstrated in multiple setups to achieve pixel rates of 1–20 MHz, pixel dwell times of 50–1000 ns, and frame rates up to 10–20 fps [20–24,27,28]. Analog output PMT systems have been further advanced with techniques such as the analog-mean delay (AMD) method of fluorescence lifetime estimation [20,27] and interleaved digitization to increase the number of timepoints measured for each pixel [21]. The time resolution of these systems is generally limited to about 1 ns or greater by their digitization rate and the impulse response function (IRF) of the PMT and associated electronics, such an amplifier [11,13]. High pixel rate systems are also limited by the fluorescence signal that can be collected in a short time period, which can be overcome with longer pixel dwell time, more frame averaging, higher excitation power, and brighter fluorophores. For this reason, there are only a few demonstrations of these fast analog output PMT FLIM systems for autofluorescence lifetime imaging of biological samples [22,23,27,29,30], which are limited by the lower quantum yield [11] and two-photon cross section [31] of autofluorescent molecules such NAD(P)H when compared to exogenous fluorophores. For comparison, the fluorescent dye Rhodamine B has a two-photon excitation cross-section of 210×10⁻⁵⁰ cm⁴ s/photon with 840 nm excitation, whereas NADH has a two-photon excitation cross-section of about 0.02×10⁻⁵⁰ cm⁴ s/photon with 700 nm excitation [31]. Collecting too little fluorescence, such as from having a very short pixel dwell time when imaging an autofluorescent sample, reduces fluorescence lifetime estimation accuracy; thus, when designing fast FLIM systems, one often has to make a difficult choice regarding the tradeoff between speed and accuracy.

There are many techniques for fluorescence lifetime estimation and analysis, notably least squares fitting of a single- or multi-exponential decay, and phasor analysis. Phasor analysis for fluorescence lifetime estimation, first demonstrated in 2008 [32], is one of the fastest and most useful FLIM analysis techniques [6,16,33,34]. In terms of algorithm complexity, a single-exponential decay fit and phasor analysis are both linear problems, whereas a multi-exponential decay fit requires an iterative nonlinear least squares method. However unlike a single-exponential fit, phasor analysis can also be used to analyze multi-exponential decays by decomposing a multi-exponential decay into a two component basis of frequency components, g and s, that are used to calculate the mean fluorescence lifetime. These components can be visualized on a 2D polar plot which is easily interpretable for a familiar user [32]. Fluorescence lifetime estimation is often done in post-processing and can be extremely resource intensive due to large file sizes and complicated fitting algorithms. Acceleration of different image processing algorithms has been achieved with the parallel computing capabilities of graphics processing units (GPUs) for a variety of biomedical imaging modalities [35]. Despite their computational speed, GPUs have not yet been widely employed for real-time FLIM processing and visualization, though some initial promising results have been reported for a variety of fluorescence lifetime estimation techniques [36], and also specifically for the AMD method [37]. Both of these previously reported GPU-accelerated FLIM processing methods achieved GPU processing speeds that were over an order-of-magnitude faster than central processing unit (CPU) processing alone and were able to process over 25 frames in a second. Additional work has examined GPU-accelerated processing of TCSPC data from a SPAD array which was able to achieve a throughput of 10 Gbit/s [38] and GPU-accelerated processing of multichannel digital frequency-domain FLIM [39]. Despite these recent advances, GPU-accelerated FLIM processing is still not widely used to enable real-time data analysis and visualization with high-frame rate FLIM systems.

Here, we present a versatile, fast, two-photon, time-domain FLIM system equipped with GPU-accelerated real-time fluorescence lifetime phasor analysis and display. We employ a fast analog output PMT that is amplified and directly sampled at 3.2 GHz. Using an 8 kHz RG for scanning, our system can acquire 256×256 pixel images with a pixel rate of up to 8 MHz, demonstrated by imaging of Rhodamine B uptake by cells at 25 fps with no frame averaging. The system also contains an alternate non-resonant galvo that can be used for lower frame rates and increased accuracy when imaging of autofluorescent species such as NAD(P)H in biological samples. These represent optimized aspects of fast scanning, detection, and analysis from previous FLIM systems, while also considering the need to alter acquisition rate for select samples in order to improve accuracy.

2. Methods

2.1 Optical setup

Similar to previous setups [23,29,30], this system (Fig. 1) uses an 8 kHz RG (SC30, EOPC) for fast-axis scanning and an analog output PMT (H10721-210, Hamamatsu) with a high bandwidth transimpedance amplifier (TA, C5594, Hamamatsu) that is directly sampled by a 12-bit, 3.2 GHz digitizer (ATS 9373, AlazarTech). This overcomes the slowdown in acquisition associated with photon counting since the maximum photon rate is not limited by electronic dead time, but instead is limited by the photon rate that will saturate the PMT. The system uses a mode-locked Titanium:Sapphire laser (Mai Tai HP, Spectra-Physics) with a repetition rate of 80 MHz and a tunable center wavelength; the presented data was acquired using either 750 nm (NAD(P)H) or 800 nm (Rhodamine B, Fluorescein) for two-photon excitation and second harmonic generation (SHG). The beam path contains a pair of chirped mirrors (CM, Thorlabs) to decrease the beam pulse width and increase the two-photon excitation efficiency at the sample. A photodiode (PD, Thorlabs) is used to measure the 80 MHz laser output, which is downsampled to create a 10 MHz clock to synchronize the acquisition electronics [23,29,30]. Fluorescence signal is collected using epi-detection with a 1.05 NA objective lens (OL, XLPLN25XWMP2, Olympus) specifically designed for multiphoton fluorescence and a dichroic mirror that directs the fluorescence signal to the PMT. Additionally, a filter wheel (FW) of different bandpass filters in front of the PMT allows for asynchronous collection of various emission bands.

 

Fig. 1. Video-rate fluorescence lifetime imaging microscopy (FLIM) system with real-time GPU-accelerated phasor analysis display: M, mirror; CM, chirped mirror; BS, beam splitter; PD, photodiode; HWP, half wave plate; A, attenuator; L, lens; P, pinhole; RG, resonant galvo; G, non-resonant galvo; DM, dichroic mirror; OL, objective lens; FW, filter wheel; PMT, photomultiplier tube; TA, transimpedance amplifier; CPU, central processing unit; GPU, graphics processing unit.

Download Full Size | PDF

For 256×256 pixel, 128×128 µm images, frame rates of up to 25 fps can be achieved using an RG for fast-axis scanning. At 25 fps, fluorescence decay curves are generated for each pixel by averaging the measured fluorescence intensity decay curves from 10 excitation pulses, yielding a pixel rate of 8 MHz and a pixel dwell time of 125 ns. To create a more versatile system, an alternate non-resonant fast-axis galvo (G1, Cambridge Technology) can be used to acquire images at <1 fps, collecting the fluorescence intensity decay curves from 400 excitation pulses for each pixel. This reduces the pixel rate to 0.2 MHz and increases the pixel dwell time to 5 µs. Raw data is copied from the CPU to the GPU (GeForce RTX 2080, NVIDIA), processed, and then intensity, fluorescence lifetime, and phasor components are displayed in real-time and saved, with the option of also saving raw data if the user is interested in the fluorescence decay profile of the sample.

2.2 GPU-accelerated real-time phasor analysis

As previously stated, phasor analysis of fluorescence lifetime data is a fast and useful analysis technique. Here, we present the use of a GPU to accelerate the calculation of multiphoton intensity, phasor components (g and s), and mean fluorescence lifetime (τ) of each pixel in real-time during image acquisition rates up to 25 fps (Fig. 2(a)). Additional factors such as a minimum intensity threshold for fluorescence lifetime calculation, spatial binning, and frame averaging can be input as well. Data acquisition is controlled through a custom LabVIEW (National Instruments) program, including a graphics user interface (GUI) where the resulting images and phasor plot are displayed (Fig. 2(b), Visualization 1).

 

Fig. 2. Real-time fluorescence lifetime imaging microscopy (FLIM) processing and display. (a) Graphics processing unit (GPU) and central processing unit (CPU) image processing diagram. (b) Custom LabVIEW graphical user interface (GUI) displaying the control pane (left), multiphoton intensity (top middle) and fluorescence lifetime (top right) images of the whole field-of-view (128×128 µm), along with a pixelwise phasor plot (bottom middle). A small amount of raw data from the digitizer is displayed in red (bottom right) for the user to examine as well.

Download Full Size | PDF

GPUs have parallel computing capabilities that can greatly speed up costly computations, such as high throughput image reconstruction from a large amount of raw data [35,40]. Our system quickly generates large volumes of data during imaging with a 3.2 GHz digitization rate, and is thus well-suited for GPU-accelerated processing. Data from a single pixel before and after each GPU kernel is presented in Fig. S1. Briefly, the raw data is first averaged over the assigned number of laser periods (10-400) into one single decay per pixel. This data is then normalized by subtracting a minimum value and circularly shifted so that the maximum value aligns with 0 ns. Next, spatial binning of the fluorescence decays occurs (3×3 binning was used for all data presented) to increase the collected fluorescence signal for each pixel. Then, the multiphoton microscopy intensity image is calculated as the average value of the decay curve, and a user-input intensity threshold is used to select pixels of interest for fluorescence lifetime estimation and phasor analysis, which occurs in the final kernel. Frame averaging is optional and can be used to increase signal and improve fluorescence lifetime estimation accuracy. Detailed descriptions of phasor analysis of fluorescence lifetime data can be found elsewhere [16,32]; briefly, g and s are calculated as the cosine and sine components, respectively, of the fluorescence exponential decay at the frequency of laser excitation. These values are often represented on a scatterplot with a shifted unit semicircle (Fig. 2(b), bottom middle), where single-exponential decays fall on the semicircle and multi-exponential decays fall at weighted positions within the semicircle between the multiple present fluorescence lifetimes. In phasor analysis, g and s values are normalized by the sum of the fluorescence decay, so our frame averaging uses intensity-weighted g and s components, equivalent to summing together the fluorescence decays from multiple frames and performing phasor analysis on those. After acquisition, visualization and characterization of GPU-processed images were performed using MATLAB 2019b (MathWorks).

2.3 Cell culture, preparation, imaging, and treatments

Two human epithelial breast cancer cell lines, MCF7 (ATCC HTB-22) and MDA-MB-231 (ATCC HTB-26) were used to observe Rhodamine B uptake kinetics in cell culture models. MCF7 cells were maintained in MEM-alpha medium (Gibco), and MDA-MB-231 cells were maintained in DMEM; both media were supplemented with 10% fetal bovine serum (FBS, Hyclone Laboratories), and 1% penicillin streptomycin antibiotic (PSA, Thermo Fisher Scientific), and cells were grown in an incubator at 37 °C with 5% CO₂. One day prior to imaging, cells were plated in poly-D-lysine coated 35 mm diameter glass-bottom imaging dishes (P35GC-0-10-C, MatTek) and left to adhere in the incubator overnight in 2 mL of media. Prior to imaging, media was removed and replaced with 2 mL of fresh media. Cells were imaged at ambient temperature but were not out of the incubator for more than 10 minutes prior to imaging. A stock solution of Rhodamine B (Millipore Sigma) at a concentration of 1 mM in phosphate buffer solution (PBS, Hyclone Laboratories) was used. As the FLIM acquisition was started, 100 µL of this stock solution was pipetted into the imaging dish for a final concentration of approximately 50 µM in the imaging dish. For Rhodamine B imaging, a two-photon excitation wavelength of 800 nm was used with an incident power of 10 mW, and an emission filter centered at 585 nm (FF01-585/40-25, Semrock). The uptake of Rhodamine B in cell culture has previously been demonstrated and it was suggested that Rhodamine B could be a useful probe for monitoring mitochondrial activity within cells [41].

2.4 Rat kidney ex vivo sample procurement and imaging

All procedures were performed under an approved protocol by the Institutional Animal Care and Use Committee (IACUC) at the University of Illinois at Urbana-Champaign. The kidney was harvested from a recently sacrificed rat (Sprague Dawley, ENVIGO) and bisected to expose the cortex. The fresh kidney was placed in an uncoated glass-bottom imaging dish (P35G-0-10-C, MatTek) in PBS. The sample was kept on ice for less than 30 min prior to imaging and was imaged at room temperature using 750 nm two-photon excitation and a 450 nm emission filter (FF01-450/75-25, Semrock) for NAD(P)H autofluorescence. An incident power of 30 mW was used on the sample.

3. Results

3.1 GPU timing analysis

Digitizing the PMT output at 3.2 GHz has the challenging consequence of generating multiple gigabytes of data within seconds, which limits the amount of raw data that can reasonably be acquired in an imaging session and requires considerable time to process. For this reason, the decrease in output file size from raw data (256×256×400 or 256×256×16000) to four 256×256 images (intensity, fluorescence lifetime, g, and s), enabled by GPU-accelerated processing that occurs simultaneously with acquisition, provides a significant benefit for the user in terms of time, convenience, and computer memory space. For imaging at 25 fps, raw data from a single frame was compressed 66 times; for imaging at 0.61 fps, raw data from a single frame was compressed 2640 times. With versatility in mind, the user was able to select on the GUI if they would like to save raw data, processed data, both, or neither. For imaging using either the RG or the non-resonant galvo for fast-axis scanning, the image parameters and notable metrics related to raw data size, processed data size, acquisition time, and processing time of a single frame are listed in Table 1. For both scanning schemes, the total time to copy data to and from the GPU and process data on the GPU was about 1/3 of the time to acquire a single frame.

Table 1. Single frame acquisition, processing, and timing

View Table

3.2 Fluorescence lifetime estimation performance

The fluorescence lifetimes of common fluorophores: Fluorescein (Millipore Sigma) in ethanol, Rhodamine B (Millipore Sigma) in water, and NADH (Millipore Sigma) in a pH 7.4 solution containing 1 mM HEPES, along with SHG from a beta barium borate crystal, were examined to assess the fluorescence lifetime accuracy and resolution of the system. Each dataset was acquired with the same amount of total pixel acquisition time (pixel dwell time × number of frames) via different amounts of frame averaging: 160 frames taken at 25 fps (40 laser pulses per pixel) or 4 frames taken at 0.61 fps (400 laser pulses per pixel). Previously reported fluorescence lifetime values for these samples are approximately: Fluorescein, 3.49 ns [42]; Rhodamine B, 1.74 ns [43]; NADH, 0.4 ns [43]; and SHG, 0 ns. The average fluorescence decay curves (Fig. 3(a)) and phasor analysis of images acquired at both 25 fps (Fig. 3(b)) and 0.61 fps (Fig. 3(c)) shows that species of different fluorescence lifetimes can be resolved, though there is diminished fluorescence lifetime accuracy for sub-ns lifetime fluorophores. This is due to the IRF of the system, determined from the system response to SHG, an instantaneous process. The IRF has a measured fluorescence lifetime contribution around 0.7–0.8 ns (Fig. 3) and a full width at half maximum (FWHM) of 1.61 ns (Fig. S2). To further examine system performance, we quantified the precision (τ/σ_τ) of fluorescence lifetime estimation for the dataset in Fig. 3 to estimate the number of photons and pixel acquisition time that would be needed for similar precision using an ideal TCSPC setup (assuming an F-value of 1 [44]) in Table S1. These data show that for the acquired NADH data, our system is able to acquire fluorescence lifetime data with a given precision almost 10 times faster than a TCSPC system that has a 10% maximum acceptable photon rate, yet for Fluorescein, which has a much longer fluorescence lifetime, a TCSPC system may be favorable.

 

Fig. 3. Fluorescence lifetime estimation of common fluorescent compounds at different frame rates but with the same total pixel acquisition time (pixel dwell time × number of frames). Data acquired at 25 fps used 160 frames with a pixel dwell time of 0.125 µs and data acquired at 0.61 fps used 4 frames with a pixel dwell time of 5 µs. (a) Normalized fluorescence intensity vs. time curves for second harmonic generation (SHG, black), reduced nicotinamide adenine dinucleotide (NADH, red), Rhodamine B (green), and Fluorescein (blue), at two different acquisition speeds, given in frames per second (fps). Phasor plots corresponding to the aforementioned compounds are given for (b) 25 fps and (c) 0.61 fps, separately; each point in the plot represents one pixel in the image.

Download Full Size | PDF

3.3 Dynamic imaging of cellular uptake of Rhodamine B at 25 fps

To demonstrate the video-rate imaging capabilities of the system, the uptake of Rhodamine B into human epithelial breast cancer MCF7 cells was examined with no frame averaging. A single field-of-view (FOV) was continuously imaged for 1 min to observe dynamics in intensity (Fig. 4(a)) and fluorescence lifetime (Fig. 4(b)). Visualization 2 shows the dynamic uptake throughout one minute, while only select frames in 1 s intervals are shown in Figs. 4(a)–4(c). Additionally, Visualization 1 shows how the dynamic uptake of Rhodamine B into MCF7 cells can be visualized on our custom GUI. Figure 4(d) shows the 24 frames between the 2 and 3 s of acquisition as Rhodamine B begins to penetrate the outer border of the cell cluster.

 

Fig. 4. Rhodamine B diffusion into MCF7 cells acquired at 25 frames per second (fps) for one minute. (a) Multiphoton intensity, (b) fluorescence lifetime, and the overlay of the two (c) are shown for times of 1 s through 16 s and 60 s of imaging, displayed with no frame averaging. To demonstrate the changes observable with 25 fps imaging, the overlay images taken between 2 and 3 s are shown in (d). Fluorescence lifetime is given in ns according to the colorbar. Scale bar represents 20 µm.

Download Full Size | PDF

The dynamics of Rhodamine B uptake by MCF7 cells can be further examined on the pixel level; 3×3 spatial binning is used on the raw fluorescence decay data as described in Section 2.2, but no other spatial or frame averaging was used to generate the data presented in Fig. 4 and Fig. 5. Figure 5 demonstrates the capability to examine and quantify the fluorescence lifetime and intensity kinetics of single pixels. Four pixels in Fig. 5(c) were selected for investigation: one in the extracellular space outside the cell cluster (red), one in the cytosol of an exterior cell in the cluster close to the extracellular space (green), one in the nucleus of the cell (blue), and one in the cell cytosol in closer proximity to the interior of the cell cluster (purple). The extracellular intensity and fluorescence lifetime initially sharply rise and then slowly fall; the intensity changes are due to photobleaching and uptake of Rhodamine B by cells whereas the decrease in fluorescence lifetime over time is likely due to estimation error introduced by the low fluorescence signal. The intracellular intensity and fluorescence lifetime rise more slowly and then equilibrate, though each of the selected three intracellular pixels has a different rate of change (Figs. 5(a) and (b)). The two points in the cytosol (green and purple) equilibrate around the same intensity and fluorescence lifetime values, but the interior cytosol takes longer to rise. While intensity values inside the cell, such as in the exterior cytosol and nucleus, tend to fall slightly in the final 30 s of imaging, likely due to photobleaching, the fluorescence lifetimes inside the cell stay constant since with an adequate amount of signal acquired, the estimated fluorescence lifetime will be independent of the intensity. Further quantitative analysis of fluorescence lifetime and intensity was performed by calculating the rate constant in s^-1 for each pixel based on the intensity (Fig. 5(d)) and fluorescence lifetime (Fig. 5(e)). This was performed using nonlinear least squares fitting to an exponential function (Fig. S3). To ensure that this analysis was repeatable, the data shown in Figs. 4(a)–4(c) and Figs. 5(c)–5(e) were recreated for two additional replicates of the same experimental setup with MDA-MB-231 cells (Fig. S4).

 

Fig. 5. Single-pixel kinetic analysis of Rhodamine B uptake by MCF7 cells. (a) ntensity and (b) fluorescence lifetime of four selected pixels imaged at 25 frames per second (fps) during 1 min of acquisition. Pixel locations within the image are marked in (c), an image of the average multiphoton intensity of the final 500 frames in the acquisition. The pixels chosen represent four locations with different intensity and lifetime profiles due to spatial and biochemical differences: extracellular space outside of the cell cluster (red), cell cytosol close to the extracellular space (green), nucleus of the same cell (blue), and cell cytosol closer to the interior of the cell cluster (purple). (d) Calculated rate constant for intensity kinetics for each pixel. (e) Calculated rate constant for fluorescence lifetime kinetics for each pixel. Scale bar represents 20 µm.

Download Full Size | PDF

3.4 NAD(P)H autofluorescence lifetime in a biological specimen

Autofluorescence lifetime imaging of fluorophores such as NAD(P)H provides useful information about the metabolic state of cells without the issues associated with exogenous fluorophore labelling [7,45]. To examine the capabilities of our system for autofluorescence lifetime imaging, we imaged the same FOV within the cortex of a bisected rat kidney at both 25 fps and 0.61 fps, and calculated intensity and fluorescence lifetime with different amounts of frame averaging (Figs. 6(a) and 6(b)). For images with a total pixel acquisition time (pixel dwell time × number of frames) of 40 µs, the data acquired at 25 fps has phasor values that are much closer to the origin (Fig. 6(c)), representing a larger constant component in the fluorescence decays due to noise. Using a paired t-test, the fluorescence lifetime distribution of the data acquired at 25 fps (Fig. 6(c)) was determined to be significantly lower (p < 0.001) compared to the data acquired at 0.61 fps (Fig. 6(d)). The fluorescence lifetimes achieved with 0.61 fps imaging showed a larger dynamic range and were less dominated by noise than the 25 fps images; specifically, the normalization and shifting process is susceptible to amplifying noise in low-signal data. Similar data was collected on NADH powder dissolved in a 7.4 pH solution at different concentrations to examine the effect of pixel dwell time on NADH fluorescence lifetime estimation accuracy (Fig. S5); data show NADH fluorescence lifetime is more precise with longer pixel dwell time and the detection limit of NADH in solution of the system lies between 0.1 and 1 mM.

 

Fig. 6. The effect of frame rate and frame averaging on fluorescence intensity and fluorescence lifetime of NAD(P)H autofluorescence in an ex vivo rat kidney. (a) Intensity (top, grayscale) and fluorescence lifetime (bottom, rainbow) images acquired at 25 fps with a pixel dwell time of 0.125 µs; averages of 1, 40, 160, and 320 frames are given under the corresponding total pixel acquisition time (pixel dwell time × number of frames). (b) Intensity (top, grayscale) and fluorescence lifetime (bottom, rainbow) images acquired at 0.61 fps with a pixel dwell time of 5 µs; averages of 1, 4, and 8 frames are given under the corresponding total pixel acquisition time (pixel dwell time × number of frames). Intensity and fluorescence lifetime scales indicated in the bottom left. Phasor plots corresponding to 40 µs of total pixel acquisition time (the rightmost column) are given for (c) 25 fps and (d) 0.61 fps with the mean fluorescence lifetime for each plot shown in ns. Scale bar represents 20 µm.

Download Full Size | PDF

4. Discussion

As shown in Fig. 1, Fig. 2, and Visualization 1, we have built a versatile two-photon FLIM system capable of acquiring data at variable frame rates and processing and displaying intensity and fluorescence lifetime images along with a pixelwise phasor plot in real time. This simultaneous processing and display is enabled by the parallel computing power of a GPU, which has not previously been shown for real-time FLIM analysis with GHz-rate digitization of a directly sampled PMT. Real-time processing is faster than acquisition and greatly decreases the required computer memory space (Table 1). FLIM is often limited by the bottlenecks associated with fluorescence lifetime calculation and image processing, and our system has overcome these issues with real-time GPU-accelerated processing.

However, the increased frame acquisition rate using a high-speed digitizer to directly sample the output of an analog PMT comes at the cost of fluorescence lifetime accuracy. As seen in Figs. 3(a)–3(c), low fluorescence lifetimes are overestimated because of the IRF of the system, 1.61 ns FWHM (Fig. S2), which is primarily due to the 0.57 ns rise time and the fall time of the PMT, but also affected by the timing jitter of the electronics and the 1.5 GHz bandwidth of the transimpedance amplifier. While a detector with a shorter time-response, such as a microchannel-plate (MCP), would decrease the system IRF width, the detector also needs a high sensitivity, such as a PMT or hybrid photodetector (HPD), at the target emission wavelengths to acquire enough signal for fast imaging with appropriate signal-to-noise ratio (SNR) [13]. A handful of similar fast FLIM systems use the same or a similar PMT, however there are also analog output HPDs with similar sensitivity and slightly shorter IRF, such as one demonstrated for FLIM with an IRF with a FWHM of 1.026 ns [46]. This is still significantly longer than the IRF of many TCSPC systems, so researchers interested in fluorescence lifetime accuracy at sub-ns lifetimes will likely need to resort to using photon counting FLIM systems that require longer acquisition times. However, despite this loss of low fluorescence lifetime accuracy, Figs. 3(b) and 3(c) show that the estimated fluorescence lifetimes of species that have previously been characterized as 0 ns (SHG), 0.4 ns (NADH), and 1.74 ns (Rhodamine B) [43] all have distinct fluorescence lifetime profiles and can be resolved from one another on the phasor plot. Thus, the system can still create contrast between pixels with lower fluorescence lifetime profiles and provides utility to applications that require high-speed fluorescence lifetime contrast but do not need TCSPC-level fluorescence lifetime accuracy. We additionally examined the precision of our system and the theoretical acquisition time that an ideal TCSPC system would require to match this precision (Table S1) and found that our system performed favorable for NADH and Rhodamine B, unfavorable for Fluorescein, and about equivalent for SHG. These precision values also depend on various factors such as the number of frames averaged, incident power, and fluorophore concentration, so they may not be representative of the best possible performance of our system. Nevertheless, these data show that depending on the desired speed, precision, accuracy, and target fluorophore, researchers must consider what type of FLIM system will be optimal for their experiment. Future work should expand on quantitative analysis of the photon economy of FLIM systems with analog output PMTs, similar to previous work examining both the experimental and theoretical SNR requirements and F-values of FLIM data with other acquisition schemes [44,47–49]. As more FLIM acquisition methods are explored, it will be important to provide quantitative comparisons on system performance under different conditions to enable users to choose the optimal FLIM setup for specific applications.

Figures 4 and 5 and Visualization 2 showcase the ability of the system to acquire video-rate FLIM images with no frame averaging and observe and quantify the dynamics within single pixels at 25 fps throughout 1 min of acquisition. Fluorescence lifetime is sensitive to many nanoscale environmental interactions [4], which likely accounts for the different fluorescence lifetime and intensity profiles of Rhodamine B in the cell cytosol, nucleus, and extracellular space. The calculated rate constants for fluorescence lifetime data were generally higher than those from intensity, corresponding to faster processes. This is because intensity depends on the concentration of Rhodamine B that is in the excitation volume, whereas the fluorescence lifetime depends on the local nanoscale environment and is independent of the amount of a fluorophore present. This ability to track sub-second cellular uptake dynamics and kinetics is of interest in a variety of research areas such as drug delivery [50–52] and extracellular vesicles [53]. Since fluorescence lifetime is dependent on the environment, rapid cellular uptake of fluorescent molecules or changes in the intracellular environment can be monitored efficiently by changes in fluorescence lifetime.

Many similar FLIM systems use an 8 kHz RG to significantly increase scanning speed, and the advantages of this fast scanning is seen in Fig. 4 and 5. However, for two-photon autofluorescence imaging of NAD(P)H, the inherently low quantum yield [11] and two-photon excitation cross section [31] of NAD(P)H create the need for frame averaging (Fig. 6(a)). Low signal decreases the ability to accurately calculate fluorescence lifetime, which can be seen clearly in Figs. 6(a) and 6(c) and Fig. S5. To address this, we added an extra non-resonant galvo (G1, Fig. 1) to our FLIM setup for imaging at <1 fps and collecting up to 40 times more raw data per pixel than when using the RG. This decreases the pixel rate from 8 MHz to 0.2 MHz, but provides the user with a means to collect more reliable FLIM data (Fig. 6 and Fig. S5). Photon counting systems use the number of photon counts to determine their acquisition parameters, generally aiming to collect 100 photons per pixel in order to maintain fluorescence lifetime estimation accuracy [54]. However, for systems that directly sample the fluorescence intensity decay, the fluorescence intensity required for accurate fluorescence lifetime estimation is presently not well standardized or understood. We further note that the fluorescent standards data from Fig. 3 was acquired with a high signal, which accounts for the similarity in calculated fluorescence lifetime values between the two frame rates. Future work should aim to better understand these requirements in order to ensure that these types of FLIM systems have accurate and repeatable results.

FLIM technologies are developing rapidly, however their application and usefulness with regard to biomedical problems has not increased proportionally [13]. As shown in Fig. 6, it is important to not only optimize systems for speed, but also to ensure that the system is appropriate for its intended use. When considering the examination of autofluorescence dynamics of NAD(P)H with label-free two-photon FLIM, one must keep in mind that the quantum yield of NAD(P)H is an order-of-magnitude lower [11] and the two photon excitation cross section is multiple orders-of-magnitude lower [31] than many other common fluorescent dyes and compounds; faster scanning, acquisition, and analysis cannot overcome a low number of photons produced from fluorescence while illuminating a sample with an appropriate amount of power below the photodamage threshold. Improvements in the excitation beam can also be used to increase the emitted fluorescence, such as optimization with CM (Fig. 1) or a pulse shaper [55,56], implementing adaptive optics [57], or using a lower repetition rate laser to increase the peak power while keeping an acceptable average power on the sample [28]. In addition to improving the fluorescence lifetime acquisition, a more descriptive, higher-dimensional analysis of biological samples can be achieved by combining two-photon FLIM with other optical imaging techniques in multimodal imaging systems such as optical coherence microscopy [58], or with other nonlinear optical microscopy methods, such as harmonic generation [33,59].

The direct collection of data in the time-domain enables the exploration of different strategies in image processing and fluorescence lifetime estimation. Presently, by collecting the fluorescence decay for each pixel, we are able to perform spatial binning on the fluorescence decays prior to phasor analysis, which increases the number of photons used for the fluorescence lifetime estimation for each pixel, and improves the accuracy. Furthermore, having access to the fluorescence decay will allow future work to examine IRF deconvolution methods and lifetime estimation by alternative techniques to improve accuracy. This versatility in data processing is enabled by the direct digitization in the time-domain and would be unavailable in a frequency-domain FLIM system using analog electronics to perform phasor analysis such as a previously published phase multiplexing FLIM system [60]. However, this flexibility in the potential processing techniques of our system comes with the added financial burden of our high-speed digitizer and, without GPU-accelerated imaging, would require a significant amount of data storage space for raw data files. There have also been many implementations of fast widefield frequency-domain [61,62] and time-domain [63,64] FLIM systems, which provide immense utility for imaging sub-second processes. Typically wide-field FLIM systems are not compatible with two-photon excitation, though frequency-domain multiphoton FLIM using spectral focusing has been achieved and was able to acquire up to 4 fps [65]. Previous work has compared the capabilities of frequency-and time-domain FLIM [66,67], and it would be interesting for future work to examine the differences in accuracy of our method with computational GPU-based phasor analysis to an analog electronics-based frequency-domain approach.

5. Conclusion

Fluorescence lifetime imaging microscopy is a rapidly evolving technique with much interest due to the wealth of information it provides about a fluorophore and its environment. However, fluorescence lifetime applications are often limited by long data acquisition and processing times. Here, we presented a system that overcame these challenges by using direct digitization of an amplified analog output PMT with GPU-accelerated real-time phasor analysis processing and display. The GPU processing further overcame the constraints associated with large raw data size from previous similar systems. The system can be used for imaging fast dynamics at 25 fps, and can also be used for imaging at <1 fps for more accurate imaging of fluorophores with lower quantum yields, such as NAD(P)H. This demonstrates how advances in FLIM technology must be adjusted to the specific needs of applications in order to provide more useful and accurate information.