We demonstrate diffraction limited multiphoton imaging in a massively parallel, fully addressable time-resolved multi-beam multiphoton microscope capable of producing fluorescence lifetime images with sub-50ps temporal resolution. This imaging platform offers a significant improvement in acquisition speed over single-beam laser scanning FLIM by a factor of 64 without compromising in either the temporal or spatial resolutions of the system. We demonstrate FLIM acquisition at 500 ms with live cells expressing green fluorescent protein. The applicability of the technique to imaging protein-protein interactions in live cells is exemplified by observation of time-dependent FRET between the epidermal growth factor receptor (EGFR) and the adapter protein Grb2 following stimulation with the receptor ligand. Furthermore, ligand-dependent association of HER2-HER3 receptor tyrosine kinases was observed on a similar timescale and involved the internalisation and accumulation or receptor heterodimers within endosomes. These data demonstrate the broad applicability of this novel FLIM technique to the spatio-temporal dynamics of protein-protein interaction.
© 2015 Optical Society of America
Fluorescence lifetime imaging microscopy (FLIM) is a powerful technique for high resolution imaging of functional spatio-temporal dynamics in situ. Förster resonance energy transfer (FRET) is, by far, the most extensively studied technique for observation of protein-protein homo- and hetero-dimer interactions in intact cells [1, 2]. For intermolecular FRET, a key benefit of performing donor FLIM (when compared to intensity based ratiometric techniques), is that fluorescence-lifetime measurements of donor emission are independent of acceptor concentration and is therefore suited to imaging studies in intact cells [3–5]. Multiphoton microscopy confers additional advantages in terms of inherent optical sectioning and enhanced depth penetration for in vivo imaging [6–9]. However, the data acquisition rate for FLIM is a significant limitation in current implementations of laser scanning microscopy.
There are a number of techniques currently available in multifocal multiphoton microscopy to generate multiple scanning points. These include the use of Nipkow spinning disks , cascaded beamsplitter arrays , microlens arrays  and diffractive patterned illumination approaches . These approaches restrict flexibility in relation to light efficiency (number of beams) or the use of patterned illumination. One technique utilizing a holographic pattern projected onto a diffractive spatial light modulator (SLM), (developed originally for use in optical tweezing applications ) is gaining a wider acceptance in multifocal multiphoton beam scanning implementations .
For high precision multiphoton FLIM, time-correlated single photon counting (TCSPC) is unparalleled in its measurement accuracy [16, 17]. In terms of imaging speed, TCSPC is fundamentally limited with respect to photon counting rate, since the stochastic nature of the emission process requires that the detection rate is much less than one photon per excitation event to prevent inaccuracies in lifetime determination . Consequently, acquisition times for laser scanning FLIM are in the order of minutes, whereas many dynamic biological events occur on significantly faster timescales . In order to overcome this limitation, parallel signal acquisition using arrays of laser beams with either photomultiplier arrays [20–22] or time-gated camera detection systems [23, 24] have been employed. However, in implementations to date, accurate determination of fluorescence lifetime with large numbers of channels in parallel is limited either due to cross-talk in multi-anode PMT’s  or subject to systematic error due to measurement methodology in camera systems [21, 26]. For a single beam scanning system, count rate limitations can be abrogated by parallel detection in a multi-anode PMT . Whilst, significantly increasing frame rates, this is at the cost of significant pulse pile-up , leading to count rate dependent errors in the observed fluorescence lifetime. Furthermore, the count rates reported for in vivo imaging with a low axial and lateral resolution are unobtainable in high NA lenses without excitation saturation and significant phototoxicity and photobleaching. Thus, the ability to quantitatively measure complex biological events with high temporal resolution remains a significant challenge.
Single-photon avalanche photodiodes (SPAD) array cameras developed for microscopy and spectroscopy techniques have been designed utilizing either custom or standard CMOS technologies [27, 28]. Whilst recent studies have shown the use of custom CMOS in a parallelized 8 × 1 SPAD array for use in single molecule fluorescence correlation spectroscopy , there are issues in scalability due to electrical power requirements (6 W for 8 SPADs) . Standard CMOS SPAD arrays are better suited for creating large, power efficient (100mW for 1024 SPADs) and affordable SPAD array cameras while maintaining good (50ps) timing resolution. In previous studies, the 32 × 32 Megaframe SPAD array has been presented where each individual pixel contained separate timing circuitry . Due to the relative size of the timing circuitry and associated logic, the active region of each pixel is ~1% of the total device area. Hence, when operated as a 32 × 32 pixel widefield camera, due to the low fill factor there is a low collection efficiency .
In this paper, we demonstrate a novel multifocal multiphoton FLIM microscope (MM-FLIM) which dramatically improves the acquisition rate of high resolution fluorescence lifetime imaging by parallelizing the excitation and detection process. The system consists of a two dimensional array of ultrafast beams (generated holographically via an SLM) which is optically conjugate and precisely aligned to a SPAD array camera. By reimaging the collected fluorescence beamlets directly onto the active region of the SPADs, we optically amplify the fill factor to 100%. With each individual SPAD operating in TCSPC mode, the microscope system effectively consists of 64 individual multiphoton FLIM microscopes operating in parallel to enable high data acquisition rates. The Megaframe SPAD array acquires time-resolved data from single-photon events on 32 × 32 pixels at up to 500 kcts/sec/pixel  leading to a frame rate which scales linearly with detector number. Such high count rates generate too much data to transfer via a USB2.0 interface (effective throughput up to 35 MB/s or 280 Mbit/s) and in this paper we optimize data transfer by limiting acquisition to 64 detectors and achieve a maximum data acquisition rate of 16 Mcts/sec. This is 20 times faster than a typical TCSPC acquisition speed with a single detector (assuming photon counting rates of 1% for an 80MHz laser repetition rate). The FLIM system presented here, represents a paradigm shift in acquisition of fluorescence lifetime data, whereby there is no longer a need to make a compromise between accuracy, speed and spatial resolution. We perform a number of live cell imaging experiments to highlight the capabilities of the system.
2.1 Optical set-up
The MM-FLIM microscopy system is shown schematically in Fig. 1. For multifocal beam generation, a phase-domain spatial light modulator based, arbitrary pattern holographic projection system was implemented, using techniques originally developed for optical tweezing [14, 24] and single molecule imaging with fluorescent correlation spectroscopy .
Horizontally polarized laser light from a Ti:Sapphire Chameleon Ultra II laser was expanded and collimated to overfill a near-IR (NIR) optimized SLM (Boulder Non-linear Systems Inc.). Using a doubly weighted Gerchberg-Saxton (DWGS) iterative phase retrieval algorithm , the appropriate holographic phase pattern was calculated and projected onto an SLM, which was positioned in the Fourier plane. Coupled with the incident laser light, the required laser beamlets were generated experimentally at the focal plane. A blazed grating phase pattern was convolved with the beamlet phase pattern to efficiently project the beamlet pattern into the first order diffraction pattern. The patterned NIR light was then optically relayed through a set of x-y galvanometer scanners and transmitted through a long-pass dichroic filter onto the back aperture of a x40 1.3N.A. Plan Fluor oil immersion microscope objective [Nikon Instruments Ltd] such that the back focal plane was conjugate to the SLM. Due to issues associated with projecting a square pattern onto a circular aperture, the beam size was set so that it slightly under-filled the pupil plane of the objective. Therefore the multifocal microscope system functioned with an effective excitation N.A. of 1.2. Since the focal plane was conjugate with the image plane of the SLM, the 2D beamlet array was re-created at the focal plane and was raster scanned across the sample in beam scanning mode (using the galvanometer scanners). Two photon excited fluorescence was then collected from the focal plane by the objective, de-scanned by the galvanometer scanners and re-imaged via a dichroic mirror onto the back aperture of a x10 0.3N.A. Plan Fluor objective [Nikon] and focused onto the Megaframe SPAD array.
The SLM based beamlet generation technique offers significant flexibility in terms of near-field resolution as well as having complete programmable control of the number and positioning of each beamlet. The excitation beamlet array generated fluorescence in the microscope focal plane via a two photon absorption process and was back-projected through the system to the detection plane. For parallelized TCSPC detection, each fluorescence image of the beamlet array was focused onto a single SPAD of a detector array (Fig. 2).
To ensure optimal efficiency of detection, the fluorescent beamlet projected onto the detector aperture was significantly smaller (1.8 µm FWHM) than the active area of the SPAD (6 µm dia.). The field of view of the microscope was determined by the magnification between object and detection planes, with beamlets separated in the detection plane by the detector spacing, 50 µm. With a 40x objective (Nikon Plan Fluor Oil Immersion 1.3 NA) the field of view was 100 μm for an 8 × 8 beamlet array. Precise alignment and matching of the beamlet spacing and angular orientation onto the detector array was performed to ensure an effective fill factor of 100%. The DWGS algorithm  was implemented (incorporating direct feedback from the detector array) to compensate for poor intensity fidelity between beamlets  leading to significant improvement in uniformity of the beamlet intensity variation.
2.2 Time-resolved detection and megaframe architecture
Based on the gated ring oscillator method [35, 36], the Megaframe SPAD array camera device is composed of a low power 32 × 32 10-bit time-to-digital converter (TDC) array (with an average temporal bin size of 52.5 +/− 0.7ps for a particular 8 × 8 array of SPADs) with integrated low dark-count SPAD, having a photon detection probability of 28% at 500 nm. Each pixel is 50 µm × 50 µm in size with a SPAD active region of 6 µm diameter .
In the current architecture, up to 0.5x106 time-stamp/s are delivered to the field programmable gate array (FPGA) for each pixel. Each timestamp is 10 bits long resulting in 56 ns dynamic range covering the needs for most fluorophores and light sources commonly used in microscopy. The FPGA circuit board (Opal Kelly XEM3050) contains the Spartan 3 FPGA from Xilinx (XC3S4000) and provides a USB2 interface to the host PC. The test software included with the FPGA circuit (pipetest.exe, FrontPanel 4.2.5, Opal Kelly) board can perform ~32MB/s streaming to the host PC. The firmware was written in Verilog using Xilinx’s ISE Design Tools, System Edition, version 13.2. FPGA board and Megaframe camera are interfaced via a custom motherboard printed circuit board (PCB). The motherboard PCB provides the inputs to laser pulse and laser scanner synchronization. Laser pulse synchronization signal is fed directly into the Megaframe camera for reverse start-stop TCSPC timing. Laser scanner synchronization signals are interfaced to the FPGA.
The Megaframe was operated in either time-correlated (lifetime) or time-uncorrelated modes (intensity). On-pixel TDCs generated raw time-correlated data, which was post-processed to generate an image. Due to the small size of the SPAD active area (6 µm diameter, ~28 µm2), the focused beamlet array required careful alignment for which a protocol was developed .
2.3 Image acquisition and reconstruction
All aspects of the microscope system were controlled via custom developed software written in the LabVIEW graphical programming environment. The appropriate region of interest for each sample was first selected whilst operating the microscope in wide-field fluorescent mode. Depending on the sample, the most suitable acquisition time and beamlet pattern laser power were chosen. The MM-FLIM system functioned in either single acquire, time-lapse or z-stack imaging modes. For each individual image acquisition, the system was set up to process 32 × 32 data points for 8 × 8 detectors producing 256 × 256 pixel images. The user sets the scan period and software generates the appropriate x-y galvanometer signals and pixel trigger clock. Once initialized, the buffered data reads were saved sequentially with the data containing an embedded x- address signal to enable sorting. Processing on the Megaframe camera was controlled through firmware loaded onto an on-chip FPGA.
Data lines from columns of the Megaframe camera were de-serialized and prepared for transfer over USB2. At each pixel clock cycle, one bit of a row was transferred from the top half (32 data lines) and bottom half (32 data lines). As each time-to-digital converter (TDC) was 10 bits, 10 pixel clock cycles were needed to transfer one top-half row and one bottom half row. The deserializer provided 640 lines (containing all data for two rows) to the module which checked if the two rows currently transferred contained pixels of interest. The data rate between the Megaframe camera and the FPGA is 5 Gbits/s (64 data lines at 80 MHz). This translates to a frame rate of 500 kHz, where each frame is 1024 pixels (32 × 32). As 5 Gbits/s data rate cannot be sustained to the PC and the FPGA did not include sufficient RAM for buffering full frame transfers, our approach was to select 64 pixels and transfer these at fastest possible rate over USB2.0 bus. The TDC timestamps for selected pixels were placed in the first-in-first-out (FIFO) buffer. Each timestamp is 16 bits long to allow for 10 bits of time data, 5 bits of addressing of 64 pixels and one spare bit. Our USB2 link allowed for up to 32 MBytes/s transfer rate. In practice, this was lower due to overheads in firmware and also lower photon budget. The interface between the microscope and detector system (Megaframe, FPGA circuit board and the interface board) was via TTL triggers whereby the scanner triggered the detector on each pixel thus ensuring that histograms were calculated for the correct pixels of interest.
Before analyzing the data, it must be pre-processed to compensate for non-linearities of the time-to-digital conversion (TDC) . These manifest as:
Differential Non-Linearity (DNL) – cyclic intensity variations in time due to phases generated by the four delay-element oscillator (see Fig. 3). This was corrected individually for each pixel. The four delay-element oscillator in the pixel has 8 phases (essentially four positive transitions round the ring followed by four negative transitions). This results in a histogram with a repeating pattern of 8. By reducing the bin size to 8 for a measurement taken with a temporally uniform illumination source, one can determine an appropriate correction factor to scale each bin number.
Integral Non-Linearity (INL) - Variations in the temporal bin size between detectors as a result of the CMOS fabrication process (see Fig. 4). This was compensated using a rational resampling technique. This technique incorporated first interpolation and decimation to change the sampling rate by a rational factor, thereby rescaling the temporal data taken from each detector into the same time bin size.
Data was saved and subsequently analyzed using TRI2 lifetime analysis software .
2.4 Sample preparation of plasmids, cell lines, and cell culture conditions
The plasmids encoding human EGFR and human ErbB2 tagged at the c-terminus with enhanced green fluorescent protein (EGFP) were constructed by inserting the cDNA for the respective receptor into a modified version of the pEGFP-N3 vector (Clontech) in which the dimerisation-defficient Ala206 to Lys mutation had been incorporated into the EGFP sequence. The plasmid encoding human ErbB3 tagged at the c-terminus with mRFP1 was constructed by inserting the ErbB3 cDNA sequence upstream of mRFP1 within a modified pcDNA3.1 vector. The plasmid encoding Grb2-mCherry was a gift from Dr Michela Perani and Prof Peter Parker (King’s College London, UK).
MCF7 breast carcinoma cells were grown in DMEM medium supplemented with 10% fetal bovine serum (FBS), penicillin G (100 U/ml)/streptomycin (100 mg/ml) and 1% L-glutamine, and cultured in an atmosphere containing 5% CO2/95% air (v/v). For imaging, cells were plated into 3.5 cm diameter glass-bottom culture dishes (PAA Laboratories) 24 hours prior to transfection. For EGFR-EGFP/Grb2-mCherry and HER2-EGFP/HER3-mCherry co-expression studies cells were co-transfected using Fugene 6 (Promega, Madison,WI, USA) with 1.5 µg and 3 µg of donor and acceptor plasmids respectively. Following transfection, cells were maintained at 37°C in a humidified 5% CO2 atmosphere for 24h to allow expression of exogenous proteins. For live cell imaging, cell culture media was replaced with phenol red-free OptiMEM (Gibco), supplemented with 10% FBS and 25 mM HEPES. The microscope body was equipped with an environment chamber (Solent Scientific Ltd, UK) enabling cell cultures to be stably maintained at 37°C for the duration of imaging experiments. Human recombinant EGF and Neuregulin were purchased from PeproTech, New Jersey, USA.
3. Experimental results
3.1 Resolution measurements
To demonstrate the diffraction limited imaging performance of the microscope across all beamlets, sub-resolution (100 nm dia.) fluorescent beads were imaged. Diffraction limited performance in lateral and axial resolutions of 0.44 ± 0.01 µm and 1.03 ± 0.01 µm respectively [N = 55 PSF’s distributed over 25 SPADs] was demonstrated with an effective 1.2 N.A. (due to under-illumination of the back pupil of the 1.3 N.A. objective). In Fig. 5(a) and Fig. 5(b) lateral and axial PSFs (used to calculate the resolution) are displayed respectively for a single bead. In Fig. 5(c) and Fig. 5(d) the average resolutions for a 25 SPADs are displayed on a colour chart.
3.2 Comparing lifetime analysis techniques for image acquisition of live cells at high speed
In order to compare the accuracies of lifetime determination techniques for image acquisition of live cells at high speed, data sets of MCF-7 human carcinoma cells transfected with EGFP were imaged for 500 milliseconds and 5 seconds per frame (Fig. 6).
Levenberg-Marquardt (L-M) lifetime fitting was compared with Rapid Lifetime Determination (RLD) , commonly used for analysis in gated image intensifier (GOI) based FLIM systems. These data demonstrate that L-M fitting with TCSPC data, even at low photon numbers gives a much more accurate determination of the lifetime per pixel than RLD. It should be noted that this comparison is at Poisson limited signal/noise which is superior to GOI performance. Our data also show that frame rates in excess of 2 Hz are readily achievable with the system, without the limitations imposed by GOI based systems [41–44].
3.3 Imaging protein-protein interactions in live cells
To demonstrate the dynamic imaging capability of the system we chose to image two critical steps involved in the signalling of receptor tyrosine kinases, namely, (1) receptor dimerization and (2) receptor recruitment of adaptor protein. In both cases we chose to image signalling components of the Epidermal Growth Factor Receptor (EGFR) family of receptor tyrosine kinases and all experiments were conducted in live human breast carcinoma cells.
3.4 HER2-EGFP and HER3-mRFP1
MM-FLIM was applied to the analysis of ligand-dependent receptor heterodimerisation. We examined the interaction between the EGFR family members HER2 and HER3, given the clinical significance of this heterodimer pair . HER3, which lacks intrinsic tyrosine kinase activity, can form competent signalling complexes through heterodimerisation with other EGFR family members following activation by its ligand neuregulin. As a potent driver of the PI3K/AKT-dependent cell survival pathway this novel kinase-deficient EGFR family member is gaining greater attention as a potential therapeutic target in cancer . In particular, the role that the HER2/HER3 heterodimer plays in tumourigenesis and treatment resistance in breast cancer has become the focus of intense study. In HER2-amplified breast cancers, it is the HER2/HER3 heterodimer which is largely responsible for driving tumour growth. Poor response to HER2-targeted therapies (for instance trastuzumab) in these patients is believed, in part, to be associated with increased expression of HER3 .
For live cell imaging experiments, MCF-7 cells were transiently transfected with expression constructs encoding EGFP-tagged HER2 (HER2-EGFP) and mRFP1-tagged HER3 (HER3-mRFP1) and compared with control cells transfected with EGFP-tagged HER2 only, before and after treatment with neuregulin ligand (50 ng/ml). Cells were imaged every 15s for 40 minutes with the addition of neuregulin occurring at the 0 seconds time point. A 7 × 7 beamlet pattern was applied which gave sufficient field of view (87.5 × 87.5 µm) to illuminate small clusters of cells over the course of time-lapse experiments. The lifetimes of 6 control and test samples were measured and the FRET efficiencies calculated (Fig. 7). In Fig. 6(a) the wide-field fluorescence, MP intensity and fluorescence lifetimes of representative control and test cells are presented. Statistical analysis comparing FRET efficiencies is shown in Fig. 6(b). The donor lifetime of control cells expressing HER2-EGFP alone (2.28 ± 0.02ns) remained unchanged throughout the course of time-lapse experiments. A basal constitutive association between HER2 and HER3 was observed within the perinuclear compartment with an average FRET efficiency of 3.3 ± 1.2%. On addition of neuregulin ligand, a significant increase in this FRET interaction occurred, with FRET efficiency rising to 6.1 ± 0.9%.
Time-lapse imaging data of one cell, representative of HER2-HER3 interaction, is presented in Fig. 8. Following expression, HER2-EGFP and HER3-mRFP1 both exhibited a distinct pattern of spatio-temporal localization within MCF-7, which would be extremely challenging to reveal using convention lifetime imaging techniques. These proteins accumulated at the cell periphery and within a perinuclear compartment, the latter of which was far more pronounced in the case of HER3-mRFP1 (Fig. 8(a) and 8(b)). Following the addition of neuregulin, time-lapse imaging revealed a gradual fall in the lifetime of the HER2-EGFP donor specifically within the perinuclear compartment (from 2.21 +/− 0.02ns to 2.15 +/− 0.02 ns) due to accumulation of vesicles with short lifetimes whilst donor lifetime at the cell periphery remained relatively constant (Fig. 8(c)). These data constituted a FRET efficiency increase from 3.1 ± 1.3% to 5.7 ± 0.9% after neuregulin addition. The fall in donor lifetime, indicating an accumulation of HER2/HER3 heterodimer within a perinuclear compartment, was detectable within the first 100s of ligand addition and stabilised after approximately 7 minutes, remaining low for the remainder of the observation period. Interestingly, the lack of any detectable change in donor lifetime at the cell periphery suggests that newly formed HER2/HER3 heterodimers are rapidly internalised and targeted to subcellular compartments following neuregulin stimulation and therefore active receptor complexes are predominantly intracellular.
The diffraction limited spatial resolution afforded by the MM-FLIM system enabled lifetime changes within these discrete subcellular compartments to be monitored over the course of imaging experiments and regions of interest were selected such that lifetime changes within these compartments could be assessed (Fig. 9,Media 1).
3.5 EGFR-EGFP and Grb2-mCherry
Epidermal Growth Factor (EGF)-dependent activation of EGFR represents the archetypal mitogenic signalling pathway for receptor tyrosine kinases, coupling extracellular growth factors to Ras/MAPK activation . In addition to its role in cell proliferation, EGFR activation can promote cell survival and differentiation and also drive the profound remodelling of the cell cytoskeleton, inducing migratory and invasive phenotypes in many cell types. The adaptor protein Grb2 is likely to represent a key node coupling EGFR activation to this diverse range of biological outputs as it provides a molecular bridge between active receptor and an array of downstream signalling protein classes including exchange factors, phosphatases, ubiquitin ligases and cytoskeletal remodelling factors [48–51]. Grb2 is recruited to activated receptor via its Src Homology 2 (SH2) domain which targets specific peptide consensuses flanking a critical central phosphotyrosine residue within the receptor’s intracellular domain. Grb2 serves to establish receptor-associated signalling scaffolds through which a downstream array of enzymatic pathways are rapidly activated: indeed Ras activation, which occurs following the Grb2-dependent recruitment of the exchange factor SOS to ligand-activated EGFR, is maximal within two minutes of addition of EGF to cultured cells . Visualisation of EGF-induced EGFR/Grb2 association (which cannot be performed in a conventional signal beam scanning FLIM due to slow acquisition rates) represents an interesting challenge for testing the capabilities of the MM-FLIM system.
For live cell experiments, MCF-7 human breast carcinoma cells, transiently transfected with an EGFP fusion of EGFR (EGFR-EGFP) and a mCherry fusion of Grb2 (Grb2-mCherry) were subjected to fluorescence lifetime time-lapse imaging to assess the effects of EGF stimulation on EGFR-EGFP/Grb2- mCherry interaction. Cell imaging data was acquired for 30 minutes at 10 second acquisitions. The lifetimes of 6 control and test samples were measured and the FRET efficiencies calculated (Fig. 10). In Fig. 10(a) the widefield fluorescence, MP intensity and fluorescence lifetimes of representative control and test cells are presented. Statistical analysis comparing FRET efficiencies is shown in Fig. 10(b).
No reduction in lifetime was observed for control cells expressing EGFR-EGFP donor alone, or for cells expressing both donor and acceptor molecules where the expression of acceptor was poor indicating reduced stoichiometry for the interaction (Fig. 10). The basal donor lifetime of EGFR-EGFP/Grb2-mCherry expressing cells (2.21 ± 0.01 ns) was lower than that of control cells (2.28 ± 0.01 ns), indicating the existence of a basal constitutive association between receptor and Grb2 and a FRET efficiency of 3.1 ± 0.4%. On addition of EGF ligand, a significant increase in this FRET interaction occurred, with FRET efficiency rising to 6.1 ± 0.9%.
A continuous time lapse image data for one representative cell is presented in Fig. 11. Cell imaging data was acquired continuously for 30 minutes (10 s/frame) (Media 2). Addition of EGF (100 ng/ml) occurred at the 0 seconds time point. Analysis of the time-lapse imaging revealed a rapid increase in FRET between EGFR-EGFP and Grb2-mCherry following stimulation as measured by a reduction in the lifetime of the EGFR-tagged EGFP donor fluorophore (Fig. 11(c)). Reduction in lifetime was detectable within 75 s of EGF stimulation and was sustained for the duration of the experiments, demonstrating rapid initiation of signalling downstream of the activated receptor.
The rapid association between EGFR and Grb2 observed in these live-cell imaging studies would be significantly more difficult to capture using conventional multiphoton excitation TCSPC techniques since association would be convolved with a long acquisition time or a loss of spatial resolution by integration of signal (c.f. increase in association shown by high speed multiphoton FLIM within 75s after adding ligand in Fig. 11). These experiments highlight the major advantage afforded by the increased temporal resolution of MM-FLIM.
In order to highlight the three dimensional sectioning capabilities of the system, a 40 image series of z-sections of the cells was taken before and after the time-lapse data (Fig. 12, Media 3). Each image stack was acquired in 400 s (10s/section). These data were therefore acquired on a time-scale comparable to that required to acquire a single z-section using conventional TCSPC.
Data from Fig. 11 were reanalysed by biexponential global analysis to determine the fraction of EGFR-EGFP interacting by FRET with Grb2-mCherry. The full data set was analysed simultaneously in a manner similar to that performed by Barber et al. . The fluorescence lifetimes for the non-interacting and interacting fractions were determined as 2.28 ± 0.01 ns and 1.40 ± 0.01 ns by global fitting of the entire data set. The fractional contribution of each lifetime component was then determined for each time-point (Fig. 13, Media 4). These data show that the drop in average fluorescence lifetime for EGFR-EGFP in response to ligand stimulation in cell 2 is due to recruitment of Grb2 to ~35% of EGFR molecules. This interpretation of these FRET data is appropriate if we assume only two populations (interacting and non-interacting) are present in the data. More complex interpretations are beyond the scope of this paper and these data serve as an example for the significant advantage of acquiring TCSPC data for FLIM post analysis rather than assuming a monoexponential model.
4. Discussion and conclusions
This paper presents development of a multifocal multiphoton imaging system for high-frame rate FLIM imaging. We have demonstrated that multifocal multiphoton imaging with a 2D SPAD array is practical and offers significant speed advantages compared to single beam laser scanning or widefield FLIM microscopes. With this implementation of MM-FLIM, the minimum acquisition time for a homogeneous image scales as where N2 is the number of pixels in the image, Δt is the pixel dwell time to acquire the required 1000 photons for a satisfactory single exponential fit  and n is the number of beams. At the maximum count rate per pixel for the current system (500 kHz) and 64 beams, the minimum frame time is 2.05 s for a homogeneous sample, which is a drastic improvement when compared with 131 s required for a single beam scanning system. Often in FLIM imaging spatial resolution is sacrificed for temporal resolution and applying a 3 x 3 spatial binning reduces this acquisition further to 228 ms in the MM-FLIM. One can also fit with a lower number of photons to obtain faster acquisition rates (500 ms acquisitions) although this will affect the accuracy of the lifetime determination (Fig. 6). We have demonstrated long-term observation of living biological specimens with diffraction limited imaging performance with no significant degradation in signal to noise. This is due to the lower photon flux of excitation in each beamlet compared to a single beam scanning system .
Whilst wide-field fluorescence lifetime imaging is possible at up to video frame rates with gated image intensifiers [54,55, 56], this is not practical in a biological setting due to sample limitations (i.e. excited state fluorophore saturation), significant imaging artefacts and excitation photon flux that may be damaging to cells [19, 26, 27,29]. Moreover, the MM-FLIM system described here, provides a platform for future improvements in speed and signal-to-noise by increasing the number of beams or improving the sensor spatial resolution with lower pitch, higher fill factor SPADs. To demonstrate the capabilities of the system we have performed quantitative imaging of protein-protein interactions of the EGFR family signalling network within living cells at unprecedented spatial and temporal resolutions for time-domain FLIM. We observe the gradual accumulation of these HER2/HER3 receptor complexes within a discrete perinuclear compartment highlighting the spatial resolution of the MM-FLIM system. We show that the Grb2 adaptor protein is recruited to EGFR receptor within 75 s of cell stimulation with EGF, an observation that would be challenging with conventional TCSPC systems. In addition, we are able to visualise the rapid heterodimerisation of HER2 and HER3 receptor tyrosine kinases in response to cell stimulation with the ligand neuregulin. We believe this technology has the potential to transform functional FLIM/FRET imaging and other FLIM applications, to enable dynamic spatio-temporal protein interactions to be studied quantitatively for the first time in situ.
The authors would like to thank: The Biotechnology and Biological Sciences Research Council UK (BB/I022074/1 and BB/I022937/1); STMicroelectronics (Imaging Division, Edinburgh, UK); Cancer Research UK, Engineering and Physical Sciences Research Council (UK), Medical Research Council (UK) and Department of Health (UK), KCL/UCL Comprehensive Cancer Imaging Centre; Dimbleby Cancer Care endowment fund to King’s College London and The Royal Society. The Megaframe project has been supported by the European Community within the Sixth Framework Programme IST FET Open, http://www.megaframe.eu.
References and links
1. J. R. Morris, C. Boutell, M. Keppler, R. Densham, D. Weekes, A. Alamshah, L. Butler, Y. Galanty, L. Pangon, T. Kiuchi, T. Ng, and E. Solomon, “The SUMO modification pathway is involved in the BRCA1 response to genotoxic stress,” Nature 462(7275), 886–890 (2009). [CrossRef] [PubMed]
2. M. Peter, S. M. Ameer-Beg, M. K. Hughes, M. D. Keppler, S. Prag, M. Marsh, B. Vojnovic, and T. Ng, “Multiphoton-FLIM quantification of the EGFP-mRFP1 FRET pair for localization of membrane receptor-kinase interactions,” Biophys. J. 88(2), 1224–1237 (2005). [CrossRef] [PubMed]
5. S. Pelet, P. T. So, and M. J. Previte, “Comparing the quantification of Förster resonance energy transfer measurement accuracies based on intensity, spectral, and lifetime imaging,” J. Biomed. Opt. 11, 034017 (2006).
7. E. B. Brown, R. B. Campbell, Y. Tsuzuki, L. Xu, P. Carmeliet, D. Fukumura, and R. K. Jain, “In vivo measurement of gene expression, angiogenesis and physiological function in tumors using multiphoton laser scanning microscopy,” Nat. Med. 7(9), 1069 (2001). [CrossRef]
12. A. Buist, M. Müller, J. Squier, and G. Brakenhoff, “Real time two-photon absorption microscopy using multi point excitation,” J. Microsc. 192(2), 217–226 (1998). [CrossRef]
13. L. Sacconi, E. Froner, R. Antolini, M. R. Taghizadeh, A. Choudhury, and F. S. Pavone, “Multiphoton multifocal microscopy exploiting a diffractive optical element,” Opt. Lett. 28(20), 1918–1920 (2003). [CrossRef] [PubMed]
14. J. Leach, K. Wulff, G. Sinclair, P. Jordan, J. Courtial, L. Thomson, G. Gibson, K. Karunwi, J. Cooper, Z. J. Laczik, and M. Padgett, “Interactive approach to optical tweezers control,” Appl. Opt. 45(5), 897–903 (2006). [CrossRef] [PubMed]
15. V. Nikolenko, B. O. Watson, R. Araya, A. Woodruff, D. S. Peterka, and R. Yuste, “SLM microscopy: scanless two-photon imaging and photostimulation with spatial light modulators,” Frontiers in neural circuits 2 (2008).
16. E. Gratton, S. Breusegem, J. Sutin, Q. Ruan, and N. Barry, “Fluorescence lifetime imaging for the two-photon microscope: time-domain and frequency-domain methods,” J. Biomed. Opt. 8(3), 381–390 (2003). [CrossRef] [PubMed]
17. A. Esposito, H. C. Gerritsen, and F. S. Wouters, “Optimizing frequency-domain fluorescence lifetime sensing for high-throughput applications: photon economy and acquisition speed,” J. Opt. Soc. Am. A 24(10), 3261–3273 (2007). [CrossRef] [PubMed]
18. K. Suhling, D. McLoskey, and D. Birch, “Multiplexed single‐photon counting. II. The statistical theory of time‐correlated measurements,” Rev. Sci. Instrum. 67(6), 2238–2246 (1996). [CrossRef]
19. A. Miyawaki, O. Griesbeck, R. Heim, and R. Y. Tsien, “Dynamic and quantitative Ca2+ measurements using improved cameleons,” Proc. Natl. Acad. Sci. U.S.A. 96(5), 2135–2140 (1999). [CrossRef] [PubMed]
20. S. Kumar, C. Dunsby, P. A. A. De Beule, D. M. Owen, U. Anand, P. M. P. Lanigan, R. K. P. Benninger, D. M. Davis, M. A. A. Neil, P. Anand, C. Benham, A. Naylor, and P. M. W. French, “Multifocal multiphoton excitation and time correlated single photon counting detection for 3-D fluorescence lifetime imaging,” Opt. Express 15(20), 12548–12561 (2007). [CrossRef] [PubMed]
21. J. L. Rinnenthal, C. Börnchen, H. Radbruch, V. Andresen, A. Mossakowski, V. Siffrin, T. Seelemann, H. Spiecker, I. Moll, J. Herz, A. E. Hauser, F. Zipp, M. J. Behne, and R. Niesner, “Parallelized TCSPC for dynamic intravital fluorescence lifetime imaging: Quantifying neuronal dysfunction in neuroinflammation,” PLoS ONE 8(4), e60100 (2013). [CrossRef] [PubMed]
22. D. McLoskey, D. J. Birch, A. Sanderson, K. Suhling, E. Welch, and P. Hicks, “Multiplexed single‐photon counting. I. A time‐correlated fluorescence lifetime camera,” Rev. Sci. Instrum. 67(6), 2228–2237 (1996). [CrossRef]
23. D. M. Grant, J. McGinty, E. J. McGhee, T. D. Bunney, D. M. Owen, C. B. Talbot, W. Zhang, S. Kumar, I. Munro, P. M. Lanigan, G. T. Kennedy, C. Dunsby, A. I. Magee, P. Courtney, M. Katan, M. A. Neil, and P. M. French, “High speed optically sectioned fluorescence lifetime imaging permits study of live cell signaling events,” Opt. Express 15(24), 15656–15673 (2007). [CrossRef] [PubMed]
24. S. Padilla-Parra, N. Audugé, M. Coppey-Moisan, and M. Tramier, “Quantitative FRET analysis by fast acquisition time domain FLIM at high spatial resolution in living cells,” Biophys. J. 95(6), 2976–2988 (2008). [CrossRef] [PubMed]
25. Y. Enari, K. Hayasaka, T. Hokuue, K. Inami, T. Ohshima, N. Sato, M. Akatsu, S. Kawakami, Y. Miyabayashi, H. Tokuda, H. Yanase, H. Shimoi, and T. Fujimori, “Cross-talk of a multi-anode PMT and attainment of a TOF counter,” Nucl. Instrum. Methods Phys. Res. A 547(2-3), 490–503 (2005). [CrossRef]
26. Q. S. Hanley, V. Subramaniam, D. J. Arndt-Jovin, and T. M. Jovin, “Fluorescence lifetime imaging: multi-point calibration, minimum resolvable differences, and artifact suppression,” Cytometry 43(4), 248–260 (2001). [CrossRef] [PubMed]
27. X. Michalet, R. A. Colyer, G. Scalia, A. Ingargiola, R. Lin, J. E. Millaud, S. Weiss, O. H. Siegmund, A. S. Tremsin, J. V. Vallerga, A. Cheng, M. Levi, D. Aharoni, K. Arisaka, F. Villa, F. Guerrieri, F. Panzeri, I. Rech, A. Gulinatti, F. Zappa, M. Ghioni, and S. Cova, “Development of new photon-counting detectors for single-molecule fluorescence microscopy,” Philos. Trans. R. Soc. Lond. B Biol. Sci. 368(1611), 20120035 (2013). [CrossRef] [PubMed]
29. R. A. Colyer, G. Scalia, I. Rech, A. Gulinatti, M. Ghioni, S. Cova, S. Weiss, and X. Michalet, “High-throughput FCS using an LCOS spatial light modulator and an 8 × 1 SPAD array,” Biomed. Opt. Express 1(5), 1408–1431 (2010). [CrossRef] [PubMed]
30. S. Antonioli, L. Miari, A. Cuccato, M. Crotti, I. Rech, and M. Ghioni, “8-channel acquisition system for time-correlated single-photon counting,” Rev. Sci. Instrum. 84(6), 064705 (2013). [CrossRef] [PubMed]
31. J. Richardson, R. Walker, L. Grant, D. Stoppa, F. Borghetti, E. Charbon, M. Gersbach, and R. K. Henderson, “A 32× 32 50ps resolution 10 bit time to digital converter array in 130nm CMOS for time correlated imaging,” in Custom Integrated Circuits Conference, 2009. CICC'09. IEEE(IEEE, 2009), pp. 77–80.
32. D. D.-U. Li, J. Arlt, D. Tyndall, R. Walker, J. Richardson, D. Stoppa, E. Charbon, and R. K. Henderson, “Video-rate fluorescence lifetime imaging camera with CMOS single-photon avalanche diode arrays and high-speed imaging algorithm,” J. Biomed. Opt. 16, 096012 (2011).
33. S. P. Poland, N. Krstajić, R. D. Knight, R. K. Henderson, and S. M. Ameer-Beg, “Development of a doubly weighted Gerchberg-Saxton algorithm for use in multibeam imaging applications,” Opt. Lett. 39(8), 2431–2434 (2014). [CrossRef] [PubMed]
34. G. C. Spalding, J. Courtial, and R. D. Leonardo, Holographic Optical Tweezers, Structured Light and Its Applications, D. L. Andrews, ed.(Academic Press, 2008) pp. 139–168 (2008).
35. B. M. Helal, M. Straayer, G.-Y. Wei, and M. H. Perrott, “A low jitter 1.6 GHz multiplying DLL utilizing a scrambling time-to-digital converter and digital correlation,” in VLSI Circuits, 2007 IEEE Symposium on(IEEE, 2007), pp. 166–167. [CrossRef]
36. V. De Heyn, G. Van der Plas, J. Ryckaert, and J. Craninckx, “A fast start-up 3GHz–10GHz digitally controlled oscillator for UWB impulse radio in 90nm CMOS,” in Solid State Circuits Conference, 2007. ESSCIRC 2007. 33rd European(IEEE, 2007), pp. 484–487.
37. S. P. Poland, S. Coelho, N. Krstajić, D. Tyndall, R. Walker, J. Monypenny, D. D. Li, R. Henderson, and S. Ameer-Beg, “Development of a fast TCSPC FLIM-FRET imaging system,” in SPIE BiOS(International Society for Optics and Photonics, 2013), pp. 85880X–85880X–85888.
38. J. Kalisz, “Review of methods for time interval measurements with picosecond resolution,” Metrologia 41(1), 17–32 (2004). [CrossRef]
39. P. Barber, S. Ameer-Beg, J. Gilbey, L. Carlin, M. Keppler, T. Ng, and B. Vojnovic, “Multiphoton time-domain fluorescence lifetime imaging microscopy: practical application to protein–protein interactions using global analysis,” J. R. Soc. Interface 6(Suppl_1), S93–S105 (2009). [CrossRef]
40. K. K. Sharman, A. Periasamy, H. Ashworth, and J. N. Demas, “Error analysis of the rapid lifetime determination method for double-exponential decays and new windowing schemes,” Anal. Chem. 71(5), 947–952 (1999). [CrossRef] [PubMed]
41. E. Van Munster and T. W. Gadella, “φFLIM: a new method to avoid aliasing in frequency‐domain fluorescence lifetime imaging microscopy,” J Microsc. 213(1), 29–38 (2004). [CrossRef]
42. E. B. van Munster and T. W. Gadella Jr., “Suppression of photobleaching-induced artifacts in frequency-domain FLIM by permutation of the recording order,” Cytometry A 58(2), 185–194 (2004). [CrossRef] [PubMed]
43. Q. S. Hanley, K. A. Lidke, R. Heintzmann, D. J. Arndt-Jovin, and T. M. Jovin, “Fluorescence lifetime imaging in an optically sectioning programmable array microscope (PAM),” Cytometry A 67(2), 112–118 (2005). [CrossRef] [PubMed]
46. Y. Park, H. Jung, M. Choi, W. Chang, Y. Choi, I. Do, J. Ahn, and Y. Im, “Role of HER3 expression and PTEN loss in patients with HER2-overexpressing metastatic breast cancer (MBC) who received taxane plus trastuzumab treatment,” Br. J. Cancer 110(2), 384–391 (2014). [PubMed]
47. S. E. Egan, B. W. Giddings, M. W. Brooks, L. Buday, A. M. Sizeland, and R. A. Weinberg, “Association of Sos Ras exchange protein with Grb2 is implicated in tyrosine kinase signal transduction and transformation,” Nature 363(6424), 45–51 (1993). [CrossRef] [PubMed]
48. J. P. Olivier, T. Raabe, M. Henkemeyer, B. Dickson, G. Mbamalu, B. Margolis, J. Schlessinger, E. Hafen, and T. Pawson, “A Drosophila SH2-SH3 adaptor protein implicated in coupling the sevenless tyrosine kinase to an activator of Ras guanine nucleotide exchange, Sos,” Cell 73(1), 179–191 (1993). [CrossRef] [PubMed]
49. L. Wong and G. R. Johnson, “Epidermal growth factor induces coupling of protein-tyrosine phosphatase 1D to GRB2 via the COOH-terminal SH3 domain of GRB2,” J. Biol. Chem. 271(35), 20981–20984 (1996). [CrossRef] [PubMed]
50. N. Li, M. Lorinczi, K. Ireton, and L. A. Elferink, “Specific Grb2-mediated interactions regulate clathrin-dependent endocytosis of the cMet-tyrosine kinase,” J. Biol. Chem. 282(23), 16764–16775 (2007). [CrossRef] [PubMed]
51. M.-F. Carlier, P. Nioche, I. Broutin-L’Hermite, R. Boujemaa, C. Le Clainche, C. Egile, C. Garbay, A. Ducruix, P. Sansonetti, and D. Pantaloni, “GRB2 links signaling to actin assembly by enhancing interaction of neural Wiskott-Aldrich syndrome protein (N-WASp) with actin-related protein (ARP2/3) complex,” J. Biol. Chem. 275(29), 21946–21952 (2000). [CrossRef] [PubMed]
52. S. Yamasaki, K. Nishida, Y. Yoshida, M. Itoh, M. Hibi, and T. Hirano, “Gab1 is required for EGF receptor signaling and the transformation by activated ErbB2,” Oncogene 22(10), 1546–1556 (2003). [CrossRef] [PubMed]
53. M. Köllner and J. Wolfrum, “How many photons are necessary for fluorescence-lifetime measurements?” Chem. Phys. Lett. 200(1-2), 199–204 (1992). [CrossRef]
54. P. C. Schneider and R. M. Clegg, “Rapid acquisition, analysis, and display of fluorescence lifetime-resolved images for real-time applications,” Rev. Sci. Instrum. 68(11), 4107–4119 (1997). [CrossRef]
55. J. Requejo-Isidro, J. McGinty, I. Munro, D. S. Elson, N. P. Galletly, M. J. Lever, M. A. Neil, G. W. Stamp, P. M. French, P. A. Kellett, J. D. Hares, and A. K. Dymoke-Bradshaw, “High-speed wide-field time-gated endoscopic fluorescence-lifetime imaging,” Opt. Lett. 29(19), 2249–2251 (2004). [CrossRef] [PubMed]
56. A. Agronskaia, L. Tertoolen, and H. Gerritsen, “High frame rate fluorescence lifetime imaging,” J. Phys. D Appl. Phys. 36(14), 1655–1662 (2003). [CrossRef]