1. INTRODUCTION

Mapping the activity of neurons across large populations and at high spatio-temporal resolution can provide insights into how neuronal ensembles collectively drive brain function [1]. Using genetically encoded fluorescent reporters [2,3], various approaches have recently been developed aimed at optically sampling large-scale neuronal activity at cellular resolution [4–31]. In some of these methods, the excitation light is scanned through the volume to sequentially sample voxels [4–12], which is often necessary when imaging within a scattering medium, but represents a fundamental trade-off between the size of the imaged volume and the acquisition speed. When imaging transparent organisms, vast improvements in speed can be achieved by relaxing the sequential acquisition in one or more dimensions by using a parallel, or partially parallel, camera-based acquisition approach that samples 2D planes [13–21] or 3D volumes [22–31].

Light-field microscopy (LFM) is a powerful approach for parallel 3D imaging in which the entire sample volume is simultaneously imaged onto the 2D camera [25–31]. LFM samples both the position and angle of the light reaching the sensor, which allows estimation of the 3D locations of fluorophores within the sample volume from the intensity recorded on the 2D sensor. Since LFM simultaneously samples the entire sample volume, it does not suffer from the trade-off between the volume size and acquisition speed. This unique aspect renders LFM a scalable approach that allows volumetric recordings at the camera frame rate. A powerful application of LFM is to image neuronal activity using genetically encoded calcium indicators, with the capacity to capture brain-wide dynamics of semi-transparent model organisms such as C. elegans and larval zebrafish at rates of up to 100 Hz [25]. The speed achievable with LFM exceeds that required for calcium imaging because of the inherent slow response of the calcium indicators, which is typically on the time scale of hundreds of milliseconds. For instance, GCaMP6s responds with a rise time of 0.2 s [3]. While conventional LFM provides exceptional speed, it trades off spatial resolution for the capacity to simultaneously image axially separated planes. Thus, when performing 3D imaging of time-varying neuronal activity, neuronal signals are typically extracted from reconstructed 3D movies using statistical methods such as independent component analysis (ICA) [25]. Alternatively, more sophisticated methods have been developed to statistically demix neuronal signals in the raw recorded data without needing to reconstruct all constituent 3D volumes, both in non-scattering media [30] and more recently in the highly scattering rodent brain [31]. While these methods allow effective demixing of the neuronal time series, they are insensitive to inactive neurons, and are compromised by any movement of the animal. Here we consider only conventional frame-by-frame reconstruction-based LFM, which would benefit from higher spatial resolution.

Here we introduce speckle LFM, a new modality of LFM that achieves higher spatial resolution using structured speckle illumination (Fig. 1). Speckle LFM achieves 1.4 times higher spatial resolution and lower background than a conventional LFM, providing better discriminability of neighboring neurons and thus further contributing to the signal demixing. While our approach achieves this by partially sacrificing acquisition speed, it is capable of 10 Hz volumetric recording of brain-wide dynamics in larval zebrafish, which is comparable with recently developed state-of-the-art methods for high-speed light-sheet microscopy [16,17].

 

Fig. 1. Speckle light-field microscopy. (a) Experimental schematic. The LFM is constructed with a microlens array placed in the primary image plane of the fluorescence microscope, which then projects onto the chip of a sCMOS camera via a 1:1 image relay (not shown). Controlled speckle illumination is introduced by imaging an SLM with a random phase mask onto the objective back-focal plane. (b) In speckle LFM, the fluorescent objects (grey disks) are illuminated with a sequence of independent speckle patterns (blue), and the resulting fluorescence is recorded. (c) The workflow of speckle and linear LFM, with example data from closely spaced fluorescent beads. The light-field images are extracted by taking the (i) variance and (ii) mean of the recorded data, for speckle and linear LFM, respectively. The use of variance offers sharper points and lower background. The source volume is then reconstructed using the PSF squared in speckle LFM (iii), or (iv) the PSF in linear LFM (see Section 2). In this example, the beads are resolved with speckle LFM but unresolved with linear LFM. (v) The square root of the speckle LFM reconstruction is then taken to renormalize it to a linear scaling with source brightness. Renormalization is not needed for linear LFM. Scale bars 2 μm.

Download Full Size | PDF

In speckle LFM, the sample is illuminated with a fluctuating sequence of speckle illuminations [Figs. 1(a) and 1(b)]. Fluorophores that are further separated than the speckle grain size experience statistically independent fluctuations of intensity, and thereby generate statistically independent fluctuations in their emitted fluorescence. These fluctuations encode an image of the object onto the variance of a recorded image sequence. Unlike the linear measurement that is based on the measured intensities, this variance image encodes nonlinear information about the sample, as it convolves the square of the fluorophore brightness with the square of the point-spread function (PSF), as discussed in Section 2. The nonlinearity in the imaging PSF provides higher resolution than the linear measurement while at the same time suppressing the background [Fig. 1(c)]. We henceforth use the term “linear LFM” to distinguish the usual linear approach from the nonlinear measurements used in speckle LFM. The principle of speckle LFM is comparable to that of super-resolution optical fluctuation imaging (SOFI), in which randomly blinking fluorophores are distinguished by their statistically independent blinking patterns [32], though speckle LFM can be applied to non-blinking fluorophores such as GCaMP.

Speckle LFM extends principles of blind structured illumination microscopy [33,34] to 3D imaging. While conventional structured illumination microscopy requires accurate prior knowledge of the illuminating field [34–38] and can degrade rapidly when applied in scattering media [37], blind structured illumination microscopy requires no knowledge of the illumination, making it immune to any scatter-induced degradation of the excitation light. This is important when imaging in vivo, where scattering and aberration of the light is unavoidable. Speckle LFM relies only on the statistical properties of speckle, which is generated whenever the spatial phase profile of an optical field is randomized, resulting in superposition of light with many random phases. As can be understood from the central limit theorem, the resulting speckle field has real and imaginary parts that are each normally distributed with zero mean but nonzero average intensity, leading to a Rayleigh distributed field amplitude. Further randomization or scattering of a speckle field does not change the speckle statistics [39].

2. THEORY

The image recorded in an LFM can be described as

This represents an image formation procedure similar to Eq. (2), but with the square of fluorophore brightness projected using the square of the PSF. The resulting variance image scales with ${\mu }^2$, so the square root of the image is taken to return a linear scaling with the sample brightness. Note that Eq. (6) becomes invalid as the resolution approaches the diffraction limit. A more elaborate analysis is therefore required to achieve super-resolution with speckle illumination in a diffraction-limited microscope, though the simple strategy described here does provide wide-field axial sectioning [39].

Our preliminary tests of speckle LFM showed that it can be challenging to achieve adequate signal-to-noise ratio (SNR). When imaging dynamic changes in neuronal populations, it is important that noisy fluctuations in the baseline can be easily distinguished from true activity. To understand the trade-offs between imaging speed, illumination brightness, and SNR, we performed Monte Carlo simulations of the detection statistics. The basic procedure first considered was to illuminate a fluorescent sample with a predetermined fluctuating sequence of speckle patterns and to then calculate the variance of the images from a bin of $N_{\mathrm{bin}}$ recorded frames. Speckle illumination was generated assuming both the real and imaginary parts of the optical amplitude to be normally distributed random variables of equal variance [41] and without noise or inefficiencies. The simulated fluorescence was the product of a stochastic speckle illumination with an artificial ground truth [Figs. 2(a) and 2(b)]. The simulated fluorescence was separated into a sequence of bins, and for each bin the sample brightness was estimated from the standard deviation of the recorded brightness. Fast temporal resolution requires the use of a small bin size, though smaller bin size reduces the SNR. An example simulated time trace is shown in Fig. 2(c), using a bin size of $N_{\mathrm{bin}}=10$ without any measurement noise or inefficiencies. When illuminating with completely random speckles, stochastic variations in the illumination lead to noise in the estimated brightness. This noise could be eliminated by using a repeating set of 10 pseudo-random speckle illuminations, such that the image brightness is repeatedly estimated using the same sequence of illuminating speckle patterns [Fig. 2(c)]. This can be understood qualitatively from the underlying statistics of the method: the source brightness is estimated from the standard deviation of the fluorescence, which in the absence of noise is the product of the ground-truth fluorophore brightness with the standard deviation of the illuminating speckles. When illuminating with completely random speckles, each estimate of fluorophore brightness employs a different sequence of speckles, which each have a different standard deviation. However, repeatedly using the same sequence of pseudorandom speckles ensures that the standard deviation of the illumination does not change between bins. Repeating illumination allows error-free estimation of relative changes in brightness ($\mathrm{\Delta }F/F$), which is the main parameter of interest in calcium imaging. However, repeating illumination does not improve estimation of the absolute fluorophore brightness, so it provides no benefit when comparing the relative brightness of different voxels. Note that the time included within one variance reconstruction is always chosen to match the repetition period of the speckle illumination. A mismatch of the period of the illumination pattern and the time interval over which the variance is calculated would result in a spurious beating signal. Thus, the reconstructed image frame rate is chosen in advance when preparing the illumination sequence.

 

Fig. 2. Monte Carlo simulation of speckle LFM detection statistics. (a) An example showing how the signal is calculated for speckle imaging; pseudo-random speckle illumination (blue) is incident on fluorophores for which the brightness can vary. (b) The resulting fluorescence (green) is the product of the fluorophore brightness and illumination. As the fluorophore brightness increases, both the mean and variance of the recorded signal increase. (c) and (d) We compare two configurations, one in which the sample is illuminated by a completely random sequence of speckle patterns, and another with a repeating pattern of 10 pseudo-random speckle patterns. We see that both (c) in the absence of shot noise, and (d) with shot noise included, the repeating pattern (blue) leads to a signal closely representing the ground truth (dashed line), while random fluctuations in the illumination introduce severe noise when using completely random speckle (light green). (e) The dependence of SNR on the mean number of photons included in each reconstruction. When using random speckle (light green), the random illumination statistics dominate the noise and preclude any improvement from increasing the intensity. A repeating pattern, however, improves consistently with photon number as expected for a shot-noise-limited measurement (blue). (f) Trade-off between SNR and reconstruction frame rate. The volume imaging rate can be increased by calculating the variance with smaller bin size, but at cost of SNR. If the intensity is fixed (blue), there is a dramatic loss of SNR with increasing image rate. Even if the illumination is increased to maintain fixed photon number in each bin (orange dashed), increasing the frame rate leads to diminishing SNR.

Download Full Size | PDF

Another important noise source is shot noise. To characterize shot noise, time traces were simulated like those in Fig. 2(a), but with each recorded signal constrained to discrete photon numbers. The photon number was calculated stochastically using Poissonian photon statistics with the mean photon number proportional to the intensity [42]. The mean photon number at the baseline fluorescence was set to 20 photons per sample. The sample brightness was then estimated from the variance of collected fluorescence over $N_{\mathrm{bin}}=10$ simulated measurements, such that the mean number of photons used in each estimate was 200 photons. With these parameters, speckle LFM with repeating speckle patterns becomes noisier than without shot noise, although the stochastic illumination is already so noisy that the added shot noise is not clearly visible [Fig. 2(d)].

To understand the relative contributions of shot noise and stochastic illumination noise, the average SNR was determined with a range of mean photon numbers, which results in a different relative contribution of shot noise [Fig. 2(e)]. With random speckle illumination, the SNR quickly saturates towards an upper bound approximately equal to $\surd N_{\mathrm{bin}}$, with no benefit from increasing photon number. As such, random illumination only permits high SNR with high $N_{\mathrm{bin}}$ and correspondingly low frame rate. When using repeating speckle illumination, the SNR increases continuously with increasing photon number $n_{\text{phot}}$, indicating that sensitivity is limited primarily by shot noise. Note that the measurement relies on estimation of the signal variance, which follows different statistics than estimation of the mean. As such, the shot-noise limit here corresponds to lower SNR than the $\surd n_{\text{phot}}$ expected for linear detection of intensity [42], and some SNR is necessarily sacrificed when using speckle illumination. To maximize SNR, speckle LFM should use repeating speckle illumination and a high photon number.

Next, we tested the relationship between SNR and the bin size. Recorded signals were simulated with a mean of 20 photons per acquisition at 100 Hz acquisition rate, corresponding to typical conditions used in our speckle LFM experiments when imaging larval zebrafish (see Methods). The variance was calculated over bin sizes varying from $N_{\mathrm{bin}}=3$ to 100, corresponding to 33 to 1 Hz volumetric imaging rate. As expected, SNR declined strongly with the imaging rate [Fig. 2(f)]. While this is partly due to the lower number of photons included at higher frame rate, we also found that the number of independent illuminations influences SNR. Increasing the frame rate reduces the SNR even when simultaneously increasing the intensity to ensure a constant number of photons $n_{\text{phot}}$ in each measurement of the variance [dashed line in Fig. 2(f), 200 photons per measurement]. The results in Fig. 2 suggest that provided we use pseudo-random repeating illumination, our experimentally achieved photon flux of approximately 2000 photons/s is suitable for 10 Hz volumetric imaging with variance calculated from $N_{\mathrm{bin}}=10$ camera frames.

3. EXPERIMENTS

To realize speckle LFM, we first constructed an LFM by placing a microlens array at the image plane of an epi-fluorescence microscope [Fig. 1(a)]. The foci of the microlenses were imaged onto the camera sensor using a 1:1 relay lens system [not shown in Fig. 1(a)]. As such, each camera pixel collected light that entered the image plane at a specific angle, thereby providing the angular information needed for 3D deconvolution. Speckle illumination was introduced using an Argon ion laser at 488 nm, which was randomized by a pseudo-random pattern applied to a phase-only spatial light modulator (SLM) that was imaged to the objective back-focal plane using relay optics (see Methods). The SLM generated a sequence of independent speckle illuminations by applying a series of random phase profiles with a camera frame acquired for each illuminating speckle pattern. The variance of each pixel was calculated across a set number of frames. The 3D volume was then reconstructed from the variance image using the Richardson–Lucy algorithm introduced in [25,27] for linear LFM, but replacing the PSF with the PSF squared to account for the nonlinear image formation (see Methods). For speckle LFM the algorithm estimates the square of the fluorophore distribution [see Eq. (6)], of which we then take the square root to return to a linear brightness scaling. To compare the speckle LFM images to linear LFM, we also reconstruct the 3D volume in the usual way from the average intensity at each pixel. This averages over the speckle fluctuations and produces an intensity image that closely approximates that achieved with the usual incoherent illumination [see Eq. (2)].

To ensure that speckle LFM allows reliable 3D imaging, we tested its performance on 2-μm fluorescent beads. The light fields recorded on the camera were compared for speckle and linear LFM, which showed that speckle LFM achieves sharper maxima with lower background, but higher noise [Fig. 1(c), i, ii]. By sharpening the effective PSF, speckle LFM effectively improves the angular resolution of the incident light. Volumetric reconstructions showed that speckle LFM accurately reproduced the 3D positions of fluorescent beads as measured using linear LFM. Speckle LFM correspondingly achieved smaller 3D image spot size than linear LFM and could resolve closely spaced beads that were unresolvable using linear LFM [Fig. 1(c), iii, iv]. Since in speckle LFM the reconstructed volume is a nonlinear estimate of the fluorophore brightness, the images were then renormalized by taking the square root [Fig. 1(c), v].

Using our speckle LFM we imaged neural activity at 10 Hz volume rate in 7-day old larval zebrafish expressing GCaMP6s (see Methods). Whole-brain imaging of the fish was performed and the acquired images reconstructed as described above using both speckle LFM and linear LFM (Fig. 3). The improved resolution of speckle LFM was clearly visible, allowing visual identification of many bright loci that were not clearly distinguished with linear LFM [Figs. 3(b) and 3(c)]. We identified regions of interest (ROIs) by taking the standard deviation in time of the dataset and then applying automated morphological segmentation to identify time-varying areas of approximately the correct size (see Methods). In this example, we identified 2989 ROIs for which activity could be clearly observed [Fig. 3(d)]. To qualitatively evaluate the differences between linear and speckle LFM, we inspected the activity recorded in closely spaced ROIs. For example, the ROIs shown in Fig. 3(e) are located 5 μm apart in the telencephalon and are not clearly resolved with linear LFM [observed dynamics shown in Fig. 3(f)]. As predicted theoretically, speckle LFM data is more noisy than linear LFM. The linear LFM data was found to have a consistently reduced $\mathrm{\Delta }F/F$ signal amplitude, which is expected, as the speckle LFM signal has a reduced background. To aid the visual comparison of speckle and linear LFM, the linear data shown in Fig. 3(f) has been scaled up by factors of 1.8 and 3.2 for ROIs 1 and 2, respectively. With this linear rescaling, the dynamics recorded with linear LFM and speckle LFM agree well at most times as represented by $>0.9$ correlation, confirming that speckle LFM can record calcium dynamics reliably. However, in some cases calcium transients were observed only with linear LFM and were absent in the speckle LFM recording [Fig. 3(f), i-iv]. The additional transients seen with linear LFM occurred simultaneously with transients on the neighboring ROI, which were observed with both speckle and linear LFM. This suggests that the additional calcium transients recorded with linear LFM originated from neighboring ROIs and were mixed into the recorded data due to the lower spatial resolution and that this problem is reduced with speckle LFM due to its improved resolution.

 

Fig. 3. Brain-wide recording of neural activity in larval zebrafish. (a) 3D projection of the standard deviation in time of the recorded movie, which thus highlights the active neurons. (b) and (c) Zoom-in on a single slice at $z=12\mathrm{\mu m}$ above the focal plane. The speckle LFM reconstruction (b) shows sharper features and more resolvable spots than the similar linear LFM data (c). (d) Brain-wide activity recorded using speckle LFM in this fish. The activity is mostly spontaneous, though increased activity is visible around 270–278 s, following a tail movement. (e) Two example ROIs from the areas indicated in (b) and (c) were selected, and their activity plotted in (f). The dynamics recorded with speckle LFM (blue) closely agrees with the linear LFM (orange) at most times. However, some transients that are observed on both ROIs with linear LFM are observed only on ROI 2 (i, ii, iv) or ROI 1 (iii) with speckle LFM. This is due to the lower spatial resolution of linear LFM partially mixing the signals. Scale bars in (a) 50 μm, (b, c) 10 μm, (e) 4 μm.

Download Full Size | PDF

To quantify crosstalk between ROI pairs we evaluated the mutual information, which quantifies the degree of common signal in a pair of separate measurements [43]. For the two example ROIs shown in Fig. 3(d), the mutual information was 2.5 bits with linear LFM, which is reduced to 2.0 bits with speckle LFM. The difference in mutual information calculated using linear LFM and speckle LFM stems purely from the differences between these two methods and is not related to any correlated activity of neurons. We found that the mutual information recorded in closely spaced ROIs was consistently higher in the linear LFM data than speckle LFM data, suggestive of measurement crosstalk arising from the limited spatial resolution of linear LFM. To systematically quantify this increased crosstalk, we evaluated the pairwise mutual information of ROI pairs that were within the same axial plane using data pooled from 9 separate data sets, recorded in 5 separate fish, and without any sensory stimulation. The average mutual information was then calculated as a function of the ROI separation for each depth relative to the focus. A constant offset was estimated by taking the average mutual information of ROIs separated by 50–100 μm and subtracted from the data. As shown in Fig. 4(a), the average mutual information is highest for ROIs that are closely separated, while at large separations there is no correlation between ROI separation and mutual information. The onset of this rise in mutual information was consistently observed at smaller separations with speckle LFM, indicating that measurement crosstalk rather than correlated activity dominated the observed rise in mutual information. Consequently, the onset of crosstalk allows quantification of the spatial resolution achieved in vivo.

 

Fig. 4. Quantification of resolution from pairwise mutual information. (a) Pairwise mutual information of ROIs as a function of lateral separation. Data is shown for two axial slices, with solid and dashed lines 24 and 84 μm above the focus, respectively. In all cases the mutual information is highest for closely spaced ROIs which cannot be completely resolved and plateaus at large separation. We define the onset of crosstalk to be the separation at which the average mutual information increases by 0.4 bits (dotted line), which provides a quantitative measure of the achievable resolution in vivo. (b) Estimated resolution in vivo. The resolution estimated from mutual information (points) is compared to the theoretically predicted resolution of the LFM (lines). Speckle LFM surpasses the resolution of linear LFM at all depths. The arrows indicate the data points extracted from the example data shown in (a). At some depths ($-36\mathrm{\mu m}$, $-24\mathrm{\mu m}$, and 12 μm) crosstalk was not statistically observable using speckle LFM; these data points are therefore omitted. The spike of poor resolution at $z=0$ is known as the artifact plane and is a well-known feature of the conventional LFM deconvolution method [25,27].

Download Full Size | PDF

While resolution generally describes the minimum separation at which points can be distinguished, the mutual information analysis here quantifies the separation at which we observe crosstalk, which occurs if there is any overlap of the images. Here the onset of crosstalk is estimated to be about twice the separation required for resolvability. We therefore estimate the spatial resolution to be half the separation at which crosstalk becomes evident, as quantified by a 0.4 bit increase in the average mutual information compared to ROI pairs that are far separated [Fig. 4(a)]. The mutual information analysis was applied to estimate the resolution of both linear LFM and speckle LFM across the entire imaged volume [Fig. 4(b)]. We found that speckle LFM achieved higher resolution than linear LFM at every recorded depth [Fig. 4(b)]. To compare the measured resolution to theory, we also calculated the expected resolution using the code introduced in [25]. This algorithm first simulates an LFM measurement and then reconstructs the source volume, with the reconstructed spot sizes providing an approximate measure of the achievable resolution. For both speckle LFM and linear LFM, the theoretically predicted resolution was found to agree well with the resolution measured in vivo using mutual information analysis [Fig. 4(b)]. Use of speckle LFM improves resolution by a factor that varies with depth with a mean enhancement of 1.4 predicted theoretically, which agrees with the factor of $2.0\pm 0.6$ extracted from experimental data. While correlated activity is expected to contribute to the observed mutual information, the agreement with the predicted resolution indicates that the increase in mutual information observed at small separations arises primarily from measurement crosstalk, both for linear LFM and speckle LFM.

We note that some recent statistical methods have been developed that can use temporal dynamics to demix and localize neurons, both in scattering [31] and non-scattering media [30]. These approaches fundamentally differ from speckle LFM, which reduces crosstalk by improving resolution and without relying on the temporal dynamics of the fluorescent signal. Further, speckle LFM estimates the locations of all fluorescent neurons while these statistical methods are only sensitive to active neurons, which in mammalian brains are estimated to comprise only 10% of the total population in typical experiments [44].

In addition to improving resolution, the results presented above show that speckle LFM achieves lower background than linear LFM. This suppression of background should apply to all sources of background, including scattered fluorescent light from the sample. The scattered fluorescent light reaching each camera pixel originates from sources across the whole illuminated volume, which in large samples contains many statistically independent illuminating patterns, so the scattered background therefore tends to average to a near-static baseline that is rejected in speckle LFM. While rejection of scattered light is not crucial in the transparent brain of larval zebrafish, scattered light and fluorescent background can be severe limiting factors when imaging in the highly scattering mammalian brain, particularly when imaging at depth. To test the capacity of speckle LFM to suppress background and scattered light in the presence of scattering, we imaged neural activity in vivo within the posterior parietal cortex (PPC) of mice (Fig. 5). Mice densely expressing GCaMP6m were imaged while startle-induced neuronal activity was evoked using air puffs (see Methods). Initial tests indicated that SNR was reduced compared to zebrafish, most likely because aberrations degrade the PSF and limit reconstruction reliability, or possibly also because scattering of the incident light tends to shrink the speckle grains to sizes below the microscope resolution, thereby reducing speckle contrast. To recover SNR, the volumes were calculated from a larger bin size of 50 camera frames, reducing the volumetric image rate to 2 Hz. The results indicate that speckle LFM suppresses background and thereby allows easier identification of neurons. Suppression of background also provides a more accurate estimate of the baseline fluorescence, which is required for accurate inference of the underlying neuronal activity [45]. Since fluorescence can be generated throughout the brain, the scattered background can exceed the ballistic signal even at superficial layers. In the example layers shown in Fig. 5, in the linear LFM measurements the background was at a level at which only a few image features other than the outlines of occluding blood vessels could be revealed [Figs. 5(d)–5(f)]. In contrast, speckle LFM could suppress the background significantly and allowed the observation of bright cells at depths up to 170 μm below the brain surface, resulting in inference of neuronal activity in 3D with improved contrast [Figs. 5(a)–5(g)]. Further developments in the illumination and reconstruction strategies are needed to recover the high speed and sensitivity achieved with zebrafish larvae.

 

Fig. 5. Speckle LFM imaging of neural activity in a mouse. 3D volumes imaged with (a)–(c) speckle LFM and (d)–(f) linear LFM, with example planes shown at (a) and (d) 80 μm, (b) and (e) 120 μm, and (c) and (f) 170 μm depth below the brain surface. This data includes a strong fluorescent background which is effectively suppressed in the speckle LFM data, allowing individual cells to be resolved which cannot be clearly distinguished from background with linear LFM. (g) Time traces from 10 example ROIs as indicated in (a)–(f). Solid lines: speckle LFM, dotted lines: linear LFM. Similar to results from fish, speckle LFM suppresses background and thereby increases the amplitude of $\mathrm{\Delta }F/F$. Scale bars 50 μm.

Download Full Size | PDF

When imaging in the mouse brain, we found that the resolution was clearly poorer than that seen in the weakly scattering zebrafish brain, which suggests that the resolution was degraded by optical scattering of the measured light. Speckle LFM appears to be as susceptible to scatter as linear LFM, and consequently we could not achieve higher resolution images with speckle LFM within the mouse brain. It is possible that the resolution and sensitivity achievable with low scattering could be extended to this scattering regime with more sophisticated volumetric reconstruction that could account for the effect of scattering on the PSF. Such an extension is beyond the scope of this work. Alternatively, structured illumination approaches based on incoherent illumination may prove to be more robust in the presence of strong scattering.

4. OUTLOOK

Here we have introduced speckle LFM, a new modality of LFM that uses speckle structured illumination to enhance resolution. We demonstrated the principle using an SLM to generate pseudorandom speckle, but it can also be constructed using different equipment. For instance, pseudorandom speckle can be readily generated at a much lower cost and with near-unlimited speed using a rotating diffuser. Such an approach can provide repeating illumination as discussed in Fig. 2 by synchronizing the rotation rate to the reconstruction frame rate, with repetitive speckle patterns generated each time the light scatters from similar regions of the diffuser. Alternatively, digital micromirror devices could also provide stable and high-speed control of the projected speckle patterns. If combined with a faster camera acquisition rate, such approaches could enable faster imaging speeds.

While our speckle LFM method is immediately applicable to functional imaging, we also note that the method has the potential to be dramatically improved with more elaborate approaches to 3D reconstruction. If the speckle pattern projected into the sample was known, one could utilize the temporal speckle sequences to reconstruct the volume with resolution limited only by the speckle grain size. As the speckle grains can be produced with sizes down to the objective diffraction limit, this could allow confocal resolution across large volumes at vastly higher speed than confocal microscopy. Such a microscope could be realized using prior knowledge of the generated speckle pattern, based on our knowledge of the hologram applied at the SLM. However, this approach is difficult within scattering samples, as scattering alters the speckle pattern. Alternatively, one could estimate the pattern, following a similar approach to blind-SIM in which unknown speckle patterns are projected into a sample and the patterns are estimated from the measured fluorescence [33,34]. Such reconstruction could utilize algorithms developed for blind structured illumination in photoacoustic imaging [40,46]. Speckle LFM also provides access to both the linear and nonlinear data, which could potentially be merged to profit from the relative advantages of the two methods, ideally combining the higher SNR of linear LFM with the higher resolution of speckle LFM. As such, we anticipate that speckle LFM could lead to exciting further advances in the future.

5. METHODS

A. Speckle LFM Setup

The speckle LFM was constructed by placing a microlens array (pitch 125 μm, focal length 2.5 mm, RPC Photonics) at the image plane of a home-made epi-fluorescence microscope with a numerical aperture (NA) 0.5 water immersion objective (Nikon CFI Fluor 20XW). The foci of the microlenses were imaged onto the camera sensor (Andor Zyla 5.5) using a 1:1 relay lens system (Nikon AF-S Micro-Nikkor 105 mm f/2.8G, not shown in Fig. 1(a). The illumination was produced with a 488-nm laser (Lasos 7812 ML5, 150 mW), which first passed through a band-pass filter to remove unwanted laser wavelengths. The polarization was then controlled using a half-wave plate followed by a linear polarizer. The phase was randomized by the phase-only SLM (Holoeye Pluto), which was imaged into the objective back-focal plane with relay optics. The SLM generated a sequence of independent speckle illuminations by applying a series of random phase profiles that repeated every 100 ms. The pixels applied phase shifts that were randomly chosen with equal probability across the range 0 to $2\pi$. The camera simultaneously acquired at 100 Hz, such that 10 camera images were recorded for each full set of speckle illuminations. The variance of each pixel was calculated across each set of 10 frames to achieve the variance image that was then used in 3D reconstructions [example variance image shown in Figs. 1(c), i].

The SLM used in our experiments is limited to 60 Hz update rate. Initially we thought that this should limit the useful rate of camera acquisition, as there is no benefit to sampling the speckle pattern faster than it changes. However, our experiments revealed that the recorded image variance increased as the frame rate was increased to 100 Hz, indicating that the illuminating speckle patterns decorrelate faster than the SLM is updated. This is likely due to phase flicker, which is a spurious high-frequency modulation of the phase applied by each pixel. Phase flicker can have amplitude of up to 1 radian in devices such as ours [47]. Phase flicker can be a major constraint in some applications [48], though here it increases the effective speed with which the speckle pattern decorrelates and therefore allows faster imaging.

B. 3D Deconvolution

Volumetric deconvolution was performed by adapting the method described in [25,27]. The recorded light field includes both spatial and angular information, which provides multiple perspective views of the sample volume and can therefore be formulated as a tomographic inverse problem. To achieve optimal resolution, the sample is reconstructed via Richardson–Lucy deconvolution. This algorithm estimates the volume distribution of fluorophores and iteratively updates the estimated distribution to minimize the difference between the predicted and measured light-field recordings [25,27].

The key difference between volumetric reconstructions in linear and speckle LFM is the calculated PSF. Linear PSFs are calculated using scalar diffraction theory to account for diffraction from both the objective and microlenses, using the code supplied in [25]. The linear PSF is squared to provide the effective PSF of speckle LFM [see Eq. (6)]. The estimated volume distribution is then square-rooted to return it to a linear brightness scaling [see Eq. (6)].

C. Zebrafish Larvae Experiments

For zebrafish experiments, transgenic fish with a pan-neuronal expression of nuclear GCaMP6s (HuC:H2B-GCaMP6s) in a mitfa-/-, roy-/-background were imaged at 7 days post-fertilization (d.p.f.). We immobilized the fish by embedding them in 2% low-melting-point agarose (Promega). The fish were imaged intermittently with multiple recordings of 6 min duration taken over sessions that typically lasted around 40 min. The results presented here were taken from $n=5$ fish recorded under identical conditions.

The fish were imaged with 2 mW illumination power. At this power, we found that bleaching reduced the fluorescence by approximately 20% over the imaging sessions. In each acquisition, the bright pixels on the sCMOS camera received around 800 photons each. Given that we resolve 40 axial planes, this suggests that each voxel contributed on average 20 photons/frame, or 2000 photons/s.

D. ROI Signal Extraction

To identify ROIs consistent with active neurons, the standard deviation in time was extracted for 3D datasets. Bright regions therefore correspond to regions with the highest temporal dynamics. Local maxima were identified using a Matlab script, which selected ROIs corresponding to 3D local maxima. ROI were included only if their peak value exceeded a threshold which was set to four times the standard deviation of the 3D data. To minimize the possible cross-talk between neighboring points, the temporal dynamics were extracted only from the single voxel at the center of the ROI. The SNR could be improved by spatially averaging the ROI, though this could compromise the validity of the resolution analysis applied in Fig. 4. For a fair comparison of speckle and linear LFM, temporal traces were always extracted using the same ROIs for both datasets. Signals were normalized into $\mathrm{\Delta }F/F$ by dividing by the mean value of the entire time trace.

E. Animal Surgery and in vivo Calcium Imaging of Awake Mice

Animals were prepared as previously described in [4], in accordance with Austrian and European regulations for animal experiments (Austrian §26 Tierversuchsgesetz 2012–TVG 2012). Adult mice were anesthetized with isoflurane (2–3% flow rate of 0.5–0.7 l/min), and craniotomy (3–5 mm diameter) was performed to expose the brain. With the skull opened and the dura intact, the virus AAV8:Hsyn-GCaMP6m was injected at 4–12 sites (25 nl per injection, at 10 nl/min; titer $\sim {10}^{12}$ viral particles/ml) with a 400-μm spacing forming a grid near the center of the craniotomy, at a depth of 400–450 μm below dura. After injection, a glass cranial window of 0.16-mm thickness was implanted in the craniotomy and sealed in place using tissue adhesive (Vetbond). The exposed skull was then completely covered with dental cement (Paladur, Heraeus Kulzer GmbH, Germany). To prevent post-surgical infections and post-surgical pain, the animals were supplied with water containing the antibiotic enrofloxacin (50 mg/kg) and the pain killer carprofen (5 mg/kg) for a period of $\sim 7\mathrm{d}$. After surgery, animals were returned to their home cages for 2–3 weeks for recovery and viral gene expression before being subjected to imaging experiments. During imaging sessions, the animals were head-fixed using a customized mount. A ventilation mask was placed in front of the mouse nose to provide air-puff mechanical stimulation to the whiskers and face as well as to provide gas anesthesia on demand. A typical imaging session lasted continuously for 2–10 min.