Model-free uncertainty estimation in stochastical optical fluctuation imaging (SOFI) leads to a doubled temporal resolution

Wim Vandenberg; Sam Duwé; Marcel Leutenegger; Benjamien Moeyaert; Bartosz Krajnik; Theo Lasser; Peter Dedecker

doi:10.1364/BOE.7.000467

1. Introduction

Recent developments have proven a strong interest in diffraction-unlimited far-field fluorescence imaging. By making use of fluorophore dynamics, the resolution limit imposed by diffraction can be overcome to allow a strongly improved spatial resolution [1]. A number of different techniques have appeared that achieve this goal, including STED/RESOLFT imaging [2,3] and PALM/STORM imaging [4–6]. These approaches deliver spatial resolutions down to the tens of nanometers scale in biological samples, though trade-offs in the temporal resolution or repeatability of the imaging are required [7].

Stochastic optical fluctuation imaging (SOFI) is an alternative way to achieve superresolution [8]. Like all diffraction-unlimited techniques, SOFI makes use of fluorescence dynamics of the fluorophores, focusing on the on/off ‘blinking’ behavior that is known to occur in most labels. In SOFI, multiple fluorescence images of the sample are acquired in rapid succession. Since the fluorophores rapidly cycle between the fluorescent and non-fluorescent states, each of these images presents a different ‘view’ on the sample structure because different combinations of emitters are active in each image. An image with a sub-diffraction spatial resolution can be reconstructed by calculating the cumulant of the fluorescence signal detected in each detector pixel in time. In principle the resulting procedure can achieve an unlimited spatial resolution since the calculation of the n^th-order cumulant results in an image with an $\sqrt{n}$ -fold resolution improvement in three dimensions, which can be enhanced to a factor of n via Fourier-reweighing or deconvolution [9]. The availability of a full theoretical model validating the spatial resolution improvement is a key feature of SOFI imaging.

In practice the need to sample the fluorescence at all points on the sample means that wide field imaging, using a two-dimensional array detector such as a CCD or CMOS camera, is the preferred method to acquire SOFI images. In fact, SOFI imaging is particularly convenient for wide-field imaging as the spatial resolution is increased in all directions, mitigating the reduced z-resolution intrinsic to wide-field imaging. Extending SOFI to cumulants of more than one pixel allows for the calculation of additional ‘virtual pixels’, effectively yielding an output image that has a higher pixel density than the input images, and making the full resolution enhancement accessible [9].

Compared to other super-resolution techniques, the main advantage of SOFI is that it can deal with a very wide range of conditions [10] and does not require special imaging hardware. For example, there is no requirement that individual fluorophores are resolvable, and the imaging can work well under conditions of low signal-to-noise-ratio (SNR), as can occur when imaging dim fluorophores in high-background conditions or when imaging at extended depth in the sample. However, the fluorophores must display fluorescence fluctuations that can be captured on the detector, and the improvements in spatial resolution have largely been limited to a two-to four-fold increase. In practice SOFI imaging has been performed using quantum dots [8,11], organic fluorophores [12, 13], and fluorescent proteins [14–18]. Other mechanisms to generate fluorescence fluctuations have also been used [19–21].

Since SOFI imaging makes use of a statistical parameter, the cumulant, the estimation of this parameter becomes more accurate and hence the quality of the SOFI image becomes better as more fluorescence images are included in the analysis. Ultimately the spatial and temporal resolution that can be achieved with SOFI is limited by the SNR of the calculated images. Higher-order calculations, which result in higher spatial resolutions, become progressively more noisy, and therefore require more experimental images to result in a usable SOFI image.

One of the complications in super-resolution imaging is that it is difficult to unambiguously verify the reliability or quality of the generated images, especially when a large amount of computer processing is involved. However, such estimations are required to ensure that only well-supported observations and conclusions are considered. Previously we have suggested performing this quality control by combining more than one super-resolution imaging technique [15], which can be be done if the fluorophore provides the required functionality. But, this is not possible if only one imaging technique is suited to the measurement conditions in question, in which case the reliability must be estimated using only the imaging data.

Based on statistical resampling we develop a strategy for the direct estimation of the spatially-resolved signal-to-noise ratio (SNR) of arbitrary SOFI images. We then show how this strategy can be used to considerably enhance the quality of the calculated SOFI images, and hence the spatial and/or temporal resolution, by optimally combining the cumulants that result in the same SOFI pixel. Our work not only provides a way for the unbiased estimation of the SOFI imaging quality, but also enables higher-quality SOFI images to be obtained while requiring fewer fluorescence images, while not imposing any additional requirements or limitations.

2. SOFI theory

In this section we briefly recapitulate the theory underlying SOFI imaging. The key to this imaging is estimating the cumulant of the intensity distribution sampled by each pixel on the detector over time. Following Ref. [8], the fluorescence F at position r is given by

F (r, t) = \sum_{j = 1}^{N} U (r - r_{j}) ε_{j} s_{j} (t)

where we assume N fluorophores with spatial coordinates given by the set {r_j}. ε_j is the brightness of the fluorophores, s_j(t) describes the intensity fluctuations of the emitter (being equal to one when the emitter is in the fluorescent on-state, and less than one in a dim or dark state), and U(r) is the point spread function (PSF) of the microscope.

In the simplest approach (which we do not recommend in practice) a SOFI image is generated simply by performing an independent cumulant calculation for each of the detector pixels. In this way, a second order image SOFI₂ and higher order images are given by [8]

{SOFI}_{2} (r, τ) = C_{2} [F (r, t), F (r, t + τ)]

{SOFI}_{n} (r, τ) = C_{n} [F (r, t), F (r, t + τ_{1}), \dots, F (r, t + τ_{n - 1})]

where C_n represents the calculation of the n^th-order cumulant. Various strategies for performing this calculation have been described [22], and many software packages include this among their standard libraries of procedures.

Using these details, it can be shown analytically that SOFI imaging leads to an improvement in spatial resolution. The second order cumulant C₂ can be written as

C_{2} [F (r, t), F (r, t + τ)] = 〈 δ F (r, t) \cdot δ F (r, t + τ) 〉

= \sum_{j = 1}^{N} \sum_{k = 1}^{N} U (r - r_{j}) U (r - r_{k}) ε_{j} ε_{k} 〈 Δ s_{j} (t) Δ s_{k} (t + τ) 〉

= \sum_{j}^{N} U^{2} (r - r_{j}) ε_{j}^{2} c_{2} (τ)

where the difference delta ∆s_j(t) is given by the difference between the instantaneous value of s_j(t) and its mean 〈s_j(t)〉. Furthermore we have assumed that the emitters blink independently from each other so that 〈∆s_j(t)∆s_k(t + τ)〉= c₂(τ)δ_jk, for some c₂(τ) and with δ_jk the Kronecker delta. Equation (6) reveals that the SOFI image contains the PSF squared, leading to the factor of two enhancement in spatial resolution in all three spatial directions. c₂(τ) is a function that describes the second-order cumulant of the fluctuating molecules. However, if we assume that the blinking behavior is identical for all molecules, then c₂(τ) becomes an identical scaling factor for each pixel. The procedure described here scales straightforwardly to higher order cumulants, where the n^th-order cumulant will contain the PSF raised to the n^th power, thus demonstrating the n-fold resolution improvement available.

In actual measurements there are two problems with the straightforward application of Eq. (3). If two or more time lags τ_i are identical, this results in artifacts in the SOFI image. Choosing distinct time lags eliminates this distortion but lowers the signal since the emitter fluctuations decorrelate when τ ≠ 0. More problematic is the fact that the calculated SOFI image has the same number of pixels as the fluorescence images, meaning that the optical size of the pixels is unchanged. Since the optical size of the detector pixels is usually chosen to sample the diffraction-limited PSF, the higher spatial resolution in SOFI may not be available in practice due to inadequate sampling of the enhanced structural information [9].

The natural solution to the latter problem is the expansion of the cumulant calculation to cumulants between different pixels, that is, cumulants of the form C₂[F(r,t),F(r + ξ,t + τ)], where ξ is a vector encoding a spatial shift as an integral number of detector pixels (another approach to this problem is the use of Fourier interpolation [23]). Adapting Eq. (6), we find

C_{2} [F (r, t), F (r + ξ, t + τ)] = U (\frac{ξ}{\sqrt{2}}) \sum_{j}^{N} U^{2} (r + \frac{ξ}{2} - r_{j}) ε_{j}^{2} c_{2} (τ)

We thus obtain ‘virtual pixels’ containing additional information that can not be recovered by interpolating the SOFI image discussed previously. In general, in a two-dimensional SOFI calculation of order n, (n−1) spatial shifts and time lags can be specified, while approximately n² virtual pixels can be generated for each pixel on the detector. In three-dimensional SOFI this increases to n³ virtual pixels. These virtual pixels are located at the geometric mean between the input detector pixels used in their calculation, while the signal of each virtual pixel is weighed by the distance between the pixels with respect to the size of the PSF.

Starting from a detector in which the pixels are laid out in a regular 2D grid, there are multiple pixel combinations that give rise to the same virtual pixel (Fig. 1). Due to the weighing with the extent of the PSF, the pixel combination in which the detector pixels are closest together will yield the highest signal (red arrow in Fig. 1(a)), though there are many more less-optimal combinations (black arrows). It is therefore natural to ask whether it is possible to combine all of these pixel combinations to produce SOFI images with optimized signal-to-noise ratios (SNRs). In other words, given a set of estimated cumulants {κ_i}, each calculated for different pixel combinations leading to the same virtual pixel, can we find a set of weights {w_i} such that the signal of each SOFI pixel

S = w_{1} κ_{1} + w_{2} κ_{2} + \dots

has an optimal signal-to-noise ratio (SNR)?

Fig. 1 a: The same virtual pixel (black box) can be generated using different combinations of detector pixels (green boxes). In general the combination(s) in which the detector pixels are closest together will contain the most signal (red arrow), but there are many more less-favorable combinations (black arrows). b: For three-dimensional SOFI the amount of combinations which exist for the same virtual pixel (black box) is even larger.

Download Full Size | PDF

This question is difficult to answer because the set of optimal weights depends on many parameters, including the size of the PSF with respect to the detector pixels, the brightness and fluorescence dynamics of the emitters, the background emission levels, etc. Note also that these weights should not be confused with the “distance factor corrections” used as part of the SOFI cross-cumulant calculation [9], which is automatically included as part of a standard SOFI calculation in our software. As we show further along in this contribution, it is indeed possible to achieve this in a way that takes all of these issues into account automatically, though doing so first requires a way to estimate the SNR of arbitrary SOFI images.

3. Estimating the uncertainty of SOFI pixels and pixel combinations

A SOFI image consists of many SOFI pixels, the values of which represent estimates for the underlying cumulants. Due to noise in the measurement and the fact that every measurement is necessarily of finite duration, the signal at each of these pixels has some level of uncertainty associated with it. We quantify this uncertainty using the SNR, defined as

SNR = \frac{S}{\sqrt{Var (S)}}

where S is the value of a particular SOFI pixel and Var(S) is the variance of S. In other words, we define the SNR as the ratio between the value of the signal and the uncertainty of its estimate. A ‘good’ SOFI image is thus one in which the SNR of the pixels is high.

The classical way to determine the uncertainty associated with a particular measurement is to repeat the experiment multiple times. However, this is barely possible for SOFI imaging due to photodestruction of the fluorophores and/or dynamics of the sample. Searching for an alternative way to determine the measurement uncertainty, we turned our attention to the well-established statistical resampling techniques. Since the calculation of a desired statistic typically involves hundreds or more data points (such as an image sequence used for SOFI), we can construct a large number of different but closely related datasets simply by leaving out one or more of the data points.

Delete-1 jackknife resampling [24] generates as many distinct datasets as there are measurement points by each time discarding a single measurement point (image). The SOFI analysis is then applied to each of these datasets, yielding results that are similar but not identical to that obtained on the full dataset (Fig. 2(a)). The spread of all these calculated results indicates how robust the calculated statistic is with respect to the input data, or in other words, indicates its uncertainty (a statistic with a low uncertainty is not changed much by leaving out a single data point [22]). The result is an estimate for the variance of the statistic.

Fig. 2 a: Uncertainty estimation using resampling: from a single SOFI dataset containing N fluorescence images, N derived datasets are created by sequentially discarding a single fluorescence image. This results in N SOFI images that can be compared to obtain the variance associated with each SOFI (virtual) pixel. b: A plot of the standard deviations of each pixel as determined by repeating a computer simulation 100 times versus the standard deviation obtained by resampling analysis on one of the simulated datasets. The black line corresponds to the function y = x.

Download Full Size | PDF

Given the set of cumulant estimates {κ₁,κ₂,…}, we applied the jackknife resampling technique as follows: from the stack of N acquired images, we construct N new image stacks each consisting of N − 1 images by leaving out a single (unique) fluorescence image each time. By applying the SOFI calculation to each of these resampled image stacks, we obtain N estimates for each cumulant. Importantly, performing these calculations requires only about four times more calculation time compared to a standard SOFI calculation, allowing the full procedure to be performed in minutes or less. Denoting the value of the i^th cumulant estimate obtained by excluding image f as κ_if, the overall i^th jackknife estimator for this statistic is given as

κ_{i (jack)} = \frac{\sum_{f = 1}^{N} κ_{i f}}{N}

The covariances between the different K-statistics and the variances of the individual cumulant estimates (equal to the covariance of a statistic with itself) can then be estimated using [24]

Covar (κ_{i}, κ_{j}) = \frac{\sum_{f = 1}^{N} (κ_{i f} - κ_{i (jack)}) (κ_{j f} - κ_{j (jack)})}{N - 1}

Where all (co-)variances between the K-statics in the set can be conveniently arranged in matrix form as follows.

A_{c \times c} = (\begin{matrix} Var (κ_{1}) & Covar (κ_{1}, κ_{2}) & \dots & Covar (κ_{1}, κ_{c}) \\ Covar (κ_{2}, κ_{1}) & Var (κ_{2}) & \dots & Covar (κ_{2}, κ_{c}) \\ ⋮ & ⋮ & ⋱ & ⋮ \\ Covar (κ_{c}, κ_{1}) & Covar (κ_{c}, κ_{2}) & \dots & Var (κ_{c}) \end{matrix})

For an arbitrary set of weights the signal given by Eq. (8) can now be shown to have an associated estimated variance given by

Var (S) = \sum_{i = 1}^{c} \sum_{j = 1}^{c} w_{i} w_{j} A_{i j}

where i and j run over all cumulant estimates in the set.

To test the usefulness of the uncertainty estimation, we generated a computer simulated set of fluorescence images containing immobilized blinking emitters, following the simulation procedure outlined in [25]. As Fig. 2(b) shows, the resulting estimated variance corresponds very well with the variance estimated by repeatedly performing the same simulation. For high uncertainties the jackknife estimation slightly underestimates the actual variance, which is a consequence of the fact that consecutive fluorescence images are not entirely independent. In principle this can be addressed by using a delete-n jackknife approach, though the deviation is sufficiently small as to not warrant this extra effort.

Figure 3 shows example SNRs calculated for different pixel combinations leading to the same SOFI pixels. As expected the calculated SNR is highest for the combination with the closest distance between the detector pixels, though the other combinations still lead to informative images.

Fig. 3 Three images showing the SNRs of the same virtual SOFI pixel, but constructed using different pixel combinations (the insets indicate the detector pixels that were combined). As expected the SNR of the image becomes progressively lower as the distance between the pixel detectors increases.

Download Full Size | PDF

We do note that our approach in effect only estimates the precision of the calculated result, but not the accuracy. As a result we cannot detect any fundamental bias. Experimentally a possible bias can only be detected via correlation with an independent imaging technique [15]. However, since the imaging enhancement of SOFI is unambiguously described by a theoretical model which does not indicate any bias, we believe that this distinction is sufficiently small to be ignored in what follows.

4. Determining the optimal weights of the cross-cumulant combinations

The combination of Eq. (8) and Eq. (13) allows us to readily determine the SNR of any virtual pixel in the SOFI image given an arbitrary set of weights. The SNR is given by

SNR = \frac{S}{\sqrt{Var (S)}} = \frac{\sum_{i = 1}^{c} w_{i} κ_{i}}{\sqrt{\sum_{i = 1}^{c} \sum_{j = 1}^{c} w_{i} w_{j} A_{i j}}}

at the extremum the following must hold for m ∈ 1,2,…,c

\frac{\partial SNR}{\partial w_{m}} = 0

\frac{κ_{m}}{\sqrt{Var (S)}} - \frac{S \sum_{j = 1}^{c} w_{j} A_{m j}}{Var (S) \sqrt{Var (S)}} = 0

\sum_{j = 1}^{c} w_{j} \frac{A_{m j}}{κ_{m}} = \frac{Var (S)}{S}

Note that the right-hand side of (17) is independent of our choice of m. As a result we can construct a set of (c − 1) linear equations by choosing m to belong to 2,3,…,c

\sum_{j = 1}^{c} w_{j} (\frac{A_{m j}}{κ_{m}} - \frac{A_{1 j}}{κ_{1}}) = 0

which can be turned into (still for m ≠ 1)

\sum_{j \neq 1}^{c} \frac{w_{j}}{w_{1}} (\frac{A_{m j}}{κ_{m}} - \frac{A_{1 j}}{κ_{1}}) = \frac{A_{11}}{κ_{1}} - \frac{A_{m 1}}{κ_{m}}

a set of linear equations which can be numerically solved for all (w_j/w₁) in a straightforward manner (e.g. using LU decomposition and back-substitution). We are now free to choose an arbitrary w₁ as the SNR does not change upon rescaling of all weights for any factor r ≠ 0

\frac{\sqrt{Var (r S)}}{r S} = \frac{r}{r} \frac{\sqrt{Var (S)}}{S} = \frac{\sqrt{Var (S)}}{S}

For example, we can arbitrarily pick w₁ = 1, from which all other weights are uniquely determined by solving the set of n − 1 variations of Eq. (19). These weights in combination with Eq. (8) give us the recipe for obtaining a virtual pixel with the maximal SNR.

5. Practical application and results

In principle the previous procedure allows us to define a set of weights for every single (virtual) pixel in the SOFI image. For example, for fluorescence images consisting of 512 × 512 pixels, this would result in approximately 10⁶ sets of weights for a second order SOFI image. Clearly this would not just be computationally demanding, but the resulting sets of weights would be sensitive to noise and to the underlying structure of the sample around each virtual pixel.

Instead we only determine a single set of weights for each virtual pixel type used in the image (e.g. 4 for second-order SOFI), averaging over multiple SOFI pixels to determine a ‘consensus’ set. The underlying assumption is that the optimal pixel weights are largely determined by the characteristics of the PSF, and that this PSF is invariant over the entire image. When calculating the w vector we thus average the κ vectors and A matrices to come up with an on average correct and structure-independent result.

Furthermore, to avoid excessive computation and exclude background pixels, we do not run the calculation for all virtual pixels of the same type in the acquired fluorescence images. Instead we randomly select between 200 and 1000 pixels for which we run the full jackknife procedure as described above. This sampling is limited to non-background regions of the image, selected in a randomized fashion. To verify whether a sufficiently large number of detector pixels has been sampled, we suggest repeating the calculation procedure using a different set of detector pixels, and checking that the calculated weights have converged.

A pseudocode listing of the complete procedure is given in algorithm 1. The algorithm was implemented in Igor Pro (WaveMetrics Inc., Lake Oswego, OR) based on the SOFI algorithms implemented in the Localizer [25] software package. This code is freely available together with Localizer. The calculation of the weights for a given dataset takes some time (minutes to tens of minutes on a typical desktop computer). When analyzing many datasets with similar conditions (same probe, same instrumental settings) it is possible to perform this process only once, reusing the weights for subsequent calculations.

To test the developed approach, we transfected HeLa cells using a plasmid encoding the fluorescent protein ffDronpa [15] targeted to the cell membrane using the N-terminal part of Lynkinase as a targeting motif. The cell culturing and washing have been described in Ref. [15]. Imaging was carried out on two different instruments, allowing different total magnifications to be used. Measurements using a 150× magnification were performed on an Olympus CellTIRF instrument in TIRF mode as described in Ref. [15]. The second instrument was a custom-built system based on an Olympus IX71 microscope body. Excitation was provided by a 488 nm laser at an approximate power of 2.4 mW at the back aperture of the microscope body. Datasets with a total magnification of 330× were recorded in EPI mode using an Olympus PLANAPO 100× NA 1.4 objective combined with a 3.3× magnifier, while datasets with a total magnification of 495× were recorded in TIRF mode using an Olympus UAPON 150×OTIRF NA 1.45 combined with the same magnifier. In all cases images were recorded using a Hamamatsu ImagEMx2 camera using an exposure time of 30 ms and an electron-multiplication gain optimized to the expression levels of the cells.

Algorithm 1. Algorithm for the calculation of the optimal weights.

View Table

Figure 4 shows an example of the improvements that can be obtained using our procedure. Shown are SOFI images at different optical magnifications calculated from 100 images acquired over 3 seconds of living HeLa cells. Comparing both the conventional and combination-optimized figures clearly shows the improvement in imaging quality possible through our procedure. In general, the attainable imaging improvement depends on the size of the PSF with respect to the optical pixel size of the detector, since the SOFI signal for each pixel combination depends on the distance between the pixels, scaled by the size of the PSF (Eq. (7)). As we detail in appendix A, we find that the dataset with 107 nm projected pixel size requires 33% fewer fluorescence images for an equivalent SNR and shows a 40% rise in image SNR for the same number of fluorescence images with the new method. The dataset with 48 nm projected pixel size requires 50 to 60% fewer frames for an equivalent SNR and shows an 80% increase in SNR for the same number of frames. The dataset with 32 nm projected pixel size requires 50% fewer frames for an equivalent SNR and shows a 93% increase in SNR for the same number of frames. Taken together, these results show that our technique allows the temporal resolution of SOFI imaging to be increased substantially while achieving the same imaging quality. Alternatively, better quality images can be constructed from the same datasets. Because our strategy is based on resampling, this strategy automatically takes all relevant specifics of the measurement into account.

Fig. 4 Example results obtained using the procedure outlined in this manuscript. In each case 100 fluorescence images of live HeLa cells expressing Lyn-Dronpa measured at different optical magnifications were analyzed. Shown are a conventional second-order SOFI analysis (where only the closest pixel combination(s) are used) and an analysis where all combinations within a 5 by 5 grid surrounding the virtual pixel are used with optimal weights as determined by the resampling methodology.

Download Full Size | PDF

6. Fast approximation to the exact calculation

In some cases, such as when performing initial exploration of the acquired datasets, it may not be desirable to wait for the full calculation of the optimal weights. During our investigations we noticed that there is frequently a strong correlation between the average SOFI signal of a certain pixel combination and the resulting optimal weight for that combination (Fig. 5).

Fig. 5 A plot of the optimal weights for pixel combinations leading to different virtual SOFI pixels, as determined using our algorithm, against the average signal of those pixel combinations. Data is shown for for 2^nd(left) and 3^rd(right) order SOFI, with different colors representing different virtual pixel types. The lines show a least-squares linear fit with intercept fixed to zero.

Download Full Size | PDF

In general the combinations with the highest signal also end up having the largest weight, with a linear dependence between them. This relation can be used to efficiently compute approximations to the exact solution (Fig. 6), by simply setting w_i = κ_i.

Fig. 6 Comparison of the conventional SOFI imaging, imaging using our optimized algorithm, and using weights derived from a linear approximation.

Download Full Size | PDF

Interestingly, we also noticed that the relation between w and S was more complex in some samples compared to others. For example, using the fluorescent protein rsGreen1 [18] as the label, we obtained the trend displayed in Fig. 5. However, in measurements using ffDronpa as the fluorophore, the weights show an initial drop when the SOFI signal increases, followed by a subsequent increase. In some cases this leads to the appearance of weights with negative values. As a result the applicability of the fast linear approximation of the weights should be used only after verifying the validity on a representative experiment. While the appearance of negative weights may appear surprising, these arise naturally, as can be shown with simulations and do indeed lead to the maximal SNR (Fig. 7).

Fig. 7 Plot of the weights versus SOFI signal as determined by resampling for a simulation. The only difference between the two simulations is that in the left graph the probe has a brightness of 30,000 detected photons per second in the on-state, in the right graph the brightness is 1,000 detected photons per second.

Download Full Size | PDF

In any case the validity of this approximation can be verified for a particular system by running the exact calculation and inspecting the obtained weights visually (Fig. 5).

7. Conclusion

In this work we presented a strategy to estimate the uncertainty associated with a particular SOFI measurement, based on statistical resampling. Our approach can be used to determine the per-pixel SNR and whole-image SNR associated with a single SOFI measurement. It automatically takes into account the particularities of the experiment, including instrumental aspects such as camera sensitivity and exposure time, and sample-related issues such as probe brightness, labeling density, and blinking kinetics. Knowledge of these uncertainties makes it possible to unambiguously determine to what extent quantitative analyses on SOFI data are supported.

We then showed how our procedure can be used to obtain a linear combination of the different pixel combinations such that the SNR of each type of SOFI pixel is maximized. This does not impose any additional requirements on the measurement. We estimate that our procedure results in an approximately 40% to 80% increase in SNR, allowing higher-quality images to be calculated from the same dataset, or alternatively allowing similar-quality images to be calculated using 30% to 60% fewer input fluorescence images. Since this improvement is entirely free, in the sense that it uses information that is always present in SOFI datasets, we expect that it will enable SOFI imaging to be performed on more challenging and more dynamic systems than was currently possible.

We also note that the resampling methodology introduced here can be expanded to any downstream result based on SOFI imaging. As a result we believe that our resampling methodology holds great promise for future quantitative analysis of SOFI data.

8. Appendix A: Quantifying the gain in SNR

We used two different strategies to quantify the improvement in imaging performance. The first strategy entails the direct estimation of the SNR using the resampling approach which was introduced in this paper. In the second strategy we approximate the noise in the image as the width of the image histogram, and compare the number of input fluorescence images required to achieve a comparable histogram width with and without our procedure.

The direct estimation of the SNR is done by calculating images such as the ones shown in Fig. 3, where the value of each pixel is the SNR of that pixel in the corresponding SOFI image. We then create histograms of the SNR images (example shown in Fig. 8(b)). In this histogram two distinct peaks are visible, corresponding to background (low-SNR) and signal (high-SNR). We estimate the SNR of the image as the maximum of the high-SNR peak.

Fig. 8 A comparison between the conventional approach and the approach described in this paper a:example histograms of SOFI images calculated for different numbers of fluorescence images using the 48 nm optical pixel size. Data calculated using the conventional method is shown as a continuous line, the equivalent histogram using the method of optimal weights as a dashed line. The optimal weights-based approach reaches a similar histogram width (and quality) using fewer images. b:example SNR histograms of a SOFI image calculated from 100 fluorescence images and using the 107 nm optical pixel size. The SNR of the non-background peak is clearly higher for the optimal weights calculation.

Download Full Size | PDF

In the second strategy we calculate SOFI images from a single dataset for different numbers of input images (including e.g. 100 input images, 200, 300, etc.), using both the standard SOFI calculation where only the closest combinations are used and the optimized approach presented in our study. For each image we calculate a histogram of the pixel values (Fig. 8(a)). In general, noise causes the histograms to broaden, and as a result the histogram width is a measure of the noise content of the associated SOFI image. Given these images, we then quantify the improvement in imaging by comparing the number of input images required to reach a particular histogram width using the standard calculation and the weights-optimized calculation.

The advantage of the second approach is that it is essentially independent of the resampling approach and thus provides an independent second assessment. However, the first approach is more straightforward since it results in a direct estimate of the SNR. In a sense the approaches are complementary: the first strategy estimates to what degree an image can be improved when including the same number of images, while the second strategy estimates the achievable increase in temporal resolution (lower number of input images) available using our technique while maintaining the same image quality.

The achievable improvement with our technique mainly depends on the size of the point spread function with respect to the optical pixel size of the camera. The dataset with 107 nm projected pixel size requires 33% fewer frames for an equivalent SNR and shows a 40% rise in image SNR for the same number of frames with the new method. The dataset with 48nm projected pixel size requires 50 to 60% fewer frames for an equivalent SNR and shows an 80% increase in SNR for the same number of frames. The dataset with 32 nm projected pixel size requires 50% fewer frames for an equivalent SNR and shows a 93% increase in SNR for the same number of frames.

Acknowledgments

B.M., S.D. and W.V. thank the Agency for Innovation by Science and Technology (IWT-Vlaanderen) for a doctoral fellowship. B.M. and P.D. thank the Research-Foundation Flanders (FWO-Vlaanderen) for a postdoctoral fellowship. P.D. thanks KULeuven for a Research Professorship. This work was supported by the FWO (grant number 1521915N). M.L and T.L thank the Swiss National Foundation (grants 200020-159945/1 and 205321-138305/1). M.L. acknowledges a postdoctoral fellowship by the NanoBiophotonics department of the Max Planck Institute of Biophysical Chemistry, Göttingen, Germany.

References and links

1. P. Dedecker, F. C. De Schryver, and J. Hofkens, “Fluorescent proteins: shine on, you crazy diamond,” J. Am. Chem. Soc. 135, 2387–2402 (2013). [CrossRef] [PubMed]

2. S. W. Hell and J. Wichmann, “Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy,” Opt. Lett. 19, 780–782 (1994). [CrossRef] [PubMed]

3. S. W. Hell, “Toward fluorescence nanoscopy,” Nat. Biotechnol. 21, 1347–1355 (2003). [CrossRef] [PubMed]

4. E. Betzig, G. H. Patterson, R. Sougrat, O. W. Lindwasser, S. Olenych, J. S. Bonifacino, M. W. Davidson, J. Lippincott-Schwartz, and H. F. Hess, “Imaging intracellular fluorescent proteins at nanometer resolution,” Science 313, 1642–1645 (2006). [CrossRef] [PubMed]

5. M. J. Rust, M. Bates, and X. Zhuang, “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM),” Nat. Meth. 3, 793–796 (2006). [CrossRef]

6. S. T. Hess, T. P. K. Girirajan, and M. D. Mason, “Ultra-high resolution imaging by fluorescence photoactivation localization microscopy,” Biophys. J. 91, 4258–4272 (2006). [CrossRef] [PubMed]

7. W. Vandenberg, M. Leutenegger, T. Lasser, J. Hofkens, and P. Dedecker, “Diffraction-unlimited imaging: from pretty pictures to hard numbers,” Cell Tissue Res. 360, 151–178 (2015). [CrossRef] [PubMed]

8. T. Dertinger, R. Colyer, G. Iyer, S. Weiss, and J. Enderlein, “Fast, background-free, 3D super-resolution optical fluctuation imaging (SOFI),” Proc. Natl. Acad. Sci. U.S.A. 106, 22287–22292 (2009). [CrossRef] [PubMed]

9. T. Dertinger, R. Colyer, R. Vogel, J. Enderlein, and S. Weiss, “Achieving increased resolution and more pixels with superresolution optical fluctuation imaging (SOFI),” Opt. Express 18, 18875–18885 (2010). [CrossRef] [PubMed]

10. S. Geissbuehler, C. Dellagiacoma, and T. Lasser, “Comparison between SOFI and STORM,” Biomed. Opt. Express 2, 408–420 (2011). [CrossRef] [PubMed]

11. Z. Zeng, X. Chen, H. Wang, N. Huang, C. Shan, H. Zhang, J. Teng, and P. Xi, “Fast super-resolution imaging with ultra-high labeling density achieved by joint tagging super-resolution optical fluctuation imaging,” Sci. Rep. 5, 8359 (2015). [CrossRef] [PubMed]

12. T. Dertinger, M. Heilemann, R. Vogel, M. Sauer, and S. Weiss, “Superresolution Optical Fluctuation Imaging with Organic Dyes,” Angew. Chem. Int. Ed. Engl. 49, 9441–9443 (2010). [CrossRef] [PubMed]

13. S. Geissbuehler, N. Bocchio, C. Dellagiacoma, C. Berclaz, M. Leutenegger, and T. Lasser, “Mapping molecular statistics with balanced super-resolution optical fluctuation imaging (bSOFI),” Opt. Nanoscopy 1, 4 (2012). [CrossRef]

14. P. Dedecker, G. C. H. Mo, T. Dertinger, and J. Zhang, “Widely accessible method for superresolution fluorescence imaging of living systems,” Proc. Natl. Acad. Sci. U.S.A. 109, 10909–10914 (2012). [CrossRef] [PubMed]

15. B. Moeyaert, N. NguyenBich, E. De Zitter, S. Rocha, K. Clays, H. Mizuno, L. van Meervelt, J. Hofkens, and P. Dedecker, “Green-to-red photoconvertible dronpa mutant for multimodal super-resolution fluorescence microscopy,” ACS Nano 8, 1664–1673 (2014). [CrossRef] [PubMed]

16. S. Geissbuehler, A. Sharipov, A. Godinat, N. L. Bocchio, P. A. Sandoz, A. Huss, N. A. Jensen, S. Jakobs, J. Enderlein, F. Gisou van der Goot, E. A. Dubikovskaya, T. Lasser, and M. Leutenegger, “Live-cell multiplane three-dimensional super-resolution optical fluctuation imaging,” Nat. Commun. 5, 5830 (2014). [CrossRef] [PubMed]

17. X. Zhang, X. Chen, Z. Zeng, M. Zhang, Y. Sun, P. Xi, J. Peng, and P. Xu, “Development of a reversibly switchable fluorescent protein for super-resolution optical fluctuation imaging (SOFI),” ACS Nano 9, 2659–2667 (2015). [CrossRef] [PubMed]

18. S. Duwe, E. De Zitter, V. Gielen, B. Moeyaert, W. Vandenberg, T. Grotjohann, K. Clays, S. Jakobs, L. Van Meervelt, and P. Dedecker, “Expression-enhanced fluorescent proteins based on enhanced green fluorescent protein for super-resolution microscopy,” ACS Nano (2015). [CrossRef] [PubMed]

19. S. Cho, J. Jang, C. Song, H. Lee, P. Ganesan, T.-Y. Yoon, M. W. Kim, M. C. Choi, H. Ihee, W. D. Heo, and Y. Park, “Simple super-resolution live-cell imaging based on diffusion-assisted förster resonance energy transfer,” Sci. Rep. 3, 1208 (2013). [CrossRef]

20. Z. Ristanović, M. M. Kerssens, A. V. Kubarev, F. C. Hendriks, P. Dedecker, J. Hofkens, M. B. J. Roeffaers, and B. M. Weckhuysen, “High-resolution single-molecule fluorescence imaging of zeolite aggregates within real-life fluid catalytic cracking particles,” Angew. Chem. Int. Ed. Engl. 54, 1836–1840 (2015). [CrossRef]

21. L. Kisley, R. Brunetti, L. J. Tauzin, B. Shuang, X. Yi, A. W. Kirkeminde, D. A. Higgins, S. Weiss, and C. F. Landes, “Characterization of porous materials by fluorescence correlation spectroscopy super-resolution optical fluctuation imaging,” ACS Nano 9, 9158–9166 (2015). [CrossRef] [PubMed]

22. A. Stuart and K. Ord, Kendall’s Advanced Theory of Statistics: Volume 1: Distribution Theory, no. v. 1; v. 1994 in Kendall’s Advanced Theory of Statistics (Wiley, 2009).

23. S. C. Stein, A. Huss, D. Hahnel, I. Gregor, and J. Enderlein, “Fourier interpolation stochastic optical fluctuation imaging,” Opt. Express 23, 16154–16163 (2015). [CrossRef] [PubMed]

24. A. J. Stromberg, “Robust covariance estimates based on resampling,” Journal of Statistical Planning and Inference 57, 321–334 (1997). Robust Statistics and Data Analysis, Part {II}. [CrossRef]

25. P. Dedecker, S. Duwé, R. K. Neely, and J. Zhang, “Localizer: fast, accurate, open-source, and modular software package for superresolution microscopy,” J. Biomed. Opt. 17, 126008 (2012). [CrossRef] [PubMed]

1:	Let N be the number of fluorescence images in the dataset
2:	Select P non-background detector pixels
3:	for v in virtual SOFI pixels do
4:	for p in selected detector pixels P do
5:	for i in pixel combinations for v do
6:	Calculate SOFI signal κ_ip for detector pixel p and pixel combination i
7:	for f is 1 to N do
8:	Calculate delete-1 SOFI signal κ_ifp for detector pixel p and pixel combination i, excluding the contribution of the f ^th fluorescence image
9:	end for
10:	end for
11:	Calculate covariance matrix Ap using Eq. (10) and Eq. (11), replacing κ_if with κ_ifp.
12:	end for
13:	Calculate ${\bar{κ}}_{i}$ and Ā by averaging A_p and κ_ip over p.
14:	Determine optimal weights {w_i} for virtual pixel v using Eq. (19), replacing κ with $\bar{κ}$ and A with Ā.
15:	end for

Model-free uncertainty estimation in stochastical optical fluctuation imaging (SOFI) leads to a doubled temporal resolution

Abstract

1. Introduction

2. SOFI theory

3. Estimating the uncertainty of SOFI pixels and pixel combinations

4. Determining the optimal weights of the cross-cumulant combinations

5. Practical application and results

6. Fast approximation to the exact calculation

7. Conclusion

8. Appendix A: Quantifying the gain in SNR

Acknowledgments

References and links

Cited By

Figures (8)

Tables (1)

Equations (20)

Biomedical Optics Express