1. Introduction

Recent progress in single-molecule localization microscopy (SMLM) has made a great impact on biological and medical research [1–4]. SMLM sparsely switches fluorophores to a ‘‘bright state’’ during the imaging acquisition and locates the positions of individual fluorophores by fitting the point spread function using a Gaussian. By repeating this process thousands of times, one obtains the three-dimensional (3D) point clouds representing the locations of fluorophores with a precision of 10 ‒ 20 nm. However, analyzing SMLM images presented in these (3D) point clouds remains challenging.

Existing algorithms for cluster identification require users to define thresholds to determine cluster boundaries [5–7]. For example, In density-based analysis, DBSCAN requires users to input two parameters for detecting and segmenting the clusters in the point clouds: a radius and the minimum number of points within the radius to qualify as a cluster [8]. In Bayesian cluster analysis, Bayesian is used to evaluate the assignment of every molecule to clusters by its marginal posterior probability [9]. The users are required to set the cutoff probability that localization is clustered. In Voronoi Tessellation analysis, the assignment of point clouds is based on the geometrical properties of the Voronoi cells, such as the cell area (or point density), which also requires its users to input a cutoff cell area. When applying SMLM to study biological samples, the challenge is that no prior knowledge is available to determine these thresholds. Often these thresholds are determined by the trial-and-error method with repeated visual inspections. Therefore, the resulting cluster assignments are highly user-dependent. The situation is further complicated by the fact that many of these parameters depend on the conditions under which the images were collected. For example, the point density in an SMLM image depends on the data acquisition time and the antibody concentration used in the immunoassay labeling. Therefore, an optimal cutoff threshold for one SMLM image may not be suitable for another image.

In the current study, we seek to find an algorithm that identifies clusters within the point clouds in SMLM without any inputs from users. Since the cutoff thresholds required in the existing methods are typically determined by repeated visual inspections, it is plausible to develop an algorithm for cluster identification using the modulation transfer function (MTF) of human ocular optics. Such an algorithm would produce cluster assignments consistent with human vision in a systematic way. We show that such a human-vision-inspired cluster identification (HVICI) can be developed so that cluster identification for SMLM can be carried out in a user-independent manner.

2. Algorithm

Human vision’s MTF acts as a low-pass filter, whereby the contrast transfer decreases with increasing spatial frequency [10,11]. The low-pass MTF allows humans to easily identify larger features (low spatial frequencies) embedded with noises (high spatial frequencies). As an example, Fig. 1(a) shows an image of the Canadian flag with 90% of the pixels removed. Although Fig. 1(a) is a noisy image, it can be easily identified by eyes as the Canadian flag. When the Fourier transform is applied to Fig. 1(a), one obtains its power spectrum in the spatial frequency domain (Fig. 1(b)). Random noise produces a relatively uniform offset over the entire frequency range (green color in Fig. 1(b)), [12] whereas the information of the flag is located in the low spatial frequency region (red color). These characteristics can be utilized in cluster recognitions for SMLM images.

 

Fig. 1. (a) Flag of Canada with 90% pixels removed. (b) Power spectrum of (a) obtained by the Fourier transform. (c) MTF obtained by fitting (b) using Eq. (2). (d) Resulting power spectrum after the MTF in (c) is applied to (b). (e) Intensity image in the Cartesian space from the inversed Fourier transform of (d). (f) Binary image of (e). (g) Image in (a) with added noise; (h) Restored image from (g) using procedures described in (a)-(f).

Download Full Size | PDF

The human ocular MTF is well described by Johnson’s equation [13]

For images that consist of point clouds, randomly scattered points (Supplement 1, Fig. S1a) produce spatial frequencies over a broad range (Supplement 1, Fig. S1b). When the points form a cluster (Supplement 1, Fig. S1c), the intensity in the low-spatial-frequency region is enhanced (Supplement 1, Fig. S1d). In the current study, n = 2 was used in Eq. (1) to depress the high spatial frequency components (noises) while remaining consistent with the human MTF [13,14]. Taking into account the anisotropic characteristics of an SMLM image, Eq. (1) can be generalized to an anisotropic MTF

Because HVICI identifies only the non-random features of the point clouds, a key advantage is that it can identify clusters over randomly distributed noise, which may be present in biological samples due to the nonspecific binding of fluorophores. When a large amount of random noise is added to Fig. 1(a) (Fig. 1(g)), HVICI restores the Canadian flag with only minor distortions (Fig. 1(h)). In contrast, coordinate-based analyses, such as SR-Tesseler [5], would require its users to raise the cut-off point density to obtain similar clusters. Because the background noise varies from image to image, it becomes challenging for coordinate-based analyses to obtain reliable results. Even when our eyes begin to experience difficulties in defining the border of the maple leaf due to a large amount of noise (Fig. 1(g)), HVICI can restore the image without a significant loss (Fig. 1(h)). Therefore, HVICI may outperform human vision.

3. Results

To recognize clusters in point clouds, a more sophisticated approach for detecting cluster boundaries is needed. Figure 2(a) shows an SMLM image of the fusion proteins of the Nipah virus (NiV) on the plasma membrane of a host cell. Following the steps described in Figs. 1(a)–1(e), one obtains its intensity map in the Cartesian space (Fig. 2(b)), which is an analog of Fig. 1(e). In contrast to the binary nature of Fig. 1(e), Fig. 2(b) contains peaks of various intensities. The peak intensity is related to the number of localizations in the cluster. A dense and large cluster has a higher peak intensity. We applied the Laplacian edge detection to identify the edges of clusters [15]. Figure 2(c) shows the second-order gradient amplitude after applying the Laplacian operator to Fig. 2(b). Theoretically, zero-crossings in the Laplacian of an image identify an edge [16]. However, ripples arising from the Fourier transform may yield false zero-crossings. These false zero-crossings may be removed by increasing the threshold. A plausible approach for determining the cutoff threshold is to assume that a single isolated point (a delta function) should not be classified as a cluster in the point clouds. Since the Fourier transform of a delta function has a constant amplitude of 1 over the entire frequency domain, the normalized MTF (Fig. 1(c)) represents the residual power spectrum of a delta function seen by the MTF. Therefore, the maximum of the MTF’s second-order gradient was used as the threshold for determining the cluster boundaries in the Cartesian space, as shown by the green curves in Fig. 2(d).

 

Fig. 2. (a) SMLM image of NiV fusion proteins labeled with Alexa Fluor 647 on the plasma membrane of the host cell (PK13). (b) Cluster intensity map in the Cartesian space after applying the MTF filter in the frequency space. (c) Amplitude of the second-order gradient after applying the Laplacian operator on (b). (d) Cluster boundaries (green lines) identified using the Laplacian edge detection.

Download Full Size | PDF

To demonstrate the advantages of HVICI, we compared HVICI with the SR-Tesseler [5], a popular method for cluster identification based on Voronoi tessellation. Figure 3(a) shows the clusters of NiV fusion proteins on the plasma membrane of a host cell (green points) and the clusters identified by HVICI (red lines). SR-Tesseler requires users to input a point density threshold (density factor). Here users face the dilemma of segregating or combining nearby clusters. A high-density threshold overlooks small clusters (yellow circles in Fig. 3(b)), while lowering the density threshold leads to the merging of nearby clusters (white boxes in Figs. 3(c) and 3(d)). The fundamental issue of coordinate-based analyses is that a fixed global threshold does not work well for all clusters with various local densities in an image. Therefore, the result is often user-dependent (Fig. 3(e)) and lacks reproducibility from one image to another or even for different regions in the same image. In contrast, the Laplacian edge detection used in HVICI is adaptive because it detects the shape of a cluster using the density gradients of an image, which is insensitive to the absolute point density. Overall, HVICI produces cluster identifications with their shapes and sizes much more consistent with human vision than the coordinate-based algorithm.

 

Fig. 3. Comparisons to the SR-Tesseler [5]. (a) Cluster boundaries identified by HVICI. (b)-(d) Cluster boundaries identified by the SR-Tesseler with the density threshold (density factor) set at 0.9 (b), 0.8 (c), and 0.5 (d). Green dots represent the SMLM image of the NiV fusion proteins labeled with Alexa Fluor 647 on the plasma membrane of the host cell, and solid red lines represent the boundaries of the identified clusters. Scale bar: 500 nm. For the SR-Tesseler, the objects were selected with the density factor = 0.15 and the cut density = 25. The minimum number of points in a cluster was set at 5. For a straightforward comparison, the minimum number of points in a cluster was also set at 5 for (a). (e) Histograms of cluster areas presented by the red cures in (a) – (d).

Download Full Size | PDF

HVICI can be easily extended to analyze 3D point clouds. Figure 4(a) shows the z-sectioned (Δz = 170 nm) SMLM images of a virus-like particle with the matrix proteins labeled with Alexa Fluor 647. The 3D point clouds were collected using optical astigmatism [17,18]. Figure 4(a) is projected cross sections for better visualization. The procedure to process 3D point clouds by HVICI is identical to that for 2D point clouds with the addition of the third dimension. Briefly, a 3D Fourier transform was applied to the 3D point clouds to obtain its power spectrum, and a 3D Gaussian fitting was carried out to obtain the 3D MTF. After the MTF was applied to the power spectrum, an inverse Fourier transform was performed to produce the cluster intensity map in the Cartesian space, and then a 3D Laplacian was used to obtain the second-order gradient. The second-order gradient of the 3D point clouds is four-dimensional (4D). To visualize the amplitude of the second-order gradient for the Laplacian edge detection, the amplitude of the second-order gradient is projected to a stack of 2D images in Fig. 4(b) (negative amplitudes not shown). Similarly, a threshold to determine the boundaries of clusters can be obtained with the assumption that a single isolated point will not be classified as a cluster. When the threshold is applied to the amplitude of the second-order gradient, one obtains the isosurfaces of the 4D object, which are 3D surfaces (Fig. 4(c)). For the virus-like particle, the algorithm produces a hollow structure, which is consistent with our understanding that the virus structure contains a capsid with a central cavity.

 

Fig. 4. (a) Z-sectioned SMLM images of a NiV virus-like particle with the matrix protein labeled with Alexa Fluor 647. (b) Second-order gradient of (a). (c) 3D cluster boundaries (green surface) identified by the Laplacian edge detection. (d) 3D SMLM image of NiV fusion proteins on cell protrusions. (e) 3D clusters identified from (d).

Download Full Size | PDF

Figure 4(d) shows a 3D SMLM image of the NiV fusion proteins (red) and attachment proteins (green) on cell protrusions. The attachment proteins diffuse on the entire protrusions, while the fusion proteins (red) form small clusters [19]. Some fusion clusters appear to be out of the cell surfaces because of the uncertainties in fluorophore localization and color overlapping. The home-built SMLM’s localization precisions were about 15 nm in the lateral dimensions and 50 nm in the axial dimension. Figure 4(e) shows the 3D clusters identified using the current algorithm. The results are consistent with our visual intuition without the need for any input parameters.

4. Conclusion

We presented an algorithm to identify clusters in 2D and 3D point clouds using the MTF of human ocular optics. The HVICI presented in the current study requires no input parameters to produce visually satisfactory cluster identification. A comparison with a coordinate-based analysis using Voronoi tessellation demonstrated that HVICI removed the difficulties a user faced when choosing a cutoff threshold for the point density. HVICI provides unambiguous cluster analysis in a user-independent manner.