Open Access
Add to BibSonomy
Abstract
Nanocomputed tomography (nanoCT) is an effective tool for the nondestructive observation of 3D structures of nanomaterials; however, it requires additional correction phantom to reduce artifacts induced by the focal drift of the X-ray source and mechanical thermal expansion. Drift correction without a correction phantom typically uses rapidly acquired sparse projections to align the original projections. The noise and brightness difference in the projections limit the accuracy of existing feature-based methods such as locality preserving matching (LPM) and random sample consensus (RANSAC). Herein, a rough-to-refined correction framework based on global mixed evaluation (GME) is proposed for precise drift estimation. First, a new evaluation criterion for projection alignment, named GME, which comprises the structural similarity (SSIM) index and average phase difference (APD), is designed. Subsequently, an accurate projection alignment is achieved to estimate the drift by optimizing the GME within the proposed correction framework based on the rough-to-refined outlier elimination strategy. The simulated 2D projection alignment experiments show that the accuracy of the GME is improved by 14× and 12× than that of the mainstream feature-based methods LPM and RANSAC, respectively. The proposed method is validated through actual 3D imaging experiments.
© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
In nanocomputed tomography (nanoCT), gradual temperature changes inside the cabinet often occur during the scanning process, resulting in X-ray focal spot drift and mechanical thermal expansion [1,2]. Such small changes are not easily detected in microCT because of its resolution. However, this gradual drift severely deteriorates the reconstruction quality of laboratory nanoCT. Hence, it is essential to develop a high-precision projection drift correction method for accurate 3D reconstruction in laboratory nanoCT.
Since the proposal of the thermal drift correction method based on reference projections by Sasov et al. [3] in 2008, the rigid alignment between the original and the reference projections has attracted widespread attention for drift correction in nanoCT. This idea of image alignment has also been widely applied in computer vision, remote sensing image matching, and drift correction in microscopy.
Existing common image alignment methods can be classified into three categories: grayscale-domain-based, frequency-domain-based, and feature-based methods. The essence of the grayscale-domain-based methods is to construct a similarity metric function to calculate the optimal alignment parameters. Commonly used methods include the sum of squared differences (SSD) [4], entropy correlation coefficient (ECC) [5], and mutual information (MI) [6]. However, these methods are sensitive to images with low contrast and large brightness differences. Recently, researchers have made efforts to improve their accuracy and robustness. Cnossen et al. [7] proposed to achieve accurate drift estimation in single-molecule localization microscopy (SMLM) [8,9] by optimizing the minimum entropy of point-based datasets.
Frequency-domain-based methods use the relationship between the projection movement and frequency-domain information. Anuta et al. [10] realized image alignment by establishing a relationship between the frequency-domain phase and image translation. Manuel et al. [11] proposed a fast frequency-domain alignment method that uses matrix multiplication for upsampling in the target neighborhood without compromising the accuracy while significantly reducing the computation time. This method is known as the single-step DFT algorithm. Although the aforementioned methods produce highly accurate results, they are sensitive to noise.
Feature-based methods realize accurate alignment by extracting the local robustness features of images and constructing feature matching relationships. Currently, the scale-invariant feature transform (SIFT) [12] and the speeded up robust features (SURF) method [13] are well-known methods, involving two main steps: feature description and matching. However, the initially matched features have many outliers [14] because of noise and other random factors (e.g., brightness difference). Therefore, many improved methods have been proposed. Mainstream methods, such as random sample consensus (RANSAC) [15] and locality preserving matching (LPM) [16], eliminate outliers based on the positions of features.
Although these methods have shown good results in their fields, a direct application to nanoCT is associated with some drawbacks. On the one hand, the high noise and brightness differences in projections reduce the accuracy of the generic alignment method. On the other hand, the differences in the data acquisition methods used in the imaging system limit the possibility of cross-system correction of the drift. Hence, in this study, a high-precision projection alignment method is proposed to estimate the drift in nanoCT. Existing outlier elimination methods based on features have two drawbacks. First, these methods only consider the positional relationships and local information of the features. The outliers remain in the optimal feature set, which adversely affects the alignment accuracy. Second, the features have inherent errors due to the difference in the noise distribution and brightness difference in the projections. In this study, a novel outlier elimination framework was developed based on a rough-to-refined strategy in which the global mixed evaluation (GME) was optimized for an accurate projection alignment (Figs. 3 and 4). The proposed method was thoroughly evaluated through experiments. The sensitivity and effectiveness of the GME and rough-to-refined correction frameworks were validated by conducting ablation experiments (Fig. 6). In addition, the performance of the proposed method on samples with different textures was evaluated by 2D imaging experiments (Fig. 7) and 3D reconstruction experiments (Figs. 8 and 9). The main contributions of this work can be summarized as follows:
- • A new evaluation criterion (GME) for projection alignment is proposed for nanoCT, which uses complementary information in the grayscale and frequency domains. Ablation experiments show that the GME outperforms the commonly used structural similarity (SSIM) [17] index in terms of the accuracy and robustness.
- • A new outlier elimination framework is constructed for projection alignment, which provides high accuracy in a large number of experiments. Although the framework is designed for projection alignment in nanoCT, it may bring new insights in the field of image alignment.
- • Compared with existing advanced methods, the proposed method is more accurate in challenging tasks (when additional noise is added to normally acquired projections).
The remainder of this paper is organized as follows. Section 2 proves the directionality of the drift and the necessity of correction. Section 3 describes the details of the proposed method. Section 4 discusses the experimental setup. Section 5 presents the experimental results and their analysis. Finally, Section 6 concludes the paper.
2. Direction of drift
In recent years, a thermal drift phenomenon has been observed in high-resolution CT using correction phantoms such as tin spheres [2], steel spheres [18], and silicon nitride spheres [19]. These studies have shown that the thermal drift has three characteristics: a process of slowly continuous variation, randomness, and nonrepeatability. However, existing studies only considered the drift parallel to the detector. Although the effect of thermal drift in laboratory nanoCT is known to be very small in the direction perpendicular to the detector, this has not been proven experimentally. In this study, a fixed-angle 2D imaging experiment was conducted on a Siemens star to analyze the main direction of the drift; this section presents the details.
The thermal drift in nanoCT eventually leads to projection drift. Therefore, the main direction of the thermal drift was analyzed by observing the projection. A geometric model of nanoCT is introduced, as shown in Fig. 1. A Siemens star was fixed on the turntable, and projections were captured continuously at a fixed angle. During the scan, the X-ray tube was operated at 60 kV and 4.5 µA. A total of 60 projections was collected, and the exposure time was 60 s. Table 1 summarizes the system parameters of the nanoCT and the scanning parameters for the Siemens star.
Fig. 1. Simple geometric model of nanoCT. Here, SDD is the source-to-detector distance, SOD is the distance from the X-ray source to the rotation axis, and $\theta$ is the rotation angle. The spatial coordinate system is established with the center of the detector as the origin. The line connecting the X-ray source and the center of the detector is taken as the Y axis; the Z axis is perpendicular to the ground and parallel to the rotation axis of the turntable.
Table 1. System parameters of the nanoCT and scanning parameters for the Siemens star.
As the drift perpendicular to the detector moves along the scaling axis (Y-axis in Fig. 1), it is referred to as “axial drift” here. “Horizontal drift” and “vertical drift” are the drifts in the plane of the detector (X–Z plane in Fig. 1). The first projection is used as a baseline. Axial drift changes the magnification of the projections. Thus, the changes in the diameter of the star center (red circle in Fig. 2(a)) are recorded as axial drift. The horizontal and vertical drifts lead to rigid movements in the projections. Thus, the changes in the horizontal and vertical coordinates of the circle center are considered the horizontal and vertical drifts, respectively.
Fig. 2. Projections of Siemens star and drift recording. Sixty projections of the Siemens star are recorded at the same rotation angle. The first projection is a baseline. The diameter of the star center is used to calculate the axial drift of the projection. The horizontal and vertical coordinates of the circle center are used to calculate the horizontal and vertical drift, respectively. (a) First projection. (b) Final projection (60th). (c) Calculation result of drift. Left-shift and up-shift of projection are defined as the negative direction of projection drift.
Figure 2 shows the imaging results of the Siemens star. Figures 2(a) and (b) show the first and final (60th) projections of the Siemens star, respectively. The red circle denotes the same position of the projections. The position of the red circle should be the same if there is no drift. However, the projection has significant horizontal and vertical movements. In 3D imaging, the reconstructed slices can show blurring and artifacts if the projections are misaligned. Figure 2(c) shows the drift induced during scanning. The red, blue, and green curves represent the horizontal, vertical, and axial drifts, respectively. With the increase in the scanning time, the drifts are gradual and continuous in the horizontal and vertical directions. This is because the temperature of the cabinet changes during the 1 h scanning period, the feedback of the internal air conditioning is slow, and the thermoelastic effect changes the scanning geometry, leading to projection movement. The maximum recorded values of the horizontal, vertical, and axial drifts are 41.9 pixels (3.14 mm), 91.3 pixels (6.85 mm), and 0.2 pixels (15 µm) respectively. The axial drift can be ignored relative to the horizontal drift (210 times smaller) and vertical drift (457 times smaller). Thus, the projection has no deformation during scanning, and a rigid motion model is considered.
3. Method
The proposed method uses the reference projection correction framework [3]. After the original projections are obtained, sparse reference projections are immediately acquired with larger rotation steps to align the original projections. For the rotation angles without the reference projections, cubic spline interpolation was used to estimate the drift.
Figure 3 shows the correction process of the proposed method. First, SURF is used to build initial drift vectors. Histogram equalization [20] and nonlocal average denoising [21] are used to improve the accuracy of the initial drift vectors. Second, the rough-to-refined outlier elimination strategy is executed. The proposed elimination method is divided into two steps. (1) The drift vectors are classified via clustering based on the density, as shown in Fig. 4(a), and the optimal class is selected by evaluating the cluster center through GME to realize the rough elimination of the outliers, which is shown in Fig. 4(b). (2) GME is used to evaluate all the drift vectors in the selected class to achieve refined outlier elimination. The optimal position in the drift neighborhood is evaluated to eliminate the inherent error in the SURF, as shown in Figs. 4(c) and (d). Third, the refined drift vectors are used to align the original projections via DFT translation. Finally, the Feldkamp–Davis–Kress (FDK) [22] algorithm is used for reconstruction.
Fig. 3. Correction process of the proposed method. The number of reference projections is 10% of the original projections. Preprocessing includes histogram equalization and nonlocal average denoising to enhance the projection quality. The process of rough-to-refined outlier elimination is shown in Fig. 4.
Fig. 4. Outlier elimination process based on GME. (a) Drift vector clustering by DBSCAN. (b) Rough elimination of outliers by GME. (c) and (d) Refined elimination of outliers and position adjustment of drift vectors.
3.1 Measurement of GME
The proposed GME comprises grayscale and frequency-domain measurements, and it can be expressed as:
where ${F_{Intensity}}$ and ${F_{Frequency}}$ denote the measurement components in the grayscale and frequency domains, respectively.${F_{Intensity}}$ contains the information of the SSIM between the original and reference projections, and it can be written as:
The long duration of a nanoCT scan results in a brightness difference between the reference and original projections. This brightness difference leads to inaccurate SSIM. A recent study on image enhancement has shown that the combination of grayscale and auxiliary terms can improve the performance [23]. Since the brightness difference does not affect the phase of the image, the average phase difference (APD) is used to enhance the robustness, which can be expressed as:
where ${\bar{\theta }_{main}}$ and ${\bar{\theta }_{ref}}$ denote the average phases of the original and reference projections, respectively. GME combines SSIM and APD to make full use of the complementary information in the grayscale and frequency domains.3.2 Rough elimination of outliers
The initial drift vector space S is constructed by matching the features extracted from SURF. Each of the drift vectors in S indicates a possible drift. However, the drift vectors in S are inaccurate because there are numerous outliers in the original SURF. Hence, the density-based spatial clustering of applications with noise (DBSCAN) [24] is used to roughly eliminate the inaccurate drift vectors.
The category center is denoted by $s_{center}^i(i \le {N_{center}})$, where ${N_{center}}$ is the number of category centers. In Section 3.1, a new global metric named GME is introduced to evaluate the similarity between the projections. Here, the GME is used to evaluate the category center $s_{center}^i$ for rough elimination by the reference projection ${P_{ref}}$ and the original projection ${P_{main}}(s_{center}^i)$ moved by $s_{center}^i$. The class with the optimal category center evaluated by GME is selected as the set of drift vectors after rough elimination ${S_{rough}}$. Table 2 summarizes the steps involved in the rough elimination process, which are illustrated in Figs. 4(a) and (b).
Table 2. Process of rough elimination based on DBSCAN and GME.
3.3 Refined elimination of outliers and position adjustment of drift vectors
The rough elimination of the outliers based on DBSCAN and GME is presented in Section 3.2. The class with the optimal category center is taken as ${S_{rough}}$, in which most of the outliers are removed. However, only cluster centers are evaluated in the rough elimination process. Outliers remain in ${S_{rough}}$ that adversely affect the accuracy of the drift calculation. Therefore, we continue to evaluate the remaining drift vectors via GME. A threshold is introduced to control the number of features after refined elimination, which is denoted by ${\varepsilon _{refined}}$. The GME of the drift vectors in ${S_{rough}}$ greater than ${\varepsilon _{refined}}$ are extracted as the refined set ${S_{refined}}$.
The refined set of drift vectors ${S_{refined}}$ is still constructed from part of the initial SURF method. Hence, the positions of the initial features are retained. However, the positions of the features have inherent errors because of the low peak signal-to-noise ratio of the projection in nanoCT. Therefore, a method for adjusting the feature positions based on the GME is proposed. The optimal position in the ${\varepsilon _{range}} \times {\varepsilon _{range}}$ neighborhood of the refined drift vector is determined by GME. To accelerate the computation, the neighborhood is divided in steps of 0.1 pixels, and cubic spline interpolation is used to find the optimal position. The adjusted features constitute the final set of drift vectors ${S_{adjustment}}$. Table 3 summarizes the complete process. Figures 4(c) and (d) clearly illustrate the refined elimination and adjustment processes.
Table 3. Refined elimination and adjustment processes of drift vectors based on the GME.
3.4 Implementation details
The proposed method requires four input parameters: two clustering parameters and two refined elimination parameters.
First, DBSCAN requires the clustering radius and the minimum number of points in the class. The two parameters affect the number of classes. In extreme cases, if there are too few classes (such as when the number of classes is 1, i.e., the clustering radius is too large), the rough elimination does not take effect. If there are too many classes (such as when the minimum number in the class is 1), clustering will not accelerate the rough elimination process. This study recommends controlling the number of clusters controlled between 2 and 10. In our experiment, the clustering radius was set to 0.5. The minimum number of points was 4.
Second, the thresholds of the refined elimination ${\varepsilon _{refined}}$ and neighborhood range ${\varepsilon _{range}}$ were introduced. Here, ${\varepsilon _{refined}}$ controls the number of drift vectors in ${S_{refined}}$. We propose to set ${\varepsilon _{refined}}$ to make the number of drift vectors in ${S_{refined}}$ be 10% of S. ${\varepsilon _{range}}$ is the neighborhood range of the position adjustment. The magnitude of ${\varepsilon _{range}}$ affects the precision of the position adjustment. In our experiment, ${\varepsilon _{range}}$ was set as 1 pixel, and the segmentation step size in the neighborhood was 0.1 pixel.
4. Experiments
First, ablation experiments were performed to consider two key aspects of the proposed method: (1) the benefits of GME by combining SSIM and APD; (2) the advantages of the rough-to-refined strategy. Second, simulated 2D imaging experiments were used to evaluate the projection alignment accuracy of the different correction methods. Finally, the correction methods were tested for 3D reconstruction affected by drift.
4.1 Ablation experiment
The origin and reference projections of the samples listed in Table 4 were used to evaluate two aspects of the proposed method: the GME and rough-to-refined correction framework. First, the GME combines SSIM and APD to make full use of the complementary information in the grayscale and frequency domains. Here, the SSIM, APD, and GME in the proposed correction framework were used to test the accuracy of the projections. Second, the rough-to-refined correction framework progressively eliminates the outliers. The correction effects of SURF (no outlier elimination), rough elimination, and rough-to-refined (by GME) methods in 2D projection alignment were evaluated to prove that each step of the proposed method contributes to improving the projection alignment results.
Table 4. Scanning parameters for 2D imaging experiments. The projections of the samples are shown in Fig. 5(b). The ground truth represents the projection drift. The drift in pixel units is transformed according to the pixel size of the detector (75 um) and the drift in physical units.
4.2 Simulated 2D imaging experiment
The simulated 2D projection alignment experiment was used to evaluate the accuracy of the correction method for drift estimation. Five representative samples were selected. Figure 5(b) shows the projections. Figure 5(b1) shows a glass bead with a sparse texture, which was fixed using an adhesive. The center of the Siemens star in Fig. 5(b2) has a similar texture as the glass bead, whereas the texture around the center is more abundant. The bamboo stick (Fig. 5(b3)) and the starfish (Fig. 5(b4)) have rich and repetitive textures; however, the contrast of the bamboo stick is low, whereas the starfish has a high contrast. The texture of the chip (Fig. 5(b5)) is diverse and rich. The interval time of the original and reference projections is typically long in the drift correction of the nanoCT. Therefore, the noise distribution and brightness difference of these two projections were quite different. The projection drift was simulated by moving a trestle to obtain the ground truth of the projection drift. Figure 5(a) shows the process of acquiring the ground truth for projection drift. After the first projection was acquired, the X-ray device was immediately turned off. After the interval time, the X-ray device was started again, and the trestle was moved to simulate the projection with drift. The ground truth of the projection drift was calculated by the enlargement ratio (SOD/SDD) and the moving distance of the trestle. Table 4 presents the interval time and ground truth of the projection drift.
Fig. 5. Simulated 2D imaging experiment. (a) Acquisition process of drift ground truth. Since drift instability eventually leads to projection drift, we simulate thermal drift phenomena by moving the trestle to make the projection move. As the results of the Siemens star scanning experiment in Section 2, the effect of axial drift is small. Therefore, rigid motion parallel to the detector is considered. The horizontal and vertical drift of the projection are defined as X drift and Z drift. (b) Projections used in the experiments. Five samples with different textures are considered. (b1) Glass bead mixed using adhesive. (b2) Siemens star. (b3) Bamboo stick. (b4) Starfish. (b5) Chip.
Further, the accuracy of the 2D projection alignment was tested in two additional cases. First, the reference projections reduce the effect of drift by increasing the rotation step. In cases where the ambient temperature changes dramatically (such as on a summer morning), the reference projection may also require reducing the exposure time for further drift mitigation. However, this can further increase the noise in the projection. Therefore, additional noise was added to the normally acquired projection (original and reference projections have the same exposure time) to test the robustness of the correction method in this challenging case. Second, the precision of the projection alignment at different rotation angles was considered. The correction methods were evaluated by the histogram of the error distribution.
4.3 Three-dimensional imaging experiment
Three-dimensional imaging experiments were conducted to evaluate the effectiveness of the proposed method in the reconstruction slices. The samples were scanned through the full range of angles (360°). The rotation step of the original projection was ten times that of the reference projection. Table 5 presents the 3D scanning parameters and samples.
Table 5. Scanning parameters for 3D imaging experiment.
4.4 Comparative approaches
In the 2D imaging experiment, two mainstream outlier elimination methods (RANSAC and LPM) were used for comparison. In addition, drift correction methods commonly used in SMLM were considered. Calculating the drift using the correlation between frames is one of the popular methods in SMLM. Features extracted from SURF are used to simulate fluorophores in SMLM. Since there are at most two projections of the same rotation angle (original and reference projections) when performing drift correction for nanoCT, the direct cross-correlation method (DCC) [25] was used to compare with the proposed method.
In the 3D imaging experiment, the proposed method was also compared with the mainstream single-step DFT algorithm (abbreviated as DFT in our experiment) and ECC.
4.5 Evaluation indicators
In 2D projection alignment, the average error (AE) of the drift calculation was used to evaluate the accuracy of drift calculation, which can be expressed as:
In the 3D imaging experiments, the profiles of the reconstructed slices were used to evaluate the correction results.
5. Results
5.1 Ablation experiment
Figure 6 shows that the rough-to-refined correction framework produces optimal results for samples with different textures. The AE of the SURF (no outlier elimination) is higher because the original features contain a high number of outliers. When simultaneously using outliers and interiors to estimate the drift, the results may deviate from the ground truth. The AE of the rough elimination is reduced by 97.16% compared to that of the SURF. However, outliers cannot be completely eliminated by clustering. Refined elimination and position adjustment can further reduce the errors. Compared with the rough elimination, the AE of the rough-to-refined framework is reduced by 85.7%. Therefore, the proposed rough elimination and refined elimination are effective in improving the correction accuracy.
Fig. 6. Results of ablation experiments. SURF (no elimination), rough elimination, and rough-to-refined elimination framework are considered in the validity tests of the calibration framework. SSIM, APD, and GME are tested to assess the advantages of the combination. AEs of the different correction methods are shown.
Further, the SSIM, APD, and GME were used as criteria to evaluate the advantages of the combination in the rough-to-refined framework. The GME was found to be sufficiently robust in samples with different textures.
5.2 Simulated 2D imaging experiment
Figure 7(a) shows the AEs of different methods in the 2D projection alignment experiments. The accuracy of the proposed method is optimized for the five samples. Due to the inaccuracy of SURF in sparse texture samples, the proposed method has the highest AE in glass bead compared with other samples. A suggested approach is to increase the range of the neighborhood in the refined adjustment.
Fig. 7. Result of 2D imaging experiment. (a) AE of different methods for normally acquired projection. (b) AE of different methods in the challenging case. (c) Projection of meltblown with dust. (d) Ground truth of the drift measured in a previous experiment. Drift here is the relative drift of the sample. (e) Histograms of the calculated errors in the horizontal (X) and vertical (Z) directions for different correction methods.
In addition, a challenging situation was considered. Additional noise was added to the projection to evaluate the effect of the correction methods (lower than the projection quality of normal acquisition). The detector noise conforms to a Poisson distribution [26,27]. Three different noise levels (5%, 10%, and 15%) were considered; Fig. 7(b) shows the results. Each box contains AEs with different noise levels (5%,10%, and 15%). Here, “°” represents the mean value of the AEs. The height of the box indicates the distribution range of AE. In Fig. 7(b), the proposed method is denoted by GME and RANSAC by RAN for simplified representation. Figure 7(b) shows that the correction results of the different methods decrease significantly as the noise increases. The GME is more significantly affected by the additional noise in the bamboo stick than in the other samples. This is because the inherent error of SURF increases in low-contrast samples with increasing noise compared to other samples. In the implementation details in Section 3.4, the feature position adjustment range for GME is 1 pixel. Therefore, if the inherent error in the initial features exceeds 1 pixel, the excess cannot be adjusted. Nevertheless, the GME still maintains its accuracy in this challenging case, demonstrating its robustness.
Table 6 shows the mean of the AEs for different methods shown in Figs. 7(a) and (b). This demonstrates the comprehensive performance of the different correction methods by averaging the AEs of different samples. Since the projection of the normal acquisition is used in Fig. 7(a), the projection with additional noise is used in Fig. 7(b). The mean values in Figs. 7(a) and (b) correspond to the normal and challenging cases listed in Table 6, respectively. The accuracy of GME is 14×, 12×, and 6× better than that of the LPM, RANSAC, and DCC, respectively, under normal acquisition conditions. In the challenging case, the accuracy of LPM, RANSAC, DCC, and GME decreased by 2.9%, 2.3%, 2.7%, and 0.57%, respectively. This shows that the proposed method is optimal in terms of accuracy and robustness.
Table 6. Average calculation error of AE (%) shown in Figs. 7(a) and (b).
The AE distribution at different rotation angles is considered. A meltblown (Fig. 7(c)) was quickly scanned to mitigate the effect of drift on the original projections. Subsequently, the previous experimentally estimated drift (Fig. 7(d)) was added to the original projection. Figure 7(e) shows the distribution of the estimation errors; the errors of the GME are more concentrated, while the other methods may produce drift-related artifacts due to a wider error distribution.
5.2 Three-dimensional imaging experiment
Three-dimensional imaging experiments were conducted to test the correction effects of the different methods in the reconstruction slices. Table 5 presents the samples and scanning parameters used in the experiments.
Figure 8 shows the correction results of the chip reconstruction slice. Figure 8(a) shows the 3D structure of the chip, and the 12th slice is selected for display, as shown in Fig. 8(b). To illustrate the correction effect more clearly, the local area marked with blue block is selected; Fig. 8(d) shows the local magnifications of the different methods. The uncorrected slices (Fig. 8(d1)) suffer from severe blurring and double-edged artifacts, and it is difficult to distinguish the types of electronics and connection relationships. The correction results of DFT (Fig. 8(d2)) and ECC (Fig. 8(d3)) show that these methods do not produce good results, and the image quality of the reconstructed slices is worse than that of the uncorrected slices. The correction results of DCC (Fig. 8(d4)), LPM (Fig. 8(d5)), and RANSAC (Fig. 8(d6)) show that the image quality is clearer than that of the uncorrected slices. However, the line details are not as clear as those in the case of the proposed method (Fig. 8(d7)). The proposed method effectively preserves the details of the chip. Figure 8(c) shows the profile of the 61th line shown in Fig. 8(d). The results of the profile are consistent with the visualization, and the proposed method can clearly distinguish the detailed information of the chip.
Fig. 8. Reconstructed slice of the chip. (a) 3D structure of the chip. (b) 12th slice of the chip. (c) Profile of 61th line in the local magnifications. (d) Local magnifications of the blue block marked in (b) corrected using different methods.
Figure 9 shows the 3D-reconstructed slices of the bamboo stick and starfish corrected using the different methods. The reconstructed slices and local magnification of the bamboo sticks show that all the correction methods can improve the image quality of the reconstructed slices. The reconstructed slice corrected by DCC (Fig. 9(a7)) did not produce the same accuracy in the 2D projection alignment experiments (Figs. 7(a) and (b)). The reduced accuracy is attributed to the inaccurate feature extraction by SURF in low-contrast projections and the lower alignment accuracy by DCC in some projections. The DFT, RANSAC, and GME produced visually better and similar correction results. Figure 9(c) shows the profile of the 369th line. Compared with the other methods, the GME has a wider range of grayscale variation and better contrast of the reconstructed slice. RANSAC, DCC, and GME produce similar results for the starfish, as the contours of the slices are sharp and without artifacts. Two image evaluations without reference (Vollath function [28]and image entropy) were considered. Figure 9(d) shows the normalized results, which show that the GME produces clearer results than the RANSAC and DCC.
Fig. 9. Correction results of bamboo stick and starfish obtained using different methods. (a) Schematic of the 3D structure of the bamboo stick; layer 385 is shown. (a1)–(a7) Reconstructed slices corrected using different methods and local magnifications of the region of interest marked by the red box in (a1). (c) Profile indicated by the green line. (b) Schematic of the 3D structure of the starfish; layer 700 is shown. (b1)–(b7) Reconstructed slices and local magnifications of yellow box in (b1). (d) Normalized image entropy and Vollath. Vollath function is a classical image sharpness evaluation function based on self-correlation. Here, the clear image has a higher value.
6. Conclusion
Through ablation experiments, the effectiveness of the GME and rough-to-refined framework was tested by 2D projection alignment experiments. Two conclusions can be drawn: (1) GME has a higher sensitivity and robustness than its components (SSIM and APD). (2) Each step in the rough-to-refined framework contributes to drift estimation. In 2D projection alignment experiments with normal acquisition, the drift estimation accuracy of GME was improved by 14× and 12× compared to LPM and RANSAC, respectively. Further, a challenging case showed that the proposed method is hardly affected by noise. The AE distribution at different rotation angles showed that the proposed method has more focused and lower errors. In 3D reconstruction, the GME exhibited optimal contrast and tomographic results.
Since the proposed method is based on SURF, the performance of SURF may deteriorate for sparsely textured samples and samples with low contrast. Although the drift estimation accuracy was optimized by rough-to-refined elimination and feature position adjustment, the error in the drift estimation may increase compared to other samples. Therefore, we recommend users to extend the neighborhood range of the feature position adjustment to counteract the increase in the inherent error in SURF.
In summary, a new rough-to-refined outlier elimination framework was established based on the GME to precisely estimate the drift encountered in nanoCT. The proposed method can produce a highly accurate correction result. Although the proposed method has been developed for drift estimation in nanoCT, it can be used for rigid alignment in other applications such as remote sensing and medical imaging. Currently, the proposed method is based on the SURF. In the future, the feature extraction strategy can be modified to improve the robustness of drift correction in nanoCT.
Funding
The National Key Research and Development Project of China (No. 2020YFC1522002).
Disclosures
The authors declare no conflict of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. N. Flay, W. Sun, S. Brown, R. Leach, and T. Blumensath, “Investigation of the Focal Spot Drift in Industrial Cone-beam X-ray Computed Tomography,” in Digital Industrial Radiology and Computed Tomography (DIR2015)(2015).
2. D. Vavřík, I. Jandejsek, and M. Pichotka, “Correction of the X-ray tube spot movement as a tool for improvement of the micro-tomography quality,” J. Instrum. 11(01), C01029 (2016). [CrossRef]
3. A. Sasov, L. Xuan, and P. L. Salmon, “Compensation of mechanical inaccuracies in micro-CT and nano-CT,” Proceedings of SPIE - The International Society for Optical Engineering 7078, N/A (2008).
4. Bajcsy Ruzena, and, Kovačič Stane, “Multiresolution elastic matching,” Comput. Graph. Image Process. 46(1), 1–21 (1989). [CrossRef]
5. G. D. Evangelidis and E. Z. Psarakis, “Parametric Image Alignment Using Enhanced Correlation Coefficient Maximization,” IEEE Trans. Pattern Anal. Mach. Intell. 30(10), 1858–1865 (2008). [CrossRef]
6. J. Pluim, J. Maintz, and M. A. Viergever, “Mutual-information-based registration of medical images: a survey,” IEEE Trans. Med. Imaging 22(8), 986–1004 (2003). [CrossRef]
7. J. Cnossen, T. J. Cui, C. Joo, and C. Smith, “Drift correction in localization microscopy using entropy minimization,” Opt. Express 29(18), 27961–27974 (2021). [CrossRef]
8. S. T. Hess, T. Girirajan, and M. Mason, “Ultra-high resolution imaging by fluorescence photoactivation localization microscopy,” Biophys. J. 91(11), 4258–4272 (2006). [CrossRef]
9. Q. Houwink, D. Kalisvaart, S. Hung, J. Cnossen, D. Fan, P. Mos, A. Ulku, C. Bruschini, E. Charbon, and C. Smith, “Theoretical Minimum Uncertainty of Single-Molecule Localizations Using a Single-Photon Avalanche Diode Array,” Opt. Express 29(24), 39920 (2021). [CrossRef]
10. P. E. Anuta, “Spatial Registration of Multispectral and Multitemporal Digital Imagery Using Fast Fourier Transform Techniques,” IEEE Trans. Geosci. Electron. 8(4), 353–368 (1970). [CrossRef]
11. S. T. T. Manuel Guizar-Sicairos and J. R. Fienup, “Efficient subpixel image registration algorithms,” Opt. Lett. 33(2), 156 (2008). [CrossRef]
12. D. G. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” Int. J Comput. Vis. 60(2), 91–110 (2004). [CrossRef]
13. H. Bay, A. Ess, T. Tuytelaars, and L. V. Gool, “Speeded-Up Robust Features (SURF),” Comput. Vis. Image. Underst. 110(3), 346–359 (2008). [CrossRef]
14. J. Ma, H. Zhou, J. Zhao, Y. Gao, J. Jiang, and J. Tian, “Robust Feature Matching for Remote Sensing Image Registration via Locally Linear Transforming,” IEEE Trans. Geosci. Remote Sensing 53(12), 6469–6481 (2015). [CrossRef]
15. P. Torr and A. Zisserman, “MLESAC: A New Robust Estimator with Application to Estimating Image Geometry,” Comput. Vis. Image. Underst. 78(1), 138–156 (2000). [CrossRef]
16. J. Ma, J. Zhao, J. Jiang, H. Zhou, and X. Guo, “Locality Preserving Matching,” Int. J Comput. Vis. (2019).
17. W. Zhou, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. on Image Process. 13(4), 600–612 (2004). [CrossRef]
18. J. Hiller, M. Maisl, and L. M. Reindl, “Physical characterization and performance evaluation of an x-ray micro-computed tomography system for dimensional metrology applications,” Comput. Vis. Image. Underst. 23(8), 085404 (2012). [CrossRef]
19. F. D. Oliveira, D. Maurício, V. C. Nardelli, F. A. Arenhart, and G. D. Donatelli, “Characterization and Correction of Geometric Errors Induced by Thermal Drift in CT Measurements,” Key Eng. Mater. 613, 327–334 (2014). [CrossRef]
20. A. J. Stark, “Adaptive image contrast enhancement using generalizations of histogram equalization,” IEEE Trans. on Image Process. 9(5), 889–896 (2000). [CrossRef]
21. A. Buades, B. Coll, and J. M. Morel, “A non-local algorithm for image denoising,” in Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference (2005).
22. L. A. Feldkamp, L. C. Davis, and J. W. Kress, “Practical cone-beam algorithm,” J. Opt. Soc. Am. A 1(6), 612–619 (1984). [CrossRef]
23. S. J. Cho, S. W. Ji, J. P. Hong, S. W. Jung, and S. J. Ko, “Rethinking Coarse-to-Fine Approach in Single Image Deblurring,” (2021).
24. M. Ester, H. P. Kriegel, J. Sander, and X. Xu, “A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise,” AAAI Press (1996).
25. Y. Wang, J. Schnitzbauer, Z. Hu, X. Li, Y. Cheng, Z. L. Huang, and B. Huang, “Localization events-based sample drift correction for localization microscopy with redundant cross-correlation algorithm,” Opt. Express 22(13), 15982–15991 (2014). [CrossRef]
26. J. Xin, L. Liang, S. Le, and Z. Chen, “Improved total variation based CT reconstruction algorithm with noise estimation,” in Spie Optical Engineering + Applications (2012).
27. S. Rose, M. S. Andersen, E. Y. Sidky, and X. Pan, “Noise properties of CT images reconstructed by use of constrained total-variation, data-discrepancy minimization,” Med. Phys. 42(5), 2690–2698 (2015). [CrossRef]
28. D. Vollath, “Automatic focusing by correlative methods,” J. Microsc. 147(3), 279–288 (1987). [CrossRef]
