Open Access
Add to BibSonomy
Abstract
High-accuracy spot target localization is an essential optical measurement technique in fields such as astronomy and biophysics. Random noise generated during the imaging process limits further improvement of centroiding accuracy. Research for centroiding methods can no longer meet the demand for higher accuracy. This limitation is even more severe for low signal to noise ratio (SNR) imaging measurements. This paper proposes an energy filtering method based on time-domain extended image sequences, which is a typical application such as a star tracker. The energy variations of the spot in continuous sequences are analyzed, and the energy is filtered at pixel level. The filtered pixel response that is closer to real energy is involved in the calculation of the centroid. Adaptive variations of filter parameters for different energy distributions are also realized. Both simulations and laboratory experiments are designed to verify the effectiveness of the approach. The results show that this method can effectively and adaptively filter the spot energy at pixel level and further improve centroiding accuracy.
© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. INTRODUCTION
In recent years, the rapid development of aerospace technology and precise measurement, such as deep space exploration, high-resolution Earth observation, and high-precision navigation and positioning, has put forward higher requirements for spacecraft attitude measurement systems [1–3]. A star tracker, considered as the most accurate attitude measurement instrument for spacecrafts [4,5], takes star observation as measurement reference and outputs absolute attitude information with high accuracy. It images stars into pixel intensities, calculates their centroids on the pixel array, matches the observation with a star catalog, then achieves attitude determination.
One of the key technologies for improving the attitude measurement accuracy of star trackers concerns precise star spot localization. Localization technology is also widely applied in biomedical research [6,7], celestial body observation [8], super-resolution imaging [9,10], and other fields. Due to the shot noise of light and the electronic noise of image detectors, the pixel intensities of a point target fluctuate, causing random error in localization results. There are mainly two classes of methods for suppressing the influence of random noise, namely, filtering and centroid iteration. The filtering method is mainly at centroid level, and we introduce one typical application. Lockheed Martin AST series star trackers [11] from the United States and Jena-Optronik star trackers [12] from Germany use the centroiding method with threshold to calculate subpixel interpolation star location. A filtering algorithm is then utilized to suppress random noise, achieving localization accuracy of 1/50 pixel. As the demand for accuracy continues to rise, more detailed study of localization errors is carried out. In 2001, JPL laboratory conducted analysis and modeling on the localization error for star trackers [13] and concluded that the systematic S-curve error and image detector noise are two key factors affecting localization accuracy. An improved centroiding method with threshold was adopted, and the localization results were processed by a Kalman filter. The localization accuracy could be better than 1/50 pixel. It was pointed out that jointly reducing systematic and random errors is necessary for further improving localization performance. Centroid iteration research mainly focuses on innovation in methods of centroid iteration. In 2012, in the research of Shack–Hartmann wavefront sensors in the field of adaptive optics, Vargas et al. proposed an improved iteratively weighted center of gravity (IWCOG) algorithm [14]. The method continuously iterates centroids by tracking the spot target imaging profile. The maximum value of the correlation function between the discrete sampled tracking profile and the real spot profile is obtained, and thus the final centroiding results are calculated. By comparison, the IWCOG algorithm is thought to be the most accurate centroiding algorithm at present [15,16]. However, Thomas [17,18] et al. stated that the mutual correlation function could not find the optimal solution when the SNR does not satisfy the tracking condition. Moreover, the profile of the tracking spot target is greatly affected by random noise.
Synthesizing the above related studies, this paper proposes an adaptive energy filtering method based on time-domain image sequences for high-accuracy spot target localization. The method utilizes the high sampling frequency characteristics of star trackers. Single-frame star images with low SNRs are extended in time domain. The information dimension is increased by perceiving the energy change of successive associated frames to perform the energy smoothing filter. Then, the smoothed energy is used for centroid calculation. A schematic of the method principle is shown in Fig. 1. The star tracker in is the NST series star tracker of Tsinghua University that has been applied by several satellites in orbit. Unlike other methods where the centroiding outputs are filtered, the proposed method in this paper filters the spot energy at pixel level.
For random signal filtering, the common methods are mean filtering, Gaussian filtering, and others. The principle of the mean filtering is to replace the output with the average value within the filter window. This method is usually destructive to the signal and does not effectively eliminate the effects of noise [19]. The Gaussian filter is a typical and common low-pass filter. It uses a Gaussian kernel for filtering and has a good suppressing effect for Gaussian noise. However, due to the various spot target energy and profiles, it is not possible to use a general Gaussian template for filtering, and this method has a high requirement for template parameters [20]. In this paper, a Savitzky–Golay (SG) filter based on a local polynomial fit is chosen for filtering the pixel intensities of spot targets. This type of filter is widely used for data smoothing [21–23] because it can effectively preserve the useful information of the signal with easily adjustable filter parameters. Furthermore, Stein’s unbiased risk estimator (SURE) unbiased estimation [24,25] is utilized in the SG filter to achieve adaptive adjustment capability for different spot target energy distributions. Compared with traditional filtering methods and centroid iteration methods, this algorithm has the following advantages: (1) this is a deep filtering method at an energy level with higher accuracy rather than the centroid level; (2) better fit in calibration for systematic error correction; (3) this method can be adaptively applied to star spot targets with different energy distributions and does not require a priori information; and (4) this approach can be adapted to different dynamic conditions and extended for other applications.
2. SPOT TARGET ENERGY FILTERING METHOD BASED ON ADAPTIVE SAVITZKY–GOLAY FILTER
For energy smoothing of continuous star image sequences, a suitable filtering method needs to be selected for this particular filtering scenario. In this section, a suitable SG filter is applied and analyzed for its filtering principle and adaptive parameter adjustment for continuous star images.
A. Theory of Savitzky–Golay Filter
The actual stellar imaging input signal produces time-varying signal fluctuations. To handle continuous data, the basic principle of the SG filter is to perform a local polynomial fit based on local least squares with a series of measurements, and then output the corresponding fitted values instead of the measurements to achieve smooth denoising of the data [26]. This is a local polynomial regression (LPR) method [27]. The simple principle of the SG filter is shown in Fig. 2.
The actual pixel response model for the stellar imaging region for multiple measurements can be expressed as
where $i[n]$ denotes actual measured pixel response, $s[n]$ indicates ideal pixel response, single measurement noise is $w[n]$, and $n = 1,2,3 \ldots N$ is the number of measurements.The sampling window ${\rm{2M}}\;{ + }\;{{1}}$ and fitting order D are chosen as the local fitting parameters. The position of the sampling window $[\; - {\rm{M}},\; - ({\rm{M}} - {{1}}),\;\ldots ,\;{{0}},\;\ldots,\;{\rm{M - 1}},\;{\rm{M}}]$ is a polynomial independent variable $x$, and ${a_k}$ are polynomial coefficients. The polynomial expression can be represented as
SG filter regression is the fitting of this polynomial to multiple sets of data and then regressing the polynomial value at the centroid of sampling coordinates. The local regression output value is
The polynomial residuals for Eq. (2) can be derived as
The residuals are solved optimally as
Transformation of the above equation is
The matrix ${\boldsymbol{A}} = \{{u_{n,j}}\} ,{u_{n,j}} = {n^j}(- M \le n \le M,j = 0,1, \ldots ,D)$ is defined as Eq. (7), and its matrix size is $(2M + 1) \times (D + 1)$:
The local polynomial coefficient matrix is defined as ${\boldsymbol{a}} = {[{{a_0},{a_1}, \ldots ,{a_D}}]^T}$, and the pixel response measurement matrix is ${\boldsymbol{i}} = {[{i[- M], \ldots ,i[- 1],i[ 0],i[1], \ldots ,i[M]}]^T}$. These parameters can be substituted into Eq. (6), which can be obtained as
The local polynomial coefficient matrix needs to be solved:
The first term in the coefficient matrix ${P_{\text{LPR}}}(0) = {a_0}$ is the smoothing value of the current SG filter for replacement of the measured pixel response.
A continuous pixel response measurement signal for stellar imaging ${\boldsymbol{i}}$ can be substituted into the matrix for calculation ${\boldsymbol{H}} = {({{\boldsymbol{A}}^T}{\boldsymbol{A}})^{- 1}}{{\boldsymbol{A}}^T}$ to obtain the polynomial fitting coefficient. From Eq. (10), it can be got that its value is related only to the filter window size M and the partial polynomial regression order D. Therefore, these two main parameter terms are the metrics to be subsequently discussed for optimization. Because of the different sampling target morphologies and energy components, adaptive adjustmtent of these two parameter values is required.
B. SURE Unbiased Estimation Principle
The design of an optimal SG filter with window size M and LPR order D requires corresponding filtering metrics for evaluation, and it is necessary to design the corresponding unbiased estimator to observe the whole filtering process. SURE is a relatively unbiased estimation method commonly used in signal processing [28]. For SG filtering of stellar imaging localization, the minimum mean squared error (MMSE) of pixel response filtering with different window sizes and regression orders needs to be found. The mean squared error (MSE) expression of pixel filtering with the SG filter is
An unbiased estimation analysis of Eq. (11) is conducted as
Equation (12), can be decomposed into an analysis of the regression points for each measurement and is defined as
For multiplying two variables to find the expectation, if the measurement pixel response ${\boldsymbol{i}}$ is a random variable distribution $N({\boldsymbol{s}},{\sigma ^2})$ obeying the ideal signal sequence ${\boldsymbol{s}}$ and the obtained estimated signal is differentiable in the sequence interval, according to Stein’s derivation theorem [29], it can be calculated as
The above equation is taken into Eq. (13):
The expression for the unbiased estimation analysis can be obtained as
From Eq. (16), it is known that the second term of the unbiased estimation analysis is the ideal signal, and the previous term is the target that needs to be minimized to find the best signal. Therefore, the objective expression of the local minimization search is the SURE unbiased estimation:
For the sequence star images, it needs to adjust the filter parameters M, D according to the different distributions and energies of the star spot targets. The filtering effect of the sequence pixel energy is observed to get the corresponding filter optimal parameters.
C. Adaptive Savitzky–Golay Pixel Filtering Method Based on SURE Unbiased Estimation
The principle of the adaptive SG filter is shown in Fig. 3. By adjusting different SG filter local correction window parameters M and fitting orders D, the local optimum of the variation of the SURE unbiased estimation parameters is calculated. Then the corresponding optimal estimated pixel response values are output and the filtering of the sequence star images is completed.
According to the simulation and analysis of sequential star images by subsequent sections, SURE can minimize the MSE of the local fit of the SG filter for the sequential star map within a certain order. For the application conditions of a star tracker, the range of fitting orders for SURE estimation is ${\rm{D}}\; \lt \;{{3}}$. Then, on this basis, the upper limit of the number of suitable seeking windows M is selected according to different SNRs. Hence, within a reasonable range, the SG filter can achieve adaptive parameter filtering for star image sequences with different motions.
3. SIMULATION OF CENTROIDING ACCURACY WITH ADAPTIVE SAVITZKY–GOLAY PIXEL ENERGY FILTERING METHOD
A. Adaptive SG Filter Performance Simulation
According to the derivation analysis in the previous section, the SG filter needs to be studied for the star image sequences of continuous motion. Further, the optimal selection of filter parameters for stellar imaging targets at different morphological and energy positions can be calculated. The continuous motion model of spot target energy simulation is a point spread function (PSF), in the form of a two-dimensional Gaussian function as
Continuous spot target energy is discretely sampled by the image detector. This process is the integration of the diffusion function over the continuous PSF function shown in Fig. 4. The output integrated energy value is used as the pixel response. Then for the pixel position ${x_k},{y_k}$ on the image detector (pixel size as unit size), the discrete sampling expression under the continuous PSF function can be expressed as
Poisson’s random noise associated with the magnitude of the spot target is superimposed on the signal during the detector sampling process. Also, the presence of dark currents in complementary metal oxide semiconductor (CMOS) image detectors generates Poisson noise associated with dark currents. Variance of Poisson noise for random motion of photons is $\boldsymbol \sigma _e^2 = {\boldsymbol \mu _e}$, where ${\boldsymbol \mu _e}$ is the number of spot target photon input. Poisson noise due to random fluctuations of dark current electrons is $\boldsymbol\sigma _d^2 = {\boldsymbol \mu _d}$, where ${\boldsymbol \mu _d}$ is the number of dark current. There is also Gaussian random noise $\boldsymbol\sigma _s^2$ due to CMOS silicon structure noise in the imaging process. The ideal pixel response is defined as $g = K{\boldsymbol \mu _e}$, where $K(DN/e)$ is the system gain of the image detector. The pixel positions are expressed as ${x_k},{y_k}$. Since the ideal pixel response is not known during the measurement, the actual response value ${i_k}$ instead approximates it within a certain reasonable range. According to the derivation of the noise link process [30,31], the random noise signal model in the imaging process can be calculated as
Simulation of a spot target for sub-pixel shifting of time-domain extended image sequences. The direction of motion is along the $x$ axis, and the sampling period is 0.01 pixel. Sampling 200 cycles is calculated by simulation, and the center of the spot target moves two pixels in the image plane. Spot target SNR is simulated as a typical value ${\rm SNR} = 30\; {\rm dB}$. The Gaussian radius is designed as $\boldsymbol \sigma = 0.5$ to be approximated by the optical structure of the star tracker in use. Corresponding pixel Poisson noise, typical dark current noise $\boldsymbol \sigma _d^2 = 2{\rm{(}}DN)$, and Gaussian noise $\boldsymbol \sigma _d^2 = 1(DN)$ are added to the imaging model. The different phases of the image sequences within a pixel are shown schematically in Fig. 5.
According to the simulated continuous spot target image sequences (as shown in Fig. 6), for locations with higher energy, they have more sequential image sampling data during continuous motion (typical energy feature 1), where the overall continuous energy variation resembles a Gaussian curve. However, the energy is lower for the edge position of the spot target (typical energy feature 2); the energy decreases rapidly when the spot target moves through its pixel region, and the continuous energy response is a linear-like response. In this paper, these two typical sequential pixel energy distributions are discussed.
The parameters M (local fitting window) and D (local fitting order) of the SG filter are adjusted separately. The SURE MSE estimate of the pixel response versus the ideal response and true MSE are calculated for each corresponding location. The contrast curves are shown in Fig. 7.
From the simulation results, it can be concluded that for the continuous energy change feature 1:
- (1) The MSE curve is obtained by taking different local fitting window sizes M, when the local fitting order D is fixed. The fixed constraint that the local window is taken to be larger than the fitting order must be satisfied. The MMSE exists and can be obtained by the local optimum of the MSE curve.
- (2) The curve of estimation ${\boldsymbol \lambda _{{{\rm SURE}_{\text{SG}}}}}$ of true-MSE using the SURE method is similar to real curve changes. The correlation coefficient of curves can be calculated from Eq. (21), as 96.34% (${\rm{D}} = {{2}}$) and 93.63% (${\rm{D}} = {{3}}$). This result indicates that the two trends are generally consistent. The MSE of the SURE estimation can well characterize the MSE of the continuous variation of the true spot target. But when the order ${\rm{D}}\; \gt \;{{4}}$, the correlation coefficient between the two curves decreases rapidly. This is due to the fact that the high-frequency fits retain more high-frequency noise and the local coefficient of SURE cannot be estimated accurately. The SURE estimation performance is significantly degraded when order is higher:$$\boldsymbol \rho = \frac{{Cov({c_1},{c_2})}}{{{\boldsymbol \sigma _{{c_1}}}{\boldsymbol \sigma _{{c_2}}}}} = \frac{{\sum\limits_{i = 1}^n {\left[{\left({{c_{1,i}} - {{\bar c}_1}} \right)\left({{c_{2,i}} - {{\bar c}_2}} \right)} \right]}}}{{\sqrt {\sum\limits_{i = 1}^n {{{\left({{c_{1,i}} - {{\bar c}_2}} \right)}^2}}} \sqrt {\sum\limits_{i = 1}^n {{{\left({{c_{2,i}} - {{\bar c}_2}} \right)}^2}}}}}.$$
- (3) In this simulation condition, the optimal parameters of the SURE optimal estimated SG energy filter can be obtained by simulation (${\rm{M}} = {{10}}$, ${\rm{D}} = {{2}}$). The optimal parameters of the SG energy filter are obtained from the True-MSE (${\rm{M}} = {{10}}$, ${\rm{D}} = {{2}}$) and are consistent with the SURE method.
- (4) Higher local regression orders will require more data to be fitted, and therefore require larger open window sizes. High-order fitting does not bring significant MSE improvement while increasing the computational effort. For typical star spot target energy distributions, the energy feature locations of order 3 or less are well characterized by MSE.
For the analysis based on the simulation of continuous pixel energy change feature 2, the continuous energy response decreases in a shorter period of time, as shown in Fig. 8. For lower energy pixel locations, the continuous sequence image has fewer effective data segments. Compared to the higher-energy pixel response, the characteristics are as follows:
- (1) The correlation coefficient of the MSE curves of the two series can be calculated according to Eq. (21), as 93.46% (${\rm{D}} = {{2}}$). The trend is basically the same, and there are filter parameter values corresponding to the optimal MSE position. However, the performance of SURE estimation decreases significantly after ${\rm{D}}\; \gt \;{{2}}$.
- (2) Since the energy characteristics approximate the low-order curve, the MSE curve drops sharply from the beginning to ${\rm{M}}\; \lt \;{{10}}$, then ${{10}}\; \lt \;{\rm{M}}\; \lt \;{{40}}$ goes through a flat interval corresponding MSE values varying within 0.01. The choice of window M during this plateau period does not have a large impact on MSE estimates. At larger windows, the MSE estimation performance shows a significant decrease. SURE estimation of optimal SG filter parameters is ${\rm{M}} = {{13}}$, ${\rm{D}} = {{2}}$. The real MSE optimal filter parameters are ${\rm{M}} = {{15}}$, ${\rm{D}} = {{2}}$.
In summary, the local optimal MSE smoothed by the SG filter using SURE estimation of continuous sequence images can be approximated as the true MSE estimation to some extent. Due to the special distribution of the spot target, it is in different positions and needs to adjust different windows for fitting. A smaller window does not allow for effective local filtering, but a larger filtering window will cause the fitted regression values to deviate further from the true response due to the involvement of sequence positions with different energy patterns in the calculation. Therefore, it is necessary to estimate the filtering window according to the different continuous sequence pixel energy variations to find the best.
In terms of order selection, higher orders are affected by high-frequency noise, resulting in poorer estimation, while higher orders require a larger window for calculation. For typical star spot energy in a continuous sequence, pixels change as a class of Gaussian curves, and low-energy regions are localized as near quadratic curves, so for this type of typical star point continuous filtering, ${\rm{D}}\; \lt \;{{3}}$ is used for local optimal selection.
Simulation of real MSE optimal filter parameters and SURE estimated filter parameter selection for typical energy change locations with different star point overall SNR. The results are shown in the Table 1. As the SNR of the spot target increases, the number of windows to be fitted and the order of fitting are gradually reduced.
Table 1. Optimal Parameters of SG Filter with True-MSE and SURE Estimation at Different SNRs
The real-time problem of the algorithm is also an important indicator of the application. CMOS image detectors are widely used in star trackers. Different from the global shutter (GS) mode used in CCDs, electronic rolling shutter (ERS) mode in CMOS can achieve imaging data readout line by line, storing pixel energy line by line according to the SG energy filtering principle, then calculating the centering result at time 0 at time M in the ARM processor. Since SG filtering is a local polynomial fit, this algorithm is not complicated. The filter parameters can be evaluated during the star finding process, and the subsequent steady state can follow the filter parameters. Hence the whole process time delay is about M times the exposure time. For example, exposure time $ts = {{30}}\;{\rm{ms}}$, and typical star spot SG filter parameters ${\rm{M}} = {{10}}$. Its update rate can reach ${\rm{1s/}}({\rm{M}} \times ts) = {{1}}\;{\rm{s/300}}\;{\rm{ms}}\; \approx \;{{3}}\;{\rm{Hz}}$. If a higher update rate is required, the value of M can be appropriately shortened, and we can now also shorten the exposure time, while also ensuring a certain amount of sampling energy. So it can meet the control frequency of about 1–10 Hz of a satellite. This method has the real-time capability for applications such as high-precision motion estimation, delayed precision control, etc.
B. Simulation of Centroiding Accuracy by Adaptive SG Filter
An adaptive SG energy filter method is used to perform spot target centroiding accuracy simulation compared with the different methods shown in Fig. 9. Spot targets produce periodic S-curve errors during sub-pixel movement using the center of gravity method [32–35], and the calculation formula is
Fig. 9. Simulation of centroiding accuracy of adaptive SG filtering method compared with other methods.
Periodic S-curve systematic error is related only to the Gaussian radius of the diffusion function of the point target. This phenomenon is caused by discrete sampling of continuous signals, using discrete pixel centers instead of continuous energy centers. S-curve error can be obtained as [36,37]
Using the traditional threshold method, the centering error curve is covered random noise, and the ideal S-curve systematic error cannot be measured effectively. The IWCOG method is now the most accurate method of centroiding. Since the output center is fitted, there is no S-curve error from discrete sampling. Median filtering of the output centroid is a commonly used method to be compared in simulation.
The simulation results show that traditional threshold methods can no longer meet the demand for high-accuracy spot target centroiding. The IWCOG method is sensitive to noise fluctuations, and it is difficult to effectively fit iterations for spot target localization with low SNR. Although the IWCOG has no S-curve error, the improvement of its centering accuracy is limited under the influence of random noise. Median filtering of the continuous centroid after calculation is also a common method of centering to reduce the effect of random noise. However, for this approach, since the final computed output prime is calculated from the pixel response containing random noise, its filtering effect is not as good as the proposed adaptive SG energy filtering method when the same window length is used. From the simulation results, it can be concluded that the centering results of the adaptive SG pixel-level filtering method are closest to the ideal centering S-curve error.
Since the periodic S-curve is a systematic error in the case where the Gaussian radius is determined, it can be eliminated by fitting with a calibration. This paper focuses on the effect of random noise on centroiding results. Equation (24) is defined as the residual equation for the residual of the calibration elimination S-curve system error. Each method uses multi-frame calibration to remove the S-curve systematic error. For the IWCOG algorithm, there is no S-curve systematic error [38,39], so calibration is not possible. The residual random error ${\boldsymbol \delta _i}$ effects are shown in Fig. 10, and the residual random centroiding errors are shown in Table 2:
Table 2. Centroiding Error of Single Simulation (Fig. 10) and 1000 Monte Carlo Simulations
In this typical simulation condition, adding the corresponding pixel Poisson response noise, 1000 times Monte Carlo residual centroiding error simulation with different approaches is performed shown as Fig. 11. The average remaining residual centroiding errors are shown in Table 2.
The simulation results show that the adaptive SG energy filtering method proposed in this paper can effectively filter the continuous energy of spot targets at different positions. The effect of random fluctuation noise in the imaging process can be effectively attenuated. Based on the traditional threshold centroiding method, the image sequences are extended to obtain higher accuracy—an accuracy comparison improvement of 75.4% (from 0.0126 pixel to 0.0031 pixel). Compared with the IWCOG algorithm, which has the higher accuracy, this method has better noise immunity—an accuracy comparison improvement of 66.3% (from 0.0092 pixel to 0.0031 pixel).
Table 3. Parameters of Experiment Materials
C. Adaptive SG Filter Application Analysis in Dynamic Conditions
SG filtering on the energy level requires continuous image sequences in the time domain. However, in dynamic conditions, the rapid movement of the star spot causes the single pixel position to fail to capture enough filtered data. Therefore, it is necessary to analyze the applicability of the proposed method in dynamic conditions.
According to key parameters of the star tracker—focal length $f$ (mm), pixel size $p$ (µm), dynamic angular velocity $v$ (°/s), and exposure time ${t_s}$ (ms)—the star spot moving position within a sampling period is calculated as
For example, for the experimental star tracker in this paper, the parameters are shown in Table 3. The star spot shifts 0.1 pixel with 0.04°/s in one sampling period, which can be calculated as
Simulation of continuous energy distribution on single pixels under different angles is shown in Fig. 12. With slow-motion speed, continuous energy data obtained from single pixel position acquisition are sufficient for adaptive SG filtering ($v = {0.04}^\circ {\rm{/s}}$, energy data number about 70). However, as the angular velocity increases, fewer energy data can be collected ($v = {0.20}^\circ {\rm{/s}}$, energy data number about 15). According to the previous simulation analysis, adaptive SG filtering does not allow for effective unbiased estimation.
Therefore, when the collected energy data are small, SG local filtering regression can be performed on all data. M can be approximated by taking the data length of the spot target region at the target regression pixel, ${M_{\text{dynamic}}} = \text{Spot\_Length}/\Delta s$. Spot_Length is the spot target imaging area in dynamic conditions. For example, for ${{7}} \times {{7}}$ star spot distribution with 0.20°/s angular velocity, $\Delta s = 0.5\; {\rm pixel}$ can be calculated by Eq. (25), and the SG filter window can be selected as ${M_{\text{dynamic}}} = 7\; {\rm pixel}/0.5\; {\rm pixel} = 14\; {\rm pixel}$. For the fitting order D, from the simulation results, the star spot does not yet show a high level of trailing in this dynamic case; therefore, in combination with low-dynamic simulation we can choose ${\rm D} = 3$.
As the angular velocity continues to increase, the star spot quickly moves across the image plane. A single pixel can get just one frame of energy data that is no longer available for energy level filtering. Therefore, further expansion of the application is needed. According to the previous simulation analysis, the improvement of spot target localization accuracy based on energy filtering is better than direct centroiding filtering. However, when energy filtering cannot continue in high-dynamic conditions, SG filtering at the centroiding level can also be used to improve localization accuracy.
Combining the above analyses, at low-angular-velocity steady state motion, the SG energy filter method can be performed. When the speed increases further, SG centroiding filtering can be extended to improve centroiding accuracy. Dynamic experiments are designed and performed in the next sections. SG filtering applications in different dynamic conditions are shown in Fig. 13.
The experimental process of high-accuracy spot target localization is shown in Fig. 14. In this experiment, due to the limitation of turntable control accuracy, the sampling period ${\rm{T}} = {{1/12}}$ pixel is adopted to generate time-domain image sequences.
4. SPOT TARGET LOCALIZATION EXPERIMENTS
A. Laboratory Spot Target Localization Experiments
High-accuracy spot target centroiding experiments are conducted using the NST star tracker series. This series of star tracker products has been applied to several micro-satellite in-orbit verifications. The whole experimental setup is shown in Fig. 15. Instrumental parameters of the experiment are listed in Table 3. The experiments simulate as closely as possible the actual star tracker operating conditions of the sampled stellar spot targets.
Based on the acquired continuous spot target image sequences, adaptive SG filtering energy smoothing is performed using. The process of selecting the optimal filter parameters for a typical energy location is shown in Fig. 16. From the experimental results, it can be concluded that this method can be effective for filter optimal parameter selection (${\rm{M}} = {{8}}$, ${\rm{D}} = {{2}}$) and effective energy smoothing filtering for following high-accuracy centroiding calculations.
Different centroiding algorithms are used to analyze the centroiding results of continuous image sequences of spot targets obtained from the designed experiment. For the calibration of S-curve error measurement, the calibration is performed by means of multi-frame averaging at different pixel phasing. In this experiment, 50 frames at each position are acquired first. Then the error calibration of the S-curve is performed using the corresponding centroiding methods. Finally, the centroiding error obtained from a single frame is subtracted from the systematic error obtained from calibration, and the remaining residual error is calculated to compare. The experimental results are shown in Fig. 17. A comparison of the corresponding residual centroiding errors is shown in Table 4.
The following conclusions can be drawn from the experimental results:
- (1) For high-accuracy centroiding methods for spot target localization, conventional methods can achieve accuracy only up to near 0.01pixel. To further improve accuracy, time-domain expansion information needs to be carried out. Combining multi-frame image filtering implementation is an effective way.
- (2) The method of filtering after calculating the output centroids does not effectively calibrate the S-curve error. In the process of high-accuracy centroiding, there are still a lot of residuals left, and the filter parameters of the centroids filtering method are not easy to determine. Too large a filter window will make it impossible to effectively calibrate system error, and too small a window will reduce the filtering effect.
- (3) The proposed adaptive SG energy filtering method is capable of adaptively finding the optimal filter parameters by combining the frame energy response relationships before calculating the centroids. This method can effectively smooth the response energy as much as possible without losing point target energy for calibration of S-curve errors and reduction of random errors.
A comparison of the final experimental centroiding residual error shows 66% improvement in accuracy relative to the threshold method and 57.6% improvement to the IWCOG method. This result verifies the effectiveness of the proposed adaptive SG energy filtering algorithm.
Table 4. Experimental Results on Centroiding Error of Image Sequences
B. Real Sky Experiments
In this section, a ground observation experiment of a star tracker is designed to test the efficiency of the proposed algorithm. Ground experiments to capture real stars are one of the closest experimental methods to actual star tracker imaging. The experimental platform and real sky observation are shown schematically in Fig. 18. The star tracker is fixed on the test platform. The turntable allows the star tracker to image the appropriate imaging sky area. The Earth’s rotation is a very precise motion, so continuous real star spot target images can be obtained, and centroiding accuracy can also be verified. This experiment is high-accuracy spot target localization with the rotation of the Earth.
The exposure time of the experimental star tracker device is ${t_s} = {{30}}\;{\rm{ms}}$. Based on time-domain imaging analysis, the target star is shifted by 1/15 pixels per sampling period. Since a star spot can move continuously in sub-pixels, we use the adaptive SG filtering method to filter the star energy; 1000 consecutive frames of sampled images are analyzed and calculated. The results of the centroiding error for the first 200 frames are shown in Fig. 19. The result of high-accuracy centroiding after correcting the system error is shown in Fig. 20.
According to the results of real sky localization experiments, the following conclusions can be drawn:
- (1) More stable centroiding results can be obtained by using the adaptive SG energy filtering method for spot targets. In low SNR conditions, it is more conducive to the fitting correction of systematic errors.
- (2) The proposed algorithm can significantly improve centroiding accuracy from 0.0787 pixel to 0.0136 pixel (82.7% improvement). Under the influence of random noise, the traditional threshold algorithm is no longer able to perform the correction of systematic errors. Further accuracy improvements can be achieved by time-domain image sequence information.
C. SG Filter Extended Application in High-Dynamic Conditions
In high-dynamic conditions, such as orbit insertion, star spot imaging will have a trailing phenomenon. Furthermore, the difference in energy distribution between two adjacent images can be significant. Based on the dynamic energy analysis in Section 3, the star spot will cross the image plane rapidly in high-dynamic conditions. Continuous energy change data cannot be captured.
In this section, we adjust the different angular velocity movements through the rotary table to make continuous star tracker imaging (exposure time, ${t_s} = {\rm{30\; ms}}$). Combined with the actual special application, an extension method of the SG filter is proposed. Local filtering can be performed based on the change centroiding data rather than specific energy data. This application method is similar to the smoothing filtering of centroiding. By experimental analysis, unlike the energy change, smooth filtering is locally approximated as steady state motion during the continuous motion. Therefore, in the dynamic experiments, we used the SG filter order ${\rm{D}} = {{1}}$. The window size M around the trailing length is selected depending on the amount of data taken because, unlike energy filtering, choosing too long a window for centroiding filtering can make local centroiding changes imperceptible.
The results of the dynamic star spot experiment are shown in Fig. 21. In dynamic conditions of 1°/s, centroiding errors can be improved from 0.5570 pixel to 0.0854 pixel with the SG centroiding filter method (${\rm{M}} = {{7}}$, ${\rm{D}} = {{1}}$). The angular velocity increases to 2°/s, and star spot imaging has a more serious trailing phenomenon. Centroiding error can be improved from 0.8889 pixel to 0.1835 pixel with the SG centroiding filter method (${\rm{M}} = {{12}}$, ${\rm{D}} = {{1}}$).
As the angular velocity increases, the star spot exhibits a trailing phenomenon during exposure imaging. The longer the trailing length, the greater the angular velocity, and more star energy is distributed over more pixels. Therefore, the positioning accuracy is more affected by random noise in high-dynamic conditions. In this case, SG filtering at centroiding level can effectively improve localization accuracy. This method can be used as an extension of SG filtering in high-dynamic conditions.
Also in high-dynamic conditions, we can filter the quaternions of the continuous variation to perform accuracy improvement at the attitude level. This part of the discussion is beyond the scope of this paper but can be carried out as an extension of future work.
5. CONCLUSION
High-accuracy spot target localization is an important requirement for deep space exploration, super-resolution imaging, biomedical imaging, and other research. As the demand for accuracy in these areas increases, spot target localization for specific frames can no longer be further improved. Time-domain extended image sequences are an effective means of achieving accuracy improvements. However, most current applications are filtered after the output of the centroid calculations, which can lead to more accurate sub-pixel localization information not being available. The proposed method, which performs filtering at the pixel energy level, is able to incorporate the energy expression of image sequences. For different spot target energy distributions, adaptive local filtering can achieve the goal of retaining as much energy information as possible. This method is a deeper filtering method compared with other methods, and can effectively improve spot target localization accuracy. The experimental and simulation results show that the method can effectively adapt to different spot target patterns for optimal selection of filter parameters to achieve pixel energy level filtering. Compared with the conventional method, the centroiding accuracy of 0.01 pixel can be improved to around 0.003 pixel (70% improvement). For other filtering iterations, it is also possible to improve the centroiding accuracy of 0.008 pixel by 62.5%. Further, we designed a real sky experiment to be closer to real working conditions. Real sky experiments also verify the effectiveness of the algorithm. Also, for dynamic conditions, we make a brief analysis and propose the corresponding SG filter extension application. This approach can be applied to other application scenarios with continuous spot target imaging.
Funding
National Natural Science Foundation of China (51827806, 51522505); Top Young Talents of Beijing High-level Innovation and Entrepreneurship (G04070017).
Acknowledgment
The authors acknowledge support from TY-Space Technology (Beijing) Ltd. for its cooperation in the experiment.
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but maybe obtained from the authors upon reasonable request.
REFERENCES
1. X. Chen, F. Xing, Z. You, X. Zhong, and K. Qi, “On-orbit high-accuracy geometric calibration for remote sensing camera based on star sources observation,” IEEE Trans. Geosci. Remote Sens. 60, 5608211 (2021). [CrossRef]
2. X. Liu, F. Xing, S. Fan, and Z. You, “A compressed and high-accuracy star tracker with on-orbit deployable baffle for remote sensing CubeSats,” Remote Sens. 13, 2503 (2021). [CrossRef]
3. M. Wang, Y. Cheng, B. Yang, S. Jin, and H. Su, “On-orbit calibration approach for optical navigation camera in deep space exploration,” Opt. Express 24, 5536–5554 (2016). [CrossRef]
4. D. Spiller and F. Curti, “A geometrical approach for the angular velocity determination using a star sensor,” Acta Astronaut. (2020). [CrossRef]
5. J. Lu, M. Hu, Y. Yang, and M. Dai, “On-Orbit calibration method for redundant IMU based on satellite navigation & star sensor information fusion,” IEEE Sens. J. 20, 4530–4543 (2020). [CrossRef]
6. E. Betzig, G. H. Patterson, R. Sougrat, O. W. Lindwasser, S. Olenych, J. S. Bonifacino, M. W. Davidson, J. Lippincott-Schwartz, and H. F. Hess, “Imaging intracellular fluorescent proteins at nanometer resolution,” Science 313, 1642–1645 (2006). [CrossRef]
7. S. Manley, J. M. Gillette, G. H. Patterson, H. Shroff, H. F. Hess, E. Betzig, and J. Lippincott-Schwartz, “High-density mapping of single-molecule trajectories with photoactivated localization microscopy,” Nat. Methods 5, 155–157 (2008). [CrossRef]
8. J. P. Greco, P. van Dokkum, S. Danieli, S. G. Carlsten, and C. Conroy, “Measuring distances to low-luminosity galaxies using surface brightness fluctuations,” Astrophys. J. 908, 24 (2021). [CrossRef]
9. A. Yildiz, J. Forkey, S. McKinney, T. Ha, Y. Goldman, and P. Selvin, “Myosin V walks hand-over-hand: single fluorophore imaging with 1.5-nm localization,” Science 300, 2061–2065 (2003). [CrossRef]
10. E. F. Schlafly and D. P. Finkbeiner, “Measuring reddening with Sloan Digital Sky Survey stellar spectra and recalibrating SFD,” Astrophys. J. 737, 103 (2011). [CrossRef]
11. A. G. V. De Brum, M. Fialho, M. Selingardi, N. Borrego, and J. Lourenço, “The Brazilian autonomous star tracker—AST,” WSEAS Trans. Syst. 12, 459–470 (2013).
12. U. Schmidt, C. Elstner, and K. Michel, “ASTRO 15 star tracker flight experience and further improvements towards the ASTRO APS star tracker,” in AIAA Guidance, Navigation and Control Conference and Exhibit (2008), p. 6649.
13. B. R. Hancock, R. C. Stirbl, T. J. Cunningham, B. Pain, C. J. Wrigley, and P. G. Ringold, “CMOS active pixel sensor specific performance effects on star tracker/imager position accuracy,” Proc. SPIE 4284, 43–53 (2001). [CrossRef]
14. J. Vargas, R. Restrepo, J. Estrada, C. Sorzano, Y.-Z. Du, and J. Carazo, “Shack–Hartmann centroid detection using the spiral phase transform,” Appl. Opt. 51, 7362–7367 (2012). [CrossRef]
15. S. Adatrao and M. Mittal, “Error analysis in determining the centroids of circular objects in images,” Comput. Vis. Sci. 18, 145–155 (2017). [CrossRef]
16. H. M. Chulani, “Weighted Fourier phase slope as a centroiding method in a Shack–Hartmann wavefront sensor for adaptive optics in astronomy,” Ph.D. thesis (Universidad de La Laguna, 2017).
17. S. Thomas, T. Fusco, A. Tokovinin, M. Nicolle, V. Michau, and G. Rousset, “Comparison of centroid computation algorithms in a Shack–Hartmann sensor,” Mon. Not. R. Astron. Soc. 371, 323–336 (2006). [CrossRef]
18. S. Thomas, “Optimized centroid computing in a Shack-Hartmann sensor,” Proc. SPIE 5490, 1238–1246 (2004). [CrossRef]
19. P. Zhang and F. Li, “A new adaptive weighted mean filter for removing salt-and-pepper noise,” IEEE Signal Process. Lett. 21, 1280–1283 (2014). [CrossRef]
20. J.-M. Geusebroek, A. W. Smeulders, and J. Van De Weijer, “Fast anisotropic gauss filtering,” IEEE Trans. Image Process. 12, 938–943 (2003). [CrossRef]
21. W. H. Press and S. A. Teukolsky, “Savitzky–Golay smoothing filters,” Comput. Phys. 4, 669–672 (1990). [CrossRef]
22. M. Sadeghi, F. Behnia, and R. Amiri, “Window selection of the Savitzky–Golay filters for signal recovery from noisy measurements,” IEEE Trans. Instrum. Meas. 69, 5418–5427 (2020). [CrossRef]
23. H. Kordestani and C. Zhang, “Direct use of the Savitzky–Golay filter to develop an output-only trend line-based damage detection method,” Sensors 20, 1983 (2020). [CrossRef]
24. I. Z. Apostolakis, J. S. Shin, C. Meral, J.-L. Robert, A. Sadeghi, and F. Vignon, “Adaptive slow-time singular value thresholding (SVT) based on Stein’s unbiased risk estimate (SURE) for ultrasound image random noise reduction,” in IEEE International Ultrasonics Symposium (IUS) (IEEE, 2020), pp. 1–4.
25. X.-P. Zhang and M. D. Desai, “Adaptive denoising based on SURE risk,” IEEE Signal Process. Lett. 5, 265–267 (1998). [CrossRef]
26. R. Cao, Y. Chen, M. Shen, J. Chen, J. Zhou, C. Wang, and W. Yang, “A simple method to improve the quality of NDVI time-series data by integrating spatiotemporal information with the Savitzky-Golay filter,” Remote Sens. Environ. 217, 244–257 (2018). [CrossRef]
27. J. M.-T. Wu, M.-H. Tsai, Y. Z. Huang, S. H. Islam, M. M. Hassan, A. Alelaiwi, and G. Fortino, “Applying an ensemble convolutional neural network with Savitzky–Golay filter to construct a phonocardiogram prediction model,” Appl. Soft Comput. 78, 29–40 (2019). [CrossRef]
28. P. C. Bellec and C.-H. Zhang, “Second-order Stein: SURE for SURE and other applications in high-dimensional inference,” Ann. Stat. 49, 1864–1903 (2021). [CrossRef]
29. P. C. Bellec and C.-H. Zhang, “Second order Stein: SURE for SURE and other applications in high-dimensional inference,” arXiv:1811.04121 (2018).
30. H. Tian, Noise Analysis in CMOS Image Sensors (Stanford University, 2000).
31. A. Boukhayma, A. Caizzone, and C. Enz, “A CMOS image sensor pixel combining deep sub-electron noise with wide dynamic range,” IEEE Electron Device Lett. 41, 880–883 (2020). [CrossRef]
32. R. Liu, Z. Yang, Q. Chen, G. Liao, and W. Zhen, “GNSS multi-interference source centroid location based on clustering centroid convergence,” IEEE Access 9, 108452–108465 (2021). [CrossRef]
33. X. Meng, Y. Wu, M. Wang, and H. Cheng, “Research on the accuracy evaluation method of ePSF model reconstruction,” in IEEE 6th International Conference on Computer and Communications (ICCC) (IEEE, 2020), pp. 1573–1577.
34. X. Wei, J. Xu, J. Li, J. Yan, and G. Zhang, “S-curve centroiding error correction for star sensor,” Acta Astronaut. 99, 231–241 (2014). [CrossRef]
35. X. Wei, J. Xu, and G. Zhang, “S-curve error compensation of centroiding location for star sensors,” Opt. Precis. Eng. 21, 849–857 (2013). [CrossRef]
36. J.-Q. Sun, J. Zhou, Z. Zhang, and Y.-P. Zhang, “Centroid location for space targets based on energy accumulation,” Opt. Precis. Eng. 19, 3043–3048 (2011).
37. X. Wan, G. Wang, X. Wei, J. Li, and G. Zhang, “Star centroiding based on fast Gaussian fitting for star sensors,” Sensors 18, 2836 (2018). [CrossRef]
38. A. Vyas, M. Roopashree, and B. R. Prasad, “Centroid detection by Gaussian pattern matching in adaptive optics,” arXiv:0910.3386 (2009).
39. V. Akondi and B. Vohnsen, “Myopic aberrations: impact of centroiding noise in Hartmann Shack wavefront sensing,” Ophthal. Physiol. Opt. 33, 434–443 (2013). [CrossRef]
