Abstract
In this paper, we apply the frequency transfer function formalism to analyze the red, green and blue (RGB) phase-shifting fringe-projection profilometry technique. The phase-shifted patterns in RGB fringe projection are typically corrupted by crosstalk because the sensitivity curves of most projection-recording systems overlap. This crosstalk distortion needs to be compensated in order to obtain high quality measurements. We study phase-demodulation methods for null/mild, moderate, and severe levels of RGB crosstalk. For null/mild crosstalk distortion, we can estimate the searched phase-map using Bruning’s 3-step phase-shifting algorithm (PSA). For moderate crosstalk, the recorded data is usually preprocessed before feeding it into the PSA; alternatively, in this paper we propose a computationally more efficient approach, which combines linear crosstalk compensation with the phase-demodulation algorithm. For severe RGB crosstalk, we expect non-sinusoidal fringes’ profiles (distorting harmonics) and a significant uncertainty on the linear crosstalk calibration (which produces pseudo-detuning error). Analyzing these distorting phenomena, we conclude that squeezing interferometry is the most robust demodulation method for RGB fringe-projection techniques. Finally, we support our conclusions with numerical simulations and experimental results.
© 2016 Optical Society of America
1. Introduction
Multichannel red, green and blue (RGB) color encoding is a very useful approach in fringe projection profilometry, which allows simultaneous acquisition of three linearly independent phase-shifted fringe patterns for single-shot measurements [1–6]. Pioneer applications of RGB fringe-projection are due to Wust and Capson [1], who in turn used the phase-demodulation methods by Bruning et al. [7] and Takeda et al. [8]. In RGB fringe-projection profilometry, a digital projector casts simultaneously up-to three sinusoidal phase-shifted fringe patterns over the tridimensional (3D) object under test, and these fringe patterns became phase-modulated proportionally to the height profile of the object [1–6]. Under ideal conditions, we can estimate the searched modulating phase from the RGB encoded data using Bruning’s 3-step phase-shifting algorithm (PSA) [1,7]. However, most color cameras and digital projectors have overlapping spectral sensitivities, which produce RGB-crosstalk distortion [9]. Phase-shifted fringe patterns distorted by RGB crosstalk have non-sinusoidal profiles and their phase-steps differ from the nominal value; this is particularly problematic for few-step algorithms [10–12]. In order to improve the quality of the measured phase proportional to the object’s profile under study, crosstalk compensation is required before feeding the data into the 3-step PSA [1–6]. Other common practices in fringe projection include gamma correction and projection defocusing in order to obtain more sinusoidal profiles [1–6,13–15].
In this paper, we employ the frequency transfer function (FTF) formalism to analyze the robustness of Bruning’s 3-step PSA when applied to fringe patterns distorted by RGB crosstalk. From our analysis, we deduce an equivalent but computationally more efficient method to perform the quadrature filtering in fringe patterns with mild to moderate RGB crosstalk. We also show that Bruning’s 3-step PSA is unreliable for severe RGB crosstalk, since it introduces pseudo-detuning distortion and preserves too many distorting harmonics. When working under these conditions we strongly recommend working with the squeezing interferometry technique (which we also analyze briefly in this paper) for robust quadrature filtering. Finally, we support our conclusions with numerical simulations and by digitizing a static 3D surface (for repetitively, in order to compare all methods discussed in this paper).
2. Theoretical background
2.1 The FTF formalism and Bruning’s 3-step PSAs
For ease of exposition and the reader’s convenience, let us begin with a brief review of the FTF formalism applied to phase-shifting algorithms [12]. Our standard mathematical model for a temporal-carrier fringe pattern is given by:
where represents the background signal, is the fringes’ contrast function, and is the searched modulating phase. Finally, the temporal carrier frequency is assumed to be known. Taking the temporal Fourier transform of Eq. (1), we have:We need to isolate one of the spectrally displaced analytic signals (without loss of generality we choose the one at ) in order to demodulate the searched phase. We do this using an -step frequency transfer function that fulfills the quadrature conditions:Note that the coefficients are complex numbers. From Eqs. (2) and (3), we have:Back on the temporal domain, we need to evaluate at in order to use all available data [12], and using the compact notation results:where , and we use a “hat” to label estimated values. Once we isolate this analytic signal, it is easy to solve for all unknowns in Eq. (1) as:Since the estimated phase is wrapped within , we generally apply a phase unwrapper as last step of the demodulation process [12,16].Particularly, in this paper we will work with the 3-step PSA by Bruning et al. [7,12], for which and , resulting:
The coefficients also define the FTF, so we have after some algebraic manipulation:As illustrated in Fig. 1, this FTF fulfills the quadrature conditions (see Eq. (3)), thus rejecting the spectral components at and amplifying the searched analytic signal at . This is the ideally expected result.2.2 RGB fringe projection profilometry with 3-step phase shifting
Three-step phase-shifted fringe projection with a single-frequency spatial carrier is arguably the most commonly used approach in RGB fringe-projection profilometry (for alternative approaches see, for instance, Zhang’s review [5] and its references). Our mathematical model for these spatial-carrier open fringes is given by:
Here represents an analog low-pass filter (usually defocusing), is the frequency of the spatial carrier (in radians per pixel), and the numerical values for are within the range . The low-pass filtering and the non-linear gamma encoding allow us to attenuate nonlinear intensity distortions during the fringe-projection process [14,15].Considering the profilometer setup in Fig. 2, one-at-a-time projection of the open fringes in Eq. (9) produces the following phase-shifted fringe-patterns [1]:
Here and represent the background illumination signal and the fringes' contrast function, respectively. The searched modulating phase is proportional to the profile of the 3D object under study plus a known term; therefore, it is trivial to solve for once we find using Eqs. (6) and (7).Using RGB color encoding (see Fig. 3), we can project a single full-color image and generate simultaneously the three phase-shifted fringe patterns in Eq. (10). Without loss of generality we assume , , and being projected in the red, green and blue channel, respectively. In absence of crosstalk distortion (which is feasible using dichromic filters and multiple sensors such as the ones inside of a 3-CCD camera), we can recover the searched fringe patterns from the recorded data in the RGB channels as [1,9]:
It is a well-known fact that most single-sensor color cameras have overlapping sensibilities [9]. Therefore, when using this kind of cameras for fringe projection profilometry, RGB crosstalk distorts the recorded data and some compensation algorithm is required [2–6]. Assuming the crosstalk phenomenon to be linear, we have:
The real-valued matrix elements represent the camera's sensibility on the m-th channel (for ) to a signal projected in the n-th channel ().Huang et al. [2], proposed a calibration method for matrix , which was later improved by Zhang et al. [3]. This method consists in generating only red, only green, and only blue phase-shifted open fringes and project them sequentially (instead of simultaneously) over a white reference plane. Due to crosstalk, the CCD camera records these single-color projected fringes on all three RGB channels. Analyzing separately each channel of the recorded data, we can easily calibrate all nine crosstalk coefficients proportionally to the average of the fringes’ contrast function [2,3]. Once we calibrate matrix , we can easily solve Eq. (12) for the crosstalk-compensated phase-shifted fringe-patterns as:
We can introduce these crosstalk-compensated fringe patterns into Bruning’s 3-step PSA:However, from Eqs. (13) and (14), it is easy to see that an equivalent but computationally more efficient approach to demodulate the searched analytic signal is given by:The complex-valued coefficients need to be computed only once for fixed experimental conditions. The above equation requires 4 operations per pixel and it works directly over the raw data . On the other hand, Eq. (14) required 16 operations per pixel and memory allocation for the intermediate results . Since RGB fringe-projection profilometry is meant for dynamic measurements [1–6], this represents a substantial reduction on the computational workload. As far as we know, Eq. (15) has never been reported before so this is our first contribution on this paper.3. Influence of systematic errors on the phase-demodulation algorithm
Here we illustrate how systematic errors produce low-quality phase estimations when applying Bruning’s 3-step PSA under non-ideal conditions, such as residual errors during the calibration of the crosstalk coefficients and distorting harmonics.
We start by modeling our estimation for the crosstalk matrix as , where describes the actual crosstalk values (see Eq. (12)), is unknown, and for small errors we assume . From Eqs. (13)-(15), taking , we have:
where ; therefore are unknown, and because .As we know from Eq. (5), the coefficients define univocally not only the phase-demodulation formula but also the FTF: . In Fig. 4 we show several computer-simulated realizations of these FTFs, where are uniformly distributed random numbers within . We must remark that these plots still correspond to Bruning’s 3-step PSA but being applied to fringe patterns distorted by RGB crosstalk.
From Fig. 4 we see that fails to fulfill the quadrature conditions and . Thus, the estimated analytic signal is given by:
Carefully controlled experimental conditions and/or fringe pattern normalization allow us to eliminate the background signal term [13]. Thus, taking , we have:For the trained eye, Eq. (18) predicts that the wrapped phase will be distorted by a double-frequency spurious signal, which is identical to the well-known distortion due to detuning error in phase-shifting interferometry [12]. We performed a numeric simulation to assess this pseudo-detuning distortion (see Fig. 5). First, we modulated three phase-shifted fringe patterns with a known phase. Next, we estimated the analytic signal according to Eq. (16) using random values for the crosstalk calibration uncertainty such as . Finally, we computed the wrapped difference between the actual phase and the estimated one, revealing the predicted double-frequency spurious signal. In this simulation, we used a low-frequency spatial carrier for ease of observation.Up to this point, we have modeled perfectly sinusoidal fringe patterns for ease of exposition but, in practice, these fringe patterns may be distorted by harmonics due to gain saturation, poorly estimated gamma, and/or non-lineal RGB crosstalk [12]:
Here represents the order of the highest harmonic with still-significant distorting energy, is the contrast function of the -th harmonic, and the wrapped location of the Dirac deltas is due the periodicity of the discrete-time Fourier transform [12]. When the fringe pattern is distorted by harmonics, we want to isolate in order to find the best-quality estimation of the modulating phase. However, since fails to fulfill the quadrature conditions , we have:It is easy to see from the above equation that back on the temporal domain we will have an analytic signal distorted by pseudo-detuning (because remains unfiltered), and corrupted by the distorting harmonics with the highest energy (both components of the second one, both components of the third one, and so on). Therefore, it is clear that we require a more robust approach to cope with non-sinusoidal fringes and severe RGB crosstalk.4. Robust quadrature filtering for crosstalk-distorted fringe patterns
As we know from the FTF formalism (see Appendix A in [12]), the PSAs with detuning error robustness require four or more phase-shifted interferograms. Nevertheless, the squeezing interferometry method allows us to achieve detuning robust quadrature filtering and decrease the harmonics distortion using just three phase-shifted samples [10].
The first step in squeezing interferometry is to rearrange the phase-shifted fringe patterns ( in our case), in order to produce a spatial-carrier fringe pattern with very high spatial-carrier frequency as follows:
where represent the horizontal and vertical sizes of . Rewriting the resulting spatial-carrier fringe pattern in terms of , we have:And computing its discrete Fourier transform results: ,The asterisk represents the convolution operation, and we are applying the translation property of Dirac delta: . Factorizing the spectral lobes by their spectral displacement, we can rewrite the above equation as:Comparing Eqs. (20) and (24), it is clear that the temporal and spatial carriers produce similar spectral distributions, but there are significant differences. First, the energy of spreads over a spectral region times as large as the bandwidth of ; thus, the -th harmonic has much lower spectral density than the searched analytic signal. Also, processing in the spatial Fourier domain is immune to detuning error (and/or pseudo-detuning) because we can filter-out all signals outside the area of interest, instead of just the components exactly at and . As illustrated in Fig. 6, this means we can achieve robust quadrature filtering with the following synchronous demodulation formula [10,11]:Here represents a narrow quadrature filter in the Fourier domain (usually a binary mask) which preserves only the first harmonic of the analytic signal. Once we estimate , we squeeze it back to the original size of ; finally, we compute its argument and unwrap the modulating phase, as usual [10,11].5. Experimental results
In our experiment, we worked with a static test subject for repeatability: digitizing a human-face model made of expanded polystyrene (see Fig. 7). This allows us to take a crosstalk-free measurement using sequential grayscale-based fringe projection as null test. We used this null test to assess the quality of the recovered phases from single-shot RGB fringe projection using Bruning’s 3-step PSA and squeezing interferometry. The open fringes were projected using a digital light projector, with defocusing and pre-encoding gamma [14,15]. The fringe patterns were captured using a single-sensor digital color camera that performs RGB color separation using a Bayer filter mosaic with a resolution and 8-bit color depth per channel. We calibrated the crosstalk matrix using 6-step least-squares phase-shifting measurements of the reference plane for increased reliability (to generalize from [2,3] is straightforward). Our numerical values for the crosstalk calibration matrix (see Eqs. (12) and (13)) and for the corresponding phase demodulation coefficients , required to work directly with raw data (see Eq. (15)), are given by:
According to these numerical values, we faced severe RGB-crosstalk distortion in comparison to other calibration matrices reported in the literature [2,3,6].Figure 8 shows the first step of the squeezing interferometry method [10], where we rearrange the information from the phase-shifted fringe patterns in order to generate the very high-frequency spatial-carrier fringe pattern . The beating distortion observed in is a presentation artifact (nonexistent in the actual data) as consequence of the down sampling from the original size ( pixels).
The next step is to compute , from where we isolate the searched analytic signal by means of a narrow one-sided band-pass filter. Back on the spatial domain, we squeezed the analytic signal back to the original size of , and then we estimated the wrapped phase from its argument. For ease of visualization, at this point we removed the low-frequency modulation , previously calibrated from the reference plane. See Fig. 9.
Figure 10 shows the recovered profiles from the RGB raw data, the crosstalk compensated fringe patterns, squeezing interferometry, and our null-test for illustrative purposes. These plots use the amplitude of the analytic signal as texture in order to visualize fine details.
Figure 10(a) shows the 3D profile recovered from the raw camera data, where crosstalk-distortion was so severe that the phase unwrapper failed to recover even the overall shape of the test subject. Figure 10(b) shows the result of processing the same raw data using our proposed algorithm (Eq. (15)); this result shows a drastic improvement respect to Fig. 10(a) but still exhibits the ripple pattern due to pseudo-detuning error and distorting harmonics. Figure 10(c) shows our best quality result, which we obtained with the combination of crosstalk-compensation and squeezing interferometry. Finally, Fig. 10(d) shows the result of our crosstalk-free null test for illustrative purposes; this result was obtained by means of sequential gray-scale based fringe projection.
Figure 11 shows horizontal slices of the unwrapped phase for qualitative comparison of the results estimated with Bruning’s 3-step PSA and with squeezing interferometry. This figure also demonstrates that the ripple pattern due to pseudo-detuning and distorting harmonics is observed on the modulated phase.
6. Conclusions
We analyzed the most commonly used phase-demodulation approaches for RGB fringe-projection profilometry by means of the FTF formalism. We proposed a computationally more efficient algorithm applicable for mild to moderate linear RGB-crosstalk conditions, which combines crosstalk compensation and phase demodulation, and that works directly with the raw data (Eq. (15)). We show that relying on Bruning’s 3-step PSA for severe crosstalk conditions (when modeling this phenomenon using linear combinations no longer applies) translates into a pseudo-detuning distortion and high-energy distorting harmonics. The pseudo-detuning error was somehow unexpected because the phase-step is a controlled parameter on fringe projection techniques. We also analyzed the squeezing interferometry phase-demodulation method to show that it allows us to recover high-quality results even with non-sinusoidal fringes and under severe crosstalk distortion. Finally, we supported our claims with numerical simulations and experimental results.
Acknowledgments
The authors want to thank the financial support from the Mexican National Council for Science and Technology (CONACYT), grant CB2010-157044. The authors also acknowledge Cornell University for supporting the e-print repository arXiv.org and the Optical Society of America for allowing the contributors to post their manuscript at arXiv.
References and links
1. C. Wust and D. W. Capson, “Surface profile measurement using color fringe projection,” Mach. Vis. Appl. 4(3), 193–203 (1991). [CrossRef]
2. P. S. Huang, Q. Hu, F. Jin, and F. P. Chiang, “Color-encoded digital fringe projection technique for high-speed three-dimensional surface contouring,” Opt. Eng. 38(6), 1065–1071 (1999). [CrossRef]
3. Z. Zhang, C. E. Towers, and D. P. Towers, “Time efficient color fringe projection system for 3D shape and color using optimum 3-frequency Selection,” Opt. Express 14(14), 6444–6455 (2006). [CrossRef] [PubMed]
4. S. Ma, R. Zhu, C. Quan, B. Li, C. J. Tay, and L. Chen, “Blind phase error suppression for color-encoded digital fringe projection profilometry,” Opt. Commun. 285(7), 1662–1668 (2012). [CrossRef]
5. Z. H. Zhang, “Review of single-shot 3D shape measurement by phase calculation-based fringe projection techniques,” Opt. Lasers Eng. 50(8), 1097–1106 (2012). [CrossRef]
6. J. L. Flores, J. A. Ferrari, G. García Torales, R. Legarda-Saenz, and A. Silva, “Color-fringe pattern profilometry using a generalized phase-shifting algorithm,” Appl. Opt. 54(30), 8827–8834 (2015). [CrossRef] [PubMed]
7. J. H. Bruning, D. R. Herriott, J. E. Gallagher, D. P. Rosenfeld, A. D. White, and D. J. Brangaccio, “Digital wavefront measuring interferometer for testing optical surfaces and lenses,” Appl. Opt. 13(11), 2693–2703 (1974). [CrossRef] [PubMed]
8. M. Takeda, H. Ina, and S. Kobayashi, “Fourier-transform method of fringe-pattern analysis for computer-based topography and interferometry,” J. Opt. Soc. Am. 72(1), 156–160 (1982). [CrossRef]
9. T. Katsuta, F. Ando, M. Kida, and M. Kawasaki, “Optical system for tricolor separation,” U.S. Patent: 3 602 637, issued date August 31, 1971.
10. M. Servin, M. Cywiak, D. Malacara-Hernandez, J. C. Estrada, and J. A. Quiroga, “Spatial carrier interferometry from M temporal phase shifted interferograms: Squeezing Interferometry,” Opt. Express 16(13), 9276–9283 (2008). [CrossRef] [PubMed]
11. B. Li, L. Chen, W. Tuya, S. Ma, and R. Zhu, “Carrier squeezing interferometry: suppressing phase errors from the inaccurate phase shift,” Opt. Lett. 36(6), 996–998 (2011). [CrossRef] [PubMed]
12. M. Servin, J. A. Quiroga, and J. M. Padilla, Fringe Pattern Analysis for Optical Metrology: Theory, Algorithms, and Applications (Wiley-VCH, Weinheim, 2014).
13. J. A. Quiroga and M. Servin, “Isotropic n-dimensional fringe pattern normalization,” Opt. Commun. 224(4-6), 221–227 (2003). [CrossRef]
14. H. Guo, H. He, and M. Chen, “Gamma correction for digital fringe projection profilometry,” Appl. Opt. 43(14), 2906–2914 (2004). [CrossRef] [PubMed]
15. S. Lei and S. Zhang, “Flexible 3-D shape measurement using projector defocusing,” Opt. Lett. 34(20), 3080–3082 (2009). [CrossRef] [PubMed]
16. D. C. Ghiglia and M. D. Pritt, Two-Dimensional Phase Unwrapping: Theory, Algorithms, and Software (Wiley, 1998).