Hilbert transform (HT) has been employed to compensate phase error arising from the nonlinear effect in phase shifting profilometry (PSP). However, in most common situations, pure HT may lead to a significant system error, which has a negative impact on subsequent phase error compensation. In this paper, system error from HT of non-stationary and non-continuous fringe is analyzed, and then a novel phase error suppression approach is presented. The cosine fringe without direct current (DC) component is reconstructed to eliminate the influence of non-smooth reflectivity, and the fractional periods at both ends of the reconstructed fringe are extended to generate fringe with integer number of periods. And then the HT is applied to the reconstructed and extended fringe. Finally, a revised phase-shifting algorithm is employed to calculate the phase with the fringe after HT. The proposed approach is suitable for PSP of the surface with non-smooth reflectivity (e.g. texture of complex colors), which is demonstrated in a series of experiments.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
Phase shifting profilometry (PSP) has been widely used in non-contact three-dimensional (3D) topography [1–10] due to its advantages of high speed, high accuracy, and high resolution. In PSP, the nonlinear response of the projection-capture system is the main source of phase error . Although this nonlinear response can be effectively suppressed by increasing the number of phase-shifting steps , it is still an unavoidable problem in many practical applications. Since considering the trade-off of measurement accuracy, computational complexity and time efficiency, small step number is also requested in some cases. Therefore, it is desired to compensate the phase error arising from the nonlinear response for phase-shifting with small step number. At present, algorithms for phase error compensation can be generally divided into two categories [13,14]: active, and passive. The active phase error compensation algorithms try to generate a corrected fringe pattern for projection with calibrated gamma factor [12,15] or response curve [16–18]. Passive phase error compensation algorithms estimate the optimal phase map based on captured fringe images and certain error model [19–22]. Most of the phase error compensations are based on certain error model and phase-shifting step, which means these methods should depend on sophisticated calibration and/or specific application conditions. While for phase error compensation algorithm  based on Hilbert transform (HT), neither pre-calibration of the gamma factor nor time-consuming iterative process is required. It is proved in theory that the phase information, which is calculated with captured fringe images before and after HT respectively, has error distribution with the same period, the same amplitude, and half period shift. Therefore, the phase error compensation can be achieved by averaging the phases before and after HT. This algorithm is based on captured fringe images thus will not be affected by changes in environment and system configuration, which has advantages of flexibility and simplicity.
However, captured fringe images in PSP should commonly be treated as non-stationary and non-continuous signals, since the amplitude of fringe is always modulated by the non-smooth reflectivity of the surface, meanwhile, the fringe pattern will contain fractional period when it is cut off by the edge of the surface. The modulation to amplitude is equivalent to the product of a modulating signal and a cosine signal. There will be significant errors after performing HT due to spectrum aliasing between the two signals. In addition, HT of non-continuous fringe with the fractional period will cause serious errors at both ends of the signal due to spectrum leakage. These problems greatly limit the practical application of HT in phase error compensation. In order to solve above problems, this paper presents a novel phase error suppression algorithm based on HT. First, the wrapped phase is calculated using the standard phase-shifting algorithm and will be employed to reconstruct fringes without direct current (DC) component. And the fractional periods will be extended to get new fringes which contain an integer number of periods. Then the HT will be applied to get transformed fringes, which will be used to calculate the final phase. With proposed approach including fringe reconstruction, extension, and transformation, the phase error arising from nonlinear response of the PSP system will be well suppressed. The proposed approach requests no pre-calibration and scenario limitation thus is appropriate for PSP of the surface with non-smooth reflectivity (e.g. texture of complex colors).
The rest of this paper is organized as follows: Section 2 introduces some background theories related to nonlinear phase error and the HT. Section 3 gives details of proposed phase error suppression algorithm. Section 4 shows experimental results and discussion. Section 5 is the conclusion.
2.1 Nonlinear response and related phase error in PSP
In PSP, the standard fringe pattern will be projected onto the surface of objects and then the reflected fringe will be captured by the camera. The ideal fringe image of the nth step during N-step phase-shifting can be formulated as
It has been proved that the high-order harmonic arising from nonlinear response will introduce phase error in phase-shifting, and the phase error between the actual phase and the ideal phase ϕ is 
It should be noticed that for N-step phase-shifting, only errors related to harmonic with an orderare remained, while errors related to harmonic with other order are suppressed. Actually, it is enough to consider just the N-order harmonic () and discard other items, since the value of Gs decreases rapidly with the increase of harmonic order s. Furthermore, can be neglected because it has relatively little absolute value comparing with . Therefore, Eq. (3) can be further simplified as
2.2 Hilbert transform (HT) of fringe
The HT of an ideal cosine signal can be formulated as 
The captured fringe image in Eq. (2) can be expanded with Fourier series26]. Therefore, the HT of the captured fringe image can be obtained directly with the combination of Eqs. (6) and (7). And then the phase error of phase-shifting after HT can be formulated. Comparing the phase error before and after HT, it is found that both errors present periodic distributions with the same period and amplitude, while there is a half period shift between them. Based on the above characteristics, the phase error can be compensated by taking the mean of the phase before and after HT .
2.3 Problems on non-stationary and non-continuous fringe
In general, the captured fringe image should be treated as a non-stationary and non-continuous signal, since the amplitude of fringe is always modulated by the non-smooth reflectivity of the surface, meanwhile, the fringe pattern will contain fractional period when it is cut off by the edge of the surface. The above situations lead to significant system error in phase shifting with fringe images after HT, which will make subsequent phase compensation invalid. The side effect of non-stationary and non-continuous signal to HT will be analyzed in detail. To make the analysis more simple and clear, only the one-dimensional signal is demonstrated.
2.3.1 The influence of non-smooth reflectivity
Many objects have surfaces with non-smooth reflectivity, e.g., for surface with different colors, the reflectivity will change steeply on the joint boundary of two colors. In this case, the reflectivity α should be treated as a function which contains a step component on this location, and the amplitude of captured fringe image is modulated by function, which produces a non-stationary fringe. The step signal has a wide spectrum thus there will be spectra aliasing between reflectivity function and cosine fringe, which means Bedrosian's theorem is invalid. HT of such non-stationary fringe will contain not only the component related to the HT of the cosine fringe, but also the components related to the HT of the non-smooth reflectivity . The phase-shifting algorithm for fringe after HT (Appendix A) is only suitable for the former component, while the other components will lead to a significant phase error. Simulation of spectra aliasing due to non-smooth reflectivity is shown in Fig. 1. Figure 1(a) is a non-stationary fringe whose amplitude is modulated by a step function and Fig. 1(b) is spectra of the step function and cosine fringe contained in Fig. 1(a). It can be seen obviously that spectra of the two signals have aliasing. Figure 1(c) shows phase errors of 3-step phase-shifting with fringes before and after HT respectively. There is a significant error in the location of step amplitude (where pixel index is 72), and this error propagates to both side. It should also be noticed that there are significant errors at both ends, which come from the step between the head and end of the signal. Errors at both ends will be further discussed in the following section.
2.3.2 The influence of fractional period
In practice, only discrete and infinite signal is processed. For a discrete signal of length N, its discrete Hilbert transform (DHT) is defined based on circular convolutionFig. 2.
In the captured fringe image, when the fringe is cut off by an edge of the surface, it commonly contains a fractional period, which means discontinuity between the head and the end of fringe. The circular signal from this fringe will contain the step in the connection of head and end, which results in spectrum leakage. The corresponding DHT of such fringe will contain complex components, which leads to the significant phase error in subsequent phase shifting. Simulation of spectrum leakage due to fractional period is shown in Fig. 3. Figure 3(a) is a fringe containing a fractional period. And Fig. 3(b) is the spectrum of fringe in Fig. 3(a), in which the spectrum leakage near the fundamental frequency is clearly shown. Figure 3(c) shows phase errors of 3-step phase-shifting with fringes before and after HT respectively. There are significant phase errors at both ends, and both errors propagate inward.
3.1 Phase error suppression based on HT of reconstructed fringes
It has been shown in Section 2.3.1 that when HT is applied to the fringe with modulated amplitude, it will lead to the significant phase error in phase-shifting with such transformed fringe. To avoid the influence of amplitude modulation, the actual phase calculated with captured fringe image is proposed to reconstruct fringe images without DC component
It can be approved (details please refer to the Appendix B) that the phase error between and isEq. (5), the suppression of phase error can be generally reflected by the following analysisEq. (11) should be
To verify proposed algorithm, phase error distributions of 3-step phase-shifting with different γ values are simulated. Table 1 comparatively shows statistics of simulated phase errors before and after suppression. It is obvious that proposed algorithm based on HT of reconstructed fringes dramatically reduces the phase error arising from the nonlinear response. The simulated phase error distribution corresponding to is shown in Fig. 4. The simulated distribution is generally consistent with the distribution formulated in Eq. (10), and the minor deviation may come from the approximation in formula derivation.
3.2 Phase-aided extension for fractional period
In order to avoid the spectrum leakage, the fractional periods at both ends of the fringe are extended into complete periods with the assistant of wrapped phase. The fringe extension is based on the periodicity of wrapped phase and the monotonicity of wrapped phase within single period. The fractional period at the left end can be extended with the following procedures (extension at the right end has similar procedures)
- (1) The starting position of source section is determined by searching from the left end of wrapped phase to the right end
where TH is a threshold close to 2π, which empirically takes the value 4π/3.
- (3) Copy the data from section to the left end of the fringe.
- (4) Repeat steps (1)-(3) for all the fringe images.
With the abovementioned procedures, extended fringe images are obtained to guarantee that all the fringe contain an integer number of periods. Hereafter the superscript * denotes the corresponding physical quantity related to the reconstructed fringe.
Fringe extension for 3-step phase-shifting with proposed approach is shown in Fig. 5. Fractional periods at both ends are extended to guarantee that all the fringes contain an integer number of periods, as shown in Figs. 5(b)-5(d). Figure 5(e) comparatively shows the phase errors of phase-shifting with fringe images after HT. Before fringe extension, there are significant errors at both ends due to spectrum leakage, while after fringe extension errors are well eliminated.
3.3 The complete approach
The complete approach of phase error suppression based on HT is schematically shown in Fig. 6. First, the actual wrapped phase is calculated with captured fringe images by using the standard phase-shifting algorithm. After that, the fringe images without DC component are reconstructed with . And the fractional periods in will be extended to get new fringe images which contain integer number of periods. And then the HT will be applied to get transformed fringe images . It should be guaranteed that all the N fringe images are processed with abovementioned reconstruction, extension, and transformation. Finally, the phase is calculated with by using standard phase-shifting algorithm once again. Comparing with original phase , the phase error arising from nonlinear response of the PSP system has been greatly suppressed in .
From Eq. (5), it can be inferred that the phase error of the standard phase-shifting algorithm will sharply decrease when the step number N is increasing, since decreases quickly for large N. Here we make simulations for different values of γ and step numbers of phase-shifting to demonstrate the performance of the proposed approach. The resulting maximum phase errors are shown in Fig. 7. It clearly shows that for 3-step phase-shifting, the proposed approach dramatically reduces the phase error. While for number of step larger than 4, the standard phase-shifting algorithm (before suppression) can well suppress the phase error by itself, thus the effectiveness of the suppression algorithm is trivial. That’s why most of the works on phase error suppression focus on 3-step (sometimes including 4-step) phase-shifting.
The proposed approach of phase error suppression based on HT is applied to a dedicated digital fringe projection system for PSP, which consists of a commercial DLP projector (ViewSonic PJD5555W) and an industrial camera (Imaging Source DFK-23GM021). The algorithm is executed in MATLAB R2016b. Two sets of experimental results related to a colored plate and a colorful craft are shown and discussed here.
4.1 Experiment on a colored plate
A plate with colored bands is chosen as a testing sample, and fringe images of 3-step phase-shifting are captured and processed. Some basic experimental results are shown in Fig. 8, where Figs. 8(a) and 8(b) are partial displays of plate photo and captured fringe image respectively. Figure 8(c) plots the intensity curve corresponding to the red dashed line in Fig. 8(b), which clearly shows the modulation to amplitude arising from different colors. Figure 8(d) plots two phase error distributions of 3-step phase-shifting, where the blue curve corresponds to phase error calculated with original fringe images while the red curve with fringe images after HT. It can be seen that pure HT does not provide help for phase error suppression, and errors at the border of different colors and at both ends of fringe are even larger, which demonstrate the influence of non-smooth reflectivity and fractional period, as analyzed in Section 2.3.
Some internal and final in phase error suppression with proposed approach are shown in Fig. 9, where Fig. 9(a) is the wrapped phase calculated with captured fringe images and Fig. 9(b) is one of the reconstrued fringes with phase in Fig. 9(a). The data related to fringe extension at both ends are also shown in Figs. 9(a) and 9(b). Figure 9(c) comparatively shows three distributions of different phase errors, where the blue curve is the original phase error as the same as the blue curve in Fig. 8(d). The green curve is the suppressed phase error based on HT of reconstructed fringes while without fringe extension, which shows apparent errors due to spectrum leakage at both ends. The red curve is the final error distribution after suppression with proposed algorithm. Comparing with the original phase error, the root mean square of suppressed phase error is reduced to 7.3%.
4.2 Experiment on a colorful craft
A colorful craft serves as another testing sample. Fringe patterns of 3-step phase-shifting are projected and deformed fringe images are captured. Figure 10 shows its photo and a captured fringe image. Then the phase is calculated and then be unwrapped with the well-known temporal phase unwrapping algorithm . The reconstructed 3D surface of the craft from the unwrapped phase is shown in Fig. 11(a), and an enlarged partial view is shown in Fig. 11(c). There are apparent ripples on the 3D surface due to the nonlinear response of the PSP system. While after phase error is suppressed with proposed approach, the reconstructed 3D surface and enlarged partial view are shown in Figs. 11(b) and 11(d) respectively, where it can be found that the ripples are well suppressed. Profiles of the 3D surface on the position marked with the green line in Fig. 10(a) are comparatively shown in Fig. 12, which demonstrates the effectiveness of proposed approach more clearly.
A problem in Fig. 11 is that the real surface of the craft is smooth, but the reconstructed 3D surface has some artificial errors at the junctions of different colors. However, it can be found that the artificial errors stably exist in PSP of any step number, e.g. 12-step PSP which we have tested. Inferred from Fig. 7, the 12-step phase-shifting itself can almost eliminate the nonlinear phase error completely. Therefore, the artificial error does not come from the nonlinear phase error. How to suppress or eliminate this error is beyond the topic of this paper, so we do not further discuss it here.
The nonlinear response of fringe projection system will lead to dramatic phase error for phase-shifting with small step number. To solve above problems, this paper presents a novel phase error suppression approach based on HT. Side effects of pure HT are analyzed and corresponding remedies are introduced to improve the performance of error suppression. Both theoretical analysis and experiments have demonstrated that with proposed approach including fringe reconstruction, extension, and transformation, the phase error arising from nonlinear response will be suppressed. The proposed approach requests no pre-calibration and scenario limitation thus is appropriate for PSP of a common surface with non-smooth reflectivity.
A. Phase-shifting algorithm
The cosine fringe image of the nth step during N-step phase-shifting can be formulated as
The HT will introduce a phase shift of π/2 to original fringe meanwhile suppress the DC component, thus the HT of can be formulated as
B. Phase error after HT
Assume that , both and are small values, so Eq. (5) can be further simplifiedEq. (9) can be extended into
The phase after HT can be calculated with standard phase-shifting algorithmEq. (22) into Eq. (24), we have
National Key Research and Development Program of China (2017YFF0106401, 2017YFF0106400, 2017YFB1402104); National Natural Science Foundation of China (NSFC) (61775121, 61701321); Sino-German Cooperation Group (GZ 1391); Science and Technology Planning Project of Guangdong Province (2017A010102023); Scientific and Technological Project of the Shenzhen government (JCYJ20160520160747570).
References and links
1. Y. An and S. Zhang, “Three-dimensional absolute shape measurement by combining binary statistical pattern matching with phase-shifting methods,” Appl. Opt. 56(19), 5418–5426 (2017). [CrossRef] [PubMed]
4. D. Zheng, F. Da, Q. Kemao, and H. S. Seah, “Phase-shifting profilometry combined with Gray-code patterns projection: unwrapping error removal by an adaptive median filter,” Opt. Express 25(5), 4700–4713 (2017). [CrossRef] [PubMed]
5. S. Feng, C. Zuo, T. Tao, Y. Hu, M. Zhang, Q. Chen, and G. Gu, “Robust dynamic 3-D measurements with motion-compensated phase-shifting profilometry,” Opt. Lasers Eng. 103, 127–138 (2018). [CrossRef]
6. P. Zhao, N. Gao, Z. Zhang, F. Gao, and X. Jiang, “Performance analysis and evaluation of direct phase measuring deflectometry,” Opt. Lasers Eng. 103, 24–33 (2018). [CrossRef]
7. C. Chen, Y. Wan, and Y. Cao, “Instability of projection light source and real-time phase error correction method for phase-shifting profilometry,” Opt. Express 26(4), 4258–4270 (2018). [CrossRef] [PubMed]
8. Z. Zhang, C. E. Towers, and D. P. Towers, “Time efficient color fringe projection system for 3D shape and color using optimum 3-frequency Selection,” Opt. Express 14(14), 6444–6455 (2006). [CrossRef] [PubMed]
9. Z. Zhang, “Review of single-shot 3D shape measurement by phase calculation-based fringe projection techniques,” Opt. Lasers Eng. 50(8), 1097–1106 (2012). [CrossRef]
10. S. Heist, P. Kühmstedt, A. Tünnermann, and G. Notni, “Theoretical considerations on aperiodic sinusoidal fringes in comparison to phase-shifted sinusoidal fringes for high-speed three-dimensional shape measurement,” Appl. Opt. 54(35), 10541–10551 (2015). [CrossRef] [PubMed]
11. G. H. Notni and G. Notni, “Digital fringe projection in 3D shape measurement: an error analysis,” Proc. SPIE 5144, 372–380 (2003). [CrossRef]
13. S. Zhang, “Comparative study on passive and active projector nonlinear gamma calibration,” Appl. Opt. 54(13), 3834–3841 (2015). [CrossRef]
14. Z. Cai, X. Liu, Q. Tang, X. Peng, and Y. Yin, “Comparison of active, passive and adaptive phase error compensation methods using a universal phase error model,” Proc. SPIE 10250, 102502Z (2017). [CrossRef]
16. C. Coggrave and J. M. Huntley, “High-speed surface profilometer based on a spatial light modulator and pipeline image processor,” Opt. Eng. 38(9), 1573–1582 (1999). [CrossRef]
18. P. S. Huang, C. Zhang, and F.-P. Chiang, “High-speed 3-D shape measurement based on digital fringe projection,” Opt. Eng. 42(1), 163–169 (2003). [CrossRef]
20. B. Pan, Q. Kemao, L. Huang, and A. Asundi, “Phase error analysis and compensation for nonsinusoidal waveforms in phase-shifting digital fringe projection profilometry,” Opt. Lett. 34(4), 416–418 (2009). [CrossRef] [PubMed]
22. A. Kamagara, X. Wang, and S. Li, “Nonlinear gamma correction via normed bicoherence minimization in optical fringe projection metrology,” Opt. Eng. 57(3), 034107 (2018). [CrossRef]
23. Z. Cai, X. Liu, H. Jiang, D. He, X. Peng, S. Huang, and Z. Zhang, “Flexible phase error compensation based on Hilbert transform in phase shifting profilometry,” Opt. Express 23(19), 25171–25181 (2015). [CrossRef] [PubMed]
25. M. Klingspor, Hilbert Transform: Mathematical Theory and Applications to Signal processing, Master thesis, (Linköping University, 2015).
26. F. R. Kschischang, The hilbert transform, (University of Toronto, 2006).
27. S. L. Hahn, “Comments on “A tabulation of Hilbert transforms for electrical engineers”,” IEEE Trans. Commun. 44(7), 768 (1996). [CrossRef]
28. J. Tian, X. Peng, and X. Zhao, “A generalized temporal phase unwrapping algorithm for three-dimensional profilometry,” Opt. Lasers Eng. 46(4), 336–342 (2008). [CrossRef]