Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Depth range enhancement of binary defocusing technique based on multi-frequency phase merging

Open Access Open Access

Abstract

Binary defocusing technique has demonstrated various merits for high-speed and high-accuracy three-dimensional measurement. However, the existence of excessive defocusing zone (EDZ) limits the depth range of binary defocusing system. To overcome this problem, this paper proposes a multi-frequency phase merging (MFPM) approach, which makes it possible to measure the object surface in large depth range (LDR). The method is based on our finding that for different fringe frequencies, the associated EDZs of binary defocusing system are different and not totally overlapped. Thus by merging the phase maps of multiple binary fringes, we could effectively enhance the measurement depth range. Meanwhile, a strategy to determine the optimal combination of fringe frequencies is also proposed by analyzing the phase error distribution under different defocusing degrees. Both simulations and experiments verify the effectiveness and robustness of the proposed method.

© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

1. Introduction

Fringe projection profilometry has been widely applied in numerous fields including biomedical monitoring, industrial quality control, computer vision, virtual reality and culture heritage preservation [13]. Conventional digital fringe projection (DFP) technique generally utilizes 8-bit sinusoidal patterns, which results in the following three major limitations [4]: 1) The measurement speed is restricted by the maximum frame rate of the projector, typically 120Hz; 2) Gamma calibration is mandatory due to the nonlinear gamma effect of the projector; 3) Precise synchronization between the projector and the camera is required because the digital-light-processing (DLP) technology produces grayscale images by time integration. To overcome these troubles, the binary defocusing technique has recently been proposed by utilizing 1-bit binary patterns and generating the pseudo sinusoidal patterns with projector lens defocusing effects [5,6]. By this means, the projection rate of binary pattern can be boosted up to over kHz [7], and also there is no need for gamma calibration and strict synchronization.

Though with aforementioned advantages, the binary defocusing technique is not trouble-free. Only when the defocusing degree of projector is within an appropriate range will good-quality fringes be acquired, thus it is still challenging for binary defocusing technique to measure objects in large depth range. Due to this, researchers have explored various approaches to extend the depth range by modulating traditional square binary pattern in spatial domain, which could be further divided into one-dimensional (1D) and two-dimensional (2D) optimization methods. 1D optimization methods mainly include pulse width modulation (PWM) technique [8], sinusoidal pulse width modulation (SPWM) technique [9,10] and optimal pulse width modulation (OPWM) technique [11]. These techniques could achieve better measurement quality when fringe period is relatively small but fail when fringe period is too large. To deal with this limitation, 2D optimization methods have been proposed such as area modulation techniques [12,13] and dithering techniques [14,15]. Numerous dithering techniques have been successfully applied for binary fringe modulation, such as Bayer-ordered dithering technique, error-diffusion dithering technique and their variants. Comparing with 1D optimization methods, 2D modulation methods demonstrate their advantages in improvement on measurement quality for large fringe period. In summary, both 1D and 2D optimization techniques only function for a limited range of fringe periods and even if we combine both techniques for absolute phase retrieval, their practical depth range is still limited.

Besides, alternative methods have also been proposed such as phase error compensation algorithms [16] and defocusing degree controlling approaches [17,18]. The former compensates the phase error with its mathematical fitting model. Specific algorithms have been developed to improve measurement accuracy such as Hilbert three-step and double three-step phase shifting profilometry [19]. Yet such algorithms are not robust enough and may request more projected fringes for high-accuracy situation. The latter quantifies defocusing degree to optimize system parameters, and adjusts the parameters of defocusing projection system to make it work more effectively, but still can not overcome its essential limitations of measurement depth range.

Though various methods have been developed, none of them could solve the problem of the existence of excessive defocusing zone (EDZ) for binary defocusing system. Excessive defocusing zone is where fringe has been excessively defocused and thus shows poor contrast and low signal to noise ratio (SNR) as the depth increases. Consequently, the phase information is unreliable in this zone, which causes the limitation of measurement depth range. In this paper, we propose a multi-frequency phase merging (MFPM) method to overcome this problem. The proposed method is based on our finding that for different fringe frequencies, the associated EDZs of binary defocusing system are different and not totally overlapped. Thus by carefully choosing multi-frequency binary fringes and merging the related phase maps, we could effectively enhance the measurement depth range. Our method incorporates the following major steps: 1) Determine the optimal combination of fringe frequencies by analyzing the phase error distributions under different defocusing degrees; 2) Evaluate fringe defocusing degree based on the normalized fringe modulation [20], and generate related masks for each fringe frequency; 3) Synthesize multi-frequency phase maps to generate a final merged phase map for large-depth-range (LDR) 3D reconstruction.

Section 2 explains the principle of the proposed method. Section 3 shows the simulated and experimental results to verify the performance of the proposed method; and Section 4 summarizes the paper.

2. Principle

2.1 N-step phase-shifting algorithm

Phase-shifting methods have been widely employed in optical metrology due to their accuracy and efficiency [21]. For N-step phase shifting algorithm with a constant phase shift of $2\pi /N$, the n-th fringe image obtained from a ideal digital fringe projection system can be described as,

$$I_n(x,y) = I'(x,y) + I^{\prime\prime}(x,y) \cos\left(\phi(x,y) + \frac{2n\pi}{N}\right), n = 0,1,2,\ldots,N-1.$$
where $I'(x,y)$ denotes the average intensity, $I''(x,y)$ is the intensity modulation, $\phi (x,y)$ represents the desired phase to be solved utilizing the least-squares algorithm as follows,
$$\phi(x,y) ={-}\tan^{{-}1} \left[\frac{\sum_{n=0}^{N-1} I_n(x,y) \sin\frac{2n\pi}{N}}{\sum_{n=0}^{N-1} I_n(x,y) \cos\frac{2n\pi}{N}}\right] .$$
The result of this equation is the wrapped phase that limited in the range of [$-\pi$, $+\pi$) with $2\pi$ discontinuities owing to the nature of arctangent function. Spatial or temporal phase unwrapping algorithms can be adopted to remove the $2\pi$ phase jumps. Here we employ a three-frequency temporal phase unwrapping algorithm to obtain the continuous phase map. Finally a simple calibration method based on reference plane is applied to converting the absolute phase into height information for 3D reconstruction.

Additionally, the quality of data collected can be evaluated by the data modulation $\gamma (x, y)$ indicated as follows,

$$\gamma(x,y) = \frac{2\sqrt{(\sum_{n=0}^{N-1} I_n(x,y) \sin\frac{2n\pi}{N})^{2}+(\sum_{n=0}^{N-1} I_n(x,y) \cos\frac{2n\pi}{N})^{2}}}{\sum_{n=0}^{N-1} I_n(x,y)} .$$
where the numerator is the intensity modulation $I''(x,y)$ and the denominator is the average intensity $I'(x,y)$.

2.2 High-speed binary defocusing technique

Binary defocusing technique utilizes 1-bit square pattern instead of 8-bit sinusoidal pattern, which boosts 3D shape measurement rate up to kHz. Conventionally, square binary pattern is adopted and projected for phase extraction. The intensity distribution of a square binary pattern can be mathematically formulated as follows,

$$B_i(x_p,y_p) = I'(x_p,y_p) +I^{\prime\prime}(x_p,y_p)\sum\limits_{k=0}^\infty \frac{1}{2k+1}cos\left[\left(2k+1\right)\left(\phi(x_p,y_p) + \frac{2\pi i}{N}\right)\right] ,$$
This equation incorporates an infinite sum of cosine waves, the first order harmonic of which indicates the ideal pattern used for phase shifting. Meanwhile, the undesired higher order harmonics would induce phase errors, which could be effectively alleviated by proper projector defocusing effect.

The defocus blur of the projector lens can be considered as a low-pass Gaussian filter. The modulation of a Gaussian function and its Fourier transform form are as follows,

$$G(x_p,y_p) = \frac{1}{2\pi \sigma^2}e^{-\frac{x_p^2+y_p^2}{2\sigma^{2}}} \rightleftharpoons G(\omega) = e^{-\frac{\sigma^{2} \omega^{2}}{2}} ,$$
where the defocus kernel $\sigma$ represents the defocusing level for projector lens. As shown in Fig. 1, there is a linear relationship between defocus kernel $\sigma$ and depth $z$ for the projection system [22,23], which could be expressed as follows,
$$\sigma = kR(z) = k\left[\left|\frac{D}{2}\left(\frac{z}{z_f}-1\right)\right|+\frac{rz}{v}\right] .$$
where $k$ is the coefficient, $R(z)$ denotes the radius of the blurred spot on the defocus plane that is corresponding to the radius $r$ of given pixel cross of the Digital Micro-mirror Device (DMD) unit, $D$ is the diameter of the aperture, $z$ represents the depth range of the projector lens, $z_f$ is the distance between the projector lens and the focus plane and $v$ is the distance between the DMD and projector lens. Figure 1(b) demonstrates that defocus kernel $\sigma$ shows a upward trend when the projection plane gets far away from the focus plane, which represents augmented defocusing effect.

 figure: Fig. 1.

Fig. 1. (a) Projector’s defocusing effect corresponding to depth $z$; (b) Defocus kernel $\sigma$ curve as a function of depth $z$: $z_1\,<\,z_2\,<\,z_3\,<\,z_f\,<\,z_4\,<\,z_5\,<\,z_6$.

Download Full Size | PDF

Based on the above defocusing model, we find that for single-frequency fringe, there will be an inevitable EDZ where the acquired patterns would be severely blurred with terrible SNR. Therefore, this will cause unreliable phase information and result in the limitation of measurement depth range. To overcome this problem, we propose the multi-frequency phase merging method to extend the depth range of binary defocusing technique.

2.3 Large-depth-range 3D measurement principle

2.3.1 Proposed multi-frequency phase merging method

The principle is illustrated in Fig. 2. For single-frequency binary fringe with period $T_1$ or frequency $\omega _1$, the pattern is properly defocused at depth range $z_1$ but excessively defocused at $z_2$, where the binary pattern will fail to work due to the low contrast. To illustrate the reason, the spectrum distribution at different depth range is plotted in the upper left corner of Fig. 2. The defocus kernels $\sigma _1, \sigma _2$ are corresponding to the depth range $z_1$ and $z_2$. It can be seen that with defocus kernels $\sigma _2$, all harmonics components including the fundamental frequency $\omega _1$ will been excessively suppressed by augmented defocusing effect , which is the leading cause of poor fringe contrast.

 figure: Fig. 2.

Fig. 2. Multi-frequency phase merging: $\sigma _1, \sigma _2$ are the defocusing kernels at depth $z_1$ and $z_2$, and $\omega _1, \omega _2$ are the fundamental frequencies for fringe periods $T_1, T_2$ with $\omega = \frac {2\pi }{T}$.

Download Full Size | PDF

To alleviate this problem, an intuitive thought is to introduce another fringe with relatively lower frequency $\omega _2$. Since $\omega _2 < \omega _1$, its fundamental frequency could be retained under augmented defocusing effect at depth $z_2$ while the higher-order harmonics are well removed. Therefore, good-quality fringe with frequency $\omega _2$ could be captured when frequency $\omega _1$ fails as shown in Fig. 2. This means that for different fringe frequencies, their associated EDZs are different and not totally overlapped. Due to this, the measurement depth range of binary defocusing technique could be effectively extended if we could incorporate all the frequencies for phase extraction.

2.3.2 Optimal frequency determination

However, how should we determine the optimal frequencies for binary patterns? In order to achieve LDR 3D measurement, the following algorithm has been proposed as demonstrated in Fig. 3.

 figure: Fig. 3.

Fig. 3. Steps for optimal frequency determination.

Download Full Size | PDF

Firstly, fringe period $T_1$ of highest-frequency fringe could be set according to the three-step phase-shifting algorithm adopted. In order to achieve higher phase resolution, $T_1$ should be picked as narrower as possible. With the chosen fringe period $T_1$, numerous binary defocusing techniques could be applied to generate the patterns. To demonstrate the universality of our proposed method, standard squared fringe patterns are used in this paper. And then a sequence of Gaussian filters with different defocusing kernels are applied to the generated patterns for simulating the defocusing effects in large depth range. Under this condition, the phase root mean square (RMS) errors could be obtained for evaluation. Representative simulation results have been shown in Fig. 4. For evaluation, in this paper, the Gaussian filter sizes $G$ were set in the range of 3 to 80 pixels with $\sigma = \frac {G}{6}$, which represents different defocusing degree from nearly focusing to excessive defocusing. Wrapped phase $\phi ^{t}$ is calculated from three-step phase shifting algorithm. And unwrapped phase $\varPhi ^{t}$ could be retrieved by some well-established phase unwrapping algorithms. In this research, we adopt a three-frequency phase unwrapping method. The phase RMS error distribution is obtained by comparing against the ideal phase generated by ideal sinusoidal waves. And the EDZ of fringe $T_1$ is determined as the range where its phase RMS error is greater than the preset threshold. For example, if the fringe period $T_1$ is set as 18 pixels, it can be seen from Fig. 4(a) that the phase RMS error of fringe $T_1$ increases steadily when the filter size is larger than 25 pixels, thus the range with larger filter size could be considered as the EDZ of fringe $T_1$.

 figure: Fig. 4.

Fig. 4. Simulation results for optimal frequency determination: (a) When $T_1$ = 18 pixels and $T_2$ = 24 pixels; (b) When $T_1$ = 18 pixels and $T_2$ = 36 pixels; (c) When $T_1$ = 18 pixels and $T_2$ = 54 pixels; (d) When $T_1$ = 18 pixels, $T_2$ = 36 pixels and $T_3$ = 54 pixels.

Download Full Size | PDF

Secondly, to further extend the depth range of this zone, another fringe period $T_2$ need to be carefully chosen as a lower-frequency fringe. The major steps are as follows: initializing the value of $T_2$, designing the binary patterns and analyzing the phase errors, adjusting $T_2$ until its phase errors are lower than the threshold in the EDZ range of $T_1$ . It should be noted that multiple $T_2$ values could satisfy the above criteria, thus the largest $T_2$ value should be chosen in order to ensure the least required frequencies. Figures 4(a)–4(c) illustrate several representative simulation results of different fringe period pairs $\left (T_1, T_2\right)$. For all these fringe period pairs listed, depth range could be extended compared with the single-frequency method, but the most appropriate one still need to be determined. As demonstrated in Fig. 4, the overlapped range of phase error curves is too large and this fringe period pair could not extend sufficient measuring range. Another fringe period pair illustrated in Fig. 4(c) provides a larger measuring range but its overlapped range of phase error curves is too small that may cause inaccurate measurement at the starting point of aforementioned EDZ. In contrast, fringe period pair illustrated in Fig. 4(b) provides acceptable measuring depth range while maintaining high-quality performance during its measuring range. Thus when the fringe period $T_1$ is selected as $18$ pixels, $36$ pixels is an ideal value for fringe period $T_2$.

Similarly, the depth range could be further enhanced by adopting additional fringe frequencies like fringe period $T_3$. For example, Fig. 4(d) indicates that the fringe period combination $T = \left (18, 36, 54\right)$ pixels has the superior measuring capability for larger depth range. It is clear that when the filter size is larger than 59 pixels, fringe with $T_2 = 36$ pixels gradually fails to function accurately and could be substituted by fringe with $T_3 = 54$ pixels in this zone. Thus depth range could be further enhanced compared with fringe period pair $T = \left (18, 36\right)$ pixels.

For the proposed MFPM method, more fringe frequencies could theoretically result in larger depth range. However, the more fringe frequencies, the slower measurement speed. Thus we need to choose a limited-number of fringe frequencies with optimal performance. In this paper, we present a strategy of phase merging for two frequencies $\left (\omega _1, \omega _2\right)$ and demonstrate its feasibility for depth extension.

2.3.3 Phase merging strategy

Another crucial problem to be solved is multi-frequency phase merging. In order to achieve this, the merging strategy and detailed procedures are proposed as follows.

  • • Acquisition of multi-frequency fringe sequences $I_n^{t}$ from binary defocusing system. $t$ and $n$ respectively denote fringe periods and phase shifting orders.
  • • Calculation of wrapped phase $\phi ^{t}(x, y)$ and related defocusing degree map $\gamma _{nor}^{t}(x, y)$ for each fringe frequency. In this paper, the normalized fringe modulation $\gamma _{nor}(x, y)$ is obtained as described in Eq. 7, and $\gamma _{max}(x,y)$ represents the maximum fringe contrast when a pure white fringe pattern $I_{w}(x,y)$ is projected. Through the normalization process, the influence of local textural variations will be eliminated and thus the index $\gamma _{nor}^{t}(x, y)$ could better represent the binary pattern defocusing degree. To further demonstrate the reliability of this index, fringe patterns of different frequencies and corresponding normalized modulation distributions are illustrated in Figs. 5(b)–5(e), and Fig. 5(f) shows the merged result. In these figures, the male sculpture is placed close to the focus plane with a distance of about 23 cm, while the female one is placed far away from the focus plane with a distance of 52.5 cm. From Figs. 5(b)–5(c), it can be seen that $\gamma _{nor}(x, y)$ in different depth range are obviously different and the female sculpture has very low fringe contrast. Generally data points with $\gamma _{nor}(x, y)$ below $0.1$ have terrible signal-to-noise ratio and the resultant phase value will be unreliable. Accordingly, phase information in this region can be substituted by those from lower-frequency fringe patterns as shown in Figs. 5(d)–5(e).
    $$\begin{aligned} \gamma_{nor}(x,y) & = \frac{\gamma(x,y)}{\gamma_{max}(x,y)}\\ & = \gamma(x,y) \times \frac{I'(x,y)}{I_{w}(x,y) - I'(x,y)} . \end{aligned}$$
  • • Generation of excessively defocusing mask $M^{t}(x, y)$ for each fringe frequency according to the distribution of $\gamma _{nor}(x, y)$:
    $$\left\{ \begin{array}{lr} M^{t}(x, y)=0, \quad if \gamma_{nor}^{t}(x, y)\,<\, thresh ;\\ M^{t}(x, y)=1, \quad otherwise . \end{array} \right.$$
  • • Determination of the optimal fringe period $t_{opt}$ for each pixel. For each pixel, the optimal fringe period is the minimum one from all the properly defocusing fringe periods with $M_t=1$.
  • • Phase mapping. Multi-frequency phase needs to be normalized on the basis that unwrapped phase is inversely proportional to fringe period. Thus the lower-frequency phase maps with fringe period $t$ are normalized and mapped according to the highest-frequency fringe using the following equation:
    $$\varPhi_{nor}^{t}(x,y)=\frac{t}{min{\left(T\right)}}\varPhi^{t}(x,y).$$
    T is a set of all the fringe periods.
  • • Multi-frequency phase merging with the following equation.
    $$\varPhi_{LDR}(x,y)=\varPhi^{t_{opt}}_{nor}(x,y).$$

 figure: Fig. 5.

Fig. 5. Defocusing degree maps for phase merging: (a) LDR scene consisting of two portrait sculptures of which the male one is placed at the distance of about 23 cm from the focus plane and the female one is placed at the distance of about 52.5 cm from the focus plane; (b) Fringe pattern when $T_1$ = 18 pixels; (c) Corresponding defocusing degree map when $T_1$ = 18 pixels; (d) Fringe pattern when $T_2$ = 36 pixels; (e) Corresponding defocusing degree map when $T_2$ = 36 pixels; (f) Merged fringe pattern calculated by aforementioned defocusing degree maps when $T_1$ = 18 pixels and $T_2$ = 36 pixels.

Download Full Size | PDF

3. Experiments

Experiments have been conducted to verify the performance of our proposed algorithm. As shown in Fig. 6, experimental system incorporates a DLP projection unit (model: DLP LightCrafter 6500) and a CMOS camera unit (model: Flea3 FL3-U3-13Y3M-C) that is attached with a 16 mm focal length CCTV lens (model: LM16JC). The projector resolution is $1920\times 1080$ pixels while the camera resolution is $1280\times 1024$ pixels.

 figure: Fig. 6.

Fig. 6. Experimental setup.

Download Full Size | PDF

3.1 Comparative analysis of different frequency combinations

As demonstrated in Sec. 2.3.2, the simulation test shows that we could adopt fringes with periods $T_1 = 18$ pixels and $T_2 = 36$ pixels to enhance the depth range. It is clear that fringe with $T_1$ has better phase quality in the slightly defocused zone while fringe with $T_2$ outperforms it when the filter size is larger than 25 pixels. This demonstrates that the combination of fringes with $T_1$ and $T_2$ could effectively extend the depth range.

To verify this, we did experiments by utilizing a white board to evaluate the phase error in a relatively large depth range. The original depth is set as the projection focus plane. And the white board is placed at the depth $z$ ranging from 14 cm to 56.5 cm with an interval of 2.5 cm. In this research, we focus on the problem of the existence of EDZ, thus these experiments were carried out from the depth where the highest-frequency fringe is properly defocusing to the depth where it is excessively defocusing. The phase error is obtained by comparing with a reference phase calculated by 36-step phase-shifting algorithm for fringe images with $T = 36$ pixels [24]. The experimental results are shown in Figs. 7(a)–7(d). It can be seen that fringe with $T_1 = 18$ pixels shows superior quality with lower phase error under 0.05 rad when $z$ is in the range of $[14, 34]$ cm. Yet its phase error becomes to increase steadily when $z > 34$ cm. In order to extend this limitation, different fringe period combinations could be adopted with the results shown in Figs. 7(a)–7(c). From Fig. 7(a), we can see that the fringe period pair $T = \left (18, 24\right)$ pixels could extend the depth range slightly to $[14, 41.5]$ cm, since the phase error of $T_2 = 24$ pixels shows worse phase quality gradually when $z > 41.5$ cm. Besides, the results of fringe period pair $T = \left (18, 54\right)$ pixels shown in Fig. 7(c) reveal that the phase quality of $T_2 = 54$ pixels is not high in the range of $[34, 40]$ cm. In contrast, Fig. 7(b) shows that fringe period combination of $T = \left (18, 36\right)$ pixels provides the best results since it could significantly improve the depth range to nearly $[14, 50]$ cm. Meanwhile, further depth enhancement could be achieved with more fringe frequencies. As shown in Fig. 7(d), three fringe period combination $T = \left (18, 36, 54\right)$ pixels has the potential to further enhance the depth range. These results are consistent with simulation results in Fig. 4. Simulated and experimental RMS error plots demonstrate basically the same trend, but RMS error values in Fig. 4 are below those in Fig. 7. This is reasonable because the simulation is based on the theoretical model with ideal condition while there are lots of noises caused by the ambient lighting and camera sampling for the real experiments.

 figure: Fig. 7.

Fig. 7. Phase RMS errors in different depth ranges for different fringe period combinations: (a) $T_1$ = 18 pixels and $T_2$ = 24 pixels; (b) $T_1$ = 18 pixels and $T_2$ = 36 pixels; (c) $T_1$ = 18 pixels and $T_2$ = 54 pixels; (d) $T_1$ = 18 pixels, $T_2$ = 36 pixels and $T_3$ = 54 pixels.

Download Full Size | PDF

3.2 Accuracy evaluation on large-depth-range object measurement

To further quantitatively evaluate the proposed method , experiments were performed to measure two flat boxes in large depth range. The upper box is placed at the depth about 22.5 cm from the focus plane while the lower one is placed at the depth about 56.5 cm. We utilize fringes with $T = 18, 36, 120, 1140$ pixels to measure the objects, the first two fringe periods are used to extend the depth range, while the last two are used for phase unwrapping [25]. Besides, a simple phase-to-height approach based on referenced plane is employed to obtain the height distribution [26].

Figures 8(a) and 8(c) show the fringes with $T_1 = 18$ pixels and $T_2 = 36$ pixels respectively, and Fig. 8(e) shows the resultant merged fringe. Figures 8(b), 8(d) and 8(f) show the corresponding 3D result. Figures 8(g) and 8(h) show the real measuring scene and ideal 3D result respectively. To better view the difference, we also plot the corresponding cross sections as shown in Fig. 9. For the upper box, the cross section of absolute phase maps are illustrated in Fig. 9, from which we can see fringe $T_1$ provides more accurate value since its phase value is close to the ideal one. And the phase error plots of different frequencies are illustrated in Fig. 9(b). The phase root-mean-square (RMS) errors for $T_1 = 18$ pixels and $T_2 = 36$ pixels are respectively 0.051 rad and 0.112 rad. Besides, for the lower box, Figs. 9(c)–9(d) shows the comparison results. On the contrary, we can see fringe $T_2$ provides more accurate result. And the phase RMS errors for $T_1 = 18$ pixels and $T_2 = 36$ pixels are respectively 0.205 rad and 0.110 rad.

 figure: Fig. 8.

Fig. 8. 3D reconstruction results for two flat boxes in large depth range: (a) Fringe pattern of $T_1 = 18$ pixels; (b) Corresponding 3D result of (a); (c) Fringe pattern of $T_2 = 36$ pixels; (d) Corresponding 3D result of (c); (e) Merged fringe pattern; (f) Corresponding 3D result based on MFPM method; (g) LDR scene consisting of two boxes; (h) Ideal 3D reconstruction result.

Download Full Size | PDF

 figure: Fig. 9.

Fig. 9. Accuracy evaluation for the MFPM method: (a) Cross sections of absolute phase maps for the upper surface; (b) Corresponding phase error plots when comparing with the ideal phase map; (c)–(d) The cross sections of absolute phase maps and phase error for the lower surface.

Download Full Size | PDF

3.3 Measurement for complex objects in large depth range

Furthermore, experiments were also carried out to measure complex objects in large depth range. The first male sculpture is placed at the depth about 23 cm from the focus plane and the second female one is put at the depth about 52.5 cm. Figures 10(a) and 10(c) show the captured fringe patterns with $T_1 = 18$ pixels and $T_2 = 36$ pixels respectively. The relevant 3D reconstructed results are demonstrated in Figs. 10(b) and 10(d). To better view the differences, the zoom-in plots for each statue have been shown in Figs. 10(g)–10(j). It is clear that for the first statue, fringe $T_1$ shows better quality in Fig. 10(g), and in contrast, fringe $T_2$ provides inferior results with obvious stripes in the region of face as shown in Fig. 10(i). This is because that the proper defocusing effect for fringe $T_1$ exactly eliminate its high-order harmonics, but for fringe $T_2$, it is too slight to remove the harmonics completely. However, as $z$ increases, the surface of second statue for fringe $T_2$ in Fig. 10(j) outperforms the result of fringe $T_1$ in Fig. 10(h).

 figure: Fig. 10.

Fig. 10. 3D reconstruction results for complex sculptures in large depth range. (a) Fringe pattern when $T_1 = 18$ pixels; (b) Corresponding 3D reconstructed result when $T_1 = 18$ pixels; (c) Fringe pattern when $T_2 = 36$ pixels; (d) Corresponding 3D reconstructed result when $T_2 = 36$ pixels; (e) Merged fringe pattern when $T_1 = 18$ pixels and $T_2 = 36$ pixels; (f) Corresponding 3D reconstructed result based on MFPM method; (g) Zoom-in 3D plot for the male statue when $T_1 = 18$ pixels; (h) Zoom-in 3D plot for the female statue when $T_1 = 18$ pixels; (i) Zoom-in 3D plot for the male statue when $T_2 = 36$ pixels; (j) Zoom-in 3D plot for the female statue when $T_2 = 36$ pixels; (k) Zoom-in 3D plot for the male statue based on MFPM method; (l) Zoom-in 3D plot for the female statue based on MFPM method.

Download Full Size | PDF

Finally, large-depth-range 3D reconstruction has been realized based on our proposed MFPM method. Figures 10(e)–10(f) respectively show the merged fringe patterns and corresponding 3D results for two statues. And Figs. 10(k)–10(l) show the zoom-in plots, it is obvious that the surfaces of both statues obtained from MFPM method are better with less errors, and outperforms single-frequency methods. These experimental results verify that the proposed MFPM method has the capability to extend the depth range with multi-frequency binary patterns.

4. Conclusion

This paper has presented a multi-frequency phase merging method to measure the object in large depth range. By carefully choosing the suitable fringe frequencies and merging the related phase maps, the excessive defocusing zone for single-frequency binary patterns could be overcome. Both simulation and experiments confirm the effectiveness of the proposed method. In this paper, we mainly focus on the depth range extension using two fringe frequencies, which is probably enough for a lot of applications. But if larger depth range is required, more frequencies could be incorporated to further improve the capability.

Funding

National Natural Science Foundation of China (61603360, 61261130587, 51605130, 51905005).

References

1. S. S. Gorthi and P. Rastogi, “Fringe projection techniques: Whither we are?” Opt. & Lasers Eng. 48(2), 133–140 (2010). [CrossRef]  

2. X. Su and Q. Zhang, “Dynamic 3-d shape measurement method: A review,” Opt. & Lasers Eng. 48(2), 191–204 (2010). [CrossRef]  

3. S. Zhang, “High-speed 3d shape measurement with structured light methods: A review,” Opt. & Lasers Eng. 106, 119–131 (2018). [CrossRef]  

4. B. Li and S. Zhang, “Microscopic structured light 3d profilometry: Binary defocusing technique vs. sinusoidal fringe projection,” Opt. & Lasers Eng. 96, 117–123 (2017). [CrossRef]  

5. X. Su, W. Zhou, B. G. Von, and D. Vukicevic, “Automated phase-measuring profilometry using defocused projection of a ronchi grating,” Opt. Commun. 94(6), 561–573 (1992). [CrossRef]  

6. S. Lei and S. Zhang, “Flexible 3-d shape measurement using projector defocusing,” Opt. Lett. 34(20), 3080–3082 (2009). [CrossRef]  

7. Y. Gong and S. Zhang, “Ultrafast 3-d shape measurement with an off-the-shelf dlp projector,” Opt. Express 18(19), 19743–19754 (2010). [CrossRef]  

8. H. Fujita, K. Yamatan, M. Yamamoto, Y. Otani, A. Suguro, S. Morokawa, and T. Yoshizawa, “Three-dimensional profilometry using liquid crystal grating,” Proc. SPIE - The Int. Soc. for Opt. Eng. pp. 51–60 (2003).

9. C. Zuo, Q. Chen, S. Feng, F. Feng, G. Gu, and X. Sui, “Optimized pulse width modulation pattern strategy for three-dimensional profilometry with projector defocusing,” Appl. Opt. 51(19), 4477–4490 (2012). [CrossRef]  

10. G. A. Ayubi, J. A. Ayubi, M. J. Di, and J. A. Ferrari, “Pulse-width modulation in defocused three-dimensional fringe projection,” Opt. Lett. 35(21), 3682–3684 (2010). [CrossRef]  

11. Y. Wang and S. Zhang, “Superfast multifrequency phase-shifting technique with optimal pulse width modulation,” Opt. Express 19(6), 5149–5155 (2011). [CrossRef]  

12. T. Xian and X. Su, “Area modulation grating for sinusoidal structure illumination on phase-measuring profilometry,” Appl. Opt. 40(8), 1201–1206 (2001). [CrossRef]  

13. W. Lohry and S. Zhang, “3d shape measurement with 2d area modulated binary patterns,” Opt. & Lasers Eng. 50(7), 917–921 (2012). [CrossRef]  

14. Y. Wang and S. Zhang, “Three-dimensional shape measurement with binary dithered patterns,” Appl. Opt. 51(27), 6631–6636 (2012). [CrossRef]  

15. L. William and S. Zhang, “Genetic method to optimize binary dithering technique for high-quality fringe generation,” Opt. Lett. 38(4), 540 (2013). [CrossRef]  

16. B. Pan, K. Qian, L. Huang, and A. Anand, “Phase error analysis and compensation for nonsinusoidal waveforms in phase-shifting digital fringe projection profilometry,” Opt. Lett. 34(4), 416–428 (2009). [CrossRef]  

17. A. Kamagara, S. Li, and X. Wang, “Optimal defocus selection based on normed fourier transform for digital fringe pattern profilometry,” Appl. Opt. 56(28), 8014–8022 (2017). [CrossRef]  

18. Y. Wang, H. Zhao, H. Jiang, and X. Li, “Defocusing parameter selection strategies based on psf measurement for square-binary defocusing fringe projection profilometry,” Opt. Express 26(16), 20351–20367 (2018). [CrossRef]  

19. D. Zheng, F. Da, K. Qian, and H. S. Seah, “Phase error analysis and compensation for phase shifting profilometry with projector defocusing,” Appl. Opt. 55(21), 5721 (2016). [CrossRef]  

20. A. Silva, J. L. Flores, A. M. noz, G. A. Ayubi, and J. A. Ferrari, “Three-dimensional shape profiling by out-of-focus projection of colored pulse width modulation fringe patterns,” Appl. Opt. 56(18), 5198–5203 (2017). [CrossRef]  

21. D. Malacara and F. Roddier, “Optical shop testing,” Appl. Opt. 97, 454–464 (1978). [CrossRef]  

22. V. P. Namboodiri and S. Chaudhuri, “On defocus, diffusion and depth estimation,” Pattern Recognit. Lett. 28(3), 311–319 (2007). [CrossRef]  

23. M. Nagase, D. Iwai, and K. Sato, “Dynamic defocus and occlusion compensation of projected imagery by model-based optimal projector selection in multi-projection environment,” Virtual Real. 15(2-3), 119–132 (2011). [CrossRef]  

24. L. Ekstrand and S. Zhang, “Three-dimensional profilometry with nearly focused binary phase-shifting algorithms,” Opt. Lett. 36(23), 4518 (2011). [CrossRef]  

25. C. Zuo, H. Li, M. Zhang, Q. Chen, S. Feng, T. Tao, and Y. Hu, “Robust and efficient multi-frequency temporal phase unwrapping: optimal fringe frequency and pattern sequence selection,” Opt. Express 25(17), 20381–20400 (2017). [CrossRef]  

26. Y. Xiao, Y. Cao, and Y. Wu, “Improved algorithm for phase-to-height mapping in phase measuring profilometry,” Appl. Opt. 51(8), 1149 (2012). [CrossRef]  

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (10)

Fig. 1.
Fig. 1. (a) Projector’s defocusing effect corresponding to depth $z$; (b) Defocus kernel $\sigma$ curve as a function of depth $z$: $z_1\,<\,z_2\,<\,z_3\,<\,z_f\,<\,z_4\,<\,z_5\,<\,z_6$.
Fig. 2.
Fig. 2. Multi-frequency phase merging: $\sigma _1, \sigma _2$ are the defocusing kernels at depth $z_1$ and $z_2$, and $\omega _1, \omega _2$ are the fundamental frequencies for fringe periods $T_1, T_2$ with $\omega = \frac {2\pi }{T}$.
Fig. 3.
Fig. 3. Steps for optimal frequency determination.
Fig. 4.
Fig. 4. Simulation results for optimal frequency determination: (a) When $T_1$ = 18 pixels and $T_2$ = 24 pixels; (b) When $T_1$ = 18 pixels and $T_2$ = 36 pixels; (c) When $T_1$ = 18 pixels and $T_2$ = 54 pixels; (d) When $T_1$ = 18 pixels, $T_2$ = 36 pixels and $T_3$ = 54 pixels.
Fig. 5.
Fig. 5. Defocusing degree maps for phase merging: (a) LDR scene consisting of two portrait sculptures of which the male one is placed at the distance of about 23 cm from the focus plane and the female one is placed at the distance of about 52.5 cm from the focus plane; (b) Fringe pattern when $T_1$ = 18 pixels; (c) Corresponding defocusing degree map when $T_1$ = 18 pixels; (d) Fringe pattern when $T_2$ = 36 pixels; (e) Corresponding defocusing degree map when $T_2$ = 36 pixels; (f) Merged fringe pattern calculated by aforementioned defocusing degree maps when $T_1$ = 18 pixels and $T_2$ = 36 pixels.
Fig. 6.
Fig. 6. Experimental setup.
Fig. 7.
Fig. 7. Phase RMS errors in different depth ranges for different fringe period combinations: (a) $T_1$ = 18 pixels and $T_2$ = 24 pixels; (b) $T_1$ = 18 pixels and $T_2$ = 36 pixels; (c) $T_1$ = 18 pixels and $T_2$ = 54 pixels; (d) $T_1$ = 18 pixels, $T_2$ = 36 pixels and $T_3$ = 54 pixels.
Fig. 8.
Fig. 8. 3D reconstruction results for two flat boxes in large depth range: (a) Fringe pattern of $T_1 = 18$ pixels; (b) Corresponding 3D result of (a); (c) Fringe pattern of $T_2 = 36$ pixels; (d) Corresponding 3D result of (c); (e) Merged fringe pattern; (f) Corresponding 3D result based on MFPM method; (g) LDR scene consisting of two boxes; (h) Ideal 3D reconstruction result.
Fig. 9.
Fig. 9. Accuracy evaluation for the MFPM method: (a) Cross sections of absolute phase maps for the upper surface; (b) Corresponding phase error plots when comparing with the ideal phase map; (c)–(d) The cross sections of absolute phase maps and phase error for the lower surface.
Fig. 10.
Fig. 10. 3D reconstruction results for complex sculptures in large depth range. (a) Fringe pattern when $T_1 = 18$ pixels; (b) Corresponding 3D reconstructed result when $T_1 = 18$ pixels; (c) Fringe pattern when $T_2 = 36$ pixels; (d) Corresponding 3D reconstructed result when $T_2 = 36$ pixels; (e) Merged fringe pattern when $T_1 = 18$ pixels and $T_2 = 36$ pixels; (f) Corresponding 3D reconstructed result based on MFPM method; (g) Zoom-in 3D plot for the male statue when $T_1 = 18$ pixels; (h) Zoom-in 3D plot for the female statue when $T_1 = 18$ pixels; (i) Zoom-in 3D plot for the male statue when $T_2 = 36$ pixels; (j) Zoom-in 3D plot for the female statue when $T_2 = 36$ pixels; (k) Zoom-in 3D plot for the male statue based on MFPM method; (l) Zoom-in 3D plot for the female statue based on MFPM method.

Equations (10)

Equations on this page are rendered with MathJax. Learn more.

In(x,y)=I(x,y)+I(x,y)cos(ϕ(x,y)+2nπN),n=0,1,2,,N1.
ϕ(x,y)=tan1[n=0N1In(x,y)sin2nπNn=0N1In(x,y)cos2nπN].
γ(x,y)=2(n=0N1In(x,y)sin2nπN)2+(n=0N1In(x,y)cos2nπN)2n=0N1In(x,y).
Bi(xp,yp)=I(xp,yp)+I(xp,yp)k=012k+1cos[(2k+1)(ϕ(xp,yp)+2πiN)],
G(xp,yp)=12πσ2exp2+yp22σ2G(ω)=eσ2ω22,
σ=kR(z)=k[|D2(zzf1)|+rzv].
γnor(x,y)=γ(x,y)γmax(x,y)=γ(x,y)×I(x,y)Iw(x,y)I(x,y).
{Mt(x,y)=0,ifγnort(x,y)<thresh;Mt(x,y)=1,otherwise.
Φnort(x,y)=tmin(T)Φt(x,y).
ΦLDR(x,y)=Φnortopt(x,y).
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.