Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Smartphone-based hybrid 3D profilometry with an extended imaging depth range based on binary code and phase-shifting

Open Access Open Access

Abstract

For the emerging demands of three-dimensional (3D) profile measurement in daily life, a smartphone-based 3D profilometry based on structured light was presented in this paper. To measure 3D profiles within a large depth range, a method combing binary code and phase-shifting was employed. Applying anti-phase complementary binary code patterns successfully overcame the defocusing issue of both the camera and projector. It also helped reduce the influence of objects’ surface reflectivity and the phase unwrapping error. For a depth range of 1100 mm, an average height reconstruction error lower than 2 mm can be achieved with this system.

© 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

1. Introduction

Optical three-dimensional (3D) profilometry nowadays plays an indispensable role in numerous application fields such as manufacturing, product inspection, and medicine. The demands of 3D profiling also emerge for daily life applications such as face recognition and volume estimation [1,2]. Currently, there are various ways to achieve noncontact and real-time 3D profiling with high accuracy. Structured light is one of the most commonly used methods [3]. A structured light system consists of a camera and a projector. The camera axis is shifted away from the projection axis by an angle. Because of this shift, the pattern projected onto an object is deformed by the surface shape in the image captured by the camera. Knowing the deformation amount, the height of the object can be reconstructed through triangulation. In contrast to time-of-flight techniques which are popular for long-distance and large range measurements [4,5], structured light methods are more suitable for a short distance and small range 3D profiling [6,7]. The structured light methods also provide high accuracy and a much simpler image reconstruction process, compared to stereo vision techniques which have limited measurement accuracy and require intensive signal processing [8,9]. Moiré profilometry has a similar system configuration to structured light, including a camera and a projector. But an additional physical grid is required for demodulation, which complicates the system configuration [10].

To acquire the pattern deformation on the object, numerous structured light methods have been developed, including single-shot and multi-shot categories [3,11]. Each method has its strengths and weakness. Many of them have been reported to achieve real-time imaging [9,12]. To uniquely label the object surface through a single-shot, the pattern should be encoded via a special coding strategy, including color encoded and grayscale indexing methods. The color encoding methods, such as color-coded stripes [13,14] and De Bruijn sequence [15], employ color as a tag for identification. The resolution of these methods is highly sensitive to the color resolution and dynamic range of both the projector and camera chip. Also, the reflection spectrum of the object surface can seriously influence the quality of the height reconstruction results. The grayscale indexing methods, such as pseudo-random 2D array [16] and grayscale coded stripes [17], are color-independent and suitable for objects with a non-uniform reflection spectrum. However, these methods recover the pattern deformation within sub-domains of the patterns so that they are highly sensitive to the surface discontinuities.

To overcome the weakness of the single-shot methods, multi-shot methods can be used instead but the imaging speed may be sacrificed. Similar to the color encoded methods, binary coded [18], Gray coded [19] and grayscale coded [3] methods are robust approaches to give a unique position code to each point on the object surface. The binary and Gray coded methods require at least N binary patterns to generate 2N position codes along one direction, while the grayscale coded method can generate MN position codes for one image dimension by projecting N grayscale patterns with M grayscale levels. In comparison, the grayscale coded method can help reduce the number of patterns per reconstructed image but it has a relatively lower tolerance for surface reflectivity variation in contrast to the binary and Gray coded methods. For these methods, the lateral resolution of the height reconstruction relies on the size of the smallest discretized regions. Increasing the number of patterns can achieve higher resolution but also reduce the imaging speed. To make a trade-off between the resolution and imaging speed, the phase-shifting method is often used along with these methods [2022].

The phase-shifting method requires N sinusoidal patterns with equally separated phases. 3-step or 4-step phase-shifting is most commonly used [23,24]. Applying the phase-shifting method, the lateral resolution can be pixel-wise. However, the most significant issue is coming from the phase unwrapping errors. The phases extracted from the N sinusoidal patterns are originally wrapped within the (π, -π] region. The phases in the adjacent regions are expected to have a phase difference of 2π, but it is usually not the case when the object has drastic height changes or discontinuous surfaces [25]. Previous researches have introduced various ways to post-correct the errors [20] or pre-avoid the errors [26,27]. Many types of composite phase-shifting schemes have also been proposed to achieve real-time 3D profiling [2830] but the capability of imaging depth range has not been demonstrated in these papers. The position codes provided by the binary or gray coded methods can also help to determine the phase jump and remove the unwrapping errors [21]. The boundary determination of the discretized regions thus plays an important role. Influenced by the non-uniform surface reflectivity or defocusing, the boundaries are usually not sharp. Many ways of adding complementary patterns, such as the unequal-period pattern [21,31] and anti-phase pattern [11], were developed to increase the accuracy of boundary determination.

Due to the issues of pattern defocusing and system calibration, the structured light methods are limited to short distance and small range 3D profiling. By employing large-scale calibration targets, the range of 3D measurement has been extended to 1200(H) ×800 (V) × 1000(D) mm3 with submillimeter accuracy [6]. In this research, we developed a smartphone-based hybrid structured light system based on both binary code and phase-shifting methods, combining a smartphone camera and a microprojector. This system aims for daily-life usage and aims to extend the depth range as well. By adding a complementary anti-phase pattern, the boundaries of the binary patterns can be accurately determined even with non-uniform surface reflectivity and pattern defocusing. The demonstration of this system on standard objects showed the 3D profiling can be achieved with an average height error lower than 2 mm within a depth range of 1100 mm. For a 3D scene reconstruction, the depth range even extended to 1380 mm. This method requires a reference plane only for the calibration process as the camera and projector were assembled. With no need for a reference plane during the imaging process, this method is suitable for daily life applications.

2. Methods

2.1 System configuration and height reconstruction

The configuration and picture of the hybrid structured light system are shown in Fig. 1(a) and 1(b). To make the system more applicable in daily life, a microprojector (Vivitek Qumi Q6) was used to project structured light and the camera of a smartphone (HTC One M9+) was employed to capture the images. The camera and the projector were separated along the y-direction but colocalized at the same x position and these two devices were combined through a homemade framework (Fig. 1(b)). Orienting the strucutred light pattern along the y-direction, a shift of the pattern can be observed on the object surface and the shift amount can be used to reconstruct the height of the object. According to the system geometry in Fig. 1(c), the displacement between the camera and the projector is d and the reference plane is L apart from the projector. The light pattern is projected onto point B at the reference plane when there is no object; while the pattern is projected onto point A at the object surface when the object exists. From the perspective of the camera, the pattern is shifted from B to C and the shift amount is S. The relationship between the shift amount (S) and the height of point A (h) can help to reconstruct the object’s 3D profile. First, the shift amount can be described as

$$S = h\tan \theta - h\tan ({\theta - \phi } ), $$
where
$$\theta - \phi = {\tan ^{ - 1}}\left( {\frac{{({L - h} )\tan \theta - d}}{{L - h - t}}} \right). $$

 figure: Fig. 1.

Fig. 1. The (a) system configuration, (b) system setup, and (c) triangulation geometry of the structured light system.

Download Full Size | PDF

Substituting Eq. (2) into Eq. (1) yields

$$S = h\tan \theta - h\left[ {\frac{{({L - h} )\tan \theta - d}}{{L - h - t}}} \right] = \frac{{ - th\tan \theta + hd}}{{L - h - t}}.$$

Then, the height can be described as

$$h = \frac{{S({L - t} )}}{{d - t\tan \theta + S}}.$$

As long as the S throughout the object is obtained, the 3D profile of the object can be reconstructed through Eq. (4).

2.2 Structured light patterns

This hybrid system employed both the binary code and phase-shifting methods. The binary code patterns were used to provide a code map throughout the imaging region. Figure 2(a) (red rectangle) displays eight patterns for the binary code. The first seven of them stood for the position numbers of 26 to 2° in the binary numbering system. Each discretized region in the 2° pattern thus had a unique 7-bit binary number and a corresponding decimal position code. Through these position codes, the shift amount, S, can be correctly determined even though the corresponding phase shift is more than 2π. Besides, the position codes also have benefits to recognizing a phase jump more than 2π in the wrapped phase, which usually occurs on a drastically depth-varying surface. Thus, the phase unwrapping errors can be greatly reduced. With the already known projection geometry of the projector, the reference plane can be removed during measurements.

 figure: Fig. 2.

Fig. 2. (a) The light patterns for the binary code method (red square); (b) intensity curves of 2° patterns with the phases of 0 and π; (c) the reconstructed code map without (black) and with (red) the boundary information; (d) The light patterns used for the phase-shifting method (blue square).

Download Full Size | PDF

Although the conversion boundaries between black and white appear to be distinct (Fig. 2(a)) they were transient regions (Fig. 2(b)). The width of these transient regions was determined by the resolution limit of the projector and camera, pattern defocusing on the object surface, and the reflectivity variation of the object. To locate all the boundaries in the seven patterns, only the 2° pattern had to be considered since this pattern contains all the boundaries in other patterns. To avoid the influence of the transient regions, a complementary anti-phase 2° pattern was used in this research. Since both the 2° and anti-phase 2° patterns experienced the same defocusing or reflectivity variation issues, they had the same intensity modulations (Fig. 2(b)). By subtracting one pattern from the other, the resulted zero points should be the boundaries we were looking for. Without using the complementary pattern, the reconstructed code map showed errors near the boundaries (black curve in Fig. 2(c)). These errors were successfully corrected by applying the anti-phase pattern (red curve in Fig. 2(c)). Moreover, finding boundaries in this way helped to overcome the defocusing issue, which also had benefits to measuring the object within a wide depth range.

To gain higher lateral resolution, the phase-shifting method was applied as well. Figure 2(d) shows the four 2° patterns with phases of 0, π/2, π, and 3π/2. Since two of them belonged to the binary code patterns, only two additional patterns were required. After filtering out the higher frequency components in the frequency domain, four binary patterns can be transformed into sinusoidal patterns,

$$\begin{aligned} &{I_1}({x,y} )= A({x,y} )+ B({x,y} )\cos [{\theta ({x,y} )} ]\\ &{I_2}({x,y} )= A({x,y} )+ B({x,y} )\cos [{\theta ({x,y} )+ {\pi / 2}} ]\\ &{I_3}({x,y} )= A({x,y} )+ B({x,y} )\cos [{\theta ({x,y} )+ \pi } ]\\ &{I_4}({x,y} )= A({x,y} )+ B({x,y} )\cos [{\theta ({x,y} )+ {{3\pi } / 2}} ]\end{aligned}, $$
where A(x,y) is the average intensity, B(x,y) is the intensity modulation, and θ (x,y) is the phase distribution we are seeking. Using the following equation,
$$\theta ({x,y} )= {\tan ^{ - 1}}\frac{{{I_4}({x,y} )- {I_2}({x,y} )}}{{{I_1}({x,y} )- {I_3}({x,y} )}}, $$
θ (x,y) is the reconstructed phases wrapped within (-π, π]. Since the 2π phase jump occurs only at the boundaries of the 2° (0) pattern, θ (x,y) can be properly unwrapped with the aid of the boundary information. In a summary, 10 patterns in total were used to get a reconstructed 3D profile. Because the highest frame rate of the smartphone’ camera is 30 fps, the imaging rate of reconstructed images is 3 fps. However, the lowest frequency pattern is responsible for the highest significant bit and is expected to cover the whole imaging region with only one black and one white region. If the object occupies a smaller imaging area, the 25 or even 24 pattern can be used as the lowest frequency pattern instead. Then, the number of the patterns used for one reconstruction can be reduced to 9 or 8.

2.3 System calibration

To increase the height reconstruction accuracy, the parameters L, d, and t in Eq. (4) should be carefully calibrated. Since the Qp and Qc points (Fig. 1(b)) are usually inside the projector and camera, they cannot be located by simply measuring the distance between them and the reference plane. Figure 3(a) shows the method for the Qp point calibration. A number of horizontal lines with single-pixel width were projected onto the reference plane. With the same focusing condition, a screen was inserted at positions, z1, z2, and z3, which were 200 mm, 300 mm, and 400 mm away from the reference plane, respectively. At these three positions, the heights of each projected line with respect to the optical table, which was at y = 0, were recorded as y1, y2 and y3 (circles in Fig. 3(b)), and one linear function can then be obtained for each projected line through curve fitting (dashed lines in Fig. 3(b)). Through more than 2 linear functions, the y and z positions of the intersection point, which was also the Qp point, can be found (Fig. 3(b)). As a result, L was calibrated to have an average value of 1218 mm.

 figure: Fig. 3.

Fig. 3. (a) The calibration methods for the projector’s Qp point; (b) the heights of the lines measured at 200 mm, 300 mm and 400 mm away from the reference plane (circles) and the fitted linear functions (dashed lines); (c) the calibration methods for the camera’s Qc point; (d) the intensity profiles of the lines, measured along y direction at 200 mm, 300 mm and 400 mm away from the reference plane.

Download Full Size | PDF

To calibrate the Qc point, a target with a number of periodic horizontal lines was inserted at positions, z1, z2 and z3, which were 200 mm, 300 mm, and 400 mm apart from the reference plane (Fig. 3(c)). The line at the very bottom was aligned with the height of Qp, obtained from the Qp calibration process. At each position, the target was recorded by the camera. The intensity profiles of the lines along the y-direction are shown in Fig. 3(d). By choosing the frequency of the lines high enough, there should be a bunch of lines on the three captured images overlapping one another (dashed lines in Fig. 3(d)). By considering the imaging system as a pinhole camera model, these overlapping lines should locate at the same viewing angle of the camera. When the heights of these overlapping lines were recorded as p1, p2, and p3 at different positions, a linear function can be obtained through curve fitting. By getting more than two linear functions, the Qc point can thus be found as the intersection point of these linear functions. The y and z positions of the Qc point were thus obtained. L-t was calibrated to have an average value of 1200 mm. t was 18 mm according to the value of L obtained above. In addition, the distance between the projector axis and the camera axis, d, can be calculated from the y positions of the Qc and Qp points. d was calibrated to have an average of 194.4 mm. The values of L, t and d were applied to the reconstruction process for the results in the following sections.

3. Results and discussions

To demonstrate the 3D profiling, a sphere with a radius of 49.8 ± 0.37 mm was used as the object. The radius was measured by using a Vernier caliper. As shown in Fig. 4(a), 10 raw images with different structured light patterns were acquired. Figure 4(b) shows the y-direction intensity profiles taken from the 2° (0) and 2° (π) raw images, along the blue and red lines in Fig. 4(a). The average intensity of the light patterns was influenced by the reflectivity variation which was caused by the surface curvature. To eliminate this influence, four 2° patterns were added together and then divided by 4 to get A(x,y) in Eq. (5), which gave the average intensity in Fig. 4(b). By subtracting A(x,y) from the raw images, only the modulation B(x,y) was left in the processed images (Fig. 4(c)). By subtracting one processed image from the other, the resulted zero points stood for all the boundaries in the 2° image.

 figure: Fig. 4.

Fig. 4. (a) The raw images of the reference plane and the object with 10 light patterns; (b) the intensity profiles along the blue and red lines in (a); (c) the normalized intensity profiles with the modulation removed; (d) the reconstructed code map of the reference plane and the object; (e) the position codes along the blue and red lines in (d); (f) the wrapped phases obtained through four 2° raw images, along the blue (or red) line in (a); (g) the unwrapped phases along the blue and red lines in (d).

Download Full Size | PDF

While knowing all the boundary information in 26 to 2° (0) raw images, the code map of the object surface was built (Fig. 4(d)). The code map of the reference plane was built from the raw images of only the reference plane. Despite 7 patterns were used in this case, according to the size of the sphere, only 5 patterns (excluding 26 and 25 patterns) were enough to build the code map. Figure 4(e) shows the position codes taken along the blue and red lines in Fig. 4(d) but only within the object region (region A). Since the binary patterns were shifted by the object, the position codes of the object were lower than that of the reference plane.

The position codes in Fig. 4(e) had a discrete variation which limited the lateral resolution and the boundaries between two adjacent position codes were expected to have an error region of several pixels. To elevate the resolution, four processed 2° images were substituted into Eq. (6) to get θ (x,y). Figure 4(f) shows the wrapped phases along the blue (or red) line in Fig. 4(a). During the unwrapping process, the errors usually occurred at the phase jump, i.e., the boundary. Although the boundary information may not be that precise, it can also help to find out the regions where the phase errors might occur. Within these hazard regions, curve fitting of the phase variation was further used to correct the phase errors, since the phase was supposed to change continuously. The unwrapped phases of the reference plane and the object along the blue and red lines (region B) in Fig. 4(d) were shown in Fig. 4(g). The shift amount of the same phase between the reference plane and the object was then used to reconstruct the height (h) through Eq. (4).

Based on the image acquisition and reconstruction procedures, three objects: a planar screen (Fig. 5(a)), a small ball (Fig. 5(b)) and a large ball (Fig. 5(c)) with radii of 19.83 ± 0.37 mm and 49.84 ± 0.35 mm were measured. The planar screen was put at distances of 200 mm, 300 mm, and 400 mm, where the distance was defined as the distance between the planar screen and the reference plane. The corresponding reconstructed height throughout the whole screen was 200.39 ± 0.38 mm, 300.51 ± 0.28 mm, and 400.82 ± 0.42 mm. The height errors were shown in Fig. 5(d). The larger the distance, the higher was the errors. Since the projector was focused on the reference plane, the increase of the error with increasing distance might result from the pattern defocusing. On the other hand, the reconstructed 3D profiles of the two balls placed at 400 mm were shown in Fig. 5(e) and 5(f). The distance used here was defined as the distance between the reference plane and the farthest point on the ball away from the reference plane. As shown in Fig. 5(e) and 5(f), these reconstructed profiles (deep blue lines) were fitted by sphere functions with radii of 19.83 mm and 49.84 mm (dashed blue lines), and the error curves between them (orange lines) are shown as well. Both the profiles and the fitted curves were across the center of the spheres. The absolute errors for the small ball were all lower than 0.25 mm but slightly higher within the y positions from 205 mm to 215 mm. The absolute errors for the large ball were mostly lower than 0.5 mm but higher than 0.5 mm within the y positions from 195 mm to 215 mm. In both cases, the errors within the lower parts of the balls were relatively higher. Since the patterns generally had higher frequency within these regions (Fig. 4(a)), the increasing frequency could lead to higher reconstruction phase errors. As a result, the accuracy of the height reconstruction reduced as well.

 figure: Fig. 5.

Fig. 5. The height reconstruction results of (a) the planar screen, (b) the small ball and (c) the large ball; (d) the average height errors throughout the whole object, measured at distances of 200 mm, 300 mm, and 400 mm; (e)(f) the reconstructed profiles (deep blue lines), the fitted curves (dashed blue lines) and the errors between the reconstructed profiles and the fitted curves (orange lines) of (e) the small ball and (f) the large ball.

Download Full Size | PDF

The average height errors throughout the ball surface, as shown in Fig. 5(d), were generally lower than 0.6 mm and slightly increased with decreasing distance. Since the height errors of the balls were measured between the reconstructed data and the fitted sphere function, the errors resulting from the physical measurement of the object distance could thus be reduced. That is possibly why the average errors of the balls were generally lower than those of the planar screen. Comparing the errors at different distances, the planar screen and balls showed an opposite trend of the error variation. In the case of balls, the decreasing of errors with increasing distance might result from the effective lower surface curvature when the balls were closer to the projector and the camera.

To further extend the depth range, the planar screen was placed at different distances from 600 mm to −500 mm. The results are shown in Fig. 6(a). Here, the negative distance indicates that the object was placed behind the reference plane. These measurements were carried out in the absence of the reference plane. The errors were much lower within the range from 400 mm to −200 mm and then grew with increasing distance in both positive and negative directions. Within an imaging range of 1100 mm, the average height errors were generally lower than 2 mm. As mentioned above, the increasing feature of the errors might result from: (1) the pattern defocusing at larger distances, since the projector was focused at the reference plane; (2) the errors of the physical distance measurement, which would become more serious as the distance became larger. In contrast to the planar screen, the average height errors of the large ball within the range from 600 mm to 200 mm are shown in Fig. 6(b). The errors were generally lower than 0.65 mm, which was better than the case of the planar screen. Besides, the errors were decreasing when the sphere moved from 200 mm to 400 mm but was then gradually increasing when the sphere moved even further. Even though the surface curvature was getting effectively lower with increasing distance, the more serious defocusing issue in the meantime would compensate for this advantage and result in higher errors.

 figure: Fig. 6.

Fig. 6. The average height errors of (a) the planar screen measured at distances from 600 mm to −500 mm and (b) the large sphere measured at distances from 600 mm to 200 mm; (c) the scene for 3D profile reconstruction; (d)(e) the reconstructed 3D profiles. Red arrows: regions with the minimum height; green arrow: regions with the maximum height; yellow arrowheads: regions of low reflectivity; blue arrowheads: regions of high reflectivity.

Download Full Size | PDF

Figure 6(c) showed a scene for 3D profile reconstruction. There were three small balls placed at distances of 400 mm, 200 mm, and −200 mm. Without adding any mask to filter out the background, the 3D profiles of both the spheres and the background were reconstructed, as shown in Fig. 6(d) and 6(e). To show the 3D profile properly, the zero-height plane was shifted by −1070 mm and the distances of the three balls became 1470 mm, 1270 mm, and 870 mm. By fitting the three balls with sphere functions, the average errors were 0.31 ± 0.56 mm, 0.32 ± 0.40 mm, and 0.50 ± 0.57 mm, respectively. Throughout the whole scene, the minimum reconstructed height (red arrows in Fig. 6(c) and 6(d)) was 90.47 mm, while the maximum height (green arrows) was 1470.67 mm. The system showed the capability to reconstruct a 3D scene within a depth range of around 1380 mm. However, the reconstructed result also revealed two issues of this system: 1) within the regions of low reflectivity (yellow arrowheads), the contrast of the patterns was not high enough to perform the reconstruction properly; 2) within the regions of high reflectivity (blue arrowheads), the intensity of the patterns saturated in the raw images. This also led to insufficient contrast for reconstruction. Both issues could be solved if a camera with a higher dynamic range has been employed.

To demonstrate the accuracy of this system to measure a complex surface. A face mask (Fig. 7(a)) was used as the object. The physical heights of several local maxima or local minimum points, including the forehead, nasal tip, cheek, chin, and eyes, were measured through a Vernier caliper. The maximum height of this mask was 69.4 mm at the nasal tip. Figure 7(b) shows the raw image of 2° pattern. Figures 7(c) and 7(d) show the reconstructed profiles. With the aid of the binary patterns, the much faster phase variation at the lower edge of the nose can be correctly reconstructed. Compared the reconstructed data with the physically measured heights, the average height error was 0.30 ± 0.21 mm.

 figure: Fig. 7.

Fig. 7. (a) The face mask used for 3D profile measurement; (b) the raw image of 2° pattern; (c)(d) the reconstructed 3D profile.

Download Full Size | PDF

4. Conclusion

To make the 3D profiling more applicable for daily life, the hybrid structured light system combing the smartphone camera and the microprojector was introduced in this research. The system employed both binary code and phase-shifting methods. Ten light patterns in total were used to reconstruct a 3D profile. For the binary code method, 8 patterns all together gave the position code map throughout the whole imaging region. The complementary anti-phase 2° pattern played the key role to locate all the boundaries in the binary patterns accurately, even though the boundaries were usually blurred due to pattern defocusing or surface reflectivity variations. To gain higher lateral resolution, two additional 2° patterns (π/2, 3π/2) were used along with the two 2° patterns (0, π) in the binary code patterns to execute the phase-shifting method. Based on the boundaries already found, the phase unwrapping process can be carried out without misjudgments of phase jump more than 2π. The ability to locate the boundaries and do the phase unwrapping accurately provided the advantages of this system to accomplish the 3D profiling within a wide depth range of imaging. Since the absence of the reference plane was allowed, the imaging range can be extended to positive and negative regions. For the planar screen, the average height errors lower than 2 mm was obtained within the imaging depth range of 1100 mm. Especially within the range from 400 mm to −200 mm, the average height errors were below 1 mm. In contrast, the average height errors were generally lower than 0.65 mm for the large ball and the minimum error occurred at 400 mm. At this distance, the system seemed to make a trade-off between the effective surface curvature and the pattern defocusing. With the capability to deal with the defocusing and phase unwrapping errors, this system was shown to be able to reconstruct a 3D profile including the objects and the background. The total depth range had been successfully extended to 1380 mm. The frame rate of the reconstruction images was 3 fps, which was limited by the required number of raw images and the speed of the smartphone camera. With the smartphone-based configuration, accuracy of measurement, extended imaging depth range and 3-fps imaging speed, this system has been demonstrated to have the capability for daily life applications.

Acknowledgments

This research was supported by the Ministry of Science and Technology MOST 108-2221-E-008-091-MY2.

Disclosures

The authors declare that there are no conflicts of interest related to this article.

Data availability

No data were generated or analyzed in the presented research.

References

1. S. Makhsous, H. M. Mohammad, J. M. Schenk, A. V. Mamishev, and A. R. Kristal, “A novel mobile structured light system in food 3D reconstruction and volume estimation,” ensors 19(3), 564 (2019). [CrossRef]  

2. M. Piccirilli, G. Doretto, A. Ross, and D. Adjeroh, “A mobile structured light system for 3D face acquisition,” IEEE Sensors J. 16(7), 1854–1855 (2016). [CrossRef]  

3. S. Van der Jeught and J. J. J. Dirckx, “Real-time structured light profilometry: a review,” Opt. Lasers Eng. 87, 18–31 (2016). [CrossRef]  

4. Y. Li and J. Ibanez-Guzman, “Lidar for autonomous driving: the principles, challenges, and trends for automotive lidar and perception systems,” IEEE Signal Process. Mag. 37(4), 50–61 (2020). [CrossRef]  

5. T. Ringbeck and B. Hagebeuker, “A 3D time of flight camera for object detection,” in Optical 3-D Measurement Techniques (2007), pp. 1–10.

6. Y. An, T. Bell, B. Li, J. Xu, and S. Zhang, “Method for large-range structured light system calibration,” Appl. Opt. 55(33), 9563–9572 (2016). [CrossRef]  

7. A. G. Marrugo, R. Vargas, L. A. Romero, and S. Zhang, “Method for large-scale structured-light system calibration,” Opt. Express 29(11), 17316–17329 (2021). [CrossRef]  

8. M. Z. Brown, D. Burschka, and G. D. Hager, “Advances in computational stereo,” IEEE Trans. Pattern Anal. Mach. Intell. 25(8), 993–1008 (2003). [CrossRef]  

9. S. Rusinkiewicz, O. Hall-Holt, and M. Levoy, “Real-time 3D model acquisition,” ACM Trans. Graph 21(3), 438–446 (2002). [CrossRef]  

10. F. Porto, J. Gurgel, T. Russomano, and P. D. T. V. Farinatti, “Moiré topography: characteristics and clinical application,” Gait & Posture 32(3), 422–424 (2010). [CrossRef]  

11. J. Salvi, J. Pagès, and J. Batlle, “Pattern codification strategies in structured light systems,” Pattern Recognit. 37(4), 827–849 (2004). [CrossRef]  

12. O. Hall-Holt and S. Rusnkiewicz, “Stripe boundary codes for real-time structured-light range scanning of moving objects,” in Eighth IEEE Int. Conf. Comput. Vision ICCV 2001, (2001).

13. W. Liu, Z. Wang, G. Mu, and Z. Fang, “Color-coded projection grating method for shape measurement with a single exposure,” Appl. Opt. 39(20), 3504–3508 (2000). [CrossRef]  

14. B. Li, Y. Fu, Z. Wang, and J. Zhang, “High-speed, high-accuracy 3D shape measurement based on binary color fringe defocused projection,” J. Eur. Opt. Soc. -Rapid 10, 15038 (2015). [CrossRef]  

15. L. Z. L. Zhang, B. Curless, and S. M. Seitz, “Rapid shape acquisition using color structured light and multi-pass dynamic programming,” in Proceedings First International Symposium on 3D Data Process Visual Transm, (2002).

16. P. Payeur and D. Desjardins, “Structured light stereoscopic imaging with dynamic pseudo-random patterns,” LNCSLect. Notes Comput. Sci. 5627, 687–696 (2009). [CrossRef]  

17. N. G. Durdle, J. Thayyoor, and V. J. Raso, “An improved structured light technique for surface reconstruction of the human trunk,” in IEEE Can. Conf. Electr. Comput. Eng., 1998), 2.

18. D. Moreno and G. Taubin, “Simple, accurate, and robust projector-camera calibration,” in Second Int. Conf. 3D Imaging, Model., Process., Visualization and Transm., (2012), pp. 464–471.

19. G. Sansoni, S. Corini, and S. Lazzari, “Three-dimensional imaging based on Graycode light projection: characterization of the measuring algorithm and development of a measuring system for industrial applications,” Appl. Opt. 36(19), 4463–4472 (1997). [CrossRef]  

20. D. Zheng, F. Da, Q. Kemao, and H. S. Seah, “Phase-shifting profilometry combined with Gray-code patterns projection: unwrapping error removal by an adaptive median filter,” Opt. Express 25(5), 4700–4713 (2017). [CrossRef]  

21. J. S. Hyun and S. Zhang, “Enhanced two-frequency phase-shifting method,” Appl. Opt. 55(16), 4395–4401 (2016). [CrossRef]  

22. Q. Zhang, X. Su, L. Xiang, and X. Sun, “3-D shape measurement based on complementary Gray-code light,” Opt. Lasers Eng. 50(4), 574–579 (2012). [CrossRef]  

23. P. S. Huang and S. Zhang, “Fast three-step phase-shifting algorithm,” Appl. Opt. 45(21), 5086–5091 (2006). [CrossRef]  

24. C. Zuo, S. Feng, L. Huang, T. Tao, W. Yin, and Q. Chen, “Phase shifting algorithms for fringe projection profilometry: a review,” Opt. Lasers Eng. 109, 23–59 (2018). [CrossRef]  

25. H. O. Saldner and J. M. Huntley, “Temporal phase unwrapping: application to surface profiling of discontinuous objects,” Appl. Opt. 36(13), 2770–2775 (1997). [CrossRef]  

26. Z. Wu, W. Guo, Y. Li, Y. Liu, and Q. Zhang, “High-speed and high-efficiency three-dimensional shape measurement based on Gray-coded light,” Photonics Res. 8(6), 819–829 (2020). [CrossRef]  

27. Z. Wu, W. Guo, L. Lu, and Q. Zhang, “Generalized phase unwrapping method that avoids jump errors for fringe projection profilometry,” Opt. Express 29(17), 27181–27192 (2021). [CrossRef]  

28. C. Zuo, Q. Chen, G. Gu, S. Feng, and F. Feng, “High-speed three-dimensional profilometry for multiple objects with complex shapes,” Opt. Express 20(17), 19493–19510 (2012). [CrossRef]  

29. K. Liu, Y. Wang, D. L. Lau, Q. Hao, and L. G. Hassebrook, “Dual-frequency pattern scheme for high-speed 3D shape measurement,” Opt. Express 18(5), 5229–5244 (2010). [CrossRef]  

30. C. Zuo, Q. Chen, G. Gu, S. Feng, F. Feng, R. Li, and G. Shen, “High-speed three-dimensional shape measurement for dynamic scenes using bi-frequency tripolar pulse-width-modulation fringe projection,” Optics and Lasers in Engineering 51(8), 953–960 (2013). [CrossRef]  

31. S. Yu, J. Zhang, X. Yu, X. Sun, and H. Wu, “Unequal-period combination approach of gray code and phase-shifting for 3-D visual measurement,” Opt. Commun. 374, 97–106 (2016). [CrossRef]  

Data availability

No data were generated or analyzed in the presented research.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (7)

Fig. 1.
Fig. 1. The (a) system configuration, (b) system setup, and (c) triangulation geometry of the structured light system.
Fig. 2.
Fig. 2. (a) The light patterns for the binary code method (red square); (b) intensity curves of 2° patterns with the phases of 0 and π; (c) the reconstructed code map without (black) and with (red) the boundary information; (d) The light patterns used for the phase-shifting method (blue square).
Fig. 3.
Fig. 3. (a) The calibration methods for the projector’s Qp point; (b) the heights of the lines measured at 200 mm, 300 mm and 400 mm away from the reference plane (circles) and the fitted linear functions (dashed lines); (c) the calibration methods for the camera’s Qc point; (d) the intensity profiles of the lines, measured along y direction at 200 mm, 300 mm and 400 mm away from the reference plane.
Fig. 4.
Fig. 4. (a) The raw images of the reference plane and the object with 10 light patterns; (b) the intensity profiles along the blue and red lines in (a); (c) the normalized intensity profiles with the modulation removed; (d) the reconstructed code map of the reference plane and the object; (e) the position codes along the blue and red lines in (d); (f) the wrapped phases obtained through four 2° raw images, along the blue (or red) line in (a); (g) the unwrapped phases along the blue and red lines in (d).
Fig. 5.
Fig. 5. The height reconstruction results of (a) the planar screen, (b) the small ball and (c) the large ball; (d) the average height errors throughout the whole object, measured at distances of 200 mm, 300 mm, and 400 mm; (e)(f) the reconstructed profiles (deep blue lines), the fitted curves (dashed blue lines) and the errors between the reconstructed profiles and the fitted curves (orange lines) of (e) the small ball and (f) the large ball.
Fig. 6.
Fig. 6. The average height errors of (a) the planar screen measured at distances from 600 mm to −500 mm and (b) the large sphere measured at distances from 600 mm to 200 mm; (c) the scene for 3D profile reconstruction; (d)(e) the reconstructed 3D profiles. Red arrows: regions with the minimum height; green arrow: regions with the maximum height; yellow arrowheads: regions of low reflectivity; blue arrowheads: regions of high reflectivity.
Fig. 7.
Fig. 7. (a) The face mask used for 3D profile measurement; (b) the raw image of 2° pattern; (c)(d) the reconstructed 3D profile.

Equations (6)

Equations on this page are rendered with MathJax. Learn more.

S = h tan θ h tan ( θ ϕ ) ,
θ ϕ = tan 1 ( ( L h ) tan θ d L h t ) .
S = h tan θ h [ ( L h ) tan θ d L h t ] = t h tan θ + h d L h t .
h = S ( L t ) d t tan θ + S .
I 1 ( x , y ) = A ( x , y ) + B ( x , y ) cos [ θ ( x , y ) ] I 2 ( x , y ) = A ( x , y ) + B ( x , y ) cos [ θ ( x , y ) + π / 2 ] I 3 ( x , y ) = A ( x , y ) + B ( x , y ) cos [ θ ( x , y ) + π ] I 4 ( x , y ) = A ( x , y ) + B ( x , y ) cos [ θ ( x , y ) + 3 π / 2 ] ,
θ ( x , y ) = tan 1 I 4 ( x , y ) I 2 ( x , y ) I 1 ( x , y ) I 3 ( x , y ) ,
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.