Target enhanced 3D reconstruction based on polarization-coded structured light

Xiao Huang; Jian Bai; Kaiwei Wang; Qun Liu; Yujie Luo; Kailun Yang; Xianjing Zhang

doi:10.1364/OE.25.001173

1. Introduction

3D reconstruction is one of the fundamental issues in computer vision, which is extensively applied to industrial automation, mapping, modeling and so forth. Structured light has become one of the mainstream methods of the 3D reconstruction of objects. Multifarious structured light methods have been proposed to cope with different tasks [1]. They could be roughly classified into discrete [2–5] and continuous [6], temporary [2–4, 6] and spatial methods [5] according to their coding strategy. However, most structured light methods require plain ambience which may not be satisfied in practical circumstances [7]. For structured light system, complex ambience are undesirable because (1) in these areas, due to defocus and intensity attenuation, loss of pattern contrast could produce severe reconstruction noise [8] and (2) processing of the region out of interest leads to unnecessary cost of time and computing resource. Hence, pre-processed enhancement of the target object is favorable to good 3D reconstruction performance and efficiency.

Polarization is one of the basic properties of light, apart from intensity, frequency and coherence [9]. In comparison with conventional intensity imaging, polarimetric imaging extends information dimension. Therefore, polarimetric imaging is extensively studied, such as animal vision [10], biomedical imaging [11], remote sensing [12] and computer vision [13]. In particular, as polarization parameters are closely related with physical properties of materials, polarimetric imaging is useful for contrast optimization [14], material discrimination [15], target detection [16], and image enhancement in underwater and hazed environment [17, 18]. Meanwhile, given that light with different polarization state behaves differently at the interface of object surface, polarization has been applied to many surface measurements techniques. In passive 3D reconstruction applications, polarization information could directly help estimate depth map of distant objects in hazed environment [18] and provide surface normal information with prior knowledge [19]. Nevertheless, the polarization cues of passive methods are generally weak. Active polarization-based 3D reconstruction could provide reliable polarization information to assist the phase-shifting [20, 21] and photometric measurements [22, 23]. However, most of the current active methods are based on temporal global polarized illumination that either require expensive elaborated device or suffer from low efficiency and potential noise due to the mechanical rotation of polarizers.

In this paper, taking advantage of the intrinsic properties of LCD projector, we propose a structured light coded by spatially-distributed polarization state of the illuminating patterns, which can (1) estimate the degree of linear polarization (DOLP) that provides additional description of the scene; (2) enhance targets with polarization cues from complex ambience;(3) reconstruct the target in 3D effectively and efficiently. In addition, the proposed approach could achieve snapshot for each structured light pattern because no rotation of polarizer is required, which is convenient for practical applications. In this paper, both principles and experimental results are presented. Comparison between the proposed approach and the conventional approach is performed and discussed.

2. Principles

2.1 Polarization coding strategy

For triangulation-based structured light system, accurate correspondence between camera pixels and projector units is an imperative prerequisite for the following ray-plane triangulation and 3D reconstruction [24]. Structured light based on Gray code (GC) patterns is a classical technique to obtain such correspondence. Gray code patterns are a set of binary patterns coded by intensity, i.e. dark for 0 and bright for 1. An example with four patterns is shown in Fig. 1, every adjacent code word with $2^{m}$ bits differ by only one bit, where m is the sequence number of the patterns. In this manner, Gray code patterns could minimize errors and tend to be robust [3]. In practice, in order to remove decoding artifacts, both patterns and their inverse are projected. In this way, points with weak response to the intensity modulation of Gray code patterns could be eliminated by a threshold of intensity contrast. The intensity contrast (IC) is defined as Eq. (1).

I C = \frac{I_{p a t} - I_{i n v}}{I_{p a t} + I_{i n v}},

where the

I_{p a t}

and

I_{i n v}

are the intensity of the pattern and its inverse pattern.

Fig. 1 Gray code structured light patterns.

Download Full Size | PDF

Inspired by the fact that polarization provides a new information channel, we suppose that structured light coded in polarization channel rather than conventional intensity channel could enhance target by polarization information and perform 3D reconstruction efficiently. Figure 2 illustrates the coding strategy of the proposed polarization-coded (PC) structured light. The basic coding strategy refers to Gray code. The polarization-coded structured light patterns have even intensity but uneven polarization state. As arrows indicate, light with horizontal polarization is coded as 1 while vertical as 0. Considering that ordinary camera only response to light intensity, a polarizer with horizontal transmission axis is required to analyze the polarization codes.

Fig. 2 Polarization-coded structured light patterns.

Download Full Size | PDF

2.2 DOLP estimation and target enhancement

The degree of linear polarization in polarization-difference imaging [25] is widely employed as a basis of polarization analysis for its convenience and effectiveness [17–19, 26]. DOLP in polarization-difference imaging is given by:

DOLP = \frac{I_{| |} - I_{⊥}}{I_{| |} + I_{⊥}},

where

I_{| |}

and

I_{⊥}

represent the intensity of polarization components that are parallel and perpendicular to the polarization state of incident light respectively. Considering horizontally polarized incident light,

I_{| |}

and

I_{⊥}

could be expressed in Stokes vector and Mueller matrix form as

\begin{array}{l} {\overset{⇀}{S}}_{| |, ⊥} = M_{P | |, ⊥} • M_{O} • {\overset{⇀}{S}}_{i n} = (\begin{matrix} 0.5 & \pm 0.5 & 0 & 0 \\ \pm 0.5 & 0.5 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{matrix}) (\begin{matrix} m_{11} & m_{12} & m_{13} & m_{14} \\ m_{21} & m_{22} & m_{23} & m_{24} \\ m_{31} & m_{32} & m_{33} & m_{34} \\ m_{41} & m_{42} & m_{43} & m_{44} \end{matrix}) (\begin{matrix} 1 \\ 1 \\ 0 \\ 0 \end{matrix}) \\ = 0.5 (\begin{matrix} (m_{11} + m_{12}) \pm (m_{21} + m_{22}) \\ \pm (m_{11} + m_{12}) + (m_{21} + m_{22}) \\ 0 \\ 0 \end{matrix}), \end{array}

I_{| |} = {\overset{⇀}{S}}_{0, | |} = m_{11} + m_{12} + m_{21} + m_{22},

I_{⊥} = {\overset{⇀}{S}}_{0, ⊥} = m_{11} + m_{12} - m_{21} - m_{22},

where

{\overset{⇀}{S}}_{i n}

,

{\overset{⇀}{S}}_{| |}

,

{\overset{⇀}{S}}_{⊥}

are Stokes vectors of incident light, parallel and perpendicular components of exit light, and

M_{P}

,

M_{O}

are Mueller matrixes of polarizer and object. Substituting Eq. (2) with Eq. (4), DOLP can be rewritten with Mueller matrix elements as

DOLP = \frac{m_{22} + m_{21}}{m_{11} + m_{12}} .

Without loss of generality, for reflection at air-dielectric interface, the Mueller matrix of object could be derived from Fresnel equations as Eq. (6) [9].

M_{O} = \frac{1}{2} {(\frac{\tan θ_{-}}{\tan θ_{+}})}^{2} (\begin{matrix} \cos^{2} θ_{-} + \cos^{2} θ_{+} & \cos^{2} θ_{-} - \cos^{2} θ_{+} & 0 & 0 \\ \cos^{2} θ_{-} - \cos^{2} θ_{+} & \cos^{2} θ_{-} + \cos^{2} θ_{+} & 0 & 0 \\ 0 & 0 & - 2 \cos θ_{+} \cos θ_{-} & 0 \\ 0 & 0 & 0 & - 2 \cos θ_{+} \cos θ_{-} \end{matrix})

where

θ_{\pm} = θ_{i n c i d e n t} \pm θ_{r e f r a c t i v e}

.Particularly, in the situations with small incident angle (

θ_{\pm} \approx 0

), it is clear that

m_{12} = m_{21} = \cos^{2} θ_{-} - \cos^{2} θ_{+} \approx 0

. Thus, DOLP in situations with small incident angle can be estimated as

DOLP = \frac{m_{22}}{m_{11}} .

Furthermore, with the same small angle condition as discussed, it is interesting to suppose that there are two orthogonally polarized, horizontally and vertically for instance, incident light but only one polarizer with a fixed transmission axis, in horizontal for consistency with before. Similar to the Eq. (3), intensities of the two exit light ${I^{'}}_{| |}$ and ${I^{'}}_{⊥}$ could be expressed as

\begin{array}{l} {S^{'}}_{(| |, ⊥)} = {M^{'}}_{P} • M_{O} • {S^{'}}_{i n} = (\begin{matrix} 0.5 & 0.5 & 0 & 0 \\ 0.5 & 0.5 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{matrix}) (\begin{matrix} m_{11} & m_{12} & m_{13} & m_{14} \\ m_{21} & m_{22} & m_{23} & m_{24} \\ m_{31} & m_{32} & m_{33} & m_{34} \\ m_{41} & m_{42} & m_{43} & m_{44} \end{matrix}) (\begin{matrix} 1 & 1 \\ 1 & - 1 \\ 0 & 0 \\ 0 & 0 \end{matrix}) \\ = 0.5 (\begin{matrix} m_{11} + m_{12} + m_{21} + m_{22} & m_{11} - m_{12} + m_{21} - m_{22} \\ m_{11} + m_{12} + m_{21} + m_{22} & m_{11} - m_{12} + m_{21} - m_{22} \\ 0 & 0 \\ 0 & 0 \end{matrix}), \end{array}

{I^{'}}_{| |} = {S^{'}}_{0, | |} = m_{11} + m_{12} + m_{21} + m_{22},

{I^{'}}_{⊥} = {S^{'}}_{0, ⊥} = m_{11} - m_{12} + m_{21} - m_{22},

where

{S^{'}}_{i n}

,

{S^{'}}_{(| |, ⊥)}

are expanded Stokes vectors of incident light and exit lights that parallel/perpendicular to the transmission axis of polarizer, and

{M^{'}}_{P}

are Mueller matrix of the fixed polarizer. In exactly the same manner as Eq. (5-7), it could be derived that

\frac{{I^{'}}_{| |} - {I^{'}}_{⊥}}{{I^{'}}_{| |} + {I^{'}}_{⊥}} = \frac{m_{22} + m_{12}}{m_{11} + m_{21}} \approx \frac{m_{22}}{m_{11}} \approx DOLP .

Equation (10) indicates that when the light source and sensor are close enough ( $θ_{\pm} \approx 0$ ) that $m_{12}$ and $m_{21}$ are negligible, DOLP could be estimated by modulation of the light source rather than the polarizer. In the context of structured light system, it means that DOLP of the scene could be estimated by projector-camera pair with sufficiently short baseline. Patterns and their inverse patterns act as the two orthogonally polarized light sources to estimate the DOLP.

Similar to the threshold of IC mentioned in Section 2.1, threshold of DOLP could also eliminate points with weak response to the polarization modulation. In other words, points that strongly depolarize the incident polarized light tend to remain the same brightness, regardless of the polarization state of incident light. As a consequence, points with low DOLP, namely low contrast between patterns, will be eliminated so that target objects with DOLP over threshold could be enhanced by the proposed polarization-coded structured light patterns.

2.3 Polarization properties of LCD projector

In general, LCD projector depends on thin film transistor liquid crystal display (TFT-LCD) units to modulate the illuminating light. When different voltages applied, liquid crystal changes the polarization state of incident polarized light in different degrees. Then, the modulated light transmits through a polarizer whose transmission axis is orthogonal to the incident polarization state. As a result, the intensity is modulated by Malus’ law while the polarization state of output light is always determined [27]. To facilitate the beam split and combination, polarization-dependent dichroic mirrors and a prism, known as X-cube, are applied [28]. Consequently, the polarization states of output light beams differ in RGB channels. For the LCD projector in this paper, light in green (G) channel is horizontally polarized, while the red (R) and blue (B) channels are vertically polarized. The described intrinsic polarization properties of LCD projector lay the foundation of the proposed spatial multiplexing of polarization states for each pattern.

3. Experimental setup

The experimental scheme is shown in Fig. 3. The polarization-coded structured light is projected by a Toshiba TLP-X2500A LCD projector. The illuminated scene is captured by a 1/2 inch CMOS camera (Daheng Mercury-310-12UC), which works in monochromatic mode. The resolution of the projector is 1024 × 768 and the camera is 2048 × 1536. Calibration for the projector and camera is performed with the Bouguet’s toolbox [29]. A static linear polarizer is fixed before the camera with a horizontal transmission axis. The polarizer is used to assist the DOLP estimation and pattern decoding mentioned in Section 2.

Fig. 3 Experimental scheme. The scale of the figure has been adapted to fit the page. The true distance between the projector and objects is around 1400mm and the baseline of projector-camera pair is 181mm. The projected pattern in this figure is an illustrative example for the polarization-coded structured light patterns.

Download Full Size | PDF

The distance from the projector-camera pair to the scene is around 1400mm. The baseline length between projector and camera is 181mm, which is relatively small to meet the approximation condition of small incident angle (less than $3.7 °$ ) for DOLP estimation. Certainly, excessively short baseline is adverse to the accuracy of 3D reconstruction so that a slight compromise between the DOLP estimation and 3D reconstruction is necessary. To suppress the discontinuity of DOLP estimation at the edges of pattern stripes, more than one pattern is used to estimate the DOLP and the results are finally merged together. As the max resolution of the projector is $1024 = 2^{10}$ , 10 patterns are adopted to make full use of the resolution. Patterns and their inverse are projected successively in line with the depth of coding. As mentioned before, horizontal polarized light is coded as 1, which is green in color; vertical polarized light is coded as 0, which is magenta (mixture of red and blue) in color. Ideally, intensity of the two kinds of light should be equal. In fact, the received intensity on camera sensor is related to the spectrum distribution of projector light source and the spectrum response of camera. In addition, it is also related to the diattenuation capability of the polarizer when polarizer is applied. Therefore, Gamma calibration should be performed to guarantee an even illumination. It is also convenient to adjust the brightness of pattern itself to accomplish such calibration.

Furthermore, it is worth noting that there are two assumptions for the scene in our experiment. Firstly, it is supposed that there is no strong inter-reflection in the scene. For most structured light systems, inter-reflection still remains a challenging problem to be completely solved in the future [19]. Secondly, it is assumed that absorption for both polarized light is relatively balanced, which is important for an accurate estimation of DOLP. As the spectrum of the projector is wide, this assumption would not be too restrictive in general case.

4. Results and discussion

To demonstrate the feasibility of the proposed polarization-coded structured light, a comparison between images without/with polarizer is shown in Fig. 4. As discussed before, the structured light is coded in polarization state rather than intensity. The illumination intensity is supposed to be even. Figure 4(a) shows the intensity distribution of the scene illuminated by an example pattern without polarizer. The brightness of the complementary pattern stripes is close as a whole, depending on the reflectivity of object itself as well. Once the polarizer is applied, as Fig. 4(b) illustrates, the polarization-coded patterns would work immediately. Response ability to the polarizer depends on DOLP of object itself. In this way, it is clear that in the scene, the contrast of stripes varies by DOLP. Thus, with a constraint of DOLP in 3D reconstruction process, target object could be selectively reconstructed.

Fig. 4 Pattern-illuminated experimental scene (a) without polarizer (b) with polarizer. For comparability, reasonable exposure compensation is employed in the capture of (b) as thoroughly investigated in [30].

Download Full Size | PDF

In the proposed case, objects with various materials are included as the scene. With fine-grained mental surface, the laptop is the main target object to be reconstructed. As the secondary target, a subtle adhesive tape on the wooden board is used to test the enhancement ability for target objects with similar surface appearance. The paper in foreground, wooden board and cloth in background as well as the boxes under the laptop are regarded as the ambience. DOLP analysis of the scene is shown in Fig. 5 to support the target enhancement. Figure 5(a) is the DOLP estimated by the method in this paper, and Fig. 5(b) is the DOLP measured by conventional method [25]. It can be observed that the DOLP estimation method works quite well. Representative areas are labeled in Fig. 5(a) and 5(b) for quantitative analysis. Line I represents the adhesive tape on rough surface, where surface reflectivity fluctuates significantly. Line II represents the marginal part of the target, where the DOLP changes smoothly. Line III represents the central part of the target and it is designed to be vertical. Area IV and V are separated out to evaluate the estimation of target as a whole. DOLP curves of the lines are presented in the Fig. 5(c), where dotted line is corresponding to the estimated DOLP and the solid line is corresponding to the measured DOLP. Standard deviations of the DOLP difference between the dotted line and solid line are 0.0572, 0.0188 and 0.0104 for Line I, II and III respectively. Furthermore, the Pearson correlation coefficients between the estimated DOLP and measured DOLP in Area IV and V are 0.9859 and 0.8797, respectively. It infers that the estimation in area with low DOLP tends to be interfered by noise while estimation in area with high DOLP fits well. In addition, The Pearson correlation coefficient between the global DOLP results are 0.9725, indicating that the linear correlation between the two results are remarkable. It could be seen that the DOLP estimation is reliable, especially for the main target, which is helpful for the subsequent target enhancement and 3D reconstruction.

Fig. 5 DOLP analysis of the scene. (a) The estimated DOLP by our method. (b) The measured DOLP by conventional method. (c) Cross-sections of DOLP results on the Line I, II and III as labeled in (a) and (b).

Download Full Size | PDF

With the constraint from the estimated DOLP, target enhanced 3D reconstruction is performed by the polarization-coded structured light system. To demonstrate the performance, we compared the proposed method with Gray code structured light method as mentioned in Section. 2.1. 3D point cloud is retrieved by decoding and ray-plane intersection calculation. Previous to the decoding, image mask, generated by the thresholds of DOLP/IC, is applied to filter out the invalid points. Without any post-reconstruction processing, 3D visualization of the original point cloud is shown in Fig. 6. Figure 6(a)-6(c) are the results acquired by the polarization-coded structured light with different DOLP constraints. Figure 6(d) is the result acquired by Gray code structured light. Figure 6(a) presents the point clouds of target objects, the laptop and the tape. Figure 6(b) gives the point cloud of the laptop, which is enhanced by a high-pass DOLP filter. Figure 6(c) shows the secondary target, the adhesive tape, which is enhanced by a band-pass DOLP filter. Figure 6(d) is the result of Gray code structured light. Obviously, merely by the intensity, the conventional structured light could not distinguish the target objects from the ambience and suffer from the undesirable extra reconstruction overhead. For Fig. 6(d), 1.416 million points are reconstructed whereas the point number in Fig. 6(a)-6(c) is only 0.300, 0.251 and 0.038 million, respectively. Runtime of the ray-plane intersection calculation are 66.1 $μ s$ , 53.5 $μ s$ , 19.2 $μ s$ and 268.4 $μ s$ for Fig. 6(a)-6(d) respectively. The processing is fulfilled by MATLAB software on a computer with an Intel i5-4460 CPU@3.20 GHz.

Fig. 6 Visualization of the 3D point cloud of (a) the enhanced targets; (b) the enhanced laptop by high-pass DOLP filter; (c) the enhanced adhesive tape by band-pass DOLP filter. (d) Visualization of the 3D point cloud retrieved by conventional Gray code structured light.

Download Full Size | PDF

For the convenience of in-depth quantitative analysis of the overall performance, we transfer the discrete 3D point cloud into depth map with an 800 × 600 resolution. It is worth considering how the pattern contrast threshold affects the target enhancement in 3D reconstruction. Figure 7 shows the depth maps retrieved by different DOLP/IC thresholds. Pixels surrounded by dotted lines belong to the designed targets. Those pixels are regarded as the region of interest. For the polarization-coded structured light, a suitable threshold helps to identify the targets from the complex ambience and maintain the integrity of the target well. On the contrary, for the conventional structured light, no matter what the threshold is, the bright paper in foreground can never be suppressed. Meanwhile, the tape on the wooden board, which shares similar appearance with the wooden board, could barely be figured out either.

Fig. 7 Depth maps retrieved by different thresholds. Figures in upper line, (a) to (c), are corresponding to the polarization-coded structured light method, where the DOLP is taken as the threshold basis. Figures in lower line, (d) to (f), are corresponding to the conventional Gray code structured light method, where the IC is taken as the threshold basis.

Download Full Size | PDF

For a permitted noise level, 0.1 is the minimum threshold of both structured light systems. Valid pixels in the region of interest in Fig. 7(a) and 7(b) are summed up as the benchmark pixel number for each method. To properly evaluate the performance, we define the integrity ratio $α$ as the ratio of valid pixel number to the benchmark pixel number. Target-scene ratio $β$ is defined as the ratio of valid pixel number to the total valid pixel number. For the simplicity, target enhancement merit factor $γ$ is defined as $γ = α β$ , which simultaneously reflects the integrity of the target itself and the proportion that the target covers in the overall reconstructed scene. Figure 8 shows the relationship between pattern contrast threshold and the evaluation factors for the targets. The labeled peak value above the entire bar indicates the target-scene ratio $β$ . The proportion that the colored bar occupies in the entire bar indicates the integrity ratio $α$ . Value of the merit factor $γ$ is calculated and marked in the colored bar. It could be seen that for the proposed polarization-coded structured light, the target-scene ratio is much higher than its counterpart. In addition, as shown in Fig. 8(a), the proposed method keeps absolute advantage in merit factor over the other one. Figure 8(b) shows how the object with middle value of DOLP is gradually eliminated.

Fig. 8 Relationship between pattern contrast threshold and the evaluation factors for the target (a) laptop and (b) tape. Data of left bar are from polarization-coded structured light and data of right bar from Gray code structured light. The value on the top of entire bar is the target-scene ratio $β$ . The proportion that the darker bar occupies in the entire bar indicates the integrity ratio $α$ of target, which is not decimally marked. Value of the merit factor $γ$ is labeled in the dark bar.

Download Full Size | PDF

Furthermore, precision is another aspect of the 3D reconstruction. Depth curves at the same location of the main target for the two approaches (orange lines shown in Fig. 7) are presented in Fig. 9. Though tilted to the camera, the central part of the laptop surface is supposed to be planar. Thus, linear regression is applied to generate local reference line. The coefficient of determination R-square is 0.991 for polarization-coded method and 0.983 for Gray code method. The root-mean-square (RMS) errors are 1.863 and 2.346 respectively. The standard deviations of the distance between points and reference lines are 0.892mm and 1.159mm, respectively. The linear regression indicates that the depth curve measured by polarization-coded structured light has a smaller data fluctuation than that of Gray code structured light. It infers that the proposed approach is more robust against decoding noise in depth measurement of target. Decoding noise often occurs in the area of which contrast is relatively low. When illuminated by structured light patterns with high spatial frequency, the global contrast decreases and decoding noise becomes remarkable. Several factors contribute to the loss of contrast, such as indirect illumination [7], misalignment of display chips, defocus blur [8] and noise from hardware. With the use of polarizer, contrast loss from indirect illumination and chip misalignment can be suppressed by the proposed method. Specifically, when the polarization direction of incident light differs from the polarization axis of polarizer, the polarizer can absorb the incident light accordingly. Indirect illuminating light in scene tends to be unpolarized. And light from relative misaligned channels tends to have a polarization direction that perpendicular to the polarization axis of polarizer, depending on the DOLP of object. Therefore, noises can be suppressed by the absorption of polarizer. With a lower noise level, the polarization-coded structured light can thus achieve a better reconstruction performance than Gray code structured light. In addition, more precise measurement could be achieved by better hardware, finer calibration methods and more comprehensive algorithms [31], which is beyond the scope of this paper.

Fig. 9 Depth curves of the main target by the polarization-coded structured light (Blue) and the Gray code structured light (Red).

Download Full Size | PDF

However, it is worth noting that as the polarizer is a parallel dielectric plate with thickness, it may bring systematic error if no compensation is done. Specifically, a parallel dielectric plate with thickness gives rise to axial shift and transverse shift of the light from object. If the plate is tilted, global shift of the light also occurs. Those shifts affect the correspondence between camera pixels and projector units and thus lead to a slightly underestimated depth value. Actually, similar to the protective glass embedded before image sensor, a fixed polarizer before camera can also be regarded as one part of their composite. Calibration of the projector and the composite camera is required to remove the systematic error brought by the existence of polarizer.

Last but not least, it is worth discussing the existing challenges and some future work for the proposed polarization-coded structured light. For the DOLP estimation, several factors may interfere in the estimation. Selective absorbers for the colored polarized light are strong noise sources, such as polarizer in the scene, objects in colors that are too similar to the patterns and some material with extraordinary photonics structures [32]. As a result, reflection within limited times could also change polarization state slightly and produce some noise for DOLP estimation. For the 3D reconstruction, similar to some structured light methods [1], the proposed method is somewhat color-based at present so that it is not sufficiently robust to colorful objects. In addition, strong scattering light, especially inter-reflection is also a challenge for correct decoding and 3D reconstruction. In the future, we intend to incorporate modification strategies, such as color compensation and inter-reflection elimination, into the proposed method to improve the robustness and performance in both DOLP estimation and 3D reconstruction. Besides, the proposed coding strategy in binary polarization channel is compatible with most existing structured light methods that are based on binary patterns. Patterns in more advanced form could be readily adapted in the future work.

5. Conclusion

In this paper, we proposed a polarization-coded structured light that enhance target from complex ambience by polarization cues and complete its 3D reconstruction effectively and efficiently. DOLP estimation approach suitable for structured light system is proposed to be the basic principle of the target enhancement. The polarization-coded structured light is realized by ordinary LCD projector and no rotation of polarizer is required. Principles are elucidated theoretically and verified by experimental results. The polarization-coded structured light is beneficial to both efficient 3D reconstruction and polarimetric target enhancement because it integrates the acquisition of polarization information and spatial information simultaneously. It has potential for practical applications, such as high-speed 3D reconstruction, underwater target recognition and industrial automation.

Acknowledgments

We acknowledge Prof. I. C. Khoo for the heuristic discussions on liquid crystals photonics during this research. This work was also supported by China Academy of Engineering Physics grants (JCKY2016212A506-0502). We also acknowledge the anonymous reviewers for their insightful comments.

References and links

1. J. Salvi, S. Fernandez, T. Pribanic, and X. Llado, “A state of the art in structured light patterns for surface profilometry,” Pattern Recognit. 43(8), 2666–2680 (2010). [CrossRef]

2. J. L. Posdamer and M. D. Altschuler, “Surface measurement by space-encoded projected beam systems,” Comput. Graph. Image Process. 18(1), 1–17 (1982). [CrossRef]

3. S. Inokuchi, K. Sato, and F. Matsuda, “Range imaging system for 3-D object recognition,” in Proceedings of the International Conference on Pattern Recognition, pp. 806–808 (1984).

4. D. Caspi, N. Kiryati, and J. Shamir, “Range imaging with adaptive color structured light,” IEEE Trans. Pattern Anal. 20(5), 470–480 (1998). [CrossRef]

5. K. L. Boyer and A. C. Kak, “Color-encoded structured light for rapid active ranging,” IEEE Trans. Pattern Anal. 1, 14–28 (1987).

6. S. Zhang, “Recent progresses on real-time 3D shape measurement using digital fringe projection techniques,” Opt. Lasers Eng. 48(2), 149–158 (2010). [CrossRef]

7. M. Gupta, A. Agrawal, A. Veeraraghavan, and S. G. Narasimhan, “A practical approach to 3D scanning in the presence of interreflections, subsurface scattering and defocus,” Int. J. Comput. Vis. 102(1–3), 33–55 (2013). [CrossRef]

8. Y. Xu, L. Ekstrand, J. Dai, and S. Zhang, “Phase error compensation for three-dimensional shape measurement with projector defocusing,” Appl. Opt. 50(17), 2572–2581 (2011). [CrossRef] [PubMed]

9. L. Goldstein Dennis, Polarized Light (Marcel Dekker 2003).

10. G. Horváth, Polarized Light and Polarization Vision in Animal Sciences (Springer, 2014).

11. S. L. Jacques, J. C. Ramella-Roman, and K. Lee, “Imaging skin pathology with polarized light,” J. Biomed. Opt. 7(3), 329–340 (2002). [CrossRef] [PubMed]

12. J. S. Tyo, D. L. Goldstein, D. B. Chenault, and J. A. Shaw, “Review of passive imaging polarimetry for remote sensing applications,” Appl. Opt. 45(22), 5453–5469 (2006). [CrossRef] [PubMed]

13. L. B. Wolff, “Polarization vision: a new sensory approach to image understanding,” Image Vis. Comput. 15(2), 81–93 (1997). [CrossRef]

14. M. Boffety, H. Hu, and F. Goudail, “Contrast optimization in broadband passive polarimetric imaging,” Opt. Lett. 39(23), 6759–6762 (2014). [CrossRef] [PubMed]

15. G. Anna, F. Goudail, and D. Dolfi, “Optimal discrimination of multiple regions with an active polarimetric imager,” Opt. Express 19(25), 25367–25378 (2011). [CrossRef] [PubMed]

16. B. Huang, T. Liu, J. Han, and H. Hu, “Polarimetric target detection under uneven illumination,” Opt. Express 23(18), 23603–23612 (2015). [CrossRef] [PubMed]

17. Y. Y. Schechner and N. Karpel, “Clear underwater vision,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2004), pp. I-536–I-546.

18. E. Namer, S. Shwartz, and Y. Y. Schechner, “Skyless polarimetric calibration and visibility enhancement,” Opt. Express 17(2), 472–493 (2009). [CrossRef] [PubMed]

19. A. Kadambi, V. Taamazyan, B. Shi, and R. Raskar, “Polarized 3d: High-quality depth sensing with polarization cues,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 2015), pp. 3370–3378. [CrossRef]

20. T. Chen, H. P. Lensch, C. Fuchs, and H. Seidel, “Polarization and phase-shifting for 3D scanning of translucent objects,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), pp. 1–8. [CrossRef]

21. B. Salahieh, Z. Chen, J. J. Rodriguez, and R. Liang, “Multi-polarization fringe projection imaging for high dynamic range objects,” Opt. Express 22(8), 10064–10071 (2014). [CrossRef] [PubMed]

22. A. Ghosh, T. Chen, P. Peers, C.A. Wilson, and P. Debevec, “Circularly polarized spherical illumination reflectometry,” ACM Trans. Graph. 30(6), 162 (2010).

23. A. Ghosh, G. Fyffe, B. Tunwattanapong, J. Busch, X. Yu, and P. Debevec, “Multiview face capture using polarized spherical gradient illumination,” ACM Trans. Graph. 30(6), 129 (2011). [CrossRef]

24. R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision (Cambridge University, 2003).

25. M. P. Rowe, E. N. Pugh Jr, J. S. Tyo, and N. Engheta, “Polarization-difference imaging: a biologically inspired technique for observation through scattering media,” Opt. Lett. 20(6), 608–610 (1995). [CrossRef] [PubMed]

26. E. Du, H. He, N. Zeng, M. Sun, Y. Guo, J. Wu, S. Liu, and H. Ma, “Mueller matrix polarimetry for differentiating characteristic features of cancerous tissues,” J. Biomed. Opt. 19(7), 076013 (2014). [CrossRef] [PubMed]

27. I. C. Khoo, Liquid Crystals: Physical Properties and Nonlinear Optical Phenomena (John Wiley & Sons, 2007).

28. H. S. Kwok, P. W. Cheng, H. C. Huang, H. F. Li, Z. R. Zheng, P. F. Gu, and X. Liu, “Trichroic prism assembly for separating and recombining colors in a compact projection display,” Appl. Opt. 39(1), 168–172 (2000). [CrossRef] [PubMed]

29. J. Y. Bouguet, “Camera calibration toolbox for Matlab,” (Accessed 30 July, 2016). http://www.vision.caltech.edu/bouguetj.

30. T. Treibitz and Y. Y. Schechner, “Polarization: Beneficial for visibility enhancement?” Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2009), pp. 525–532.

31. D. Scharstein and R. Szeliski, “High-accuracy stereo depth maps using structured light,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE 2003), pp. I-195–I-202. [CrossRef]

32. T. M. Jordan, D. Wilby, T. H. Chiou, K. D. Feller, R. L. Caldwell, T. W. Cronin, and N. W. Roberts, “A shape-anisotropic reflective polarizer in a stomatopod crustacean,” Sci. Rep. 6, 21744 (2016). [CrossRef] [PubMed]

Target enhanced 3D reconstruction based on polarization-coded structured light

Abstract

1. Introduction

2. Principles

2.1 Polarization coding strategy

2.2 DOLP estimation and target enhancement

2.3 Polarization properties of LCD projector

3. Experimental setup

4. Results and discussion

5. Conclusion

Acknowledgments

References and links

Cited By

Figures (9)

Equations (12)

Optics Express