Polarimetric target detection under uneven illumination

Bingjing Huang; Tiegen Liu; Jiahui Han; Haofeng Hu

doi:10.1364/OE.23.023603

1. Introduction

Polarimetric imaging systems can obtain the information that is not visible on intensity images. Therefore, it is applied in many fields such as machine vision, remote sensing, biomedical imaging and industrial control [1–4 ]. Compared with classical photoelectric imaging system, polarimetric imaging systems can obtain not only the intensity information, but also the polarization information. One of the most important applications of polarimetric imaging is target detection, in which enhancing the contrast between a target of interest and the background is a key issue for the objective of discrimination. Up to now, many efforts have been done to enhance the contrast of the polarimetric imaging in an optimal way. In particular, one of the representative methods is optimizing the contrast in scalar polarimetric images, in which the intensities of the target and the background are modulated by adjusting the state of polarization state generator (PSG) and polarization state analyzer (PSA) to achieve the optimal contrast [5–8 ]. However, in some situations such as remote sensing, some illumination conditions (such as the inhomogeneous shadow on the scene, the uneven characteristic of the optical lens, and the attenuation of atmosphere) could cause the significant spatial fluctuations of light intensity on the scene [9]. In such cases, the intensity modulation by adjusting PSG and PSA may not be dominant compared to the spatial intensity fluctuations, and therefore, the previous method of maximizing the contrast with the optimal PSG and PSA states may not be competent for target detection.

In this paper, we focus on the passive polarimetric imaging under the condition of uneven illumination, and we investigate the characteristic of the contrast function for the polarimetric image in such case. We show the limitation of the previous method of contrast optimization by just optimizing the state of PSA, and a method of contrast enhancement and optimization with illumination compensation is proposed. We theoretically analyze the effect of illumination compensation on the contrast enhancement, and demonstrate it by numerical simulation and real-world experiment.

2. Stokes polarimetric imaging and contrast optimization

We consider a passive polarimetric imaging system that illuminates the scene with light coming from an unpolarized monochromatic light source. The polarization state in illumination is defined by a Stokes vector S _n with degree of polarization of zero. We use Mueller matrix M to characterize the polarimetric properties of a region of the scene that is corresponding to a pixel in the image. For the sake of simplicity, the scene is supposed to be divided into two regions: a Mueller matrix M^a represents a target of interest and a Mueller matrix M^b represents a background. The Stokes vector of the light scattered by region u is given by the equation

S^{u} = M^{u} S_{n},

where

u \in {a, b}

. The scattered light is analyzed by a PSA, which is a generalized polarizer whose eigenstate is the Stokes vector T. The light intensity measured by the sensor is

i_{u} = \frac{1}{2} T^{T} S^{u},

where the superscript T denotes matrix transposition. In this equation, T is a purely polarized Stokes vector with unit intensity. It can be seen that by changing the polarization state of PSA, the intensity coming from the objects in the scene can be modulated. In the case where the measurement is perturbed by the additive noise, Eq. (2) can be written as

i_{u} = \frac{1}{2} T^{T} S^{u} + n,

where n is a random variable of zero mean and variance σ ² .

In general, the contrast between the target of interest in region a and the background in region b in the image is quantified by the Fisher ratio [10], which is given by

ℱ (T) = \frac{{[〈 i_{a} 〉 - 〈 i_{b} 〉]}^{2}}{var [i_{a}] + var [i_{b}]},

where

〈 i_{u} 〉

and

var [i_{u}]

refers to the mean and the variance of i_u in regions

u \in {a, b}

respectively. In the case where the fluctuations of S ^u and n are independent, the mean and variance of i_u in regions

u \in {a, b}

are given by

〈 i_{u} 〉 = \frac{1}{2} T^{T} 〈 S^{u} 〉,

var [i_{u}] = \frac{1}{4} T^{T} 〈 (S^{u} - 〈 S^{u} 〉) {(S^{u} - 〈 S^{u} 〉)}^{T} 〉 T + σ^{2},

where

〈 . 〉

denotes the ensemble averaging. Each pixel belonging to region

u \in {a, b}

has a Stokes vector S that deviates from the average vector

〈 S^{u} 〉

of this region. It can be seen from Eq. (6) that

var [i_{u}]

considerably depends on the variance of the Stokes vectors

S^{u}

.

The Stokes vectors S and T can be expressed by S = [S ₀,s ^T]^T and T = [1,t ^T]^T, where s and t are 3-dimensional partial Stokes vectors given by s = [S ₁,S ₂,S ₃]^T and t = [T ₁,T ₂,T ₃]^T respectively. In particular, t is a unit norm vector defining the state of the PSA.

Consequently, $var [i_{u}]$ given by Eq. (6) can be expressed as:

\begin{matrix} var [i_{u}] = \frac{1}{4} T^{T} 〈 (S^{u} - 〈 S^{u} 〉) {(S^{u} - 〈 S^{u} 〉)}^{T} 〉 T + σ^{2} \\ = \frac{1}{4} (〈 {[Δ S_{0}^{u}]}^{2} 〉 + 〈 t^{T} Δ s^{u} Δ s^{u}^{T} t + Δ S_{0}^{u} Δ s^{u}^{T} t + t^{T} Δ s^{u} Δ S_{0}^{u} 〉) + σ^{2}, \end{matrix}

where

Δ S_{0}^{u} = S_{0}^{u} - 〈 S_{0}^{u} 〉

and

Δ s^{u} = s^{u} - 〈 s^{u} 〉

.

It is noticed that the term $〈 {[Δ S_{0}^{u}]}^{2} 〉$ , which refers to the variance of $S_{0}^{u}$ , does not depend on the state of PSA t. If the value of $〈 {[Δ S_{0}^{u}]}^{2} 〉$ is large, then the possibility to adjust $var [i_{u}]$ by t will be limited. Consequently, the influence of t on the function F could be not significant. In this case, the previous method of contrast optimization by just adjusting the state of PSA t [5] could not have good performance. In such case, it can be seen from Eqs. (4) and (7) that in order to achieve a high value of $ℱ$ , one needs to decrease the value of $〈 {[Δ S_{0}^{u}]}^{2} 〉$ . On the other hand, decreasing the value of $〈 {[Δ S_{0}^{u}]}^{2} 〉$ could enhance the ability of modulating Fisher ratio $ℱ$ by t, and thus could lead to a stronger contrast enhancement by optimizing t.

According to the analysis above, based on the light intensity information contained in the Stokes vector, a stronger contrast enhancement could be achieved with further computational steps, as described as follows. If we compensate the illumination fluctuation by dividing the intensity of the polarimetric scalar image i_u by $S_{0}^{u} / 〈 S_{0}^{u} 〉$ , the intensity of the image after illumination compensation can then be expressed as:

i_{u}' = \frac{i_{u}}{S_{0}^{u}} 〈 S_{0}^{u} 〉 = \frac{1}{2} T^{T} (\frac{S^{u}}{S_{0}^{u}}) 〈 S_{0}^{u} 〉 + n',

where

n'

is a random variable representing the noise after illumination compensation with the variance of

σ'^{2}

. The variance of

n'

can be expressed as

var [n'] = var [\frac{n}{S_{0}^{u}} 〈 S_{0}^{u} 〉] = var [\frac{n}{S_{0}^{u}}] {〈 S_{0}^{u} 〉}^{2} = σ'^{2} .

It is noticed that after illumination compensation, the noise term

n'

depends on

S_{0}^{u}

. By employing S = [S ₀,s ^T]^T and T = [1,t ^T]^T to represent the Stokes vector S and T respectively, the variance of

i_{u}'

can be thus written as

\begin{matrix} var [i_{u}'] = \frac{1}{4} T^{T} 〈 [(\frac{S^{u}}{S_{0}^{u}}) 〈 S_{0}^{u} 〉 - 〈 (\frac{S^{u}}{S_{0}^{u}}) 〉 〈 S_{0}^{u} 〉] {[(\frac{S^{u}}{S_{0}^{u}}) 〈 S_{0}^{u} 〉 - 〈 (\frac{S^{u}}{S_{0}^{u}}) 〉 〈 S_{0}^{u} 〉]}^{T} 〉 T + σ'^{2} \\ = \frac{1}{4} [1, t^{T}] 〈 [[\begin{matrix} 1 \\ s^{u} / S_{0}^{u} \end{matrix}] 〈 S_{0}^{u} 〉 - 〈 [\begin{matrix} 1 \\ s^{u} / S_{0}^{u} \end{matrix}] 〉 〈 S_{0}^{u} 〉] {[[\begin{matrix} 1 \\ s^{u} / S_{0}^{u} \end{matrix}] 〈 S_{0}^{u} 〉 - 〈 [\begin{matrix} 1 \\ s^{u} / S_{0}^{u} \end{matrix}] 〉 〈 S_{0}^{u} 〉]}^{T} 〉 [\begin{matrix} 1 \\ t \end{matrix}] + σ'^{2} \\ = \frac{1}{4} t^{T} 〈 (\frac{s^{u}}{S_{0}^{u}} 〈 S_{0}^{u} 〉 - 〈 \frac{s^{u}}{S_{0}^{u}} 〉 〈 S_{0}^{u} 〉) {(\frac{s^{u}}{S_{0}^{u}} 〈 S_{0}^{u} 〉 - 〈 \frac{s^{u}}{S_{0}^{u}} 〉 〈 S_{0}^{u} 〉)}^{T} 〉 t + σ'^{2} . \end{matrix}

By comparing Eq. (6) and Eq. (10), it can be seen that by illumination compensation given by Eq. (8), the variance of the noise term is amplified, or in other words, $σ'^{2}$ is greater than $σ^{2}$ . On the other hand, the uneven illumination could lead to the great variance of the Stokes vectors $S^{u}$ . By illumination compensation, the variance of the Stokes vectors can be decreased, and thus the first term (the signal term) on the right side in Eq. (6) could decrease to that in Eq. (10). In the case of great spatial illumination fluctuation or low noise level, the decrease of the variance of the signal term by illumination compensation is dominant compared to the increase of the variance of the noise term, and thus $var [i_{u}']$ should be lower than $var [i_{u}]$ .

According to Eqs. (8) and (10) , the corresponding Fisher ratio for $i_{u}'$ can be deduced as

ℱ^{'} (T) = \frac{{[〈 i_{a}' 〉 - 〈 i_{b}' 〉]}^{2}}{var [i_{a}'] + var [i_{b}']} .

By comparing Eq. (10) and Eq. (7), it can be seen in Eq. (10) that the term $〈 {[Δ S_{0}^{u}]}^{2} 〉$ , which does not depend on t, is eliminated through illumination compensation described in Eq. (8), and thus the dependence of $var [i_{u}']$ and $ℱ^{'}$ on t is increased.

For optimizing the contrast, our goal is to maximize the Fisher ratio, which depends on the mean and the variance of the light intensity in the scene. According to the analysis above, if the processing given by Eq. (8) is performed on the polarimetric scalar images, the variance of the intensity can be decreased and thus the contrast can be enhanced. In addition, the processing given by Eq. (8) leads to a new Fisher ratio function given by Eq. (11) and thus the potentiality of a further contrast optimization. In order to demonstrate the effect of illumination compensation in the contrast enhancement, we take one example based on numerical simulation and another one based on real-world polarimetric images in the following.

3. Numerical simulation

In our numerical simulation, we model the scene according to Eq. (1) and Eq. (3). The illumination beam is the unpolarized natural light, and thus the Stokes vector of the scattered light S ^u is equal to the first column of the Mueller matrix M^u. The mean of the Stokes vectors of regions a and b are assumed to be

〈 S^{a} 〉 = [\begin{matrix} 1 \\ - 0.09 \\ - 0.62 \\ 0.08 \end{matrix}], 〈 S^{b} 〉 = [\begin{matrix} 1 \\ - 0.10 \\ - 0.45 \\ 0.21 \end{matrix}] .

We assume each coefficient of the Mueller matrix M^u randomly fluctuates with the variance of 0.015 and the variance of the addictive noise is set to be 0.005. In addition, in order to simulate the uneven illumination, we assume that the illumination on the scene is spatially fluctuated with the variance of 0.06, and the corresponding spatial intensity distribution of the illumination light on the scene is shown in Fig. 1 . In such case, the spatial fluctuation of light intensity i_u induced by uneven illumination is dominant compared to those induced by the nonuniformity of M^u and the noise.

Fig. 1 Intensity distribution of the illumination light on the scene.

Download Full Size | PDF

In order to perform the optimization of the Fisher ratio and the illumination compensation given by Eq. (8), one needs to know the ensemble averages and the variances of the Stokes vectors in regions a and b, and therefore, the boundary between regions a and b has to be identified in advance. Figure 2(a) shows the measured Stokes vectors of the scene, and it can be seen that the boundary between regions a and b is hard to be identified due to the significant uneven illumination. However, by normalizing the Stokes vectors, the influence of the uneven illumination can be considerably decreased, and the boundary between regions a and b becomes clear for some elements of the Stokes vector, as shown in Fig. 2(b). By employing the image segmentation method of GrabCut [11], the boundary between regions a and b can be identified based on the normalized Stokes vector shown in Fig. 2(b). According to the measured Stokes vectors of the scene as well as the identified boundary between regions a and b, the ensemble averages and variances of the Stokes vectors in regions a and b can be consequently calculated.

Fig. 2 (a) Measured Stokes vectors of the scene in numerical simulation. (b) Normalized Stokes vectors of the scene in numerical simulation.

Download Full Size | PDF

Let us first consider the case where the uneven illumination is not compensated in the optimization process, which is the case reported in [5,8,12 ]. In this case, the variance of $S_{0}^{u}$ given by $〈 {[Δ S_{0}^{u}]}^{2} 〉$ is a constant with great value. The optimal state of t maximizing the Fisher ratio are found to be (α = 43°, ε = 6°), where α and ε denotes the azimuth and ellipticity of t respectively. The obtained image is shown in Fig. 3(a) , in which the Fisher ratio is $ℱ$ = 0.44. It can be seen that the object is hard to be discriminated in Fig. 3(a).

Fig. 3 (a) Optimal image without illumination compensation (α = 43°, ε = 6°). (b) Image with compensation illumination (α = 43°, ε = 6°). (c) Optimal image with illumination compensation (α = −41°, ε = −18°).

Download Full Size | PDF

Let us now consider the case where the uneven illumination has been compensated by performing the processing given by Eq. (8). In this case, we can obtain a much more clear image with Fisher ratio of $ℱ$ = 32.88 by illumination compensation while keeping the state of PSA unchanged [see Fig. 3(b)]. Furthermore, if we optimize the state of PSA after the illumination compensation, the Fisher ratio can be further improved to be $ℱ$ = 40.60, as shown in Fig. 3(c), and the corresponding optimal state of PSA is found to be (α = −41°, ε = −18°).

It is noticed that the optimal states of t for in Fig. 3(b) and 3(c) are different. By comparing Eq. (7) and Eq. (10), it can be seen that the relation between variable t and the variance of the intensity are different due to the illumination compensation, and thus the optimization of $ℱ$ should lead to different optimal states of t. The result in Fig. 3 indicates that the illumination compensation not only enhances the contrast, but also generates the capacity of further contrast optimization.

Besides, it can be seen from Eq. (10) that the variance of $i_{u}'$ partially depends on $var [n']$ , and consequently, the noise term could influence the maximum value of Fisher ratio that can be achieved after illumination compensation. However, it is noticed in Eq. (9) that $var [n']$ does not depend on the state of PSA t. Therefore, the optimal state of PSA for maximal Fisher ratio does not depend on the noise term.

4. Real-word experiment

Let us now consider a real-world polarimetric imaging scenario. The PSA consists of a rotating quarter wave plate and a rotating linear polarizer. By adjusting the orientations of the polarizer and the wave plate, the azimuth and ellipticity of t can be modulated. The target region is constituted by a piece of translucent adhesive tape, on the backside of which another piece of translucent adhesive tape has been stuck on a polarizer [see Fig. 4(a) ]. The mask between the LED light source and the scene is a piece of PMMA mask with significant spatial transmittance fluctuations. The standard intensity image of this scene is shown in Fig. 4(b), in which the adhesive tape is hard to be discriminated.

Fig. 4 (a) Scheme of the observed scene. (b) Intensity image of the scene.

Download Full Size | PDF

The Stokes vectors of all pixels can be calculated based on four intensity measurements at four different configurations of the PSA states [13,14 ], as shown in Fig. 5(a) , and it can be seen that the boundary between the target and the background is hard to be identified. While by normalizing the Stokes vectors, the boundary between the target and the background becomes clear, as shown in Fig. 5(b), which can be identified by the image segmentation method of GrabCut. Consequently, based on the measured Stokes vectors of the scene and the identified boundary between the target and the background, the means and variances of the Stokes vectors in region a (target) and b (background) can be estimated. Based on the estimated Stokes vectors of all the pixels, we first search the optimal state of PSA that maximizes the Fisher ratio without illumination compensation, which is found to be (α = −11°, ε = 24°). The corresponding image is shown in Fig. 6(a) . Compared with the intensity image in Fig. 4(b), the Fisher ratio of the polarimetric scalar image in Fig. 6(a) is slightly increased from 0.31 to 0.75. However, it is still difficult to discriminate the adhesive tape even at the optimal state of t without illumination compensation. If we compensate the illumination while keeping the state of PSA unchanged, the Fisher ratio can be significant increased to as high as 17.44 [see Fig. 6(b)], and it is easy to discriminate the target from the background.

Fig. 5 (a) Measured the Stokes vectors of the scene in real-world experiment. (b) Normalized Stokes vectors of the scene in real-world experiment.

Download Full Size | PDF

Fig. 6 (a) Optimal image of the scene in Fig. 3 without compensation with (α = −11°, ε = 24°). (b) Compensating the illumination by keeping the PSA state unchanged.

Download Full Size | PDF

In order to further enhance the contrast of the image shown in Fig. 6(b), we perform an exhaustive search to find the optimal PSA settings that maximize the contrast of the illumination compensated image. The search was performed with a resolution step of 1 degree for the azimuth and ellipticity of t. We have represented in Fig. 7 the Fisher ratio as a function of azimuth and ellipticity when optimizing t with and without illumination compensation. Comparing Fig. 7 (a) and 7(b), it is found that the optimal states of t are different. In addition, the order of the value of Fisher ratio with illumination compensation [see Fig. 7(b)] is much greater than that without illumination compensation [see Fig. 7(a)], and the maximum value of Fisher ratio in Fig. 7(b) is much greater than that in Fig. 7(b). In addition, it can be seen that the rate of variation of $ℱ$ in Fig. 7(b) is greater than that in Fig. 7(a), which indicates that the Fisher ratio is more sensitive to t after illumination compensation.

Fig. 7 Fisher ratio $ℱ$ as a function of azimuth and ellipticity of t without compensate the illumination (a) and with compensation (b). Both maps where created using a step of 1 degree.

Download Full Size | PDF

Therefore, if we compensate the illumination and obtaining the scalar image with the optimal state of PSA in Fig. 7(b) (α = 35°, ε = −32°), we can further enhance the contrast between the target and background to $ℱ$ = 22.07. The corresponding image is shown in Fig. 8(b) . This is a clear illustration that by optimization t after the compensation of illumination, the contrast can be further improved. In addition, comparing the parameters of the two images presented in the right part of Fig. 8, it can be seen that with the illumination compensation, the improvement of Fisher ratio is mainly attributed to the decrease of the variances of the intensities in the object region and background region.

Fig. 8 (a) Compensating the illumination on the basis of Fig. 4(a). (b) Optimal image of the scene with illumination compensation.

Download Full Size | PDF

5. Conclusion

In conclusion, we have shown that a significant spatial illumination fluctuation can disturb the target discrimination in the polarimetric scalar image, and it is found that the light intensity information contained in the Stokes vector can be utilized for further improving the contrast of the polarimetric image. Based on this point, we propose a methodology of illumination compensation and contrast optimization, which addresses successfully the issue of the spatial illumination fluctuations. The results of numerical simulation and real-word experiment in this paper show that it is possible to improve and further optimize the contrast of the image by illmunation compensation to make the object easy to be discriminated in the polarimetric image. The method proposed in this paper is particularly effective in the cases with great spatial illumination fluctuations, such as remote sensing scenarios.

This work opens up perspectives for further contrast enhancement of the polarimetric imaging with the proper processing by taking full advantage of the obtained polarization information, and we believe different processings should be performed under different imaging conditions. We are currently investigating the proper approaches for the polarimetric scalar images under other conditions, such as polarimetric imaging of objects in scattering media.

Acknowledgments

This work is supported by the National Natural Science Foundation of China (No. 61405140), the National Instrumentation Program (No. 2013YQ030915), and the Natural Science Foundation of Tianjin (No.15JCQNJC02000). Haofeng Hu acknowledges the Fondation Franco-Chinoise pour la Science et ses Applications (FFCSA) and the China Scholarship Council (CSC).

References and links

1. L. B. Wolff, Polarization Methods in Computer Vision (Columbia University, 1991).

2. J. S. Tyo, D. L. Goldstein, D. B. Chenault, and J. A. Shaw, “Review of passive imaging polarimetry for remote sensing applications,” Appl. Opt. 45(22), 5453–5469 (2006). [CrossRef] [PubMed]

3. A. Pierangelo, A. Benali, M.-R. Antonelli, T. Novikova, P. Validire, B. Gayet, and A. De Martino, “Ex-vivo characterization of human colon cancer by Mueller polarimetric imaging,” Opt. Express 19(2), 1582–1593 (2011). [CrossRef] [PubMed]

4. F. Meriaudeau, M. Ferraton, C. Stolz, O. Morel, and L. Bigué, “Polarization imaging for industrial inspection,” Proc. SPIE 6813, 681308 (2008). [CrossRef]

5. G. Anna, F. Goudail, and D. Dolfi, “Polarimetric target detection in the presence of spatially fluctuating Mueller matrices,” Opt. Lett. 36(23), 4590–4592 (2011). [CrossRef] [PubMed]

6. F. Goudail and A. Bénière, “Optimization of the contrast in polarimetric scalar images,” Opt. Lett. 34(9), 1471–1473 (2009). [CrossRef] [PubMed]

7. F. Goudail, “Optimization of the contrast in active Stokes images,” Opt. Lett. 34(2), 121–123 (2009). [CrossRef] [PubMed]

8. G. Anna, F. Goudail, and D. Dolfi, “Optimal discrimination of multiple regions with an active polarimetric imager,” Opt. Express 19(25), 25367–25378 (2011). [CrossRef] [PubMed]

9. T. Lillesand, R. W. Kiefer, and J. Chipman, Remote Sensing and Image Interpretation (John Wiley & Sons, 2014).

10. K. Fukunaga, Introduction to Statistical Pattern Recognition (Academic, 2013).

11. C. Rother, V. Kolmogorov, and A. Blake, ““GrabCut”-interactive foreground extraction using iterated graph cuts,” ACM Trans. Graph. 23(3), 309–314 (2004). [CrossRef]

12. M. Boffety, H. Hu, and F. Goudail, “Contrast optimization in broadband passive polarimetric imaging,” Opt. Lett. 39(23), 6759–6762 (2014). [CrossRef] [PubMed]

13. H. Hu, G. Anna, and F. Goudail, “On the performance of the physicality-constrained maximum-likelihood estimation of Stokes vector,” Appl. Opt. 52(27), 6636–6644 (2013). [CrossRef] [PubMed]

14. L. Goldstein Dennis, Polarized Light (Marcel Dekker, 2003).

Polarimetric target detection under uneven illumination

Abstract

1. Introduction

2. Stokes polarimetric imaging and contrast optimization

3. Numerical simulation

4. Real-word experiment

5. Conclusion

Acknowledgments

References and links

Cited By

Figures (8)

Equations (12)

Optics Express