Abstract
In polarimetric imaging, the uneven illumination could cause the significant spatial intensity fluctuations in the scene, and thus hampers the target detection. In this paper, we propose a method of illumination compensation and contrast optimization for Stokes polarimetric imaging, which allows significantly increasing the performance of target detection under uneven illumination. We show with numerical simulation and real-world experiment that, based on the intensity information contained in the polarization information, the contrast can be effectively enhanced by proper approach, which is of particular importance in practical applications with spatial illumination fluctuations, such as remote sensing.
© 2015 Optical Society of America
1. Introduction
Polarimetric imaging systems can obtain the information that is not visible on intensity images. Therefore, it is applied in many fields such as machine vision, remote sensing, biomedical imaging and industrial control [1–4 ]. Compared with classical photoelectric imaging system, polarimetric imaging systems can obtain not only the intensity information, but also the polarization information. One of the most important applications of polarimetric imaging is target detection, in which enhancing the contrast between a target of interest and the background is a key issue for the objective of discrimination. Up to now, many efforts have been done to enhance the contrast of the polarimetric imaging in an optimal way. In particular, one of the representative methods is optimizing the contrast in scalar polarimetric images, in which the intensities of the target and the background are modulated by adjusting the state of polarization state generator (PSG) and polarization state analyzer (PSA) to achieve the optimal contrast [5–8 ]. However, in some situations such as remote sensing, some illumination conditions (such as the inhomogeneous shadow on the scene, the uneven characteristic of the optical lens, and the attenuation of atmosphere) could cause the significant spatial fluctuations of light intensity on the scene [9]. In such cases, the intensity modulation by adjusting PSG and PSA may not be dominant compared to the spatial intensity fluctuations, and therefore, the previous method of maximizing the contrast with the optimal PSG and PSA states may not be competent for target detection.
In this paper, we focus on the passive polarimetric imaging under the condition of uneven illumination, and we investigate the characteristic of the contrast function for the polarimetric image in such case. We show the limitation of the previous method of contrast optimization by just optimizing the state of PSA, and a method of contrast enhancement and optimization with illumination compensation is proposed. We theoretically analyze the effect of illumination compensation on the contrast enhancement, and demonstrate it by numerical simulation and real-world experiment.
2. Stokes polarimetric imaging and contrast optimization
We consider a passive polarimetric imaging system that illuminates the scene with light coming from an unpolarized monochromatic light source. The polarization state in illumination is defined by a Stokes vector S n with degree of polarization of zero. We use Mueller matrix M to characterize the polarimetric properties of a region of the scene that is corresponding to a pixel in the image. For the sake of simplicity, the scene is supposed to be divided into two regions: a Mueller matrix Ma represents a target of interest and a Mueller matrix Mb represents a background. The Stokes vector of the light scattered by region u is given by the equation
where . The scattered light is analyzed by a PSA, which is a generalized polarizer whose eigenstate is the Stokes vector T. The light intensity measured by the sensor iswhere the superscript T denotes matrix transposition. In this equation, T is a purely polarized Stokes vector with unit intensity. It can be seen that by changing the polarization state of PSA, the intensity coming from the objects in the scene can be modulated. In the case where the measurement is perturbed by the additive noise, Eq. (2) can be written aswhere n is a random variable of zero mean and variance σ 2 .In general, the contrast between the target of interest in region a and the background in region b in the image is quantified by the Fisher ratio [10], which is given by
where and refers to the mean and the variance of iu in regions respectively. In the case where the fluctuations of S u and n are independent, the mean and variance of iu in regions are given by where denotes the ensemble averaging. Each pixel belonging to region has a Stokes vector S that deviates from the average vector of this region. It can be seen from Eq. (6) that considerably depends on the variance of the Stokes vectors.The Stokes vectors S and T can be expressed by S = [S 0,s T]T and T = [1,t T]T, where s and t are 3-dimensional partial Stokes vectors given by s = [S 1,S 2,S 3]T and t = [T 1,T 2,T 3]T respectively. In particular, t is a unit norm vector defining the state of the PSA.
Consequently, given by Eq. (6) can be expressed as:
where and .It is noticed that the term , which refers to the variance of , does not depend on the state of PSA t. If the value of is large, then the possibility to adjust by t will be limited. Consequently, the influence of t on the function F could be not significant. In this case, the previous method of contrast optimization by just adjusting the state of PSA t [5] could not have good performance. In such case, it can be seen from Eqs. (4) and (7) that in order to achieve a high value of , one needs to decrease the value of . On the other hand, decreasing the value of could enhance the ability of modulating Fisher ratio by t, and thus could lead to a stronger contrast enhancement by optimizing t.
According to the analysis above, based on the light intensity information contained in the Stokes vector, a stronger contrast enhancement could be achieved with further computational steps, as described as follows. If we compensate the illumination fluctuation by dividing the intensity of the polarimetric scalar image iu by, the intensity of the image after illumination compensation can then be expressed as:
where is a random variable representing the noise after illumination compensation with the variance of . The variance of can be expressed asIt is noticed that after illumination compensation, the noise term depends on. By employing S = [S 0,s T]T and T = [1,t T]T to represent the Stokes vector S and T respectively, the variance of can be thus written asBy comparing Eq. (6) and Eq. (10), it can be seen that by illumination compensation given by Eq. (8), the variance of the noise term is amplified, or in other words, is greater than . On the other hand, the uneven illumination could lead to the great variance of the Stokes vectors . By illumination compensation, the variance of the Stokes vectors can be decreased, and thus the first term (the signal term) on the right side in Eq. (6) could decrease to that in Eq. (10). In the case of great spatial illumination fluctuation or low noise level, the decrease of the variance of the signal term by illumination compensation is dominant compared to the increase of the variance of the noise term, and thus should be lower than.
According to Eqs. (8) and (10) , the corresponding Fisher ratio for can be deduced as
By comparing Eq. (10) and Eq. (7), it can be seen in Eq. (10) that the term, which does not depend on t, is eliminated through illumination compensation described in Eq. (8), and thus the dependence of and on t is increased.
For optimizing the contrast, our goal is to maximize the Fisher ratio, which depends on the mean and the variance of the light intensity in the scene. According to the analysis above, if the processing given by Eq. (8) is performed on the polarimetric scalar images, the variance of the intensity can be decreased and thus the contrast can be enhanced. In addition, the processing given by Eq. (8) leads to a new Fisher ratio function given by Eq. (11) and thus the potentiality of a further contrast optimization. In order to demonstrate the effect of illumination compensation in the contrast enhancement, we take one example based on numerical simulation and another one based on real-world polarimetric images in the following.
3. Numerical simulation
In our numerical simulation, we model the scene according to Eq. (1) and Eq. (3). The illumination beam is the unpolarized natural light, and thus the Stokes vector of the scattered light S u is equal to the first column of the Mueller matrix Mu. The mean of the Stokes vectors of regions a and b are assumed to be
We assume each coefficient of the Mueller matrix Mu randomly fluctuates with the variance of 0.015 and the variance of the addictive noise is set to be 0.005. In addition, in order to simulate the uneven illumination, we assume that the illumination on the scene is spatially fluctuated with the variance of 0.06, and the corresponding spatial intensity distribution of the illumination light on the scene is shown in Fig. 1 . In such case, the spatial fluctuation of light intensity iu induced by uneven illumination is dominant compared to those induced by the nonuniformity of Mu and the noise.
In order to perform the optimization of the Fisher ratio and the illumination compensation given by Eq. (8), one needs to know the ensemble averages and the variances of the Stokes vectors in regions a and b, and therefore, the boundary between regions a and b has to be identified in advance. Figure 2(a) shows the measured Stokes vectors of the scene, and it can be seen that the boundary between regions a and b is hard to be identified due to the significant uneven illumination. However, by normalizing the Stokes vectors, the influence of the uneven illumination can be considerably decreased, and the boundary between regions a and b becomes clear for some elements of the Stokes vector, as shown in Fig. 2(b). By employing the image segmentation method of GrabCut [11], the boundary between regions a and b can be identified based on the normalized Stokes vector shown in Fig. 2(b). According to the measured Stokes vectors of the scene as well as the identified boundary between regions a and b, the ensemble averages and variances of the Stokes vectors in regions a and b can be consequently calculated.
Let us first consider the case where the uneven illumination is not compensated in the optimization process, which is the case reported in [5,8,12 ]. In this case, the variance of given by is a constant with great value. The optimal state of t maximizing the Fisher ratio are found to be (α = 43°, ε = 6°), where α and ε denotes the azimuth and ellipticity of t respectively. The obtained image is shown in Fig. 3(a) , in which the Fisher ratio is = 0.44. It can be seen that the object is hard to be discriminated in Fig. 3(a).
Let us now consider the case where the uneven illumination has been compensated by performing the processing given by Eq. (8). In this case, we can obtain a much more clear image with Fisher ratio of = 32.88 by illumination compensation while keeping the state of PSA unchanged [see Fig. 3(b)]. Furthermore, if we optimize the state of PSA after the illumination compensation, the Fisher ratio can be further improved to be = 40.60, as shown in Fig. 3(c), and the corresponding optimal state of PSA is found to be (α = −41°, ε = −18°).
It is noticed that the optimal states of t for in Fig. 3(b) and 3(c) are different. By comparing Eq. (7) and Eq. (10), it can be seen that the relation between variable t and the variance of the intensity are different due to the illumination compensation, and thus the optimization of should lead to different optimal states of t. The result in Fig. 3 indicates that the illumination compensation not only enhances the contrast, but also generates the capacity of further contrast optimization.
Besides, it can be seen from Eq. (10) that the variance of partially depends on , and consequently, the noise term could influence the maximum value of Fisher ratio that can be achieved after illumination compensation. However, it is noticed in Eq. (9) that does not depend on the state of PSA t. Therefore, the optimal state of PSA for maximal Fisher ratio does not depend on the noise term.
4. Real-word experiment
Let us now consider a real-world polarimetric imaging scenario. The PSA consists of a rotating quarter wave plate and a rotating linear polarizer. By adjusting the orientations of the polarizer and the wave plate, the azimuth and ellipticity of t can be modulated. The target region is constituted by a piece of translucent adhesive tape, on the backside of which another piece of translucent adhesive tape has been stuck on a polarizer [see Fig. 4(a) ]. The mask between the LED light source and the scene is a piece of PMMA mask with significant spatial transmittance fluctuations. The standard intensity image of this scene is shown in Fig. 4(b), in which the adhesive tape is hard to be discriminated.
The Stokes vectors of all pixels can be calculated based on four intensity measurements at four different configurations of the PSA states [13,14 ], as shown in Fig. 5(a) , and it can be seen that the boundary between the target and the background is hard to be identified. While by normalizing the Stokes vectors, the boundary between the target and the background becomes clear, as shown in Fig. 5(b), which can be identified by the image segmentation method of GrabCut. Consequently, based on the measured Stokes vectors of the scene and the identified boundary between the target and the background, the means and variances of the Stokes vectors in region a (target) and b (background) can be estimated. Based on the estimated Stokes vectors of all the pixels, we first search the optimal state of PSA that maximizes the Fisher ratio without illumination compensation, which is found to be (α = −11°, ε = 24°). The corresponding image is shown in Fig. 6(a) . Compared with the intensity image in Fig. 4(b), the Fisher ratio of the polarimetric scalar image in Fig. 6(a) is slightly increased from 0.31 to 0.75. However, it is still difficult to discriminate the adhesive tape even at the optimal state of t without illumination compensation. If we compensate the illumination while keeping the state of PSA unchanged, the Fisher ratio can be significant increased to as high as 17.44 [see Fig. 6(b)], and it is easy to discriminate the target from the background.
In order to further enhance the contrast of the image shown in Fig. 6(b), we perform an exhaustive search to find the optimal PSA settings that maximize the contrast of the illumination compensated image. The search was performed with a resolution step of 1 degree for the azimuth and ellipticity of t. We have represented in Fig. 7 the Fisher ratio as a function of azimuth and ellipticity when optimizing t with and without illumination compensation. Comparing Fig. 7 (a) and 7(b), it is found that the optimal states of t are different. In addition, the order of the value of Fisher ratio with illumination compensation [see Fig. 7(b)] is much greater than that without illumination compensation [see Fig. 7(a)], and the maximum value of Fisher ratio in Fig. 7(b) is much greater than that in Fig. 7(b). In addition, it can be seen that the rate of variation of in Fig. 7(b) is greater than that in Fig. 7(a), which indicates that the Fisher ratio is more sensitive to t after illumination compensation.
Therefore, if we compensate the illumination and obtaining the scalar image with the optimal state of PSA in Fig. 7(b) (α = 35°, ε = −32°), we can further enhance the contrast between the target and background to = 22.07. The corresponding image is shown in Fig. 8(b) . This is a clear illustration that by optimization t after the compensation of illumination, the contrast can be further improved. In addition, comparing the parameters of the two images presented in the right part of Fig. 8, it can be seen that with the illumination compensation, the improvement of Fisher ratio is mainly attributed to the decrease of the variances of the intensities in the object region and background region.
5. Conclusion
In conclusion, we have shown that a significant spatial illumination fluctuation can disturb the target discrimination in the polarimetric scalar image, and it is found that the light intensity information contained in the Stokes vector can be utilized for further improving the contrast of the polarimetric image. Based on this point, we propose a methodology of illumination compensation and contrast optimization, which addresses successfully the issue of the spatial illumination fluctuations. The results of numerical simulation and real-word experiment in this paper show that it is possible to improve and further optimize the contrast of the image by illmunation compensation to make the object easy to be discriminated in the polarimetric image. The method proposed in this paper is particularly effective in the cases with great spatial illumination fluctuations, such as remote sensing scenarios.
This work opens up perspectives for further contrast enhancement of the polarimetric imaging with the proper processing by taking full advantage of the obtained polarization information, and we believe different processings should be performed under different imaging conditions. We are currently investigating the proper approaches for the polarimetric scalar images under other conditions, such as polarimetric imaging of objects in scattering media.
Acknowledgments
This work is supported by the National Natural Science Foundation of China (No. 61405140), the National Instrumentation Program (No. 2013YQ030915), and the Natural Science Foundation of Tianjin (No.15JCQNJC02000). Haofeng Hu acknowledges the Fondation Franco-Chinoise pour la Science et ses Applications (FFCSA) and the China Scholarship Council (CSC).
References and links
1. L. B. Wolff, Polarization Methods in Computer Vision (Columbia University, 1991).
2. J. S. Tyo, D. L. Goldstein, D. B. Chenault, and J. A. Shaw, “Review of passive imaging polarimetry for remote sensing applications,” Appl. Opt. 45(22), 5453–5469 (2006). [CrossRef] [PubMed]
3. A. Pierangelo, A. Benali, M.-R. Antonelli, T. Novikova, P. Validire, B. Gayet, and A. De Martino, “Ex-vivo characterization of human colon cancer by Mueller polarimetric imaging,” Opt. Express 19(2), 1582–1593 (2011). [CrossRef] [PubMed]
4. F. Meriaudeau, M. Ferraton, C. Stolz, O. Morel, and L. Bigué, “Polarization imaging for industrial inspection,” Proc. SPIE 6813, 681308 (2008). [CrossRef]
5. G. Anna, F. Goudail, and D. Dolfi, “Polarimetric target detection in the presence of spatially fluctuating Mueller matrices,” Opt. Lett. 36(23), 4590–4592 (2011). [CrossRef] [PubMed]
6. F. Goudail and A. Bénière, “Optimization of the contrast in polarimetric scalar images,” Opt. Lett. 34(9), 1471–1473 (2009). [CrossRef] [PubMed]
7. F. Goudail, “Optimization of the contrast in active Stokes images,” Opt. Lett. 34(2), 121–123 (2009). [CrossRef] [PubMed]
8. G. Anna, F. Goudail, and D. Dolfi, “Optimal discrimination of multiple regions with an active polarimetric imager,” Opt. Express 19(25), 25367–25378 (2011). [CrossRef] [PubMed]
9. T. Lillesand, R. W. Kiefer, and J. Chipman, Remote Sensing and Image Interpretation (John Wiley & Sons, 2014).
10. K. Fukunaga, Introduction to Statistical Pattern Recognition (Academic, 2013).
11. C. Rother, V. Kolmogorov, and A. Blake, ““GrabCut”-interactive foreground extraction using iterated graph cuts,” ACM Trans. Graph. 23(3), 309–314 (2004). [CrossRef]
12. M. Boffety, H. Hu, and F. Goudail, “Contrast optimization in broadband passive polarimetric imaging,” Opt. Lett. 39(23), 6759–6762 (2014). [CrossRef] [PubMed]
13. H. Hu, G. Anna, and F. Goudail, “On the performance of the physicality-constrained maximum-likelihood estimation of Stokes vector,” Appl. Opt. 52(27), 6636–6644 (2013). [CrossRef] [PubMed]
14. L. Goldstein Dennis, Polarized Light (Marcel Dekker, 2003).