Abstract

Long-wave infrared (LWIR) imaging has been successfully used in surveillance applications in low illumination conditions. However, infrared energy reflected from smooth surfaces such as floors and metallic objects may reduce object detection and tracking accuracies. In this paper, we present a novel reflection removal method using polarization properties of the reflection in LWIR imagery. Reflection can be distinguished from the scene by two unique characteristics of polarization: the difference of two orthogonal polarized components (OPC) and the uniformity of angle of polarization (AoP). The OPC difference helps locate the regions of reflection. The uniformity of AoP in the reflection region pose a strong constraint for reflection detection. The proposed joint reflection detection method combines the OPC difference and the uniformity of AoP can detect actual reflection region. Then the closed-form matting method improves the robustness of the method and removes the reflection from the scene. Experiment results demonstrate that the proposed scheme effectively removes the reflection in challenging situations where many existing techniques may fail.

© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Full Article  |  PDF Article
OSA Recommended Articles
Multiscale contrast direction adaptive image fusion technique for MWIR-LWIR image pairs and LWIR multifocus infrared images

A. Onur Karali, Serdar Cakir, and Tayfun Aytaç
Appl. Opt. 54(13) 4172-4179 (2015)

The effects of thermal equilibrium and contrast in LWIR polarimetric images

J. Scott Tyo, Bradley M. Ratliff, James K. Boger, Wiley T. Black, David L. Bowers, and Matthew P. Fetrow
Opt. Express 15(23) 15161-15167 (2007)

Real-time robust direct and indirect photon separation with polarization imaging

Yitian Ding, Amit Ashok, and Stanley Pau
Opt. Express 25(23) 29432-29453 (2017)

References

  • View by:
  • |
  • |
  • |

  1. C. Dai, Y. Zheng, and X. Li, “Layered representation for pedestrian detection and tracking in infrared imagery,” Comput. Vis. Image Underst. 106, 288–299 (2007).
    [Crossref]
  2. Z. Li, Q. Wu, J. Zhang, and G. Geers, “SKRWM based descriptor for pedestrian detection in thermal images, ” in Proceedings of IEEE Workshop on Multimedia Signal Processing (IEEE, 2011), pp. 1–6.
    [Crossref]
  3. H. Torresan, B. Turgeon, C. Ibarra-Castanedo, P. Hebert, and X. P. Maldague, “Advanced surveillance systems: combining video and thermal imagery for pedestrian detection,” Proc. SPIE 5405, 506–516 (2004).
    [Crossref]
  4. Y. Schechner, J. Shamir, and N. Kiryuati, “Polarization-based decorrelation of transparent layers: the inclination angle of an invisible surface,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 1999), pp. 814–819.
    [Crossref]
  5. H. Farid and E. H. Adelson, “Separating reflections from images by use of independent component analysis,” J. Opt. Soc. Am. A 16(9), 2136–2145 (1999).
    [Crossref] [PubMed]
  6. A. M. Bronstein, M. M. Bronstein, M. Zibulevsky, and Y. Y. Zeevi, “Sparse ICA for blind separation of transmitted and reflected images,” Int. J. Imaging Syst. Technol. 15(1), 84–91 (2005).
    [Crossref]
  7. R. Wan, B. Shi, T. A. Hwee, and A. C. Kot, “Depth of field guided reflection removal,” in Proceedings of the IEEE International Conference on Image Processing (IEEE, 2016), pp. 21–25.
  8. T. Sirinukulwattana, G. Choe, and I. S. Kweon, “Reflection removal using disparity and gradient-sparsity via smoothing algorithm,” in Proceedings of the IEEE International Conference on Image Processing (IEEE, 2015), pp. 1940–1944.
    [Crossref]
  9. Y. Li and M. S. Brown, “Exploiting reflection change for automatic reflection removal,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 2013), pp. 2432–2439.
    [Crossref]
  10. A. Teschioni and C. S. Regazzoni, “A robust method for reflections analysis in color image sequences,” in Signal Processing Conference (EUSIPCO 1998) (IEEE, 1998), pp. 1–4.
  11. E. J. Carmona, J. Martínez-Cantos, and J. Mira, “A new video segmentation method of moving objects based on blob-level knowledge,” Pattern Recognit. Lett. 29(3), 272–285 (2008).
    [Crossref]
  12. D. Conte, P. Foggia, G. Percannella, F. Tufano, and M. Vento, “Reflection removal in colour videos,” in Proceedings of IEEE International Conference on Pattern Recognition (IEEE, 2010), pp. 1788–1791.
  13. T. Zhao and R. Nevatia, “Tracking multiple humans in complex situations,” IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1208–1221 (2004).
    [Crossref] [PubMed]
  14. D. Conte, P. Foggia, G. Percannella, and M. Vento, “Removing object reflections in videos by global optimization,” IEEE Trans. Circ. Syst. Video Tech. 22(11), 1623–1633 (2012).
    [Crossref]
  15. M. Karaman, L. Goldmann, and T. Sikora, “Improving object segmentation by reflection detection and removal,” Proc. SPIE 7257, 725709 (2009).
    [Crossref]
  16. S. V. U. Ha, N. T. Pham, L. H. Pham, and H. M. Tran, “Robust Reflection Detection and Removal in Rainy Conditions using LAB and HSV Color Spaces,” REV J. Electron. Commun. 6, 13–19 (2016).
  17. Y. Ding, A. Ashok, and S. Pau, “Real-time robust direct and indirect photon separation with polarization imaging,” Opt. Express 25(23), 29432–29453 (2017).
    [Crossref]
  18. D. H. Goldstein, Polarized Light, 2nd ed. (Marcel Dekker, Inc, 2003).
  19. J. S. Tyo, B. M. Ratliff, J. K. Boger, W. T. Black, D. L. Bowers, and M. P. Fetrow, “The effects of thermal equilibrium and contrast in LWIR polarimetric images,” Opt. Express 15(23), 15161–15167 (2007).
    [Crossref] [PubMed]
  20. T. J. Rogne, F. G. Smith, and J. E. Rice, “Passive target detection using polarized components of infrared signatures,” Proc. SPIE 1317, 242–252 (1990).
    [Crossref]
  21. E. Hecht, Optics, 4th ed. (Pearson Education, 2002).
  22. N. Kong, Y. W. Tai, and J. S. Shin, “A physically-based approach to reflection separation: from physical modeling to constrained optimization,” IEEE Trans. Pattern Anal. Mach. Intell. 36(2), 209–221 (2014).
    [Crossref] [PubMed]
  23. L. Zhang, Q. Zhang, and C. Xiao, “Shadow remover: Image shadow removal based on illumination recovering optimization,” IEEE Trans. Image Process. 24(11), 4623–4636 (2015).
    [Crossref] [PubMed]
  24. A. Levin, D. Lischinski, and Y. Weiss, “A closed-form solution to natural image matting,” IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 228–242 (2008).
    [Crossref] [PubMed]
  25. Y. Zhao, Multi-band Polarization Imaging and Applications. (Springer, 2016).
  26. V. Gruev, R. Perkins, and T. York, “CCD polarization imaging sensor with aluminum nanowire optical filters,” Opt. Express 18(18), 19087–19094 (2010).
    [Crossref] [PubMed]
  27. C. Xu, J. Ma, C. Ke, Y. Huang, Z. Zeng, and W. Weng, “Numerical study of a DoFP polarimeter based on the self-organized nanograting array,” Opt. Express 26(3), 2517–2527 (2018).
    [Crossref] [PubMed]

2018 (1)

C. Xu, J. Ma, C. Ke, Y. Huang, Z. Zeng, and W. Weng, “Numerical study of a DoFP polarimeter based on the self-organized nanograting array,” Opt. Express 26(3), 2517–2527 (2018).
[Crossref] [PubMed]

2017 (1)

Y. Ding, A. Ashok, and S. Pau, “Real-time robust direct and indirect photon separation with polarization imaging,” Opt. Express 25(23), 29432–29453 (2017).
[Crossref]

2016 (1)

S. V. U. Ha, N. T. Pham, L. H. Pham, and H. M. Tran, “Robust Reflection Detection and Removal in Rainy Conditions using LAB and HSV Color Spaces,” REV J. Electron. Commun. 6, 13–19 (2016).

2015 (1)

L. Zhang, Q. Zhang, and C. Xiao, “Shadow remover: Image shadow removal based on illumination recovering optimization,” IEEE Trans. Image Process. 24(11), 4623–4636 (2015).
[Crossref] [PubMed]

2014 (1)

N. Kong, Y. W. Tai, and J. S. Shin, “A physically-based approach to reflection separation: from physical modeling to constrained optimization,” IEEE Trans. Pattern Anal. Mach. Intell. 36(2), 209–221 (2014).
[Crossref] [PubMed]

2012 (1)

D. Conte, P. Foggia, G. Percannella, and M. Vento, “Removing object reflections in videos by global optimization,” IEEE Trans. Circ. Syst. Video Tech. 22(11), 1623–1633 (2012).
[Crossref]

2010 (1)

V. Gruev, R. Perkins, and T. York, “CCD polarization imaging sensor with aluminum nanowire optical filters,” Opt. Express 18(18), 19087–19094 (2010).
[Crossref] [PubMed]

2009 (1)

M. Karaman, L. Goldmann, and T. Sikora, “Improving object segmentation by reflection detection and removal,” Proc. SPIE 7257, 725709 (2009).
[Crossref]

2008 (2)

E. J. Carmona, J. Martínez-Cantos, and J. Mira, “A new video segmentation method of moving objects based on blob-level knowledge,” Pattern Recognit. Lett. 29(3), 272–285 (2008).
[Crossref]

A. Levin, D. Lischinski, and Y. Weiss, “A closed-form solution to natural image matting,” IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 228–242 (2008).
[Crossref] [PubMed]

2007 (2)

J. S. Tyo, B. M. Ratliff, J. K. Boger, W. T. Black, D. L. Bowers, and M. P. Fetrow, “The effects of thermal equilibrium and contrast in LWIR polarimetric images,” Opt. Express 15(23), 15161–15167 (2007).
[Crossref] [PubMed]

C. Dai, Y. Zheng, and X. Li, “Layered representation for pedestrian detection and tracking in infrared imagery,” Comput. Vis. Image Underst. 106, 288–299 (2007).
[Crossref]

2005 (1)

A. M. Bronstein, M. M. Bronstein, M. Zibulevsky, and Y. Y. Zeevi, “Sparse ICA for blind separation of transmitted and reflected images,” Int. J. Imaging Syst. Technol. 15(1), 84–91 (2005).
[Crossref]

2004 (2)

H. Torresan, B. Turgeon, C. Ibarra-Castanedo, P. Hebert, and X. P. Maldague, “Advanced surveillance systems: combining video and thermal imagery for pedestrian detection,” Proc. SPIE 5405, 506–516 (2004).
[Crossref]

T. Zhao and R. Nevatia, “Tracking multiple humans in complex situations,” IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1208–1221 (2004).
[Crossref] [PubMed]

1999 (1)

H. Farid and E. H. Adelson, “Separating reflections from images by use of independent component analysis,” J. Opt. Soc. Am. A 16(9), 2136–2145 (1999).
[Crossref] [PubMed]

1990 (1)

T. J. Rogne, F. G. Smith, and J. E. Rice, “Passive target detection using polarized components of infrared signatures,” Proc. SPIE 1317, 242–252 (1990).
[Crossref]

Adelson, E. H.

H. Farid and E. H. Adelson, “Separating reflections from images by use of independent component analysis,” J. Opt. Soc. Am. A 16(9), 2136–2145 (1999).
[Crossref] [PubMed]

Ashok, A.

Y. Ding, A. Ashok, and S. Pau, “Real-time robust direct and indirect photon separation with polarization imaging,” Opt. Express 25(23), 29432–29453 (2017).
[Crossref]

Black, W. T.

J. S. Tyo, B. M. Ratliff, J. K. Boger, W. T. Black, D. L. Bowers, and M. P. Fetrow, “The effects of thermal equilibrium and contrast in LWIR polarimetric images,” Opt. Express 15(23), 15161–15167 (2007).
[Crossref] [PubMed]

Boger, J. K.

J. S. Tyo, B. M. Ratliff, J. K. Boger, W. T. Black, D. L. Bowers, and M. P. Fetrow, “The effects of thermal equilibrium and contrast in LWIR polarimetric images,” Opt. Express 15(23), 15161–15167 (2007).
[Crossref] [PubMed]

Bowers, D. L.

J. S. Tyo, B. M. Ratliff, J. K. Boger, W. T. Black, D. L. Bowers, and M. P. Fetrow, “The effects of thermal equilibrium and contrast in LWIR polarimetric images,” Opt. Express 15(23), 15161–15167 (2007).
[Crossref] [PubMed]

Bronstein, A. M.

A. M. Bronstein, M. M. Bronstein, M. Zibulevsky, and Y. Y. Zeevi, “Sparse ICA for blind separation of transmitted and reflected images,” Int. J. Imaging Syst. Technol. 15(1), 84–91 (2005).
[Crossref]

Bronstein, M. M.

A. M. Bronstein, M. M. Bronstein, M. Zibulevsky, and Y. Y. Zeevi, “Sparse ICA for blind separation of transmitted and reflected images,” Int. J. Imaging Syst. Technol. 15(1), 84–91 (2005).
[Crossref]

Brown, M. S.

Y. Li and M. S. Brown, “Exploiting reflection change for automatic reflection removal,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 2013), pp. 2432–2439.
[Crossref]

Carmona, E. J.

E. J. Carmona, J. Martínez-Cantos, and J. Mira, “A new video segmentation method of moving objects based on blob-level knowledge,” Pattern Recognit. Lett. 29(3), 272–285 (2008).
[Crossref]

Choe, G.

T. Sirinukulwattana, G. Choe, and I. S. Kweon, “Reflection removal using disparity and gradient-sparsity via smoothing algorithm,” in Proceedings of the IEEE International Conference on Image Processing (IEEE, 2015), pp. 1940–1944.
[Crossref]

Conte, D.

D. Conte, P. Foggia, G. Percannella, and M. Vento, “Removing object reflections in videos by global optimization,” IEEE Trans. Circ. Syst. Video Tech. 22(11), 1623–1633 (2012).
[Crossref]

D. Conte, P. Foggia, G. Percannella, F. Tufano, and M. Vento, “Reflection removal in colour videos,” in Proceedings of IEEE International Conference on Pattern Recognition (IEEE, 2010), pp. 1788–1791.

Dai, C.

C. Dai, Y. Zheng, and X. Li, “Layered representation for pedestrian detection and tracking in infrared imagery,” Comput. Vis. Image Underst. 106, 288–299 (2007).
[Crossref]

Ding, Y.

Y. Ding, A. Ashok, and S. Pau, “Real-time robust direct and indirect photon separation with polarization imaging,” Opt. Express 25(23), 29432–29453 (2017).
[Crossref]

Farid, H.

H. Farid and E. H. Adelson, “Separating reflections from images by use of independent component analysis,” J. Opt. Soc. Am. A 16(9), 2136–2145 (1999).
[Crossref] [PubMed]

Fetrow, M. P.

J. S. Tyo, B. M. Ratliff, J. K. Boger, W. T. Black, D. L. Bowers, and M. P. Fetrow, “The effects of thermal equilibrium and contrast in LWIR polarimetric images,” Opt. Express 15(23), 15161–15167 (2007).
[Crossref] [PubMed]

Foggia, P.

D. Conte, P. Foggia, G. Percannella, and M. Vento, “Removing object reflections in videos by global optimization,” IEEE Trans. Circ. Syst. Video Tech. 22(11), 1623–1633 (2012).
[Crossref]

D. Conte, P. Foggia, G. Percannella, F. Tufano, and M. Vento, “Reflection removal in colour videos,” in Proceedings of IEEE International Conference on Pattern Recognition (IEEE, 2010), pp. 1788–1791.

Geers, G.

Z. Li, Q. Wu, J. Zhang, and G. Geers, “SKRWM based descriptor for pedestrian detection in thermal images, ” in Proceedings of IEEE Workshop on Multimedia Signal Processing (IEEE, 2011), pp. 1–6.
[Crossref]

Goldmann, L.

M. Karaman, L. Goldmann, and T. Sikora, “Improving object segmentation by reflection detection and removal,” Proc. SPIE 7257, 725709 (2009).
[Crossref]

Gruev, V.

V. Gruev, R. Perkins, and T. York, “CCD polarization imaging sensor with aluminum nanowire optical filters,” Opt. Express 18(18), 19087–19094 (2010).
[Crossref] [PubMed]

Ha, S. V. U.

S. V. U. Ha, N. T. Pham, L. H. Pham, and H. M. Tran, “Robust Reflection Detection and Removal in Rainy Conditions using LAB and HSV Color Spaces,” REV J. Electron. Commun. 6, 13–19 (2016).

Hebert, P.

H. Torresan, B. Turgeon, C. Ibarra-Castanedo, P. Hebert, and X. P. Maldague, “Advanced surveillance systems: combining video and thermal imagery for pedestrian detection,” Proc. SPIE 5405, 506–516 (2004).
[Crossref]

Huang, Y.

C. Xu, J. Ma, C. Ke, Y. Huang, Z. Zeng, and W. Weng, “Numerical study of a DoFP polarimeter based on the self-organized nanograting array,” Opt. Express 26(3), 2517–2527 (2018).
[Crossref] [PubMed]

Hwee, T. A.

R. Wan, B. Shi, T. A. Hwee, and A. C. Kot, “Depth of field guided reflection removal,” in Proceedings of the IEEE International Conference on Image Processing (IEEE, 2016), pp. 21–25.

Ibarra-Castanedo, C.

H. Torresan, B. Turgeon, C. Ibarra-Castanedo, P. Hebert, and X. P. Maldague, “Advanced surveillance systems: combining video and thermal imagery for pedestrian detection,” Proc. SPIE 5405, 506–516 (2004).
[Crossref]

Karaman, M.

M. Karaman, L. Goldmann, and T. Sikora, “Improving object segmentation by reflection detection and removal,” Proc. SPIE 7257, 725709 (2009).
[Crossref]

Ke, C.

C. Xu, J. Ma, C. Ke, Y. Huang, Z. Zeng, and W. Weng, “Numerical study of a DoFP polarimeter based on the self-organized nanograting array,” Opt. Express 26(3), 2517–2527 (2018).
[Crossref] [PubMed]

Kiryuati, N.

Y. Schechner, J. Shamir, and N. Kiryuati, “Polarization-based decorrelation of transparent layers: the inclination angle of an invisible surface,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 1999), pp. 814–819.
[Crossref]

Kong, N.

N. Kong, Y. W. Tai, and J. S. Shin, “A physically-based approach to reflection separation: from physical modeling to constrained optimization,” IEEE Trans. Pattern Anal. Mach. Intell. 36(2), 209–221 (2014).
[Crossref] [PubMed]

Kot, A. C.

R. Wan, B. Shi, T. A. Hwee, and A. C. Kot, “Depth of field guided reflection removal,” in Proceedings of the IEEE International Conference on Image Processing (IEEE, 2016), pp. 21–25.

Kweon, I. S.

T. Sirinukulwattana, G. Choe, and I. S. Kweon, “Reflection removal using disparity and gradient-sparsity via smoothing algorithm,” in Proceedings of the IEEE International Conference on Image Processing (IEEE, 2015), pp. 1940–1944.
[Crossref]

Levin, A.

A. Levin, D. Lischinski, and Y. Weiss, “A closed-form solution to natural image matting,” IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 228–242 (2008).
[Crossref] [PubMed]

Li, X.

C. Dai, Y. Zheng, and X. Li, “Layered representation for pedestrian detection and tracking in infrared imagery,” Comput. Vis. Image Underst. 106, 288–299 (2007).
[Crossref]

Li, Y.

Y. Li and M. S. Brown, “Exploiting reflection change for automatic reflection removal,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 2013), pp. 2432–2439.
[Crossref]

Li, Z.

Z. Li, Q. Wu, J. Zhang, and G. Geers, “SKRWM based descriptor for pedestrian detection in thermal images, ” in Proceedings of IEEE Workshop on Multimedia Signal Processing (IEEE, 2011), pp. 1–6.
[Crossref]

Lischinski, D.

A. Levin, D. Lischinski, and Y. Weiss, “A closed-form solution to natural image matting,” IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 228–242 (2008).
[Crossref] [PubMed]

Ma, J.

C. Xu, J. Ma, C. Ke, Y. Huang, Z. Zeng, and W. Weng, “Numerical study of a DoFP polarimeter based on the self-organized nanograting array,” Opt. Express 26(3), 2517–2527 (2018).
[Crossref] [PubMed]

Maldague, X. P.

H. Torresan, B. Turgeon, C. Ibarra-Castanedo, P. Hebert, and X. P. Maldague, “Advanced surveillance systems: combining video and thermal imagery for pedestrian detection,” Proc. SPIE 5405, 506–516 (2004).
[Crossref]

Martínez-Cantos, J.

E. J. Carmona, J. Martínez-Cantos, and J. Mira, “A new video segmentation method of moving objects based on blob-level knowledge,” Pattern Recognit. Lett. 29(3), 272–285 (2008).
[Crossref]

Mira, J.

E. J. Carmona, J. Martínez-Cantos, and J. Mira, “A new video segmentation method of moving objects based on blob-level knowledge,” Pattern Recognit. Lett. 29(3), 272–285 (2008).
[Crossref]

Nevatia, R.

T. Zhao and R. Nevatia, “Tracking multiple humans in complex situations,” IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1208–1221 (2004).
[Crossref] [PubMed]

Pau, S.

Y. Ding, A. Ashok, and S. Pau, “Real-time robust direct and indirect photon separation with polarization imaging,” Opt. Express 25(23), 29432–29453 (2017).
[Crossref]

Percannella, G.

D. Conte, P. Foggia, G. Percannella, and M. Vento, “Removing object reflections in videos by global optimization,” IEEE Trans. Circ. Syst. Video Tech. 22(11), 1623–1633 (2012).
[Crossref]

D. Conte, P. Foggia, G. Percannella, F. Tufano, and M. Vento, “Reflection removal in colour videos,” in Proceedings of IEEE International Conference on Pattern Recognition (IEEE, 2010), pp. 1788–1791.

Perkins, R.

V. Gruev, R. Perkins, and T. York, “CCD polarization imaging sensor with aluminum nanowire optical filters,” Opt. Express 18(18), 19087–19094 (2010).
[Crossref] [PubMed]

Pham, L. H.

S. V. U. Ha, N. T. Pham, L. H. Pham, and H. M. Tran, “Robust Reflection Detection and Removal in Rainy Conditions using LAB and HSV Color Spaces,” REV J. Electron. Commun. 6, 13–19 (2016).

Pham, N. T.

S. V. U. Ha, N. T. Pham, L. H. Pham, and H. M. Tran, “Robust Reflection Detection and Removal in Rainy Conditions using LAB and HSV Color Spaces,” REV J. Electron. Commun. 6, 13–19 (2016).

Ratliff, B. M.

J. S. Tyo, B. M. Ratliff, J. K. Boger, W. T. Black, D. L. Bowers, and M. P. Fetrow, “The effects of thermal equilibrium and contrast in LWIR polarimetric images,” Opt. Express 15(23), 15161–15167 (2007).
[Crossref] [PubMed]

Regazzoni, C. S.

A. Teschioni and C. S. Regazzoni, “A robust method for reflections analysis in color image sequences,” in Signal Processing Conference (EUSIPCO 1998) (IEEE, 1998), pp. 1–4.

Rice, J. E.

T. J. Rogne, F. G. Smith, and J. E. Rice, “Passive target detection using polarized components of infrared signatures,” Proc. SPIE 1317, 242–252 (1990).
[Crossref]

Rogne, T. J.

T. J. Rogne, F. G. Smith, and J. E. Rice, “Passive target detection using polarized components of infrared signatures,” Proc. SPIE 1317, 242–252 (1990).
[Crossref]

Schechner, Y.

Y. Schechner, J. Shamir, and N. Kiryuati, “Polarization-based decorrelation of transparent layers: the inclination angle of an invisible surface,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 1999), pp. 814–819.
[Crossref]

Shamir, J.

Y. Schechner, J. Shamir, and N. Kiryuati, “Polarization-based decorrelation of transparent layers: the inclination angle of an invisible surface,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 1999), pp. 814–819.
[Crossref]

Shi, B.

R. Wan, B. Shi, T. A. Hwee, and A. C. Kot, “Depth of field guided reflection removal,” in Proceedings of the IEEE International Conference on Image Processing (IEEE, 2016), pp. 21–25.

Shin, J. S.

N. Kong, Y. W. Tai, and J. S. Shin, “A physically-based approach to reflection separation: from physical modeling to constrained optimization,” IEEE Trans. Pattern Anal. Mach. Intell. 36(2), 209–221 (2014).
[Crossref] [PubMed]

Sikora, T.

M. Karaman, L. Goldmann, and T. Sikora, “Improving object segmentation by reflection detection and removal,” Proc. SPIE 7257, 725709 (2009).
[Crossref]

Sirinukulwattana, T.

T. Sirinukulwattana, G. Choe, and I. S. Kweon, “Reflection removal using disparity and gradient-sparsity via smoothing algorithm,” in Proceedings of the IEEE International Conference on Image Processing (IEEE, 2015), pp. 1940–1944.
[Crossref]

Smith, F. G.

T. J. Rogne, F. G. Smith, and J. E. Rice, “Passive target detection using polarized components of infrared signatures,” Proc. SPIE 1317, 242–252 (1990).
[Crossref]

Tai, Y. W.

N. Kong, Y. W. Tai, and J. S. Shin, “A physically-based approach to reflection separation: from physical modeling to constrained optimization,” IEEE Trans. Pattern Anal. Mach. Intell. 36(2), 209–221 (2014).
[Crossref] [PubMed]

Teschioni, A.

A. Teschioni and C. S. Regazzoni, “A robust method for reflections analysis in color image sequences,” in Signal Processing Conference (EUSIPCO 1998) (IEEE, 1998), pp. 1–4.

Torresan, H.

H. Torresan, B. Turgeon, C. Ibarra-Castanedo, P. Hebert, and X. P. Maldague, “Advanced surveillance systems: combining video and thermal imagery for pedestrian detection,” Proc. SPIE 5405, 506–516 (2004).
[Crossref]

Tran, H. M.

S. V. U. Ha, N. T. Pham, L. H. Pham, and H. M. Tran, “Robust Reflection Detection and Removal in Rainy Conditions using LAB and HSV Color Spaces,” REV J. Electron. Commun. 6, 13–19 (2016).

Tufano, F.

D. Conte, P. Foggia, G. Percannella, F. Tufano, and M. Vento, “Reflection removal in colour videos,” in Proceedings of IEEE International Conference on Pattern Recognition (IEEE, 2010), pp. 1788–1791.

Turgeon, B.

H. Torresan, B. Turgeon, C. Ibarra-Castanedo, P. Hebert, and X. P. Maldague, “Advanced surveillance systems: combining video and thermal imagery for pedestrian detection,” Proc. SPIE 5405, 506–516 (2004).
[Crossref]

Tyo, J. S.

J. S. Tyo, B. M. Ratliff, J. K. Boger, W. T. Black, D. L. Bowers, and M. P. Fetrow, “The effects of thermal equilibrium and contrast in LWIR polarimetric images,” Opt. Express 15(23), 15161–15167 (2007).
[Crossref] [PubMed]

Vento, M.

D. Conte, P. Foggia, G. Percannella, and M. Vento, “Removing object reflections in videos by global optimization,” IEEE Trans. Circ. Syst. Video Tech. 22(11), 1623–1633 (2012).
[Crossref]

D. Conte, P. Foggia, G. Percannella, F. Tufano, and M. Vento, “Reflection removal in colour videos,” in Proceedings of IEEE International Conference on Pattern Recognition (IEEE, 2010), pp. 1788–1791.

Wan, R.

R. Wan, B. Shi, T. A. Hwee, and A. C. Kot, “Depth of field guided reflection removal,” in Proceedings of the IEEE International Conference on Image Processing (IEEE, 2016), pp. 21–25.

Weiss, Y.

A. Levin, D. Lischinski, and Y. Weiss, “A closed-form solution to natural image matting,” IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 228–242 (2008).
[Crossref] [PubMed]

Weng, W.

C. Xu, J. Ma, C. Ke, Y. Huang, Z. Zeng, and W. Weng, “Numerical study of a DoFP polarimeter based on the self-organized nanograting array,” Opt. Express 26(3), 2517–2527 (2018).
[Crossref] [PubMed]

Wu, Q.

Z. Li, Q. Wu, J. Zhang, and G. Geers, “SKRWM based descriptor for pedestrian detection in thermal images, ” in Proceedings of IEEE Workshop on Multimedia Signal Processing (IEEE, 2011), pp. 1–6.
[Crossref]

Xiao, C.

L. Zhang, Q. Zhang, and C. Xiao, “Shadow remover: Image shadow removal based on illumination recovering optimization,” IEEE Trans. Image Process. 24(11), 4623–4636 (2015).
[Crossref] [PubMed]

Xu, C.

C. Xu, J. Ma, C. Ke, Y. Huang, Z. Zeng, and W. Weng, “Numerical study of a DoFP polarimeter based on the self-organized nanograting array,” Opt. Express 26(3), 2517–2527 (2018).
[Crossref] [PubMed]

York, T.

V. Gruev, R. Perkins, and T. York, “CCD polarization imaging sensor with aluminum nanowire optical filters,” Opt. Express 18(18), 19087–19094 (2010).
[Crossref] [PubMed]

Zeevi, Y. Y.

A. M. Bronstein, M. M. Bronstein, M. Zibulevsky, and Y. Y. Zeevi, “Sparse ICA for blind separation of transmitted and reflected images,” Int. J. Imaging Syst. Technol. 15(1), 84–91 (2005).
[Crossref]

Zeng, Z.

C. Xu, J. Ma, C. Ke, Y. Huang, Z. Zeng, and W. Weng, “Numerical study of a DoFP polarimeter based on the self-organized nanograting array,” Opt. Express 26(3), 2517–2527 (2018).
[Crossref] [PubMed]

Zhang, J.

Z. Li, Q. Wu, J. Zhang, and G. Geers, “SKRWM based descriptor for pedestrian detection in thermal images, ” in Proceedings of IEEE Workshop on Multimedia Signal Processing (IEEE, 2011), pp. 1–6.
[Crossref]

Zhang, L.

L. Zhang, Q. Zhang, and C. Xiao, “Shadow remover: Image shadow removal based on illumination recovering optimization,” IEEE Trans. Image Process. 24(11), 4623–4636 (2015).
[Crossref] [PubMed]

Zhang, Q.

L. Zhang, Q. Zhang, and C. Xiao, “Shadow remover: Image shadow removal based on illumination recovering optimization,” IEEE Trans. Image Process. 24(11), 4623–4636 (2015).
[Crossref] [PubMed]

Zhao, T.

T. Zhao and R. Nevatia, “Tracking multiple humans in complex situations,” IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1208–1221 (2004).
[Crossref] [PubMed]

Zheng, Y.

C. Dai, Y. Zheng, and X. Li, “Layered representation for pedestrian detection and tracking in infrared imagery,” Comput. Vis. Image Underst. 106, 288–299 (2007).
[Crossref]

Zibulevsky, M.

A. M. Bronstein, M. M. Bronstein, M. Zibulevsky, and Y. Y. Zeevi, “Sparse ICA for blind separation of transmitted and reflected images,” Int. J. Imaging Syst. Technol. 15(1), 84–91 (2005).
[Crossref]

Comput. Vis. Image Underst. (1)

C. Dai, Y. Zheng, and X. Li, “Layered representation for pedestrian detection and tracking in infrared imagery,” Comput. Vis. Image Underst. 106, 288–299 (2007).
[Crossref]

IEEE Trans. Circ. Syst. Video Tech. (1)

D. Conte, P. Foggia, G. Percannella, and M. Vento, “Removing object reflections in videos by global optimization,” IEEE Trans. Circ. Syst. Video Tech. 22(11), 1623–1633 (2012).
[Crossref]

IEEE Trans. Image Process. (1)

L. Zhang, Q. Zhang, and C. Xiao, “Shadow remover: Image shadow removal based on illumination recovering optimization,” IEEE Trans. Image Process. 24(11), 4623–4636 (2015).
[Crossref] [PubMed]

IEEE Trans. Pattern Anal. Mach. Intell. (3)

A. Levin, D. Lischinski, and Y. Weiss, “A closed-form solution to natural image matting,” IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 228–242 (2008).
[Crossref] [PubMed]

N. Kong, Y. W. Tai, and J. S. Shin, “A physically-based approach to reflection separation: from physical modeling to constrained optimization,” IEEE Trans. Pattern Anal. Mach. Intell. 36(2), 209–221 (2014).
[Crossref] [PubMed]

T. Zhao and R. Nevatia, “Tracking multiple humans in complex situations,” IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1208–1221 (2004).
[Crossref] [PubMed]

Int. J. Imaging Syst. Technol. (1)

A. M. Bronstein, M. M. Bronstein, M. Zibulevsky, and Y. Y. Zeevi, “Sparse ICA for blind separation of transmitted and reflected images,” Int. J. Imaging Syst. Technol. 15(1), 84–91 (2005).
[Crossref]

J. Opt. Soc. Am. A (1)

H. Farid and E. H. Adelson, “Separating reflections from images by use of independent component analysis,” J. Opt. Soc. Am. A 16(9), 2136–2145 (1999).
[Crossref] [PubMed]

Opt. Express (4)

Y. Ding, A. Ashok, and S. Pau, “Real-time robust direct and indirect photon separation with polarization imaging,” Opt. Express 25(23), 29432–29453 (2017).
[Crossref]

V. Gruev, R. Perkins, and T. York, “CCD polarization imaging sensor with aluminum nanowire optical filters,” Opt. Express 18(18), 19087–19094 (2010).
[Crossref] [PubMed]

C. Xu, J. Ma, C. Ke, Y. Huang, Z. Zeng, and W. Weng, “Numerical study of a DoFP polarimeter based on the self-organized nanograting array,” Opt. Express 26(3), 2517–2527 (2018).
[Crossref] [PubMed]

J. S. Tyo, B. M. Ratliff, J. K. Boger, W. T. Black, D. L. Bowers, and M. P. Fetrow, “The effects of thermal equilibrium and contrast in LWIR polarimetric images,” Opt. Express 15(23), 15161–15167 (2007).
[Crossref] [PubMed]

Pattern Recognit. Lett. (1)

E. J. Carmona, J. Martínez-Cantos, and J. Mira, “A new video segmentation method of moving objects based on blob-level knowledge,” Pattern Recognit. Lett. 29(3), 272–285 (2008).
[Crossref]

Proc. SPIE (3)

M. Karaman, L. Goldmann, and T. Sikora, “Improving object segmentation by reflection detection and removal,” Proc. SPIE 7257, 725709 (2009).
[Crossref]

H. Torresan, B. Turgeon, C. Ibarra-Castanedo, P. Hebert, and X. P. Maldague, “Advanced surveillance systems: combining video and thermal imagery for pedestrian detection,” Proc. SPIE 5405, 506–516 (2004).
[Crossref]

T. J. Rogne, F. G. Smith, and J. E. Rice, “Passive target detection using polarized components of infrared signatures,” Proc. SPIE 1317, 242–252 (1990).
[Crossref]

REV J. Electron. Commun. (1)

S. V. U. Ha, N. T. Pham, L. H. Pham, and H. M. Tran, “Robust Reflection Detection and Removal in Rainy Conditions using LAB and HSV Color Spaces,” REV J. Electron. Commun. 6, 13–19 (2016).

Other (10)

D. H. Goldstein, Polarized Light, 2nd ed. (Marcel Dekker, Inc, 2003).

D. Conte, P. Foggia, G. Percannella, F. Tufano, and M. Vento, “Reflection removal in colour videos,” in Proceedings of IEEE International Conference on Pattern Recognition (IEEE, 2010), pp. 1788–1791.

Y. Schechner, J. Shamir, and N. Kiryuati, “Polarization-based decorrelation of transparent layers: the inclination angle of an invisible surface,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 1999), pp. 814–819.
[Crossref]

Z. Li, Q. Wu, J. Zhang, and G. Geers, “SKRWM based descriptor for pedestrian detection in thermal images, ” in Proceedings of IEEE Workshop on Multimedia Signal Processing (IEEE, 2011), pp. 1–6.
[Crossref]

R. Wan, B. Shi, T. A. Hwee, and A. C. Kot, “Depth of field guided reflection removal,” in Proceedings of the IEEE International Conference on Image Processing (IEEE, 2016), pp. 21–25.

T. Sirinukulwattana, G. Choe, and I. S. Kweon, “Reflection removal using disparity and gradient-sparsity via smoothing algorithm,” in Proceedings of the IEEE International Conference on Image Processing (IEEE, 2015), pp. 1940–1944.
[Crossref]

Y. Li and M. S. Brown, “Exploiting reflection change for automatic reflection removal,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 2013), pp. 2432–2439.
[Crossref]

A. Teschioni and C. S. Regazzoni, “A robust method for reflections analysis in color image sequences,” in Signal Processing Conference (EUSIPCO 1998) (IEEE, 1998), pp. 1–4.

E. Hecht, Optics, 4th ed. (Pearson Education, 2002).

Y. Zhao, Multi-band Polarization Imaging and Applications. (Springer, 2016).

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (18)

Fig. 1
Fig. 1 Procedures of the proposed reflection removal scheme. (a) Original LWIR image. (b) The OPC difference. (c) Joint reflection detection result. (d) Reflection matte result. (e) Reflection removal result.
Fig. 2
Fig. 2 Mode of how reflection is formed in LWIR and we consider the measured polarization signature is primarily due to reflections from the surface of the medium.
Fig. 3
Fig. 3 Two polarized components perpendicular and parallel to the plane of incidence of reflectance vary with respect to the angle of incidence, and the two components are different under most angles of incidence (the refractive index is about 6.74 for common marble tiles in LWIR spectrum).
Fig. 4
Fig. 4 Reflection Detection Based on Difference of OPC. (a) Perpendicular component I . (b) Parallel component I . (c) Difference of OPC. (d) Binary result of (c).
Fig. 5
Fig. 5 The AoP of reflection region vary with respect to θ and δ.
Fig. 6
Fig. 6 The target for the study of AoP in reflection region is a plane blackbody, and we put a marble tile in front of it as a reflective medium.
Fig. 7
Fig. 7 Experiments on uniformity of AoP under different viewing angle. (a)-(d) correspond to viewing angle 50, 60, 70 and 80 degrees, respectively. The first row is original infrared image, the second row is AoP image, and the third row is corresponding statistical results of the distribution of AoP in reflection regions (regions in red circle).
Fig. 8
Fig. 8 Experiment on uniformity of AoP under different blackbody temperature. (a) is original infrared image, and (b)-(f) represent AoP images in 40,50,60,70, and 80C, respectively. (g) shows the statistical results of the distribution of AoP in reflection regions under different blackbody temperature in a single chart.
Fig. 9
Fig. 9 Joint reflection detection. (a) Difference of OPC. (b) Joint reflection detection result. (c) Binary result of (a). (d) Binary result of (b).
Fig. 10
Fig. 10 Reflection matte result. (a) Joint reflection detection result. (b) Binary result of (a). (c) Reflection matte map, where the white regions represent reflection regions and the black regions are non-reflection regions.
Fig. 11
Fig. 11 Reflection removal result. (a) Original LWIR image. (b) Reflection matte map. (c) Background reference image. (d) Reflection removal result.
Fig. 12
Fig. 12 Experiment results in real DoFP infrared polarization data. (a)-(h) represent different selected frame. The first row is infrared intensity image, the second row is the reflection detection map and the third row is reflection removal result.
Fig. 13
Fig. 13 Detection before and after reflection removal.
Fig. 14
Fig. 14 The ground truth reflection maps and our detected reflection maps. (a)-(h) represent different selected frames. The first row is infrared intensity image, the second row is the ground truth reflection map and the third row is corresponding our detected reflection map.
Fig. 15
Fig. 15 (a) The detection error. (b) The TH before and after reflection removal.
Fig. 16
Fig. 16 Comparison of different methods. (a) Image without reflection removal. (b) The ICA-based reflection removal results. (c) The SPICA-based reflection removal results. (d) The MI-based reflection removal results. (e) EO photon separation results. (f) Our method. And the different rows represent different frames.
Fig. 17
Fig. 17 Experiment on complex situations that contain several persons. (a) Three persons stood in different position. (b) Two person when one was blocked by another and the reflection region in red line belongs to the front person and the reflection region in yellow line belongs to another one. (c) The combination of the above two cases. And the first row is infrared intensity image where the green dotted lines are baselines, the second row is reflection detection map and the third row is reflection removal result.
Fig. 18
Fig. 18 Experiments on complex situations that the camera is not vertical. (a) Infrared intensity image where the green dotted lines are baselines. (b) Reflection detection map. (c) Reflection removal results. The first row camera is slightly sloping and the second row is a person sit near a glass wall.

Tables (2)

Tables Icon

Table 1 Algorithm outline of reflection alpha matting

Tables Icon

Table 2 Performance of the proposed reflection removal method

Equations (29)

Equations on this page are rendered with MathJax. Learn more.

L (θ)= R + E =P( T t ) r (θ)+P( T m ) ε (θ)
L (θ)= R + E =P( T t ) r (θ)+P( T m ) ε (θ)
L (θ) R =P( T t ) r (θ)
L (θ) R =P( T t ) r (θ)
r (θ,n)= sin 2 (θ θ t (θ,n)) sin 2 (θ+ θ t (θ,n)) and r (θ,n)= tan 2 (θ θ t (θ,n)) tan 2 (θ+ θ t (θ,n))
[ ϕ 1 ϕ (x)]= 1 2 tan 1 ( I 1 (x)+ I 3 (x)2 I 2 (x) I 1 (x) I 3 (x) )
I (x)= I 1 (x)+ I 3 (x) 2 + I 1 (x) I 3 (x) 2cos2[ ϕ 1 ϕ (x)]
I (x)= I 1 (x)+ I 3 (x) 2 I 1 (x) I 3 (x) 2cos2[ ϕ 1 ϕ (x)]
D o (x)= | I (x) I (x) | 2
E R (t)= E 0 exp[ i(ωt+ δ ) ]
E R (t)= E 0 exp[ i(ωt+ δ ) ]
E 0 = R =P( T t ) r (θ)
E 0 = R =P( T t ) r (θ)
S 0R =cosθ( E R E R + E R E R )
S 1R =cosθ( E R E R E R E R )
S 2R =cosθ( E R E R + E R E R )
S 3R =icosθ( E R E R E R E R )
S 0R =cosθ( E 0 2 + E 0 2 )=cosθ( R 2 + R 2 )
S 1R =cosθ( E 0 2 E 0 2 )=cosθ( R 2 R 2 )
S 2R =cosθ(2 E 0 E 0 cosδ)=cosθ(2 R R cosδ)
S 3R =cosθ(2 E 0 E 0 sinδ)=cosθ(2 R R sinδ)
Ao P R = 1 2 tan 1 ( S 2R S 1R )
Ao P R = 1 2 tan 1 ( cosθ(2 R R cosδ) cosθ( R 2 R 2 ) )= 1 2 tan 1 ( 2 R R cosδ R 2 R 2 )
Ao P R = 1 2 tan 1 ( 2( R / R )cosδ ( R / R ) 2 1 )
Ao P R = 1 2 tan 1 ( 2( r (θ)/ r (θ) )cosδ ( r (θ)/ r (θ) ) 2 1 )
D joint (x)= | I (x) I (x) | 2 i=1 k exp[ η ( AoP(x) P i ) 2 ]
α=argmin α T Lα+λ( α T b S T ) D S (α b S )
E= N over + N under N truth ×100%
TH= h t h r +h ×100%