Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Polarization-based smoke removal method for surgical images

Open Access Open Access

Abstract

Smoke generated during surgery affects tissue visibility and degrades image quality, affecting surgical decisions and limiting further image processing and analysis. Polarization is a fundamental property of light and polarization-resolved imaging has been studied and applied to general visibility restoration scenarios such as for smog or mist removal or in underwater environments. However, there is no related research or application for surgical smoke removal. Due to differences between surgical smoke and general haze scenarios, we propose an alternative imaging degradation model by redefining the form of the transmission parameters. The analysis of the propagation of polarized light interacting with the mixed medium of smoke and tissue is proposed to realize polarization-based smoke removal (visibility restoration). Theoretical analysis and observation of experimental data shows that the cross-polarized channel data generated by multiple scattering is less affected by smoke compared to the co-polarized channel. The polarization difference calculation for different color channels can estimate the model transmission parameters and reconstruct the image with restored visibility. Qualitative and quantitative comparison with alternative methods show that the polarization-based image smoke-removal method can effectively reduce the degradation of biomedical images caused by surgical smoke and partially restore the original degree of polarization of the samples.

Published by Optica Publishing Group under the terms of the Creative Commons Attribution 4.0 License. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

1. Introduction

In surgery, the production of surgical smoke, vapor and aerosols (referred to throughout as ‘smoke’) during tissue dissection not only reduces the tissue visibility, but also introduces an error during further image processing and tissue diagnosis [1]. Therefore, it would be useful to reduce the influence of surgical smoke, enhance image contrast and texture, and remove ‘noise’ in order to improve the biomedical image quality.

Polarization is a fundamental property applying to transverse waves that specifies the geometrical orientation of the oscillations. According to the oscillation trajectory of the electric field in the plane perpendicular to the propagation direction, polarized light can be divided into different polarization states, namely linearly polarized light, circularly polarized light and elliptically polarized light. Polarized light can be produced from the common physical processes that deviate light beams, including scattering, absorption, refraction, reflection and birefringence. When light propagates through a scattering medium, it undergoes multiple scattering with particles, leading to randomization of its direction, phase, and polarization state compared with the incident light. Therefore, compared to general non-polarized optical imaging, polarization imaging can detect additional or ‘hidden’ information from the way the scattering causes variation in the polarization state of light. On this basis, polarization has been widely applied in many different fields, including for biomedical imaging and tissue diagnosis [25].

Although the problem of surgical smoke removal is a recent topic of interest with only a few published studies [610], it builds on the basis of the well-established outdoor image dehazing research. Outdoor dehazing methods can be roughly divided into three main categories: enhancement approaches, restoration approaches and learning approaches [11]. Enhancement approaches seek to remove the haze (‘noise’) and improve the contrast by using image processing algorithms including, for example, adaptive histogram equalization (AHE) [12], retinex [13], wavelets [14], etc. These methods are simple and direct but their dehazing effect is limited. Restoration approaches are usually based on various models that estimate the dehazed image color and intensity from the image based on specific priors or assumptions, for example, based on the dark channel prior [15], maximizing contrast [16], color lines prior [17], haze lines prior [18,19], color attenuation prior [20] and polarization based [2125], etc. These methods usually have a better dehazed output, but the assumptions and prior information have limitations and are not applicable in all scenarios. Learning approaches are recently becoming popular, some of which still follow the idea of estimating model parameters, while others propose a trainable end-to-end network for direct haze-free output [26,27]. However, learning approaches need advanced equipment, specific datasets and a lot of training data and time.

Even though there are examples of polarization imaging in outdoor scenes such as for foggy days [21], underwater imaging [2225], and for biomedical image processing and tissue diagnosis [25], there are currently neither related research about polarization based surgical smoke removal, nor polarized surgical smoke datasets [1]. In this paper, we propose a polarization-based smoke removal method for biomedical images. We analyzed and simulated the propagation of polarized light in a mixed medium, and implemented the polarization difference method to estimate the transmission parameters of the smoke. Real experiments were performed to verify the proposed method, and qualitative and quantitative results show that our method effectively reduced the degradation of biomedical images caused by surgical smoke and partially restored the original degree of polarization of the samples. The proposed method has advantages of simplicity, speed and effectiveness, without a requirement for extensive datasets or training process. Medical images can be restored from the effect of surgical smoke in real time.

The remainder of this paper is organized as follows. In Sec. 2, the general image degradation model is introduced and a redefined polarization-based image degradation model is proposed. In Sec. 3, the propagation of polarized light is analyzed and simulated and the detailed process of parameter estimation from smoke data is proposed, including the polarization difference calculation for the estimation of transmission. Then, in Sec. 4, the experimental results and qualitative and quantitative analysis are presented and finally some conclusions are drawn in Sec. 5.

2. Model and simulation

2.1 General image degradation model

In large-scale scattering media such as atmospheric and underwater environments, a common general outdoor image degradation model is [15]:

$$I\textrm{(x)} = J\textrm{(x)}t\textrm{(x)} + \textrm{A}(\infty )(1 - t\textrm{(x)}),$$
where the two terms represent the direct transmission and scattered air-light, respectively. I and J describe the intensity of the haze and haze-free image, respectively, x is a pixel in the image, $\textrm{A}(\infty )$ denotes the atmospheric radiance at an infinite distance that scatters into the imaging path, t is the medium transmission rate which describes the proportion of radiance attenuation. The transmission rate t can be expressed as:
$$t\textrm{(x)} = {e^{ - \beta d(x)}},$$
when the medium extinction coefficient $\beta$ is depth invariant and d refers to the scene depth.

2.2 Polarization-based Image degradation model

The image degradation model introduced above is applicable to a wide range of media where the estimated transmission map is related to the scene depth and haze density does not vary extensively across the scene. However, in a local, limited depth range, some definitions of the parameters of the degradation model should be modified. Depth information is independent from the attenuation of signal transmission t, because the depth of the local area is almost the same. Instead, the transmission rate t is determined by the optical depth of the local smoke $\tau$, i.e. the higher the concentration of smoke, the lower the transmission rate. Therefore, we redefine the transmission t as:

$$t\textrm{(x)} = {e^{ - \beta \tau (x)}},$$

Correspondingly, $\textrm{A}(\infty )$ denotes the intensity of ambient light with infinite optical depth, meaning that the smoke is heavy enough that no object radiance can be received through the smoke. Therefore, the polarization-based smoke removal model in a specific polarization state can be rewritten as:

$${I_{pol}}\textrm{(x)} = {J_{pol}}\textrm{(x)}{e^{ - \beta \tau (x)}} + {\textrm{A}_{pol}}(\infty )(1 - {e^{ - \beta \tau (x)}}),$$

2.3 Propagation of polarized light in a medium

Before developing solutions to estimate parameters of the degradation model and remove the smoke, it is important to understand how the polarized light interacts with the biological object and the smoke. In general, for surgical imaging the illumination must travel through the smoke layer, with photons then scattered by the object being imaged in reflection mode. As the light propagates into the scattering medium, its direction, phase, and polarization state become randomized due to multiple scattering [28,29]. The characteristics of the resulting scattered polarized light strongly depend on different parameters, such as the medium particle size and density, the initial polarization state, the wavelength, and the depth, among others [29,30].

Ideally, the scene would be simplified as a double layered scattering medium, with an upper layer of smoke and a deeper layer of tissue, as shown in Fig. 1. For linear polarization illumination, the superficial smoke layer, as indicated in the orange semicircle area in Fig. 1, maintains the initial polarization, which means the backscattered polarized light contains a substantial component in its original polarization state, while the deeper layers contain a multiply-scattered component [28,31]. In order to study the propagation of polarized light in the medium under different illumination and medium conditions, simulation experiments were completed with a Polarized Monte Carlo program [32,33]. Polarized Monte Carlo provides a solution for understanding the propagation of polarized photons in the medium. It can sequentially track the movement of photons in the medium, and can realize the update of the photon’s polarization state after each movement. The detailed Polarized Monte Carlo modeling method can be found in the supplemental document. As surgical smoke consists of particles with diameters mostly in the range from 0.01 µm to 6.0 µm, depending on the type of electrocauterized tissue and surgical tool [34,35], we chose two particle diameters, 0.2 µm and 6.0 µm. The scattering coefficient (expressed as a number proportional to the amount of photons scattered per distance) is a measure of the ability of particles to scatter photons out of a beam of light and it is determined by the particle diameters, medium density, medium relative refractive index and incident wavelength [36]. The density was set to ensure that the calculated scattering coefficient was consistent with real smoke [37] and the absorption coefficient was set to a small value [38,39]. The scattering coefficient and anisotropy g were calculated by calling a Mie function. The refractive indexes of smoke particle and tissue were selected according to [40,41]. The detailed simulation parameters are listed in Table 1.

 figure: Fig. 1.

Fig. 1. Schematic of polarized light propagation in the medium.

Download Full Size | PDF

Tables Icon

Table 1. Simulation parameters

In the Monte Carlo calculations, the receiving plane was placed at different heights above the tissue – corresponding to different medium depths – to capture the backscattered radiance. The receiving plane had a size of 25×25 cm, with 100×100 sampling points, and the pencil beam illuminant was incident perpendicularly at the central point of the plane. The 3D distributions of linear polarized backscattered radiance in different types of medium at different heights, 2 cm, 4 cm, 6 cm and 8 cm, are shown in Fig. 2. For two media with particles of different sizes, regardless of medium depth, the co-polarized component dominated the backscattering radiance but decreased rapidly with increasing medium depth. Specifically, due to the small scattering coefficient of the small-sized particles, the probability of photon-particle interactions was low, and most photons were detected after being reflected directly by the object. Therefore, the backscattered co-polarized radiance was high and the cross-polarized radiance remained low. The scattering coefficient and the probability of photons-particle interactions increased for large-sized particles, and photons gradually lost their initial polarization state and became randomly polarized. The difference in radiance between co-polarized and cross-polarized channels decreased, especially when the depth of the medium exceeded the mean free path of photon transmission.

 figure: Fig. 2.

Fig. 2. Radiance of polarization channels at different heights in the medium with (a) small-sized particles (0.2 µm) and (b) large-sized particles (6.0 µm). The radiance of the brightest central region is marked in each subgraph.

Download Full Size | PDF

Considering real scenarios, with a size distribution of particles within surgical smoke, we carried out a simulation for a polydisperse medium [42], and assumed that the particle size followed a Gaussian distribution, with a range set to match surgical smoke particle sizes. The detailed simulation parameters are listed in Table 2. The distribution of particle sizes and the radiance of orthogonal linear polarization channels are shown in Fig. 3. We could conclude that, the particles polydispersity affects the contrast of orthogonal linear polarization channels, and the degree of impact was determined by the particle size distribution and weight. However, the co-polarized component still dominated the backscattering radiance.

 figure: Fig. 3.

Fig. 3. Backscattered radiance of linear polarization illumination in a polydisperse medium: (a) the distribution of the particle sizes and the radiance of (b) co-polarized and cross-polarized channel at different heights.

Download Full Size | PDF

Tables Icon

Table 2. Simulation parameters

From the simulation results we can conclude that, for the linear polarization imaging:

  • i) the co-polarized channels consist of large amounts of polarization-maintaining light from the superficial layers;
  • ii) the cross-polarized channels eliminate dominance of radiance from superficial layers, which means the cross-polarized channel has a certain dehazing effect, and this is consistent with the conclusions in Ref. [23,25];
  • iii) at greater depths, co-polarized and cross-polarized channels have similar intensities.

2.4 Polarization difference for transmission estimation

For linearly polarized incident light and a pair of orthogonal (co-polarized and cross-polarized) backscattered channels, the polarization difference calculation can be used to estimate parameters of the degradation model. As was demonstrated in the previous section and summarized in Table 3, the co-polarized channel was dominated by a polarization-maintaining component from smoke superficial layers [28]. After multiple particle interactions during propagation into deeper layers, the light had near-equal contribution in all polarization channels (including the co-polarized and the cross-polarized channels) [43]. The cross-polarized channels reduced the radiance from superficial layers and included a multiply scattered component. Therefore, compared with co-polarized channels, cross-polarized channels contained relatively more radiance from deeper smoke layers closer to the object.

Tables Icon

Table 3. Backscattered light characteristics for linear polarization illumination

The polarization difference calculation was implemented to estimate the smoke component and transmission rate of the medium. The polarization-based image degradation model for co-polarized and cross-polarized states was rewritten as:

$$\left\{ {\begin{array}{{l}} {{I_{co}}\textrm{(x)} = {J_{co}}\textrm{(x)}{e^{ - \beta \tau (x)}} + {\textrm{A}_{co}}(\infty )(1 - {e^{ - \beta \tau (x)}})}\\ {{I_{c\textrm{r}o}}\textrm{(x)} = {J_{cro}}\textrm{(x)}{e^{ - \beta \tau (x)}} + {\textrm{A}_{cro}}(\infty )(1 - {e^{ - \beta \tau (x)}})} \end{array}} \right.,$$

When the incident light underwent multiple scattering to reach the object layer, it lost its initial polarization state and became randomly polarized before being detected, therefore, the radiance from the object, ${J_{co}}\textrm{(x)}$ and ${J_{cro}}\textrm{(x)}$ could be regarded as approximately equal (when the detection distance approached the photon transport mean free path in the medium) [43] in Eq. (5). So that polarization difference calculation could be written as:

$${I_{co}}\textrm{(x)} - {I_{c\textrm{r}o}}\textrm{(x)} = {\textrm{A}_{co}}(\infty )(1 - {e^{ - \beta \tau (x)}}),$$

Notably, ${\textrm{A}_{co}}(\infty )$ was a constant and was estimated from all the pixels of co-polarized channel, in the presence of complete smoke component. Therefore, ${\textrm{A}_{co}}(\infty )$ should be maintained in the polarization difference calculation. Then the transmission rate of the degradation model could be estimated as:

$$t\textrm{(x)} = {e^{ - \beta \tau (x)}} = 1 - \frac{{{I_{co}}\textrm{(x)} - {I_{c\textrm{r}o}}\textrm{(x)}}}{{{\textrm{A}_{co}}(\infty )}},$$

A smoke removal result was reconstructed by doubling the radiance of the processed cross-polarized channel because:

  • i) cross-polarized channels were less affected by the polarization-maintaining light as well as the reflectance of the medium [23,25];
  • ii) cross-polarized channels contained relatively more detailed information of the object (an advantage of polarization-resolved imaging);
  • iii) in deeper layers, the contribution of co-polarized and cross-polarized channels was nearly equal.

In summary, the restored image could be calculated as:

$$J\textrm{(x)} = 2 \cdot {J_{cro}}\textrm{(x)} = 2 \cdot \frac{{{I_{c\textrm{r}o}}\textrm{(x)} - {\textrm{A}_{cro}}(\infty )}}{{{e^{ - \beta \tau (x)}}}} + \textrm{2} \cdot {\textrm{A}_{cro}}(\infty ),$$

Note that even if the smoke is inhomogeneous, the difference of pixels under different optical depths is still reflected in the polarimetric imaging results and polarization difference results. The processing of each pixel is entirely independent and the estimated transmission rate t will vary corresponding to the smoke density, so our model can handle inhomogeneous smoke.

2.5 Estimation of ambient light with infinite optical depth

The estimation of ${\textrm{A}_{co}}(\infty )$ and ${\textrm{A}_{cro}}(\infty )$ can be contrasted with outdoor haze imaging methods, where a pixel is selected from a background area containing no visible object, representing an infinite depth point, i.e. an estimate of the air-light at infinity. However, in biomedical imaging where the tissue is generally close to the camera, there may be no areas with infinite optical depth to estimate the ambient light at infinity. Therefore, the algorithm in [44] was adopted to estimate the parameter $\textrm{A}(\infty )$.

In brief, a specific color at different positions within the image will be affected by haze to varying degrees, and its red-green-blue radiance constitutes a linear distribution called a ‘Haze Line’. Pixels in a hazy image can be modeled using these haze-line priors in red-green-blue space that all intersect with the air-light value at infinity [44].

The method contains three main steps [44]:

  • i) converting the RGB image into an indexed image with 64 clusters (64 typical color values);
  • ii) projecting the color clusters onto three different 2D RGB planes, RB, GB, RG;
  • iii) using a Hough transform to vote for the location of the air-light in 2D RGB planes.

2.6 Color information

Color is affected by haze to varying degrees, and it is important to estimate the transmission rate t per pixel in R, G, B channels respectively, instead of using a single unique value per pixel for three channels. For example, if haze-free red pixels are affected by smoke, the intensities of the G and B channels are more significantly affected compared to the R channel and therefore, the transmission rate for the G and B channels must be lower than for the R channel [45].

3. Experiments

3.1 Experiment setup

The Polarization State Analyzer (PSA) for the experiments was a LUCID-TRI050S integrated with Sony polarized sensor IMX250MYR. With an intuitive graphical user interface, (Arenaview), four polarimetric channels, 0°, 45°, 90°, 135°, could be captured in color simultaneously. In addition, a linear polarization luminant was used as the polarization state generator (PSG) and consisted of a LED light source GI-0604 and a calibrated linear polarizer. The polarization setup is shown in Fig. 4. All the equipment was placed in a container with a black foam cover, and a fogger AB-900 could inject a controllable level of fog (from heated fluid) into the container through a pipe to simulate the surgical smoke environment. The created smoke contained multi-size particles and had a diameter mainly ranging from 1-5 µm [46]. A series of polarimetric images that varied with the concentration of the smoke were collected.

 figure: Fig. 4.

Fig. 4. The polarization setup

Download Full Size | PDF

3.2 Results and analysis

Before smoke injection, the incident linear polarized light was calibrated to the sample and smoke-free ground truth reference data was captured. Since the sample signal was completely obscured by the smoke, the polarimetric images of the mixed medium were collected at regular intervals (different smoke concentrations). A comparative experiment with a SpyderCheckr sample for different smoke concentrations is shown in Fig. 5, including examples of estimating the ambient light with infinite optical depth (Fig. 5(g-i)). A pig kidney and liver were chosen as ex vivo tissue samples, as shown in Fig. 6. A pair of orthogonal polarimetric images with 1134 × 917 pixels took an average processing time of 5.5 seconds using MATLAB R2019a on a Windows laptop with Intel Core i7-9750H and 16GB RAM. The experimental results show that the polarization-based smoke removal method reduced the effect of smoke on image quality in near real time for different smoke concentrations and samples.

 figure: Fig. 5.

Fig. 5. Comparison of different smoke concentrations (levels 1–5, where level 1 was extremely heavy smoke, level 3 was moderate and level 5 was mild concentration) with SpyderCheckr: (a) co-polarized smoke images, (b) cross-polarized smoke images, (c) polarization difference images, (d) estimated transmission maps, (e) smoke removal output images, (f) ground truth. (g-i) Examples of ambient light estimation with infinite optical depth (diamond) for level 2 smoke in different indicated color spaces.

Download Full Size | PDF

 figure: Fig. 6.

Fig. 6. Comparison between different smoke concentrations for tissue samples (kidney and liver): (a) co-polarized smoke images, (b) cross-polarized smoke images, (c) polarization difference results, (d) estimated transmission maps, (e) smoke removal output images, (f) ground truth.

Download Full Size | PDF

Three prior-based dehazing methods were also implemented for comparison, including methods based on a dark channel prior (DCP) [15], haze line prior (HLP) [19] and color attenuation prior (CAP) [20], with different sample surfaces. Both the co-polarized and cross-polarized images were processed by different dehazing methods, as shown in Fig. 7. According to our tests, these methods have similar processing times.

 figure: Fig. 7.

Fig. 7. Comparison of dehazing results between different methods: (a) haze images, (b) ground truth, results processed by (c) DCP, (d) HLP, (e) CAP, (f) POL.

Download Full Size | PDF

It can be concluded that all methods can reduce the impact of smoke on image quality. After comparing the dehazing result with the ground truth, we conclude that DCP method underestimate the transmission parameters, leading to a very low luminance, oversaturation effect and information loss, as shown in Fig. 7(c). The HLP method had a better dehazing performance, as indicated by Fig. 7(d), but compared to the ground truth, the restoration of the original color was not accurate. The CAP method overestimated the transmission parameters so that the smoke was not completely removed, and for the liver sample it was not accurate enough to restore the color, as shown in Fig. 7(e). Our polarization-based method (POL) reduced the impact of smoke, as indicated in Fig. 7(f), and through the comparison of the SpyderCheckr scenario, the original color was restored very well.

The Peak Signal-to-Noise Ratio (PSNR) [47], the Structural Similarity Index (SSIM) [48], the CIEDE2000 [49] were adopted to quantitatively evaluate the dehazing results, as shown in Table 4, for reference. The PSNR index was defined by Mean Square Error [47] and the SSIM index was determined by three factors, luminance, contrast and structure [48]; the higher the values of these two indicators, the better the image quality. The CIEDE2000 could evaluate color difference of dehazing results and ground truth [49], for color restoration performance, the lower the value of this indicator, the better the color restoration performance. We manually selected the pixels of the sample area and performed quantitative evaluation so as to reduce interference from the background. From these values we concluded that different methods usually have different performance in different scenarios. However, sometimes the conclusion from qualitative and quantitative analysis were not consistent, for example, we qualitatively considered that CAP did not have a good smoke removal performance on the co-polarized channel of the SpyderCheckr and liver sample because the parameters were overestimated and the color restoration was inaccurate, as shown in Fig. 7(e), but it had the best PSNR and SSIM results.

Tables Icon

Table 4. Quantitative evaluation of different dehazing methods: SSIM and PSNR (the higher the better), CIEDE2000 (the lower the better)

However, in addition to restoring the image visibility, for biomedical images, the restoration of the original polarization properties of the samples can be further applied for biomedical information processing and disease detection. Therefore, it may be useful to restore the original polarization information of the samples as far as possible while reducing the influence of surgical smoke. Muller matrices can characterize the sample’s polarization properties and evaluate the accuracy of polarization information restoration. However, it needs multiple measurements to obtain and is not suitable for dynamic smoke scenarios. Therefore, the degree of polarization (DOP), one of a number of possible parameters describing the polarization information of the samples, was adopted and calculated using Eq. (9). The restored DOP results are shown in Fig. 8 and quantified in Table 5. In the SpyderCheckr scenario, the DCP and HLP method were not accurate enough for color recovery, which would affect the balance between polarization channels and therefore affect the DOP restoration. The CAP method had limited ability to recover DOP information. Compared with the qualitative and quantitative results, our POL-based method had the best DOP restoration performance.

$$\textrm{DOP} = \frac{{{I_{co}}\textrm{(x)} - {I_{c\textrm{r}o}}\textrm{(x)}}}{{{I_{co}}\textrm{(x)} + {I_{c\textrm{r}o}}\textrm{(x)}}},$$

However, our method also has shortcomings, since cross-polarized radiance can inevitably be affected by smoke, especially at high concentrations where the estimated smoke information from polarization difference calculations is lower than expected, which in turn leads to overestimation of the transmission parameters. Therefore, in the case of the environment with high smoke concentration, our experimental results still retained some smoke. On the other hand, in real surgical smoke scenarios, the concentration of smoke would not be expected to reach that level [1], so the proposed polarization-based smoke removal method remains applicable. In addition, the proposed quantitative evaluation indexes often evaluate from a certain aspect, and cannot comprehensively evaluate the smoke removal performance, so they are not entirely suitable for our scenario. Specifically, PSNR only considers the absolute difference and is not sensitive to structural information; SSIM is partially determined by a luminance factor, however, in our surgical smoke scenario, the captured smoke images have higher luminance than processed images, so they have higher scores in luminance factors than processed images. This is confounding because the processed images should have a higher SSIM score than the smoke images, and it is therefore important to better balance the weights of these three SSIM factors, or to develop other quantitative evaluation indexes for our scenario.

 figure: Fig. 8.

Fig. 8. Comparison of restored DOP results between different methods: (a) the DOP of haze images, (b) the DOP of ground truth, results processed by (c) DCP, (d) HLP, (e) CAP, (f) POL.

Download Full Size | PDF

Tables Icon

Table 5. Quantitative evaluation of DOP results of different methods: SSIM and PSNR (the higher the better), CIEDE2000 (the lower the better)

4. Conclusion

In endoscopic/laparoscopic surgery, image quality can be severely degraded by surgical smoke, which reduces the visibility for the surgeons and introduces errors when using image processing. A simple, real-time, polarization-based smoke removal method for surgical images was proposed. Unlike the outdoor dehazing problem, we redefined the parameters of transmission rate and ambient light with infinity and proposed the polarization-based degradation imaging model. Experiments with a calibrated polarization setup, for different smoke concentrations, and different sample surface materials were conducted. Qualitative and quantitative analysis and comparisons with alternative methods showed that our method can effectively reduce the degradation of biomedical images caused by surgical smoke and partially restore the original degree of polarization of the samples.

Funding

China Scholarship Council (201906690041); Engineering and Physical Sciences Research Council (EP/R004153/1); National Natural Science Foundation of China (61971177); National Natural Science Foundation of China (61905152).

Acknowledgments

Infrastructure support is acknowledged from the NIHR Imperial Biomedical Research Centre (BRC) and the CRUK Imperial Centre at Imperial College London.

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Supplemental document

See Supplement 1 for supporting content.

References

1. I. C. London, “Hamlyn centre laparoscopic / endoscopic video datasets,” Hamlyn Centre Laparoscopic, 2020, http://hamlyn.doc.ic.ac.uk/vision/. Accessed: 2020-10.

2. N. T. Clancy, S. Arya, J. Qi, D. Stoyanov, G. B. Hanna, and D. S. Elson, “Polarised stereo endoscope and narrowband detection for minimal access surgery,” Biomed. Opt. Express 5(12), 4108–4117 (2014). [CrossRef]  

3. C. He, H. He, J. Chang, B. Chen, H. Ma, and M. J. Booth, “Polarisation optics for biomedical and clinical applications: a review,” Light: Sci. Appl. 10(1), 1–20 (2021). [CrossRef]  

4. J. Qi and D. S. Elson, “Mueller polarimetric imaging for surgical and diagnostic applications: a review,” J. Biophotonics 10(8), 950–982 (2017). [CrossRef]  

5. J. Qi and D. S. Elson, “A high definition Mueller polarimetric endoscope for tissue characterization,” Sci. Rep. 6(1), 1–11 (2016). [CrossRef]  

6. K. Tchaka, V. M. Pawar, and D. Stoyanov, “Chromaticity based smoke removal in endoscopic images,” in Med. Imaging 2017: Image Processing, (International Society for Optics and Photonics, 2017), 10133, p.101331M.

7. X. Luo, A. J. McLeod, S. E. Pautler, C. M. Schlachta, and T. M. Peters, “Vision-based surgical field defogging,” IEEE Trans. Med. Imaging 36(10), 2021–2030 (2017). [CrossRef]  

8. A. Leibetseder, M. J. Primus, S. Petscharnig, and K. Schoeffmann, “Real-time image-based smoke detection in endoscopic videos,” in Proceedings of the on Thematic Workshops of ACM Multimedia, (2017), p. 296–304.

9. C. Wang, F. A. Cheikh, M. Kaaniche, and O. J. Elle, “A smoke removal method for laparoscopic images,” arXiv preprint arXiv:1803.08410 (2018).

10. C. Wang, A. K. Mohammed, F. A. Cheikh, A. Beghdadi, and O. J. Elle, “Multiscale deep desmoking for laparoscopic surgery,” in Med. Imaging 2019: Image Processing, (International Society for Optics and Photonics, 2019), 10949, p.109491Y.

11. D. Singh and V. Kumar, “A comprehensive review of computational dehazing techniques,” Arch. Comput. Methods Eng. 26(5), 1395–1413 (2019). [CrossRef]  

12. R. Singh and M. Biswas, “Adaptive histogram equalization based fusion technique for hazy underwater image enhancement,” in IEEE Int. Conf. Comput. Intell. Comput. Res., (2016), pp. 1–5.

13. W. Wang and L. Xu, “Retinex algorithm on changing scales for haze removal with depth map,” Int. J. Hybrid Inf. Technol. 7(4), 353–364 (2014). [CrossRef]  

14. S. G. Chang, B. Yu, and M. Vetterli, “Adaptive wavelet thresholding for image denoising and compression,” IEEE Trans. on Image Process. 9(9), 1532–1546 (2000). [CrossRef]  

15. K. He, J. Sun, and X. Tang, “Single image haze removal using dark channel prior,” IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2011). [CrossRef]  

16. R. T. Tan, “Visibility in bad weather from a single image,” in IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., 1–8 (2008).

17. R. Fattal, “Dehazing using color-lines,” ACM Trans. Graph. 34(1), 1–14 (2014). [CrossRef]  

18. D. Berman, T. Treibitz, and S. Avidan, “Single image dehazing using haze-lines,” IEEE Trans. Pattern Anal. Mach. Intell. 42(3), 720–734 (2020). [CrossRef]  

19. D. Berman and S. Avidan, “Non-local image dehazing,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. (2016), pp. 1–9.

20. Q. Zhu, J. Mai, and L. Shao, “A fast single image haze removal algorithm using color attenuation prior,” IEEE Trans. on Image Process. 24(11), 3522–3533 (2015). [CrossRef]  

21. Y. Y. Schechner, S. G. Narasimhan, and S. K. Nayar, “Polarization-based vision through haze,” Appl. Opt. 42(3), 511–525 (2003). [CrossRef]  

22. Y. Y. Schechner and N. Karpel, “Recovery of underwater visibility and structure by polarization analysis,” IEEE J. Oceanic Eng. 30(3), 570–587 (2005). [CrossRef]  

23. I. Léonard, A. Alfalou, and C. Brosseau, “Sensitive test for sea mine identification based on polarization-aided image processing,” Opt. Express 21(24), 29283–29297 (2013). [CrossRef]  

24. M. Dubreuil, P. Delrot, I. Leonard, A. Alfalou, C. Brosseau, and A. Dogariu, “Exploring underwater target detection by imaging polarimetry and correlation techniques,” Appl. Opt. 52(5), 997–1005 (2013). [CrossRef]  

25. K. O. Amer, M. Elbouz, A. Alfalou, C. Brosseau, and J. Hajjami, “Enhancing underwater optical imaging by using a low-pass polarization filter,” Opt. Express 27(2), 621–643 (2019). [CrossRef]  

26. B. Cai, X. Xu, K. Jia, C. Qing, and D. Tao, “Dehazenet: An end-to-end system for single image haze removal,” IEEE Trans. on Image Process. 25(11), 5187–5198 (2016). [CrossRef]  

27. S. Bolkar, C. Wang, F. A. Cheikh, and S. Yildirim, “Deep smoke removal from minimally invasive surgery videos,” in Proc. - Int. Conf. Image Process. ICIP (2018), pp. 3403–3407.

28. E. Gnanatheepam, U. Kanniyappan, K. Dornadula, A. Prakasarao, and G. Singaravelu, “Polarization gating technique extracts depth resolved fluorescence redox ratio in oral cancer diagnostics,” Photodiagnosis Photodyn. Ther. 30, 101757 (2020). [CrossRef]  

29. C. Brosseau, “Fundamentals of polarized light: a statistical optics approach,” Wiley-Interscience (1998).

30. A. Lizana, A. Van Eeckhout, K. Adamczyk, C. Rodríguez, J. C. Escalera, E. Garcia-Caurel, I. Moreno, and J. Campos, “Polarization gating based on Mueller matrices,” J. Biomed. Opt. 22(05), 1 (2017). [CrossRef]  

31. S. Sridhar and A. Da Silva, “Enhanced contrast and depth resolution in polarization imaging using elliptically polarized light,” J. Biomed. Opt. 21(7), 071107 (2016). [CrossRef]  

32. J. C. Ramella-Roman, S. A. Prahl, and S. L. Jacques, “Three Monte Carlo programs of polarized light transport into scattering media: part I,” Opt. Express 13(12), 4420–4438 (2005). [CrossRef]  

33. J. C. Ramella-Roman, S. A. Prahl, and S. L. Jacques, “Three Monte Carlo programs of polarized light transport into scattering media: part II,” Opt. Express 13(25), 10392–10405 (2005). [CrossRef]  

34. B. C. Ulmer, “The hazards of surgical smoke,” AORN J. 87(4), 721–738 (2008). [CrossRef]  

35. M. Karjalainen, A. Kontunen, S. Saari, T. Rönkkö, J. Lekkala, A. Roine, and N. Oksala, “The characterization of surgical smoke from various tissues and its implications for occupational safety,” PLoS ONE 13(4), e0195274 (2018). [CrossRef]  

36. “Definition and units of scattering coefficient,” Available at: https://omlc.org/classroom/ece532/class3/mus-definition.html. Accessed: 2022-02.

37. “Smoke Density,” Available at: https://dieselnet.com/calculator/smoke1.html. Accessed: 2021-09.

38. I. V. Meglinsky and S. J. Matcher, “Modelling the sampling volume for skin blood oxygenation measurements,” Med. Biol. Eng. Comput. 39(1), 44–50 (2001). [CrossRef]  

39. “Cloud based monte carlo tool for photon transport,” Available at: http://www.biophotonics.fi/. Accessed: 2021-09.

40. S. Li, X. Cheng, P. Mei, S. Lu, H. Yang, and H. Zhang, “Multiple scattering of light transmission in a smoke layer,” Optik 125(9), 2185–2190 (2014). [CrossRef]  

41. S. L. Jacques, “Optical properties of biological tissues: a review,” Phys. Med. Biol 58(11), R37–R61 (2013). [CrossRef]  

42. A. J. Radosevich, J. D. Rogers, V. Turzhitsky, N. N. Mutyal, J. Yi, H. K. Roy, and V. Backman, “Polarized enhanced backscattering spectroscopy for characterization of biological tissues at subdiffusion length scales,” IEEE J. Select. Topics Quantum Electron. 18(4), 1313–1325 (2012). [CrossRef]  

43. P. Shukla, R. Sumathi, S. Gupta, and A. Pradhan, “Influence of size parameter and refractive index of the scatterer on polarization-gated optical imaging through turbid media,” J. Opt. Soc. Am. A 24(6), 1704–1713 (2007). [CrossRef]  

44. D. Berman, T. Treibitz, and S. Avidan, “Air-light estimation using haze-lines,” in IEEE Int. Conf. Comput. Photogr. (ICCP) (2017), pp. 1–9.

45. J. El Khoury, J. B. Thomas, and A. Mansouri, “Haze and convergence models: Experimental comparison,” in AIC (2015). https://hal.archives-ouvertes.fr/hal-01202989.

46. “Particle Size & Settling Velocities,” Available at: http://www.smokemachines.com/settling-velocities-particle-size.aspx. Accessed: 2021-10.

47. A. Hore and D. Ziou, “Image quality metrics: PSNR vs. SSIM,” in 2010 20th Int. Conf. Pattern Recognit. (2010), pp. 2366–2369.

48. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. on Image Process. 13(4), 600–612 (2004). [CrossRef]  

49. C. Li, S. Anwar, J. Hou, R. Cong, C. Guo, and W. Ren, “Underwater image enhancement via medium transmission-guided multi-color space embedding,” IEEE Trans. on Image Process. 30, 4985–5000 (2021). [CrossRef]  

Supplementary Material (1)

NameDescription
Supplement 1       Detailed Polarized Monte Carlo modeling method

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (8)

Fig. 1.
Fig. 1. Schematic of polarized light propagation in the medium.
Fig. 2.
Fig. 2. Radiance of polarization channels at different heights in the medium with (a) small-sized particles (0.2 µm) and (b) large-sized particles (6.0 µm). The radiance of the brightest central region is marked in each subgraph.
Fig. 3.
Fig. 3. Backscattered radiance of linear polarization illumination in a polydisperse medium: (a) the distribution of the particle sizes and the radiance of (b) co-polarized and cross-polarized channel at different heights.
Fig. 4.
Fig. 4. The polarization setup
Fig. 5.
Fig. 5. Comparison of different smoke concentrations (levels 1–5, where level 1 was extremely heavy smoke, level 3 was moderate and level 5 was mild concentration) with SpyderCheckr: (a) co-polarized smoke images, (b) cross-polarized smoke images, (c) polarization difference images, (d) estimated transmission maps, (e) smoke removal output images, (f) ground truth. (g-i) Examples of ambient light estimation with infinite optical depth (diamond) for level 2 smoke in different indicated color spaces.
Fig. 6.
Fig. 6. Comparison between different smoke concentrations for tissue samples (kidney and liver): (a) co-polarized smoke images, (b) cross-polarized smoke images, (c) polarization difference results, (d) estimated transmission maps, (e) smoke removal output images, (f) ground truth.
Fig. 7.
Fig. 7. Comparison of dehazing results between different methods: (a) haze images, (b) ground truth, results processed by (c) DCP, (d) HLP, (e) CAP, (f) POL.
Fig. 8.
Fig. 8. Comparison of restored DOP results between different methods: (a) the DOP of haze images, (b) the DOP of ground truth, results processed by (c) DCP, (d) HLP, (e) CAP, (f) POL.

Tables (5)

Tables Icon

Table 1. Simulation parameters

Tables Icon

Table 2. Simulation parameters

Tables Icon

Table 3. Backscattered light characteristics for linear polarization illumination

Tables Icon

Table 4. Quantitative evaluation of different dehazing methods: SSIM and PSNR (the higher the better), CIEDE2000 (the lower the better)

Tables Icon

Table 5. Quantitative evaluation of DOP results of different methods: SSIM and PSNR (the higher the better), CIEDE2000 (the lower the better)

Equations (9)

Equations on this page are rendered with MathJax. Learn more.

I (x) = J (x) t (x) + A ( ) ( 1 t (x) ) ,
t (x) = e β d ( x ) ,
t (x) = e β τ ( x ) ,
I p o l (x) = J p o l (x) e β τ ( x ) + A p o l ( ) ( 1 e β τ ( x ) ) ,
{ I c o (x) = J c o (x) e β τ ( x ) + A c o ( ) ( 1 e β τ ( x ) ) I c r o (x) = J c r o (x) e β τ ( x ) + A c r o ( ) ( 1 e β τ ( x ) ) ,
I c o (x) I c r o (x) = A c o ( ) ( 1 e β τ ( x ) ) ,
t (x) = e β τ ( x ) = 1 I c o (x) I c r o (x) A c o ( ) ,
J (x) = 2 J c r o (x) = 2 I c r o (x) A c r o ( ) e β τ ( x ) + 2 A c r o ( ) ,
DOP = I c o (x) I c r o (x) I c o (x) + I c r o (x) ,
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.