Effective solution for underwater image enhancement

Ye Tao; Ye Tao; Lili Dong; Luqiang Xu; Wenhai Xu

doi:10.1364/OE.432756

1. Introduction

Underwater world provides numerous treasures such as: varieties of marine creatures, amazing landscapes, mysterious shipwrecks, and amounts of available resources. Thus many researching fields are emerging such as: underwater man-made facility inspection [1], underwater target detection [2], marine life discovery [3], and underwater vehicle control [4]. However, images and videos captured in underwater medium, which always suffer from degradation of visual quality viz. contrast degradation, color distortion, and detail loss. This seriously limits people to exploring and understanding the underwater world. Then ‘How to effectively improve the visual quality of underwater images?’ has become a fundamental but vital topic in underwater optics research.

Unlike regular fogged images, light is wavelength dependent absorbed when propagating in underwater medium, besides scattering. Ancuti et al. demonstrated that red light with the longest wavelength almost disappears below depth of 5 m, then orange light (10m), yellow light (15m), which results in the appearance of greenish or bluish tone in underwater images generally [5]. The scattering changes the propagation direction resulting in contrast reduction and blur, while the absorption causes color distortion. McGlamery [6] first proposed the underwater imaging model in 1980, which could be expressed as a linear additivity of three major components called the forward-scattering component ${E_{forward}},$ the back-scattering component ${E_{back}},$ and the direct component ${E_{direct}}.$ The model was represented as follows:

(1)$${I^c}(x,y) = {E_{direct}}(x,y) + {E_{forward}}(x,y) + {E_{back}}(x,y)$$

where ${I^c}(x,y) = [{{I^R}(x,y),{I^G}(x,y),{I^B}(x,y)} ]$ denoted the captured image, parameter $c$ signified $R,G,B$ color channel respectively. $(x,y)$ was the pixel location. ${E_{direct}}(x,y)$ denoted the light directly reflected from target scene and reaching the camera. ${E_{forward}}(x,y)$ on behalf of the light which reflected by suspending particles but still getting into the camera. And ${E_{back}}(x,y)$ was the light component coming from surroundings without reaching the target but getting into the camera. Considering that the imaging target was always far away from the camera, ${E_{direct}}(x,y)$ could be ignored in computation of underwater imaging process. Then the model could be simplified as follows:

(2)$$\begin{aligned} {I^c}(x,y) &= {E_{forward}}(x,y) + {E_{back}}(x,y)\\ &= {J^c}(x,y){e ^{ - {t^c}(x,y)}} + {A^c}(1 - {e ^{ - {t^c}(x,y)}}) \end{aligned}$$

where ${J^c}(x,y)$ denoted the ideal dehazed image, ${t^c}(x,y) \in [0,1]$ signified the underwater light transmission rate. Figure 1 shows the model.

Fig. 1. Underwater imaging model.

Download Full Size | PDF

Many techniques have been proposed to dehaze underwater images. A few researchers, for instance, proposed to use dedicated hardware or polarization-based strategies [7,8] to dehaze underwater images/frames. Although excellent results obtained from these approaches, the exorbitant expense limited them to being applied for common users. Besides, some scientists proposed to use multi-images based strategies [9] to dehaze underwater images. However, acquiring multiple versions of one scene in underwater environment was operated quite difficultly. Scholars therefore began to pay more attention to developing underwater single image dehazing methods, which could be roughly divided into three subclasses: physical model-based methods, non-physical model-based methods, and data-driven methods.

Physical model-based methods were also called the underwater image restoration techniques, which usually established a degradation model based on the transformation of Eq. (2), then deduced related parameters via some prior knowledge, and finally restored inputs. The most representative one could be the dark channel prior (DCP) technique [10], which was initially dedicated designed for dehazing fogged images. DCP assumed that most local patches in fogged images always had at least one considerable low-intensity color channel. Although DCP could exactly dehaze fogged images, it failed in restoring underwater images due to ignoring the light absorption in underwater medium. Inspired by DCP, other underwater restoration approaches were presented. Hung et al. [11] proposed to apply a color-correction operation after the traditional DCP process. Drews et al. [12] proposed the famous underwater dark channel prior (UDCP), which was based on the observation that green and blue lights were the information source of inputs. Those methods usually employed a color correction operation and a reversion of underwater imaging model formular, which required prior knowledge or extra assumptions. Besides, their restored results were always unsatisfactory, especially in removing color distortion. Later, according to the optical characteristics in underwater imaging, Berman et al. [13] proposed to expand the haze-line model to address the issue of wavelength dependent attenuation (Uw-HL). But Uw-HL always made restored results appearing in an ‘over-red’ tone. Song et al. [14] proposed a new underwater dark channel prior restoration method (NUDCP), which made a great improvement both in enhancing contrast and removing color cast. However, since underwater images were always in an undesired exposure, NUDCP performed not well in recovering detail information. Besides, NUDCP also required long running time when processing large size inputs.

Non-physical model-based methods (underwater image enhancement methods) were always implemented by transforming pixel intensities in spatial/frequency domain, which aiming at correcting color-distortion, enhancing contrast, and maintaining detail-information. Li et al. [15] proposed a minimum information loss and histogram distribution prior enhancement approach, which could enhance contrast and remove color distortion of inputs, but easily introduced red artifacts into results. Fu et al. [16] proposed a ‘two-step’ strategy, but it easily caused results suffering extra noise. Ancuti et al. [5] proposed a multi-scale fusion strategy. It blended the gamma-corrected version and white-balanced version into fusion strategy, while enhanced results always appeared in an unrealistic greyish tone with low contrast. Recently, Marques et al. [17] proposed a low-light underwater image enhancement method (L²UWE) which also utilized the multi-scale fusion scheme to enhance underwater images. Although L²UWE performed well in maintaining detail-information, it caused outputs suffering too much artificial noise. Later, Zhuang et al. [18] developed a Bayesian retinex algorithm (BR), which overcame most disadvantages of previous techniques to a certain degree. But it could not clearly restore the detail-information, especially for images captured with artificial illuminatio.

With the rapid development of deep-learning techniques, superior achievements have been made in optical fileds including underwater optics. Iqbal et al. [19] proposed an unsupervised color-correction method (UCM), which easily caused dim regions of inputs being red over-compensated. Wang et al. [20] proposed the famous convolutional neural network (CNN) to enhance brightness and contrast of inputs. Then, Song et al. [21] proposed a rapid scene depth estimation model combining date-driven supervised linear regression to train the model coefficient. But undesired color cast still existed in their results. In more recently, Li et al. [22] proposed a novel underwater convolutional neural network model (UWCNN & UWCNN+), which required a highly trained dataset. Islam et al. [23] developed a fast underwater image enhancement method based on the Generative Adversarial Network (F-GAN). Although F-GAN could enhance underwater images without a lot training, it performed not well in inputs with uneven illumination.

In conclusion, current techniques were inadequate either in color correction or detail restoration, especially in dehazing underwater images with uneven illumination (artificial illumination). In this paper, we propose an effective solution for underwater image enhancement, which contains three main steps. First, we employ an adaptive-adjusted artificial multi-exposure fusion strategy (A-AMEF) and a parameter adaptive-adjusted local color correction (PAL-CC) operation respectively on the input. This could help us achieve a contrast enhanced version in the optimal exposure, and a color corrected version as a color reference image. Second, we use the famous guided filtering [24] on the contrast enhanced version, which aims at getting a base-layer and a detail-layer. And we utilize the color channel transfer operation to transfer color information from the color-corrected version to the base-layer. Last, we simply make a linear additivity of color corrected base-layer and detail-layer to reconstruct the enhanced result.

The technical contributions of this paper are summarized as follows:

1) we propose an effective solution for underwater image enhancement. The proposed solution could effectively improve the visual quality of different kinds of underwater images through our comprehensive validation both in the quantitative and qualitative evaluations.
2) considering the characteristics of underwater images with different exposures, we first propose an entropy-based algorithm to choose images with optimal exposure in 7 different value-ranges. Then we utilize ‘contrast’ and ‘well-exposedness’ as two weights, blending with the chosen versions into the multi-scale fusion strategy. A-AMEF could help us get a contrast-enhanced version with detail restored.
3) different from current global color correction methods, we first propose a novel color-channel attenuation map-based algorithm to find the dominated channel. Then inspired by the computation process of guided filtering, we propose a parameter adaptive adjusted color channel compensation method. Last, we utilize the traditional Gray-World assumption to get the color corrected version. Our PAL-CC removes color distortion accurately. Besides, it also addresses the issue of choosing the optimal color reference image in CCT process.
4) we also demonstrate the superiority of our solution in dehazing fogged images and improving accuracy of other optical applications such as: image segmentation and local feature point matching.

The rest of this paper is organized as follows: Section 2 introduces the motivation and basic knowledge of our solution. Section 3 introduces the process of our solution in details. Section 4 discusses the experimental results. In Section 5, we conclude our research and discuss the future direction.

2. Related work

In this section, we intend to expound the motivation of our solution first. And then, we introduce the basic knowledge of some technics applied in our solution, such as: artificial multi-exposure fusion, color correction, and guided filtering.

2.1 Motivation

To improve the visual quality of underwater images, we first summarize the deficiencies in underwater images as follows: 1) color attenuation, which makes images appearing in a bluish or greenish tone generally; 2) contrast degradation, which makes edge hardly being recognized, besides, it also makes underwater images less vivid; 3) improper exposure, which always makes detail information lost ether in bright or dim area.

Then we intend to address issues above separately. First, we want to enhance contrast of underwater images. Image contrast can be simply represented as follows:

(3)$$C(\Omega ) = I_{\max }^\Omega - I_{\min }^\Omega $$

where $\Omega $ signifies a given region in $I(x).$ In general, $I_{\max }^\Omega$ denotes the darkest pixels in the given region of the image. On the contrary, $I_{\min }^\Omega$ always denotes the brightest pixels. In other words, maintaining detail information in both dim and bright areas of inputs, which can also assist to enhance contrast. Therefore, the artificial multi-exposure fusion strategy (AMEF) could not only adjust inputs into the optimal exposure, but also enhance contrast. As for removing color distortion, an effective color correction should be operated on the inputs. Unfortunately, we have observed that simply making the multi-exposure fusion strategy after the color correction process, or making it in an inverse way, which could both severely degrade effects of the color correction. We therefore propose to use the multi-exposure fusion strategy and color correction operation on the input respectively. After obtaining two processed versions of inputs, we set the color-corrected version as the color reference image first. Then we separate the contrast-enhanced version into a base-layer and a detail-layer through guided filtering process and make a CCT process from the reference image to the base-layer. At last, we simply add the base-layer and detail layer together again to reconstruct the enhanced result. Through above operations, the enhanced result not only restores the lost detail-information, enhances contrast in an optimal exposure effectively, but also compensate the color attenuation accurately. The related observations are shown in Fig. 2.

Fig. 2. Comparison of discussed situations. (a) Input. (b) only A-AMEF. (c) only PAL-CC. (d) A-AMEF + PAL-CC. (e) PAL-CC + A-AMEF. (f) The proposed solution. (Best viewed at 390% zoom).

Download Full Size | PDF

2.2 Artificial multi-exposure fusion

Considering the advantages of multi-exposure fusion strategy, scientists began researching in artificial multi-exposure fusion strategy [25–28]. Interestingly, most of them were dedicated designed for dehazing regular fogged images. Recently, the most famous two approaches were probably AMEF [27] and fast multi-exposure fusion (FMEF) [28], they both got multi-exposure versions through the process expressed as follows:

(4)$${I_\gamma }(x,y) = I(x,y).\hat{} \gamma $$

where $\gamma $ is the positive constant, ${I_\gamma }(x,y)$ denotes the adjusted exposure version of the input $I(x,y).$ And $\gamma $ influences the exposure of output directly. In addition, considering the characteristics of regular fogged images: high luminance, degrading contrast and less saturation, both AMEF and FMEF aimed at generating under-exposure versions $(\gamma \ge 1).$ In AMEF, authors got five different exposure versions $({\gamma \textrm{ = }1,2,3,4,5} )$ through Eq. (4). Then they set ‘contrast’ and ‘saturation’ as two weight measures, fusing into the multi-scale fusion strategy through the process expressed follows:

(5)$${J_l}(x,y) = \sum\limits_{k = 1}^K {{G_l}\{ {{\bar{W}}_k}(x,y)\} {L_l}\{ {E_k}(x,y)\} } $$

where ${J_l}(x,y)$ denotes the ${l^{th}}$ level of fusion-result. And ${G_l}\{{\cdot} \}, {L_l}\{{\cdot} \} $ means Gaussian and Laplacian pyramid decomposition transformation respectively, which is processed on the ${l^{th}}$ level of normalized weights and inputs. The final fusion result $J(x,y)$ comes from reconstructing the Laplacian pyramid from the bottom level to the top level. And in FMEF, authors also got five different exposure versions $({\gamma \textrm{ = }1,1.2,3,4,8} )$ through Eq. (4). Then they put these versions into a guided filter, to get the correspondent base-layer and detail layer. They set quite amount parameter-values to define the weight measures, reconstructed dehazed result. Both AMEF and FMEF could perform well in dehazing fogged images. However, light condition was quite different for underwater images. The optimal values of $\gamma $ should be reset. Besides, since we just want to enhance contrast and recover details without changing the colorfulness, weight coefficients should be redesigned again. Thus, we develop our A-AMEF method to satisfy these requirements.

2.3 Color correction

Conventional color correction approaches for underwater images always refer to the Gray-Edge assumption [29], the Shades-of-Gray assumption [30], the Max-RGB assumption [31] and the Gray-World assumption [32]. These are designed for achieving the color constancy through dividing each channel by its corresponding normalized light source intensities. Although Gray-World assumption yields a color-corrected version for underwater image, it introduces undesired red artifacts due to the extremely large changing in red-channel mean-value of input. Considering the light absorption in underwater medium, scientists began to investigate other effective approaches. Ancuti et al. [33] proposed to use a Gray-World procedure, then employed an automatic contrast adjustment to correct color-cast. Fu et al. [34] proposed to utilize the mean-value and the mean-square-error to adjust the maximum and minimum values of each color respectively. But both [33] and [34] still made the color-corrected result existing the ‘red artifacts’ deficiency. Recently, Ancuti et al. [35] developed a color channel compensation method (3C), which can be expressed as follows:

(6)$$I_d^c(x,y) = {I_d}(x,y) + \alpha .({\bar{I}_D} - {\bar{I}_d})(1 - {I_d}(x,y)).{I_D}(x,y)$$

where $I_d^c(x,y)$ denoted the compensated degraded color. ${I_d}(x,y),$ ${I_D}(x,y)$ denoted the original degraded/ dominant color respectively. ${\bar{I}_D}$ signified the mean-value of dominant color, while ${\bar{I}_d}$ represented the mean-value of degraded color. And parameter $\alpha $ was usually set to 1. 3C was a global color compensation method, but corrected results still existed red over-compensated deficiency. Hence, ‘how to choose the dominant color channel appropriately?’ and ‘how to compensate the degraded color channel locally?’ have become two solutions for improving effectiveness of existing color correction approaches. We therefore propose our parameter adaptive-adjusted local color correction (PAL-CC) approach.

On the other hand, we develop to set the color-corrected image as the reference image preparing for color channel transfer operation (CCT) [36], which also addresses the issue discussed in the CCT-based approaches [37,38]: how to generate or choose the optimal color reference image. It is because that color correction operation is always at the cost of changing the pixel-values in RGB color space of inputs, the more complexity the operation, the higher degree pixel-values change. While CCT is used to ‘borrow’ the color-characteristics from the reference image through a transformation in the opponent color space (CIE-Lab) at the least cost above. And it can be expressed as follows:

(7)$$\begin{aligned} I_L^a(x,y) &= [{{I_L}(x,y) - \overline {{I_L}} } ]\times \frac{{\sigma _L^r}}{{{\sigma _L}}} + \overline {I_L^r} \\ I_a^a(x,y) &= [{{I_a}(x,y) - \overline {{I_a}} } ]\times \frac{{\sigma _a^r}}{{{\sigma _a}}} + \overline {I_a^r} \\ I_b^a(x,y) &= [{{I_b}(x,y) - \overline {{I_b}} } ]\times \frac{{\sigma _b^r}}{{{\sigma _b}}} + \overline {I_b^r} \end{aligned}$$

where $I_L^a(x,y),I_a^a(x,y),I_b^a(x,y)$ denotes the adjusted channel of corrected image respectively. ${I_L}(x,y),{I_a}(x,y),{I_b}(x,y)$ means the original channel respectively. $\overline {{I_L}} ,\overline {{I_a}} ,\overline {{I_b}} $ signifies the mean-values of original channels of inputs, while $\overline {I_L^r} ,\overline {I_a^r} ,\overline {I_b^r} $ represents mean-value of each channel $L,a,b$ of the reference image respectively. And ${\sigma _L},{\sigma _a},{\sigma _b}$ signifies the standard deviation of each channel of input respectively, while $\sigma _L^r,\sigma _a^r,\sigma _b^r$ defines each standard deviation of reference image in a similar way.

2.4 Guided filtering

As discussed above, we first intend to directly use CCT process on the contrast enhanced result from the A-AMEF. Although, this procedure could maintain the detail information to the utmost extent, it may introduce additional noise into the enhanced result. Inspired by FMAF [28], we propose to use the guided filtering [24] to separate the enhanced result into a smooth base-layer and a detail information containing detail-layer. Then we just put the CCT on the base-layer to remove color distortion accurately, while the detail-layer could still maintain the detail information. The guided filtering generates a filtering out as a linear transform of the guidance image. And if we assume the guidance image as the input itself. Then the base-layer could be achieved through the process as follows:

(8)$$B(x,y) = {G_{\gamma ,\varepsilon }}(I(x,y),I(x,y)), \forall (x,y) \in \omega $$

where $G\{{\cdot} \}$ denotes the guided filtering process. $\gamma $ is the radius of the given region $\omega ,\varepsilon $ controls blurriness. $I(x,y),B(x,y)$ is the input and output respectively. And the larger values of $\gamma $ and $\varepsilon $ are, the more blurring the output image is. So, we get the base-layer of contrast enhanced version through Eq. (8). Then the detail layer $D(x,y)$ could be achieved through simple process as follows:

(9) $$D(x,y) = I(x,y) - B(x,y)$$

3. Solution of this paper

The overview of the proposed solution is shown in Fig. 3.

Fig. 3. Overview of the proposed solution.

Download Full Size | PDF

Considering the characteristics of AMEF and FMEF, we propose our adaptive-adjusted artificial multi-exposure fusion strategy (A-AMEF). First, we intend to address the issues of how to choose the appropriate values of $\gamma $ in Eq. (4) to generate the optimal inputs. To our best knowledge, previous artificial multi-exposure fusion strategies always chose the values of $\gamma $ depending on the visual perception. Such source-inputs are not probably the optimal ones. Hence, we propose to choose the optimal values of $\gamma $ based on the entropy in different value-ranges. Entropy is a statistical measure of randomness that used to characterize the texture of an input image, which also represents the amount of information contained in input. Entropy can be calculated as follows:

(10)$$Entropy = \sum {{P_i}({I_i}) \times {{\log }_2}({I_i})} $$

where I signifies the gray-form of input image, P denotes the number of occurrences of gray-level $i(i \in [0,255]).$ The higher the entropy is, the larger amount of information the image includes. And according to the observation that when $\gamma > 6,$ image appears too dim to be recognized as shown in Fig. 4.

Fig. 4. Comparison of different values of $\gamma .$ (a) Input. (b) $\gamma = 0.2.$ (c) $\gamma = 0.5.$ (d) $\gamma = 2.$ (e) $\gamma = 3.$ (f) $\gamma = 4.$ (g) $\gamma = 5.$ (h) $\gamma = 6.$ (Best viewed at 390% zoom).

Download Full Size | PDF

We therefore set seven value-ranges as: (0, 0.5], (0.5, 1], (1, 2], (2, 3], (3, 4], (4, 5], and (5, 6]. Then we vary $\gamma $ in an increasement of 0.1 in each value-ranges getting 35 versions of different exposures. Finally, we calculate the entropy of each version and choose seven $\gamma $ with the highest value in each value-ranges. So far, we have gotten the adaptive-adjusted exposure inputs $\{{{E_{\gamma 1}},{E_{\gamma 2}},{E_{\gamma 3}},{E_{\gamma 4}},{E_{\gamma 5}},{E_{\gamma 6}},{E_{\gamma 7}}} \}$ preparing for the multi-exposure fusion process. Comparing to FMAF [28] which needed setting amounts of coefficients to get the optimal fusion result, we prefer AMEF [27] which simply employed multi-scale fusion strategy with appropriate weights. Since intending to enhance contrast and maintain details at the cost of changing color information as little as possible, we just set the ‘contrast’ and the ‘exposedness’ as two weights different from AMEF. Contrast weight $W_c^k$ can be simply signified as the absolute value of the response to a Laplacian-filter [39]. It can be expressed as follows:

(11)$$W_c^k(x,y) = \frac{{{\partial ^2}{E_k}}}{{\partial {x^2}}}(x) + \frac{{{\partial ^2}{E_k}}}{{\partial {y^2}}}(y)$$

Exposedness weight $W_e^k$ determines the amount of light being reflected in the $k - th$ input. $W_e^k$ can be defined as follows:

(12)$$W_e^k(x,y) = {\exp ^{\frac{{ - 0.5(E_k^c(x,y) - \beta )}}{{{\sigma ^2}}}}}$$

where the standard deviation $\sigma $ and the illumination $\beta $ could be set to 0.25 and 0.5 from the conclusion of [40]. And c denotes each $R,G,B$ color channel respectively. Then the sum weight ${W_k}$ is defined by simply combining multiplicatively $W_c^k$ and $W_e^k$ for each input ${E_k}(x,y)$. And a normalized operation is processed on ${W_k}$ to get the ${\overline W _k}$, which is used for making sure intensities of the fusion result in range. ${W_k}$ can be obtained as follows:

(13)$${W_k}(x,y) = W_c^k(x,y) \times W_e^k(x,y)$$

Finally, we get the fusion result by put the ${\overline W _k}$ and ${E_k}(x,y)$ into Eq. (5). And the differences between AMEF, FMEF, and A-AMEF can be shown in Fig. 5. A-AMEF could achieve our original intention well.

Fig. 5. Comparison of different artificial multi-exposure fusion strategies. (a) Input. (b) AMEF. (c) FMEF. (d) The proposed A-AMEF.

Download Full Size | PDF

3.1 PAL-CC

To correct color distortion, we develop a parameter adaptive-adjusted local color correction (PAL-CC) approach. First, we intend to propose a simple algorithm to choose the dominant color channel rationally. Inspired by [41], light intensities of each channel should be equal to the directly reflected from the object ones without attenuation. Hence, the attenuation map of each color channel could be simply expressed as follows:

(14)$${A^c}(x,y) = 1 - {I^c}(x,y)$$

where ${A^c}(x,y)$ denotes the attenuation map of channel $c.$ ${I^c}(x,y)$ represents each original color channel. So, the dominant color channel should be the one with the minimum mean-value of ${A^c}(x,y),$ which signifies the least degradation. Since red color with the longest wavelength among $R,G,B,$ we just need to choose one with the smaller mean-value of ${A^c}(x,y)$ from blue and green. And the rest two colors can be both defined as degraded ones. After defining the dominant channel, we intend to solve the remaining problem: how to compensate the degraded channel locally?

Fortunately, compensation algorithm proposed by 3C [35] is well-recognized by numerous researchers. We intend to solve this problem based on Eq. (6). Compensating the degraded color locally which equals to achieving adaptive value of $\alpha $ in each pixel $(x,y).$ Inspired by parameter adjusted color correction method (PAC) [42], we also introduce the guided filtering into this procedure. The difference value of $I_d^c(x,y)$ and ${I_d}(x,y)$ can be obtained as follows:

(15)$$\begin{aligned}\Delta {I_d}(x,y) &= I_d^c(x,y) - {I_d}(x,y)\\ &= (\overline {{I_D}} - \overline {{I_d}} ) \times [{a_\Omega }(x,y) \times {I_D}(x,y) - {a_\Omega }(x,y) \times {I_d}(x,y) \times {I_D}(x,y)] \end{aligned}$$

where $(\overline {{I_D}} - \overline {{I_d}} )$ can be a constant in sliding window as region $\Omega .$ Then the rest of computational process could ignore $(\overline {{I_D}} - \overline {{I_d}} )$ temporarily. And we define a new function $p(x,y)$ as follows:

(16)$$\begin{aligned} p(x,y) &= {a_\Omega }(x,y) \times {I_D}(x,y) - {a_\Omega }(x,y) \times {I_d}(x,y) \times {I_D}(x,y)\\ &= {a_\Omega }(x,y) \times {I_D}(x,y) + {b_\Omega }(x,y) \end{aligned}$$

where ${a_\Omega }(x,y)$ and ${b_\Omega }(x,y)$ are two parameters changing in region $\Omega .$ In other words, achieving each value of ${a_\Omega }(x,y),{b_\Omega }(x,y)$ which means we could compensate the degraded color channel locally. Interestingly, solving the problem of Eq. (16) can be seen as solving the guided filtering problem in [24]. Here, we define ${I_D}(x,y)$ as the guided image, while ${I_d}(x,y)$ and $p(x,y)$ defined as the input channel and the output. Then, according to the computational process in [24], ${a_\Omega }(x,y)$ and ${b_\Omega }(x,y)$ can be obtained by minimizing the cost function between $p(x,y)$ and ${I_d}(x,y),$ which can be expressed as follows:

(17)$$E[{a_\Omega }(x,y),{b_\Omega }(x,y)] ={\parallel} [{a_\Omega }(x,y) \times {I_D}(x,y) + {b_\Omega }(x,y) - {I_d}(x,y)] + \varepsilon \times {a_\Omega }(x,y)\parallel _F^2$$

where ${\parallel}{\cdot} {\parallel _F}$ denotes the Frobenius norm, $\varepsilon $ is set to 0.1 avoiding ${a_\Omega }(x,y)$ getting too large. Then the linear regression solutions to Eq. (17) can be expressed as follows [24]:

(18)$$\begin{aligned} {a_\Omega }(x,y) &= \frac{{\frac{1}{{|n |}}\sum\nolimits_{(x,y) \in \Omega } {{I_D}(x,y) \times } {I_d}(x,y) - \overline {{I_{D\Omega }}} \times \overline {{I_{d\Omega }}} }}{{{\sigma ^2} + 0.1}}\\ {b_\Omega }(x,y) &= \overline {{I_d}} - {a_\Omega }(x,y) \times \overline {{I_D}} \end{aligned}$$

where ${\sigma ^2},|n |$ denotes the variance of ${I_D}(x,y)$ and number of pixels in region $\Omega $ respectively. Since aiming at compensating degraded color locally, we set the region $\Omega $ at the minimum size of 2×2, that $|n |= 2.$ Since $\Omega $ could cover one pixel ${|n |^2}$ times, when sliding over the whole image, which can be observed in Fig. 6.

Fig. 6. The relationship between $a(x,y)$ and ${a_\Omega }(x,y)$ when $|n |\textrm{ = }2$.

Download Full Size | PDF

Therefore, the final $a(x,y)$ could be obtained as follows:

(19)$$a(x,y) = \frac{1}{{{{|n |}^2}}}\sum\limits_{(x,y) \in \Omega } {{a_\Omega }(x,y)} \textrm{ = }\frac{1}{4} \times [{{a_{\Omega \textrm{ = }1}} + {a_{\Omega \textrm{ = 2}}} + {a_{\Omega \textrm{ = 3}}} + {a_{\Omega \textrm{ = 4}}}} ]$$

Now, we could compensate two degraded channels from the dominant channel through above processes. And last, we utilize the famous Gray-World assumption (GW) on the color-compensated image to generate the color-corrected result. To demonstrate the superiority of our PAL-CC, we compare with the corrected results obtained from some state-of-the-art techniques viz. 3C [35], PAC [42], PAC & GW, reference image constructed CCT (C-CCT) [37], reference image selected CCT (S-CCT) [38], and our PAL-CC. The comparison is shown in Fig. 7. Transmission map estimated by DCP [10] of each image in Fig. 7 is shown in Fig. 8.

Fig. 7. Comparison of the results obtained from different color correction techniques. (a) Input. (b) 3C. (c) PAC. (d) PAC & GW. (e) S-CCT. (f) C-CCT. (g) The proposed PAL-CC. (Best viewed at 390% zoom).

Download Full Size | PDF

Fig. 8. Comparison of the transmission estimation maps obtained from different color correction techniques based on DCP [10]. (a) Input. (b) 3C. (c) PAC. (d) PAC & GW. (e) S-CCT. (f) C-CCT. (g) The proposed PAL-CC. (Best viewed at 390% zoom).

Download Full Size | PDF

As shown in Fig. 7, since prior selected reference image is unlikely suitable for all kinds of inputs, S-CCT [38] could not remove color distortion for all inputs accurately obviously. Also, PAC is just a pre-processing operation, results directly obtained from it need to be further processed. The rest techniques could correct color distortion to a certain extent. But 3C [35] always yields a rather dim version while C-CCT [37] often generates a too bright one, both reduce contrast. Only PAC & GW and PAL-CC could effectively remove color distortion. But comparing to PAC & GW, our PAL-CC could yield more vivid versions from all four inputs. Additionally, Fig. 8 also proves the improvement in estimating transmission from our PAL-CC based on the DCP technique [10].

3.2 Reconstruction

Last, we take the contrast-enhanced version into the guided filter to generate a detail-layer $D(x,y)$ and a base-layer $B(x,y)$ through Eq. (8) and Eq. (9). To maintain the detail information as much as possible, we make such a comparison (shown in Fig. 9) of detail maintaining effect of detail-layer from different value-setting of $\gamma $ and $\varepsilon .$ As can be observed, when $\gamma \textrm{ = 8}$ and $\varepsilon \textrm{ = }0.4,$ the generated detail-layer contains sufficient edge information to reconstruct the final result well, which also means that the obtained base-layer could be color-corrected at the least cost of degrading details. Therefore, we decide to set $\gamma \textrm{ = 8}$ and $\varepsilon \textrm{ = }0.4$ as the optimal parameter settings in Eq. (8). After that, we define the color corrected version as the reference image and utilize the CCT process to get the final corrected base-layer through Eq. (7). Last, we simply make a linear additivity of the corrected base-layer ${B^c}(x,y)$ and detail-layer $D(x,y)$ to reconstruct the final enhanced result ${I^E}(x,y),$ which can be expressed as follows:

(20)$${I^E}(x,y) = {B^c}(x,y) + D(x,y)$$

Fig. 9. Comparison of the detail maintaining effect of detail-layer from different value-setting of $\gamma $ and $\varepsilon .$ (a) $\gamma = 2.$ (b) $\gamma = 4.$ (c) $\gamma = 8.$ Row 1. $\varepsilon = 0.1.$ Row 2. $\varepsilon = 0.2.$ Row 3. $\varepsilon = 0.4.$ (Best viewed at 390% zoom).

Download Full Size | PDF

4. Experiments and results

In this section, the improvement in effectiveness, computational complexity, and robustness of the proposed solution is demonstrated on the various kinds of underwater images. Additionally, we also demonstrate other applications of our solution such as: dehazing regular fogged images, improving accuracy of key-points matching and image segmentation.

4.1 Experiment preparation

Input images: To prove the effectiveness and robustness of our solution, our comprehensive experiments are conducted on images from different datasets.

1) Underwater image enhancement benchmark (UIEB) [43] : A total of 850 underwater images with their corresponding well-reconstructed reference images. We randomly choose 50 pairs of them to testify the improvement of our solution comparing to some current techniques.
2) Real-world underwater image enhancement dataset (RUIE) [44] : A total of 4230 underwater images, which are divided into 8 kinds of datasets viz. green cast, green-blue cast, blue cast, and ‘A-E’ degree visual quality datasets. ‘A’ is the best, while ‘E’ is the worst. We randomly choose 10 images from each dataset (a total of 80 images) to demonstrate the robustness and effectiveness of our solution on all 8 kinds of degraded images.
3) Real-world underwater diving scenes dataset: A total of 50 underwater diving images captured and provided by Liaoning Port Group Co., Ltd, which are used to demonstrate the practical applicated results of the proposed solution. Besides, we also choose 5 images from this dataset, which captured in extremely turbid medium with artificial illumination, to further verify the practical effects of our solution.
4) Non-homogeneous hazy and haze-free image dataset (NH-HAZE) [45] : A total of 55 fogged images, which also have their corresponding ground-truth reference images. We utilize all pairs of them, to show the superiority of our solution in dehazing fogged images.

Comparative candidates: To make sure the processed results reliable, we choose the comparative techniques for following two reasons: 1) techniques are proposed in recent years, 2) techniques can be operated through codes provided by their authors themselves. 6 comparative techniques for dehazing underwater single image are shown in Table 1. And Table 2 shows 4 comparative techniques for dehazing fogged single image.

Table 1. Comparative underwater image dehazing methods

View Table | View all tables in this article

Table 2. Comparative fogged image dehazing techniques

View Table | View all tables in this article

Quantitative Evaluations: To evaluate results comprehensively, we utilize both the non-reference quantitative evaluations metrics and full-reference ones to prove the improvement of our solution.

Non-reference evaluations:

1) Underwater color image quality evaluation (UCIQE) [48]: A linear combination of chroma, saturation and contrast. It is proposed to quantify the nonuniform color cast, blurring, and low contrast that characterize underwater engineering and monitoring images. Higher value of UCIQE means better visual quality which evaluated image appears.
2) Underwater image quality measure (UIQM) [49] : A linear combination of underwater image colorfulness measure (UICM), underwater image sharpness measure (UISM), and underwater image contrast measure (UIConM), which is dedicated used to evaluate the visual quality of underwater images. Higher value of UIQM also means better visual quality which evaluated image appears.
3) Nature image quality evaluator (NIQE) [50]: Constructed based on visual perception to high contrast regions in image. It employs multi-variate gaussian (MVG) to estimate the feature model of sensitive areas. And lower value of NIQE means better visual quality which evaluated image appears.
4) Entropy: A statistical measure of randomness that can be used to characterize the texture of an input image, which also represents the amount of information contained in input. Higher value of entropy means more information which evaluated image carries.

Full-reference evaluations:

1) Structural similarity index (SSIM) [51]: A combination of luminance, contrast, and structure comparison between two images. It can be calculated through a pooling strategy. And higher value of SSIM means less differences between two images.
2) Peak signal-to-noise ratio (PSNR) [52]: PSNR indicates the energy ratio of signal and noise. And higher PSNR means that evaluated image with less extra noise.

Other techniques: As introduced above, our solution is also suitable for image segmentation and local feature points matching. We therefore employ a scale invariant feature transform (SIFT) [53] operator to compare the valid key-point matching number from images processed by comparative underwater dehazing image techniques. Meanwhile, we also employ an image segmentation algorithm: fast fuzzy c-means clustering (FCM) [54] to compare the effects from comparative underwater image dehazing techniques.

Experiment environment: All the experiments are programmed in Python 3.7.5 or Matlab R2021a (student use) on an Intel i9-9900k CPU@ 3.60 GHz laptop with 32 GB RAM.

4.2 Comparison in dehazing underwater images

We first show the comparison of results obtained from comparative techniques and our solution on some images from UIEB dataset [43] in Fig. 10. And the corresponding transmission maps estimated by DCP [10] are shown in Fig. 11. Although L²UWE maintains details well in dehazing underwater images, it fails in removing color distortion. Besides, it introduces artificial noise into results. Transmission maps estimation obtained from L²UWE shown in Fig. 11 also demonstrates the deficiencies. Obviously, simply employ UWCNN+ in dehazing underwater images without prior training, which hardly improves visual quality of inputs. It is also the main disadvantage for CNN-based techniques, which limits their practical application severely. On the contrary, GAN model-based technique FGAN-UP overcomes this disadvantage to a certain extent, but its performance still falls behind than some comparative techniques. NUDCP and Uw-HL could not remove color distortion accurately. Besides, NUDCP always makes result contrast degradation, while Uw-HL often makes results too dim. BR performs well in dehazing underwater images in general, but our solution performs better in accurately removing color cast, enhancing contrast, and restoring details. Results obtained from our solution appear more vivid in all six images. In addition, transmission map estimation from our solution shows more accurate among comparative techniques, even better than some reference images provided by UIEB.

Fig. 10. Comparison of the dehazed results obtained from comparative techniques. (a) Input. (b) L²UWE. (c) UWCNN+. (d) NUDCP. (e) FGAN-UP. (f) Uw-HL. (g) BR. (h) The proposed solution. (i) Reference images provided by UIEB. (Best viewed at 390% zoom).

Download Full Size | PDF

Fig. 11. Comparison of the transmission estimation maps obtained from comparative techniques based on DCP [10] of Fig. 10. (a) Input. (b) L²UWE. (c) UWCNN+. (d) NUDCP. (e) FGAN-UP. (f) Uw-HL. (g) BR. (h) The proposed solution. (i) Reference images provided by UIEB. (Best viewed at 390% zoom).

Download Full Size | PDF

Table 3 shows the quantitative evaluation results of Fig. 10 in terms of UCIQE, UIQM, entropy, PSNR, and SSIM. Additionally, it also provides the quantitative evaluation average results in terms of above metrics of 50 underwater images from UIEB dataset. As can be observed, our solution achieves the top three in terms of UIQM, entropy, and PSNR metrics of all six images, while in UCIQE and SSIM metrics, our solution ranks even better. The average results of our solution ranks 1^st in terms of all evaluation indexes, whether of the demonstrated six images or 50 extra images from UIEB, which strongly proves the superiority.

Table 3. Comparison of the UCIQE, UIQM, Entropy, PSNR, and SSIM of Fig. 10.

View Table | View all tables in this article

We then employ different kinds of images from RUIE dataset [44] to test the robustness of comparative techniques and our solution. Figure 12 shows comparison of different techniques effecting on some examples in different sub-datasets of RUIE. Table 4 shows the quantitative evaluation results in terms of UCIQE, UIQM, an entropy of Fig. 12. Also, Table 4 provides the average results in terms of above evaluation indexes of a total of 80 (10 images in each sub-dataset) images from RUIE dataset.

Fig. 12. Comparison of the dehazed results obtained from comparative techniques on RUIE dataset. (a) Input. (b) L²UWE. (c) UWCNN+. (d) NUDCP. (e) FGAN-UP. (f) Uw-HL. (g) BR. (h) The proposed solution. Row 1 to 8 are images with different classified degrees in RUIE. Row 1. ‘A’ visual quality (best). Row 2. ‘B’ visual quality. Row 3. ‘C’ visual quality. Row 4. ‘D’ visual quality. Row 5. ‘E’ visual quality (worst). Row 6. Blue color cast. Row 7. Blue-Green color cast. Row 8. Green color cast. (Best viewed at 390% zoom).

Download Full Size | PDF

Table 4. Comparison of the UCIQE, UIQM, and entropy of Fig. 12.

View Table | View all tables in this article

L²UWE still does not correct color distortion accurately but introduce extra noise. Worse more, image 2 and image 6 processed by L²UWE, which perform in a rather low contrast appearance. The deep-learning based techniques: UWCNN+ and FGAN-UP perform not well this time. And NUDCP, Uw-HL, and BR could not remove color cast accurately. Images obtained from our solution perform better visual quality than other techniques generally, whether in contrast enhancement, detail restoration, or color correction. The corresponding quantitative evaluation results in terms of UCIQE, UIQM, and entropy also proves the robustness and effectiveness of our solution for enhancing different kinds of inputs.

For the sake of further testifying the superiority of our solution in practical applications, we utilize underwater diving images to compare the dehazing effects of comparative techniques in Fig. 13. Also, the corresponding quantitative results in terms of UCIQE, UIQM, and entropy are shown in Table 5. Besides, we also choose 5 images from our diving scenes dataset to show the dehazing ability of our solution for inputs with extremely uneven illumination, which are shown in Fig. 14. The quantitative results of Fig. 14 are shown in Table 6. No matter the qualitative results or the quantitative results above, which both demonstrate the better performance of our solution. So, we conclude that the proposed method is a more effective solution for underwater image enhancement than comparative techniques.

Fig. 13. Comparison of the dehazed results obtained from comparative techniques on diving scene dataset. (a) Input. (b) L²UWE. (c) UWCNN+. (d) NUDCP. (e) FGAN-UP. (f) Uw-HL. (g) BR. (h) The proposed solution. (Best viewed at 390% zoom).

Download Full Size | PDF

Fig. 14. Comparison of the dehazed results obtained from comparative techniques on extremely illumination images. (a) Input. (b) L²UWE. (c) UWCNN+. (d) NUDCP. (e) FGAN-UP. (f) Uw-HL. (g) BR. (h) The proposed solution. (Best viewed at 390% zoom).

Download Full Size | PDF

Table 5. Comparison of the UCIQE, UIQM, and entropy of Fig. 13.

View Table | View all tables in this article

Table 6. Comparison of the UCIQE, UIQM, and entropy of Fig. 14.

View Table | View all tables in this article

4.3 Computational efficiency comparison

When processing underwater images with the size of $1080 \times 1080,800 \times 800,400 \times 400,$ the proposed solution takes about average 7.0624 seconds (s), 3.5037 (s), and 1.2336 (s), which is shorter than the L²UWE, UWCNN+, NUDCP, and Uw-HL with each size. But the average running time of BR and FGAN-UP are shorter than ours with each size respectively. Interestingly, when processing underwater images with the size of $256 \times 256$ and $128 \times 128$, our solution takes less time than BR and FGAN-UP but longer time than NUDCP. Since FGAN-UP first adjusts input to the size of $256 \times 256$ and takes average running time 0.5613 (s) with any size of input. Additionally, it is worth mentioning that the Matlab code of our solution is not optimized to its best programming formation. We believe that the running time of our solution could be shorter by professional optimization, which could also expand the area of our solution for practical applications. The details of computational efficiency comparison results are shown in Table 7.

Table 7. Comparison of computational efficiency of comparative techniques on inputs with different sizes.

View Table | View all tables in this article

4.4 Other applications

Although our solution is dedicated designed for dehazing underwater images, it performs well in dehazing regular fogged images too. To demonstrate the effectiveness of the proposed solution, we employ images from NH-HAZE dataset [45] to compare the dehazed results obtained from AMEF [17], FMEF [28], MOFD [46], FFA-Net [47], and our solution. Figure 15 shows the comparison results obtained from comparative techniques. Table 8 shows the corresponding quantitative evaluation results of Fig. 15 in terms of NIQE, entropy, SSIM, and PSNR. Table 8 also provides the average evaluation results in terms of above metrics of all 55 image pairs from NH-HAZE dataset.

Fig. 15. Comparison of the dehazed results obtained from comparative techniques on NH-HAZE dataset. (a) Input. (b) AMEF. (c) FMEF. (d) MOFD. (e) FFA-Net. (f) The proposed solution. (g) Reference image provided by NH-HAZE. (Best viewed at 390% zoom).

Download Full Size | PDF

Table 8. Comparison of the NIQE, entropy, SSIM, and PSNR of Fig. 15.

View Table | View all tables in this article

As can be observed that dehazed results obtained from AMEF and FMEF still suffer from contrast degradation. Worse more, detail information is restored little too. While FFA-Net makes dehazed result performing in an undesired dim appearance. Only MOFD and our solution could effectively enhance contrast and maintain detail information from hazed inputs. However, MOFD makes dehazed results appearing in an unrealistic bluish tone. In conclusion, the dehazed results obtained from our solution perform better visual quality than comparative techniques in all 5 example pairs. The average evaluation results in terms of NIQE, entropy, SSIM and PSNR of these 5 pair images and all 55 pairs from NH-HAZE dataset, which both demonstrate the improvement of our solution.

In addition, our solution also makes improvement in image segmentation. Image segmentation is an important topic in optics, which aims at dividing input image into some disjoint and homogeneous areas according to some target characteristics. This time, we employ FCM algorithm [54] to compare the effects from comparative underwater image dehazing techniques. Figure 16 shows that segmentation results are more consistent when inputs are processed by our solution.

Fig. 16. Comparison of the segmentation results based on the FCM technique obtained from images processed by comparative techniques. (a) Input. (b) L²UWE. (c) UWCNN+. (d) NUDCP. (e) FGAN-UP. (f) Uw-HL. (g) BR. (h) The proposed solution. (Best viewed at 390% zoom).

Download Full Size | PDF

Meanwhile our solution is also suitable for improving accuracy of local feature points matching task. We employ the SIFT operator [53] to compare the valid key-point matching numbers from images processed by comparative underwater dehazing image techniques. Figure 17 and Table 9 show the significant improvement of our solution working in this field.

Fig. 17. Comparison of the valid key-point matching number based on the SIFT operator obtained from images processed by comparative techniques. (a) Input. (b) L²UWE. (c) UWCNN+. (d) NUDCP. (e) FGAN-UP. (f) Uw-HL. (g) BR. (h) The proposed solution. (Best viewed at 390% zoom).

Download Full Size | PDF

Table 9. Comparison of the valid matching numbers of Fig. 17.

View Table | View all tables in this article

5. Conclusion

The research of underwater optics plays an important role in understanding and exploring underwater world for people. So, ‘how to effectively improve the visual quality of underwater image’ comes to be a fundamental but vital issue needing to be addressed. This paper introduces an effective solution for underwater images, which can be roughly divided into three steps. First, an adaptive-adjusted artificial multi-exposure fusion (A-AMEF) and a parameter adaptive-adjusted local color correction (PAL-CC) are both operated on the input, which could generate a contrast-enhanced image and a color-corrected image. Then the famous guided filtering process is operated on the contrast-enhanced image which aims at yielding a smooth base-layer and a detail containing detail-layer. After that, color channel transfer (CCT) is operated on the base-layer from the color-corrected image. Last, simply making a linear additivity of the color-corrected base layer and the detail layer to reconstruct final enhanced output. Comparing to some current image dehazing techniques, the proposed method could generate outputs with better visual quality through our comprehensive validation of both quantitative and qualitative evaluations. It is worth mentioning that the proposed solution is not only suitable for dehazing underwater images, but also for dehazing regular images. Besides that, the proposed solution also can be utilized for improving accuracy of image segmentation and local feature points matching, which also makes some extra contribution to optical application research.

Although the proposed solution has achieved good performance, it has limitations too. When the input is in extremely small size, the proposed solution could blur some distant details sometimes. Thus, we intend to continue our research for proposing a more robust solution for dehazing underwater and fogged images.

Funding

National Key Research and Development Program of China (2019YFB1600400); National Natural Science Foundation of China (61701069); Fundamental Research Funds for the Central Universities (3132019200, 3132019340).

Acknowledgements

We would like to thanks to Liaoning Port Group., Ltd that provides us some real-world underwater diving scene images to complete the related experiments of the proposed solution. We also thanks to the anonymous reviewers for their insightful comments on this article.

Disclosures

We declare that there are no conflicts of interest related to this article.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. K. Liu and Y. Liang, “Underwater image enhancement method based on adaptive attenuation-curve prior,” Opt. Express 29(7), 10321–10345 (2021). [CrossRef]

2. Z. Chen, N. Qiu, H. Song, L. Xu, and Y. Xiong, “Optically guided level set for underwater object segmentation,” Opt. Express 27(6), 8819–8837 (2019). [CrossRef]

3. Q. Liu and M. P. Fok, “Bio-inspired photonics - marine hatchetfish camouflage strategies for RF steganography,” Opt. Express 29(2), 2587–2596 (2021). [CrossRef]

4. H. W. Wijesekera, W. S. Pegau, and T. J. Boyd, “Effect of surface waves on the irradiance distribution in the upper ocean,” Opt. Express 13(23), 9257–9264 (2005). [CrossRef]

5. C. O. Ancuti, C. Ancuti, C. De Vleeschouwer, and P. Bekaert, “Color Balance and Fusion for Underwater Image Enhancement,” IEEE Trans. on Image Process 27(1), 379–393 (2018). [CrossRef]

6. B. L. McGlamery, “A Computer Model For Underwater Camera Systems,” Proc. SPIE 0208, 221–231 (1980). [CrossRef]

7. F. Liu, Y. Wei, P. Han, K. Yang, L. Bai, and X. Shao, “Polarization-based exploration for clear underwater vision in natural illumination,” Opt. Express 27(3), 3629–3641 (2019). [CrossRef]

8. K. O. Amer, M. Elbouz, A. Alfalou, C. Brosseau, and J. Hajjami, “Enhancing underwater optical imaging by using a low-pass polarization filter,” Opt. Express 27(2), 621–643 (2019). [CrossRef]

9. E. Costa, “THE PROGRESS of SURVEY TECHNIQUES in UNDERWATER SITES: The CASE STUDY of CAPE STOBA SHIPWRECK,” presented at annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Cyprus, 2-3 May. 2019.

10. K. He, J. Sun, and X. Tang, “Single image haze removal using dark channel prior,” IEEE Trans. Pattern Anal. Mach. Intell 33(12), 2341–2353 (2011). [CrossRef]

11. H.-Y. Yang, P.-Y. Chen, C.-C. Huang, Y.-Z. Zhuang, and Y.-H. Shiau, “Low complexity underwater image enhancement based on dark channel prior,” in Proceedings of International Conference on Innovations in Bio-inspired Computing and Applications (IBICA, 2011), pp. 17–20.

12. P. L. J. Drews, E. R. Nascimento, S. S. C. Botelho, and M. F. M. Campos, “Underwater depth estimation and image restoration based on single images,” IEEE Comput. Grap. Appl. 36(2), 24–35 (2016). [CrossRef]

13. D. Berman, D. Levy, S. Avidan, and T. Treibitz, “Underwater Single Image Color Restoration Using Haze-Lines and a New Quantitative Dataset,” IEEE Transactions on Pattern Analysis and Machine Intelligence 43, 2822–2837 (2021). [CrossRef]

14. W. Song, Y. Wang, D. Huang, A. Liotta, and C. Perra, “Enhancement of Underwater Images with Statistical Model of Background Light and Optimization of Transmission Map,” IEEE Trans. Broadcast. 66(1), 153–169 (2020). [CrossRef]

15. C.-Y. Li, J.-C. Guo, R.-M. Cong, Y.-W. Pang, and B. Wang, “Underwater image enhancement by Dehazing with minimum information loss and histogram distribution prior,” IEEE Trans. on Image Process 25(12), 5664–5677 (2016). [CrossRef]

16. X. Fu, Z. Fan, M. Ling, Y. Huang, and X. Ding, “Two-step approach for single underwater image enhancement,” in Proceedings of International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS, 2017), pp. 789–794.

17. T. P. Marques and A. Branzan Albu, “L2UWE: A framework for the efficient enhancement of low-light underwater images using local contrast and multi-scale fusion,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2020), pp. 2286–2295.

18. P. Zhuang, C. Li, and J. Wu, “Bayesian retinex underwater image enhancement,” Engineering Applications of Artificial Intelligence 101, 104171 (2021). [CrossRef]

19. K. Iqbal, M. Odetayo, A. James, R. A. Salam, and A. Z. H. Talib, “Enhancing the low quality images using unsupervised colour correction method,” in Proceedings of IEEE Conference on Systems, Man and Cybernetics (IEEE, 2010). pp. 1703–1709.

20. Y. Wang, J. Zhang, Y. Cao, and Z. Wang, “A deep CNN method for underwater image enhancement,” in Proceedings of IEEE Conference on Image Processing (IEEE, 2017), pp. 1382–1386.

21. W. Song, Y. Wang, D. Huang, and D. Tjondronegoro, “A rapid scene depth estimation model based on underwater light attenuation prior for underwater image restoration,” Proc. SPIE 11164, 678–688 (2018). [CrossRef]

22. C. Li, S. Anwar, and F. Porikli, “Underwater scene prior inspired deep underwater image and video enhancement,” Pattern Recognition 98, 107038 (2020). [CrossRef]

23. M. J. Islam, Y. Xia, and J. Sattar, “Fast Underwater Image Enhancement for Improved Visual Perception,” IEEE Robot. Autom. Lett. 5(2), 3227–3234 (2020). [CrossRef]

24. K. He, J. Sun, and X. Tang, “Guided image filtering,” IEEE Trans. Pattern Anal. Mach. Intell. 35(6), 1397–1409 (2013). [CrossRef]

25. Y. Que, W. Wan, and H. J. Lee, “Multi-exposure image fusion based on sparsity blur feature,” Proc. SPIE 10836, 1083604 (2018). [CrossRef]

26. T. Li, K. Xie, T. Li, Y. Yan, and L. Huang, “Multi-exposure image fusion based on recursive filter,” J. Phys.: Conf. Ser. 1693, 012154 (2020). [CrossRef]

27. A. Galdran, “Image dehazing by artificial multiple-exposure image fusion,” Signal Processing 149, 135–147 (2018). [CrossRef]

28. Z. Zhu, H. Wei, G. Hu, Y. Li, G. Qi, and N. Mazur, “A Novel Fast Single Image Dehazing Algorithm Based on Artificial Multiexposure Image Fusion,” IEEE Trans. Instrum. Meas. 70, 1–23 (2021). [CrossRef]

29. J. van de Weijer, T. Gevers, and A. Gijsenij, “Edge-based color constancy,” IEEE Trans. on Image Process 16(9), 2207–2214 (2007). [CrossRef]

30. G. D. Finlayson and E. Trezzi, “Shades of gray and colour constancy,” in Proceedings of Color Imaging Conference on Color Science and Engineering: Systems, Technologies, Applications. (Society for Imaging Science and Technology, 2004), pp. 37–41.

31. D. Marini and A. Rizzi, “A computational approach to color illusions,” in Proceedings of International Conference on Image Analysis and Processing. (Springer Verlag, 1997), pp. 62–69.

32. G. Buchsbaum, “SPATIAL PROCESSOR MODEL FOR OBJECT COLOUR PERCEPTION,” J. Franklin Inst. 310(1), 1–26 (1980). [CrossRef]

33. C. Ancuti, C. O. Ancuti, T. Haber, and P. Bekaert, “Enhancing underwater images and videos by fusion,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 81–88.

34. X. Fu, P. Zhuang, Y. Huang, Y. Liao, X.-P. Zhang, and X. Ding, “A retinex-based enhancing approach for single underwater image,” in Proceedings of IEEE Conference on Image Processing (IEEE, 2014), pp. 4572–4576.

35. C. O. Ancuti, C. Ancuti, C. De Vleeschouwer, and M. Sbert, “Color Channel Compensation (3C): A Fundamental Pre-Processing Step for Image Enhancement,” IEEE Trans. on Image Process. 29, 2653–2665 (2020). [CrossRef]

36. E. Reinhard, M. Ashikhmin, B. Gooch, and P. Shirley, “Color transfer between images,” IEEE Comput. Grap. Appl. 21(4), 34–41 (2001). [CrossRef]

37. C. O. Ancuti, C. Ancuti, C. De Vleeschouwer, L. Neumann, and R. Garcia, “Color transfer for underwater dehazing and depth estimation,” in Proceedings of IEEE Conference on Image Processing (IEEE, 2017), pp. 695–699.

38. C. O. Ancuti, C. Ancuti, C. De Vleeschouwer, and M. Sbetr, “Color Channel Transfer for Image Dehazing,” IEEE Signal Process. Lett 26(9), 1413–1417 (2019). [CrossRef]

39. T. Mertens, J. Kautz, and F. Van Reeth, “Exposure fusion,” in Proceedings of 15th Pacific Conference on Computer Graphics and Applications (IEEE, 2007), pp. 382–390.

40. X. Fu, D. Zeng, Y. Huang, Y. Liao, X. Ding, and J. Paisley, “A fusion-based enhancing method for weakly illuminated images,” Signal Processing 129, 82–96 (2016). [CrossRef]

41. C. O. Ancuti, C. Ancuti, C. D. Vleeschouwer, and R. Garcia, “Locally Adaptive Color Correction for Underwater Image Dehazing and Matching,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops (IEEE, 2017), pp. 997–1005.

42. J. Liu and X. Zhang, “Parameter-Adaptive Compensation (PAC) for Processing Underwater Selective Absorption,” IEEE Signal Process. Lett 27, 2178–2182 (2020). [CrossRef]

43. C. Li, C. Guo, W. Ren, R. Cong, J. Hou, S. Kwong, and D. Tao, “An Underwater Image Enhancement Benchmark Dataset and beyond,” IEEE Trans. on Image Process 29, 4376–4389 (2020). [CrossRef]

44. R. Liu, X. Fan, M. Zhu, M. Hou, and Z. Luo, “Real-World Underwater Enhancement: Challenges, Benchmarks, and Solutions under Natural Light,” IEEE Trans. Circuits Syst. Video Technol 30(12), 4861–4875 (2020). [CrossRef]

45. C. O. Ancuti, C. Ancuti, and R. Timofte, “NH-HAZE: An image dehazing benchmark with non-homogeneous hazy and haze-free images,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops (IEEE, 2020), pp. 1798–1805.

46. D. Zhao, L. Xu, Y. Yan, J. Chen, and L.-Y. Duan, “Multi-scale Optimal Fusion model for single image dehazing,” Signal Processing: Image Communication 74, 253–265 (2019). [CrossRef]

47. Xu Qin, Zhiling Wang, Yuanchao Bai, Xiaodong Xie, and Huizhu Jia, “FFA-Net Feature Fusion Attention Network for Single Image Dehazing,” in Proceedings of AAAI Conference on Artificial Intelligence (AAAI, 2019), pp. 11908–11915. (2019).

48. M. Yang and A. Sowmya, “An Underwater Color Image Quality Evaluation Metric,” IEEE Trans. on Image Process 24(12), 6062–6071 (2015). [CrossRef]

49. K. Panetta, C. Gao, and S. Agaian, “Human-visual-system-inspired underwater image quality measures,” IEEE J. Oceanic Eng. 41(3), 541–551 (2016). [CrossRef]

50. A. Mittal, R. Soundararajan, and A. C. Bovik, “Making a ‘completely blind’ image quality analyzer,” IEEE Signal Processing Letters 20(3), 209–212 (2013). [CrossRef]

51. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Trans. on Image Process 13(4), 600–612 (2004). [CrossRef]

52. Q. Huynh-Thu and M. Ghanbari, “Scope of validity of PSNR in image/video quality assessment,” Electron. Lett. 44(13), 800–801 (2008). [CrossRef]

53. D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” International Journal of Computer Vision 60(2), 91–110 (2004). [CrossRef]

54. T. Lei, X. Jia, Y. Zhang, S. Liu, H. Meng, and A. K. Nandi, “Superpixel-Based Fast Fuzzy C-Means Clustering for Color Image Segmentation,” IEEE Trans. Fuzzy Syst. 27(9), 1753–1766 (2019). [CrossRef]

Methods	Brief Introduction
L2UWE [17]	Low-light underwater image enhancement
UWCNN+ [22]	Underwater convolutional neural network model and pre-processing
NUDCP [14]	A new underwater dark channel prior restoration
FGAN-UP [23]	Fast generative adversarial networks-based enhancement
Uw-HL [13]	Underwater image restoration using haze-lines
BR [18]	Bayesian retinex underwater image enhancement

Techniques	Brief Introduction
AMEF [17]	Artificial multiple-exposure image fusion
FMEF [28]	Fast artificial multi-exposure fusion
MOFD [46]	Multi-scale optimal fusion model
FFA-Net [47]	Feature fusion attention network

Image	Method	UCIQE	UIQM	Entropy	PSNR	SSIM
1	L²UWE	0.581	2.257	7.778	14.476	0.613
	UWCNN+	0.458	2.223	6.863	17.675	0.720
	NUDCP	0.687	1.999	7.459	20.071	0.730
	FGAN-UP	0.582	2.650	7.304	16.934	0.728
	Uw-HL	0.612	1.488	6.943	14.839	0.710
	BR	0.604	2.419	7.638	16.675	0.845
	Our	0.628	2.257	7.617	18.334	0.919
2	L²UWE	0.460	3.219	7.219	13.974	0.627
	UWCNN+	0.392	1.566	6.084	9.631	0.170
	NUDCP	0.510	2.632	7.477	15.230	0.581
	FGAN-UP	0.397	2.338	6.848	15.525	0.581
	Uw-HL	0.629	1.989	7.267	11.255	0.605
	BR	0.582	3.086	7.623	19.793	0.627
	Our	0.632	3.094	7.722	21.067	0.929
3	L²UWE	0.447	3.133	7.304	14.922	0.683
	UWCNN+	0.449	2.529	6.622	17.317	0.548
	NUDCP	0.466	2.806	6.943	13.797	0.720
	FGAN-UP	0.380	2.609	6.512	14.993	0.723
	Uw-HL	0.661	2.358	7.532	13.108	0.519
	BR	0.643	2.473	7.662	17.574	0.881
	Our	0.699	2.144	7.532	16.173	0.862
4	L²UWE	0.494	2.595	7.853	10.374	0.086
	UWCNN+	0.449	2.529	7.387	12.829	0.554
	NUDCP	0.591	2.932	7.401	13.797	0.720
	FGAN-UP	0.449	3.189	6.565	17.781	0.774
	Uw-HL	0.580	1.929	7.468	8.551	0.506
	BR	0.609	2.842	7.564	16.800	0.541
	Our	0.648	2.790	7.437	19.079	0.799
5	L²UWE	0.482	2.165	7.842	14.865	0.648
	UWCNN+	0.430	2.264	6.777	12.963	0.603
	NUDCP	0.571	2.396	7.461	19.289	0.775
	FGAN-UP	0.509	2.900	7.296	17.781	0.774
	Uw-HL	0.608	2.226	7.762	17.243	0.766
	BR	0.584	2.688	7.641	18.050	0.901
	Our	0.609	2.873	7.760	19.235	0.894
6	L²UWE	0.446	1.764	6.202	9.786	0.323
	UWCNN+	0.430	2.264	7.304	17.340	0.622
	NUDCP	0.453	1.849	6.736	11.654	0.387
	FGAN-UP	0.434	2.598	7.731	10.796	0.325
	Uw-HL	0.596	1.575	6.198	7.829	0.283
	BR	0.578	2.815	7.631	13.475	0.693
	Our	0.614	3.175	7.735	18.337	0.868
Average results of displayed 6 images	L²UWE	0.485	2.522	7.366	13.066	0.497
	UWCNN+	0.455	2.429	6.839	14.626	0.536
	NUDCP	0.546	2.436	7.246	15.486	0.596
	FGAN-UP	0.459	2.714	7.043	14.627	0.585
	Uw-HL	0.614	1.927	7.195	12.138	0.565
	BR	0.600	2.721	7.627	17.061	0.748
	Our	0.638	2.722	7.634	18.704	0.878
Average results of 50 images	L²UWE	0.475	2.478	7.483	12.979	0.567
	UWCNN+	0.461	2.391	7.160	13.914	0.487
	NUDCP	0.549	2.191	7.490	16.165	0.515
	FGAN-UP	0.433	2.533	7.230	12.945	0.548
	Uw-HL	0.620	1.874	7.163	10.754	0.591
	BR	0.609	2.693	7.587	17.679	0.778
	Our	0.628	2.893	7.616	19.039	0.893

Image	Method	UCIQE	UIQM	Entropy	Image	Method	UCIQE	UIQM	Entropy
1	L²UWE	0.545	2.782	7.756	6	L²UWE	0.595	2.186	7.747
	UWCNN+	0.476	3.180	6.803		UWCNN+	0.524	3.350	7.308
	NUDCP	0.636	2.655	7.764		NUDCP	0.546	3.371	7.666
	FGAN-UP	0.549	3.225	7.603		FGAN-UP	0.569	3.318	7.682
	Uw-HL	0.590	2.222	7.114		Uw-HL	0.634	2.576	7.580
	BR	0.580	3.267	7.750		BR	0.593	3.228	7.765
	Our	0.584	3.421	7.712		Our	0.618	3.316	7.858
2	L²UWE	0.486	2.708	7.738	7	L²UWE	0.476	2.940	7.709
	UWCNN+	0.453	3.300	6.891		UWCNN+	0.495	3.087	7.191
	NUDCP	0.614	2.949	7.632		NUDCP	0.534	3.379	7.395
	FGAN-UP	0.500	3.387	7.391		FGAN-UP	0.512	2.514	6.880
	Uw-HL	0.545	2.762	7.277		Uw-HL	0.616	2.275	7.561
	BR	0.597	3.225	7.609		BR	0.644	2.835	7.830
	Our	0.599	3.377	7.703		Our	0.574	3.016	7.781
3	L²UWE	0.518	2.833	7.790	8	L²UWE	0.518	2.803	7.751
	UWCNN+	0.470	3.208	7.060		UWCNN+	0.518	2.909	7.370
	NUDCP	0.569	3.044	7.695		NUDCP	0.570	3.017	7.705
	FGAN-UP	0.517	2.910	6.947		FGAN-UP	0.468	1.472	6.273
	Uw-HL	0.589	2.383	7.520		Uw-HL	0.568	2.302	7.641
	BR	0.577	3.225	7.746		BR	0.568	2.617	7.720
	Our	0.583	3.186	7.758		Our	0.582	3.028	7.784
4	L²UWE	0.488	2.838	7.723	Average results of displayed 8 images	L²UWE	0.515	2.750	7.742
	UWCNN+	0.489	3.276	7.189		UWCNN+	0.487	3.185	7.106
	NUDCP	0.501	3.132	7.396		NUDCP	0.560	3.097	7.594
	FGAN-UP	0.501	2.812	6.936		FGAN-UP	0.504	2.856	7.054
	Uw-HL	0.607	2.521	7.449		Uw-HL	0.591	2.450	7.478
	BR	0.597	2.922	7.409		BR	0.592	3.034	7.688
	Our	0.637	3.117	7.780		Our	0.596	3.215	7.778
5	L²UWE	0.496	2.908	7.720	Average results of 80 images	L²UWE	0.506	2.696	7.691
	UWCNN+	0.469	3.171	7.039		UWCNN+	0.473	3.111	7.380
	NUDCP	0.510	3.229	7.502		NUDCP	0.559	3.008	7.503
	FGAN-UP	0.419	3.213	6.719		FGAN-UP	0.490	2.999	6.979
	Uw-HL	0.583	2.562	7.680		Uw-HL	0.582	2.746	7.698
	BR	0.579	2.955	7.680		BR	0.594	2.849	7.708
	Our	0.590	3.259	7.848		Our	0.596	3.198	7.766

Image	Method	UCIQE	UIQM	Entropy	Image	Method	UCIQE	UIQM	Entropy
1	L²UWE	0.630	2.852	7.014	5	L²UWE	0.607	1.930	7.642
	UWCNN+	0.543	1.984	7.134		UWCNN+	0.431	3.106	6.473
	NUDCP	0.651	2.172	7.134		NUDCP	0.629	2.068	7.620
	FGAN-UP	0.605	2.888	7.237		FGAN-UP	0.601	3.084	7.402
	Uw-HL	0.631	1.629	6.675		Uw-HL	0.634	1.703	7.395
	BR	0.567	1.803	7.045		BR	0.589	3.097	7.755
	Our	0.650	2.852	7.792		Our	0.610	3.564	7.553
2	L²UWE	0.534	2.146	7.644	6	L²UWE	0.599	2.339	7.696
	UWCNN+	0.446	2.990	6.868		UWCNN+	0.457	3.299	6.760
	NUDCP	0.608	2.610	7.786		NUDCP	0.624	2.467	7.634
	FGAN-UP	0.520	3.268	7.239		FGAN-UP	0.575	3.497	7.216
	Uw-HL	0.614	2.449	7.697		Uw-HL	0.621	2.291	7.440
	BR	0.582	3.089	7.818		BR	0.570	3.338	7.263
	Our	0.629	2.871	7.779		Our	0.616	3.673	7.560
3	L²UWE	0.546	2.930	7.207	Average results of 6 displayed images	L²UWE	0.577	2.401	7.480
	UWCNN+	0.481	2.698	7.008		UWCNN+	0.467	2.928	6.849
	NUDCP	0.595	3.005	7.309		NUDCP	0.623	2.530	7.414
	FGAN-UP	0.521	3.286	7.461		FGAN-UP	0.565	3.277	7.321
	Uw-HL	0.604	1.754	7.212		Uw-HL	0.622	2.077	7.319
	BR	0.578	2.746	7.148		BR	0.581	2.917	7.459
	Our	0.635	3.493	7.291		Our	0.624	3.314	7.606
4	L²UWE	0.545	2.211	7.678	Average results of 50 images	L²UWE	0.592	2.167	7.263
	UWCNN+	0.447	3.491	6.854		UWCNN+	0.471	2.867	7.008
	NUDCP	0.632	2.856	7.003		NUDCP	0.617	2.842	7.428
	FGAN-UP	0.566	3.638	7.373		FGAN-UP	0.550	3.307	7.512
	Uw-HL	0.630	2.637	7.495		Uw-HL	0.611	2.146	7.698
	BR	0.602	3.431	7.729		BR	0.583	2.932	7.738
	Our	0.603	3.430	7.664		Our	0.630	3.311	7.818

Effective solution for underwater image enhancement

Abstract

1. Introduction

2. Related work

2.1 Motivation

2.2 Artificial multi-exposure fusion

2.3 Color correction

2.4 Guided filtering

3. Solution of this paper

3.1 PAL-CC

3.2 Reconstruction

4. Experiments and results

4.1 Experiment preparation

4.2 Comparison in dehazing underwater images

4.3 Computational efficiency comparison

4.4 Other applications

5. Conclusion

Funding

Acknowledgements

Disclosures

Data availability

References

Data availability

Cited By

Figures (17)

Tables (9)

Equations (20)

Optics Express

Image	Method	UCIQE	UIQM	Entropy	Image	Method	UCIQE	UIQM	Entropy
1	L²UWE	0.537	2.297	7.402	4	L²UWE	0.459	2.987	7.552
	UWCNN+	0.512	1.814	6.743		UWCNN+	0.447	1.460	6.453
	NUDCP	0.627	1.781	7.687		NUDCP	0.538	1.427	7.450
	FGAN-UP	0.484	2.620	7.273		FGAN-UP	0.476	2.605	7.105
	Uw-HL	0.600	1.850	7.273		Uw-HL	0.592	1.356	7.007
	BR	0.546	2.087	7.725		BR	0.544	1.802	7.685
	Our	0.626	2.593	7.756		Our	0.595	2.175	7.725
2	L²UWE	0.540	2.680	7.444	5	L²UWE	0.508	2.789	7.548
	UWCNN+	0.477	2.150	6.338		UWCNN+	0.441	2.381	6.606
	NUDCP	0.618	2.397	7.577		NUDCP	0.544	1.665	7.774
	FGAN-UP	0.506	3.058	7.326		FGAN-UP	0.424	2.954	7.413
	Uw-HL	0.619	2.340	7.386		Uw-HL	0.618	1.723	7.674
	BR	0.551	2.760	7.635		BR	0.522	2.523	7.715
	Our	0.622	2.951	7.699		Our	0.594	3.086	7.748
3	L²UWE	0.513	2.821	7.508	Average results of 5 displayed images	L²UWE	0.511	2.715	7.491
	UWCNN+	0.512	2.117	6.911		UWCNN+	0.478	1.984	6.610
	NUDCP	0.654	2.212	7.384		NUDCP	0.596	1.896	7.574
	FGAN-UP	0.515	2.246	7.499		FGAN-UP	0.481	2.697	7.323
	Uw-HL	0.623	2.088	7.427		Uw-HL	0.611	1.871	7.354
	BR	0.581	2.594	7.763		BR	0.549	2.353	7.704
	Our	0.626	2.786	7.746		Our	0.613	2.718	7.748

Method	1080×1080	800×800	400×400	256×256	128×128
L²UWE	60.6083 s	32.0705 s	8.1922 s	1.2722 s	0.6608 s
UWCNN+	9.8052 s	6.6534 s	5.1581 s	4.0642 s	2.0546 s
NUDCP	45.2477 s	24.2253 s	5.7377 s	0.5933 s	0.1483 s
FGAN-UP	0.5613 s	0.5613 s	0.5613 s	0.5613 s	0.5613 s
Uw-HL	53.1753 s	29.1614 s	7.6148 s	1.3058 s	0.8434 s
BR	4.4218 s	2.0730 s	1.0188 s	0.6943 s	0.6545 s
Our	7.0624 s	3.5037 s	1.2336 s	0.5520 s	0.5333 s

Image	Method	NIQE	Entropy	SSIM	PSNR
1	AMEF	2.760	7.603	0.637	15.774
	FMEF	3.521	7.682	0.807	14.828
	MOFD	2.316	7.721	0.506	12.591
	FFA-Net	3.198	7.543	0.786	13.359
	Our	2.608	7.775	0.790	13.647
2	AMEF	2.348	7.123	0.806	14.827
	FMEF	3.170	7.174	0.845	16.203
	MOFD	2.569	7.756	0.629	14.054
	FFA-Net	3.221	6.627	0.765	11.790
	Our	2.184	7.798	0.907	18.076
3	AMEF	2.525	7.319	0.559	15.187
	FMEF	3.187	7.469	0.814	14.949
	MOFD	2.645	7.756	0.509	14.488
	FFA-Net	2.896	7.595	0.750	12.856
	Our	2.417	7.789	0.843	14.971
4	AMEF	2.541	7.089	0.454	14.069
	FMEF	3.080	7.153	0.794	14.509
	MOFD	2.483	7.715	0.390	13.484
	FFA-Net	3.126	7.285	0.787	13.722
	Our	2.504	7.571	0.823	15.790
5	AMEF	2.663	7.348	0.434	14.498
	FMEF	3.260	7.508	0.779	14.304
	MOFD	2.539	7.688	0.459	13.413
	FFA-Net	3.248	7.456	0.760	12.828
	Our	2.778	7.711	0.716	14.227
Average results of 5 displayed images	AMEF	2.567	7.296	0.553	15.322
	FMEF	3.243	7.397	0.808	14.959
	MOFD	2.510	7.727	0.498	13.606
	FFA-Net	3.138	7.301	0.770	12.000
	Our	2.498	7.729	0.816	15.342
Average results of 55 Images	AMEF	2.501	7.273	0.588	15.333
	FMEF	3.252	7.266	0.800	15.013
	MOFD	2.525	7.699	0.501	13.488
	FFA-Net	3.097	7.229	0.761	12.163
	Our	2.385	7.703	0.803	15.336

Pair	Method	Valid matches	Pair	Method	Valid matches
1	Input	133	2	Input	2
	L²UWE	117		L²UWE	9
	UWCNN+	132		UWCNN+	5
	NUDCP	292		NUDCP	35
	FGAN-UP	94		FGAN-UP	2
	Uw-HL	203		Uw-HL	27
	BR	262		BR	42
	Our	306		Our	83