Enhancement of Images Degraded by Hazy Light Scattering and Attenuation

We propose a method for enhancing images degraded by hazy scattering and attenuation of lights. Images are enhanced in two steps. The first step is restoration of the hue of colors on the basis of the gray world assumption. This hue restored images are next enhanced with magnifying the contrast of the luminance and the saturation of colors while preserving the hue of colors. Performance of the proposed method is verified for some example images of foggy and underwater photographs.


Introduction
We consider, in this paper, a unified method for restoring the color and tone of images photographed under an illconditioned environment such as shots in a foggy weather or underwater images. Such bad exposures alter the color of objects and the light attenuation degrades the contrast with blurring scene details. White fogs changes the scene whity and yellow sands discolor the sky yellowish. Underwater photographs are generally bluish due to absorption of the red component in the light. In addition to these shifts in the color hue, the saturation of the color is decayed by the light scattering and the contrast is decreased by the light attenuation. These degrading processes require restoration of the original color hue and enhancement of the contrast together with magnification of the color saturation.
Various white balance methods (WB) [1] [2] have been developed for restoring the hue of colors of photographed scenes. Conversion of the luminance of colors by using tone curves is used for the contrast compensation [3] of illexposed photographs. For restoration of images degraded by haze, defogging [4]∼ [14] and enhancement of underwater photographs have been presented [15]∼ [23].
We propose, in this paper, a method for enhancing photographs by firstly restoring the color hue while preserving the luminance and the saturation of colors and next increasing the contrast and the saturation with maintaining the restored color hue.
In the mathematical descriptions below, matrices are denoted by upper-case letters, vectors by thick lower-case letters and scalars by thin lower-case letters.

Hue Conversion
Let the color at pixel (i, j) in an input photograph be c i j = [r i j , g i j , b i j ] T which is converted to c ′ i j = S c i j +u i j at the first stage in our method. The gray world assumption is widely adopted for tuning the white balance (WB). Let the average of c i j over an entire image bec = [r,ḡ,b] T and the average of converted The gray world assumption is declared as follows.
i.e.c ′ is proved to be gray. This color transformation from c i j to c ′ i j according to the gray world assumption is equivalent to the estimation of the color of illumination light toc. Actually, however, calculation ofc by averaging the color over the entire image yields unnatural colors by excessive equalization of color distributions. For preventing such over-balancing, the average is better to be calculated not over whole image but for only parts in the image. [24]. The following two rules have been used for selecting the partial regions. 1) Bright Region WB: BRWB It is generally known that the color in bright regions is close to the illumination light [1]. Let the luminance of input image c i j be l i j = q · c i j where x · y is the inner product of x and y and we set q = [0.296, 0.514, 0.112] in the experiments below. We define the bright region by the set of pixels satisfying over the entire image. We calculate the above-mentioned matrix U from the average of colors of pixels in this bright area. We call the WB using this matrix U "the bright region WB" (BRWB).

2) Low Saturation Region WB: LSRWB
It is also known that the color in lower saturated regions tends close to the illumination light [25]. Let the saturation of colors be s i j = ∥c i j − l i j u∥. Low saturation regions are defined by the set of pixels where s i j > [zs max + (1 − z)s min ] with s max = max i, j s i, j , s min = min i, j s i, j and the same z as the above z. We call the WB using the matrix U by averaging colors of these lower-saturated pixels the "low saturation region WB" (LSRWB).

3) Bright and Low Saturation Region WB (BLSRWB)
Selection is refined to the intersection of the above two regions. We call the WB using the matrix U by averaging colors both of bright and low saturated regions the "bright and low saturation region WB" (BLSRWB). This is the combination of the above BRWB and LSRWB.
In these WB methods, the color hue is changed by the matrix U. On the other hand, the color saturation is controlled by α. The average of the saturation of the image Similarly, the luminance is adjusted by β i j . The luminance at each pixel is preserved if we set By setting the parameters α and β i j in this way, we can alter only the hue of colors while preserving the saturation and the luminance. Thus we can execute an ideal WB.
For instance, if we apply the gray world assumption over the entire image in Fig.1(a), we get the converted image in Fig.1(b) where whole colors become too reddish. This red shift is relaxed in the above BRWB as is shown in Fig.1 (c) and further in the LSRWB as in Fig.1(d) where the color of the fish is almost close to its true color in Fig.1(f). However, the red shift is still slightly remained in Fig.1(d). The combined BLSRWB outputs the image in Fig.1(e) where white skins in the fish is clearly brighter than Fig.1(d) and more like to Fig.1(f). From this observation of a preliminary experiment, we use this BLSRWB hereafter in experiments in this paper.

Hue Preserving Contrast Enhancement
After this color hue conversion from c i j to c ′ i j by the BLSRWB, the color saturation is needed to be magnified and the contrast is desired to be enhanced for hazy or dark photographs. This image enhancement should preserve the color hue which is already repaired to intrinsic values in the first stage mentioned above.
It is known that the hue of [26]. In this color transformation from c ′ i j to c ′′ i j , the color saturation is adjusted by γ i j and the luminance by δ i j . We set γ i j as We set s = 3 and t = 50 in all experiments below. These values are determined empirically for yielding appropriate results for various images.
Another parameter δ i j is set as δ i j = m i j − γq · c ′ i j where m i j is an enhanced luminance of c ′ i j . By setting δ i j to this value, the luminance of c ′′ i j becomes m i j . We adopt the locally projected luminance [27] as m i j . The locally projected luminance is the value of projection of c ′ i j to the axis along the local variation in the luminance This conversion of the luminance from the ordinary luminance l ′ i j to the locally projected luminance m i j shows a contrast enhancing effect similar to the local histogram equalization (details are described in the reference [27]). Through this conversion from c ′ i j to c ′′ i j , the color saturation is magnified and the contrast is enhanced while preserving the color hue.
As is explained above, the method proposed in this paper is a cascade of two color conversion processes. The original color c i j is firstly converted to c ′ i j via the process explained in section 2 and next c ′ i j is changed to c ′′ i j by the method explained above in this section 3. All of the color components, i.e. the luminance, the saturation and the hue are refined throughout the overall conversion from the input color c i j to c ′′ i j which is outputted from our method.

Experiments for Comparison with Other Methods
We have experimented the above explained method for some images commonly dealt with many dehazing meth-(a) input (b) Yeh [11] (c) Shiau [10] (d) Fattal [4] (e) Li [7] (f) Lai [14] (g) Tarel [9] (h) He [5] (i) ours ods, some of which execute the WB without tone conversion while some methods target only tone enhancement preserving the color hue and their combined techniques have been also presented for total color restoration. We experimented our method for these images and compared its result with those of various other methods. The first example is a hazy image shown in Fig.2(a) where scene becomes whitish due to white fog. The result of a dehazing method by Yeh et al. [11] in Fig.2(b) is unnaturally reddish due to improper color conversion. The next result by Shiau et al. [10] in Fig.2(c) is too dark with low contrast in shaded areas. Compared to these two results, the images shown from Fig.2(d) to Fig.2(h) are clearer, however sharpening is insufficient for trees and grasses. In comparison with these previous methods, the result of our method in Fig.2(i) is bright and sharp over the entire image.
The second example is also a hazy image in Fig.3(a) where the scene is yellowish due to yellow sand in the sky. Dehazed results are shown from Fig.3(b) to Fig.3(h). In Fig.3(b)∼(e), yellow tint is still remaining in the sky. Such yellow tint disappears in Fig.3(g) while dark areas cannot be discriminated clearly. In Fig.3(h), the yellow tint disappears and dark areas are brightened. Our result shown in (a) input (b) Li [12] (c) Son [8] (d) Lai [14] (e) He [5] (f) Li [13] (g) Shiau [10] (h) Fattal [4] (i) ours (a) input (b) Lu [23] (c) Fattal [4] (d) Wen [16] (e) C.-B. [15] (f) ours   3(i) is clearer and brighter than all of these results and bluish sky is restored. Next in Fig.4(a) is an example of underwater photograph. The result by Lu et al. [23] shown in Fig.4(b) is insufficient in restoration of the reddish tinge of the scene. The result in Fig.4(c) is too dark over the whole image. The reddish tinge is restored in Fig.4(d) where, however, shaded regions are still dark, while bright regions become too whitish in Fig.4(e). In our result in Fig.4(f), the reddish tinge is sufficiently restored and dark regions become clearly visible.
Finally, Fig.5(a) is also an example of underwater photograph which contains much dark shadows requiring their contrast enhancement. The shadow regions are darkened in Fig.5(b) and Fig.5(c), the color saturation is lowered in Fig.5(d) and the bluish tinge is strengthened in Fig.5(e) and Fig.5(f). In Fig.5(g)∼(i), the dark shadow becomes unnaturally reddish. Figure 5(j) is still bluish. Figure 5(k) is sharpest among these results, while the color becomes reddish over the entire image and the color saturation is lowered. In our result in Fig.5(l), the dark regions are sharpened and the reddish tinge is appropriately restored.

Conclusion
We have presented a method for restoring the color hue, enhancing the contrast of luminance and magnifying the color saturation in photographs discolored by various haze. We verified the performance of the proposed method for some examples of foggy and underwater images. In our method, conversion of the color hue and the contrast enhancement are divided to separate stages which are executed sequentially. Their unification into an overall processing of a photograph is under study.