Enhanced Restoration of Ill-Conditioned Photographs

We propose a unified method for enhancing color images photographed under bad conditions, such as foggy, ill-exposed and underwater photographs. Input images are firstly enhanced by hue-preserving unsharp masking and next their color is changed by a cross white balance procedure. The hue of colors is almost preserved for foggy and ill-exposed images while colors are changed for underwater photographs.


Introduction
Light is scattered and attenuated in foggy air or hazy waters.Colors are hence degraded in hazy or underwater photographs.Similar color degradation is caused by bad tuning of cameras, e.g.under-exposure darkens photographs while over-exposure brightens them.Blur is also added to their color degradation.These degraded photographs need to be post-processed for restoring the true color of scenes and sharpening blurred edges and textures.
Various enhancing restoration methods have been presented according to individual degradation conditions.Restoration methods adopting a dark channel prior are widely used for dehazing foggy photographs (1,2) .These dehazing methods darkened photographs of colors brightened by fogs.This is also the case for over-exposed photographs where colors are brightened hence their darkening is needed.Conversely in under-exposed photographs, scene colors are darkened hence they need to be brightened.The bright channel prior is used for it (3,4) .
These fogs and under or over exposures change mainly the luminance and the saturation of colors while the hue is not much changed.Hence enhancement methods for these photographs are desired to preserve the hue of colors.On the other hand, in underwater photographs the hue of colors is shifted because waters absorb mainly the red component while the green and blue are not much decayed.Hence scenes become bluish in waters.Such color change is needed to be corrected in enhancement of underwater photographs.Several methods have been presented for enhancing underwater photographs (5,6) .
Thus, various methods have been developed for enhancing individually degraded photographs.When we use one of them, we should discriminate which type of degradation is matched to a target photograph.This judge is not so easy in some cases, and furthermore in some photographs, multiple degradation conditions are simultaneously included in one scene.For such photographs, we should apply different methods to each region adequately.
This observation leads us to unify the above methods for automatically enhancing all types of degraded photographs.In this paper, we present such a universal image enhancing method.Our proposed method is composed of two steps: the first is the hue-preserving unsharp masking (7) and the second is a cross white balancing technique (see Fig. 1).In this paper, vectors are denoted by capital letters and scalars are represented by lower-case letters.

Hue-Preserving Unsharp-Masking
We denote the color of pixel ( , )  i j in an input image This UM, however, does not preserve the hue of colors.In order to preserve the hue, this equation is modified to ( ) where 0 where Hence the hue of ij D in eq.( 1) is equal to that of ij C , i.e. eq.( 1) is the hue-preserving unsharp masking (HPUM) (7) .

Cross White Balance
The above HPUM is sufficient for dehazing and exposure compensation because the hue of colors is almost preserved.Underwater photographs, however, need to alter colors.Hence we next change ij D by using a white balance (WB) technique.We adopt here the most popular WB method based on the gray world assumption (9) where the average of each , , r g b channel over the entire image is almost the same, i.e. the average color is gray.
Straightforward way of the white balance according to this assumption is transforming the color where , , over the entire image.This ' , ' , ' is gray.This full white balancing, however, alters the color too strongly and produces an unnaturally colored image.
We weaken this full white balance procedure as follows: 1) Compute the average of [ , , ] where x and y are the width and height of the image.
In this procedure, the weighted mean in eq.( 8) is due to if we output the perfectly white-balanced color , , , then the color is too strongly changed and its output is unnatural as was mentioned above.Hence we average the hue-preserved output and the hue-changed output , , . This automatic adjustment of hue-preserving degree is the aim of the above weighted mean and a benefit of our method.In the above procedure, the white balance coefficients , , which is the target of white balance.Hence we call the above procedure the cross white balance (CWB).

Experiments
For some example photographs, we have done experiments for the following three cases: 1) We stop at the HPUM and omit the following CWB.
2) We bypass the HPUM and directly apply the CWB to the original input image, i.e. we execute the procedure in the previous section by setting 3) Our proposed method, i.e. execute the HPUM firstly and next apply the CWB to its output.

Dehazing
First, some examples of foggy photographs are shown from Fig. 2 to Fig. 8 where (a) is an input image, (b) is the output of HPUM, (c) is the result of direct application of WCB to the input image without HPUM and (d) is the output of our method.The value of h is written below (c), which is the same in (d).In Fig. 2, Fig. 3 and Fig. 6, h is not so large, hence the color is slightly changed by CWB.
In Fig. 5, Fig. 7 and Fig. 8, the value of h is close to 1, hence no change is produced by CWB.
Examples of exposure compensation are shown from Fig. 9 to Fig. 12 which are under-exposed cases and Fig. 13 which is an over-exposed photograph.In these photographs except for Fig. 10, h takes large values, hence the hue of colors is almost preserved and the contrast is enhanced by the HPUM.
Last examples are underwater photographs shown from Fig. 14 to Fig. 17.In all of these underwater photographs, h is rather small, hence their colors are changed by the CWB.The red channel is mainly reduced in the water, hence the CWB mainly restores the red channel and makes the color reddish.
As is observed from these results, our method restores the color of photographs automatically according to their degree of color unbalance.If the condition under which each photograph is taken is known, the result of our method is inferior to those of previous methods specialized to individual scenes, but our method can deal with broader class of photographs without detailed information about their scene condition.

Conclusions
We have presented a general image enhancing method.This method can deal with variously degraded photographs, such as hazed, under or over exposed and underwater photographs.Textures in images are sharpened using the hue-preserving unsharp masking and colors are restored by the cross white balance procedure.Improvement of the contrast enhancement and refinement of the white balance are under study.
input image is already balanced, then h is nearly equal to 1 and