Effective image dehazing by multiband image fusion

Image dehazing is of great importance since haze-free images are visually pleasuring and the clear visibility of image is a prerequisite for most modern intelligent systems. Although different kinds of methods have been presented, they tend to be complex and their performance may vary with scenes. In terms of this, a novel method using a visible and an near-infrared image of the same scene for haze removal is proposed in this paper. It is achieved through a multiband image fusion approach, followed by a segmentation and a filtering strategy. And there is no need to infer the atmospheric parameters or generate the depth information. Experiments on real outdoor images validate the proposed method.


Introduction
Images of outdoor scenes are usually degraded in bad weather conditions such as haze, fog and smoke, a phenomenon due to atmospheric absorption and scattering.This process offers a poor visibility of scene content which is a prerequisite of most modern systems for surveillance (1) , intelligent vehicles (2) and remote sensing.Meanwihle, the contrasts of the degraded images are reduced and there surface colors become faint (3,4) .Image dehazing is therefore of great importance and is an inevitable task for social life.
However, haze removal is a challenging task and many methods have been proposed.. General contrast enhancement approaches such as histogram equalization, wavelet transform or Retinex (5) can be applied for image dehazing to improve the visual effect.Since these techniques ignore the fact that haze is not constant over an image and is dependent on the unknown depth information, they would distort haze-free regions at the same time.
Therefore, methods using multiple images or additional information have been proposed.Multiple images of the same scene under different weather conditions are used to compute depth information and hence to get rid of haze (6)(7)(8) .Polarization based methods use two or more images with different degrees of polarization to remove haze effect (9,10) .And haze was removed by using rough depth information from user interaction or existing 3D models (11,12) .While these approaches are better than the general contrast enhancement methods, they are not flexible to handle dynamic scenes and not easy for image acquisition.
Recently, single image haze removal that rely on a strong prior or assumption has made significant progress.Fattal infers the medium transmission under the assumption that transmission and surface shading were locally uncorrelated (13) .This approach, however, might be failed in the cases of heavy haze.Tan et al. observes that the haze-free image has higher contrast compared with the haze image and he removes the haze by maximizing the local contrast of the input image (14) .The results of such method are visually sound, but they tend to be oversaturated on very saturated scenes.Lately, He et al. introduces a simple and effective dark channel prior to estimate the transmission map and thus achieved compelling results (15) .However, a common disadvantage of these physical model based approaches is their complexity on estimation of atmospheric parameters.Consequently, they suffer from poor efficiency.
Then, multiband image fusion technique achieved by using an additional near-infrared(NIR) image of the same scene has been applied in the field of image dehazing of late (16,17) .Because of the transmission characteristics of NIR, haze is much less present in NIR images.It is therefore an effective way to get rid of haze.An algorithm with manual work to select different parts of the image as the preprocessing work for image fusion was proposed (16) .It cannot work automatically and in real time.Another multiresolution approach is used to fuse both visible and NIR images together (17) .Since the texture details in the non-haze regions will also be boosted in luminance channel, the results of such method suffer from color shifting artifacts.
In this paper, a novel image dehazing algorithm based on segmentation and multiband image fusion is proposed.With respect to the previous approaches, the novelty of the proposed method is three-fold.1) the complexity of the proposed fusion approach is more reduced than the previous strategies since no scattering model is required in our framework.This is highly desired since the parameters, especially the depth information of the image, in the scattering model are difficult to estimate accurately and the model based algorithms tend to be time-consuming.2) A simple but effective segmentation approach by using the dark channel prior (15) is used to separate the original hazy image into haze part and haze-free part automatically.Such segmentation is critical for subsequent image fusion processing, since it makes the most of the useful information of both visible band(VIS) and NIR images, and regions without haze remain unaltered.3) Except the poor visual effects in terms of color and intensity information, the ability that NIR "see" through the haze surpasses any image processing based approaches.This has greatly expanded the range of image dehazing application.Even in the heavy haze situation, the NIR is still physically valid.

NIR Approach 2.1 Rayleigh Scattering Structure
Light is scattered in different directions by small particles when traveling through the atmosphere.When these particles are much smaller than the wavelength of the light involved, a physical phenomenon called rayleigh scattering occurs (18)   The intensity of the scattered light s I is related to that of the incident light 0 I by the inverse of the fourth power of the wavelength λ (19) .Consequently, some wavelengths of the light are scattered more by particles than others.Therefore, sky appears blue most of the time because the blue light is scattered more efficiently due to its relatively short wavelength.And thus the near infrared waves can pass through haze whereas visible light cannot.As a result, NIR is able to capture images of objects that otherwise would have been hidden from view.This phenomenon is at times striking.

NIR Image Acquisition
As opposed to the human visual system, camera sensors are made of silicon, sensitive to radiation from 200nm to 1100nm.The 400-700 nm range belongs to the visible band while 700-1100nm is the NIR band.The ultraviolet part of the spectrum, 200-400 nm, is generally filtered out by the camera's optical elements.By replacing the filter that protects the sensors from incoming NIR light by a piece of glass, the camera is able to capture both visible and NIR light (16) .The possible movements between two shots are corrected using a registration algorithm (19) .

Color Space Conversion
Since the NIR image is one channel only containing intensity information, a color space transformation process need to be carried out.Specifically, the luminance and chrominance components of the original visible band image are required to be separated.The transformation can be achieved through commonly-used HSV or YCbCr conversion.The HSV conversion, however, suffers from computation complexity where trigonometric function is calculated on each pixel (20) .Therefore, the YCbCr conversion is selected.

Haze Thickness Evaluation
Dark channel prior is proposed based on the statistics of the haze-free outdoor images (15) : in most of the non-sky patches, at least one color channel has very low intensity at some pixels.In other words, the minimum intensity in such a patch should have a very low value.For a given image J , the prior Due to the additional atmospheric light, a haze image is brighter than its original haze-free counterpart.Therefore, the dark channel intensity of the haze image will significantly increase in regions with dense haze.Visually, the intensity of the dark channel is a good approximation of the thickness of the haze.Fig. 2.VIS image(L) and its corresponding dark channel image(R)

NIR and VIS Image Fusion
The visible spectrum and the NIR are inherently complementary, since the visible spectrum image contains robust color and comparatively low noise while the NIR image gives better results in foggy weather condition.Therefore, the combination of them can be of great advantages when either may perform poorly.Multi-resolution algorithm is a widely-used and effective technique for image fusion, for the important characteristics of the image varies from different resolutions.The quality of fusion result, however, has decreased as compared to the haze-free part of the VIS image while increased in comparison with the fuzzy part.This means that the multiresolution methods promote the quality of blurring part of the original image by reducing the quality of its clear part (21) .This is not the perfectly way of fusion.The ideal rule of fusion is supposed to make the most of the clear part of the VIS image while replace the blurred part by the corresponding NIR image.In other words, it is first required to segment the haze and haze-free part of the VIS image.Such segmentation process tends to be tough.However, the brightness distribution of the aforementioned dark channel image has already segmented the original hazy image.The brighter the denser, the darker the clearer.Thus, the haze-free part of the VIS image and the anti-haze part of the NIR image can be combined through a simple and efficient way.For the output fused image 0

Boundary Artifact Removal
Segmentation based fusion, though a better approach than global fusion, suffers from boundary artifacts.This is because the intensity value between visible and NIR images at the boundaries may differ enough to introduce a visible edge.Thus, a two dimensional matrix M is defined and , i j M is the value of α corresponding to the image pixel at ( , ) i j .The elements of the matrix M is eventually completely defined after the aforementioned steps since α is determined by the intensity value of the dark channel image.Due to the fact that haze is usually continuous, the value of α changes sharply only along boundary edges and relatively stable in rest areas.The classic and fast Gaussian filter is therefore introduced to remove the boundary artifact and in this paper the standard deviation of the Gaussian kernel is set to 1.0.As shown in Fig. 4, the boundary gap has been successfully removed and barely can be seen.

Experimental Results And Analysis
This section performs a series of experiments to demonstrate the effectiveness of the proposed method and also compares the results with those generated by some other methods.A set of VIS hazy images and their corresponding NIR images are used in the experiments.These images are available in the EPFL Datasets (22) .By default, the local patch Ω used to generate the dark channel image is set to 15 15  × for consistency over all of the tested images.
The first set of experiment was performed on a landscape image in which the mountain far away is shrouded by haze, as shown in the top-left in Fig. 5.The rest of images in Fig. 5 are dehazing results by histogram equalization, He et al. (15) method and our algorithm, respectively.In order to distinguish the details of the results more clearly, we took part of the image in red rectangle and enlarge it in Fig. 6.As can be seen, our approach can unveil the details in very dense haze regions while keep vividness of the image colors.
Comparing with the state-of-the-art dehazing technique, as shown in the bottom-left in Fig. 5 that the result of He et al. (15) suffers from unrealistic colors and halo artifacts, as its approach is based on the haze formation equation and influenced heavily by scene information.Our method recovers the hazy image without sacrificing the fidelity of the colors and halo artifacts can be barely seen.
Meanwhile, our method outperforms the classic histogram equalization method.Histogram equalization enhances the contrast of the original image globally and hence brought color distortion or oversaturation problems.Differently, our result not only contains more details and texture information in the hazy part of the images, but also remain the haze-free part almost unchanged.Another set of experimental results are shown in Fig. 7 and Fig. 8.They also illustrate that our result not only unveils the details but also remain the picture naturally.From the experiments we can see that the proposed method removed the haze effectively and gave better results than others.Predictably, our method performs well and stable as long as the NIR is valid while others may vary with scenes.

Copyright
Papers submitted to ICISIP must describe original and unpublished researches.They should not have been submitted to other conferences or journals.Any paper which is under review elsewhere will not be considered for publication.Once an eligible paper is accepted, it will be published under a Creative Commons Attribution 3.0 (CC BY 3.0)License.

Conclusions
In this paper, a novel image dehazing method based on NIR and VIS image fusion has been presented.The algorithm put forward was carried out in three phases.In the first phase, the dark channel prior was used to segment the haze-part and the haze-free part of the original VIS image.In the second fusion phase, we made the best use of both VIS and NIR images and fused them in according to the thickness of the haze.Finally, a filtering strategy was introduced to solve boundary artifact.The experimental results illustrated that the proposed method could remove haze effectively and stably.
Compared with the existing image dahazing method that based on model or image enhancement, the presented algorithm has following characteristics: 1) manual interaction is not required during the whole dehazing process.The automatic operation is achieved.2) calculation of complex atmospheric model and depth information can be ignored which simplified the algorithm complexity greatly.3) the use of NIR is physically valid which guarantee the effectiveness of image dehazing.4) the NIR and VIS image are easy to get which makes the proposed method can be applied pervasively.However, the proposed method has its limitation when the dark channel prior is failed.Therefore, it is necessary to design more robust algorithm to estimate the thickness distribution map of hazy image and this will be the focus in the subsequent work. .

J
is a color channel of J and ( )x Ω is a local patch centered at x.The statistics reveals that dark J has a very low value and tends to be zero if J is a haze-free outdoor image.dark J is called the dark channel of J , and the above statistical observation is referred to as the dark channel prior.

I
is luminance channel of the VIS image, NIR I is the NIR image.The weighting coefficient α is the normalized dark channel intensity defined by the intensity of the dark channel image dark J and grayscale L .The fusion result shown in Fig.3 illustrates that the distant mountain shrouded in fog in the VIS image is well-distinguished and the nearer haze-free part almost stays the same.This suggests that the quality of haze-degraded image can be greatly improved through the proposed fusion procedure and thus make invisibles visible.However, the fusion result has a clear fragment boundary which is not visual friendly.This is because the thickness distribution of haze in VIS image has a clear boundary and consequently affects its dark channel version.To solve such drawback, a filtering strategy for α is bringing in.

Fig. 5 .
Fig. 5.The fusion results of different image dehzazing algorithms, from top to bottom, let to right: original image, image dehazing using histogram equalization, He et cl..method and the proposed fusion approach.

Fig. 6 .
Fig. 6.Part of image dehazing results taken from Fig. 5 in red rectangle.

Fig. 7 .Fig. 8 .
Fig. 7.The fusion results of different image dehzazing algorithms, from top to bottom, let to right: original image, image dehazing using histogram equalization, He et cl.method and the proposed fusion approach.