Adaptive Combination of Additive and Multiplicative Algorithms for Color Image Enhancement

Recently, Nikolova and Steidl have proposed a hue and range preserving histogram specification method which outputs a weighted average of two images generated by additive (shifting) and multiplicative (scaling) algorithms, where the weights are fixed for all pixels in an image. In this paper, we propose a method for selecting the one of the two algorithms at each pixel adaptively. Therefore, there is no need to prepare the weights for combining additive and multiplicative algorithms in the proposed method. Experimental results show that the proposed method can improve the saturation better than the convex combination method by Nikolova and Steidl. We also verify the property of saturation improvement in the proposed method theoretically.


Introduction
Color image enhancement is an important technique in digital image processing for human visual perception and computer vision, where color images are typically recorded in the format of RGB color channels. However, the RGB color space is not intuitive for human vision. Therefore, we often need to use the alternative intuitive color spaces such as HSV (hue, saturation, value), HSL (hue, saturation, lightness), HSB (hue, saturation, brightness), HSI (hue, saturation, intensity). Among the three attributes of color, hue, saturation and intensity, hue determines the appearance of color such as red, green and blue. Therefore, in a number of applications of color image enhancement, it is preferable to preserve hue throughout the process. The basic flow of color image enhancement procedure may proceed as follows: (1) transform a color from RGB to HSI, (2) enhance the color in a hue-preserving manner, and (3) inversely transform the enhanced color from HSI to RGB. However, such naive procedures can cause a gamut problem, i.e., the enhanced color may not exist in the RGB color cube. Naik and Murthy have proposed a scheme for huepreserving color image enhancement without gamut problem [1]. However, as pointed out by themselves, their scheme always decreases the saturation in a common condition. To alleviate this problem, Murahira and Taguchi have proposed improved methods [2] [3], an Inoue et al. have proposed a method for maximizing the saturation while preserving the hue [4]. Recently, Nikolova and Steidl have proposed new algorithms for hue and range preserving color † Corresponding: k-inoue@design.kyushu-u.ac.jp * Department of Communication Design Science, Kyushu University 4-9-1, Shiobaru, Minami-ku, Fukuoka 815-8540, Japan image enhancement [5], where a fast ordering algorithm for exact histogram specification [6] is used. Minami and Yamada have also proposed an algorithm for exact histogram equalization [7]. In their paper [5], Nikolova and Steidl combine an additive algorithm and a multiplicative algorithm convexly by introducing a weight parameter, which is fixed for all pixels in a target image. However, their convex combination method occasionally decreases the saturation because of the fixed weight parameter. In this paper, we propose a method for improving their method to alleviate the saturation deterioration. That is, the proposed method selects the additive algorithm or the multiplicative algorithm that outputs better saturation at each pixel adaptively. Experimental results show the effectiveness of the proposed method compared with the convex combination method.
The rest of this paper is organized as follows: Section 2 summarizes related algorithms proposed by Nikolova and Steidl [5] [6]. Section 3 proposes an adaptive combination method for improving saturation. Section 4 shows experimental results. Finally, Section 5 concludes this paper.

Additive and Multiplicative Algorithms for Color
Image Enhancement In this section, we briefly summarize two algorithms for color image enhancement proposed by Nikolova and Steidl [5].
Let w ≡ (w r , w g , w b ) be an RGB color image with M × N pixels, where w c ∈ {0, 1, . . . , L − 1} for c ∈ {r, g, b} are the red, green and blue channels of w, where L denotes the number of brightness intensities, e.g., for 8-bit images we have L = 2 8 = 256. Then we reorder the pixels in each color channel w c columnwise into an n-dimensional vector, where n ≡ MN, and address each pixel by the index i ∈ I n ≡ {1, 2, . . . , n}. Let be the intensity [8] of w, and let f [i] be the ith pixel value of f (w). Then we see that Nikolova and Steidl have proposed a method for exact histogram specification (HS) [6], which transforms an image f intof whose histogram matches a specified target histogramĥ ≡ (ĥ 1 ,ĥ 2 , . . . ,ĥ L ), i.e.,ĥ k = #{i ∈ I n :f [i] = k − 1} for k = 1, 2, . . . , L, where # stands for the cardinality. 2n − M − N denotes the discrete gradient operator (horizontal and vertical forward differences), whose elements are ∇ l,i = −1 and ∇ l,i+M = 1 for the horizontal differences, and ∇ l ′ ,i = −1 and ∇ i ′ ,i+1 = 1 for the vertical differences. The other elements of ∇ are all zeros. ∇ T denotes the transpose of ∇, and where α = 0.05 is a default value. Given a histogram-specified intensityf from the original intensity f of w, we want to have the corresponding color imageŵ in a hue-preserving manner. It is confirmed that w preserves the original hue of w ifŵ ≡ (ŵ r ,ŵ g ,ŵ b ) is expressed as follows: where a[i] and b[i] are constants to be specified. Substitut- . Substituting this into (3), we have another expression ofŵ c [i] as follows: Let us define the following values: so that On the other hand, a lower gamut problem arises when m[i] < 0. In this case, to avoid the lower gamut problem, so that From (5) we have an additive transform called shifting for a is chosen to avoid the gamut problem as follows: for all i ∈ I n and c ∈ {r, g, b}. Nikolova and Steidl combined the shifting and scaling models aŝ with upper and lower gamut corrections (8) and (10) if necessary, and defined the critical points for lower and upper gamut problems as follows: Then the additive algorithm for computing a color-enhanced imageŵ is given in Algorithm 2, and the multiplicative algorithm is given by Algorithm 3. Letŵ + c andŵ × c be the output of Algorithms 2 and 3, respectively. Then Nikolova and Steidl also combined them bỹ Algorithm 2 Additive Color Enhancement [5] 1: Compute the intensity f of w and the target intensityf using Algorithm 1.
and for all c ∈ {r, g, b}: Algorithm 3 Multiplicative Color Enhancement [5] 1: Compute the intensity f of w and the target intensityf using Algorithm 1.
and for all c ∈ {r, g, b}: and showed an equivalence betweenw c andŵ c given by (11).
We have the saturation of w[i] in the HSI (hue, saturation and intensity) color space as follows [8]: Then the saturation ofw[i] is given by where

Proposed Method
In (17), the second case depends on λ, and when λ = 0, we have , which means that only (1−λ)ŵ + c in (15) contributes to the modification of the saturation, which will be maximized when λ = 0. On the basis of this observation, we propose a method that switches the value of λ adaptively to improve the saturation of each pixel in a given image. From (18), iff [i] < f [i], then we can maximize the saturation by setting λ = 0 for using the additive algorithm, otherwise we switch to the multiplicative algorithm by setting λ = 1 to prevent the saturation deterioration. This method is formally described as follows: for c ∈ {r, g, b}, where the second to fourth cases correspond to the additive algorithm (λ = 0), and the fifth to sixth cases correspond to the multiplicative algorithm (λ = 1). The procedure of the proposed method is summarized in Algorithm 4.
Algorithm 4 Proposed method 1: Compute the intensity f of w and the target intensityf using Algorithm 1.
and for all c ∈ {r, g, b}: and for all c ∈ {r, g, b}: The saturation of w * [i] obtained by Algorithm 4 is given by In Appendix A, we show the derivation of (20). We have a relationship between the saturation S (w[i]) in (17) and S (w * [i]) in (20) as follows: In Appendix B, we prove this property.

Experimental Results
In this section, we show experimental results with real images. First, we show an example of the exact histogram equalization by Algorithm 1 proposed by Nikolova and Steidl [6]. Figure 1(a) shows a grayscale image, which is enhanced by Algorithm 1 as shown in Fig. 1(b). Their histograms are shown in Figs. 1(c) and (d), respectively, where the positively skewed distribution in Fig. 1(c) (the right tail is longer) is transformed into the uniform distribution in Fig. 1(d). In the following experiments, we use the uniform distribution for the target histogramf , i.e., we perform histogram equalization in color image enhancement. Figure 2 shows the results of color image enhancement by Algorithms 1 to 4. First, the original input images in Figs Fig. 2(c). On the other hand, the right image (Peppers) exhibits similar appearance to Fig. 2(d). Figures 2(g) and (h) show the results by the proposed Algorithm 4, which successfully enhances both images as well as Algorithm 3. Figure 3 shows the results of color image enhancement by convex combinations of Algorithms 2 and 3 by Nikolova and Steidl [5] for Couple and Peppers images in the first and second rows, respectively. These images are the weighted averages of the output images of Algorithms 2 and 3 with different values of λ. We compared the average saturation per pixel between the convex combination method by Nikolova and Steidl [5] and the proposed method. Figures uses λ = 0, and the horizontal and vertical axes denote the saturation obtained by the convex combination and the proposed methods, respectively. All pixels or points denoted by'+' are above or on the line through the points (0, 0) and (1,1), that indicates that the saturation obtained by the proposed method is not smaller than that obtained by the convex combination method. These results also support the claim in Property 1. Figure 6 shows the results of color image enhancement of photographs of various scenes by the proposed method, where the left and right columns display the input and output images. For all images, the hue is preserved, and the contrast and saturation are enhanced.

Conclusion
In this paper, we proposed a method for hue-preserving color image enhancement, where the additive and multiplicative algorithms by Nikolova and Steidl [5] are adaptively combined. Experimental results demonstrated that the proposed method achieves higher saturation compared with the convex combination method by Nikolova and Steidl [5]. We also showed that this property of saturation improvement is valid for any color image theoretically.

Appendix
A. Derivation of (20) In this section, we derive (20), which is composed of six cases. Each case is separately described below. If Substituting this into (16), we have  Figure 6: Color image enhancement results.