Adaptive Aggregated Histogram Equalization for Color Image Enhancement without Gamut Problem

In color image enhancement, gamut problem is one of the fundamental issues for practical image processing tasks. However, recent color image enhancement methods without gamut problem fail to preserve or enhance color saturation, which is an essential property for visual perception. In this paper, we propose a color image enhancement method, in which we introduce a new histogram named aggregated histogram (AgH), which is computed with R, G and B components of all pixels in a color image, and then equalize AgH to obtain an enhanced color image in a similar manner to conventional grayscale histogram equalization (GHE). As a result, the proposed method is free from gamut problem as well as GHE. Moreover, we extend the proposed AgH equalization (AgHE) method to adaptive AgHE (AAgHE), which equalizes a local histogram at each pixel of a color image. Experimental results show that the proposed methods produce better results than existing color image enhancement methods. Consequently, the proposed methods can be a good alternative to conventional color image enhancement methods without gamut problem.


Introduction
Image enhancement is a necessary technique for visual perception of human beings and machine perception in various environments. Many grayscale image enhancement techniques have extended to their color versions so far. However, it should be noted that hue, saturation and intensity are a set of characteristics inherent in colors. Therefore, the extension of the techniques for grayscale images to color images is not straightforward in many realistic situations. In color image enhancement, it is frequently required that the hue of original images is preserved after the procedures for preserving the appearance of the contents in the original images.
Naik and Murthy [1] have proposed a scheme to generalize any grayscale image enhancement method to color images without encountering gamut problem, and used the scheme to generalize the histogram equalization (HE) method for grayscale images to that for color images. Han et al. [2] also proposed a 3-D color HE method which is equivalent to one of the methods presented by Naik and Murthy. Recently, Zahedi and Rahimi [3] also proposed a 3-D color HE method using principal component analysis. A drawback of Naik and Murthy's method is that it cannot only increase the saturation of colors, but also decreases it frequently. To solve this problem, Yang and Lee [4] proposed a hue-preserving gamut mapping method that * Corresponding: k-inoue@design.kyushu-u.ac.jp Department of Communication Design Science, Kyushu University 4-9-1, Shiobaru, Minami-ku, Fukuoka 815-8540, Japan provides higher saturation than that of Naik and Murthy's method. In fact, Yang and Lee's method can improve the saturation of dark and bright colors. However, the other colors having middle luminance are processed in the same manner as Naik and Murthy's method. Therefore, the saturation of colors having middle luminance is not increased even if Yang and Lee's method is used. Another way of improving HE is adaptive HE (AHE), which equalizes a local histogram at each pixel for enhancing the contrast better than global HE. Pizel et al. [5] proposed an interpolated AHE to speed up the AHE procedure, and also presented a clipped AHE, which is now called the contrast limited AHE (CLAHE) [6], to overcome the problem of overenhancement of noise contrast. Amorim et al. [7] proposed a 3D CLAHE method in the context of medical imaging. Stimper et al. [8] generalized CLAHE to arbitrary dimensions for contrast enhancement of complex multidimensional imaging and spectroscopy datasets.
In this paper, we propose a new HE method for color image enhancement without gamut problem. First, we make a histogram from an input color image, and refer to it as aggregated histogram (AgH). Next, we equalize AgH by transforming R, G and B values of each color in the image into the enhanced ones by a single transform function derived from AgH. We call this method the AgH equalization (AgHE). By using a single transform function for all color channels, the relative relationship among R, G and B values or their order is preserved during the procedure. As a result, the hue of the processed colors are moderately preserved without gamut problem.
We also propose an adaptive AgHE (AAgHE), which equalizes a local AgH at each pixel, in which we utilize an integral histogram [9] technique for efficient computation.
Experimental results show that the proposed methods can enhance the saturation of color images better than Naik and Murthy's and Yang and Lee's methods while the hue is moderately preserved.
The rest of this paper is organized as follows: Section 2 describes grayscale and color HE methods, where, first, GHE is summarized, next, a separable method for color image enhancement is summarized, and then the AgHE method is proposed for improved color image enhancement. Section 3 extend the proposed AgHE to adaptive AgHE by using an integral histogram technique. Section 4 shows experimental results on the standard image database SIDBA [10], and compares the proposed AgHE and AAgHE. Finally, Section 5 concludes this paper.

Histogram Equalization (HE)
In this section, we first summarize grayscale histogram equalization (GHE), and then it is applied to color images in a straightforward manner. After that, we propose an aggregated histogram equalization (AgHE) method for saturation-improved color image enhancement.

Grayscale Histogram Equalization (GHE)
Let for l = 0, 1, . . . , L, where δ l, f i j denotes the Kronecker delta [11] defined by Next, from the histogram h, the cumulative histogram H = [H 0 , H 1 , . . . , H L ] is computed as follows: for l = 0, 1, . . . , L, which can be computed recursively as where ϕ(·) denotes the function of a tone curve for HE, and round(·) is a rounding function that rounds a given argument toward the nearest integer, and a and b are constants, for example, given by where f min denotes the minimum pixel value in F. The difference between (6) and (7) is apparent when f min is substituted for f i j in (5), i.e., ϕ( f min ) = round( L mn H f min ) for (6), and ϕ( f min ) = 0 for (7). For the maximum pixel value f max in F, both (6) and (7) give the same pixel value for l = 0, 1, . . . , L. The difference quotient of H HE l can be computed approximately as follows: where l 2 > l 1 , and k 1 and k 2 are the maximum values that satisfy l 1 = ϕ(k 1 ) and l 2 = ϕ(k 2 ), respectively. The result in (11) shows that the slope of the cumulative histogram H HE is approximately constant, which is similar to that of the cumulative histogram of a histogram with uniform distribution.

Separable Histogram Equalization (SHE) for Color Image Enhancement
A simple extension of the above GHE to color images is that we apply GHE to each RGB color channel of a color image separately, and then combine the histogram-equalized RGB color channel images into an RGB color image [2]. We would like to call this method the separable HE (SHE).
The SHE method equalizes the histograms of RGB color channels separately with different tone curves. Therefore, the hue of a color may not be preserved after SHE procedure. To overcome this problem, Naik and Murthy [1] proposed a hue-preserving color image enhancement method without gamut problem, which is also proposed by Han et al. [2] from a different viewpoint. However, Naik and Murthy's method often decreases the saturation of input colors to result in grayish output images. To overcome this problem, Yang and Lee [4] proposed a modified method for achieving higher saturation compared with Naik and Murthy's method. However, the effectiveness of Yang and lee's method is limited due to the use of Naik and Murthy s method in a part of their method. In the next subsection, we propose a more effective method than the above methods.

Aggregated Histogram Equalization (AgHE)
In this subsection, we propose an HE method for color images named aggregated HE (AgHE), where all RGB color components are transformed with a single tone curve. Let . . , m and j = 1, 2, . . . , n, and each of R, G and B values, r i j , g i j or b i j , has a discrete value in L. Then we define the aggregated histogram (AgH) of for l = 0, 1, . . . , L. We also define the cumulative AgH for l = 0, 1, . . . , L, which can be computed recursively as for l = 1, 2, . . . , L with ] denotes the RGB color vector at a pixel (i, j). Then the R, G and B values are given by respectively, where ϕ A (·) denotes the function of a single tone curve for AgHE, and is defined as where a A and b A are constants given by where x min denotes the minimum RGB value in F, i.e., The difference between (17) and (18) is apparent when x min is substituted for x in (16), i.e., ϕ A (x min ) = round( L 3mn H A x min ) for (17), and ϕ A (x min ) = 0 for (18). For the maximum RGB value (17) and (18) where l 2 > l 1 , and k 1 and k 2 are the maximum values that satisfy l 1 = ϕ A (k 1 ) and l 2 = ϕ A (k 2 ), respectively. The result (22) shows that the slope of the cumulative AgH H AHE is approximately constant similarly to that of the cumulative histogram of a histogram with uniform distribution.

Adaptive Aggregated Histogram Equalization (AAgHE)
The above AgHE method transforms all pixel values with a single function. Therefore, AgHE can be viewed as a global tone mapping method. In this section, we propose a local tone mapping method for AgHE, adaptive AgHE (AAgHE), which equalizes a local AgH at each pixel in a color image. We implement AAgHE with an integral histogram [9] for computing local AgHs in an efficient way. Let for l = 0, 1, . . . , L. Next, for i = 2, 3, . . . , m with fixed j = 1, we compute Similarly, for j = 2, 3, . . . , n with fixed i = 1, we compute For the remaining indices i = 2, 3, . . . , m and j = 2, 3, . . . , n, we compute After the above computation, we obtain the integral histogram H I of F. Let us proceed to the computation of a local AgH with H I . Assume that the size of a rectangular region (or contextual region [5]) on F is w × w pixels, in which a local AgH will be constructed. Then we determine the value of w by for a given nonnegative integer parameter d. An example of the rectangular region centered at (i, j) is shown in Fig. 1 in blue. In this situation, the local AgH in the rectangular region is given by  , j), we compute the local AgH h A i j , from which a local tone mapping function is derived to transform the RGB value at the position (i, j) in the same manner as the global AgHE.

Experimental Results
In this section, we show the experimental results of grayscale and color image enhancement using the standard image database SIDBA [10], and demonstrate that the proposed AgHE method can enhance the saturation of color images better than existing color image enhancement methods based on HE technique. We also compare AgHE and AAgHE for demonstrating the superior performance of AAgHE in color image enhancement. Figure 2 shows an example of GHE with the constants in (6), where the contrast of an original grayscale image in Fig. 2(a) is enhanced by GHE as shown in Fig. 2(b). Figure 2(c) shows the histogram of the original image in Fig. 2(a), where we can see that this image has few black or white pixels, and therefore has low contrast. On the other hand, Fig. 2(d) shows the histogram of the enhanced image in Fig. 2(b), which contains both black and white pixels that contribute to enhance the contrast. Next, we show the cumulative histograms of Figs. 2(a) and (b) in Figs. 2(e) and (f), respectively, where the curved shape of the graph in Fig. 2(e) is stretched to a straight line in Fig. 2(f), where a straight line y = x a with a slope 1 a derived in (11) is shown in red, which fits the obtained cumulative histogram faithfully.
Next, Fig. 3 shows an example of the proposed AgHE with the constants in (17). As well as the above example in Fig. 2 Fig. 3(a), the contrast of the original image is low, which can also be confirmed from AgH in Fig. 3(c), where the entities concentrate in the middle range of the horizontal axis. As a result, the cumulative histogram becomes an Sshaped curve as shown in Fig. 3(e). On the other hand, Fig. 3(b) shows the enhanced image by the proposed AgHE method. The corresponding AgH is shown in Fig. 3(d), where the entities have a wide range distribution on the horizontal axis. Figure 3(f) shows the corresponding cumulative AgH, where the S-shaped curve in Fig. 3(e) is stretched to a straight line, and a straight line y = x a A with a slope 1 a A derived in (22) is shown in red, which fits the obtained cumulative AH faithfully. In Fig. 4, we compare the proposed method with SHE de-  by the separable, Naik and Murthy's, Yang and Lee's and the proposed AgHE methods, respectively. In Fig. 4(b), since the SHE method equalizes the histogram of each RGB color channel separately, the hue of some colors are not preserved; for example, the peppers in the 2nd image from the right have different hue from the above original image. On the other hand, Naik and Murthy's and Yang and Lee's methods in Figs. 4(c) and (d) are hue-preserving ones. Therefore, the resultant images preserve the original hue in Fig. 4(a). However, Naik and Murthy's method decreases the saturation of colors, and the resultant images approach their grayscale ones; for example, the 2nd image from the left looks grayish in both Figs. 4(c) and (d). Yang and Lee's method improves the saturation as shown in the 3rd and 7th images from the left. However, in many cases, Yang and Lee's method does not improve the saturation, because Yang and Lee's method uses Naik and Murthy's method under the condition that pixels have a mid-range luminance.
On the other hand, the proposed method can improve the saturation of wide-ranging images as shown in Fig. 4(e).
To evaluate the performance quantitatively, we compute the saturation [12] defined by where Figure 5 shows the mean saturation of each image computed by where the vertical and horizontal axes denote the mean saturation and the abbreviated names of images, respectively. The bars in cyan, yellow and red denote Naik and Murthy's, Yang and Lee's and the proposed methods, respectively. The proposed method achieved higher mean saturation value MS (F) than the other methods for all images.
The results in Fig. 5 for 12 images are summarized in Table 1, where 12 saturation values for each method are   averaged to evaluate the performance numerically. The proposed method achieved the highest value 56.59 among the compared methods. Figure 6 compares the results of AAgHE with that of AgHE, where the original color image in Fig. 6(a) is taken against the light, and has colors of low saturation. As shown in Fig. 6(b), the performance of AgHE in terms of saturation improvement is marginal. Figures 6(c) and (d) show the results of AAgHE with d = 200 and d = 100, respectively, where we can see that AAgHE outputs colorful images compared with AgHE in Fig. 6(b). The mean saturation of the four images in Fig. 6 is summarized in Table 2, where AAgHE achieves higher saturation values than AgHE.

Conclusion
In this paper, we proposed a color image enhancement method, where a new histogram aggregating R, G and B color channels of a color image is defined, and equalized to enhance the visual quality of the color image. The proposed aggregated histogram equalization (AgHE) method converts R, G and B values of each pixel in an input color image with a single transform function. As a result, the relative relationship among R, G and B values of each color is preserved, that brings a moderate hue preservation property to the proposed AgHE method. We also extended the proposed AgHE method to an adaptive AgHE (AAgHE), which equalizes a local AgH at each position, where an integral histogram technique is applied for efficient computation. Experimental results showed that the proposed AgHE method produces enhanced color images with higher saturation than that of the state-of-the-art color HE methods. The performance was also quantitatively evaluated by mean saturation values to demonstrate the effectiveness of the proposed AgHE method. Furthermore, we showed an example that AAgHE outperforms AgHE in terms of saturation improvement.
Future work will include the development of a contrast limited [6] AAgHE method and an aggregated histogram specification [13] [14] method for improving the performance of the proposed AgHE method.