Statistical and Geometrical Methods: A Comparative Study on Color Transfer to Dark Image

Color transfer is a technical means of changing the appearance of an image (source image) such that the image has characteristics similar to those of a specified image (target image). There exist first-order statistics-based and geometry-based color transfer methods. These color transfer methods can be applied in different color spaces via space transformation. In this report, we review the fundamental principles of two color transfer methods in different color spaces and present a comprehensive comparative study on color transfer using statistical and geometrical methods in different color spaces. We assess the quality of color transfer by computing the consistency between the normalized histograms of the source image and target image and discuss the performance of each method for different color spaces. Our experiments show that the best color space is the RLAB color space while the geometrical method is quantitatively better than the statistical method for most of a large number of images.


Introduction
Color is an important feature of an image in the fields of painting and photography. Colors express the personal emotion and mood of the artist and present useful information for technical use such as in visualization. A common operation on an image is to alter the color in the image. Color transfer, also called color mapping, is a type of color manipulation that conveys the color information from a target image such that it has a color appearance similar to that of the target image. Figure 1 shows an example of color transfer, panel (a) is the source image, panel (b) is the target image, and panel (c) is the result of transferring the color information in panel (b) to panel (a) so that panel (c) has a color appearance similar to that of panel (b) while the contents of the scene remain unchanged. Color transfer is useful for color editing in image processing. As an example, color transfer is used to convey color information from a target image to a source image to change the appearance of the source image, such as from autumn to spring. It is also applied to reduce color differences between images in stitching a panorama using a series of images that may have slight color variations [1]. In the movie industry, consecutive shots taken at different times may have different colors due to slight changes in natural or artificial lighting, * Hiroshima Institute of Technology, Hiroshima, Japan.
(xinyue001112@163.com) and this requires the color to be adjusted consistently across all shots [2]. In an image style transfer algorithm [3], when transferring the style in a given image to another image, the color distribution of the style image is also copied to the content image, which might be undesirable, and preservation of the color in the content image using a color transfer method may therefore be required [4].
In the graphics community, the pioneering work of color transfer was reported [5] by Reinhard et al. in 2001. It is well known that the three channels of the RGB color space have strong correlation. Reinhard et al. thus established a decorrelated color space lαβ using a series of conversions from the RGB color space to the XYZ, to the LMS space, and finally to the lαβ space. The decorrelation between the three color axes is computed through principal component analysis [5] [6]. Color is transferred on the decorrelated lαβ color space and then converted back to the RGB color space. Xiao et al. proposed a method that transfers color from one image to another directly in the correlated RGB color space [7] without color space conversion. Both methods mentioned above transfer color globally (i.e., to the entire image); however, they mix up colors in different regions. In fact, these methods may be applied for corresponding parts of two images; i.e., swatch-based color transfer [5] [7]. An intensive review of color transfer, or color mapping, and its applications is presented in [1] [13]. Readers are strongly urged to find useful information in these references.
There are two predominant methods of color transfer. The method proposed by Reinhard et al. [5] computes the mean and standard deviations of all pixels along each of the three axes in the lαβ color space for target and source images. These statistical data are used to change the values of pixels in the source image in the lαβ color space, and we thus refer to the method as the statistical method. Meanwhile, Xiao's method transforms the pixel distribution of the source image so that the image has a distribution similar to that of the target image. The method is essentially a geometrical transform using a series of translations, scalings, and rotations and we therefore refer to the method as the geometrical method. Reinhard et al. used the lαβ color space but their color transfer algorithm is not limited to this space. Many other related spaces, such as the la * b * , lu * v * , and RLAB [8] color spaces, are also used in the literature. Jonathan et al. conducted an intensive series of experiments and claimed that the RLAB color space is the best space for transforming an image of a dark scene into an image of a lightened scene [9] using the statistical color transfer algorithm. Theoretically, the geometrical color transfer algorithm can also be applied in any color space and is not limited to the RGB color space.
In this paper, we conduct a comparative study of the statistical method [5] and geometrical method [7], which are the methods most commonly used in transferring color globally in various color spaces. We focus on five color spaces, namely the RGB, lαβ, la * b * , lu * v * , and RLAB color spaces. The comparison uses distance metrics for histograms of the output image and target image in evaluating the quality of transferring color. We employ the four most commonly used distance metrics, namely, The Euclidean diastance, Bhattacharyya distance [10] [11], chi-square metric [12], and histogram intersection, to measure the similarity of histograms. Section 2 describes the five typical color spaces used in the comparison and the conversions. The statistical and geometrical methods of color transfer between a source image and target image are detailed in Section 3. Section 4 presents experimental results and observations. Section 5 presents conclusions of the study and discusses future work.

Color spaces
Color transfer, or color mapping, is an operation that rearranges the pixel distribution of the source image in a color space, such that the distribution matches that of the target image. Many color spaces are adopted for color trans-fer algorithms. In our comparison, we apply the statistical method and geometrical method of color transfer in five commonly used color spaces, namely, the RGB, lαβ, la * b * , and lu * v * , and a special RLAB color space. Conversions between the RGB color space and other spaces are described for completeness in this section.

RGB color space
The first color space is the RGB space commonly used for image processing and computer graphics. The RGB color space is an additive color model that employs RGB primaries to define other colors by the three chromaticity values of red, green, and blue. In terms of digital image processing and computer graphics, the color model most commonly used in practice is the RGB model for a color monitor. In this model, each color appears in its primary spectral components as a triple of red, green, and blue. The model is based on a Cartesian coordinate system and the subspace of interest is a cube.
Theoretically, different colors representing a pixel in an image are points on or inside the cube in the model. It is often assumed that all colors are normalized so that the cube is the unit cube; i.e., all values R, G, and B are in the range [0, 1]. In practice, an RGB image generally uses eight bits for each channel such that each color pixel has a depth of 24 bits, resulting in what is called a full-color or true color image [14].

lαβ color space
As a non-standard color space, the lαβ space was used by Reinhard et al. in their report [5]. Unlike the RGB color space, which has strong correlations among the three channels, the lαβ color space has maximal decorrelation among the three axes computed in principal component analysis [15]. To compute lαβ values from the RGB color space, we first convert the RGB color space to the XYZ space using the equation given by Reihard et al. [5]: This XYZ space is a device-independent color space. After converting the RGB space to the XYZ space, the next step is to convert the image to the LMS cone space: Equations (1) and (2) can certainly be combined for conversion from the RGB space to the LMS space directly. Data distribution in the LMS space is heavily skewed, and thus converted to a logarithmic space to eliminate skewness. Data distribution in the logarithmic space is then converted to the decorrelated lαβ color space: After transferring color in the decorrelated lαβ color space, it is easy to convert the result image in the lαβ color space to an image in the LMS space, XYZ space, and finally RGB space by a series of inverse conversions.

CIE la * b * color space
The la * b * color space was designed by the International Commission on Illumination (CIE) in 1976 as a device-independent color model. In the la * b * color space, the magnitudes of numerical change in l, a * , and b * correspond roughly to the same magnitudes of change visually perceived by human. The conversion between RGB and la * b * color spaces uses the deviceindependent CIEXYZ space converted from CIERGB and its reverse version. The forward transformation from the XYZ space to the where and σ = 6/29. Note that X n ,Y n , and Z n are a reference white point set to (1.0, 1.0, 1.0) in this paper. The reverse transformation from the la * b * space to the RGB space is simple and expressed as where The XYZ space is then converted to the RGB color space.

CIE lu * u * color space
The lu * v * color space was designed by the CIE in 1976 as a simple transformation of the XYZ space. After converting from the CIERGB space to the CIEXYZ space, the conversion from the XYZ space to the lu * v * space is as follows. Chromaticity coordinates (u ′ , v ′ ) are computed in the first step of conversion. Given a reference white point ( where σ = 6/29. The reverse transformation from Again, it is easy to convert the XYZ space back to the RGB color space.

RLAB color space
The RLAB color space was originally designed for fixing problems with the CIE la * b * color space that arise under unusual lighting conditions [8].
In the conversion between the RGB and RLAB color spaces, the CIEXYZ space is converted to an XYZ space under reference viewing conditions: where RAM =           1.0020 −0.0401 0.0084 −0.0042 0.9666 0.0008 0.0000 0.0000 0.9110 In the forward transformation, the three components (l, a, b) in the RLAB color space are computed as where the exponent σ takes the value of 1/2.3 for an average-brightness surround, 1/2.9 for a dim surround, and 1/3.5 for a dark surround as suggested in [8]. The backward transformation from the RLAB space back to the XYZ is trivial: The XYZ space is then converted back to the RGB color space.

Color transfer
The operations of color transference are conducted in one of the above five color spaces. Without loss of generality, we refer to any color space as C and the three channels of the color space as C k , k = 1, 2, 3. The statistical and geometrical color transfer methods are considered for comparative study and are briefly described in this section.

Statistical method
Reinhard et al. proposed a statistical method of color transfer [5]. They used first-order statistics; i.e., the global mean and standard deviation of both source and target images. The method modifies the content of the source image using statistics from the target image. This method was originally used in the lαβ decorrelated color space but is not limited to this space. We apply the method in other color spaces for a comparison of performance. Given a color space C with three channels C k , k = 1, 2, 3, the first statistics are computed for each channel and for both the source and target images. Supposing that the image size is M s × N s for the source image S and M t ×N t for the target image T , the means and standard deviations are computed as where µ and σ are respectively the mean and standard deviation, s and t respectively represent the source image and target image, k is the channel index, and S k (x, y) and T k (x, y) respectively indicate the source and target pixel values at (x, y) in channel k of a given color space, which is the RGB, lαβ, la * b * , lu * v * , or RLAB color space. The color transfer from the target image T to the source image for channel k is implemented by subtraction, scaling, and addition operations. This statistical color transfer is simple and intuitive and can be implemented adopting where D k (x, y) represents the destination image for channel k and pixel (x, y) after color transfer. All channels are processed the same way.

Geometrical method
The geometrical method of color transfer focuses on the distribution of pixels in a given  The Italic number in a numerical cell indicates the best value of a metric. stat. = statistical method; geom. = geometrical method. color space[7] [16]. The method rearranges the pixel distribution of a source image so that it has a distribution sim-ilar to that of a target image. This rearrangement of pixel distributions comprises a series of geometrical transforma- where the matrix T {t,s} represents translations between the center of the distribution and the origin of the coordinate system, R {t,s} represents rotations between the principal axes and the coordinate axes, and S {t,s} represents the scalings between unit scales and distribution scales. The matrix M s transforms the pixel distribution of the source image to a normalized distribution, where the matrix T s translates pixels of the source image so that the center of the distribution is at the origin, R s rotates pixels so that the principal axes are in accordance with the axes of the coordinate system, and S s scales pixels to the unit scale. The matrix M t acts in a reverse manner by transforming the normalized distribution so that the distribution is similar to that of the target image. The matrix S t scales the unit scale to that of the distribution of the target image, R t rotates the axes of the coordinate system to the principal axes of the distribution of the target image, and T t translates the origin to the center of the distribution of the target image. The translation matrices T {t,s} are computed by finding the centroids of distributions of both source and target images. The rotation R {t,s} and scaling S {t,s} matrices are calculated from covariance matrices of source and target images, using SVD decomposition [17]. Details are found in [7] [16]. Figure 2 presents an example to explain the effect of color transfer using the geometrical method. Figure 2(a), (b), and (c) shows distributions of pixels sampled from images in Fig. 1(a), (b), and (c), respectively, in the RGB color space. It is seen that the source and target images have very different distributions; however, after transferring the color information in the target image to the source image, the resulting image has a distribution very similar to that of the target image.

Quantitative evaluation
The color transfer operation maps color information from the target image to the source image. The resulting image should have color appearance similar to that of the target image. A method of measuring the similarity of the appearances of the images is to adopt a histogram similarity metric. Suppose a histogram has b bins. A histogram can then be considered as a point in a b−dimensional space. A histogram similarity metric is in fact the distance metric between the two points representing the two histograms. This metric has been used in evaluating image similarity [18] previously.
Although there are many metrics, in our comparative study, we used four metrics to compute how the histograms match, namely the Euclidean distance d k E , Bhattacharyya distance d k B , chi-square metric d k C , and intersection d k I for channel k. In the following metric calculations, H k d and H k t represent the histograms of result and target images, and all histograms have bins of b = 256 for each channel. Moreover, these histograms are normalized so that each histogram has a unit sum of b = 256 bins; i.e., Σ b−1 i=0 H(i) = 1.0. The Euclidean distance between two histograms H k d and H k t is defined as the distance between two points in b−dimensional space: where the Euclidean distance d k E is in the range [0.0, . A histogram is considered a probability distribution of pixel values. The Bhattacharyya distance measures the similarity of two distributions. Note that the histograms have been normalized. We have where the Bhattacharyya distance is in the range [0.0, 1.0]. There are several definitions for the chi-square metric.   The Italic number in a numerical cell indicates the best value of a metric. stat. = statistical method; geom. = geometrical method.
For the comparison of histogram, we use the equation where chi-square distance is in the range [0.0, 2.0]. The last measurement is the intersection metric that is the intersection of the two histograms and is defined as where the intersection metric is in the range [0.0, 1.0]. Because a color image has three channels, a color histogram comprises three histograms calculated for each channel. For the comparison of color histograms, we combine the three metrics for each channel as a vector in threedimensional space. A comprehensive distance metric is then computed from the metric d k of each channel as a length of the vector using d * = (d 1 * ) 2 + (d 2 * ) 2 + (d 3 * ) 2 , where * represents E, B, C, or I for each metric. A smaller distance value indicates better similarity of histograms for Euclidean, Bhattacharyya, and chi-square metrics and thus the better color transfer. However, for the intersection metric, a higher value means better similarity.

Experimental results
Experiments were conducted on many images to compare the performance of the statistical method with that of the geometrical method in different color spaces. The two color transfer algorithms, written in C language, were implemented on a notebook computer running Windows 10.
We used 100 natural dark color images from a dataset and 300 color images from the BSD300 dataset, with both datasets downloaded through the Internet [19]. For the sake of convenience, we refer to the former as dataset 1 and the Figure 6: Another example of color transfer using the BSD300 dataset: (a) a source image, (b) a target image, (c)-(l) results obtained using statistical and geometrical methods in the order of RGB, lαβ, la * b * , lu * v * , and RLAB color spaces. Note that stat. = statistical method, geom. = geometrical method.  The Italic number in a numerical cell indicates the best value of a metric. The bold number is the best value in a column. stat. = statistical method; geom. = geometrical method.
later as dataset 2. We used the dark color images in dataset 1 as source images and selected 10 natural images in dataset 2 as target images. We transferred color information from the 10 target images in the five color spaces using the two methods, resulting in 100 images for one dark image. Therefore, 10,000 images were produced. Figure 3 presents experimental results obtained when using the two color transfer methods in different color spaces. Figure 3(a) shows a source image, which is an extremely dark image taken from dataset 1. The intensity is so low that the content can hardly be observed by the human eye. Figure 3(b) shows a target image taken from dataset 2, in which the color information is transferred to the source image (a). Figure 3(c) and (d) respectively shows the results of direct transference in the RGB color space obtained using statistical and geometrical methods. Figure 3(e) and (f) respectively shows the results of color transference in the lαβ color space obtained using statistical and geometrical methods. Likewise, Fig. 3(g) and (h) shows results for the la * b * space, Fig. 3(i) and (j) shows results for the lu * v * space, and Fig. 3(k) and (l) shows results for the RLAB color space, with Fig. 3(g), (i), and (k) presenting the results for the statistical method and Fig. 3(h), (j), and (l) the results for the geometrical method. Table 1 presents the results of quantitative evaluation using histogram matching metrics mentioned in the previous section. The first column gives the color spaces, the second to fifth columns give the Euclidean, Bhattacharyya, chisquare, and intersection metrics in that order. In the second row, stat. and geom. respectively refer to the statistical method and geometrical method. Each cell includes the evaluated numerical results. A larger value indicates a better result for the intersection metric while a smaller value indicates a better result for the other three metrics. The best value is given in italics. The table reveals that the geometrical method performs better than the statistical method in all color spaces for all evaluation metrics, except for the chisquare metric in the lαβ color space. In this space, the geometrical method (2.7925) performs a little worse than the statistical method (2.7918) according to the chi-square metric. At first glance, the results of the statistical method seem better than those of the geometrical method owing to slight over-saturation of the sky. However, careful observation reveals that the colors of the lawn are more similar to those in the target image when using the geometrical method.
In this experiment, we obtained 10,000 resulting images and calculated the four metrics for each image and its corresponding target image. The geometrical method certainly outperforms the statistical method for most images, but not for all images. We therefore calculated the average of each metric in each color space for each color transfer method as listed in Table 2. The notations in this table are the same as those in Table 1, but the values in the cells are the average values for each metric. The italic number in a numerical cell indicates the best value of a metric, while the bold number indicates the best value in a column. By observing the data in each cell, we find that the geometrical method outperforms the statistical method for all color spaces and all metrics, except for the combination of the lu * v * color space and Bhattacharyya metric. Note that, except for the combination of the Bhattacharyya metric and geometrical method, among the five color spaces, the RLAB color space has the best performance. This result is consistent with the conclusion drawn by Jonathan et al., who claimed that the RLAB color space is the best space for color transfer [9], although they only used the statistical method in their experiments.
In the next experiment, we used 300 images from dataset 2. All images had been converted to dark images; i.e., nighttime images [20]. We directly used the converted dark images downloaded [19]. We used the original image as the target image, and the converted dark image as the corresponding source image. Color information in the original images was transferred to the dark images. Each pair of source and target images provided 10 resulting images in five color spaces for the two color transfer methods, and we thus obtained 3000 resulting images. Figure 5 presents experimental results for dataset 2 (i.e., the BSD300 dataset). Figure 5(a) shows a dark image created from the original in Fig. 5(b) to simulate a night scene. The dark image was used as the source image, while the original image was used as the target image, in which the color was transferred to the source image. Figure 5(c) and (d) respectively show resulting images obtained using the statistical method and geometrical method in the RGB color space. Similarly, Fig. 5(e) and (f), (g) and (h), (i) and (j), and (k) and (l) respectively show images produced in lαβ, la * b * , lu * v * , and RLAB color spaces using the two methods. Table 3 presents the quantitative evaluation of images in Fig. 5 by histogram matching. The notations are the same as those in Tables 1 and 2. The table shows that the geometrical method outperforms the statistical method for all metrics in all color spaces.
We transferred color information in original images to the 300 artificially created dark images individually to obtain 3000 images as described above. One result is shown in figure 6, where Fig. 6(a) shows the dark image created from the original image in Fig. 6(b) to simulate a night scene. For the transference of color information of the image in Fig. 6(b) to the image in Fig. 6(a) using statistical and geometrical methods, the results are shown in Fig. 6(c) and (d) for the RGB color space, Fig. 6(e) and (f) for the lαβ color space, Fig. 6(g) and (h) for the la * b * color space, Fig.  6(i) and (j) for the lu * v * color space, and Fig. 6(k) and (l) for the RLAB color space. As before, the four metrics were calculated for each histogram pair of the original image and resulting image. Average values of the metrics are presented in Table 4. This quantitative evaluation reveals that the geometrical method outperforms the statistical method for all metrics and color spaces except for the RLAB space. When applying the statistical method for color transfer, the RLAB color space had the best performance according to all metrics. This is again consistent with the opinion of Jonathon et al. The next best color space when using the statistical method was the lαβ space. However, for geometrical method, as shown in bold numbers, the lαβ space provided the best performance among color spaces.
The following finds are taken from the above experiments. The geometrical method generally outperforms the statistical method in most color spaces according to the histogram matching metrics in both cases of natural dark images and artificially created night scenes. When transferring color information to a dark image using the statistical method, the RLAB space is generally the best color space, although not in all cases. When using the geometrical method for color transference on the BSD300 dataset, the lαβ space is a better color space in most cases. We recognize that there is no best color space and color transfer method for all situations.

Conclusions
In this comparative study on statistical and geometrical methods of transferring color information from a given original image to a dark image, we adopted four commonly used color spaces and a special RLAB color space to conduct color transfer and used four histogram matching metrics to evaluate the methods and color spaces. Through quantitative comparison, we found that the geometrical method outperforms the statistical method in most cases. Generally speaking, the RLAB space is the best color space for color transfer, seemingly followed by the lαβ space. It is clear that the best color space and method depends on the specific contents of source and target images. We are still addressing what dependencies should be considered. Color transfer in the present study is only from image to image, and we will evaluate color transfer from swatch to swatch in future work. 2016-ZJ-745, the Project of the Chongqing Nature Science Foundation under Grant No. cstc2019jcyj-msxmX0683, and the National Nature Science Foundation of China under Grant Nos. 61662062 and 61602088.