The Optimum Frequency Band Partition Based on Kullback–Leibler Divergence in Subband Image Coding

In order to improve the coding performance of subband image coding, the optimum frequency band partition (OFBP) based on the parameters of subband signal such as signal power or kurtosis has been proposed. The coding performance of two kinds of OFBP is quite different for input image or coding rate. When an input image is given, for instance, OFBP using the signal power is superior at the lower bit rate, on the other hand, OFBP using the kurtosis is superior at the higher bit rate. In this paper, it is clarified that the similarity between the probability density functions (PDFs) in subband signals is closely related to the improvement of coding performance of the OFBP. We propose a new method to determine the optimum partition pattern on the 2-dimensional frequency domain using the Kullback– Leibler divergence (KLD) which is known as a similarity measurement between given two PDFs. It is seen in the experimental results that the PSNR of the proposed method is at most 3.0[dB] larger than the previous method in OFBP and JPEG at the same coding rate.


Introduction
Subband coding is one of powerful means for highly efficient coding of image signals [1].In order to improve the coding performance of the subband coding, it is effective to change the partition pattern on the 2-dimensional frequency domain adaptively in consideration of the stochastic characteristics of input image.Wavelet packet [2] is a representative one in such a method, however, this method has the problem that the computational costs to determine the partition pattern is larger than the general wavelet transform, because the number of subbands in the wavelet packet is too many.In order to the problem of the computational costs in the wavelet packet, the optimum frequency band partition [3] (OFBP) with small number of subbands has been proposed as a method that can determine the optimum partition pattern on the 2-dimensional frequency domain in the view point to maximize the coding gain [4], when the coding rate is given.
Applying the OFBP to any real-world image, first, the 2-dimensional frequency domain is divided into a set of small sized blocks which is called the band blocks.In the next, each subband is reconstructed by the adequate set of band blocks to maximize the coding gain.In our previous method, the signal power or the kurtosis has been used as the parameter for band block signals [5].It has been seen in the experimental results using the previous method that the partition pattern on the 2-dimensional frequency domain by the signal power is different from the one by the kurtosis.The coding performance of two kinds of OFBP is quite different for input image or coding rate.When an input image is given, for instance, OFBP using the signal power is superior at the lower bit rate, on the other hand, OFBP using the kurtosis is superior at the higher bit rate.The reason why the parameter can't be decided uniquely to obtain the optimum solution that both two parameters are not faithful to the change of the signal in band block, when each subband signal is quantized.
This paper proposes a new OFBP method to minimize the entropy of the coded image, after all band block signals are quantized, namely, the image quality of the coded image is determined.To minimize the entropy in OFBP in our proposed method can solve to minimize the Kullback-Leibler divergence [6] (KLD) among the probability density where KLD is one of similarity measurement between given two PDFs.We propose an algorithm to realize the proposed method which means the optimum partition pattern on the 2-dimensional frequency domain based on the KLD between PDFs of band blocks.In our experiments using some test images, the coding performance by the new partition pattern is improved compared with both the previous method of the OFBP and JPEG.

Optimum Frequency Band Partition
The optimum frequency band partition [2] (OFBP) is defined as a method that is divided the 2-dimensional frequency domain of an input image into M subbands to maximize the coding gain [4] .(1) In Eq.(1), σ k 2 and λ k is the signal power and the coding rate in subband   ( = 0, 1, … ,  − 1), respectively.In order to apply the OFBP to any input image, the optimum solution is realized approximately so that each subband is reconstructed by a set of band blocks   ( = 0, 1, … ,  − 1;  ≫ ) to maximize the G V .Here, the band block means a small region on the frequency domain which is calculated by the 2-dimensional separable filter bank using QMF [1].To maximize the coding gain of Eq.( 1) is equivalent with each subband is reconstructed by the set of band blocks with similar signal power.On the other hand, it is possible to obtain the partition pattern on the 2-dimensional frequency domain using the kurtosis which is an another parameter to express the feature of band block signal instead of the signal power [5].Fig. 1 shows the examples of the optimum frequency band partition pattern for image "Lenna" in the previous method.Fig. 1(a) and Fig. 1(b) illustrates the optimum partition pattern based on the signal power and the kurtosis of band blocks, respectively.In this example, band blocks N=64 are classified into subbands Ω 0 ~Ω4 (M=5), and a linear quantizer with the quantization level=256 is applied to all band blocks.It is seen in Fig. 1 that the partition pattern on the 2-dimensional frequency domain by the signal power is different from one by the kurtosis.Moreover, it has been clarified from the experimental results that the coding performance of two kinds of OFBP is quite different for input image or coding rate.When an input image is given, for instance, OFBP using the signal power is superior at the lower bit rate, on the other hand, OFBP using the kurtosis is superior at the higher bit rate.Therefore, it is necessary to define a new parameter which corresponds to the change of band block signal by the quantization accurately to improve the coding performance in the OFBP.

Basic Idea
It is well known that the entropy is given by the probability density function (PDF), is used as a measurement amount of information, where () is probability for an event .In order to discuss the entropy, namely, the coding rate in the OFBP, we focus on the PDF of band block signal.When two band blocks whose PDFs are given as () and (), the simple problem in the OFBP is which measure is better (i) () and () are grouped into same subband, or (ii) () and () are divided into different subbands, in the viewpoint to reduce the total entropy.According to Eq.( 2), the entropy in the case of the measure (i) and (ii) is calculated by and , respectively.If () () is close to () () , it is desirable to choose the measure (i), because the computational costs by applying band partitioning to an original signal is laborious, in spite of the reduced entropy by the band partitioning is a trifling amount.Contrary to   this, if () () is much smaller than () () , it is desirable to choose the measure (ii).In fact, we will show two typical examples by using band blocks for image "Lenna".Fig. 2 shows a pair of PDFs of band blocks, where Fig. 2(a) and Fig. 2(b) is correspond to red frames and green frames are in respectively.Moreover, () () () Fig. 2 denote entropies in the case of Eq.(3) and Eq.( 4).And, ()  is the difference value between () () and () () , it means the amount of reduced entropy by applying band partitioning.It is seen in Fig. 2 that ()  in the case of Fig. 2(a) is smaller than ()  in the case of Fig. 2(b), and the two PDFs in Fig. 2(a) are similar each other.On the other hand, two PDFs in Fig. 2(b) are significantly different each other.According this observation, if two PDFs are much different each other, it should choose the measure (ii) to reduce the entropy in the OFBP.Therefore, it is need to define a new parameter which can evaluate the similarity between the PDFs to obtain the optimum frequency band partition in the viewpoint of the entropy minimization.In [6], it is possible to measure the similarity between the PDFs by using the Kullback-Leibler divergence (KLD), which is known as a similarity measurement between given two PDFs.The KLD between two PDFs () and () is defined by

𝐷 (𝑃||𝑄) = ∑ 𝑃(𝑥) 𝑙𝑜𝑔 𝑃(𝑥) 𝑄(𝑥) .
(5) After the PDFs of all band blocks are given, the OFBP to minimize the total entropy is formulated as the problem to obtain the partition pattern on the 2-dimensional frequency domain to minimize KLD among PDFs of band blocks in each subband.

Derivation of New OFBP
We will formulate the entropy minimization problem in the OFBP under the condition that a same quantizer is applied to all band blocks.Let us define the probability density functions (PDFs) of band blocks   ( = 0, 1, … ,  − 1) and subband   ( = 0, 1, … ,  − 1) is as   () and   () , respectively.The total entropy for the all subband signals which is obtained by the partition pattern on the 2-dimensional frequency domain can compute by In Eq.( 6),   is the number of band blocks in subband   , and  and  are the minimum and maximum value of  with   () ≠ 0.Then, PDF of subband   () is replaced by using   ().Furthermore, when is applied to Eq.( 6), is obtained.Based on Eq.( 5), term 1 in Eq.( 9) is considered as the average of Kullback-Leibler divergences (KLDs) between the PDF of each band block   () in   () and PDF of subband   ().Also, term 2 in Eq.( 9) is the average of entropies using PDFs of all band blocks   ().Because term 2 is independent on the result of the partition pattern, minimizing  of Eq.( 9) can be seen as equivalent To minimize  means that each subband   is reconstructed by the band blocks   whose PDF are similar each other.Therefore, it is clarified that the entropy of the OFBP can be reduced by minimizing the KLDs among the PDFs of band blocks in each subband.

Algorithm
We propose an algorithm to determine the optimum frequency band partition (OFBP) based on the similarity between the probability density functions (PDFs) of band blocks.However, it is difficult to search the partition pattern to minimize  in the viewpoint of the computational costs, because the number of the partition patterns is enormous under the condition of OFBP is N=64 and M=5.On the other hand, to maximize the coding gain in Eq.( 1) is equivalent that each subband is reconstructed by the set of band blocks with similar value of parameter.In order to maximize the coding gain in Eq.(1) after the signal power of band block is given, first, we have sorted all band blocks in descending order as S  = {∆ , ∆ , … , ∆  , … , ∆ }, where ∆  means the -th band block.In such case, the optimum solution is contained in the restricted partition pattern (11) , where ( , ) is the -th subband in the partition pattern S  .Therefore, it is possible to determine the optimum partition pattern with small computational costs by searching the restricted partition pattern under the especial condition of Eq.(11).At this time, the costs required for search can be reduced from 5.42 × 10 44 to 595,665 under the above condition.Similarly, we determine the optimum partition pattern to minimize  in Eq.(10) by sorting the band blocks according to the similarity of PDF.In our proposed method, first, all band blocks is sorted in descending order of signal power, because the signal power is considered as one of parameter which is correspond to the shape of PDF approximately.

Experimental Results
The proposed method is applied to test image "Lenna" and "Barbara" under the condition that band blocks N=64 classified into M=5 subbands, and a linear quantizer with the quantization level (256,128,64,32,16) is applied to all band blocks.Table .1 and Table.2 shows the results of the PSNR vs. entropy of the proposed method for "Lenna" and "Barbara", respectively, compared with the previous Fig. 3. Optimum frequency band partition by proposed method ("Lenna").2 that the entropy of the proposed method is reduced than the previous OFBPs.Fig. 3 illustrates the partition pattern of the proposed method for image "Lenna" when a linear quantizer with the quantization level=256 is applied to all band blocks.It is seen in Fig. 3 compared with Fig. 1 that the area of subband Ω 1 and Ω 2 in Fig. 3 tend to be smaller than two partition patterns are shown in Fig. 1, because the shape of band block PDFs in the lower-frequency domain are different each other.Also, Fig. 4 shows the PSNR vs. entropy characteristics of the proposed method for "Lenna" compared with JPEG.It is seen in Fig. 4 that the proposed method is superior about 3.0[dB] than JPEG at the lower coding rates.

Conclusion
In this paper, we have proposed a new optimum frequency band partition (OFBP) method that the partition pattern on the 2-dimensional frequency domain to minimize the entropy for the coded image.is clarified that the entropy minimization problem in the OFBP can solve to minimize the Kullback-Leibler divergence (KLD) among the probability density functions (PDFs) of band block signals in each subband.In the experimental results, the entropy of the proposed method is reduced than the previous OFBP, and the PSNR of the proposed method is about 3.0[dB] larger than JPEG at the same coding rate.
However, our proposed method OFBP has the problem that the computational costs to determine the partition pattern is larger than the previous OFBP, because the calculation of KLD between PDFs requires a several hundred histogram bins.To propose a new simple parameter which can evaluate the similarity between the PDFs instead of the KLD will be defined as a future work.

1 .
Optimum frequency band partition by previous method ("Lenna").) of band block signals in each subband, Comparison of two PDFs of red frames in Fig.1.(b) Comparison of two PDFs of green frames in Fig.1.
using signal power and kurtosis as parameter of band blocks.It is seen in Table.
1 and Table.