Efficient Non-Local Image Denoising Using Binary Descriptor Classification

Non-local mean (NLM) is one of the most effective image denoising methods currently available. It calculates weights of neighboring pixels based on the similarity between two image patches. The pixel is then estimated by the weighted sum of the neighboring pixels and itself. Because the large number of the patches needs to be compared, NLM incurs extremely high computational cost. In this paper, we propose to use binary descriptors to reject dissimilar patches from the computation and improve the performance of NLM. A binary descriptor of a patch compares the pixels in the patch with a given threshold and produces a simple binary string to describe the patch. It is very simple to generate and the comparison between two descriptors is computationally efficient. Experimental results show that using a simple binary descriptor can effectively increase the denoising performance of NLM and significantly reduce the execution time.


Introduction
With the popularity of digital imaging devices, image denoising has become an important application in image processing.One of the most effective image denoising methods is the non-local mean (NLM) (1,2) .For a pixel to be denoised, NLM first calculates weights for its neighboring pixels based on the similarity between the image patch of the current pixel and the image patches of the neighboring pixels.The current pixel is then replaced by the weighted sum of its neighboring pixels and itself.Because pixels from similar patches are given large weights, the collection of similar pixels can effectively remove unwanted noise in practice.
Because the area of the neighborhood to be searched usually needs to be sufficiently large so that the non-local nature of the algorithm can be exploited, a large number of image patches need to be compared and their corresponding weights to be calculated in order to find the weighted sum of the current pixel.This means that NLM incurs extremely high computational cost and leads to many researches in reducing the computational burden.These acceleration methods can be divided into three categories.The first category is to avoid dissimilar patches by pre-classification (3- 9) .The criteria used to reject image patches include: means (3,5,7) , variances (5,7) , higher-order statistical moments (4,6) , gradients (3) , and pixel intensities (8,9) .The second category is to reduce the dimensions of the image patches so that the comparison between patches can be simplified (10)(11)(12)(13)(14)(15) .The dimension-reduction techniques include principle component analysis (10,11,(13)(14)(15) and singular value decomposition (12) .The third category is to speed up the computation by using either efficient algorithms or data structures (16)(17)(18)(19)(20)(21)(22)(23) .The efficient algorithms being used are random selections (16,23) , fast Fourier transform (17,18,22) , squared image (18,19) , and probabilistic early termination (21) .The data structures for improving efficiency include Laplacian pyramid (18) and multiresolution representations (20) .
Among these three categories of acceleration techniques, the first category has an additional advantage of producing images with less blurring.This is because that the small weights given to pixels from dissimilar patches eventually accumulate to a significant amount and bias the estimation (24) .By rejecting dissimilar patches in the weight-calculating process, the bias from these small weights can be removed and result in an image of better quality.However, in order for the pre-classification to work, additional statistical parameters such as means or gradients need to be calculated before the NLM process.This adds additional complexity in the algorithm.Therefore, the motivation of this paper is to find a simplified parameter for the pre-classification process while still maintain the advantages of more complex parameters.To this end, we propose to use binary descriptors in the pre-classification of NLM.A binary descriptor of an image patch compares the pixels in the patch with a given threshold and produces a simple binary string to describe the patch.The reasons to choose binary descriptors as an alternative to other statistical parameters are that they are very simple to generate and the cost of comparison between two descriptors is extremely low.
In this paper, we study two kinds of binary descriptor.The first is called "simple binary pattern" (SBP) which uses half of the maximum pixel value as the threshold.The second is a very popular binary descriptor called "local binary pattern" (LBP) (25) which uses the value of center pixel as the threshold.Only image patches with the same description are considered in the weight calculation.The experimental results show that using SBP can effectively increase the denoising performance of NLM and significantly reduce the execution time.
The rest of the paper is organized as follows.In the second section, we present the binary descriptors and the modified NLM algorithm used in this paper.In the third section, we show the experimental results.In the fourth section, we conclude.

Non-Local Mean
Let X be the original noise-free image, X ˆ be the estimated image from a noisy image Y which is interfered by an additive Gaussian noise with zero mean and variance  2 .
We assume that the pixel values of all the images are normalized to the range between 0 and 1.Let  is the Euclidean norm and h is an adjustable parameter.
Then, the estimated pixel value is the weighted mean:

Binary Descriptors
A binary descriptor of an image patch compares the pixels within the patch with a given threshold and produces an integer to describe the patch.

Let
) ( q be a binary operator (or a quantizer) defined as below: ( The relative positions of the pixels to be considered in the description with respect to the center of the patch are given by a set of vectors V: For example, if the center of patch and its four neighbors are considered in the description, the set V is: In this paper, we considered four variations of neighbors in the description.We called these variations: 4-neighbor, 8neighbor, 12-neighbor, and 16-neighbor descriptions.The pixels used in these four variations are shown in Fig. 1.
The description for a patch m Y is then calculated as: To put it simply, the description calculates the number of ones within the pixels being considered.Some descriptors multiply a weight of 2 i-1 to each of the quantized values in the summation of (4).However, adding the weights leads to too many different classes in the description and, in turn, results in too few patch matches in NLM.We find that adding weights in the calculation does not produce desirable results in denoising.Therefore, the weights are not included in the description.Two kinds of binary descriptor are considered in this paper.The first is called "simple binary pattern" (SBP) and its threshold, Th, in (3) is equal to 0.5.The second is called "local binary pattern" (LBP) and its threshold is equal to the value of the center pixel, ) (m Y .

Descriptor Classification
With the binary descriptors defined, the original NLM algorithm is modified as follows:

Experimental Results
Our proposed algorithm was compared with the original NLM algorithm (1,2) and the accelerated NLM algorithm using the mean of the patch for classification (3,5,7) .For all algorithms, h is set to 100 2 .The mean of a patch is calculated over a 7x7 neighborhood and weights are only calculated for patches with means within % 10  of the mean of the current patch.These parameters are directly adopted from reference 3. The reason that mean-classified NLM is used for comparison is that the binary descriptor we used can be viewed as an approximate estimator of the sum of the pixels being considered and this sum is directly related to the mean.
Images from USC-SIPI image database were tested.All algorithms are implemented in C under Microsoft Visual Studio Community 2015.The test platform is a personal computer with Intel Core i7-4790 CPU running at 3.6 GHz and 8 GB of memory.Fig. 2 shows the average execution time for all the methods.In the figure, we can see that the execution time for 12-neighbor SBP is only 28% of the original NLM and 45% of the mean-classified NLM.This indicates that SBP NLM does indeed have significant advantage in terms of reducing computational complexity.Fig. 3 shows PSNR vs. the standard deviation  of the additive Gaussian noise for four of the images tested.From the figure, we can see that 12-neighbor SBP generally performs better than mean-classified NLM when  is less than 40.Even when it is not better than mean-classified NLM when  is larger, it is still better than the original NLM.The largest gap between 12-neighbor SBP NLM and the original NLM is 2.5 dB which occurs at = 30 for the image House.
The only exception is for the image Lena but the performance of SBP NLM is still on the par with the original NLM.In the worst case, PSNR of 12-neighbor SBP NLM is 0.57 dB lower than that of the original NLM which happens at = 50 for the image Lena.However, it is worth noting that the execution time for SBP NLM is much less than that of the original NLM as shown in Fig. 2. Therefore, we believe that 12-neighbor SBP NLM is still better than the original NLM even in the worst case scenario.The reason why SBP  NLM does not perform as well as mean-classified NLM in some images still needs further investigation.On the other hand, the performance of LBP NLM is worse than the other approaches.Therefore, we conclude that LBP is not suitable as a classifier in NLM.Fig. 4 shows the images produced by the original NLM, the mean-classified NLM, the 12-neighbor SBP NLM, and the 4-neighbor LBP NLM when  =30.From the figures, we can see that SBP can preserve the details of the image much better than the other approaches.This fact is the most distinctive in the image Baboon.Therefore, despite the fact that PSNR for SBP is not better than the original NLM for the image Lena.Subjectively, SBP is still competitive.
The reason that SBP is effective can be attribute to the fact that SBP can be viewed as a mean estimator for the quantized values of the pixels.In other words, SBP provides an approximation of the original mean.Therefore, it can approach the performance of mean-classified NLM when the approximation is accurate.On the other hand, the purpose of an LBP is to describe the relationship between pixels and it appears that this relationship is not a good classifier for NLM.Therefore, our conclusion is that SBP is a good classifier in accelerating NLM.

Conclusions
In this paper, we show that using a simple binary descriptor to reject dissimilar patches from consideration can significantly reduce the computation time while still increase the denoising performance in NLM.Because binary descriptors are much simpler to calculate than previously proposed parameters such as mean and gradients, it has significant advantage as compared to other approaches in complexity reduction for NLM.
two locations in the images.Let ) (m Y denote the pixel value at location m and n Y denote the image patch which is centered at location n .The patch n Y extends to k pixels around n in all four directions (top, bottom, left, and right), i.e., the size of the patch is (2k+1) 2 .The NLM algorithm works as follows:For a location m , let m S be a square search window centered at m and extended to s pixels in all four directions, i.e., the size of the search window is (2s+1)2 .For each patch nm w in (1) is calculated.Otherwise, the weight nm w is set to zero.In other words, only patches with the same binary description are included in the denoising calculation of (2).

Fig. 1 .
Fig. 1.The relative positions of the pixels consider in the description.The cross indicates the location of the center pixel: (a) 4-neighbor, (b) 8-neighbor, (c) 12-neighbor, and (d) 16-neighbor descriptions.

Fig. 2 .
Fig. 2. The average execution time for all the methods