Optimal Binary Descriptor for Pre-Classification in Efficient Non-Local Mean Image Denoising

With the popularity of consumer digital imaging devices, image denoising has become an important issue in image processing. Among currently available image denoising algorithms, non-local mean (NLM) is one of the most effective methods. NLM calculates weights of neighboring pixels based on the similarity between two image patches. The pixel is then estimated by the weighted sum of the neighboring pixels. Because the large number of the patches needs to be compared, NLM incurs high computational cost and this makes it impractical in many situations. On the other hand, a binary descriptor produces a simple binary string to describe an image patch by comparing pixels with a given threshold. Comparing binary descriptors between two image patches is much more efficient than comparing the two patches directly. Preliminary results show that using binary descriptors to reject dissimilar patches from the computation can significantly improve the performance of NLM. In this paper, we seek to find the best binary descriptor which can provide large speedup and enhance denosing performance. Experimental results show that using two types of 18neighbor binary descriptors can effectively increase the denoising performance of NLM and significantly reduce the execution time.


Introduction
With the popularity of consumer digital imaging devices, image denoising has become an important application in image processing especially when images are acquired in undesirable condition such as in poor illumination or under high ISO setting.One of the most effective image denoising methods currently available is the non-local mean (NLM) (1,2) method.For a pixel to be denoised, NLM first calculates weights for its neighboring pixels based on the similarity between the image patch of the current pixel and the image patches of the neighboring pixels.The current pixel is then replaced by the weighted sum of its neighboring pixels and itself.Because pixels from similar patches are given large weights, the collection of similar pixels can effectively remove unwanted noise if a large number of image patches are considered.
Among these three categories of acceleration techniques, the first category has an additional advantage of producing images with less blurring.This is because that the small weights given to pixels from dissimilar patches eventually accumulate to a significant amount and bias the estimation (24) .Rejecting dissimilar patches can remove these small weights from computation and produce less blurred image.However, additional statistical parameters such as means or gradients need to be calculated before the NLM process and this adds additional complexity in the algorithm.In a preliminary study (25) , we have shown that using binary descriptors in the pre-classification can significantly improve NLM in both reducing computation time and enhancing denoising performance.A binary descriptor of an image patch compares the pixels in the patch with a given threshold and produces a simple binary string to describe the patch.The advantages of binary descriptors are that they are very simple to generate and the cost of comparison between two descriptors is extremely low.Therefore, the computation overhead is very low.
Encouraged by these preliminary results, we seek to find the best binary descriptor for NLM in this paper by extensively testing various binary descriptors.Overall, 50 binary descriptors have been tested.Experimental results show that using two types of 18-neighbor binary descriptors can effectively increase the denoising performance of NLM and significantly reduce the execution time.Therefore, our conclusion is that these two types of binary descriptors are the optimal choice in NLM pre-classification.
The rest of the paper is organized as follows.In the second section, we give a brief introduction on NLM algorithm and present the tested binary descriptors.In the third section, we show the experimental results.In the fourth section, we conclude.

Non-Local Mean
Let X be the original noise-free image, N X Y   be a noisy image with an additive Gaussian noise N of zero mean and variance  2 , and X ˆ be the denoised image estimated from Y .We assume that the pixel values of all the images are normalized to the range between 0 and 1.Let where 2  is the Euclidean norm and h is an adjustable parameter.
The estimated pixel value is the weighted mean:

Binary Descriptors
A binary descriptor of an image patch compares the pixels within the patch with a given threshold and produces an integer to describe the patch.

Let
) ( q be a binary operator (i.e., a quantizer) defined as below: Based on previous experimental results (25) , T h is set to 0.5 and we called this descriptor "simple binary pattern" (SBP).The relative positions of the pixels to be considered in the description with respect to the center of the patch are given by a set of vectors V: For example, if the center of patch and its four neighbors are considered in the description, the set V is: If N neighbors are considered, we called the descriptor a "Nneighbor description".Therefore, the above example is a 4neighbor description.
The description for a patch m Y is calculated as: To put it simply, the description calculates the number of ones within the pixels being considered and it can be as a quantized mean estimator.
For our NLM algorithm, only when , the weight nm w in (1) is calculated.Otherwise, the weight nm w is set to zero.In other words, only patches with the same binary description are included in the denoising calculation of (2).

Tested Binary Descriptors
Although preliminary study (25) shows that 12-neighbor and 16-neighbor SBP's produce better results than other descriptors, there are only a very limited number of descriptors being tested in the study.In order to comprehensively understand the behavior of SBP, we tested five kinds of SBP based on the number of neighbors being included in the description.They are: 10-neighbor, 12neighbor, 14-neighbor, 16-neighbor, and 18-neighbor SBP's.For each kind of SBP, 10 variations on the relative positions of neighboring pixels are studied.Overall, 50 SBP's have been tested.Fig. 1~5 shows the representative variations which have better performance than the other SBP's being tested.

Experimental Results
The SBP pre-classified NLM algorithm was compared with the original NLM algorithm (1,2) and the accelerated NLM algorithm using the mean of the image patch for classification (3)(4)(5) .For all algorithms, h is set to 100 2 .The mean of a patch is calculated over a 7x7 neighborhood and weights are only calculated for patches with means within % 10  of the mean of the current patch.The reason that mean-classified NLM is chosen for comparison is that the SBP can be viewed as an approximate estimator of the pixel sum and it is directly proportional to the mean.
Images from USC-SIPI image database were tested under the platform of a personal computer with 3.6 GHz Intel Core i7-4790 CPU and 8 GB of memory.All algorithms are implemented in C under Microsoft Visual Studio Community 2015.
For the 10-neighbor SBP in Fig. 1, type-10B and type-10C are better and the performances of both descriptors are almost the same because they are symmetric.For the 12neighbor SBP in Fig. 2, type-12D is better.For the 14neighbor SBP in Fig. 3, type-14B and type-14D are better.When the number of neighbors in the descriptor increases, there are more variations in the performance among different types.For the 16-neighbor SBP in Fig. 4, type-16F and type-16H are better for most cases but type-16A, type-16B, type-16E, and type-16G are better for a few cases.For the 18neighbor SBP in Fig. 5, type-18C and type-18D are better for most cases but type-18A and type-18B are better for some cases.
Fig. 6 shows PSNR vs. the standard deviation  of the additive Gaussian noise for four of the images tested.Since the differences among the better-performed SBP's with the same number of neighbors are small, we only show the (a) ( Fig. 8 shows PSNR vs. the number of neighbors in the descriptors for  = 20 and  = 60.From the figure, we can see that the differences among different descriptors are small when noise is small.However, when noise is large ( = 60), performance becomes better as the number of neighbors increases but the improvement flattens out as the number of neighbors reaches beyond 16.The largest improvement is 0.7502 dB from type-10B to type-18D for the image Lena.
Fig. 9 shows the images produced by the original NLM, the mean-classified NLM, the type-10B SBP NLM, and the type-18D NLM when  =40.From the figures, we can see that SBP-classified NLM can preserve image details better and it is still competitive for the image Lena despite the fact that its PSNR is not better than the other two algorithms.
From these results, we can observe that neighbors whose distance are closer to the center of the patch are more important to the denoising performance than neighbors that

Conclusions
In this paper, the optimal choice of using binary descriptor to reject dissimilar patches in NLM has been studied through extensive experiments.Experimental results show that using two types of 18-neighbor simple binary descriptors can effectively maintain the denoising performance of NLM while significantly reduce the execution time.Therefore, our conclusion is that these two types of binary descriptors are the optimal choice in NLM pre-classification.

Fig. 6 .
PSNR vs. standard deviation of additive Gaussian noise for four images: (a) Baboon, (b) F-16, (c) House, and (d) Lena.are further away from the center.As the number of neighbors increases, there are more neighbors of the same distance to choose from and the choices of the best outlying neighbors depend more on the characteristics of the image.Summarizing from the experimental results, our conclusion is that type-18C and type-18D SBP's are the optimal choices.

Fig. 7 .
Fig. 7.The average execution time for all the methods
2 let m S be a square search window centered at m and extended to s pixels in all four directions, i.e., the size of the search window is (2s+1)2.For denote the pixel value at location m and n Y denote the image patch which is centered at location n .The patch n Y extends to k pixels around n in all four directions (top, bottom, left, and right), i.e., the size of the patch is (2k+1) 2 .For a location m