Efficient Non-Local Means Image Denoising Using Halftone Image for Pre-Classification

Non-local means (NLM) is one of the most effective image denoising methods among currently available algorithms. NLM calculates weights of neighboring pixels based on the similarity between two image patches. The pixel is then estimated by the weighted sum of the neighboring pixels. Because the number of patches needed to be calculated for their similarities are very large, NLM incurs extremely high computational cost. In the meantime, small weights given to pixels from dissimilar patches could accumulate to a significant amount and bias the denoising results. Rejecting dissimilar patches can remove both the similarity computation and the small weights associated with it, thus, improve the denoising performance. However, comparing two patches directly requires additional computation and removes some of the benefits in dissimilar patch rejection. On the other hand, a halftone image is a binary approximation of the original image and it can be generated with very little computation using error diffusion. Therefore, in this paper, we propose to use the error-diffused halftone image of the input as an efficient way to pre-classify patches before weight calculations in NLM. Experimental results show that using halftone image as a reference can effectively increase the denoising performance of NLM and significantly reduce the execution time.


Introduction
Image denoising is an important step in the process of image acquisition especially in undesirable conditions such as poor illumination or low contrast.One of the most effective image denoising algorithms currently available is non-local means (NLM) (1,2) .For a pixel to be recovered, NLM first calculates weights for each of its neighboring pixels based on the similarity between the image patch of the current pixel and the image patches of the neighboring pixels.The output for the current pixel is the weighted sum of its neighboring pixels and itself.Because pixels from similar patches are given larger weights, the averaging of similar pixels can effectively remove unwanted noise if a large number of image patches are considered.
Among these three categories, dissimilar patch rejection of the first category has an additional benefit of producing images with less blurring.This is because that the small weights given to pixels from dissimilar patches eventually accumulate to a significant amount and bias the estimation (24) .However, additional statistical parameters such as means or gradients need to be calculated and this computation adds additional complexity in the algorithm.Therefore, the motivation of this paper is to find a simplified approach for the pre-classification process.Our idea is that a halftone image is a binary approximation of the original image and it can be generated with very little computation using error diffusion (25) .Thus, we propose to use the error-diffused halftone image of the input as an efficient way to pre-classify patches before weight calculations in NLM.Experimental results show that using halftone image as a reference for preclassification can effectively increase the denoising performance of NLM and significantly reduce the execution time.
The rest of the paper is organized as follows.In the second section, we present the NLM algorithm, the error diffusion halftone algorithm we used, and the proposed NLM algorithm.In the third section, we show the experimental results.In the fourth section, we conclude.The NLM algorithm works as follows:

Non-Local Means and Error Diffusion
For a location , let be a square search window centered at and extended to pixels in all four directions such that the size of the search window is 2 1 .For each patch where ∈ , the weight is calculated as follows: where ‖•‖ is the Euclidean norm and is an adjustable parameter.
Then, the estimated pixel value is the weighted mean:

Error Diffusion
Halftoning is the process of generating a binary image which approximates the original image.Its original use is to print an image to a device which can only reproduce on and off pixels.Error diffusion is one of the most efficient halftoning algorithms because it can generate a halftone image in only one pass with simple calculations.It works by adjusting pixel value through subtracting errors made at previous pixel locations.In this paper, we use the error diffusion algorithm proposed by Floyd and Steinberg (25) .The reason to choose this algorithm is because of its efficiency and simplicity.An error-diffusion halftoning algorithm works as follows: where is the halftone output, is the error made at previous pixel locations, and • is a quantizer defined as below: For Floyd-Steinberg algorithm, the values of are defined as follows:

Proposed Non-Local Mean Algorithm
With the halftoned image as a reference, only image patches with the same halftoned pattern are selected for the weight calculation in Equation (1).In other words, only patches with the same halftoned pattern are included in the denoising calculation of Equation (2).However, to simplify the halftoned pattern comparison and prevent too many patches being excluded from denoising calculation, we sum the binary values of the current pixel and its surrounding 8 pixels and use the sum for comparison instead of the entire binary halftoned pattern.

Experimental Results
Our proposed algorithm was compared with the original NLM algorithm (1,2) .For all algorithms, h is set to 100 2 .Images from USC-SIPI image database were tested.All algorithms are implemented in C under Microsoft Visual Studio Community 2017.The test platform is a personal computer with Intel Core i7-7700 CPU running at 3.6 GHz and 8 GB of memory.
Table 1 shows the average execution times for the two methods.From the table, we can see that the execution time for the proposed method is only 1/3 of the original NLM or, to be exact, a 66.1% reduction in execution time.This indicates that the proposed NLM does have significant advantage in terms of reducing computational complexity.From the two figures, we can see that the proposed NLM can preserve image details much better.In Fig. 4(c), it is worth noting that the image "Hill" from the proposed method is still visually better than that from the original NLM despite the fact that the PSNR's of the two images are almost the same in Fig. 1(b).
As a reference, the halftoned images of the input are shown in Fig. 5. From the figure, we can see that a slight disadvantage of the proposed method is that the deonised image preserves some of the characteristics of the halftoned image.The way to reduce this influence will be the direction for our future researches.

Conclusions
In this paper, we show that using error diffusion halftoned image as a reference to reject dissimilar patches from consideration can significantly reduce the computation time while still increase the denoising performance in NLM.Because error diffusion halftoned image is much simpler to generate than previously proposed parameters such as mean and gradients, it has significant advantage as compared to other approaches in complexity reduction for NLM.

2. 1
Non-Local Means Let be the original noise-free image and is the noisy image with being by a Gaussian noise with zero mean and variance  2 .Let be the estimated image from a noisy image .We assume that the pixel values of all the images are normalized to the range between 0 and 1.Let , and ′, ′ denote two coordinates in the images.Let denote the pixel value at location and denote the image patch which is centered at location .The patch extends to k pixels around , i.e., the size of the patch is (2k+1) 2 .

Fig. 1
Fig. 1 shows PSNR vs. the standard deviation  of the additive Gaussian noise and Fig. 2 shows SSIM vs  for four of the images tested.From the figure, we can see that PSNR of the proposed NLM method performs better than the original NLM when  is less than 40 while SSIM of the proposed method are better in the entire test range.The largest gap of PSNR between the two methods is 1.45 dB at = 20 for the image Baboon, 0.98 dB at = 30 for the image Boat, 0.51 dB at = 20 for the image Hill, and 1.56 dB at = 30 for the image House.The largest gap of SSIM between the two methods is 0.26 at = 40 for the image Baboon, 0.07 at = 30 for the image Boat, 0.13 at = 30 for the image Hill, and 0.10 at = 30 for the image House.

Fig. 3
Fig. 3 and 4 shows the images generated by the proposed method and original NLM at  =20 and  =40, respectively.

Table 1 .
The average execution times for 512 512 images.