Sharpening Interpolated Image by Using Cube of Pixel Difference and Evaluation in Frequency Domain

In order to improve image resolution, some interpolation has to be applied. However, in general, an interpolation generates smooth connections between adjacent sampling points and causes blurred edges in the interpolated image. To suppress such blur in edges, it is desired that some sharpening effect is included in the interpolating process. In this paper, we propose a sharpening method using cubic operations of the differences of pixel values. Experimental results show that this method produces sharp high resolution image which is more similar than that by any other method to a target image.


Introduction
Improving image resolution is a process for making a high resolution image from a low resolution image. In order to improve image resolution, some interpolation has to be applied. However, in general, interpolation generates smooth connections between adjacent sampling points and causes blurred edges in the interpolated image. To suppress such blur in edges, it is desired that some sharpening effect is included in the interpolating process.
For sharping images, conventional linear filters such as Laplacian operator and unsharp masking based on Gaussian filter are often used in general. A cubic unsharp masking method is proposed by G. Ramponi [1] [2]. The method uses an operator which consists of the product with Laplacian and a quadratic function of the local gradient. The cubic unsharp masking method performs effectively for the images with noise since the method suppresses the noise enhancement. However, for a noiseless image the method may not produce better result than any other sharpening method.
In order to produce a high resolution image which has extra-high frequency components, a method using cube of high pass filter is proposed [3]. In the paper, the cube of Laplacian filter is applied to video images to produce high resolution images of them and the validity of the method are shown for producing high frequency components.
In this paper, we propose a sharpening method which is not cube of linear filter but uses weighted sum of cubic operations of the differences between a value of pixel and values of the surrounding pixels. Experimental results for an interpolated image show that this method produces a sharp high resolution image which is more similar to a target image than that by any other sharpening method. * Corresponding: ono@design.kyushu-u.ac.jp † Faculty of Design, Kyushu University, Shiobaru 4-9-1, Minami-ku, Fukuoka Japan 815-8540

Sharpening for interpolated image 2.1 Unsharp masking by liner operation
The fundamental idea of unsharp masking is to add high frequency components to the input image. The high frequency components of an input image is obtained by subtraction of a low pass filtered version of the input image from the input image itself. In other words, the input image is modified by its own high frequency component.
Given interpolated image {d i j (i=1, · · · , M; j=1, · · · , N)}, an unsharp masked image f i j , which is derived by the standard unsharp masking with linear low pass filter, is given by whered i j is a low pass filtered version of d i j . For making a low pass filtered imaged i j , using Gaussian filter the linear unsharp masking (LUM) produces an enhanced image f gi j described by where α(> 0) and p are parameters which smoothness and size of mask respectively. Furthermore we control the effect of sharpening by a parameter δ(> 0).

Sharpening by cube of liner masking
In the process of unsharp masking, if we adopt the cube of the difference given by the subtraction of a low pass filtered version from the input image as the high frequency components, slight changes in the image can be enhanced and detail information becomes more visible effectively. According to this idea, an enhanced image f ci j is produced by The second term in eq.4 is the cube of LUM and the input image d i j is modified by the cube of LUM. Using the cube of Laplacian instead of second term in eq.4 as the high frequency component, the sharpening procedure is equivalent to that in [3].

Sharpening by using cube of pixel differences
We propose a sharpening method which is not cube of linear filter but uses weighted sum of cubic operations of the differences between a value of pixel and values of the surrounding pixels, i.e.
This method also enhances slight changes effectively. However, since the value of modification in eq.5 often becomes extremely large, we suppress the excessive modification by setting a limit value L . Needless to say, the limitation should be applied to the cube of LUM in eq.4.

Experiments 3.1 Sharpening image
An original image, shown in Figure 1, consists of 256 × 256 pixels with 256 gray levels. Firstly, we interpolated the original image into an enlarged image which consists of 512 × 512 pixels by the bilinear interpolation. Figure 2 shows the interpolated image. Applying the three sharpening methods described above to the interpolated image, we had three enhanced images. In the experiment, we set the parameters p = 2, α = 0.5, δ = 0.4 and L = 4.
We compared the proposed method with a conventional LUM and the cube of LUM experimentally. Figure 3, Fig.  4 and Fig. 5 show improved images by the LUM, the cube of LUM and the proposed method respectively. The image by the proposed method shows clear details of textures with suppressing excessive edge enhancement such as overshoot and undershoot. Although the LUM also gave clear texture details, all edges are enhanced too much. The cube of LUM lost the texture details especially in the area of a hat.

Comparison on the similarities of local complexity
In general, a performance of image processing is evaluated by using PSNR. However PSNR does not always show a reasonable value especially on sharpness of images. Since we now focus on the sharpness and enhancement of image details, we use the local complexity to evaluate the performance.
We employ the local standard deviation as the local complexity. Assuming that the local standard deviation of an enhanced image and that of the target image are represented by s i j and s 0i j respectively, the sum of difference of local complexities between the enhanced image and the target image     is obtained by The value derived by Eq.6 gives the similarity between an enhanced image and a target image in the meaning of local complexity. The smaller S d is, the more similar the enhanced image and the target image are. In this experiment, we used a high resolution 512×512 image shown in Fig. 6 as the target image, where this image is the true high resolution image of the input 256×256 image. Standard deviations s 0i j for the target image and s i j for created images shown in Fig. 3, 4, 5 are calculated with 7×7 square window. Table 1 shows the sum of difference of local complexities between the enhanced image and the target image by Eq.6. S d for resultant images by the LUM and the cube of LUM are larger than S d for the bilinear interpolated image which is before enhancement. Especially the difference

Comparison in the frequency domain
By improving the resolution of image, the components in higher frequency domain are newly created. We evaluate the components in the low frequency domain included in the original low resolution image and those in the high frequency   domain generated by the interpolation respectively.
After improving the image, it is desired to preserve the frequency components included in an input image before interpolation in order not to change the original image information. On the other hand, the high frequency components which is not contained before interpolation should be create effectively to give a better impression as a high precision image. Figure 7 shows the FFT result of the original low resolution image. Although the frequency domain of the original image is 256×256 , the components are superimposed in the domain of 512×512 to compare with the interpolated high resolution images. Figure 7 also shows the original image does not have high frequency components in the outside of 256×256 low frequency domain. In the graph the values over 0.01 are truncated in order to use the same scale in the following graphs (Fig.8, 9, 10 and 11).   Figure 8 shows the difference of frequency components between the bilinear interpolated image shown in Fig.2 and the input low resolution image. Needless to say, difference in the outside of 256×256 is newly created high frequency components. Figure 9, 10, 11 show the differences between frequency component of Fig.2 and those of Fig. 3, 4, 5 respectively. We evaluated whether the method creates high frequency components with suppressing the change of components in 256×256 low frequency domain.
Although the bilinear interpolated image has new high frequency components which look like thin walls in Fig.8, these components are yielded by the linear interpolation process and are not created from the characteristic of the input image.
From Fig.9 and 10, although enhanced images by the LUM and the cube of LUM have high frequency components, components in low frequency domain have changed remarkably. This shows that the information of original image was changed obviously. Furthermore, the differences in low frequency domain for resultant images by the LUM and the cube of LUM are larger than those for the bilinear interpolated image which is before enhancement. This means that the LUM and the cube of LUM distorted the original information from the bilinear interpolated image. Figure 11 shows that the result by the proposed method have high frequency components and the difference of components in 256×256 low frequency domain are smaller than those of other resultant images. From this experiment, the proposed method yields high frequency components with suppressing change of information of original image reasonably.

Conclusions
In this paper, we proposed a sharpening method which is not cube of linear filter but uses weighted sum of cubic operations of the differences between a value of pixel and those of the surrounding pixels.
We applied some enhancement procedures for a linear interpolated image and compared the similarities of local complexity of the resultant images and a target image experimentally. The proposed method improved the interpolated image and gave most similar image to the target image at a point of complexity. We also evaluated the resultant images in the frequency domain. The proposed method yielded high frequency components with suppressing change of information of the original image reasonably.
Experimental results for an interpolated image show that the proposed method produces sharp high resolution image which is more similar to a target image than that by any other method.