Development of Single Camera Finger Touch Position Coordinate Transform Algorithm

This paper has focused on Image processing-based touch sensing systems using new methods. Image processing based single camera figure touch position detection approach has several significant advantages: (1) this method need not extra sensing instruments on the surface or under side of touch screen like resistive membrane-based systems and capacitive-based sensing systems. (2) this approach can convert non-touch screen into touch screen by only attaching cameras on the screen. (3) this approach enables various operations not only by touch but also by gesture operations. The most characteristic point of our system is using only single camera. Using a single view camera, we cannot get the depth information from a single view image. To solve this problem, we use the reflected finger image appears on the back of the screen. Detecting the fingertip and the reflected image on the screen effectively, we enable to detect touch position only using single camera. In this paper, we provide concrete method, algorithm, implement details, and several experimental results.


Introduction
This paper has focused on Image processing-based touch sensing systems.Touch sensing system provides more usable and convenience touches action by allowing user directly operation through finger than traditional interfaces such as keyboard or mouse.For this reason public equipment with large screen such as digital signage and vending machines are increasing these days.
The major touch sensing methods are resistive membrane-based system and capacitive-based system.
Resistive membrane-based system is cheap to implement, but it is easy to damage and hard to provide sufficient high sensing accuracy.Therefore, membrane-based touch sensing system is difficult to support more sophisticated or complex user operations.Capacitive-based system can provide more accurate sensing result.But Capacitive-based system requires some extra sensing attachments under the screen [1].For these reason, Image processing-based touch sensing system is collecting a lot of attention these days.Image processing-based touch sensing system is used to large touch screen for TV weather forecast, Digital Signage.Image processing-based touch sensing systems has advantages: (1) this method need not extra sensing instruments on the surface or under side of touch screen like resistive membrane-based systems and capacitive-based sensing systems.(2) this approach can convert non-touch screen into touch screen by only attaching cameras on the screen.(3) this approach enables various operations not only by touch but also by gesture operations.
However, most developed image processing-based touch sensing system are using a multi cameras to detect the position of the finger by using the principle of triangular surveying.
Therefore, existing image processing-based method has some weak points: (1) Distance is required between the cameras in order to detect the finger.(2) it is necessary to adjust the optical axis precisely of multiple cameras.Moreover, it is not possible to determine the depth (distance between finger and the screen) using the principle of triangular surveying.From this reason, strictly speaking, the triangular surveying based image processing based system cannot recognize the touch operation.Therefore, we propose the Single Camera Finger Touch Position Detection System.The most characteristic point of our system is using only single camera.Using a single view camera to detect the touch action and touch position is a challenging task.Because general speaking, we cannot get the depth information from a single view image.
To solve this problem, we use the reflected finger image appears on the back of the screen.Detecting the fingertip and the reflected image on the screen effectively, we enable to detect touch position only using single camera.Image processing based single camera figure touch position detection approach has several significant advantages: (1) No sensing devices need to be instrumented on the surface of the touch screen.(2) Minimum sensor construction can reduce the failure rate to realize maintenance free system.
(3) This approach enables an easy installation and a low-cost touch sensing.
In this paper, chapter 2 describe the proposed system, and fingertip coordinate transform algorithm which calculate the position of the finger exactly, chapter 3 shows the evaluation experiment of fingertip coordinate transform algorithm.Finally, the conclusion and future work is mentioned in chapter 4. Our system performs process as follows:
Capture a single view image includes fingertip and reflected image from the camera fixed at the top of the screen. 2.
From the capture image, recognize fingertip and reflected image using Haar-like features and Adaboost learning algorithm. 3.
Get fingertip and reflected image coordinates.4.
From two obtained coordinates, determine overlap of two coordinates. 5.
If coordinates of fingertip and reflected image coordinates overlapping, determine finger has touched the screen. 6.
After detecting fingertip, convert the coordinate of fingertip on the camera into the coordinate on the screen. 7.
Output the result.

Fingertip Coordinate Transform Algorithm
Because the screen looks trapezoidal shape in camera image.For this reason, we have to convert the coordinate of fingertip on the camera into the coordinate on the screen after detecting fingertip.

Evaluation Experiment
In order to confirm the speed and accuracy of our coordinate transform algorithm, we conducted an evaluation experiment and created an experimental program using C++ on Visual Studio 2012.The experimental environment is by using web camera to capture 320✕240 pixel image, OS is Windows7, CPU is Core-i3 2.93GHz and Memory is 2GB.
We attached a web camera at the top of the tablet like Figure.4.
Figure .5 shows the original captured image form web camera.
Fig. 4 The state that attaches a web camera to the tablet Fig. 5. Original captured image

Transformation Speed Experiment Result
Table 1 shows Resolution and Calculation time of our fingertip coordinate transform algorithm.

Conclusions
In this paper, we proposed a Finger Touch Detection and Position Detection System using a Single Camera.The coordinate transformation experiment result shows that our coordinate transform algorithm calculates very fast in real time, and converts the coordinate of fingertip on the camera image into the coordinate on the screen correctly.However, the unstable of finger coordinate transformation is a problem had not been solved.As a future work, we would like to devise a correction algorithm to get improved transformation accuracy and processing time.

Figure. 1 Fig. 1
Figure.1 shows the concept of proposed system.A single camera is fixed at the top of the screen.From the camera image, recognize the fingertip to get x-coordinate and y-coordinate and recognize fingertip and reflected image to get z-coordinate.If coordinates of fingertip (blue frame in Figure.1)and reflected image (green frame in Figure.1)overlapping, determine finger has touched the screen.

Figure. 3
shows the image of coordinate transformation.The method of fingertip coordinates transform algorithm based on the idea of projective transformation.(k,Nx,Ny) is calibration parameters which add to projection transformation.Normal projective transformation cannot correspond to the noise and the camera gap.Therefore, Parameter k works as scale parameter and Parameter Nx,Ny works as sift parameter.Converted fingertip coordinate X' and Y' can be computed by Equation (1) by original fingertip coordinate x,y, conversion parameters a, b, c, d, e, f, g, h and calibration parameters k,Nx,Ny.

Fig. 3
Fig. 3 Image of coordinate transformation

Figure. 6
Figure.6 shows the result of our coordinate transform algorithm from original captured image (Figure.5).

Fig. 6
Fig.6result of coordinate transform algorithm Due to our system using projection transform algorithm, spatial resolution will decrease at the bottom side.Assuming an angle θ between the upper side and aspect side shows in Figure.7,spatial resolution will decrease as Equation (4).

Fig. 8
Fig.8 Reduction rate of the spatial resolution

Table 1
Resolution and Calculation time of our fingertip coordinate transform algorithm