Mobile Display Stabilization by Visual Tracking Techniques

In this work, we implement three visual tracking techniques on mobile devices, i.e., a face tracking method using Haar-like features, a motion estimation method using 2D logarithmic search, and a circle tracking method by detecting a red circle attached on the user’s forehead. We use these three methods for mobile display stabilization alternatively and compare their performances. As visual tracking is a time consuming process and difficult running smoothly on mobile devices, the circle tracking method outperforms the other two by its simple computing process. We then validate the circle tracking based stabilization mechanism using the tumbling E eye examination and subjective 10-point scale questionnaire. Experimental results revealed that the proposed mechanism successfully reduced sight discomfort during motion, such as walking, taking a bus, and taking the Taipei MRT.


Introduction
Although visual tracking [1,2,3] has been an important research topic and gains increasing attention due to popular distribution of camera systems, visual tracking techniques are still time consuming and take a lot of computing resources.Limited to the large scale of data size, these techniques can even hardly perform smoothly on mobile devices.In our previous work [4,5], we proposed a motion compensation based anti-shake mechanism for mobile display.To improve the anti-shake performance, it is necessary to develop visual tracking functions on mobile devices.This way we can analyze the relative motion between the head and the hand, and thus to preciously predict the displacement of the mobile device.Herein, we develop three visual tracking techniques on mobile devices, and apply these techniques for mobile display stabilization.
Visual tracking processes usually involve several critical steps, namely, preprocessing, feature selection, image segmentation, and object alignment.To implement visual tracking for mobile display stabilization, we can simplify some steps as if the displacement sequence can be predicated.In this study, we propose three simplified methods, including a Haar-like feature based method [6,7], a 2D logarithmic search method [8] and a Hough circle algorithm [9] based method.All of these methods can serve for motion analysis and thus can be combined into our display stabilization mechanism to improve the anti-shake performance.
In the preprocessing step, we read a frame captured by the front panel camera and transform it into an intensity image.Then, as shown in Fig. 1(a)-(c), the first method we propose use three Haar-like feature types to detect the position and scale of the user's face.Fig. 1(c) also shows one of these feature types positioning on the user's eyes.Thus, we can obtain the center of this feature in each frame and figure out the motion of the mobile device.Fig. 1(d)-(f) shows the second method, in which we setup an N x N sub image at the center of the current frame and a p x p search area on the previous frame.Then, we applied a 2D logarithmic search algorithm for moving estimation.Fig. 1(f) shows an example of running this search algorithm, in case N = 15 and p = 8.We initially computed the mean absolute error (MAE) of the N x N = 15 x 15 sub image within a window of [-p/2, p/2] = [-4, 4] at nine locations as shown in green.Then, we repeated the following steps three times until the search region is one pixel wide: Fig. 1(g) illustrates the third method, in which we stick a red circle on the user's forehead and then use Hough circle algorithm to identify the circle.Advantages of this method include not only the computational efficiency but also that we can precisely figure out the moving pixels since we know both the real radius of the red circle and the radius in the captured frame.The disadvantage is that rare users want to attach a red sign on their forehead in real case.

Mobile Display Stabilization
Figure 2 shows the configuration and setup of our antishake mechanism.We created a drawing panel for mobile display called the active display area.Therefore, stabilization results from moving this display area on the screen.As stated Fig. 2. Configuration and Setup of the Anti-shake Mechanism.
in our previous work [4] [5], four parameters are defined to implement and enhance this mechanism, i.e., the border space "b", the moving bound "d", the suspending time "t", and the scale factor "c".As shown in Fig. 2, there are three functional units in our design.These units carry out analysis, prediction, and compensation processes.The dataflow of the anti-shake mechanism is as follows.First, the analysis unit receives video stream from the front panel camera.The video stream is captured frame by frame such that each of our proposed visual tracking methods can be implemented between two adjacent frames to gain the displacements along X-axis and Y-axis.The prediction unit then processes X and Y data separately and estimates the next displacements of the mobile device.Finally, the compensation unit moves the active display area in the opposite direction.
In the prediction unit, we processed X and Y data separately to predict the next displacement of the device.We formulated an autoregressive (AR) model for system modeling and used the recurrent algorithm for least-squares estimation (LSE) to calculate the coefficients of the model.The coefficients were updated real-time, therefore, the displacement at each moment can be predicted.The AR model with weight set 's can be described by Eqn (1), where the system parameter n indicates the window size of filtering.
Motion prediction at time k can be realized by Eqn (1) and the following recurrent LSE equations described in Eqn (2)-(4).^ (3) 1 x n k x n k x k The predicted move was then multiplied by the scale factor c. If the modified value is not larger than the moving bound d, and the suspending timer has ended countdown, the compensation unit will change this value to a negative value and form a compensatory offset C[k+1] (Eqn ( 5)).Finally, the active display area was moved to stabilize the mobile display.

Experimental Results
We selected the hTC Nexus 9 tablet computer as our test device.Three Android apps were developed to test the proposed visual tracking methods, respectively.The participant users took walk at speed of around 0.5 meter per second, so that we formed a moving environment for using mobile devices.We validated the performance of these two methods by the upper bounds of the frame rates and the prediction errors.Given that the predicted displacement is d[k+1] and the analyzed displacement is D[k+1], the prediction errors are Table 1 lists the experimental results that are averaged from twenty experiments, where we let N=15, p=8 and set the radius of the red sign as 20 mm.According to Table 1, we find that the Circle Tracking method outperformed the other two.The 2D logarithmic search method also performed better than the face tracking method; implying that we may not need to track objects for some kinds of mobile applications, such as device moving prediction and display stabilization we implement in this study.

User Test
The stabilization mechanism built upon circle tracking technique was then verified by user test experiments.After the user takes 12 min to answer some pre-test prointellet questions, we let him or her to conduct (a) a tumbling E examination [10] and (b) a 10-point scale questionnaire about visual fatigue [11] (Fig. 3).We compared the  test results from the case when the stabilization mechanism was turned on and when it was turned off.The motion includes walking, taking a bus, and taking the Taipei MRT.
Participant users include 20 college students.All the students were with normal eye sight, i.e., above 20/20 vision after necessary correction.Table 2 lists the experimental results averaged from 20 participants.Compared to the case where we turned off the anti-shake mechanism, the users had sharper vision when anti-shake mechanism was turned on.The result of the 10-point questionnaire also revealed that this stabilization technique provided a more comfortable feeling during motion.

Conclusions
In this paper, we implemented three visual tracking techniques on mobile devices, i.e., a Haar-like feature based face tracking method, a 2D logarithmic search based motion estimation method, and a Hough circle algorithm based sign tracking method.We compared these three methods in mobile display stabilization and found that the sign tracking method is the most efficient and the most precise.The circle tracking based stabilization mechanism was then verified by using the tumbling E eye examination and subjective 10point scale questionnaire.Experimental results revealed that the proposed mechanism successfully reduced sight discomfort during motion.Although implementation like this performed well so far.The problem is that few people would like to stick a sign on the head for the front panel camera to track.We can search for some unobvious material to form the tracking sign in the future, so that this mechanism could be more practical in use.

Table 2 .
User Test Results of Eye Examination and Sight Fatigue Questionnaire