Development of EOG Mouse Using Learning Vector Quantization

Recognition of eye motions has attracted more and more attention of researchers all over the world in recent years. In particular studies to make lives more convenient for patients as ALS who cannot move even their muscles expect eye have been actively done. Many kinds of eye motion recognition methods have been proposed, for example, using infrared to track pupil position, or using image processing technologies to find the pupil. However as they use either infrared or cameras, they may have some effect on the eye. Therefore, in our study, we use an EOG method to recognize eye motions: attaching wet disposable electrodes on the patients’ face to obtain eye movement signals. Then Learning Vector Quantization algorithm is used to recognize each eye motion. Finally, corresponding recognition results to various mouse operations can be obtained. In our study, we recognized eye motions of rolling eyes upward, downward, rolling left, rolling right, diagonal eye motions which contain rolling up-left, rolling up-right, rolling down-left, rolling down-right ( the angle of the diagonal motion is 45°), blink, blink string of three times motion, left wink and right wink, in total 12 kinds of eye motions. The average recognition accuracy was over 98%. Using this recognition system, we achieved 8 direction cursor movements and double click action, scroll page upward and downward. This study would be used as a means of communication to help those patients as ALS.


Introduction
For patients with severe physical disabilities, for example, amyotrophic lateral sclerosis and serious paraplegia, eye motion is the last spontaneous motion for them to respond and communicate with others.It is important to develop a new form of communication tool for these patients and to make their lives more convenient.
Many eye motion recognition methods have been proposed, such as Corneal Reflection Method [1], Search Coil Method [2], VTR Method [3] and EOG Method.
(1) Corneal reflection method First, it irradiates user's face with an infrared LED, then uses a camera that can shoot an infrared to search the eye, after that, determines the position of the reflected light on the cornea as a reference point.On the basis of the position of the pupil obtained by the position of the corneal reflection, eye motion can be recognized [1].
(2) Search coil method Search coil method is a method to track the movement of coil by wearing special contact lenses.The accuracy of search coil method is very good, and using this method the rotation of eyeball axis direction can also be detected [2].However, owing to the use of the special contact lens, it is necessary to reduce the wearing time.As a result, it is inconvenience for patients.(3) VTR method The VTR method is that: it records a moving image of eye motion using a camera, and then makes a decision of each motion by image processing technology [3].
However, there are some problems with these works, for example: the number of eye motion types that can be recognized is less; infrared and contact lenses may have some effect on the eye; and recognition accuracy is not high.
(4) EOG method The electrooculography (EOG) is an electrical signal produced by potential difference between the retina and the cornea of eye.The difference is due to the large presence of electrically active nerves in retina compared to the front of the eye.Many experiments show that the corneal part is a positive pole and the retina part is a negative pole in the eyeball as shown in Fig. 3.
In our study we use EOG method to recognize eye motions.Compared with other methods, EOG method has advantages as follows: it does not need to fix the head, the installation of a device is easy, and it can be relatively easy to detect the signal for a long time.

Construction of recognize system
This section describes a recognition system of eye movements.The recognition system is constituted from 4 parts; an input EOG acquisition part, a motion extraction part, a data transform part and a recognition part.The detail of each part is shown below.

Input EOG acquisition part
In our study we use Personal-EMG to measure the EOG signals.Personal-EMG is produced by Oisaka Electronic Equipment Ltd (Japan), and is a muscle surface potential measuring device.It plays an active part in a wide range of fields.We can measure and save the EOG and EMG simply by connecting the A/D card.The use connecting to the external device is also possible, and it can be used in a wide range of fields such as welfare, sports field and ergonomics.
In our system the data obtained using Personal-EMG are all amplified 1,000 times and the A/D transform is carried out.The sampling frequency in the device is 3,000Hz.Considering skin softness around the eye and recognition accuracy we used wet electrodes in our study.
We attached seven wet disposable electrodes around eyes as shown in Fig. 5 to measure the EOG signals.We obtained horizontal and vertical channel signals by one trail.The horizontal channel electrodes were attached on the outer corner of the eyes.In the vertical channels 1 and 2, electrodes were attached on the two vertical sides of two eyes, and the reference electrode was attached on the neck as Fig. 5 shows.Another reference location including ear, nose and their surround are also tested for EOG measurement.However the location in Fig. 2 gave a best result.

Motion extraction part
In this system, we set the moving frequency of cursor as 1.8 seconds one time.Before cursor operation we have to obtain the EOG signals online and extract signals at 1.8 seconds interval one time.After extraction, we have to make a judge whether an eye motion subsists in the 1.8 seconds EOG data.If there is any motion, we detect the data of that motion and make a recognition on it.Furthermore, if there are two motions in the same 1.8 seconds EOG data, we also need to detect both of the two motions.In the motion extraction part, we use a moving average method to detect signals of each eye motion [5,6,7].

Data transform part
In the data transform part, we use Fast Fourier Transform to obtain frequency features and humming window is used so that a high frequency ingredient may not appear [8].
In order to increase recognition accuracy, in this system we use 3 channel EOG signals to make up frequency features.As shown in Fig. 6, compared with our previous work, we add vertical channel 2 and attached electrodes on the vertical sides of left eye.In the previous system, we use FFT to transform each signal from time domain to frequency one, and combine vertical channel FFT and horizontal channel FFT data as one.Then the combined data are sent into recognition part.In the new 3 channel system, more information of each motion will be used to generate frequency features, using the new data by which we can get higher recognition accuracy.In the same way as our previous work, only from 0 Hz to 300 Hz components with a large amplitude value among the obtained frequency components are used.Then, we sent the combined data into recognition part to make an recognition on it.

Recognition part
In the recognition part, we use the LVQ3 [9], which is a prototype-based supervised classification algorithm, to recognize a class of each motion.In LVQ we use FFT features of each motion as an input vector and set a weight vector for each output neuron.The explanation of LVQ is as follows.
Input vector: X = { 1 ,  2 ,  3 … .  } Weight vector: Wj = { 1 ,  2 … . .  }   : Category represented by the  ℎ neuron T: Correct category for the input X.First, we define Euclidean distance between an input vector and the weight vector of the  ℎ neuron in the following: Then, we train the weight vectors.Let Ya be the neuron with the weight vector that is closest to X, and let Yb be the neuron with the weight vector that is next closest to X. Let Da be a distance between X and the weitht vector of Ya.Let Db be a distance between X and the weight vector of Yb.Weights of the both Ya and Yb are updated if: (where ε depends on the number of training samples, in this paper, ε =0.3)If   is the same as T, and   is different from T, the weight is updated as follows: where α is a learning rate and is reduced with training times.
When α is 0, the update stops.If both   and   are the same as T, the weight update is given as: If neither   nor   is the same as T, the weight update is given as: After training, the LVQ network classifies an input vector by assigning it to the same category as the output neuron whose weight vector is closest to the input vector.In our system, left wink and right wink are recognized separated.We use the difference between two vertical channels to recognize them.Through observation we found that, when we do simple eye wink motion, the difference between two vertical channels is very large compared with normal eye motion.By using this characteristic we can recognize left wink and right wink at 100% recognition accuracy.

EOG Mouse
In our study, we set the moving frequency of EOG-Mouse as 1.8 seconds one time.The achievement of online function is based on a part of code provided by Oisaka Electronic Equipment Ltd (Japan) which designed the personal-EMG.Of course, we can adjust the moving frequency according to individual needs of each user.We extract eye motion from each 1.8 seconds signal, and then recognize each motion to control the movement of cursor.In our study we control cursor movement through Mouse-Event function.In the current stage we have finished the recognition of rolling eye upward, rolling downward, rolling left, rolling right, blink and diagonal eye motions which contain rolling up-left, rolling up-right, rolling down-left, rolling down-right (the angle of the diagonal motion is 45° [6]), blink string of three times motion, left ink and right wink, in total 12 kinds of motions.
Motions in 8 directions correspond to 8 directions cursor movement in this system.We regard blink motion as invalid signal and define blink string motions as double click action.Left wink and right wink motions correspond to rolling page upward and downward functions, respectively.By using these functions patients can simply conduct the online activities.

Experimental and results
In this paper, we carried out recognition experiments only using 1 subject data, the recognition accuracy was 98%, it is higher than any subject of our 2 channel system previous work (97.8%, 97.6%, 93.9%), in the next step we will increase the number of subjects.The detail of LVQ is shown in Table 1.Table 2 shows the experimental results of recognition accuracy for the subjects A. We carried out online experiments of EOG mouse in this paper.We let 2 subjects do eye movement to open a set of files as Figure 11 shows: 2 subjects tried to open the files in the order of 6,1,3,9.The two subjects successfully operated the cursor movement with no failure recognition.
About moving distance of cursor by every eye movement, if the distance is too large, the selection accuracy can be reduced.If the distance is too small, the number of operations can increase and we must bear a burden on eyes.After trial and error we set them as shown in Fig. 12.By repeatedly moving up and down, we can achieve a small adjustment of the distance.

Conclusions
In this paper we developed an EOG Mouse application which used Learning Vector Quantization algorithm to recognize each eye motion.We achieved recognition of rolling eyes upward, downward, rolling left, rolling right, diagonal eye motions, blink, blink string 3 times motion, left wink and right wink, in total 12 kinds of eye motions.Using this recognition system we got a high recognition accuracy at average 98%.Compared with our previous work, its recognition accuracy was increased.
In future plans, we will increase the number of eye motions, for example some continues combined motions, left wink 2 times motion, right wink 2 times motion, rolling eyeball clockwise, rolling eyeball counterclockwise.Furthermore, we will obtain more training data for a LVQ network to increase recognition accuracy in the next step.

Figure 7 Fig. 8 .Fig. 9 .
Figure 7 shows the distribution of 10 motions using Simple-principal component analysis which is an approximation algorithm of PCA.From the picture we can observe, 10 eye motions are divided into 4 classes, rolling left and rolling right belong to class 1, rolling upward, rolling downward and blink belong to class 2, and blink string motion belongs to class 3, diagonal motions belong to class 4. The resulting classification in Fig. 7 depended on the distribution range of EOG frequency.The motions which belong to the same class have a similar frequency property and it is difficult to separate them, especially classes 2 and 4. In order to solve this difficulty we use another characteristic of EOG features at each motion to separate similar motions in the same class at the next step.If a motion belongs to class 1, we use the horizontal EOG data of this motion.Then we seek the locations (A) with the maximum value and (B) with the minimum value in the horizontal axis.The classification rules are then given as: