Development of Eye Mouse Using EOG signals and Learning Vector Quantization Method

Recognition of eye motions has attracted more and more attention of researchers all over the world in recent years. Compared with other body movements, eye motion is responsive and needs a low consumption of physical strength. In particular, for patients with severe physical disabilities, eye motion is the last spontaneous motion for them to make a respond. In order to provide an efficient means of communication for patients such as ALS (amyotrophic lateral sclerosis) who cannot move even their muscles except eye, in this paper we proposed a system that uses EOG signals and Learning Vector Quantization algorithm to recognize eye motions. According to recognition results, we use API (application programming interface) to control cursor movements. This system would be used as a means of communication to help ALS patients.


Introduction
It is difficult for patients with severe physical disabilities to communicate with others, such as Amyotrophic Lateral Sclerosis (ALS) and serious paraplegia.Owing to the illness in which they lost their limb motor function and language function, they cannot move even their muscles except eye.It is important to develop a new form of communication tool for these patients and to make their lives more convenient.
In order to provide an efficient means of communication for those patients, in this paper we proposed a system that uses EOG signals and Learning Vector Quantization algorithm to recognize eye motions.According to recognition results, we use API (application programming interface) to control cursor movements.A recognition part in the system consists of four steps.First, we measure EOG signals by every 1.8 seconds.Next, we make a judge whether eye motion exists in the 1.8 seconds EOG data.If any, we extract the data of each motion from the 1.8 seconds EOG data.After that we use Fast Fourier Transform to obtain the frequency features of the extracted motion.Finally we use a Learning Vector Quantization network and characteristics of EOG features at each motion to recognize eye motions.The LVQ network is trained beforehand.In this paper we recognized motions of rolling eye upward, rolling downward, rolling left, rolling right, blink and diagonal eye motions which contain rolling up-left, rolling up-right, rolling down-left, rolling down-right (the angle of the diagonal motion is 45 • ), left wink, right wink and blink string of three times motion.8 directions motions correspond to 8 directions cursor movements in this system.We regard a blink motion as an invalid signal and define a blink string motion as double click action.Left and right winks correspond to the rolling page up and down, respectively.Using this system we have obtained a high recognition accuracy of eye motions.

Related works
Eye motion is regarded as the last spontaneous motion of patients with severe physical disabilities.We can move it smoothly and need only a low consumption of physical strength.In recent years, recognition of eye motions is attracted more and more attention of researchers all over the world.Many eye motion recognition methods have been proposed, such as Corneal Reflection Method [1], Search Coil Method [2], VTR Method [3] and EOG Method.
(1) Corneal reflection method First, it irradiates user's face with an infrared LED, then uses a camera that can shoot an infrared to search the eye, after that, determines the position of the reflected light on the cornea as a reference point.On the basis of the position of the pupil obtained by the position of the corneal reflection, eye motion can be recognized [1].
(2) Search coil method Search coil method is a method to track the movement of coil by wearing special contact lenses.The accuracy of search coil method is very good, and using this method the rotation of eyeball axis direction can also be detected [2].However, owing to the use of the special contact lens, it is necessary to reduce the wearing time.As a result, it is inconvenient for patients.(3) VTR method The VTR method is that: it records a moving image of eye motion using a camera, and then makes a decision of each motion by image processing technology [3].However, there are some problems with these works, for example: the number of eye motion types that can be recognized is less; infrared and contact lenses may have some effect on the eye; and recognition accuracy is generally not so high.
(4) EOG method The electrooculography (EOG) is an electrical signal produced by potential difference between the retina and the cornea of eye.The difference is due to the large presence of electrically active nerves in retina compared to the front of the eye.Many experiments show that the corneal part is a positive pole and the retina part is a negative pole in the eyeball as shown in Fig. 1 In our study we use EOG method to recognize eye motions.Compared with other methods, EOG method has advantages as follows: it does not need to fix the head, the installation of a device is easy, and it can be relatively easy to detect the signal for a long time.

Construction of recognition system
This section describes a recognition system of eye movements.The recognition system is constituted from 4 parts; an input EOG acquisition part, a motion extraction part, a data transform part and a recognition part.The detail of each part is shown below.

Input EOG acquisition part
In our study we use Personal-EMG to measure the EOG signals.Personal-EMG is produced by Oisaka Electronic Equipment Ltd (Japan), and is a muscle surface potential measuring device.It plays an active part in a wide range of fields.We can measure and save the EOG and EMG simply by connecting the A/D card.The use connecting to the external device is also possible, and it can be used in a wide range of fields such as welfare, sports field and ergonomics.The cost of Personal-EMG is not cheap, but in our study we only used a little part of function of this device.Therefore in the next step, we will develop s set of simple device to measure EOG signals and to decrease the cost of this system.
In our system the data obtained using Personal-EMG are all amplified 1,000 times and the A/D transform is carried out.The sampling frequency in the device is 3,000Hz.
Figure 2 shows the proposed method of our study.Figure 3 shows the image of Personal-EMG and electrodes.The  We attached five wet disposable electrodes around eyes as shown in Fig. 4 to measure the EOG signals.We obtained horizontal and vertical channel signals by one trail.The horizontal channel electrodes were attached on the outer corner of the eyes.The vertical channel electrodes were attached on the two vertical sides of two eyes, and the reference electrode was attached on the neck as Fig. 4 shows.Another reference location including ear, nose and their surround are also tested for EOG measurement.However the location in Fig. 4 gave a best result.

Motion extraction part
In this system, we set the moving frequency of cursor as 1.8 seconds one time.Before cursor operation we have to obtain the EOG signals online and extract signals at 1.8 seconds interval one time.After extraction, we have to make a judge whether an eye motion subsists in the 1.8 seconds EOG data.If there is any motion, we detect the data of that motion and make a recognition on it.Furthermore, if there are two motions in the same 1.8 seconds EOG data, we also need to detect both of the two motions.In the motion extraction part, we use a moving average method to detect signals of each eye motion [5]∼ [7].

Data transform part
In the data transform part, we use Fast Fourier Transform to obtain frequency features and humming window is used so that a high frequency ingredient may not appear [8].We use FFT to transform each signal from time domain to frequency one, and combine vertical channel FFT and horizontal channel FFT data as one.Then the combined data are sent into recognition part.Only from 0 Hz to 300 Hz components with a large amplitude value among the obtained frequency components are used.Then, we sent the combined data into recognition part to make an recognition on it.

Recognition part
In the recognition part, we use the LVQ3 [9], which is a prototype-based supervised classification algorithm, to recognize a class of each motion.In LVQ we use FFT features of each motion as an input vector and set a weight vector for each output neuron.The explanation of LVQ is as follows.
Input vector: X = x 1 , x 2 , x 3 ..., x n Weight vector: Wj = w 1 j , w 2 j ... . .w n j C k : Category represented by the K th neuron T: Correct category for the input X.
First, we define Euclidean distance between an input vector and the weight vector of the J th neuron in the following: Then, we train the weight vectors.Let Ya be the neuron with the weight vector that is closest to X, and let Yb be the neuron with the weight vector that is next closest to X. Let Da be a distance between X and the weitht vector of Ya.Let Db be a distance between X and the weight vector of Yb.Weights of the both Ya and Yb are updated if: (Where ε depends on the number of training samples, in this paper, ε =0.3) If C Ya is the same as T, and C Y b is different from T, the weight is updated as follows: where α is a learning rate and is reduced with training times.When α is 0, the update stops.If both CY a and CY b are the same as T, the weight update is given as: If neither CY a nor CY b is the same as T, the weight update is given as: After training, the LVQ network classifies an input vector by assigning it to the same category as the output neuron whose weight vector is closest to the input vector.
Figure 5 shows the distribution of 10 motions using Simple-principal component analysis which is an approximation algorithm of PCA.From the picture we can observe, 10 eye motions are divided into 4 classes, rolling left and rolling right belong to class 1, rolling upward, rolling downward and blink belong to class 2, and blink string motion belongs to class 3, diagonal motions belong to class 4. The resulting classification in Fig. 5 depended on the distribution range of EOG frequency.The motions which belong to the same class have a similar frequency property and it is difficult to separate them, especially classes 2 and 4. In order to solve this difficulty we use another characteristic of EOG features at each motion to separate similar motions in the same class at the next step.
If a motion belongs to class 1, we use the horizontal EOG data of this motion.Then we seek the locations (A) with the maximum value and (B) with the minimum value in the horizontal axis.The classification rules are then given as: A lies to the right of B (A>B) Rolling Left B lies to the right of A (A<B) Rolling Righ If a motion belongs to class 2, we use the vertical EOG data of this motion and seek the locations of A and B as above.In the same way, the classification rules are given as: In our system, left wink and right wink are recognized separated.In order to recognize left wink and right wink, we added 2 electrodes and attached them on the vertical sides of left eye.We use the difference between two vertical channels to recognize them.Through observation we found that, when we do simple eye wink motion, the difference between two vertical channels is very large compared with normal eye motion.By using this characteristic we can recognize left wink and right wink at 100% recognition accuracy.

EOG Mouse
The objective of our study is only to use eye movements to control cursor and to achieve some functions like browse a page and click on files.Depending on the achievement of these functions, we can make patients lives more convenient.In this EOG mouse system, 8 directions eye motions About moving distance of cursor by every eye movement, if the distance is too large, selection accuracy is reduced.If the distance is too small, the number of operations increases and we must bear burden on eyes.After trial and error, we set them as shown in Fig. ??.The moving distance of upward and left is 50 pixels, the moving distance of down and right is 40 pixels, and the moving distance of diagonal motion is based on the distances of the four directions.By repeatedly moving up and down, we can achieve a small adjustment of the distance (50 pixels-40 pixels=10 pixels), it is broadly enough for normal computer use.In our study, we set the moving frequency of EOG-Mouse as 1.8 seconds one time.The achievement of online function is based on a part of code provided by Oisaka Electronic Equipment Ltd (Japan) which designed the personal-EMG.Of course, we can adjust the moving frequency according to individual needs of each user.We extract eye motion from each 1.8 seconds signal, and then recognize each motion to control the movement of cursor.In our study we control cursor movement through Mouse-Event function.In the current stage we have finished the recognition of rolling eye upward, rolling downward, rolling left, rolling right, blink and diagonal eye motions which contain rolling up-left, rolling up-right, rolling down-left, rolling down-right (the angle of the diagonal motion is 45 • [6]), blink string of three times motion, left wink and right wink, in total 12 kinds of motions.
We extract an eye motion from 1.8 seconds signals.Then we make a recognition on each motion and control the movement of the cursor.In our study we use Mouse-Event functions to control cursor movement and double click action.
Mouse-Event Function:  given by the number of dwData.

Recognition results
We carried out recognition experiments using 3 subjects data, that is subjects A, B and C (20's,male).EOG data are measured for four months.Before experiments we place reference picture in front of the subject.Then the subjects performed 10 eye motions disorderly.The LVQ network is trained by the data which obtained from the subjects A and B. Table 1 shows the experimental results of recognition accuracy of subjects A ,B and C.

3 Channel Recognition System
In order to increase recognition accuracy, we used a 3 channel signal system to make EOG Mouse, Fig. 9.As shown in the following figure, we add a vertical channel 2 and attached electrodes on the vertical sides of left eye.We used 3 channel signals to generate the frequency feature of each eye motion.In the previous system, we use FFT to transform each signal from time domain to frequency domain, and combine vertical channel FFT data and horizontal channel FFT data as one.In the new 3 channel system, more information of each motion is used to generate the frequency data.We use the new data in order to make the grouping of all motions more explicitly.We carried out recognition experiments only using 1 subject data for 3 channel system.The average recognition accuracy was 98%.It is higher than any subject of our 2 channel system in the previous work (97.8%, 97.6%, 93.9%).In the next step we will increase the number of subjects.The detail of LVQ is shown in Table 2. Table 3 shows the experimental results of recognition accuracy for the subjects A.
We carried out online experiments of EOG mouse in this paper.We let 2 subjects do eye movement to open a set of files as Figure 5 shows: 2 subjects tried to open the files in the order of 6,1,3,9.The two subjects successfully operated the cursor movement with no failure recognition.

Conclusions
In this paper we developed an Eye Mouse application which used EOG signals and Learning Vector Quantization algorithm to recognize each eye motion.We achieved recognition of rolling eyes upward, downward, rolling left, rolling right, diagonal eye motions, blink, blink string 3 times motion, left wink and right wink, in total 12 kinds of eye motions.Using this recognition system we got a high recognition accuracy at average 98%.
In future plans, we will increase the number of eye motions, for example some continues combined motions, left wink 2 times motion, right wink 2 times motion, rolling eyeball clockwise, rolling eyeball counterclockwise.Fur-

Figure 6 :
Figure 6: Signals of rolling left and rolling right