Binary Brain Machine Interface Using Modulation of Steady-State Visual Evoked Potential Associated with Mental Focus in Eyes-Closed State for Visually Impaired Persons

Modulation of electroencephalogram (EEG) can be applied to express one’s intent to others without language and body motion; brain machine interfaces (BMIs) using EEG are expected to one of the communication means between disabled with severe motor disability such as amyotrophic lateral sclerosis patients and caretakers. There have been various BMIs developed using event-related potentials such as P300, steady-state visual evoked potential (SSVEP), event-related desynchronization and slow cortical potential, some of which have achieved the outstanding performance for communication in information transfer rate and classification accuracy. However, most of conventional BMIs are difficult to use for patients with impaired oculomotor control because of requirement of visual modality. The present study aimed at developing a novel binary class of BMI which was independent of oculomotor control using event-related modulation of SSVEP associated with execution of mental task under eyes-closed condition. The amplitude of SSVEP in the posterior region of twelve healthy subjects aged 21-22 years old was seen to be significantly modulated when they repeated mental focusing on the flicker under the conditions of the flickering frequency of 10 Hz and stimulus intensity of 5 lx. Moreover, such modulation also occurred in performing image recalling and mental calculation for other six healthy subjects. Mean classification accuracy on whether the subjects performed task or not, corresponding to an intention of Yes or No, reached 87.5 %, 75.0 % and 72.9 % for the mental focusing, image recalling and mental calculation, respectively. Reproducibility of the modulation of SSVEP associated with these tasks were observed at least four times each task. On the other hand, the amplitude of SSVEP in the relaxed state without task tended to decrease with repetition. Maintenance of amplitude modulation of SSVEP in the relaxed state with repetition may improve the classification accuracy of the proposed BMI.


Introduction
Brain machine interface (BMI) typically supports a communication between a paralyzed patient with motor disability such as amyotrophic lateral sclerosis and caregivers through the translation of voluntary change in the electroencephalogram (EEG) into a patient's intention in the form of displaying his/her intention or control command to a computer or a nursing-care robot (1)(2)(3) .The state-of-the-art BMIs have outstandingly progressed in the information transfer rate and classification accuracy (4,5) .Recently the range of application of BMIs are extended to hands-free and voice-free games and control of home electronics such as channel zapping of television.
Conventional BMIs used various types of EEG components for estimating user's intent (4)(5)(6) ; sensorimotor rhythms (SMRs), slow cortical potentials (SCPs) and visual evoked potentials (VEPs).SMRs are recorded over the sensorimotor cortex which could be modulated by motor imagery, referred to as "event-related desynchronization" (ERD).SMRs are recovered and could be increased associated with relaxation, which is referred to as "event-related synchronization" (ERS).Such ERD/ERS modulation have been used in many BMIs.
SCPs are the slowest response in event-related potentials (ERPs) which change over 0.5 s.Negative and positive SCPs accompany mental preparation and mental inhibition, respectively.After training for several weeks, users may control the amplitude of SCPs.Some of BMIs based on SCP have been developed to assist basic word-processing (5,8) .
VEPs also belong to ERP which are transient and stationary response of EEG to visual stimulus and observed after several hundred milliseconds after a visual stimulus.Specifically, P300, a positive peak potential which occurs about 300 ms after the onset of a target stimulus which is rarely presented among a series of frequent and non-target stimuli, has been also used in many conventional BMIs, namely, P300-based BMIs.P300-based BMIs have advantages in high detectability and high information transfer rate (ITR) (2,5) .
Repetitive flicker stimuli or pattern reversal stimuli can elicit another type of VEPs; steady state visual evoked potentials (SSVEPs) which are defined as the EEG components with the same frequency as the stimulus frequency (flickering frequency).SSVEP recorded over the parietal and occipital regions have suitable properties for BMI for which the amplitude is larger than that of other EEG components used for conventional BMIs and the sustainability is high.The underlying mechanism of SSVEPs are not clarified, but is estimated to originate from wide region of cortex, not only primary visual cortex, striate and extra-striate visual cortex but also motion sensitive area (9,10) .
In typical SSVEP-based BMIs, a user chooses and gazes at one of the flickering icons (images and characters) with different frequencies designating user's demands, leading to increase the SSVEP amplitude at the flickering frequency of the icon the user chooses.By analyzing the EEG in the frequency domain, the user's intent can be classified.Information transfer rate of the SSVEP-based BMIs reached more than 50 bit/min, comparable to the SMR-based BMIs (5,(11)(12)(13) .
Such SSVEP-based BMIs, however, are almost unavailable for severely paralyzed patients, for example, "locked-in" patients who loose motor function including eye-movement.Some laboratories proposed improved BMIs using superimposed icons with different colors corresponding to user's intent for users who could not control their eye-movement (14,15) .They also noted the difficulty in gazing at one of the superimposed icons and slowness in the information transfer rate less than 1 bit/min.Moreover, for "locked-in" patients, such BMIs are still unavailable.
From this viewpoint, there has been proposed a new type of SSVEP-based BMI for visually impaired patient.In the proposed BMI, a user has glasses in each of which an attached light emitting diode (LED) flickers at different frequency from one another.User's attention to one of the two LEDs assigned to user's intent ("Yes" or "No", or 1 or 0, …) elicits the SSVEP with a corresponding frequency. (16).ITR of the system achieved a relative high value, 10.8 bit/min, in BMIs which could be used in the eyes-closed condition.It is however better to reduce the light intensity and use the single stimulus frequency to reduce user's mental fatigue from a practical perspective.
Recently the amplitude of the SSVEP with eyes-closed was found to be significantly reduced in the mental focusing on flicker stimuli of LEDs at single stimulus frequency compared to the amplitude in the relaxed state under the flicker stimuli (17) .Such modulation is similar to ERD in SMRs, but seems to be different from the conventional observation in SSVEP-based BMI with eyes-opened that the SSVEP amplitude at the flickering frequency of an icon which a user gazes at is mostly increased.
Practically it is interesting to investigate a novel binary class of eye-opening independent BMI using SSVEP modulation through the mental focus on flicker stimuli.Moreover, stimulus specification used in the previous research 5 lx and 10 Hz for stimulus frequency and intensity may be sufficiently weak and moderate to allow patients to use for a relatively long time.
One of the issues in applying the above phenomenon to SSVEP is reproducibility of ERD-like modulation of SSVEP in concentrating on the flicker stimuli (17) .Change of the amplitude modulation of SSVEP by repeating of mental focusing would be one of the impediments to overcome to establish the BMI using the SSVEP modulation with mental task.
Moreover, it should be investigated whether such modulation also occurs in other mental tasks and could be used for the SSVEP-based BMI which is independent of eye-open.
The purpose of the present study is thus to investigate the reproducibility of the modulation of SSVEP with repeating mental focusing and relaxing and to examine effects of other mental tasks concentrating user's attention which were expected to modulate SSVEP.

Experiments
Two types of experiments were separately conducted in order to (1) investigate the reproducibility of SSVEP modulation associated with mental focusing and (2) examine task-dependence of SSVEP modulation.Twelve male volunteer students aged from 21 to 24 years old participated in the former, while six male volunteer students from 21 to 25 years old participated in the latter.
All the measurements were performed in the electromagnetic shield room in the Intellectual Property Center, Yamaguchi University.EEG was derived from 13 electrode sites with Ag electrodes with diameter of 8 mm according to the 10-20 Electrode System with reference to averaged earlobe potentials (see Fig. 1).
Electrooculogram (EOG) was also recorded over left and right angulus oculi lateralis of each subject for detecting his eye-movement.Note that specific directional eye movements to flicker stimuli were not recognized in performing task as well as in relaxed state without task through the two experiments.As SSVEP was elicited mainly in the posterior region, the present study focuses to discuss the modulation of SSVEP at P3, P4, O1 and O2.
These physiological signals were amplified and recorded using Nihon Koden EEG-5532 with a gain of 86 dB and digitized with a sampling frequency of 200 Hz per site and an amplitude resolution of 12 bit under the control of a PC.
We took informed consent in accordance with the tenets of the Declaration of Helsinki after the aim and possible effect of flicker stimuli used in the experiment on health were fully explained to them.Test stimuli were applied to each subject for 10 s before the first run to confirm that test stimuli did not affect the subject's affect.
Eight LEDs (OptoSpply, OSHR516A-QR, half intensity angle: 30deg, wavelength: 625 nm) were used as the flicker stimulator as shown in Fig .2. The stimulator was allocated immediately above eyes of a subject who was on his back with a bed in the laboratory with a distance of 70 mm and flickered at 10 Hz in both experiments.All the subjects with eyes-closed could perceive the flickering light through their eyelids but could not specify the location and focus of the flickering LEDs under this condition.
The tasks and conditions are shown in Table 1.In both experiments, each subject were directed to close his eyes during EEG recording.All the experiments were conducted in the eyes-closed state.
(1) Reproducibility Each run lasts 20 s with 20 s rest between runs.In Table 1, in "relaxed", all the subjects were directed to relax without task under the flicker stimuli, whereas in "focused", they were directed to mentally focus on the flicker stimuli.Differently from the BMI with eye-opened, it is difficult to gaze at the LED directly, this instruction tended to confuse the subjects or make them a little nervous.Eight test stimuli consisting of four relaxed and focused runs were performed to ensure that each participant got familiar with the mentally focusing task before the first session.The light intensity of the flicker was set at 5 lx at the position of eyelids of each subject using an illumination meter (Topcon IM-5).
(2) Dependence on engaged task Each run lasts 15 s with 20 s rest between runs and 2-3 minutes rest between every four-runs.The three tasks was randomly ordered each subject to avoid the order effect on SSVEP modulation.
In the image recalling trials, each subject was instructed to recall his favorite still image such as dog and cat during a run according to a cue from the experimenter.
The assignment of mental calculation is a repeat of subtraction of seven from an initially given triple-digit number randomly generated.For example, when 777 is given, a subject should repeat subtraction as 770, 763, 756, ….At the end of each run, the subjects were asked to answer.
Task condition of run was classified into "relaxed without task" and "engaged in a task" using the SSVEP modulation as described in the next section 2.2 in order to examine the task-dependence of classification accuracy.This experiment had no test run under consideration of fatigue of the subjects.The light intensity of the flicker stimuli was 15 lx, stronger than the reproducibility experiment to elicit more recognizable SSVEP modulation.

Analysis
The fundamental SSVEP amplitude synchronized at the stimulus frequency (10 Hz) was estimated for a 10-s data window from 3 to 13 s after the onset of the flicker stimuli each run using the discrete Fourier transform of EEG data obtained in both experiments.Generally SSVEP includes harmonics of the fundamental frequency, the harmonics were excluded from analysis because the amplitude of the harmonic SSVEP with eyes-closed was much smaller than that of the fundamental SSVEP as previously reported 17) .
(1) SSVEP modulation The fundamental SSVEP amplitude was estimated from EEG records in the reproducibility experiment using the discrete Fourier transform.The effect of mental focusing on the flicker stimuli was evaluated using the modulation rate of the average of the fundamental SSVEP amplitude during the mental focusing to that during the relaxed state as follows, r f a a r a  where a r f a and r a (i = 1, ... 4) denote the amplitude modulation rate of SSVEP, the average of SSVEP amplitude over four runs of mental focusing and the average of SSVEP amplitude over four runs of the relaxed state without mental focusing, respectively.Significance of the amplitude modulation was tested using one-sample t-test with null hypothesis that 1  a r .Reproducibility of the amplitude modulation of the fundamental SSVEP was assessed by means of evolution of normalized SSVEP amplitude given by the following equation, where i a f ~ (i = 1, ... 4) respectively denote the normalized SSVEP amplitude at 10 Hz in the i-th run of the mental focusing.
Statistical difference of the normalized SSVEP amplitude each run was analyzed in terms of parametric statistical test.The main effect of runs was checked through one-factor analysis of variance (ANOVA) with repeated measures and pairs of runs with significant difference was sought using paired t-test as post hoc.
(2) Classification dependence on engaged task Classification accuracy of the proposed BMI was evaluated using an offline classification of SSVEP modulation.The proposed BMI classified a run condition into "engaging in a task assigned" if the absolute value of the SSVEP modulation in each run exceeded 20 %, which was empirically set, with reference to an average of SSVEP amplitude over two runs chosen in the relaxed state, otherwise classified the data into "relaxed without any task".Here the absolute value of the SSVEP modulation each run, i, was defined as where i denotes the SSVEP amplitude at the i-th run.Classification accuracy of the proposed BMI was given by the grand mean of the classification rate into a correct task condition of run over the eight runs across the six subjects.Practically, the task condition corresponds to a user's intent to care givers; e.g., "engaging in task" corresponds to "Yes" response to them while "relaxed without task" corresponds to "No" response.

Reproducibility
The grand mean of modulation rate of SSVEP amplitude a r is illustrated in Fig. 3 (n =12).The amplitude of fundamental SSVEP (amplitude at 10 Hz) is seen to be reduced with mental focusing on the flicker stimuli, with the modulation rate a r ranging 0.7 ~ 0.75 over the six electrode sites from the central to occipital areas.This amplitude reduction is similar to ERD of the alpha wave in response to the mental task such as mental arithmetic and also ERD of SMRs in preparation for movement.
There was also significant difference for the amplitude modulation rate a r by using one-sample t-test with T < -3.7, p < 0.01 at all of the six electrode sites.Such amplitude modulation of SSVEP was also observed in the frontal/frontal pole, but the effect of volume conduction should be considered.
Fig. 4 shows the evolution of the average of the normalized SSVEP amplitude across the subjects.The abscissa shows the repeat counts of relax/task runs.The actual order of runs is shown in Table 1.The averages of normalized amplitude in the relaxed condition exceed those in the mental focusing conditions in the first three runs over the six electrode sites (figures for C1, P1 and O1 are omitted).Thus, the evolution of the normalized SSVEP amplitude provides evidence on a certain amount of reproducibility of amplitude modulation accompanying the mental focusing on the flicker stimuli.Nevertheless, the averages of normalized SSVEP amplitude in the relaxed state were reduced in the second half runs on run-by-run basis.The main effect of runs was found at C4 and P4 by ANOVA (F(7,11) = 2.3, p < 0.05 at C4 and F(7,11) = 3.2, p < 0.01 at P4), but differences between the normalized SSVEP amplitudes in the relaxed and focused states each run become small to have no significant difference in the second half runs at the three electrode sites.Moreover, there was no main effect of runs at O2 (F(7,11) = 1.7, p > 0.1) and no significant difference between the relaxed and focused runs in the first run at C4 due to large inter-individual difference (standard error in Fig. 4).
For reference, non-parametric Friedman test also showed the main effect of runs at C4 and P4 ( 2 (7) = 14.1, p < 0.05 at C4 and  2 (7) = 18.1, p < 0.05 at P4) but not at O2 ( 2 (7) = 9.9, p = 0.19).Wilcoxon signed-rank test was done for as a post-hoc test and there was significant difference between the normalized SSVEP amplitudes in the relaxed and focused states in each of the first and second runs at both C4 and P4 (p < 0.05 in the first and p < 0.01 in the second runs, at both C4 and P4), respectively.
Amplitude reduction in the relaxed state may due to insensitivity to the flicker stimuli caused by habituation and/or mental fatigue with repeating flicker stimuli.The present study applied flicker stimuli for 20 s each run, which might so long as to induce mental fatigue in the late runs.Duration of flicker stimuli should be shorten to maintain the responsibility of the SSVEP modulation.
On the effect of the ERD of the alpha wave (8~13 Hz) on the EEG modulation, the amplitudes at 10 Hz under the flicker stimuli in the relaxed and focused states were compared to the amplitude at 10 Hz in the free running.Most of the amplitudes under the flicker stimuli showed the significant difference from the latter by one one-sample t-test for the amplitude ratio to the latter with null hypothesis that the population mean of the ratio was equal to 1.The effect of the ERD of the alpha wave could be thus significantly smaller than that of the SSVEP modulation.

Task-dependence
Dependence of task the subject engaged in was investigated using the EEG modulation associated with each task.Classification accuracy in terms of SSVEP modulation in engaging each task was evaluated on a trial basis (Table 2) (n = 6).In Table 2, classification accuracy shown was estimated at an electrode site where the total of classification accuracy took a maximum value in the four electrode sites (P3, P4, O1and O2) each subject.The mental focusing task has the total classification accuracy of 87.5%, which is larger than the classification accuracies the other tasks have.Moreover, the classification accuracy of "No (relax)" is much smaller than that of "Yes (engaging in task"; the former is in the range of 58~66 %, whereas the latter exceeds 85 % in the three tasks.Such disproportion in classification accuracy is originated from the degeneration of responsibility to the flicker stimuli in the relaxed state as discussed above.From the practical point of view, enhancement of reproducibility of the SSVEP is crucial issue to be resolved to develop the proposed BMI. Besides, the present study used 15-s-lentgh EEG data to classify the user's intent, corresponding to a few bit/min of ITR.ITR of the BMI independent of eye-movement is limited to be less than 10 bit/min (18) , which is inconvenient for potential users with impaired eyes.ITR of the proposed system could be raised up by adopting an autoregressive parametric estimation method of the EEG spectra with short-time data (19) .

Conclusion
It is practically helpful to develop the BMI with eyes-closed for paralyzed patients who cannot control their eye-movement.The aim of the present study is to achieve a novel type of BCI using the SSVEP modulation associated with users' mental task.The SSVEP modulation with the mental focusing on the flicker stimuli was seen to be able to reproduce more than four times of runs.However, the responsiveness of EEG to the flicker stimuli was degraded with repetition of runs.Ensuring the steady SSVEP is critical issue to establish the proposed BMI system.Task-dependence of the classification accuracy was investigated on a trial basis.Although the mental focusing took the maximum classification accuracy, the classification accuracy for other tasks should be improved to reduce users' mental fatigue by repeating mental focusing on the flicker stimuli.

Fig. 3 .Fig. 4 .
Fig. 3. Grand mean with standard error of amplitude modulation rate of SSVEP amplitude a r across 12 subjects.Error bar designates standard error.**: p < 0.01 for independent one-sample t-test for population mean of a r with the hypothetical mean of