Driving Support System by Face Directions and Voice Recognition

Many people use cars for transportation. Owners of conditioned driving license are less than 1% of the normal driving license owners. This means that it is difficult such as getting on or off a car, operating a wheel, stepping on a brake or accelerator and turning on a blinker or hazard for disabled persons to drive a car. To resolve this problem, we developed a driving support system for handicapped persons. This system consists of Arduino, two servo motors, microphone and camera. It turns on blinker or hazard by voice recognition and face directions. After camera is started by voice recognition, it starts image processing. Direction determination of the face uses Optical Flow of Open CV. According to the face direction, the system turns blinker or hazard and cancel of these actions. If face turns right, the system starts right blinker. If face turns left, the system starts right blinker. If face turns up, the system starts hazard. If face turns down, the system cancels these actions. Thanks to this system, physically handicapped people especially in limb can drive a car easily.


Introduction
According to a national census of transportation method by Statistic Bureau, Ministry of Internal Affairs and Communications, people who use only their cars as a transportation method for around 45%.Although this survey was done in 2000, the percentage of people who use only cars go up by 7.1% compared with 1900.On the other hand, the percentage of people who use buses or on foot has decreased (1) .That is, today, half or more people use private cars for transportation.According to driving licenses by Metropolitan Police Department , in 2017, the statistics of disabled conditional driver's license is 241913 and, this is less than 1% of the driving license owners (2) .This means that it is very difficult to drive a car for physical disabled people.Get on or off a car, operate a wheel, step on a brake or an accelerator and turn on the blinker, hazard, wiper or lights.These are very difficult for disabled people.In addition, when they do such multiple actions at the same time, it's more difficult for physically handicapped people especially in limb.Also, not a few people have difficulty moving limb among the nonhandicapped driving license owners.And, be made to ride for those people, or customized cars, Hard to buy for expensive.
To this, the number of using cars is increasing in modern society, but a car is not familiar with a physically handicapped people.Now, there are some systems for physically handicapped people such as rehabilitation hand (3) and visual art interface that draws pictures using virtual brush (4) .In this study, we propose a driving support system for those who have handicap particularly in arm and hand.Now, rehabilitation hand, As a similar study, there is a system that turn signals by voice and face direction for those who have handicap particularly in arm and hand, it was the system that blinker is turned off after a little while automatically (5) .But, the actual blinker is turned off automatically just after a car turn left or right.In the previous study, that system has problems that blinker turns off during waiting a signal or turning left or right.In addition, that system didn't process correctly when microphone recognizes the voice mistakenly.In this study, we improved such problems and add the function that flashes hazard light.
This system was created by Arduino, camera, microphone and motion parts.Voice recognition was used Julius that is a free soft.

Driving Support System 2.1 Constitution
The system configuration diagram is shown in Fig. 1.The overall view of the system is shown in Fig. 2. The location of the camera to recognize the face direction shows in Fig. 3.This location is at the sun visor.I think it as the best location because this location is almost the same as any cars.
Next, the easy flow of work is described below.In this system, after obtaining the feature points in the animation, a vector is calculated as to how much the feature point moved.A feature point is a point where it is always present and can be clearly distinguished from other feature points.Feature points are shown in Fig. 4. In (a), a radio-control car is located in the middle of the image.The radio-control car moved only a small distance on the left is shown in (b).In this time, the system except to the extent not fit the requirements between (a) and (b), and draws vectors.This is (c).Also, the image size is 640×480.More of 80 Pixel in left and right at a time, on 50 Pixel, under 140 to calculate the vector in the range narrowed pixels.Vectors calculate at this time have conditions that both the start point and end point within the range to a minimum.Also, too big vector is false recognition of feature point, too small vector is noise, so, to specify the range of the length of the vector.The distinction of left and right condition for 0°line relative ±20°range of conditions.

Motion Part
The motion part of blinker is shown in Fig. 7, the motion part of hazard is shown in Fig. 8.
If the face direction is judged right by image processing, Arduino drives the servo in Fig. 7.As linear position in Fig. 7 rotate 50° left and turn right signal flashing.And then, the servo motor rotate to the right 50°, and back to the original line.Because of this motion, the system can be applied to make the blinker turned off automatically when you turn the wheel back to its original position.
Next, if the face direction is judged left by image processing, Arduino the servo in Fig. 7 drive.Like the right signal, fig.7 in relation to the position.From there the servo motor is rotated 50° right to the left signal flashing.And then, the servo motor rotate to the left 50°.
Then, if the face direction is judged up by image processing, Arduino the servo in Fig. 8 drive.This is the contrivance that switch is pushed by the servo motor 45° rotation.Besides, if the hazard lights are activated, camera can read the face direction only up.This makes, the hazard lights turn off.

Experiment
In this study, we checked the operation in the following fixes.
(a) Microphone recognizes the voice.After that, the camera starts image processing.(b) The system discriminates the face directions of the relative position.(c) We confirm the motion of the system according to face directions.

Conclusions
In this study, we developed a Driving Support System that can start the blinker and the hazard from the voice recognition and cancel the motion when the system starts in error.As a result, we succeed that people who struggle with the tip of a finger or arm can be start the blinker or hazard easily.
As the challenges for the future, we need to accelerate image processing, and so on.In addition, when we change lines, the blinker isn't returned automatically.So, we need to improve the system that can deal such situation.

①
To recognize a "winker" sound ② To start processing by the camera image ③ To determine the face directions ④ Send data based on face directions to Arduino ⑤ Operate the actuator by Arduino received the data These actions do using two servo motors.Also, processing face detections is described below. Up…Start of the hazard，and stop it  Down…If you start by mistake，the system does operation stop  Left…Start the left blinker  Right…Start the right blinker 2.2 Distinction of Face Directions Optical Flow of Open CV is used to recognize.Optical Flow is time take advantage of image columns, image velocity (speed of object + camera speed) it represented by the vectors obtained in.To use the direction in which the person in the image moved， camera determines as following conditions.① The same brightness of moving back and forth.② Image is smooth.③ Moving value is very slightly.④ Neighborhood Pixel does the same motion.
(a) initial position (b)position short distance moved (c) drawing of vectors Fig. 4.feature point The upper and lower discrimination condition for a line rotate 90°left/right distinction, it as ±20 °range of conditions.Also, valid vectors have 10 pieces or more, and vectors have other than 1.5 times more vectors are the orientation to determine the face direction.These conditions are shown Fig.5, Fig.6.

Fig. 5 .
Fig. 5.recognition condition of vector (1) (a)initial position of blinker (b)start blinker (c)servo return to initial position Fig. 7.action part of blinkerIn the end, if the face direction is judged down by image processing, Arduino the servo in Fig.7drive and rotate 20°both right and left.Because of this motion, you can stop the system when you start camera accidentally or blinker don't return to the initial position.
(a)servo rotate 45° (b)start hazard Fig. 8.action part of the hazard