Warning System for Abnormal Movement of Driver ’ s Head

The paper proposes a robust warning system to detect abnormal movement of driver’s head. The LED light source is designed and placed behind the driver’s head. The CMOS sensor is used to acquire frames including the region around the driver’s head. The acquired frame is analyzed by the proposed abnormal head movement warning algorithm. Any head movement will result in changes of the LED region acquired by the CMOS sensor. By analyzing these changes, the monitoring algorithm can detect abnormal head movement and issue a warning signal correctly. The system is robust to violent illumination variation outside the vehicle and eye-sheltered effect from sunglasses. Software simulations are also given to demonstrate its effectiveness.


Introduction
When a driver is exhausted or absent-minded, his perception ability to surrounding environmental, handling ability to the car, and decision ability to traffic situation all diminish.They all easily result in serious traffic accident and loss of life and property.Hence, the development of safe driving system to detect whether a driver is exhausted is very important to prevent traffic accident.Actually different brands of car corporations and research centers devote themselves to develop this kind of system over the past decade (1)(2) .For example, the monitoring system "City safety" is the standard equipment in VOLVO XC60.
In the first category, it is necessary to attach sensors to the driver to measure the physiological signal.The attachment will interfere with the driver and hence limit its practical application.On the contrary, to monitor the state of driver's head or face in the second category, it just needs non-contact CMOS sensors instead of contact ones.Well-developed image processing techniques make the second category to be a promising approach.However, how to overcome the violent illumination change outside the vehicle is a big challenge in image processing.Some techniques shine infrared rays on driver's eyes to produce bright effect on pupil of driver's eyes, use the infrared sensor to acquire the frame with bright pupil of driver's eyes, and analyze area change of pupil of driver's eyes to judge whether the driver is tired.The eyes can faithfully reflect whether the driver is exhausted and the pupil area can be adopted to estimate PERCLOS [6] to measure the driver's fatigue.However, this kind of system has three major disadvantages. 1) Shining infrared rays on driver's eyes continuously may be harmful to driver's eyes.2) If the driver wears glasses, the light reflection from glasses will produce bright area which will be confused with the bright pupil.3) The bright pupil effect is obvious in the nighttime, but it is unobvious in the daytime.Practical application is still a problem in such kind of system.Hence, the development of practical monitoring system safe to driver's eyes, robust to violent illumination variation, and not being effected by glasses is an important topic in the safe driving system.
In the paper, we propose a new system which preserves the advantages of the second category and improves its disadvantages.

System Architecture
To monitor whether the movement of the driver's head is abnormal, the warning system for abnormal movement of driver's head (WSAMDH) is proposed.Lateral and front views of the system are shown in Figs.1(a) and 1(b) respectively.In the system, LED light source is adhered to the back cushion behind the driver's head.The CMOS sensor is used to acquire frames including the region around the driver's head.The acquired frame is analysed by the proposed abnormal head movement warning algorithm.Head movement causes driver's head covers different LED region and the LED region in the acquired frame changes.By analyzing these changes, the warning algorithm can detect abnormal head movement.

Abnormal Head Movement Warning Algorithm
In WSAMDH, the abnormal head movement warning algorithm (AHMWA) shown in Figure 2 is proposed to monitor the changes of the LED region.To quantitatively describe AHMWA, the following notations are adopted and defined: fdi: the acquired ith frame of size 540960, fd_Hi: horizontal region of interest (ROI) of size 17170 in fdi, fd_Vi: vertical ROI of size 5016 in fdi, fd_Hi R (p, q), fd_Hi G (p, q), and fd_Hi B (p, q): red, green, and blue components of fd_Hi (p, q), 1p17, 1q170, fd_Vi R (p, q), fd_Vi G (p, q), and fd_Vi B (p, q): red, green, and blue components of fd_Vi (p, q), 1p50, 1q16, fd_HSi: binarization result from segmenting fd_Hi, fd_VSi: binarization result from segmenting fd_Vi, area_HSi: foreground area of fd_ HSi, area_VSi: foreground area of fd_ VSi, ccog_HSi: column coordinate of center of gravity (COG) of fd_HSi, warningi: indicator whether to issue a warning signal to the driver for frame fdi.Based on the defined notations, AHMWA is proposed, shown in Figure 2, and described as follows.
Step 1: Acquire the new frame fdi, i = 1, 2, 3, fd H p q fd H p q fd HS p q fd H p q where 1p17, 1q170, 1r50, and 1s16.
Step 4: Compute the areas area_HSi and area_VSi After segmenting fd_Hi and fd_Vi, compute their foreground areas by the following methods: Step 5: Determine whether the head movement is abnormal If area_HSi  35, inspect the following criterion: Criterion 1: area_VSi 10.
If it is true, set warningi to be 1 and issue a warning signal to the driver.If area_HSi  35, compute the column coordinate of COG of fd_HSi by 17 170 area HS q fd HS p q ccog HS fd HS p q (7)   and inspect the following criteria: If either criterion is true, set warningi to be 1 and issue a warning signal to the driver.
In Step 2 of AHMWA, fd_Hi and fd_Vi of Fig. 1(b) are marked and shown in Fig. 3. fd_Hi and fd_Vi are regions of size 17170 and 5016 respectively.Both of them are small regions.Segmentation of ROIs by Eqs.(3)(4), computing areas of segmented ROIs by Eqs.(5) (6), and computing the column coordinate of COG of fd_HSi by Eq. ( 7) are all simple operations and performed on the two small areas.Hence, the computational complexity of AHMWA is very low.Fig. 3. fd_Hi and fd_Vi.
If the drive's head is in the normal position, the LEDs in fd_Hi and fd_Vi are completely covered by driver's head.There is nothing in fd_HSi and fd_VSi.The values of area_HSi, area_VSi, and ccog_HSi are all zero, hence the head movement is judged to be normal.If abnormal head movement occurs, the LEDs in fd_Hi and fd_Vi will be exposed.Hence, Step 5 can detect abnormal head movement and issue a warning signal to the driver.

MATLAB Simulation Results
To demonstrate the effectiveness of the proposed WSAMDH, the system is set up in an automobile.Two strips of SMD LED are adhered to the back cushion behind the driver's head.The vehicle video recorder CASA HDR-550 is used to record the region around the driver's head.The adopted video is of length about 59 seconds and includes the following two driving environments.1) The driver wears the sunglasses.2) The vehicle goes through the tunnel two times.
898 frames of size 540960 in total are extracted from the video by the software "Free Video to JPG Converter".The acquisition rate is 15 frames per second.According to AHMWA, MATLAB simulation is performed on all the 898 frames.Use fd510 as a representative to demonstrate the simulation results about ROI.fd510 is shown in Fig. 4(a).The horizontal and vertical ROIs fd_H510 and fd_V510 are enlarged and shown in Figs.4(b) and 4(c) respectively.Notice that fd_H510 and fd_V510 are regions of size 17170 and 5016 respectively.After segmenting fd_H510 and fd_V510 by Eqs.(3)(4), the results fd_HS510 and fd_VS510 are shown in Figs.4(d) and 4(e) respectively.According to the execution results of AHMWA, the related values of fd510 are listed in Table 1.From Table 1, we have area_HS510 = 171, area_VS510 = 82, and ccog_HS510 = 122.Since area_HS510 is larger than 35 and both Criteria 1 and 2 are true, the head movement in fd510 is judged to be abnormal.Hence, set warning510 to be one and issue a warning signal to the driver.As a representative of the proposed AHMWA, fd39, fd52, fd105, fd157, fd326, fd380, fd471, and fd510 are shown in Fig. 5.They are used to represent that the acquired frames are in poor and normal illumination.In the eight frames, the head motions of the driver's head can be classified into four types, normal position, departure to left, departure to right and departure to front.According to the procedure of AHMWA, the related values of the eight representative frames are listed in Table 2.If the movement of driver's head is judged to be abnormal and the warning signal to the driver should be issued, we mark a red square on the middle right region of the frame.All the warning red squares are also demonstrated in Fig. 5.For the simulation results of all the 898 frames, the warning indicators warningi, 1  i  898, are shown in Fig. 6.Based on the warning indicators, we mark a red square on the corresponding frame.The complete simulation results of all the 898 frames are concatenated to a video again by the application software "Vegas Pro 8.0".The video is uploaded to YouTube (12) and can also be played in Figure 7.In the acquired video, there are twelve times of driver's nods and two times of actions watching the rear-view of mirror.Inspecting the video demonstrated in Figure 7, all the nodding behaviors are detected exactly and the action the driver watches the rear-view mirror is judged to be a normal movement.The video also clearly demonstrates the features of the proposed WSAMDH.The driver wears the sunglasses and the automobile he drives goes through the tunnel two times.At the moment that the automobile enters or leaves the tunnel, the illumination variation outside the vehicle is very fiercely.Even so, WSAMDH overcomes the two challenges and detects the abnormal head movement correctly.
In the real system, the warning signal issued to the driver can be the sound from loudspeaker, figures in screen, music from player, light from source, vibration from vibrator, and so forth.

Conclusions
In the paper, the high-performance warning system for abnormal movement of driver's head has been proposed.The features of the proposed system are as follows.1) The system can overcome the violent illumination change when the vehicle goes through the tunnel.2) It is not influenced by sunglasses the driver wears.3) The action the driver watches the rear-view mirror is judged to be a normal movement.4) The AHMWA is a fast algorithm because only simple operations are performed on the two small areas.5) It is a low-cost system.
Finally, we believe that the active safe driving system can greatly benefit by adopting the proposed system.

Table 1 .
The related values of fdi according to AHMWA.

Table 2 .
The related values of fdi according to AHMWA.