Recognition of Pedestrian Traffic Light Independent of Lights Type and Day or Night Time

Several studies have addressed image recognition of a pedestrian traffic light for visually impaired persons. However, few studies have depended on type of traffic lights (blub and LED) and daytime or night time. So, we have carried out a research on the recognition of a pedestrian traffic light color independent of lights type and day or night time. We adopted the aspect ratio of the light in the daytime and the machine learning in the night time as image recognition method. In the case of daytime images, the recognition rate is 98.6%. In the case of nighttime images, the recognition rate is 98.1%.


Introduction
When visually impaired persons cross an intersection, it is very difficult for them to recognize the pedestrian light color, and so they are prone to an accident.Special pedestrian traffic lights (Acoustic traffic light and PICS) have been installed for the visually impaired to cross the intersection safely.Acoustic traffic light notifies visually impaired persons of traffic light color using sound when it turns green.PICS (Pedestrian Information and Communication Systems) conveys light color by communicating with the white stick of a visually impaired person.However, Acoustic traffic lights a little noisy, and PICS is costly.Consequently, they account for only 8.8 % of all traffic lights (1) .
For this reason, visually impaired persons need a special system to indicate a light color.The Telecommunications Advancement Foundation conducted a survey of a desired function of smartphones targeted at visually impaired persons in 2013 (2) .It is reported that they want a walking support application to notify them of the light color at an intersection.
In developing walking support application to indicate the light color, it is effective to recognize the color of a pedestrian traffic light using image processing.The recognition method by focusing on the feature points of the four corners of a bulb type signal was proposed by Horiba (3) .Although recognition accuracy by the experimental results is not shown and recognition time required per an image is about one minute in this paper.Matoba et al. proposed that recognition method by blinking LED type signal is recognition rate of 73% (4) .However, these studies have focused on both of the signal types: Blub-type and LED-type.Many of these researches focus on a particular signal type.Ikeda Keigo and Shiba Kojiro researched on the recognition of signal color independent signal types (5) .They have reported that recognition rate is 57.5% for the bulb type, and 2.4% for the LED type.The walking support application requires the recognition rate of 100% because a miss recognition can lead the crossing impaired person to a fatal accident.This result indicates a fair number of mistakes in recognition.In addition, them researches were conducted in the daytime.We are still faced with the problem of recognizing images at night.
For this reason, we have carried out a research on the recognition of a pedestrian traffic light color independent of lights type and day or night time.We propose method the aspect ratio of the light in the daytime and the machine learning in the night time as image recognition method.

Features of traffic light images 2.1 Type of pedestrian traffic light
LED type light and bulb type light have been installed in Japan.According to National Police Agency, 66.2% bulb type lights and 33.7% LED type lights were installed by the end of 2012 (6) .The installation rate of LED type lights is increasing, because it has an energy-saving effect and a long service life.Therefore, it is necessary to image recognize the light color of the pedestrian traffic light independently of lights type.Images of (a) the LED type pedestrian traffic light and (b) bulb type pedestrian traffic light are shown in Fig. 1.Fig. 2 shows images of human form in a pedestrian traffic lights.
Human forms in these lights are identical, and the aspect ratio of each of them is 5: 3. Additionally, bulb type lights shows a white human form against the background of a light color.In contrast, LED type lights shows a human form in a light color against the black background.
Therefore, in order to image recognize the pedestrian traffic light independently of light type, we must carry out different processing for bulb type and LED type respectively.

Features of the night emission images
The images of pedestrian traffic lights at night are shown in Fig. 3. Details of the emitting light are blurred.Additionally, the emitting portion becomes white, and its original color is spread around it.Thus, images of emitting objects at night is quite different from that in daytime.
Therefore, in order to image recognize the pedestrian traffic light independently of day or night time, different processing must be done during the day and at night respectively.

Fig. 3
Pedestrian traffic lights at night

Process
Because pedestrian traffic light colors have distinguishing features as shown in the preceding section, they should be recognized through the process shown in Fig. 4. First, we input image to determine the image at day or night time.Next, bulb type and LED type go through different processing.Finally, recognize the lights color conduct assertive processing of emission portion.Each of the processing will be described in detail bellow.

Processing of daytime images
Because the images in the daytime remain without a blurring of the details of lights, those with the aspect ratio of 1:1 or 5:3 are detected so as to extract a candidate of the light.In order to calculate the aspect ratio, we conduct the labeling processing of the image to limit the processing range.
As the pedestrian traffic, light image in the daytime remains without blurring the detail of lights, recognition of the light is carried out by using template matching.We calculate the difference between the template of the light images and the candidate region of the light.If the difference is below a certain amount, we conclude the candidate region to be the light.To calculate difference, we resize the template image to match the candidate region.

Processing of night time images
Images in the night time are crashed detail, and so we perform labeling processing after noise removal processing.The noise removal processing makes use of the expansion and contraction.As the use of expansion and contraction process alone do not produce much effect on the removal of noise, we adopt multiple expansion followed by contraction of the same number to remove small noise.
There is a case where the aspect ratio of a pedestrian traffic light is not within the range of 1:1 or 5:3.Therefore, instead of extracting the candidate region of the light, we have removed the candidate region other than that of the light.In this study, we have regarded the light area which is either greater than 5% or smaller than 0.05% of the entire image, and the aspect ratio which is outside the range of 2:1 to1:2, as the region not of a light, and removed it from the candidate region of the light area.
Because the images in the night time is blurred in detail, it is difficult to recognize the pedestrian traffic light by template matching.So, we use the machine learning which enables our system to learn features of the pedestrian traffic light from teachers' images of many pedestrian traffic lights.The experiment carries out learning by using a Convolution Neural Network (CNN) which performs convolution and pooling operation repeatedly and learns in a neural network.We have used CNN with 3 layers of convolution and pooling and 2 layers of neural network.Additionally, 50 images of night traffic lights were used for each of the teachers' images.However, number of teachers' images is not sufficient to do learning.Therefore, add various noises (contrast and gamma adjustment etc.) to increase the number.Learning was carried out using 950 teachers' images.

Estimation of light color range
In order to estimate threshold used for color extraction, the color of a light is estimated from the images in the daytime and night time in HSV color space.Fig. 6 show (a) the experimental results and (b) the range of light color defined by CIE (Commission Internationale de l'Eclairage) in 1975 (7) .It shows that the light color range is out of the color range in the CIE recommendations.This is considered to be caused by the fact that the color photographed by the camera is different from the color sensed by human beings through the eyes.Therefore, in this research we extract color by using the threshold value of the following formula (1).

Image recognition of a pedestrian traffic light
We have applied light color recognition process to the images photographed in the daytime and the night time.Recognition rate is shown in Table 1.90 images for the experiment were taken with an ordinary smartphone from 11 to 12 in the daytime and from 19 to 20 at night.The time required for recognition the signal light color was about 1 second.This recognition time is considered to be a sufficient time in the walking support application.The recognition rate for images photographed in the daytime is 97.8% in the aggregate while the recognition rate for blue images photographed at night is 100%.However, the recognition rate for the nighttime red signal was low.
In the results of both the recognition rate and the recognize time, our proposed method is better than the previous methods.So it is clear that our proposed method is superior to other methods.

Daytime light color recognition
In the case of daytime images, two errors were made in the recognition of the bulb type red light.A sample image that caused a wrong recognition of bulb type red light is shown in Fig. 7.The bulb type light was accurately recognized in Fig. 7, but the red lantern behind the light was erroneously recognized as LED light.Our system has made the same mistake twice in the experiment because the conditions of the red lantern are very close to those of the LED light: the red lantern also has an aspect ratio of 5:3 and the wall behind the lantern is black.This happened because we have used the human form and the color around it as the features for matching.We therefore think we can remove the errors in recognition by using the whole light including its frame as matching features.

Night-time light color recognition
In recognizing the night time images, the system has made two errors, one with the bulb type red light and the other is with LED type red light.As to the former, it was mistaken for LED type red light; as to the latter, it was taken for bulb type red light.Because both were recognized as red light, we think that there is practically no problem.

Summary
In order to safely cross an intersection for visually impaired persons, we have carried out a research on the recognition of a pedestrian traffic light color independent of lights type and day or night time.
We have succeeded in extracting a candidate region of the light by applying a linear process to the input image.Daytime images were recognized at the rate of 98.6% by template matching on the candidate region extracted using the aspect ratio.Night time images were recognized at the rate of 98.1% by machine learning using CNN.
The incorrect recognition of signal type occurred in this experiment.However, it was confirmed that the light color could be recognized.Therefore, it is an effective method for the walking support application.In the future, more experiments under various circumstances are considered to increase greatly the degree of completion.

Fig. 4 Flowchart of light recognition 3 . 1
Fig. 4 Flowchart of light recognition 3.1 Judge daytime or night time Input image is converted into the HSV color space which is close to the way humans recognize colors.A brightness histogram of the daytime and night time traffic light images are shown in Fig. 5. From this figure, we know that the brightness of images in the night time is stronger than that in the daytime.Therefore, images of daytime and night time are distinguished according to brightness.

Fig. 5
Fig. 5 Histogram in the daytime and night time Day time Night time

Fig. 7
Fig. 7 Bulb type pedestrian traffic light and lantern