Development of Sealant Inspection System Using Image Processing with Reduced Effect of Ambient Light

In recent years, a variety of manufacturing is being developed. In the factory production line, an automatic inspection system with lower cost and higher precision is needed. Many inspection systems use image processing. However, generally the inspection system using image processing cannot be examined in certain optical condition, i.e., room light or sunlight can greatly affect the performance. Another problem is high cost for capital investment of the inspection system. In order to solve these problems, an image processing method using the YCbCr color space characteristics and cyan light is proposed. Also, an inexpensive system for its implementation is developed. Finally, the inexpensive image processing inspection system for reducing the influence of ambient light can be realized.


Introduction
In recent years, the variety of manufacturing is being developed.Users require the high quality of industrial product made from the process.It can improve the reliability of the manufacturer.In this way, the inspection process for the purpose of security of society and quality assurance of the product has been introduction into the production line of the factory (1) .
In the factory production line, the inspection process visual have been performed for a long time.However, in recent years, mass production of industrial products are required in the factory to meet the user demands.Therefore, rigor of standards for product quality, and in conjunction with the spread of total inspection, automation of the inspection process has become essential.Further, recently, the hardware such as the camera and the LED have high performance.At the same time, the price has been reduced.Therefore, many inspection processes have realized automation by using a camera (2)(3)(4) .Inspection process of sealant in Nissan Motor Kyushu plant that is covered in this research is automated.
This research has been conducted by the joint research of Nishinippon Institute of Technology and Nissan Motor Kyushu.At the moment, the automotive sealant that has been applied to the seam of iron plate of the car wheel parts is inspected on a production line of Nissan Motor Kyushu plant.In addition, this sealant is used in order to prevent the water leak from the outside of a car.The proper conditions at this time are shown as follows.
 Specified range within the coating width of the automotive sealant．  There are no cracks in the automotive sealant.
The automotive sealant is inspected using image processing whether conditions are fulfilled and it is applied.The picture of the automotive sealant to be examined to acquire by this inspection process is shown in Fig. 1.
The following problems can be mentioned in the inspection system current.
(a) Specular reflection (white light) occurs on the sealant surface in response to the influence of various ambient light.(Fig.2) (b) Capital investment cost of the license is expensive.
We describe the details of the above-mentioned problems.The first problem is that (a).Ambient light changes from moment to moment : outside light from the skylight, work light, and sputtering or the like is emitted always in the factory.Therefore, the specular reflection occurs on the sealant surface with high reflectivity.The image processing of this portion is difficult.There is a solution that can block the ambient light by covering with a box-shaped inspection system.However, the case studies covered in the present research, a large-scale introduction like this is very difficult.Therefore, it has become a problem.The second problem is that (b).The capital investment for the software license of image processing, i.e., about 2.81 million yen of introducing cost, about 1.56 million yen of horizontal development cost, is very expensive now.Thus, it is also a problem.
In this research, we aim to solve the above problems.We propose an optimal image processing inspection system with reduced specular reflection (5,6) due to the influence of ambient light using an inexpensive system components.

System Configuration
High cost of capital investment of the license is a problem.In order to solve this problem, we have constructed an image processing inspection system using inexpensive system components.
First, the used software of the system components were free software freely available on the Internet.We used Microsoft Visual Studio Express 2012 for Windows Desktop as an integrated development environment.In addition, we used OpenCV2.4.3 as image processing library.
Subsequently, the used hardware of the system components were LED lighting and cameras.We used the web camera (i.e., LogicoolHDProWebcamC920) manufactured by Logicool as a camera for an experiment.On the other hand, we also used the network camera (i.e., acA2500-14gc) manufactured by Basler as a camera for an installation.The camera uses two types.As the history, the used web camera for the experiment is a very low cost.However, there is a problem, i.e., hard to get the same camera at the time of failure, because the change of a standard is intense.Moreover, since it is USB connection, there is a problem that the maximum transmission distance is short.We have used as a factory installed for the network camera to solve this problem.Finally, we used MEGA BAR LED RC manufactured by American DJ as LED lighting to illuminate the target object.
A system configuration diagram is shown in Fig. 3 and Fig. 4, where the solid line ( ) indicates signal, and the dash line ( ) indicates power supply.Then, round rectangle ( ) indicates power source, the rectangle ( ) indicates the device.
(a) System configuration for the experiment The used system configuration as the experimental verification is described in Fig. 3.

(b) System configuration for installation
The used system configuration as the field-installed sealants inspection process of Nissan Motor Kyushu plant is described in Fig. 4.
The automotive sealant (Black) Part of the specular reflection (White light)

YCbCr Color Space
YCbCr color space is the method of expressing a color by Y of a luminosity ingredient, and Cb (B-Y) and Cr (R-Y) of two color difference ingredients.This is also called YUV color space.Experimental results showed that human vision is generally sensitive to changes in brightness, but it is insensitive to changes in color.
There is property of the human eye that is sensitive to changes in brightness.However, it is insensitive to color change.Therefore, it is possible to reduce the color information of Cb and Cr components.In consequence, those are applied in various fields such as lossy compression technique or television broadcast (7)(8)(9)(10)(11) .
We focus on the YCbCr color space characteristics in this research.And, we have applied to the reduction of the specular reflection component and to the extraction of the sealant.

Conversion to YCbCr Color Space from RGB Color Space
An example of a conversion formula to YCbCr color space from the RGB color space is described in Eq.( 1).{  = 0.299 + 0.587 + 0.114  = 0.564( − ) + 128  = 0.713( − ) + 128 (1) From Eq.( 1), conversion to the YCbCr color space can be performed from the RGB color space.

Color Space Separation
YCbCr color space or RGB color space is a multichannel image with three components.Color space separation is separated into only a single channel component from a specific multi-channel image.
We investigated YCbCr color space characteristics in this research.One obtained by the color space separation is described in Fig. 5.

Application and Consideration of YCbCr Color Space Characteristics
We made a comparison verification of the four images in Fig. 5.We compared Fig. 5(a) with Fig. 5(c).From this results, we found that Cb channel increase the brightness values of the blue component only.Then, we compared Fig. 5(a) with Fig. 5(d).From this results, we found that Cr channel increase the brightness values of the red component only.
As a result, region extraction of the red component and blue component is possible done by the YCbCr color space characteristics.In this research, we extract the sealant region of the test object by a combination of the YCbCr color space characteristics and the LED lighting of a specific wavelength.The specular reflection component due to the influence of the ambient light at the same time can be reduced.

Experiment Verification of the Property of Sealant
We examined the characteristics of the sealant.In consequence, we have selected wavelength of optimal LED lighting.In addition, we have selected the best channel of YCbCr color space that suits it.Finally, it is confirmed by experiment to be useful in reducing the specular reflection component due to the influence of ambient light and extracting the sealant areas of the test object.

Experimental Condition
We used the automotive sealant as a verification object in this experiment.And, the distance to the verification object was 500 mm.Fig. 6 shows the experimental device．

Measurement Experimentation of Cb and Cr
The images were acquired verification object by web camera when illuminated LED lighting with a wavelength of various patterns.(Pattern of irradiation wavelength : OFF(extinction)，green light, cyan light, blue light, violet light.This is except for the wavelength that is contained in the ambient light : red light, yellow light, white light.) We have measured the Cb and Cr value of the iron plate portion and the sealant black portion by using the images.The comparisons were made using the measured data.We found that from the measurement results of Fig. 7 and Fig. 8, the difference between the Cr-value of the iron plate portion and the Cr-value of the sealant black portion is largest by cyan light irradiation.Therefore, it is possible to extract of the sealant when illuminated cyan light in Cr channel image.

Experiment for Verification and Comparison of Cr Channel Image
We compared the input image with Cr channel image obtained by illuminating the cyan light that is selected in Section 4.2.As a result, we confirmed the extraction of the sealant area.In addition, we also confirmed that some reduction of the specular reflection component due to the influence of ambient light can be.Figure 9 shows an input image of a cyan light irradiation.Figure 10 shows the Cr channel image of cyan light irradiation.We compared the Cr channel image (Fig. 10(b)) with the input image (Fig. 9(b)) in cyan light irradiation.Thus, we determined the specular reflection component of the sealant surface using a discriminant analysis method (12,13) .Table 1 shows the number of pixels of the specular reflection component of the surface of the sealant.In addition, Table 1 shows reduction ratio of the specular reflection component.
From Table 1, the Cr channel images of cyan light irradiation was found that 39.4 % of specular reflection components can be reduced.Finally, the usefulness of the Cr channel image of cyan light irradiation was confirmed.

Conclusion
It is possible to extract the sealant using the characteristics of the Cr channel and cyan light.Reduction of specular reflection due to influence of ambient light of the sealant was confirmed.Further, the area dividing the automotive sealant to define a threshold value for each area was verified.Therefore, we expected that the specular reflection can be removed completely.

Fig. 1 .
Fig. 1.The sealant at the time of the inspection.

Fig
Fig. The Cr channel image of cyan light irradiation.
LED lighting The black part of the automotive sealant Part of iron plate (a) Input image (b) Enlarged image of the square part (a) Output image (b) Enlarged image of the square part Part the reduced

Table 1 .
The number of pixels of the specular reflection component.