Online Inspection System for Maize Kernel Based on Computer Vision

In this paper, we developed an online inspection system for maize kernel gesture recognition based on computer vision. The data ， such as the aspect ratio, the area, the width of the maize kernel and the angle of the deflection can be acquired. The angle of the deflection was transmitted to the PLC by communications, the rotation of the turntable can be achieved by PLC control. According to the selection criteria for the maize kernel ， those which cannot meet the requirements ， such as too small or damaged ， will be excluded ， while the qualified maize kernels will be clamped by the manipulator for subsequent processing.


Introduction
Due to a variety of factors (such as genes, diseases and pests, external forces, etc.), for practical applications (such as maize grading, maize breeding, etc.), the shape of maize sometimes occur incomplete phenomena (such as be damaged, mildewed, etc.), which will affect the selection of varieties.The traditional maize appearance quality detection still relies on manual inspection.The manual detection of maize is a time-consuming, labor-intensive process resulting in bias and human error which drastically affects the efficiency.Due to superior performance, ongoing improvements in cost, ease of use, and algorithmic robustness ， computer vision systems have become a common and scientific tool in industrial and agricultural manufacturing automation (1) .
1) Inspection for maize variety identification In 2013, Cheng Hong (2) from Agricultural University of Hebei presented a method of multi-corn kernel embryos detection based on threshold segmentation and morphology from a 2D digital image obtained by the scanner.
2) Inspection for damaged maize kernel In 2011, K. Nowakowski (3) from Poznan University of Life Sciences selected a set of universal features that distinguish damaged and healthy caryopses and developed a neural model for identification of mechanical damage to grain caryopses based on digital photographs;In 2014, Jose´ M. Valiente-Gonza´lez from Universitat Polite`cnica de Vale`ncia (4) proposed an inline acquisition system to capture a single dent corn kernel face and explores an algorithm using PCA based novelty detection to identify damaged kernels.
3) Inspection for the shape of maize kernel In 2004,Ning Jifeng from Northwest sci-tech university of agriculture and forestry (5) established two algorithms by analyzing mo-rphological features of tip cap and intensity and provided overall successful identification of 92.50% for tip cap and 89.58% for germ surface feature among four corn varieties feature of germ surface.In 2011,Wang Chuayu (6) from Chinese Academy of Sciences present a computer Vision method based on watershed segmentation, contour and pixel counting.The projected area, perimeter and axis length are selected as the kernel shape descriptor.In 2011,Wan Peng (7) designed a detection algorithm of the maize shape .The maize kernel pictures were firstly obtained using the detection system of the maize shape, then eight characteristic parameters, such as area, perimeter, length, width of the maize kernel and so on，was extracted according to the characteristics of maize kernel and eight characteristic parameters as input values were obtained，and finally，a BP neural network was built and used to test the maize shape． In summary, in recent years wide range of research using computer vision technology have been carried out to inspect agricultural products and grain kernel quality, to provide a more universal or targeted computer vision processing for the acquisition, storage and marketing of crops and recognition.However, automated production lines of real-time detection, processing, packaging for maize, rarely reported.Since a real time inspection is very necessary in judging the quality of maize kernels, in this paper ,this paper an online inspection system for maize kernel based on computer vision was presented.

(a) Hardware
The hardware consists of the following parts: 1) Computer ：Processor: i3-4160CPU@3.60GHz； Hard disk: 40G；memory(RAM)4.00GB； 2) PLC: MITSUBISHI QJ71C24  3) Software interface  2 is the software interface of Real-time inspection system for the shape of the maize kernel.It consists of five windows: the window of video display, gray image display, image processing, processing report, and parameter setting.First in the window of the parameter setting, the upper limit and lower limit of the aspect ratio, the area threshold and the width threshold was filled in the blanks, then after starting the camera, the video of the maize kernel can be seen on the window of video display.After the operation of the binarization, the gray image can be seen on the window of the gray image display.After the operation of image processing, the fitting ellipse, axes, and the outline of the maize kernel can be seen on the window of image processing.Finally, the report of the shape of the maize kernel can be seen on the window of processing report, for example, the aspect ratio, the area, the width of the maize kernel and the angle of the deflection.The software for this system was developed using the integrated development environment of Microsoft Visual C++ and OpenCV.The model structure of component object was used as shown in Fig. 3.Each object has its own data members and the corresponding function to complete the basic operation of the object.One such component is a software unit having a certain function, which is to achieve a clear function.

Image Processing
(a) The images was captured from the video files.If a frame was captured, the result was shown in Fig. 2(1),the code is as following: IplImage*img=0; if(!cvGrabFrame(capture)){ // Capture each frame printf("Could not grab a frame\n\"); exit(0);} img=cvRetrieveFrame(capture);// Retrieve the captured frame (b) In the captured image, since the image of the maize kernel is two small and not easy to determine ,in this paper, a specific area for the maize kernel was added.The result was shown in Fig. 2(2),the code is as following: CvSizesize= cvSize(200,150);//the size of ROI cvSetImageROI(ImageBefgray,cvRect(120,200,size.width, size.height));//Set the ROI of the source image (c) Then the captured color image was converted into gray image.The result was shown in Fig. 2 (3),the code is as following: cvCvtColor(&Frame,src,CV_RGB2GRAY);// Convert The image color space from RGB to GRAY (d) The contour of the maize kernel was fitted as an ellipse.Using OpenCV functions can allow ellipse fitting for the contour of the maize kernel easier, which mainly included the following functions: i) cvFindContours for searching the contour of the maize kernel; ii) cvDrawContours for drawing the contour; iii) cvFitEllipse for obtaining the contour of the fitting ellipse; iv) cvEllipseBox for drawing the bounding box surrounded by the ellipse.
(e) Searching for the tip point of the contour of the maize kernel.First, the perpendicular line was made from the contour of the maize kernel to the major axis of the ellipse.Then by comparing the distances from the vertical point and the center point of the ellipse, the longest distance was corresponding to the tip point of the contour of the maize kernel.The result was shown in Fig. 2(3).

Communication
After the MX Component was installed on a personal computer, the ActiveX library for communication will be attached to your computer, which supported the communications from the computer to the PLC.According to a deflection angle of the fitting ellipse of the maize kernel, through the communication with the host computer and PLC-based MX components, The angle information was transmitted to the PLC, the rotation of the turntable can be achieved by PLC control.the code is as following: CActqj71c241 m_qCpu00J; m_qCpu00J.put_ActCpuType(0x31);m_qCpu00J.put_ActBaudRate(9600);m_qCpu00J.put_ActPortNumber(0x01);As shown in Fig. 4 ， firstly, Pushing the button "OpenCom",filling the blank of "DeviceName" with "D200" and "Set Value" with "6",and Pushing the button"SetDevice"，that means the D200 was assigned as 6.Secondly, filling the blank of "DeviceName" with"D200", and Pushing the button"GetDevice".Finally，the blank of "Get Value" was appeared with "6"， that means PLC has received the information of "6".Fig. 4. Communication from the computer to the PLC

The selection criteria for the maize kernel
Experiments shows that, as for the normal maize kernel, the aspect ratio of the fitting ellipse should be within a fixed range of 1.3∼1.5.Only when the maize kernel is too small, or damaged or stand on the side, the value will jump out of the range.Repeated experiments showed that when the aspect ratio is 1.5∼2.0 or less than1.3, the maize kernel is too small or damaged, It should be excluded; and when the aspect ratio is greater than 2.0, then sending a signal of vibration for the platform ， the maize kernel will be processed again, as shown in Table .3.
Table 3.The selection criteria for the maize kernel.

Conclusions
When in the detection for the shape of the maize kernel, external force was prohibited to be applied on it.So it is necessary to adopt a non-contact method for non-invasive detection for the shape of the maize kernel.In this paper, we developed an online inspection system for maize kernel gesture recognition based on computer vision.The data，such as the aspect ratio, the area, the width of the maize kernel and the angle of the deflection can be acquired.The angle of the deflection was transmitted to the PLC by communications, the rotation of the turntable can be achieved by PLC control.According to the selection criteria for the maize kernel ， those which cannot meet the requirements，such as too small or damaged ，will be excluded， while the qualified maize kernels will be clamped by the manipulator for subsequent processing.
Therefore, the developed inspection system was developed for the identification and detection of biomass species, breeding biomass automation, greatly reducing the manual work intensity, improving the working efficiency.

( 1 )Fig. 1 .
Fig. 1.Schematic diagram of the online inspection system As shown in Fig.1, the schematic diagram of the online inspection system for maize kernel based on computer vision was presented.It is consist of camera, light source, manipulator , maize kernel ,PLC control cabinet , and the computer.

( 1 )
Fig. 2. Software interfaceAs shown in Fig.2is the software interface of Real-time inspection system for the shape of the maize kernel.It consists of five windows: the window of video display, gray image display, image processing, processing report, and parameter setting.First in the window of the parameter setting, the upper limit and lower limit of the aspect ratio, the area threshold and the width threshold was filled in the blanks, then after starting the camera, the video of the maize kernel can be seen on the window of video display.After the operation of the binarization, the gray image can be seen on the window of the gray image display.

Table 1 .
3) Industrial Camera：The manufacturer is baumer Co. from Germany.Product type is TXG12C.The technical specifications are as shown in table.1 : The technical specifications of the camera.

Table 2 .
The technical specifications of the camera.