Development of a robot that recognizes active landmarks by image processing and performs autonomous movement

We propose a robot that recognized active landmarks by image processing for autonomous movement in warehouse. With this proposed method, it is possible to introduce autonomous mobile system easily and it will not cost installation cost. Specifically, we decided to make the landmark which is the landmark of the autonomous movement not to be the fixed type like the conventional one, but to be easily installed in every place. In addition, the robot following the landmark recognizes landmarks using image processing technology. In fact, we developed 2 landmarks and 1 autonomous mobile robot and conducted experiments. As an experimental result, the limit distance that can recognize and follow the landmark was 60 m. Moreover, it was confirmed that it is possible to move autonomously accurately regardless of how the landmark and the robot are arranged.


Introduction
In recent years, the utilization of Internet mail order sales has increased worldwide, giving us various benefits in everyday life.People all over the world can easily order what they want using the Internet, and a distribution system that delivers to the house in a few days is established.In order to support these convenient systems, we must manage and transport huge amounts of goods accurately in the warehouse that manages inventory.Therefore, in the warehouse, the robot moves autonomously on behalf of human beings and carries the goods.
Two methods can be cited as representative methods of conventional autonomous movement.The first is a technique called line tracing [1].This enables autonomous movement by sticking a tape to the path the robot moves and recognizing that tape by the robot.The second method is a robot that performs autonomous movement by scanning a surrounding situation with a laser to create a map [2].The first method, the line tracing method, is the easiest method.The merit of this technique is that you can follow the route accurately.The disadvantage is that when installing an autonomous mobile system or changing the route, it is costly to install such as attaching and peeling the tape on the floor.Also, restrictions such as obstacles must not be placed on the route.The disadvantage of the second method is that it is necessary to acquire data again with a laser when changing the route.The common disadvantage of these two methods is that the place of use is limited.Also, it is difficult to do autonomous movement in any environment, such as a scene where a human and a robot work together, with the current method.
Therefore, We focused on the points common to the two conventional methods in order to develop a new autonomous mobile system.Both methods are that the tapes and buildings, etc. which are the markers of autonomous movement are passive markers that do not emit light or signals themselves.Here we define a passive sign as a passive landmark.Because it is a passive landmark, you can not send a signal to the robot himself and change the route freely.So, We thought that if we become active markers that can send and receive signals themselves, we can improve the problems of the conventional method.Define active marks as active landmarks, as opposed to passive landmarks.In this research, we develop an autonomous mobile robot that can follow the landmark by applying image processing to the active landmark and the camera image that light up in red and can exchange information by wireless communication, thereby reducing the cost of introducing the system It is aimed at reducing it.

Principle
Explain the principle that the robot recognizes red and follows the landmark.

OpenCV
In this research, a library called OpenCV was used for image processing.OpenCV is an open source library.OpenCV contains many algorithms related to image processing, computer vision, mathematical processing and machine learning, and various functions necessary for processing images and moving images are implemented.Image processing described below was executed using OpenCV.[3], [4], [5]

Conversion of RGB color space to HSV color space
The conversion formula from the RGB color space to the HSV color space is expressed by equation ( 1 The hue H varies from 0.0 to 360.0 and is expressed by an angle along the color wheel in which the hue is indicated.The saturation S and the lightness V vary within the range of 0.0 to 1.0.By converting to the HSV color space, there is the merit that setting of the threshold at the time of discriminating colors becomes easy.

Binarization processing
Processing for generating a binary image from grayscale images and color images based on appropriate conditions is called binarization processing.Fig. 1 shows the original image, and Fig. 2 shows the binarized image.
By performing binarization processing in this way, it is possible to extract only the target color in the image.There is a series of connected parts in this image, which is called blob.Next, the processing for the blob will be described.

Find the center of Blob
If we regard the blob as a single uniform plate, we can find the center of gravity.The coordinate of the center of gravity is (  ,   ), the coordinate value of the white pixel is (  ,   ), and the pixel value of the (, ) coordinate of the input image is   (, ).A is the area of the blob.
By using the center of gravity, it can be used for processing when tracking the target object.

Particle filter
Particle filter is a prediction method of time series data by probability distribution, also called sequential Monte Carlo method.In this research, red color is detected by using a particle filter and is followed in real time.In brief description of the particle filter, it is assumed that a number of possible subsequent states from the current state are represented by a large number of particles, and the weighted average calculated according to the likelihood of all the particles is assumed to be the next state.Real-time object recognition becomes possible by using particle filter

System configuration
The conceptual diagram of the system of this research is shown in Fig. 3.The model proposed in this research consists of a total of three devices, two landmarks that can communicate with each other and a two -wheeled robot that follows that landmark.The landmark 1 serves as a master unit that plays the central role of wireless communication and the other devices form a star type network that acts as a slave unit.

Two wheeled robot
The configuration of the two-wheeled robot is classified into five major categories, "control unit", "motor unit", "communication unit", "power supply unit", and "detection unit".The control section consists of m-stick and Arduino UNO 3. Motor part consists of DC motor and motor driver.The communication section uses wired serial communication and wireless communication by Xbee.The power supply unit uses a mobile battery.The detection unit uses a USB camera.Next, I will explain the detailed operation of each part.
In the m stick, we mainly perform image processing.First, HSV conversion is performed on the video obtained from the camera.You can detect landmarks in the video by applying a particle filter to the video.Then, depending on the bubble center coordinates, we decide whether the landmark is on the right side, the left side or the front side with respect to the robot.Then, it transmits direction data to Aruduino and performs motor control.
Also, in the communication section, you can inform the landmark as to whether it is close to the placemark.With this, you can head to the next landmark.

Landmark 1
The structure of the landmark 1 is classified into four major types, "control unit", "communication unit", "lighting unit", and "power supply unit".The control unit is composed of Arduino UNO 3. In the communication section, wireless communication by Xbee is used.In the light section, a red cellophane is wrapped around a commercially available LED lantern.The power supply unit uses a mobile battery.Landmark 1 plays a central role in this autonomous mobile system.

Landmark 2
The structure of the landmark 2 is classified into four major types, "control unit", "communication unit", "lighting unit", and "power supply unit".The control unit is composed of Arduino UNO 3. In the communication section, wireless communication by Xbee is used.In the light section, a red cellophane is wrapped around a commercially available LED lantern.The power supply unit uses a mobile battery.The landmark 2 plays a role of a mobile device operating with a signal from the landmark 1.

Experiments and result
Experiment using an autonomous mobile robot and two landmarks.

Measurement experiment of trackable distances
Use the two-wheel robot and the landmark 1.When the robot reaches the landmark 1 and the landmark 1 turns off, it is judged that the measurement is completed.Figure 5 shows the experiment.Fig. 3. Overview of the system Fig. 4

. A complete view of the device
As an experiment result, the robot followed the landmark straight up to 60 m.It was found that it was possible to follow a considerable distance with only one landmark.

Follow-up experiment with various patterns
Use the two-wheel robot and the landmarks 1 and 2. When arriving at the first landmark and arriving at the second landmark, it is judged that tracking is completed.
First of all, experiments were conducted to accurately track three devices when they were placed in a triangle.This was done when the landmark 1 was lit first and the landmark 2 was lit. Figure 6 shows the layout of the device.
(a), (b) in both cases, It was found that it follows accurately.Regardless of which landmark glows first, it follows exactly.In this way, the landmarks themselves transmit signals and flexibly decide the route, and the robot follows it accordingly.
Next, experiments are carried out with a pattern which devices are randomly arranged.Figure 7 shows the layout of the device.
Even in this case it followed exactly.As you can see, keep track of landmarks anyway.It is also possible to extend the tracking distance by using several landmarks.From the above experimental results, it was found that flexible movement which is not in the conventional method is possible.

Examination of results
Experimental results showed that the landmarks themselves are operating as active landmarks exchanging signals with each device and issuing commands.In this research we conducted experiments with three devices, but we can expect that more complex movements are possible if we add more numbers.
We also found that the robot can follow the red light and reach the landmark.In the experiment of 5.2.1, it was shown that tracking can be performed with only one landmark up to 60 m.The result of this experiment was able to be useful for future development of research.In this experiment, I wrote a program to detect the red light of the landmark, calculate the area of the red color one by one, and determine the threshold by the size of the area so that the correct landmark will follow, but with that algorithm it is insufficient There was a place.However, this makes it possible to accurately track the landmark by blinking the light of the landmark and recognizing it by image processing.
Therefore, this study showed that we could develop a mobile system using autonomous mobile robot using active landmark and image processing.

Conclusion
In this research, we have been conducting research aiming at the development of active landmarks that themselves actively transmit and receive signals and autonomous mobile robots with two wheels.As a result of device development and operation experiment, we showed that the landmark has turned on and turned off by transmitting and receiving signals, and that the two wheeled robot can follow up with the red light.Moreover, it showed that landmark 1 became the parent machine, and it was possible to gather information from all the devices and issue instructions.The system developed this time is more effective when the number of devices increases.It showed that we were able to develop a flexible autonomous mobile system that has never been done since we followed the device in any way.