Developing Wiimote Indoor Localization Environment for Mobile Robot and Quadcopter Interaction in Motion Planning Applications

In this work, we integrate Wiimote 3D localization scheme with mobile device for performing indoor localization applications. Specifically, the scheme is applied on a quadcopter for interacting with omni wheels to demonstrate the possible applications of this Wiimote 3D localization scheme in indoor living space applications. To navigate the robot, a new distance-efficient path planning method is proposed to find an obstacle-free path. We also developed a man-machine user interface with LabVIEW for monitoring the measured results. Finally, with the integration of navigation methods and Wiimote 3D localization scheme, the system has demonstrated the potential in indoor smart living applications.


Introduction
Both intelligent life application and smart factory concept from Industry 4.0 rely on mobile robot as the executers.In a smart factory, mobile robot can transport goods to the next process or work in random different areas.On the other hand, in a living space, mobile robot may offer eldercare, improve indoor security, clean the house, and run errands for people.Technically, both types of applications are based on perfect trajectory planning, navigation and feedback control.The key to complete these control methods or plans is accurate indoor localization.Various solutions currently exist for indoor positioning, navigation, and communications such as GPS localization (1) , inertial navigation system (INS) (2) , and ultrasonic localization (3) , have been developed.These techniques each have their own strengths and weaknesses.For details, please refer to (4) .
IR localization is another possible choice.Among them, Wiimote could potentially be a cost-effective solution.With embedded image processing capability, Wiimote lumps an observed IR image to a few points.The relationship between the sensing distance and the displayed pixel is simple.In addition, it has been demonstrated that the millimeter scale sensing resolution can be achieved (4) with a bandwidth approximately 107 Hz.Finally, the CMOS camera attached on a Wiimote can be desoldered and to be used individually without using of the rest components to work as compact IR-LED sensor.It is possible to integrate Wiimote with other indoor applications such as mobile service robot manipulations in intelligent living technology.
In our previous studies, Chen et al utilized Wiimotes to develop an accurate 2D localization system for indoor environment (4) .Fu and Chen proposed a Wiimote 2D localization system with a controllable IR LED array (5) .By the controlled input of the IR LED array, it is possible to extend the sensing area to cover the entire living space and the target can be identified correctly even if it passes through a shelter.Although Fu and Di Gu (6) have demonstrated the applicability of using Wiimote as an accurate position sensor for controlling the mobile robot.However, all above developments are all limited in planner motion detections.
Li proposed a 3D Wiimote localization technique (7) .The technique utilized two parallel Wiimotes and simple trigonometry to perform the stereo range finding.Li also uses this technique to determining both the position and attitude of two robot arms simultaneously (shown in Fig 1 .).However, the sensing area in this experiment is too small to cover the entire indoor living space.As a result, the sensing space should be extended and an experiment for demonstrating the feasibility of Wiimote 3D in typical living space should be performed.To achieve this goal, several tasks must be finished in prior.First, a significant extended 3D localization environment should be established by combining the 3D Wiimote technique (6) and the controllable IR LEDs system (7) to expand the Wiimote 3D localization area.Second, individual testing, performance evaluation, and calibration for the major carrier: a mobile robot and a quadcopter should be performed prior to the final integration.Finally, with the ready of both the environment and the carriers, additional path planning should also be designed to achieve the final demonstrations.

Fig. 2. Schematic plot of proposed scenario
The entire scenario is shown in Fig. 2. In a living space, two carriers, a mobile robot and a quadcopter, moves in an interactive manner under the monitoring of the Wiimote 3D localization system.The computer receives and calculates the quadcopter and the mobile robot position data by the localization system.With the path planning , the computer will navigate the mobile robot move toward the quadcopter in a collision-avoidance manner.

Fig. 3. Overall approach of entire work
To accomplish the design goal, the overall research approach is listed in Fig. 3. Establishment of Wiimote 3D localization environment, vehicles realization such as the quadcopter, and the mobile robot, and the motion planning, are developed in parallel.Essential testing and calibration of the 3D localization scheme on these vehicles are followed.Finally, a full integration conducted using the quadcopter and mobile robot is performed to demonstrate the possible application of the proposed scheme in smart living technology.

Development of Indoor Wiimote
Localization Scheme

Wiimote Basis
As shown in Fig. 4, Wiimote is the primary controller for Nintendo's Wii Remote originally designed for TV games.It contains Bluetooth wireless communication and a built-in high pixel CMOS image sensor, which is called "CMOS camera" in this paper.CMOS camera has great tracking ability with the sampling rate of 107 Hz.In addition, the CMOS camera is able to detect up to four IR LEDs.Fig. 4. The Nintendo Wiimote and its circuit board

Wiimote 3D Basis
Li proposed a Wiimote 3D localization technique (7) .Fig. 5 shows the schematic concept.When the IR LED is in the overlapped zone of the sensing zone of two Wiimotes, both Wiimotes detect the pixel coordinate of the IR LED and then transmit the information to PC through the built-in Bluetooth wireless communication to calculate the 3D position of the IR LED by a kinematic transform using simple trigonometry and known setup parameters.

Establishing Wiimote 3D Localization Environment
To capture the position of an object, a Wiimote mounted on the robot as a sensor to detect the IR LED landmarks attached on a ceiling.As shown in Fig. 6, we deploy an IR LED array on the ceiling.The distance between each IR LED is carefully designed so that the Wiimote attached on the carrier can detect four IR LEDs simultaneously.The total sensing area this IR LED array is 720cm  180cm.The distance between two IR LEDs along the length and the width is 40cm and 30cm, respectively.The IRLED array should be worked in a controlled manner and a switching board is realized for this requirement.
As shown in Fig. 7(a), we use NPN and PNP transistors design a circuit and use Arduino mega to control the I/O pins connected to emitter high and low.Fig. 4(b) shows the built circuit board for performing the switching control.Based on 3D Wiimote localization technique (7) and new controllable IR array, the 3D Wiimote localization environment is then established.

Experimental Design
Next, after setup of the environment, the test vehicles should be realized, including a quadcopter and an omni wheel mobile robot.

Localization of Mobile Robot Using Wiimote 3D
By detecting four IR LEDs shown in Fig. 8(a

Mobile Robot Realization
As shown in Fig. 9, we choose an omni wheel type mobile robot in this work because it moves in full direction and is very easy for motion control in indoor environment.
Fig. 9 shows the omni wheels built (6), which is driven by three DC motors (powered by a 12V battery) with an (BASIC Stamp 2) microcontroller powered by a 6V battery and a (eb500) Bluetooth communication protocol for the basic motion and communication tool.Wiimote attached to the omni wheels is also shown in Fig. 9(c) and it communicates with the computer via Bluetooth protocol.

Quadcopter Realization
The assembled quadcopter in this research is shown in

Communication and HMI of the Entire System
The quadcopter is controlled remotely .A LabVIEW interface is designed to acquire the position data of both the quadcopter and the omni robot via Bluetooth input trajectory by drawing path on the front panel, which is a convenient way for manipulating the robots.Since we have built the interface, it is convenient to discuss our effort in trajectory planning first, before presenting the integration experiments, which will be demonstrated in section V.

Design and Simulation of Indoor Motion Planning
For fulfilling a specific task in indoor space using a mobile robot, the navigation and motion planning are essential.In particular, the abilities on collision-avoidance and time or fuel efficient path generation are vital.There are many studies on path planning using various approaches.Wang proposed a dynamic-based path planning method (8) (dynamic programming (DP) method) for mobile robots in an indoor environment.Although Wang has proved that DP method is more efficient than other algorithms such as Bug or Potential Field algorithms, we believe that the DP method should be further improved.
The operating principle of the used algorithm is shown in Fig. 11(a).A robot moves from the start point to the final target.The ideal trajectory is the connection line between these two points and the desired path (shown in orange color) varies step by step.If the orange point is inside the blue obstacle, the point will be pushed to the edge in the radial direction to avoid collision.Eventually, an planned collision-free trajectory is generated and this path actually represents the shortest traveling trajectory. .To prove the above shortest length argument on , it is possible to align all orange points with checking the distance between two points repeatedly until the length r 1 plus r 2 equals r 3 shown in Fig. 11(b).Fig. 12.The simulation result of the method used in this paper Fig. 13.Path planning result by Wang's method (8) and the method used in this paper

Draw the Path and Control the Omni Wheels
With a drawing human machine interface (HMI), user can create path by dragging the mouse pointer.A typical trajectory we drew is shown in Fig. 14(a).Once the path is created, LabVIEW will search the local coordinate of the omni wheels by turning on and off the IR LEDs sequentially.After it find the omni wheels position, it will start to navigate the omni wheels and ensure the omni wheels go through the point on the path before LabVIEW send out next command.
The control scheme used in this work is the simple

Quadcopter Static 3D Calibration
On the other hand, to validate the feasibility of using Wiimote 3D localization on the quadcopter.As shown in Fig. 15(a), a calibration system is established to mimic the real operation scenario.That is a pair of CMOS camera is mounted on the ceiling and an IR LED is attached on a moving object.By adjusting the position of the moving object, the calibration of the Wiimote 3D scheme can be achieved and the experiment results is shown in Fig. 15(b).The blue point is the error between real height and the height calculated from CMOS cameras.It can be seen that the positioning errors increase with the sensing distance.However, the error is still acceptable for the indoor localization

Quadcopter Dynamic Measurement
After finishing the static calibration, localization for the quadcopter during operation is then follow in the built environment.The quadcopter is remotely operated to fly along a corridor in up and down manner.Figure 16 shows the pictures during operation and the fly trajectory.The localization error, based on the static calibration, should be within a few centimeters.

Quadcopter and Omni Wheels Interaction
Finally, the interaction between the mobile robot and the quadcopter is demonstrated here as the first step to realize the scenario shown in Fig. 2. The experiment is shown in Figure 17.The quadcopter takes off first and sending a signal to command the mobile robot moves toward the quadcopter via straight path.Then, the mobile robot reaches the destination, which is approximately 50 cm away from the quadcopter.Finally, the quadcopter lands in the front of the mobile robot.
The experimental results are shown in Fig. 18.The localization results of the above experiment is shown in Fig. 19.The Wiimote 3D localization works properly for locating the positon of both the mobile robot and the quadcopter.Notice that the quadcopter is remote manipulated manually and is not able to achieve a steady hoover.As a result, its position exhibit a considerable scattering shown in Fig. 18.Nevertheless, for demonstrating the feasibility, this experimental result should still be acceptable.

Discussion
In this study, a Wiimote 3D localization scheme is proposed.The scheme provides a reliable global position sensing for monitoring of both the aerial vehicle and the mobile robot.The global position monitoring over the entire living space can be achieved for subsequent task planning.
Furthermore, the experimental results also indicate that the interaction between vehicles through proper localization information and wireless communication has been successfully realized and demonstrated in this work.With such a capability in hand, it is possible to plan more complicate tasks utilizing the collaboration of two carriers together.
On the other hand, a new algorithm is also developed for performing efficient trajectory planning for mobile robots .This method is capable of determining paths for avoiding static obstacles and the result is more efficient than other methods such as DP.In the future, it is possible to develop dynamic obstacle avoidance based on this method.W

Conclusions and Future Work
Indoor localization of mobile units is critical for subsequent tasks in intelligent living technology.In this work, a novel Wiimote 3D localization scheme is proposed and validated.By operating IR-LEDs in a controlled manner and using two CMOS cameras as the sensors, possible carriers such as quadcopters and mobile robots can be properly located.In addition, by the developed path planning algorithm and the human machine interface programmed, an effective motion planning protocol for mobile robot has been established and is possible to be integrated in a living space.Finally, an experiment to show the interaction between a mobile robot with a quadcopter in the indoor environment successfully demonstrates the possible application of this localization scheme in indoor living space applications.
In the future, it is possible to apply more advanced control schemes in motion control.Meanwhile, the maneuverability of the quadcopter should also be improved.It is believe that through the efforts, more dedicated tasks can be designed and realized.This should bring significant impacts in intelligent living technology.

Fig. 6 .
Fig. 6.IR LED array for 3D localization environment (a) Part of circuit diagram.(b) Circuit on the board.

Fig. 7 .
Fig. 7. Switching circuit for controlling IR LED array ) simultaneously, the local coordinate can be determined.With the known IR deploy pattern and the absolute location can be found.Fig. 8(b) indicates the communication frow between the central host computer and the rest of the testing environment and the associate vehicles.When the omni wheels is about to leave a specific local coordinate, Arduino turns on other four IR LEDs mounted right above the Wiimote and the computer uses new local coordinate recalculate the position.

Fig. 8 .
Fig. 8. Communication relation flow between PC, IR LED array and mobile robot

Fig. 10 .
Which is driven by four SDMODEL brushless motor equipped with an (UNO MEGA2560 powered by 5V BEC) Arduino control card, a Pixhawk flight controller, a Bluetooth module, a PWM Encoder for translating multi PWM signals into one PPM signal, and a 11.1V lithium polymer battery for providing the power.As shown in Fig 10(b), two CMOS cameras is also attached to the quadcopter for sensing the IR LEDs.Fig. 10(c) shows the signal communication flow of the entire setup.

Fig. 10 .
Fig.10.Quadcopter use in this experiment Fig. 9. Mobile robot used for this experiment Fig. 11.Scheme plot for the proposed motion planing algorithm Fig. 14.Omni wheels control experiment result Fig. 15.Quadcopter static localization experiment Fig. 18.Interaction experiment result Fig. 17.Interaction experiment procedure