A Dual-loop Approach with Visual Servoing Fuzzy Control for Marker Navigation Quadcopter

This paper is to present a dual-loop control systems for the AR.Drone 2.0 quadcopter with internal controller treated as a black-box system to perform visual-servoing hovering task based on the vision feedback from its bottom camera and a visual marker placed on the ground. This system is implemented by using Robot Operating System (ROS). This paper describes the development, testing and implementation of marker detection method using Open Source Computer Vision (OpenCV) for image processing and also Fuzzy Logic Control to a real quadcopter, AR.Drone for controlling its position based on the marker on ground.


Introduction
In recent years, there is an increased interest on the research of Micro Aerial Vehicles (MAVs).By making an autonomous Micro Aerial Vehicles (MAV) which can perform the aforementioned tasks, it will open a lot of possibilities such as following or picking up ground objects, charging wireless ground sensors, landing on docking or charging station without human intervention in order to increase its flight time, aerial monitoring and inspection.Quadcopter is a class of four-rotor aerial vehicles which has the capabilities to provide stable acrobatic flight, navigate into places that ground robots or human cannot reach and perform tasks which are difficult or dangerous for human being such as agricultural surveillance and explosive landmines detection.Many commercially-available, off-the-shelf (COTS) quadcopters are available on market.The price range of COTS quadcopters varies depending on their size, weight, functionalities, onboard sensors and hardware.The Parrot AR.Drone 2.0 has been chosen as the quadcopter platform for this research due to its low price and rich onboard sensors (inertial measurement unit, ultrasonic sensor and cameras).The Software Development Kit (SDK) of AR.Drone which released publicly by the manufacturer has allowed developers to create programs that can communicate wirelessly (WiFi) with the quadcopter.Hence, this has opened up possibilities for a variety of research purposes, for instance high level autonomous and control algorithm can be developed for the quadcopter to perform specific tasks.Some previous studies have been made on the vision-based control of quadcopters.One of the visual servoing method for quadcopter is by using color-based tracking algorithm (1)(2)(3) .This color-based tracking method is simple and fast by filtering the color of object from the background having different color.When the pixel position of colored object in camera is obtained, its difference with the center pixel position of camera is used as input for the control algorithm to compute control signal for quadcopter.Besides that, a low cost sensor such as the Wii remote infrared (IR) camera is also used for tracking a pattern of IR lights under conditions with no direct sunlight as introduced in literature (3,4) .However, the methods mentioned above are easily being affected by the environment.Several research have employed motion capture system such as Vicon or OptiTrack which can obtain the pose of quadcopter with good accuracy.However, the high price and the short visual operation range have limited the vision range of the system.The method proposed in this paper has greater flexibility by the usage of onboard sensors and cameras only for its operation without the need of external devices.Besides that, multiple control algorithms are used for quadcopter system: PID control (1,3,5,6) , PD control (2,3) , Predictive Functional Control (1) and Fuzzy Logic Control (7,8) .Every control algorithm has its advantageous in performance (stability, robustness and disturbance rejection) over the others.
In this paper, we present a new visual servoing method by using onboard camera and ArUco marker that combining with inertial measurement unit.A fuzzy logic controller is designed to navigate quadcopter in indoor environment and compare with the original onboard controller loaded on AR.Drone 2.0.

Quadcopter Platform 2.1 Parrot AR.Drone 2.0
The quadcopter platform used in this research is AR.Drone 2.0 which manufactured by the Parrot company.
It comes with onboard flight controller which can give users the stable and secure flying experience wirelessly through smartphone.

Hardware Specifications
Its weight and size are 420g and 51.7cm x 51.7cm with indoor hull and 380g and 45.1cm x 45.1cm without hull as shown in Fig. 1.Its hardware specifications are listed in Table 1.

Black-box Model and Body of Reference
The firmware running onboard is not open-source, hence it is not possible to modify the software for tuning the internal controller or adding new functionalities such as visual-servoing.The internal controller can be treated as a black-box model where it receives inputs (roll angle, pitch angle, yaw rate and vertical velocity) and gives outputs as depicted in Fig. 2.Besides that, the body reference frame and motors arrangement are also illustrated in Fig. 2.

Visual-Servoing Control for AR.Drone
In this paper, the visual-servoing technique used is Position-based Visual-Servoing (PBVS) with eye-in-hand configuration since the camera is located at the bottom.The position relative to the tracking object (marker) is obtained by adopting the open source marker detection algorithm, ArUco (10) .

ArUco Marker
ArUco is a minimal C++ library for Augmented Reality applications developed by the University of Cordoba and it is based on OpenCV.The marker detection process is simple, fast and only requires a single camera.Guide SDK 2.0 (9) .

Tuning the Marker Detection Process
From experiments, the low resolution image from bottom camera contains noises which can affect the detection results are shown in Fig. 4. Thus, a low-pass filter called Gaussian Blur filter is proposed to reduce the amplitude of image variations by replacing the intensity of each pixel by weighted average of intensity values in its local neighborhood.The weighted scheme follows 2D Gaussian function (x and y is horizontal and vertical distance in pixel respectively while  is standard deviation: In this paper, Gaussian Blur filter with 3x3 block size and  = 1 is applied on the image from bottom camera and the result is shown in Fig. 3 and 5.By comparing the results of Fig. 4 and Fig. 5, it is can be concluded that the application of Gaussian Blur filter has greatly improved the marker detection result.It eliminates the high-frequency contents (noises) of the image captured by the low resolution bottom camera of AR.Drone and thus allows a more stable marker detection result.

Vision-Based Quadcopter Pose Estimation
The position of marker relative to camera can be obtained from tuned ArUco marker detection algorithm as introduced previously through camera extrinsic matrix [R,T], where R and T denote the rotation and translation matrix respectively.Then, the inverse of camera extrinsic matrix is computed to obtain the transformation matrix of camera with respect to marker, [R cam ,T cam ] as follows: Experiments showed that slight chattering (to simulate the real condition of quadcopter when flying) can cause the roll  and pitch angles  of detected marker with respect to camera to become unstable while the yaw  angle and translational distances remained stable.As a result, when these unstable values are applied to compute the inverse of camera extrinsic matrix [R,T] by using equation ( 2), it will lead to an unstable result of [R cam ,T cam ].To remedy this problem, a solution is proposed by utilizing IMU data of AR.Drone to obtain a more robust camera pose relative to marker.Assuming the marker is placed on the ground where AR.Drone takes off, then the roll and pitch angles measured by IMU of AR.Drone can be utilized in place of roll and pitch angles obtained from camera in computing the inverse of camera extrinsic matrix.The flowchart of this proposed algorithm is depicted in Fig. 6 where it takes camera image and IMU data as the inputs and gives the quadcopter pose relative to marker {X

Vision-Based Quadcopter Pose Estimation
AR.Drone 2.0 and a notebook computer as ground-based computer were used to conduct the experiments.The ground-based computer runs the developed ROS programs which can communicate with AR.Drone.The AR.Drone driver in ROS (12) is able to receive sensor data, camera image and send control commands to AR.Drone.The system architecture is shown in Fig. 10.
The output command follows the order of roll, pitch and lastly yaw is sent to AR.Drone every 0.2 seconds.The input limit for euler angle and yaw rate is set to 0.1 rad and 1.2 rad/sec respectively.

Experimental Results
The results of displacements and orientation of quadcopter relative to the ground marker by using fuzzy control are displayed in Fig. 11 and 12.These results are compared to that of onboard controller which is displayed in Fig. 13 and 14 where the quadcopter pose converges to zero value after 40 seconds because the quadcopter had moved away and hence the marker was outside the viewing angle of its bottom camera.These two controller performance in terms of hovering on top of ground marker are evaluated by calculating the root mean square error (RMSE) which are listed in Table 2.The RMSE of fuzzy control is calculated over the full time period of experiment: 0s ~ 60s while that of onboard controller is calculated over 0s ~ 40s because the marker was lost after 40s.

Conclusions
In this paper, we have presented a fuzzy control system added to onboard internal controller for AR.Drone to perform hovering on top of the ground marker.The performance of using the designed fuzzy controller is better in overall than onboard internal controller.Moreover, the fuzzy controller lets the quadcopter always hovering on top of marker during the experiment while onboard controller cannot.However, the fuzzy controller can be improved by introducing more inputs such as velocity difference and its performance will be evaluated to confirm its improvement.Besides hovering, other visual-servoing tasks such as following and landing can also be designed in future.

Fig. 2
Fig. 2 AR.Drone black-box model and body reference frame {X B ,Y B ,Z B }.
b } as the outputs.A translation is made to move the frame of reference from the camera frame {X C ,Y C ,Z C } to the body frame {X b ,Y b ,Z b } since the camera is not located at body center of quadcopter.

Fig. 3
Fig.3The result of applying Gaussian Blur filter on image from bottom camera: raw image (left) and binary image (right).

Fig. 4
Fig.4The result of original marker detection (marker pose relative to camera) with bottom camera and without filter when quadcopter in static condition.

Fig. 11
Fig. 11 X and Y displacement of quadcopter relative to ground marker with the designed Fuzzy Control.

Fig. 12
Fig. 12 Orientation of quadcopter relative to ground marker with the designed Fuzzy Control.

Fig. 14
Fig. 14 Orientation of quadcopter relative to ground marker with the onboard internal controller.

Table 1 The
Hardware Specifications of AR.Drone 2.0.

Table 2
Controller Performance Results.