Constrained Fuzzy Control for Wheeled Mobile Robot with Visual-Inertial Sensing

This study investigates the constrained fuzzy control design for the wheeled mobile robot through visual-inertial sensing. The visual-inertial-based localization system constructed from a binocular vision system and an inertial measurement unit (IMU) is used to measure the motion of the robot. The constrained fuzzy control law is designed based on the Takagi-Sugeno (T-S) fuzzy model of the mobile robot. The constraint induced by the visual localization system is also considered in the control design. The designed fuzzy control law is derived from the Lyapunov stability criterion. The experimental results demonstrate the effectiveness of the proposed constrained fuzzy control design for the wheeled mobile robot through visual-inertial sensing.


Introduction
The control design for wheeled mobile robots have been widely investigated.The position and posture sensing are key issues of the point stabilization for the wheeled mobile robots (1)(2)(3)(4)(5) .The visual sensing has been used in the point stabilization for the mobile robots (3)(4)(5) .Many studies employed the camera(s) mounted on the roof or extra tripod to sense the position and posture of the mobile robot (3) .Still, there are other studies use the camera directly mounted on the mobile robot (4)(5) to sense its position and posture by the parallel tracking and mapping (PTAM) (6) , simultaneous localization and mapping (SLAM) (7) , and visual odometry (VO) (8) algorithms.The PTAM and SLAM algorithms not only estimate the position of a moving mobile robot, but also build a map along the trajectory of the robot.However, the VO algorithm only focuses on measuring the trajectory of a moving mobile robot, which implies relative low computation demand.Therefore, the VO algorithm is considered in this study.
Recently, the inertial measurement unit (IMU) is also employed to improve the posture sensing of the VO algorithm.In this study, the visual-inertial sensing procedure is as follows.First, the binocular camera mounted on the mobile robot captures the images.By upright version of speeded-up robust features (U-SURF) algorithm (9) , the salient blob features are detected from the captured images.After repeatability check, the relative repeatable visual features are set as the landmarks.Through binocular vision geometry, the landmarks can be located.After landmark localization, the robot can move to next position.The moved robot can be positioned based on the located landmarks (8) .The posture of the moved robot can be measured by the IMU.In addition, the random sample consensus (RANSAC) algorithm (10) is used to reduce the localization error caused by the contaminated landmarks.Aforementioned visual-inertial sensing procedures are with heavy computation requirement.To persist in tracking enough visual landmarks, the mobile robot cannot move too fast, especially in rotation.The moving constraint limits the control design for the mobile robots.
The control design for the wheeled mobile robot represented in the polar kinematic model has been widely investigated.In fuzzy control method, the polar kinematic model of the wheeled mobile robot (2) is converted to the T-S fuzzy model by using sector nonlinearity method (11) in advance.Then the parallel distributed compensation (PDC) is employed to the T-S fuzzy model.The stabilization with control input constraint can be guaranteed by Lyapunov direct method.Comparing with most nonlinear control methods, the fuzzy-model-based control design with input constraint is much easier.
In this study, the practical control input constraint induced by the visual-inertial sensing system are considered in fuzzy control design procedure.Based on Lyapunov stability criterion, the stabilization conditions for the fuzzy control design are derived.The stabilization conditions are converted into linear matrix inequalities (LMIs) and then effectively solved by Matlab LMI tools (12) .The limitation of the visual-inertial sensing system is also represented as LMI conditions and considered in the control design.Finally, the experimental results validate the proposed constrained fuzzy control design for the wheeled mobile robot through visual-inertial sensing.

Wheeled Mobile Robot and Vision System 2.1 Wheeled Mobile Robot and Visual-Inertial Sensing System
Figures 1 shows the wheeled mobile robot (i-90) used in this paper.The vehicle body of i-90 robot measures 42*49*50 cm.The binocular vision system and inertial measurement unit (IMU) mounted on the robot (as shown in Fig. 2) are employed to sense the position and posture of the mobile robot.

Landmark Detection
During the landmark detection step, the salient and repeatable visual features are detected by using the upright version of speeded-up robust features (U-SURF) algorithm (9) .During the feature detection step, the U-SURF algorithm evaluates the approximated determinants of the Hessian matrices which are constructed from the convolution operations of the box filters in the captured images.And then it detects blob-like features at locations where the determinants are relative large.Next, the detected features are described by quadruple descriptor vectors ) descriptor vector.After repeatability check, the relative repeatable visual features are set as the landmarks.

Landmark Localization
The landmark is projected to the left image plane at as follows: Fig. 2. The binocular vision system and inertial measurement unit (IMU).IMU in {C} is: Then the m-th landmark m l in the world coordinates {W} is denoted as: where L C is the left camera position in {W}; W L R is the rotational matrix from {W} to {C}.

Robot Localization
After landmark localization, the robot can move to the next position.And then the moved robot is positioned by using perspective-three-point method (8,10) with the existing and located landmarks, the triangulation method.As shown in the Fig. 4 ).Then, the location of the robot can be geometrically derived.The posture of moved robot can be measured by the IMU.
In addition, the random sample consensus (RANSAC) algorithm (10) is utilized to reduce the localization error caused by the contaminated landmarks.The RANSAC and perspective-three-point method are incorporated to improve camera localization.The RANSAC first evaluates a model hypothesis from randomly sampled three landmarks and then verifies these hypotheses on the other landmarks.The quality of the model hypothesis is based on the number of the landmarks consistent with the model.The model hypothesis with more consistent input data is set as the better robot localization.The inconsistent (contaminated) landmarks and wrong camera localization are removed by choosing a better model.

Control Input Constraint from Visual-Inertial Sensing System
Iteratively executing landmarks localization and robot localization, the trajectory of a moving mobile robot can be measured.To persist in tracking enough visual landmarks, the mobile robot cannot move too fast, especially in rotation.Figures 5 and 6 show the influence on rotation motion of the robot.The squares in Figs. 5 and 6 denote the detected and tracking landmarks.In Fig. 5 the robot rotates in low velocity (0.65 rad/s).Figures 5(a -th frames, respectively.In Fig. 6 the robot rotates in high velocity (2.23 rad/s).Figures 6(a -th frames, respectively.Obviously, there are much fewer landmarks in Fig. 6 than those in Fig. 5.In this study, the constraint of the rotation control input is set as

Fuzzy Control Design
Figure 7 shows the coordinate system and kinematics of the wheeled mobile robot. 1 u and 2 u are the forward motion control and rotation control, respectively.The kinematics of the wheeled mobile robot in polar coordinates is as follows: The control law is intuitively considered as follows: The robot can rotate the heading toward the goal without moving the body by setting , the proposed constrained fuzzy control law is considered.Substituting Eqs. ( 8) and ( 9) into Eq.( 7), then we have Eq.( 10) as follows: By using sector nonlinearity method (11) , the nonlinear terms  cos and  sin in Eq. ( 10) can be exactly represented as a Takagi-Sugeno (T-S) fuzzy model in where , and 4 .0   k .After defuzzification, the T-S fuzzy model of the mobile robot is inferred as: The T-S fuzzy model is synthesized using parallel distributed compensation (PDC), and then the final output of the closed-loop T-S fuzzy system is as follows: The PDC design typically follows the Lyapunov stability criterion.The following displays a common quadratic Lyapunov function: The following Theorem provides the stabilization conditions for the mobile robot.
Theorem 1: Assume that the initial condition ) 0 ( p x is given.The T-S fuzzy model for the mobile robot can be stabilized by using the PDC, and the constraints can be enforced at all time, 0  t , if the following linear matrix inequalities (LMIs) are satisfied.
. I denotes an identity matrix of the appropriate dimensions, and arises from the visual-inertial sensing system as shown in section 2.5.A large control input ) ( 2 t u induces large visual landmarks disappear between two images and blurred images.Hence, the input constraint should be considered in the fuzzy control design. Remark 2: Unlike the traditional Mamdani-type fuzzy control design, the T-S fuzzy model based control design (Theorem 1) follows the Lyapunov stability criterion and can be solved by Matlab LMI tools (12) .Hence, the adjustment of the optimal parameter for the fuzzy control law is not necessary.

Experimental Results
Two experimental results are shown in this section.Example 1 is a ground truth experiment, which performs the localization ability of the visual-inertial sensing system.Example 2 is a stabilization experiment, which performs the effectiveness of the proposed constrained fuzzy control design for the wheeled mobile robot with visual-inertial sensing.

Ground Truth Experiment
The ground truth experiment is implemented on a 1m*1m square path as shown in Fig. 8. Four corners of the square path are set as the check points (A-D) to evaluate the accuracy of the visual-inertial sensing system.The visualinertial sensing system is mounted on a moveable platform and moved along the square path five rounds in a counterclockwise direction.The platform passes through all checkpoints in each circulation.Figure 9 shows the captured images frames, detected landmarks, and estimated trajectories of the visual-inertial sensing system.The squares in Fig. 9 Table 1 lists the position estimation of the check points. and  denote the average estimated position and the standard deviation in the localization, respectively.The ground truth experiment performs the localization ability of the visual-inertial sensing system.

Mobile Robot Stabilization Experiment
This experiment performs the effectiveness of the proposed constrained fuzzy control design for wheeled mobile robot through the visual-inertial sensing.and the control gains can be solved to satisfy LMIs ( 22)-(25) by using Matlab LMI toolbox (12) , where: Figure 10 shows the trajectories of the mobile robot based on eight initial conditions.The red block (bold solid line), green block (thin solid line), and black block (bold dashed line) in Fig. 10, denote the initial states, some transient postures and goal of the mobile robot, respectively.First, the mobile robot turns the heading toward the goal.Next, the mobile robot is stabilized by the designed PDC control design.The mobile robot can be stabilized in all of eight different initial conditions.The simulation results demonstrate the effectiveness of the constrained fuzzy control for the mobile robot.
Then the control design is implemented on the practical mobile robot, as shown in Fig. 1. Figure 11 demonstrates the experimental result of the mobile robot moving from

Conclusions
In this paper, the constrained fuzzy control has been designed for the wheeled mobile robot.Owing to the sensing frequency (constraint) is considered in the control design, the visual-inertial sensing system can be used to estimate the posture and position of the robot.Then the fuzzy control law can realize the position stabilization to the mobile robot.The constrained fuzzy control design can effectively solved by existing LMI tools.The simulation and experimental results demonstrate the effectiveness of the proposed visual-inertial-based fuzzy control method for the wheeled mobile robot.Further studies could apply the proposed control design to other robots with low state sensing rate constraint.
dy , in which the elements are the sums of the horizontal-and vertical-Haar wavelet responses and their absolute values, respectively.Based on efficiency consideration, the describing region of a visual feature is only within the central four sub-regions.Therefore, a visual feature is represented as a 16-dimensional (i.e.

Figure 3
Figure 3 displays the localization of the landmark.The center of perspective of the left camera is set as the origin of the binocular camera coordinates {C}.Let the position of any landmark in left and right camera coordinates are

f
are the focal lengths in x and y directions, respectively.From binocular visual geometry,
(a) are the detected and tracking landmarks.The green spot on the upper region of Fig. 9(b) denote the location of the landmark.The colored squares on the lower region of Fig. 9(b) denote the estimated trajectory of the visual-inertial sensing system.
a) shows the trajectory (red block) and the transient postures (green block) of the mobile robot.Figure 11(b) shows the response of the state variables  ,  ,  and  .The experimental results demonstrate the effectiveness of the proposed control design.
, the angles

Table 1 .
Ground truth experimental results.
The goal of the position and posture control is the origin of the reference coordinates; that is,