CAD Model-based 3D Object Pose Estimation using an Edge-Based Nonlinear Model Fitting Algorithm

.


Introduction
In recent years, the application of computer vision has become more and more popular in robotic studies.For example, in industrial applications, a robot manipulator needs to know pose information of an object-of-interest (OOI) in order to perform pick-and-place tasks.In this case, computer vision techniques provide an efficient solution to detect and track the OOI for the robot to complete the task automatically.
Estimating pose information of an OOI is one of the most important grasping tasks in robot manipulation, but most visual sensing systems only provide two-dimensional (2D) image information, which is difficult to complete the task of object pose estimation in 3D space.Therefore, how to estimate the 3D pose information of an OOI from the 2D image of the object is still an important issue in the development of a visual sensing system.To achieve this purpose, this paper presents a novel model-based 3D object pose estimation algorithm, which employs a priori-known 3D CAD model of an OOI to estimate its 6-DOF pose with respect to (w.r.t.) the camera frame of a vision system from an observed object image.This feature allows increasing the applicability of a robotic vision system.

Related Work
In a robotic vision system, the existing object pose estimation techniques can be roughly divided into two categories: feature-based and model-based tracing methods.The feature-based tracking methods use basic geometric features (such as points, lines, and circles, etc.), boundary contours (1,2) , and regions of interest (3) of the object to detect and track the OOI appeared in the image plane.By contrast, the model-based tracking methods use a priori-constructed 3D CAD model of the object to assist the tasks of object detection and tracking (4) .Compared with the feature-based tracking approaches, the model-based ones usually provide more robust tracking results, even the OOI is suffered from the partial occlusion problem (5) in a complex work environment.
To achieve 6-DOF pose tracking, an efficient method is to use an artificial mark attached to the OOI to simplify the design of object detection and tracking processes (6) .This method provides a simple and robust way to resolve the 6-DOF pose estimation problem, but it cannot work without an additional marker on the OOI.Therefore, many researchers have focused on the development of natural feature-based 3D pose tracking methods, which can be divided into keypoint-based and edge-based pose tracking methods (7) .Keypoints are image features that can be repeatedly detected in images robust to translation, scaling, and rotation transformations.In keypoint-based approachs, the keypoints are first extracted from the incoming images to detect the OOI.The 6-DOF pose of the OOI can then be detected by a keypoint classifier, which was trained from a large training set with different view ports of the OOI.By doing so, the pose tracking problem is simplified as a keypoint recognition problem and can be efficiently resolved using random trees (8) for fast nearest neighbor keypoint matching.
On the other hand, edge-based pose tracking methods usually use a 3D CAD model to estimate the 6-DOF pose of the OOI detected in the image.For example, in reference (9) , Choi and Christensen proposed a RAPiD (10) style tracking system, which consists of a global and a local pose estimation unit.In the global pose estimation, a keypoint-based object recognition method was employed to detect the OOI in the image and decide its initial pose information w.r.t.current camera frame.In the local pose estimation, an edge-based pose tracking algorithm was developed to estimate the 6-DOF object pose w.r.t. the camera via back-projection optimization given an initial pose and a 3D CAD model of the OOI.The proposed system provides an automatic object detection, recognition, and pose tracking solution; however, only polygon mesh models can be used as the object CAD model in this system.
In this study, a novel edge-based nonlinear model fitting algorithm is proposed to implement an image-based 6-DOF pose tracking system using a monocular camera.The proposed method is a nonlinear optimization process based on an edge-based distance metric, which can work with 3D polygon edge and polygon mesh models.Experimental results demonstrate the performance of the proposed system.
In the remainder of the paper, Section 3 introduces the proposed CAD model-based 3D object pose estimation algorithm.Section 4 presents the experimental results to evaluate the effectiveness and efficiency of the proposed 6-DOF pose tracking method.Section 5 concludes the contributions of this paper.

Proposed Method
Figure 1 shows the flowchart of the proposed pose tracking system.Suppose that the OOI in the image has been detected by an object detection and recognition module.Then, the line features of the OOI are extracted from the input object image by using edge detection and Hough transform techniques.In this work, we use Canny edge detector combined with straight line Hough transform to extract line features of the OOI detected in the image.Given a CAD model of an OOI, a line matching process is employed to find the nearest edges between the detected line features and the projection of the CAD model without hidden lines.Next, the 6-DOF pose of the OOI can be estimated through a nonlinear model fitting process, which is the main subject of this paper.
After matching the detected line features of the object and the projected lines of the CAD model, the proposed nonlinear model fitting process aims to estimate the 6-DOF pose of the object by fitting the projected boundaries of the CAD model to the detected Hough lines of the object.Let R and T denote a quaternion rotation matrix (11) and a translation vector such that ) , , , ( q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q y x x w z y y w z x , where (w q , x q , y q , z q ) are the four components of a quaternion, (t x , t y , t z ) are the three components of a translation vector.Suppose that the camera intrinsic matrix K is computed a priori via a camera calibration process (12) .Then, the projection of a CAD model point X c =[x c , y c , z c ] T onto an image coordinate p i =[x i , y i ] T with respect to an object pose P=(w q , x q , y q , z q , t x , t y , t z ) in camera coordinates can be computed by where s is a scale factor, where d is the distance between a projected control point p i and the corresponding Hough line h.Suppose that there are m Hough lines, each of them having n j corresponding projected control points.Then, a nonlinear cost function to evaluate the error distance between the projection of the CAD model and the boundaries of the object w.r.t. an object pose in the camera frame is defined as )) ( , ( ) where P is the object pose in camera coordinates defined previously, jk i p denotes the k-th projected control point corresponding to j-th Hough line.Finally, the optimal object pose can be estimated by minimizing the cost function f(P) such that where P ˆ is the optimal object pose.Note that the initial value of P is set as (1, 0, 0, 0, 0, 0, 0) in our implementation.The nonlinear model fitting algorithm described above is validated in the next section.

Experimental Results
Figure 2 shows the 3D polygon CAD model used in the experiments, which is a rectangular cube model containing twelve line features.A rectangular box was used as an OOI in the experiments.Figure 3 1 records the 6-DOF pose tracking results of the detected OOI shown in Fig. 3. From Table 1, one can see that the yaw angle of the detected OOI is changed from -0.1515˚ to -14.8174˚ as the OOI rotates left along y-axis.This implies that the proposed tracking algorithm successfully estimates the motion trajectory of the detected OOI in the camera frame.Therefore, the above experimental results validate the tracking performance of the proposed method.
p i and X c , respectively.Now we define an edge-based distance metric for the following optimization process.Suppose that a 2D Hough line located on one of the boundaries of the object is parameterized by h=(a, b, c).Then, the distance between the projection of a CAD model control point (termed as a projected control point) and the 2D Hough line is given by Figure2shows the 3D polygon CAD model used in the experiments, which is a rectangular cube model containing twelve line features.A rectangular box was used as an OOI in the experiments.Figure3presents the

A
novel CAD model-based 6-DOF pose tracking algorithm is presented in this paper.The proposed algorithm is a nonlinear model fitting process based on a new edge-based distance metric.Experimental results validate the tracking performance of the proposed method.