Double-Ring Marker Based 3 D Pose Estimation For Rod-Shaped MIS Instrument From a Single 2 D Image

In this paper, we propose a hologram like Double-Ring Marker based method which can estimate the six (6) 3D pose parameters of a Rod-shaped body such as MIS (Minimally Invasive Surgery) instrument using just a single 2D image. The six 3D pose parameters are derived from a set of geometric equations that the predefined Double-Ring Markers on the rod be projected onto the image plan through the camera’s perspective transformation. Compared to existing MIS pose estimation methods, which normally require time consuming processes including depth estimation, instrument identification and position parameters estimation as the pre-processing, the proposed Double-Rings marker based algorithm is very efficient and accurate.


Introduction
Minimally Invasive Surgery (MIS) become very popular because it can greatly reduce the pain and recovery span for patients.3D pose estimation and tracking of the MIS instrument is an important technique toward remote surgery through robotics [1-5].
Among the 3D pose parameters, the depth information of MIS instrument is currently obtained through the widely adopted stereo-imaging algorithm which requires the stereo cameras system and the time-consuming matching process between left and right (stereo) images of a scene as the pre-process.
In this paper, we proposed a Double-Ring-Markers based method for estimation of 3D pose parameters for any rod-shaped body including the MIS instrument.The proposed Double-Ring-Markers system requires only a single 2D image obtained from a camera, with additional two (double) ring markers of same width but different color painted at a predetermined distance on the surface of the rod-shaped body.Using the predefined information of the two rings and the perspective transformation of the camera, we are able to derive a set of deterministic equations for calculating the six 3D pose parameters of a rod-shaped body.Compared to the stereo camera system which is more costly in hardware and computation, the proposed method can estimate the 3D pose of any rod-shaped body very precisely and very efficiently in terms of both hardware and computational cost.
This paper is organized as follows: In section 2, we describe the system and derive the deterministic equations for calculating the 6 pose parameters.In section 3, we describe the process of the proposed algorithm.In section 4, we conduct experiments to prove the correctness of the proposed algorithm.Conclusion is made in section 5.

2.Derivation Of 3D Pose Parameters Of a Rod-Shaped Body
We assume that the origin of the 3D world coordinate system (X, Y, Z) and the 2D image plan coordinate system (x, y) are coincident as shown in Figure 1(a), where X and x and are horizontal axis, Y and y are vertical axis, while Z is the depth axis.Since the reference point AA1 is on the rod's axis, it cannot be seen and cannot be detected from the 2D image.Instead, the upper edge point A1 on the surface of Ring A with coordinate (X A1 , Y A1, Z A1 ) as shown in Fig. 1(b) is estimated before estimation of the reference (X AA1 , Y AA1, Z AA1 ).Fortunately, the 3D point A1 can be detected in the 2D image as an intermediate step toward the estimation of the reference point AA1.
Let line BA be the intersection of the rod's surface and the trajectories of the axis line   is the angle between X(x)-axis and the projection of directional line vector BA onto x-y (X-Y) image plan.
Angle  , rotating around Z axis, is also referred to as the uniquely determine the 3D position of the Double-Ring marked Rod-Shaped body.

Estimation of in-plane angle
Let the 2D directional line vector ba be the projection of directional 3D line vector BA onto x-y image plan, and  is the angle between ba and positive x-axis.ba can be found by connecting image point A(x a ,y a ) and B(x b ,y b ), where A(x a ,y a ) is the centroid of the area that Ring-A projected onto the image plan through the camera lens of focal length  .Similarly, B(x b ,y b ) be the centroid of the area that Ring-B projected onto the image plan through the same camera.There are four possibilities of angle  accordingly to the direction of the rod as shown in Fig. 2, where In this paper, we limit our discussion to rod direction BA pointing from left to right and away from the camera, due to page space constraint.
Fig. 2 Four possibilities of pose angle accordingly to direction of the rod

Estimation of out-plane angle β and depth Z A1
Figure 3 shows the rod pose projected onto the XZ plane, where x a1 , x a2 are the x-coordinates of the projected upper and lower edges of Ring-A respectively, while x b1 , x b2 are that of Ring-B.L and length between Ring-A and Ring-B L AB , we would like to find the out-plane angle β and depth Z A1 by using the perspective geometric transformation of the camera.
From Ring A of Figure 3, we have the following relations in 3D coordinates,  Using image perspective geometric relationships, we have Substitute Eq. (2) (3) to Eq.( 1), we get Simplify Eq.( 4) further by using the 3D geometric relationship  After derivations, we eventually obtain the out-plane angle β in Eq.( 5) and depth Z A1 in Eq.( 6) where

Estimation of out-plane angle  and depth Z A1
Similarly, given y a1 , y a2 , y b1 , y b2 ,  , L and L AB , by using the perspective geometric transformation of the camera on the YZ plane, we obtain the out-plane angle  in Eq.( 7) and depth Z A1 in Eq.(8) as follows: Note that Z A1 can be obtained either using Eq.(6) or Eq. ( 8), if |x a1 -x a2 | > |y a1 -y a2 | then Eq.( 6) is used; otherwise, Eq. ( 8) is used.

Estimation of 3D coordinate of reference point AA1
In Fig.Given image coordinate (x a1 , y a1 ), Z A1 and  , we would estimate the 3D coordinates of point A1 then the reference point AA1 as follows: Step 1: Estimation of 3D coordinates for A1(X A1 , Y A1, Z A1 ) on rod's surface.Since Z A1, the depth of point A1 , is available by eq.( 6) or eq.( 8) (see sec.2.3), we may obtain X A1 and Y A1 using the inverse perspective transformation, Step 2: Estimation of 3D coordinates for the reference point ) , , ( If x a1 >= 0 (reference point at right side of focal point) then ; If y a1 >= 0 (reference point at upper side of focal point) then , if |x a1 -x a2 | > |y a1 -y a2 | (the larger the difference the better the precision) then and R is the radius of the rod.

The Algorithm
Given the system described in section 2, the 3D pose described by the 6 parameters namely (X AA1 , Y AA1, Z AA1 ,   β, , ) of a Double-Ring marked rod can be estimated using just a single 2D image f(x,y) as follows: Step1: (a) Capture the double ring marked rod by a camera, then input the image or a video frame f(i,j) of size M, N and (b) Convert f(j,i) to f(x,y) which has coordinate origin the center of the image i.e.Step3: Obtain the intersections of line ba with Ring A front edge A1( x a1 ,y a1 ) and rear edge A2(x a2 ,y a2 ) as well as the intersections of line ba with Ring B front edge B1(x b1 ,y b1 ) and rear edge B2(x b2 ,y b2 ) as shown in Figure 5. Step5: Estimate ange  using Eq.(1) in sec.2.1 Step6: Estimate ange β using Eq.(24) in sec.2.2 .3 Step8: Estimate Z A1 .if |x a1 -x a2 | > |y a1 -y a2 | then Eq.( 6) is used; otherwise, Eq. ( 8) is used.
The six (6) 3D pose parameters namely (X AA1 , Y AA1, Z AA1 ,   β, , ) of a Double-Ring marked rod can be estimated with a single 2D image as described above.In addition, the tracking of the rod position can be achieved by repeating the above algorithm to each video frame.

Experiments 4.1 Experimental Set ups
The purpose of the experiment is to prove the correctness of the derived equations and evaluation of the proposed algorithm.Figure 5 shows the experimental setup as well as the tools for measurements of the six 3D pose parameters.

Experimental Results
While the 6 estimated parameters are obtained by the proposed algorithm which are unique for a given pose, the measured parameters are the average parameters measured by N observers (N=2 in this experiment).A common set of observation procedure is required to reduce the discrepencies among the measured data from different observers.The measured as well as estimated (X AA1 , Y AA1, Z AA1 ,   β, , ) and their differences of a double-ring marked rod at 4 poses are listed in Table 1 with the angles in degree and distance in cm.

Conclusions
We have proposed an algorithm for estimation of the six 3D pose parameters for a double-ring marked rod just a single 2D image.Similar to hologram, the 3D pose of a rod shaped object can be reconstructed from a single 2D image or a video frame.
The core of the propsoed algorithm is the equations for estimation of the 6 parameters (X AA1 , Y AA1, Z AA1 ,   β, , ) derived using the camera's perspective transformation relationships between the 3D rods (with double-ring markers) and the corresponding image.Experimental results prove the correctness of the derived equations and show that the proposed algorithm can estimate the six 3D pose parameters very efficiently with accurately.
It is also noted that the size of the ring markers as well as the distance between the two markers may affect the estimation accuracy.Further improvements on the estimation accuracy as well as estending to rod pose of any orientation is the future research topics.
Ring-A and Ring-B (Double-Rings) of the same width L are painted or marked surrounding the surface near the top (front end) of the Rod-Shaped body at a predefined distance AB L as shown in Figure 1(a)(b).Two different colors easily distinguishable from the scene are assigned to Ring-A and Ring-B respectively for easy and precise segmentation of Ring-A and Ring-B in the 2D image.The 3D pose of a rod-shaped object can be define by six (6) parameters i.e.X AA1 , Y AA1, Z AA1 and  Y AA1, Z AA1 are the 3D coordinate of the chosen reference point AA1 as shown in Figure 1(b).Points AA1 (and BB 1 ) is the intersection of the rod's axis and the plane containing the upper edges of Ring-A (and upper edges of Ring-B).The rod's position and direction is presented by the directional line 1 1 AA BB which pass through the rod's axis.
the image plan, as shown in Fig.1(b).Since we assume that the diameters of the rod is even, line BA on the rod's surface is parallel to the axis 1 1 AA BB .It is noted that point A1 is at the intersection of BA and upper edges of Ring A, see Fig. 1(b).

Fig. 1 :
Fig. 1: (a) The Imaging Relationship of a Rod-Shaped Body with Double-Ring-Markers (b) Reference point AA1 at the intersection of rod's axis and the plan containing the upper edges of Ring-A Three angles parameters   β, , In-Plan angle.(2) Angle β is the angle between X(x)-axis and the projection of directional line BA onto X(x)-Z plan.Angle β , rotating around Y(y)-axis, is also referred to as the out-plan angle.(3)Angle  is the angle between Y(x)-axis and the projection of directional line BA onto Z-Y(y) plan, rotating around X(x) axis, also referred to as Out-Plan angle.These 6 parameters (X AA1 , Y AA1, Z AA1 and   β, , )

Fig. 3
Fig. 3 Projection of 3D rod-shaped body onto x-axis in image plan Given x a1 , x a2 , x b1 , x b2 and camera focal length  , width of Ring-A and Ring-B


And the depth of point A1 1(b), note that the estimation of point A1's coordinate (X A1 , Y A1, Z A1 ) is an intermediate step to find the final uniquely defined reference point AA1's coordinate (X AA1 , Y AA1, Z AA1 ).
Thus, we have obtained the six (6) 3D pose parameters of the Double-Ring Marked rod-shaped object: X AA1 , Y AA1,

Fig. 3 .
Fig. 3.The geometric relationship between point A1 and the reference point AA1 From Figure 3 , Fig4.Convert coordinates Step2.(a) the areas of Ring A and Ring B of the rod separately and compute the centroids of Ring A and Ring B accordingly.(b) obtain the projected 2D line ba by connecting the two centroids.(c) Find the edges of Ring A and Ring B.Step3: Obtain the intersections of line ba with Ring A front edge A1( x a1 ,y a1 ) and rear edge A2(x a2 ,y a2 ) as well as the intersections of line ba with Ring B front edge B1(x b1 ,y b1 ) and rear edge B2(x b2 ,y b2 ) as shown in Figure5.

Step 4 :Fig. 5 .
Fig. 5. Intersections of line ba and front and rear edges of Ring A (A1&A2 respectively) and Ring B (B1&B2).

Table I .
Experimental Measured/Estimated Parameters (angles in degree, distances in cm)