Multi image stitching with cylindrical surface base on local feature matching for solving the distortion problem

This paper presents a method to solve the problem encountered in wide view panorama image stitching. The stitching problem is appearing on the registration process using projective transformations model (homography) that will make the distort image severely, increasingly, and continually. In this paper, all images are transformed into the cylindrical surface instead of rectangle surface, then computed the translation of relation from image to find the placement. Experimental result using cylindrical surface shows that image is not enlarged and distorted by cylinder surface transformation.


Introduction
Panorama image is a wide view angle than common image.Typically, a panorama image will be used for the landscape, and it is able to represent more realistic than the common image, because of the wide range of scene.The advantage of the panorama is a beautiful scene from surrounding such as travel photography, landscape and natural photography, and is also used for various medias.In addition, panorama images used for various applications or scientific research such as creating 3D virtual reality simulation, 3D medical diagnosis (1)(2)(3) .
The Algorithm used for merge multiple images with overlapping area is a well-known stitching image algorithm.Moreover, this technique is able to find the relation of image such as image segmentation (4) , and moving object detection (5) .The stitching image algorithm is simple.Firstly, the features are extracted from the images and computes the matching feature using the Euclidian distance then the result is the pair of points with relation or similarity called corresponding point.The output from the previous process shows the overlap area of the images.In generally, the homography is computed from the estimated geometric transform and adjusts the image into the same perspective called warping images.After that, the image with warping set to the main coordinates system, then the output will produce the high resolution panorama image with a wide view angle.
However, some problem of stitching image will continually appear in multi image.Warped image with homography can be distorted view (6) .This problem can be avoided by converting image to cylindrical surface.For this method, lens (Focal length) is important to convert the normal image to the cylindrical image.
This paper has organized as following; Section 2 discusses how to stitch the regular panorama image, section 3 explains the cylindrical panorama, section 4 discusses how to stitching with cylindrical panorama, section 5 discusses the experimental result, and section 6 discusses the conclusion and direction for future work.

Stitch regular panorama image
Stitching image algorithm has been applied in many tasks such as capturing the overview of city, complex image form surface of the earth by satellites and image of travel location (7) .The main deployment will focus on realistic results instead of effective results

Image stitching Algorithm
The stitching image algorithm is divided into 3 steps as shown in Fig.In this paper, SURF algorithm (8) will used to extract the features, which is fast and powerful resistance to change of image such as rotation, scale, blur, and value of illuminance.Normally, the similarity between the images is calculated by the Euclidian distance.The small value of distance means that the two points is similar.In conversely, they are different.In this paper, the threshold of similarity is set to 0.6 as shown in Fig. 3 Fig. 3. Position of similar key points between 2 images (left) main image (right) reference image Form Fig. 3, the overlapping area can be obtained from the position of matching key points.Form the left and right image, points of similarity will be matched with line and drawn between 2 images.After this process the overlapping area will show.
(b) Estimated transformation matrix and warping This step is for transforming the images to the same view.Some overlapping image may be found the problems because of difference of view between images such as translation, rotation, scale, etc as shown in Fig.This problem can be solved by transformation of image to the same view, and use the corresponding points from the previous step for estimated transformation matrix.The wellknown estimated algorithm is the random sample consensus (RANSAC) (9) .Advantage of this method is the robust

Rotation Scale
Skew Translation estimation of the homography matrix.RANSAC selectes or prunes the corresponding points by fitting the homography.Fig. 5. Types of transformation function (10) The transformation functions are various types as shown in Fig.
where  2 is the result after warping,  1 is image aspiring warping and  is the 3x3 homography matrix.
H is able to solve the equation using normalized direct linear transformation (DLT) (11) which is computed by the corresponding points at least 4 pairs.The number of corresponding points effect with the error, and that are categorized into 2 types; inlier, and outlier by RANSAC classification.Inlier points used to compute the homography for warping image as shown in Fig.      Finally Homography For Warping Image Fig. 9 Homography for warping image process From Fig. 9, the panorama images must stitch the image from left to right.The number image is .In this process, image 2 needs to transform to the same view of image 1, and image 3 needs to transform to the same view of image 2, and respectively.The number of homography matrix is n-1.From these homography matrix, the images are still not stitching.homography matrix must calculate the cross product such as H2 cross product with H1 with present H, H3 cross product with H2 with present H, respectively.The cross product will make for the cumulative value of translation matrix for correct position.In conversely, homography matrix produces the cumulative distort value, that the panorama result will distort, indefinitely.From Fig. 10, the position of T7 and T8 in the matrix are controlling the distorting value.(d) Distortion of image 3 Fig.11 The example of distortion matrix Fig. 11 (a) shows the original image and initial homography matrix (identity matrix), (b) and (c) show the distortion image that the weight are equal (-0.0005 and -0.001).(d) shows that the x is greater than 0, the images is smaller than original, when x tends to 1.In contrast, images is bigger than original when x tends to -1.The value of T7 and T8 value will cumulate distortion in case of panorama.
This problem is able to solve by converting the images from rectangle to cylindrical projection, instead of homography matrix for estimating translation transform.

Cylindrical Panorama
Cylindrical image is different view from the original image, as shown in Fig. 12   where (xc,yc) is the old coordinate of (x,y), (x ' ,y') is the new coordinate of (x,y), x is the width of image, y is the height of image, and f is a focal length.

Stitching with Cylindrical Panorama
This chapter discusses to solve the panorama image stitching problem.The main objective is not distorting result.This process uses translation matrix instead of homography matrix, and estimates translation transform by RANSAC.Translation matrix is shown in equation ( 5) where ( 1 ,  1 ,  1 ) is old coordinates and ( 2 ,  2 ,  2 ) is new coordinates after image transformation and (Tx, Ty) is the control translation image value.One pair of corresponding points for estimated translation transform is ( 1 ,  1 ) and ( 2 ,  2 ) calculated by equation ( 6) and ( 7) where Tx is the value of translation in x axis, Ty is the value of translation in y axis and ( 1 ,  1 ,  2 ,  2 ) is position coordinate system.Result is shown in Fig. 16.In generally, the regular panorama stitching for 2 or 3 images performs faster than cylindrical panorama stitching because of projection process.However, in case of 4 image up the cylindrical panorama stitching performs faster than regular panorama stitching as shown in Table 1

Conclusion
Normally, wide panorama stitching using regular panorama stitching are distortion.Most of distortion occur from horizontal image of four or more images.Moreover, the stitching image process is failure.Experimental results show that the cylindrical panorama stitching successfully solves these problems.
The output image still found the unsmooth, thus the future work will propose the blending algorithm and try to increase the efficiency of stitching by automatic estimated focal length.

Fig. 1 .
Fig. 1.Stitching image algorithm SURF algorithm is detecting the feature from the second derivative of Gaussian and descripts using haar wavelet for integral image.That is less computational time.The output called key points is able to identify the location, scaling, and orientation as shown in Fig 2.

Fig. 4 .
Fig. 4. Geometric transformation If the image are not transformed to the same view the result after stitch will distort, and it is uncontinue.This problem can be solved by transformation of image to the same view, and use the corresponding points from the previous step for estimated transformation matrix.The wellknown estimated algorithm is the random sample consensus (RANSAC)(9) .Advantage of this method is the robust

5 .
Each model has different properties such as Translation model can slide image to aspiring position, Euclidean model can increase rotation from Translation model, Similarity model can increase the scale from Euclidean model, Affine model can increase shearing from similarity work at same depth, and Projective model can transform all types of images.This paper selected to use the transformation function by Projective model, which uses the homography matrix 3x3 to transform and warping image calculated by equation (1).

Fig. 7 .
Fig. 7. Results after the end of the process, the panorama.

2. 2
Encountered problem in the regular panoramaThis problem is appearing on the warped image for stitching on many images.Example of distortion result is shown in Fig.8

Fig. 10
Fig. 10 Distortion control matrix Example distortion from T7 and T8 as shown in Fig. 11 (a) Original image (b) Cylindrical image Fig. 12 Original to cylindrical image Original image are square shape but top and bottom of cylindrical image are curve.The advantage of cylindrical transformation for wide panorama stitching, that it is able to create the non-distorted panorama.The vital parameter of cylindrical transformation is focal length (distance from the focus point of the system to the surface and control field of view) to control projection of cylindrical image (curving).
Fig. 13 Stitching cylindrical image algorithm

Fig. 15
Fig. 15 Corresponding points from cylindrical images (a) Before translation (b) After translation Fig. 16 Translation Transform From Fig. 16 (a) the Tx and Ty are equal to 6 and -7, will move to the new coordinate as shown in Fig. 16 (b) (d) Cylindrical panorama stitching In this paper, image stitching is performed begin from left to right and first to last of image.The result is shown in Fig. 17