Image Processing in 3 D Profilomerty Measurement Based on MATLAB

Phase-shifting profilometry (PSP) combined with gray-code patterns projected onto the object surface have been widely used to 3D shape measurement. Because of the change of the 3D surface, the distortion stripes appear when the sinusoidal grating or gray-code pattern is projected onto the object surface, which includes information about the 3D surface profilometry. The deformed grating pattern is taken into a computer by CCD camera and processed by a series of procedure: image pre-process (cropping and filtering), fractal image coding, thresholding and morphology process. Then according to the corresponding mathematics model and arithmetic of digital image process analyze to obtain the unwrapped phase of the superficial of the measured object.


Introduction
In recent years, high-speed, high-accuracy, and non-contact three dimensional (3D) measurement has been an extensively studied research area due to the diversity of potential application which extends to a variety of fields including but not limited to mechanical engineering, industrial production, aerospace, virtual reality, biomedicine and other industrial applications (1)(2)(3) .
With the recent advancements in digital technology, the use of fringe projection techniques for 3D shape measurement techniques has become one of the most active research areas in optical metrology (4) .Among these existing 3D shape measurement techniques, the use of fringe projection techniques for generating 3D surface information has become one of the most popular research areas in optical metrology because of its ability to provide high-resolution, whole-field 3D reconstruction of objects in a non-contact manner at high speed.
MATLAB is a scientific calculation, interactive high-tech high-performance visualization software.As a visual programming tool, MATLAB has the advantages of other programming software cannot match, it compresses many widely functions together, placed centrally in the interactive window for easy operation, such as numerical analysis, nonlinear system dynamic simulation, matrix operation, etc. MATLAB as a programming language and visualization tools, provides the scientific research and numerical operations as a good visual interactive platform, represents the scientific computing software with advanced level.
There are many authors have researched a lot about 3D reconstruction, but few paper given the detail about the image processing during the reconstruction process.So it is meaningful to present the image processing part using MATLAB for some new researcher.

PSP Combined with Gray-code Patterns
Phase-shifting profilometry (PSP) combined with Gray-code patterns projection has been widely used for 3D measurement due to its merits of flexibility, high-speed and high resolution.The PSP uses the following sinusoidal patterns.
Where and N denotes the number of phase steps, ' ( , ) I x y is the average intensity, '' ( , ) I x y is the intensity modulation,  is the phase to be calculated, ( , )   x y denotes the spatial coordinate, and n  is the phase-shifting amount, selected as 2 ( 1) / , 1, 2,..., .
The phase can be obtained as equation ( 3) by using a least-squares algorithm (5) .
Generally, more fringe images that are used, the better the measurement that can be obtained.However, taking more fringe images will reduce the measurement speed.In this research, we use a three-step phase-shifting algorithm with a phase shift of 2π/3.
The Equation (3) indicates that the phase value obtained ranges from -π to + π.To determine a continuous phase map, a set of m gray-code patterns ( 1,2,..., ) i G i m  are used to generate 2 m code words for phase unwrapping as follows (6) .
where C denotes the codeword calculated from gray-code patterns.Each code word can uniquely determine a phase order K , which helps to unwrap the phase as Three gray-code is selected for illustration In Fig. 1.From top to bottom of Fig. 1, three gray-code, 1 G , 2 G and 3 G , the corresponding codeword C , the wrapped and unwrapped phases are shown.Note that to keep the unique relationship between C and K , the coding patterns perfectly align with the 2π phase jump.
In this experiment, the tested reference plane is placed in front of the camera about 40cm.One series of 1024  768 fringe patterns includes three sinusoidal fringe patterns and five Gray-code patterns.The period of sinusoidal fringe patterns is 32 DLP pixels and the number of pattern periods is 32.After they are projected and then captured, we will deal with all the images by the steps presented in the following section to get the absolute phase for the test reference plane.

Image Processing Process Analysis
Image processing part mainly includes image reading, filtering, binarization, morphology, stripe analysis and so on.The flow is shown in fig. 2.

Image pre-processing
Image pre-processing mainly includes two parts: image cutting and image filtering.It is to improve the resolution of the original image, eliminate all kinds of interference noise, enhance the recognition signal, and avoid blurring the image, and prepare for the image processing work in the rear.
(a) Image cropping Because the image usually contains too much unnecessary background information or the image size is not meet the requirement, it is necessary to cut the original image and cut off the unnecessary background and edge, which can reduce the interference, improve the effect of image processing and save processing time.In the process of image cutting, we must carry out according to specific requirements.In this paper, the contour of the object is retained, and the edge part which is not related to the detected object is removed.We can use the command of "imrect", "getPosition" and "imcrop" to get the region of interest for an image by mouse directly.Fig. 3 shows the original image of one gray-code pattern adopted by a camera with the resolution of 1280  (

c) Filtering
The image is inevitably disturbed by various noises during the reading process, which may be produced in the process of image acquisition, quantization, etc., and may also be produced in the process of image transmission.The information is polluted by noise.In order to eliminate noise, image filtering must be adopted (7) .
The purpose of image filtering is to improve the resolution of the original image, eliminate all kinds of interference and noise, and enhance recognition signal.Generally, noise can be reduced by means of average filter or median filter in the space domain.In frequency domain, because the noise spectrum is usually in the high frequency section, all kinds of low-pass filters can be used to reduce the noise.
In an image, we can look at the mean ( , ) L m x y of its neighbors (2 1) (2 1) x y : Then, the new gray levels can be calculated as k is the ratio of the new local standard deviation to the original standard deviation.If 1 k  , it is like a low-pass, smoothing filter; if 1 k  , it is a high-pas filter.If 0 k  , it equals the average value.
When dealing with the distortion grating fringe of this subject, the smoothing filter can be used generally.In this paper, two smoothing filtering algorithms, namely average filtering and median filtering, are provided to determine the filtering algorithm based on the fringe image.
We use the MATLAB command of "filter2" and "medfilt2" with a 3 3  region to filter one of the gray code strip project to a background plane.Fig. 5 shows the results of using average filter and median filter.
Due to the field condition do not have strong interference and noise, the original image can reflect the real situation, filtering effect is not very significant.But it could eliminate the interference of unknown or pollution, to prepare for further image processing, therefore is an essential part of the image processing process.

Image Binarization
Image binarization is to transform gray image into only black (gray value is 0) and white (gray value is 255).Image threshold is a widely used image segmentation technology.Experts and scholars have carried out extensive research and extensive experiments on this subject, but there is still not a threshold selection method that can be effectively segmented for all images.Some threshold segmentation method can only be applied to a certain type of image, but the effect of other image segmentation is not ideal (8) .Suppose the original image ( , ) f x y , find a suitable gray value in ( , ) f x y as a threshold T with certain criteria, then the image ( , ) g x y after binarization can be expressed by the following form: T is binarization threshold, "1", "0" are used to represent the gray value of "white" and "black" respectively.Obviously, the key problem is how to select threshold T. Therefore, it is very important to choose T when doing this step.
From above, we can see that the key of threshold segmentation is the determination of threshold value.At present, the method of automatic selection of threshold T has simple segmentation method, optimal threshold method, maximum class variance method (Otsu's method), maximum entropy threshold and so on.Among these methods, the Otsu's method (9) has a better effect, and the computation is not too large.
In practical measurement, many uncertain factors such as non-uniform reflectivity of tested objects, background intensity, impact of defocus, and so on.So it is necessary to try different binarization method for the gray-code images.
In this paper, we project two more patterns with all black and all white to the test reference plane.Then we calculate the average intensity as binarization threshold T for each pixel between the black and white patterns.Finally we performed binarization operation for all the five captured gray-code image by the threshold T calculated by the average intensity.Moreover, we also have used Otsu's method to perform threshold segmentation treatment of stripe grayscale image pretreated with median filtering, and its threshold segmentation effect is shown in fig.6.

Morphology
Image morphological operation is for binary image based on mathematical morphology developed the image processing method of set theory method.Its main content is to design a set of transform (operation), the concept and algorithm used to describe the basic characteristics of the image.The purpose of morphology is mainly to obtain objects topology and structure information, through the interaction between objects and structure elements of some operations, get the more essential form.Its application in image processing, mainly: (1) using image basic morphological operation, observation and processing of image, so as to achieve the purpose of improving image quality.(2) The description and definition of various geometric parameters and characteristics of the image, such as area, perimeter, connectivity, granularity, skeleton, direction and so on.
Basic operation of mathematical morphology is erosion, in which pixels matching a given pattern are deleted from the image, and dilation, in which a small area about a pixel is set to a given pattern.There are several most common operations: erosion, dilation, opening and closing operation, thinning and coarsening, which are all morphological basis.With these operations and their combination can analyze and process image shape and structure, including image segmentation, feature extraction, edge detection, image enhancement and recovery, etc.
(a) Erosion In general, the erosion of image X by structuring element B can be defined as (10) { , | }  S is a binary image set after erosion in the formula, B is structural elements used for erosion, each element of structural elements is 0 or 1, which translated by ( , )   x y corresponds to a set of black pixels in X.That the result of an erosion is a subset of the original image seems clear enough; any pixels that do not match the pattern defined by the black pixels in the structuring element will not belong to the result.
According to the direction of erosion and structural elements for erosion, image erosion is divided into horizontal erosion, vertical erosion and omnidirectional erosion, etc.The structural elements are respectively like .
Because the subject adopts grating is horizontal stripe or vertical grating stripe, so the structural elements using horizontal or vertical pixels will be better.
(b) Dilation A dilation of the set X by the set is: Where S is a binary image set after dilation in the formula, X represents the image being operated on, and B is a second set of pixels, a shape that operates on the pixels of X to produce the result; the set B is called a structuring element, and its composition defines the nature of the specific dilation.
It can be seen from the fig.7 that the effect of operation by horizontal element is not much improvement.
According to its processing effect can analyze that we should adopt vertical element to do morphology operation in later processing.
(c) Opening The application of an erosion immediately followed by a dilation using the same structuring element is referred to as an opening operation.The name opening is a descriptive one, describing the observation that the operation tends to ''open'' small gaps or spaces between touching objects in an image.This effect is most easily observed when using the simple structuring element.
(d) Closing A closing is similar to an opening, except that the dilation is performed first, followed by an erosion using the same structuring element.If an opening creates small gaps in the image, a closing will fill them, or ''close'' the gaps.

Experiment Results
Decoding the already gray-code words after morphological operation, the phase order K can be achieved by the equation ( 4) shown in fig.9.The wrapped phase is calculated by three-step phase shift technique shown in fig.10.Finally, we can get the unwrapped phase which is shown in fig.11 for the test plane by using the equation ( 5).And we can see that the resulted unwrapped phase is smooth and without sharp phase jumps.

5.
This paper mainly research on the stripe image processing in structured light 3D profilometry system, through the stripe image read, filtering, binarization and stripe encoding.We used different threshold algorithm to binary the gray-code pattern and tried different structural elements to do image morphology.Finally we obtained an unwrapped reference phase with smooth and without sharp jumps by phase shift technique combined with gray-code method.
(a) Average filter image.(b) Median filter image.

Fig. 8
is the image after opening and closing operation respectively.It can be seen from the figure that the broken columns of small black continuous stripes are connected after operation.Morphological operation has big difference related to image processing by use of structural elements; Different structural elements have different effects, and the same structural elements have different effects on different images.This requires further analysis and experiments based on image morphology.

Fig. 9 .
Fig. 9.One cross section of the phase order.
(a) Morphology operation by vertical element.(b) Morphology operation by horizontal element.

Fig. 10 .
Fig. 10.One cross section of the wrapped phase.

Fig. 11 .
Fig. 11.Unwrapped phase of test plane by phase shift technique combined with gray-code method.