Smooth and Texturing Blending Framework for Marginal Area of Facial Prosthesis

The purpose of this paper is to describe how to model the smooth and texturing blending surfaces for marginal area of facial prosthesis adjoined to surrounding tissues based on the characteristic of areal scan data. First in the geometry modelling stage, the implicit function was used to segment 3D models into parts and re-triangulate the resulting gap and after several iterations of the umbrella operator, the smooth transition between facial prosthesis and surrounding tissues can be achieved. Then in the texture mapping stage, a seamless textured surface can be produced quickly based on the method of solving the Poisson equation for the colour field that best fits the colour gradients with a grid search structure. Clinical application showed that the resulted facial prosthesis can fit the patient’s appearance well.


Introduction
Facial prostheses can restore the physical appearance of patients.There are two purposes for constructing the smooth and texturing blending surfaces for marginal area between facial prosthesis and surrounding tissues.First, a good facial prosthesis should fit the patient's appearance well, make the patient feel comfortable and maintain its aesthetic.Second, If there is a lack of realistically visualization for a facial prosthesis, it is difficult for the patient to communicate to the surgeon to identify his desired ultimate appearance.This paper is organized as follows.Section 2 presents a thorough review of the literatures on constructing the smooth and texturing blending surfaces; Section 3 describes the composition of this proposed system and its detailed method; Section 4 presents an example of the applications by the proposed approach.Section 5 provides the summary and some extensions of our study.
The process of geometry modelling can be divided into two stages: how to construct the smooth blending surfaces; how to realize the texture mapping in the blending surfaces.

Construct smooth blending surface
(1) Zipper adjacent meshes Greg Turk (1) zippered together adjacent meshes to form a continuous surface that correctly captures, The solution consists of three steps: Remove overlapping portions of the meshes;Clip one mesh against another; Remove the small triangles introduced during clipping.
(2) Volumetric Diffusion James Davis (2) began by constructing a signed distance function, the zero set of which defines the surface.Initially, this function is defined only in the vicinity of observed surfaces.Then he applied a diffusion process to extend this function through the volume until its zero set bridges whatever holes may be present.
(3) Establish the correspondence on two boundaries Hongbo Fu (3) established the correspondence between the source merging boundary and the target merging boundary.Then zip the target merging boundary and the deformed source merging boundary.

Texture mapping
For the texture mapping, many scholars had done the in-depth researches, which can be divided into five categories: (1) Based on feature points R.Kurazum (4) looked for the corresponding features from the extracted edges of three-dimensional model surface.Based on the feature points, the texture can be mapped into the model surface, however, the reliability of the feature points may influence the texture mapping result.
(2) Based on block combination Zhao (5) proposed a block-based texture mapping method in the human face synthesis.In this method, the model is divided into some parts first and each part is mapped with the clearest and non-redundant texture.The model then is rendered one time block by block, at last the virtual muscles are embedded into the model and the realistic face expressions are synthesized.The benefit of this method is only to deal with the transition between blocks, and the drawback is that it requires three orthogonal face images, such conditions are not always able to be satisfied; (3) Search an optimized stitching path Li Xiaolan (6) proposed a method for searching an optimized stitching path in the overlapped area of partial texture maps.This approach has the advantage in the elimination of joints at the same time to maximize the preservation of the original image details.Its disadvantage is that the cylindrical shape expansion method is only suitable for relatively simple models.(4) Set Weights in overlap area Ding Yabin (7) put forward a method of adding the fine texture on 3D surface.A weight function is introduced into blending textures in the overlap area.But how to set the weight value is also affect the final quality of texture mapping.
(5) Optimized Colour gradient field Su (8) and Levin (9) put the texture mapping into image gradient field in the 2D image domain.Ming Chuang in Johns Hopkins University (10) extended the method to the triangle mesh.He first obtained a colour gradient field describing the local change in the texture over the mesh by pulling colour gradients from the closest views, and then he got a seamlessly textured surface produced by solving the Poisson equation for the colour field that best fits these colour gradients.
Above discussion shows that the existing methods still have many shortcomings.In order to design more satisfying 3D smooth and texturing blending surfaces, this paper describes a new processing system, which major features are in the following: Since both range image and its corresponding texture from one direction could be acquired by areal 3D scanner simultaneously, the texture can be integrated into the range image exactly.In the geometry modelling stage, the algorithm first use the implicit function to segment 3D models into parts, re-triangulating the resulting gap and apply several iterations of the umbrella operator for the smoothness.The smooth transition between facial prosthesis and surrounding tissues can be achieved.In the texture mapping stage, based on grid search structure, the method of solving the Poisson equation for the colour field that best fits the colour gradients can produce a seamlessly textured surface quickly.

Acquire the range image with exact texture
As shown in Fig. 1, 2D patterns is projected by an areal 3D scanner (11)(12) on an object and the relevant scene is observed by a camera.The 3D coordinates of measurement points on surfaces is calculated.According to the measurement principles of areal 3D scanners, a measurement point is generated for a pixel of the imaging sensor inside the 3D scanner.Therefore, by the image row and column of each point, one can get the corresponding colour value.By using this property, each measurement point consists of eight components , , , , , , , x y z R G B pc pr , as shown in Fig. 2. ,, x y z are 3D coordinates of each point, pc is the pixel coordinates in the horizontal direction and pr is the pixel values in the vertical direction.
,, R G B are the colour of the measurement point.3D coordinate points and texture coordinates can apply the same transformation matrix, so as to ensure the accuracy of texture mapping.( , , ) For example, a sphere S of radius R and centered at the origin can be described by the equation ( , , ) the equation describes S exactly, and when ( , , )>0 P X Y Z , we describe a sphere that lies outside the sphere S. We use implicit functions to computing the clicked points on the screen.Using the subtract, we can complete the operation of pruning.
Step2 Re-triangulating the resulting gap After pruning, there is a gap between the target merging boundary and the source merging boundary.We re-triangulating the resulting gap to form a seam sheet.

Step3 Improve smoothness
To improve smoothness at the seam, we apply several iterations of the umbrella operator to change the irregular triangles to be regularized uniformly.A popular surface fairing measure is the so-called total curvature functional k and min k are the surface principal curvatures, and dA is the surface area element.The total curvature can be minimized by applying the following modified bi-umbrella operations (3,13) :  3),( 4)until ( ) 0 Tp ,a smooth mesh result can be obtained.

Texture mapping
The idea actually is transforming the problem of texture mapping into an optimization problem of defining the color value of each point on the triangular mesh.
Step1 A group of the triangular meshes with texture (refer as: View) are acquired, as shown in Fig. 4(a) and (b); Step2 Without considering the texture information, these views are processed by Multi-view registration and integration, as shown in Fig. 4(c); Step3 After the integrated mesh is processed by hole filling, and the resulting data is called as base mesh, as shown in Fig. 4(d); Step4 Construct the initial color field for the base mesh; Construct the initial color field for the base mesh, That is, determine an initial color value on each point of the base mesh. 1) For each point B in the base mesh, its index value B I is calculated; 2) If the index value of the point in the views is the same as B I , we collect these points as the point series i D ; 3) The color of the nearest point in point series i Step5 Get the optimized color field for the base mesh; Solving the Poisson equation to minimize 22 a f g f V     , we obtain the color field that best fits these gradients for the base mesh.In above equation, g is the color fields of various views; V is the color gradient fields of various views; a is the weighting term determining the importance of the value-interpolation constraints and f is the optimized color field for the base mesh, as shown in Fig. 4(d).
Step6 The optimized color field is applied to the base mesh, and finally we get the geometric model with exact texture.
The first difference between our algorithm and Ming Chuang's is to realize texture hole filling.First in Step3 the geometric model is processed by hole filling, then in Step4, as for the filled points, the search for the color of nearest point in the views is done as its initial color.After doing the optimization in Step5, the texture holes filling is achieved to ensure color smooth transition between the hole and the surrounding area; the second difference is to use the grid search instead of the original KD tree.Using the grid search structure, the closest point computation has a complexity of ) (Np O while using KD tree, the complexity is ) log ( Nx Np O (14) .

System Application
As shown in Fig. 5,this prosthesis was fabricated for a 42-year-old man who had surgery for carcinoma of the right maxillofacial face.

4.Conclusions
In this paper we developed a processing system for smooth and texturing blending surfaces for marginal area of facial prosthesis adjoined to surrounding tissues based on above techniques.We used the areal 3D scanner to acquire both range image and its corresponding texture from one direction, the texture can be integrated into the range image exactly.In the geometry modelling stage, we combined implicit function segment method, triangulation with umbrella operator smoothing method to get the smooth transition between facial prosthesis and surrounding tissues.In the texture mapping stage, we realized texture hole filling and used grid search structure to speed up the method of solving the Poisson equation for the colour field that best fits the colour gradients.All in all, Case studies showed that this system can be used effectively for the fabrication of facial prostheses.In the near future, we will try to research the influence of age,sex,race on the geometry modelling and texture mapping for the marginal area of facial prosthesis adjoined to surrounding tissues.

Fig. 1 .
Fig. 1.One to one relationship between the image pixel and the 3D data

Fig. 2 .
Fig.2.Customized structure of point 2.2 Construct smooth blending surface Step1 The operation of pruning We use the powerful modeling technique of implicit functions.Implicit functions have the form: sum of the two areas that share the common edge i PT and  is a step size for smooth.By iteratively Calculating (2),(

pN
is the point number of the fixed mesh andx N is the point number of the moving mesh for aligning to the fixed mesh.For large data sets, most of the time is spent for closest point computation, so use the grid search can speed up the search procedure.
Fig. 4. Texture mapping based on the color gradient field Fig. 5. Constructing maxillofacial prosthesisFollowing the workflow, first the alignment algorithm was used to construct the mid-plane since maxillofacial prosthesis is characterized by symmetrical features.Then the healthy maxillofacial region was mirrored along the mid-plane to the defect area.Using the clicked points to pruning the data of the origin face and the mirrored ,the result can be seen in Fig.5(a).Then using the algorithm, we got the merged data by constructing the smooth blending surface in Fig.5(b).Afterwards, the texture mapping operation was performed in Fig.5(c).And in the Fig.5(d), Complete data of the patient's face were generated.we can find the vast majority of area is green in the surrounding tissues of facial prosthesis.Therefore, we can conclude that the effect of the processing system can meet the actual requirement.