3 D Face Modeling base on the Efficient Detecting of Facial Feature Parts

Various kind of 3D model tools for creating are required for 3D printers popular rapidly in recent years. In particular, 3D CG software is used when creating a 3D model of a complicated object such as a person, but it takes time to create that details. There is a method of extracting feature points and performing simulation in the conventional 3D model creation method, but we need a device with high performance for simulation. In recent years, there are methods using deep learning. However, in order to improve accuracy, it is necessary to learn a large amount of data. In this paper, we propose a method to create a 3D model of a human face briefly without using high-performance equipment and deep learning, and actually create the 3D model. In addition, we propose a method to measure automatically data necessary for creating a 3D face model using image prosessing such as cascade classifiers of Haarlike feature, extraction of hue component in HSV color space for extracting hair and skin color, and edge detection. Finally, we investigated the error on the measurement results of the face images by the proposed method.


Introduction
In recent years, as 3D printers are rapidly spreading, a technique for shaping a three-dimensional object by a 3D printer is attracting attention.In order to shape a threedimensional object with a 3D printer, a 3D model of the object is required.Various kinds of 3D models are distributed and sold on websites, etc.However, when creating things suitable for the purpose with 3D printers, basically the target of the real thing or a 3D model, should be designed by using 3D-CAD or 3D-CG software or should be scanned by 3D modeling devices.In the case of creating a 3D model of a complicated normally object such as a person, 3D-CG software is used.However, there is a problem that it takes time to create details.
Therefore, we propose a method to easily create from a photograph in creating a 3D model of a person face.The proposed method aims to easily create a 3D face model of a person and, in order to obtain a required data for creating a 3D face model, this method extracts each facial part by detecting automatically the edge and the hue data of HSV color space.Furthermore, the difference between the image measurement value of the proposed method and the manual measurement value was verified using several face photographs.

Procedure for creating 3D model
When creating a 3D face model from a face picture, we need to measure the length and position of the face parts eyes, ears, nose, mouth.Therefore, we developed a method for preparing accurate 3D data by preliminarily creating 3D data Fig. 1. 3D model creation system from images.

Paste each face parts
Measure the length and position of the part of the face Modify the shape of the face of the facial part and pasting 3D data of the corrected facial part from the measured value.Fig. 1 shows the 3D model creation procedure of this method, and Fig. 2 shows the part where we measure the face.

Extraction of hue in HSV color space
HSV color space is one of color spaces composed of three components of hue, saturation, and value(lightness).It is easy to extract specific colors unlike the RGB color space.Fig. 3 shows an example of actually extracting human skin color and hair color.by this color space.As shown in Fig. 3, it is possible to appropriately extract the outline of the skin color portion and the hair portion.

Detection of facial features using classifiers
Classifiers is a learning feature such as density and color of image by machine learning, adapts the learned one to the target image, and detects the feature from the images.Fig. 4 shows an example in which a classifier is applied to the image.In this paper, we use the features of face (eyes, nose, mouth, ears) by using a classifier of Haar-like feature (image contrast difference) that exist standard in OpenCV which is an image processing library (3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14)(15) .

Procedure to measure the length of facial parts
In the extraction of the hue in the HSV color space, there is a possibility of extracting colors in areas other than the face and hair.In addition, although extraction of a face using a classifier can detect the position of a rough face or any other facial part, accurate length and position can not be extracted.Therefore, we devised a method to automatically measure the face length from the photograph by combining these two methods.Fig. 5 shows the procedure for automatically measuring the face length.We can automatically measure the face length without extracting the color in the range other than the face and hair by detecting the face by the classifier and extracting the hue in the HSV color space only around the detected position (16)(17)(18)(19)(20) .

Procedure to measure the length of facial parts
Since we can detect facial parts (eyes, nose, mouth) by using the classifier, we devised the method to automatically measure the length as well as the procedure to automatically measure the face length.Fig. 6 shows the procedure for automatically measuring eye length.The face is detected by the classifier and the facial parts (eye, nose, mouth) were accurately detected from the range of the face.We can measure the length by performing edge detection in this detected range.

Evaluation of created system
3.1 3D created model Fig. 7 shows an example of a 3D model actually created by this method from the photograph.In order to create a 3D model, we used a total of four pictures of the front, the profile viewed from the left and right, and the back.

Measurement of face
As shown in Figures 3 and 4, we measured the length and width of the face using the method of detecting the face with the classifier and the method of extracting the color of person skin and hair color.In addition, measurements are made manually, and the difference between the automatically measured value and the manually measured value and the error rate based on manual measurement are also found.Fig. 8 shows six photos we used.In this case, since the color is extracted by the HSV color space, we selected images with uniform background.Table 1 shows the comparison of measured values of length and width of the face.The accuracy of the classifier was high and it was able to detect the rough position of the face for all the images.Also, the difference between the automatic measurement results and the manual measurement results was small and the error rate was all 3% or less.For photos with uniform backgrounds, it is possible to automatically measure the face length by the invented method.

Measurement of facial parts
Next, we measured the facial parts (eye, nose, mouth).A approximate position is detected from the images by using the classifier in any facial parts.From the approximate obtained range, the length each facial parts was measured using edge detection.Regarding the part of the face, we calculated the error between the manual measurements and the automatical measurements by using the images of Fig. 8.
Tables 2, 3, and 4 show the comparisons of the measured values of the lengths (vertical and horizontal) of eyes, nose and mouth measured automatically and manually.Regarding the eye value, it is a value obtained by averaging the measured values of both eyes.From Tables 2 to 4, although the eye measurement the result of shows that the difference in the number of pixels is several pixels, the error rate becomes large.The difference between the automatic measurement result and the error rate of the measured value by the method devised as manual measurement of the nose were relatively small values.In the case of mouth measurement, the difference between the proposed method and the manual measurement result was higher in many cases Fig. 10.Eye edge detection result.than in other parts.As a result, the error rate of the measured value also showed a high value.Fig. 9 shows an example in which an eye is detected by a classifier and Fig. 10 shows and edge detection is performed within the detected range.Fig. 11 shows the detection result of the mouth of image 1 and Fig. 12 shows and edge detection is performed within the detected range.The part of mouth can not be accurately detected by the classifier and the value of the difference in measurement becomes large.As a cause of this, we can be inferred that Haar-like features were not be able to obtained because the shape of the lip changes due to facial expression of person.Besides, although the mouth can be accurately detected, there were also same images in which the difference between the measured values became large.
Fig. 12 shows the detection result of the mouth of Image 5 and Fig. 12 shows the result of edge detection.As shown in Fig. 13, The mouth of Image 5 can be detected.However, it was considered that the color of the lip and the skin are very close, and the contour due to the edge detection did not appear so much and so the edge could not be accurately detected.

Comparison of 3D model creation methods
According to the time to rebuild 3D face model, we compared how much difference occurs for creation with the 3D model creation method proposed this time and the 3D model creation with only manual work using the 3D modeling software.The processing of the proposed method in this paper and the software used for the each processing are shown below.
(a) Measurement of the length and position of facial parts The length and position of each part representing facial features shown in Figures 5 and Fig. 6 are measured by the proposed method.Python as the programming language is used to detect and calculate these peculiarities.
(b) Paste the facial parts Next, each specified facial parts using the length measured in the above measurement (a) are applied to the basic face model.This procedure uses free software called Blender that is able to be controlled with Python.
(c) Modification of the shape of the face Finally, it is necessary to finely modify the shape of the obtained face to resemble the face of the original person.The software for this facial modification processing is used ZBrush of Pixologic, Inc.
As a method to compare with our proposed method, a means of creating a 3D model only by manual work was verified.The verification was carried out using a method of creating the shape of the face only by operations of ZBrush.
Table 5 shows the approximate required time for each process using the proposed method and that for creating only by ZBrush.In the case of only manual work, it is common to first create the shape of the face to some extent and then make minor modifications a little by little.On the other hand, in the case of the proposed method, as the measurements of the facial parts are obtained by using the program, the shape of the face is able to be created by merely pasting the facial parts based on the measured data and the shape of the face is adjusted to some extent.Furthermore, since the positions of the facial parts are almost equal to the original positions of the photograph, the processing for alignment of each part are slight.So it is possible to create a 3D model of the face much faster than in the case of only ZBrush.Therefore, it is possible to create a 3D model of a face at a much faster speed than in the case of ZBrush alone, requiring only about one third of manual work.

Conclusions
We devised a novel method for efficiently creating a 3D model of a person and actually created a person 3D model using that method.With respect to the part of the face, the eyes and nose could automatically measure values are very close to the manually measured values.However, it was not able to automatically obtain the mouth length with an accurate length due to the accuracy of the classifier of Haarlike features and the relation of edge detection.
For future work, we improve this method to be able to automatically measure the length in the same way, using the photographs in directions other than the front photo.Besides,

Table 4 .
Comparison of measurements of mouth length.

Table 3 .
Comparison of measurements of nose length.

Table 5 .
Approximate time each process takes.order to improve accuracy of measurement of the mouth length which was not accurate this time, we are going to try classifiers based on features other than Haar-like features and try other edge detection methods. in