3D Face Modeling System Method Based on the Database of Facial Form Classification

Recent developing 3D printers are rapidly spreading in various fields such as industry, medical association, entertainment, and so on. Along the flow of such dissemination, many kind of 3D model creating tools are required for 3D printers. In particular, 3D CG software with superior operability is used efficiently when creating a 3D model of a complicated object such as a person and an animal, but it takes time to create the form in details. There are some methods of extracting feature points and performing simulation in the conventional 3D model creation method, but it needs devices with high performance for simulation (1-3) . On the other hand, when a model of a real person or an animal is created with a 3D printer, measurement systems of the shape with a laser or a plurality of cameras is necessary, which makes the system large and expensive. Therefore, we have developed a method that create a 3D model of a human face briefly without using high-performance equipment and deep learning, and actually create the 3D model. However, it still takes time to modify the face shape to resemble the model person. In this paper, we propose a novel 3D modeling system that is using facial photographs from four directions and is based on a database of facial outline feature classifications without using special measuring instrument and hardware.


Introduction
In recent years, as 3D printers are rapidly spreading due to improved performance and lower price, a technique for shaping a three-dimensional object by a 3D printer is attracting attention.In order to shape a three-dimensional object with a 3D printer, a 3D model of the object is required.Various kinds of 3D models are distributed and sold on websites and others.However, when creating things suitable for the purpose with 3D printers, basically the target of the real thing or a 3D model, should be designed by using 3D-CAD or 3D-CG software or should be scanned by 3D modeling devices.In the case of creating a 3D model of a actual complicated object such as a person, 3D-CG software is used normally.However, there is a problem that it takes time and need skilled operation technique to make up an object in details.
This project is aims to easily create 3D face model of an actual person for outputting with 3D printer.Therefore, we propose a method to easily create from a photograph in creating a 3D model of a person face.In this system, we do not use any measurement systems of the shape with a laser or a plurality of cameras, but only utilize the information obtained from four images taken from four directions which are the front, both sides, and the rear.First, the outline of a person face is classified by analyzing the front image, and a 3D model close to that form is selected from the database of face form classification.Second, to obtain a required data for creating a 3D face model, this method extracts each facial part by detecting automatically the edge and the hue data of HSV color space.Such as these, data necessary for creating a 3D face model is automatically extracted using image processing such as cascade classifiers of Haar-like feature, hue component in HSV color space, and edge detection.
Finally, the 3D created model is modified by comparing the photographs in four directions obtained from the model with the original images.We investigated the error on the difference between the image measurement value of the proposed method and the manual measurement value was verified using several face images.

3D model creation system from images 2.1 Procedure for creating 3D model
When we create a 3D face model from a face picture, we need to measure the length and position of the face parts eyes, ears, nose, mouth.Therefore, we developed a method for preparing accurate 3D data by preparing 3D data of face parts in advance and pasting 3D data of face parts from measured values.Fig. 1 shows the three-dimensional model creation procedure of this method, and Fig. 2 shows the part to measure the face.

Extraction of hue in HSV color space
HSV color space is color spaces composed of three components of hue, saturation, and value(lightness) (4)(5)(6) .It can extract specific colors unlike the others color space.Fig. 3 shows an example actually extracting human skin color and hair color by this color space.As shown in Fig. 3, it can appropriately extract the outline of the skin color portion and the hair portion.

Detection of facial features using classifiers
Classifiers is a learning feature such as density and color of image by machine learning, adapts the learned one to the target image, and detects the feature from the images (7)(8)(9)(10)(11)(12) .Fig. 4 shows an image in which a classifier is applied.In this paper, we use the features of face (eyes, nose, mouth, ears) by using a classifier of Haar-like feature (image contrast difference) that exist standard in OpenCV which is an image processing library (13)(14)(15) .In the extraction of the hue in the HSV color space, there is a possibility of extracting colors in areas other than the face and hair.In addition, although extraction of a face using a classifier can detect the position of a rough face or any other facial part, accurate length and position can not be extracted.Therefore, we devised a method to automatically measure the face length from the photograph by combining these two methods.Fig. 5 shows a method of measuring the length of a face using the classifier and the HSV color space.We can automatically measure the face length without extracting the color in the range other than the face and hair by detecting he face by the classifier and extracting the hue in the HSV color space only around the detected position (16)(17)(18)(19)(20) .

Edge detection of face outline
It is impossible to decide the boundary between the jaw and the neck by using the HSV color space, because the color difference of the contour part of the jaw contour is ambiguous.Even if it is possible, it is difficult to obtain the perfect full contour of the jaw by an ordinary edge detection operator.However, as shown in Fig. 6, the contour of a face is able to be approximately extracted using the Fast Local Laplacian Filtering method.By emphasizing the contrast of the edge detected with this filter, the boundary between the jaw and neck can also be detected as an edge (21) .

Procedure to measure the length of facial parts
In order to resemble a face of an actual person, it is necessary to resemble not only the position of the part but also the shape of the face.In the 3D model making up method proposed in this paper, it is necessary to finally modify the features of the face shape.So, we classified the face into several kinds and thought that it would be easy to modify the 3D model by selecting as a basic model with its features.And we devised a method to classify it from the front image into several kinds of face types.Fig. 7 shows the measurement points necessary for classifying the face type.To classify faces, we need to measure the following seven points.
(1) Length from the center of a face to the edge at the height of eye (2) Length from the center to the edge of the face at the height of nose (3) Length from the center of the face to the edge at the height of mouth (4) Length from the center of the mouth to chin (5) Length from the edge of the mouth to bottom (6) Vertical length of face (7) Length of the side of face

3D created model
Fig. 9 shows an example of a 3D model actually created from the photograph the proposed method.In order to Fig. 9.
Original photo and 3D model actually created.

Measurement of face length
As shown in Fig. 3 and 4, we measured the height and width of the face utilizing the method of detecting a face with the classifier and the method of extracting the skin color and hair color of a person.In addition, the difference between automatically measured value and the manually measured value, and the error rate based on manual measurement are also found.Fig. 10 shows ten photos we used.In this case, since the color is extracted by the HSV color space, we selected images with uniform background.
Table 1 shows the comparison results of measured values of the length and the width of a face.The accuracy of the classifier of face was high and it was able to detect the rough position of the face for all the images.Consequently, the difference between the automatic measurement results and the manual measurement results was small and the error rate was all 3% or less.For photos with uniform backgrounds, it is possible to automatically measure the face length by the developed method.

Setting a base model of adaptive face
In order to verify whether the length necessary for classification of face type can be measured, the length from (1) to (5) in Fig. 7 was measured manually and automatically, and the error rate was calculated.Ten images shown in Fig. 9 were used as verification images.Table 2 shows the measurement results of the lengths of the measurement points (1) to (5) in Fig. 7.The average of the error rate of each image was 6.11 [%] at the maximum, but the other error rate fell below 5 [%].
From the measurement results in Table 2, since the error rate is within the allowable range, the face image of Fig. 10 was classified into four types using the face classification method of Fig. 8. Table 3 shows the results of verifying face classification.Comparing the measurement method of Fig. 8 with the classification result by itself, the results agreed with all 10 images.Therefore, it can be said that the classification method of Fig. 8 is effective.

Comparison of 3D model creation methods
In order to demonstrate the effectiveness of the proposed method, we compared how much difference was created between the 3D modeling method proposed at the present time and the 3D model creation with only the manual work using the 3D modeling software.The processing of the method proposed in this paper and the software used are shown below.
(a) Measurement the length and the position of facial parts The length shown in Fig. 5 is measured by the proposed method.HSV color space and classifiers were implemented using a programming language called python.
(b) Paste the facial parts Each specified facial parts using the length obtained in Table 1.Comparison of measurements of face.
the above measurement (a) are adapted to the based face model.In this process, free software called Blender that is able to be managed with Python is used.(c) Modification of the shape of the face Finally, it must fine-tune the shape of the made face to mimic the actual face of a photo.The software for this facial modification processing is used ZBrush of Pixologic, Inc.
As a method to compare with the proposed method, it is compared with the method of creating 3D model by manual work only.The method of making by only manual work was done using a method of modifying the shape of the face only by operating ZBrush.
Table 4 shows the approximate required time for each process using the proposed method and that for creating only by ZBrush.In the case of only manual work, it takes a considerable amount of time to first create the shape of the face to some extent and add some modifications.On the other hand, the proposed method uses the program to measure the face parts, and can create the shape of the face simply by pasting the face parts on the selected face from face classified database.And the shape of the face is adjusted to some extent has been done.Since the position of the face part is almost equal to the original position of the photograph, the processing for alignment of each part is small.Therefore, we can make up a 3D model face much faster than creation with ZBrush alone.It is just a third of manual comparing with ZBrush alone, namely, at a much faster rate.

Conclusions
We devised a novel method for efficiently creating 3D face modeling system, which utilizes face database made by classifying faces into four categories.And we do not use any high-performance measuring equipment such as a laser or a plurality of cameras, but only utilize the information obtained from four face images taken from four directions.Both the face measurement data required for classification and the measurement result of the contour of the face were close to the manual measurement result, and the classification result also showed the same result as our own classification result.
For future work, we improve this method so that we can automatically measure the length in the same way by using the photos taken from the left and right.Also, we aim to add the minute model of face classified from the images taken from both sides.We are considering improving face modeling system with higher accuracy by using database based on this information.

Fig. 8
Fig.8shows the flow chart of classifying a face.We grasp the rough shape of the face from the above measurement part and classify it from the aspect ratio of the face into the above four kinds.Face types are set to the following four types.

Table 2 .
The lengths of the measurement points.