A novel face detection algorithm for occluded images based on the geometrical model of nose point extraction

Face detection is one of the developing trends in the field of biometrics. A lot of face detection algorithm has been developed, but still the face detection rate haven’t reached to 100%. It is due to the presence of various environmental conditions on an image. The occluded facial image is also one of the conditions for high performance face detection. A face is occluded if some area of the face is hidden behind on an object like a mask, sunglass, hand or we can say the face is visible partially. In this paper, we have developed a system that at first extracts the eyes, nose and mouth also known as the facial features by using the skin color model and the Haar cascades. The geometrical model is then applied to the extracted features using the three triangle method that calculates the height, width, distance and the position of the extracted features. Our proposed face detector then detects the presence of the face in an image even with an occlusion. The combination of the three triangle method with UKF and Haar cascade classifier represents the novelty of our proposed system. We performed our experiments on various images under different conditions, using different facial databases to clarify the effectiveness of our proposed method.


Introduction
The physiological biometric identifier that is widely used in person recognition is known as Face.In-fact, it is the face that attracts our attention the most.Human face detection is the first step of any face processing systems, computer vision and computational image analysis.In the recent years, the face detection has shown tremendous progress in this area, with emphasis on such application as biometric analysis, human machine interaction, robotics vision, image and video based coding and surveillance system.To process or step in the face recognition or face identification system, the face detection process has to be completed at the first step.A lot of face detection algorithm has been developed, but the face detection hasn't reached to the 100%.It is due to the presence of the different environmental conditions in an image.The environmental conditions for the face detection are image noise, illumination variant images, occlusion, pose, scale, expression etc.In our previous works we developed a face detection system (1) ,that works under the noisy images and the system (2) works on illuminant variant images.We were able to solve these conditions, but still some environmental conditions are left.
Our current research is based on the occlusion Condition of face images.A face is occluded if some area of the face is hidden behind on an object like a mask, sunglass, hand or we can say the face is visible partially.Facial occlusions can degrade the performance of the face detection system.To overcome up with these problems, face detection approaches have been classified into knowledge-based (3) , feature invariant (4) , template matching (5) and appearancebased (6) methods as a global approach.In recent years, the component based approaches have produced better results than global approaches.The component based approaches can be used to construct detectors that can handle partial occlusions (7) .Similarly, Naser and Soderstrom (8) have used PCA (Principal Component Analysis) for occluded image reconstruction.The geometric based approaches (9) uses width, length and angle information of the facial points for face detection.
In our current research, we are developing a face detection system that detects the face under different occluded conditions.We are using the modified skin color detector (1) for extracting the face region from the image.The face is then cropped for the feature extraction process.Then, we extract the nose point, right and left eyes and mouth from the cropped face image by applying the geometrical model for locating the position of the facial features.Different from the other process of geometrical model, we generate three triangles by using the detected eyes and nose as the first, detected nose and the mouth as the second and detected eyes and mouth as a third triangle.The formation of these three triangles helps us to obtain the position of the facial features so whenever a single triangle can be developed from the extracted the facial features, the face can be detected even in different conditions of occlusion.Finally, the face is detected by using the modified Haar cascade classifier after clearing the film grain noise by using the UKF (1) .The three triangle method combined with the UKF, Haar cascade classifier and skin color detector reflects the novelty of our paper.We perform various experiments on different types of occluded images for clarifying the effectiveness of our research.We have used the CMU-MIT (10) , INRIA (11) and FDDB datasets (12) for our experimentation.

Proposed algorithm 2.1 Structure of the Proposed algorithm
The structure of our proposed algorithm is shown by the flow diagram in figure 1.The image is first passed to the modified skin color detector (1) , where the facial region is extracted.Once the facial region is extracted, the face is cropped and the facial features which include the nose, both eyes, and the mouth is extracted by using the geometrical model.The geometrical model is applied to the extracted facial features to locate the position of the eyes, nose and mouth by calculating the height, width, distance and angle between the face components.The geometrical model creates three triangles by locating the facial components of an image.The image is then processed to the UKF for removing the noise from the image.Finally, the image is passed to the modified Haar cascade classifier (1) for final face detection.

Modified skin color detector
The modified skin color detector is developed from the idea of skin color detector (13) .In case of the skin color detector, the skin color regions are detected, but the pixels similar to the skin color are also detected together as a false positives as shown in figure 2b.These false positives are known as noise.To remove these noises, we developed a modified skin color detector by adding a low pass filter, Sobel edge detector and the color region detector to improve the effectivity of the system.The figure 2c and 2d

Face cropping
Once the facial region is detected by using the modified skin color detector, the faces are cropped from hair to chin and from right ear to left ear as shown in figure 3.In this process, we separate the skin part of the face from those of the non-face part such as arms, hands and shoulders.The face cropping process is taken as a preprocessing for the facial feature extraction.The size of the image is standardized as having a maximum of 200 pixels in width in order to minimize the variation due to the head tilt, rotation and shift of the image.

Nose point extraction
The nose point extraction is the process of detecting the nose point from the given facial image.To extract the nose point, we need to detect the nose region at first.We extract the noise region by using the Haar cascades classifier as shown in figure 4a.The defined width and height of the noise is 0.6d, where d represents the distance between two eyes as shown in figure 4b.The height and width of the nose is calculated and determined by using the geometrical model as shown in figure 4b.The nose is close to the center of the face so that the pixels that are closer to the centroid should be extracted using the morphological operations of the geometric model.The algorithm for extracting the nose point is mentioned in the following steps.
1. Translate the co-ordinate to the centroid of the image.
2. Shift the position to the desired location.
3. Change the size of an object by scaling the image to the fixed size of square dimensions as shown in figure 5.
4. Taking the centroid, find the closest region of the nose point by using morphological processing.
The morphological processing is used to extract the tip point of the nose.The largest connected component which is closest to the centroid is the point which is the tip of the nose.The figure 5 reflects the detection of the nose point or the tip of the nose.

Right eye and Left eye extraction
In our previous works, we have used modified Haar cascade classifier for detecting the presence of the two eyes in a face.In our current process, for making the occlusion 2. Find the gray image of the face.
3. Find the Max-Index on which the maximum difference between two adjacent intensities in the gray image of the face lies.

Mouth / Lip extraction
The mouth / lip extraction is the process of extracting or detecting the region of the mouth in a face.The lips indicate the position of the mouth.Referring to the previous researches, there are lack of mouth detection results specifically dealing with the thick beard, mustache and open mouth.
In our process, we are using the geometrical calculation for detecting or extracting the lip region as shown in figure 8.The algorithm for detecting the lip region is mentioned below.
1. We can predict the lip region as

The Unscented Kalman filter
In our proposed system, we have used the unscented Kalman filter (UKF) for removing the film grain noise from an image in our process.The UKF is a method for calculating the statistics of a random variable which undergoes a nonlinear transformation.We have used the two dimensional UKF for image restoration (14) .The film grain noise in an image is determined by the least squares method using the state equation and the measurement equation.The image is scanned by using the non-symmetric half plane model (NSHP model).The image is scanned from the top-left pixel and scan the first column from top to the bottom.This process is applied for several times to achieve a better result with the noise filtered image.A noise filtered image helps to increase the face detection rate.

Modified Haar cascade classifier
In our current research, we are using the Haar cascade classifier for detecting the multiple faces or the final faces in an image.We have modified the Haar cascade classifier by combining with the clustering algorithm (15) for clearing the false positives from the images.We choose clustering algorithm for our process because the algorithm can be used for basic pixel level to feature level for the classification of the facial images.As our current research is based on the occluded faces, this process is effective for working with the geometrical calculation of the facial features.The figure 10 shows the detected faces with the major facial components or facial features.

Geometrical Model
We are able to find the nose point, eyes and lips as a facial components by using our proposed algorithm.To make our algorithm work together for detecting the position of the facial components even if they are occluded, we use a flexible component based face detection based on their geometrical relations (16) as shown in figure 11a.The relationship between the facial features can be presented by their relative distance and positions.In a frontal face image, the eyes, nose and the mouth can be represented by using three triangles method.Where the eye comes at the top, nose in the middle and the lips at the bottom.The figure 11b reflects the detection of face and facial components along with their geometrical relations.This process is useful for detecting the occluded facial component of the facial images as shown in figure 12.

Experimental results and Evaluation
In order to validate our proposed algorithm, we developed a C programming code for implementing our proposed system.We performed experiments using the PC (i7-3770 CPU, 8GB main memory and Windows 8.1 64 bit) machine.Our system detects the face in a yellow box and the facial components in a white box.We performed some experiments on different types of occluded images from CMU-MIT database, FDDB database and INRIA databases along with some images taken by ourselves under different occlusion conditions.The images that were not detected by our previous works due to the occlusion conditions are now detected by using our current proposed method.We are using only the occluded facial images in our current experimentation.The experimental results of the occluded images by using our proposed method is shown in figure 13.
In the figure 13, a and b is taken from the CMU-MIT database, c, d, e, f and h from the FDDB database and g from the INRIA graz-01 database.In all of these figures the faces partially visible or occluded are detected except the figure a..In our proposed system, if two of the facial components are visible, then our system can detect the face with the geometrical relation and calculation of the position of the face, but in figure a, the facial components are unclear for the undetected face, so one face hasn't been detected in this image.The faces with two or more than two visible facial components are detected properly.The same conditions are satisfied with the rest of the images shown in figure 13.These images were not satisfied by our previous works (1) .
We performed benchmarking test based on the FDDB database and draw ROC curves to evaluate the position of our proposed algorithm.The ROC curve in figure 14 shows the face detection rate of our proposed algorithm compared with our previous works (1) XZYZ method (16) and viola Jones face detector (16) .The ROC curve is drawn from the first 2000 images by referring the ROC curves shown in Reference 16.The proposed algorithm shows the highest face detection rate compared to our previous work, XZYZ method and the Viola Jones face detector.The ROC curve clearly reflects the increase in the face detection rate on FDDB database after clearing the condition of occlusion.

Discussion
We performed various experiments and the benchmark tests under the occlusion condition by using our proposed algorithm.The proposed algorithm can detect the faces under different occlusion conditions.We started this research as the development of our previous face detector (1) , we are able to clear the environmental condition of Occlusion by using this proposed method.
The three triangle geometrical calculation process combined with our previous face detector, helps us to detect the facial components and face from the image by calculating the distance, height and width of the facial components.The three triangle methods detect either of the two facial components and calculates the position of the other occluded facial features and finds out the actual position of the face.The presence or absence of the structural components in a face such as the faces with beard, masks, and glasses on the eyes, hidden facial parts and partially visible faces are detected by our proposed algorithm.
The combination of the three triangle method geometrical calculation using noise point method and the Unscented Kalman filter reflects the novelty of our current research.

Conclusion
In this paper, we presented a novel face detection algorithm for occluded images based on the geometrical model of nose point extraction.The face and the facial components were detected by using the method of noise point extraction.
This process helped us to clear the environmental condition of occlusion for face detection and helped us to increase the face detection rate on FDDB database when compared to our previous works.The face detection rate of our proposed algorithm by using FDDB database was 84.1%.The calculation time of our proposed system is 30fps that equals to the 0.33ms.

Fig. 1 .
Fig.1.Proposed algorithm for occluded face detection represents the effectiveness of the modified skin color detector.a. Original image b.Skin color detector c.Noise filtered skin region d.Face region extraction Fig.2.Execution of modified skin color detector

4 . 2 (
Extract a portion of a gray face image by using the given calculation.X1=Max-index -(0.06*W)X2= Max-index + (0.06*W) Y1=0.14*WY2=W/Where, W is width of the face) 5. Apply median filter on the eyes 6. Normalize the eyes by adding 127 with the eyes and get a new matrix normalized eye.7. Find edges of the eyes 8. Apply boundary detection algorithm and get the exact eye region.The figure 7a, 7b and 7c shows the detection of right and left eye from the image, extracted right eye and extracted left eye respectively.

X1=Fig. 6 .
Fig.6.Geometrical construction for right and left eyes detection using nose point

Fig. 10 .
Fig.10.Face detection with facial features and nose point