Adaptive Shadow Compensation for Face Recognition Under Illumination Variations

Robust face recognition under variant illumination conditions is a challenging research problem. In this paper, a new shadow compensation method, called adaptive shadow compensation (ASC), will be proposed to eliminate shadows in the face image due to non-frontal lighting directions. ASC performs shadow compensation in each local region, based on Fourier analysis on each local image block, to remove as much shadows as possible. Null-space linear discriminant analysis is then employed to extract discriminant features from ASC compensated images. Experimental results on the Yale B face database show that the proposed ASC method can achieve high face recognition accuracy under illumination variations.

In the face modeling based approaches, a number of face images under varying lighting conditions was used to construct an illumination model such as illumination cone model [4], spherical harmonic model [5], 3D linear subspace [6], etc.One major problem with the face modeling approaches is that it needs a number of training images in the model learning process.
In the image preprocessing techniques, some preprocessing/normalization methods were employed to normalize the face images such that the preprocessed images appear to be consistent even though they were taken under different illumination conditions.Several well-known image preprocessing methods, including histogram equalization (HE) [14,15], Gamma correction [15], logarithm transform [16], retinex algorithm [17], DCT-compensation [18,19], and DFT-compensation [20,21], were widely used to adjust the image intensities such that the processed images will have similar appearance to those images taken in proper illumination conditions.Choi et al. [20] proposed the shadow compensation (SC) method, based on Fourier analysis of the input image, to solve face recognition problem under illumination variations.Their experimental results on face images having variant lighting conditions have shown that SC outperforms HE and other well-known approaches in terms of recognition accuracy.
In this paper, we will propose a novel shadow compensation method, called adaptive shadow compensation (ASC), for robust face recognition under different illumination conditions.The proposed ASC method performs shadow compensation in each local region, based on Fourier analysis, rather than on the whole image.Thus, local details can be well compensated.We will first review the SC method [20] in Section 2 and then describe the proposed ASC method in Section 3. Experimental results showing the effectiveness of the proposed ASC method will be shown in Section 4. Finally, a brief conclusion will be given in Section 5.

Review of Shadow Compensation Method
Oppenheim et al. have shown that the phase information retains most of the intelligibility property of an image [22,23].Typically, the phase spectrum acts to the locations of edges in an image.Thus, the discriminant information will be kept in the phase spectrum of the image.Give a face image having shadow regions, the Fourier phase spectrum usually keeps unchanged while the Fourier magnitude spectrum will be changed greatly in those shadow areas.Therefore, the SC method proposed by Choi et al. [20] tries to compensate for the magnitude spectrum of an input image in order to get a recovered image having proper intensities in those shadow regions.To get the shadow compensated image, a set of reference images having proper illuminations were collected and transformed to the frequency domain by using 2D Discrete Fourier Transform (2D-DFT).The average of the magnitude spectra, notated by |F avg (u, v)|, of these reference images will be used for shadow compensation purpose.Given an input image, it is first transformed into frequency domain.Then, its magnitude spectrum |F(u, v)| and the average magnitude spectrum |F avg (u, v)| are combined to obtain the modified magnitude spectrum: The phase spectrum of the input image Φ(u, v) and the modified magnitude spectrum |F M (u, v)| will constitute the modified Fourier spectrum of the shadow compensated image: Thus, the recovered face image can be obtained by performing inverse 2D-DFT on the modified Fourier spectrum.
The SC method can generate a recovered image with some shadow regions being property compensated.However, there still exist some distorted regions in the recovered image (please Fig. 2(c)).This is due to the fact that the SC method modified the magnitude spectrum of the whole image, without considering the characteristics of different local regions, to reduce the effect of shadows.As a result, those local details cannot be properly compensated.In this paper, we will propose a novel shadow compensation method, called adaptive shadow compensation (ASC), for robust face recognition under variant illumination conditions.The proposed ASC method performs shadow compensation in each local region, based on Fourier analysis on each local image block, rather than on the whole image.Thus, intensity in local details can be well compensated.

Adaptive Shadow Compensation (ASC) for Face Recognition
The proposed ASC method, robust to face recognition under variant illumination conditions, tries to obtain a restored face image with each local region being well compensated.Fig. 1 shows the block diagram of the proposed ASC method.First, each face image I(x, y) is enhanced using HE to get an enhanced image I HE (x, y).The HE enhanced image will be decomposed into a number of image blocks, with each block of size K×K pixels.Shadow compensation using Fourier analysis will be conducted on each local image block such that shadows can be properly removed, in order to get a restored face image appearing as consistent with normal face images as possible.

Adaptive Shadow Compensation
Then, we can compute the average magnitude spectrum of the co-located image blocks of all training images:  Fig. 2 shows a number of images and the processed images using HE, SC, and ASC.From this figure, we can see that either HE or SC will produce an image with shadow regions being enhanced to some extent.However, noises in the shadow region are also visibly amplified.Overall, the proposed ASC method will generate a restored image with every local region being properly compensated, than HE or SC.Thus, it is expected that high recognition accuracy can be obtained if the ASC processed images were used for face recognition.

Feature Extraction and Recognition
Similar to the SC approach [20], the null space based linear discriminant analysis (NLDA) [24,25] will be used to extract discriminant features from the restored images using ASC.In NLDA, the between-class scatter is maximized in the null space of the within-class scatter matrix.Assuming that the training set consists of C classes, with N i samples in the i-th class.The between-class scatter matrix S b and within-class scatter matrix S w are defined as follows: where x is a training image represented in a vector form, μ G is the mean vector of all training images, μ i is the mean vector of the training images in class C i .In general, the dimension of the image space is larger than the sample number in the training set.As a result, S w will become singular and results in the so-called small sample size (SSS) problem.Since the null space of S w contains much discriminative information, the objective function of NLDA is defined by W opt is the transformation matrix consisting of the d eigenvectors that maximizes the between-class scatter.This matrix will be used to project each image vector x to get a d-dimensional discriminant feature vector y: x y In the recognition process, the Euclidean distance is used to measure the distance between two discriminant feature vectors.Nearest neighbor rule will be employed to determine the recognition class of each input image.

The Face Database
In this paper, The Yale B face database [4] is used to evaluate the performance of the proposed ASC method.The Yale B database consists of 10 individuals.The images of each individual have nine different poses and with 64 illuminations per pose.We used the face images in the front pose, which was divided into 5 subsets, for performance evaluation.The division of the image subsets depends on the angle between the direction of light source and the camera's axis.The light direction angles for these five subsets are 0°-12°, 12°-25°, 25°-50°, 50°-77°, and >77° (please see Fig. 3).In this paper, the images belong to subset 1, which have proper illuminations, were chosen as the training set and images in the other subsets were used as the testing set.These training images will be treated as the reference images for generating the average magnitude spectrum in ASC.For each image, the eyes are aligned to fixed positions and each image is rescaled to be size of 120×100 pixels.

Comparison of Recognition Accuracy
In the experiment, NLDA performed on the raw images (I RAW ), image enhanced using HE (I HE ), restored image using SC (I SC ), and restored images using ASC (I ASC ), were compared.Fig. 4 compares the recognition accuracy of these methods for different number of discriminant feature vector (d).From this figure, we can see that SC outperforms HE.Further, the proposed ASC yields the best recognition accuracy.
Table 1 compares the recognition accuracy of each subset given that the feature dimension d = 9.This table reveals that the proposed ASC method always yields the best recognition result for every subset.That is, ASC is more robust to illumination variations than HE or SC.

Conclusions
In this paper, a new shadow compensation method, called adaptive shadow compensation (ASC), is proposed to eliminate shadows in the face image due to non-frontal lighting directions.ASC performs shadow compensation in each local image block, based on Fourier analysis, to remove as much shadows as possible.Null-space linear discriminant analysis is then employed to extract discriminant features from ASC compensated images.Experiments performed on the Yale B face database have shown that the proposed ASC method can reserve more detailed information in shadow regions, and thus can yield high face recognition accuracy.In the near future, we will try to apply the proposed ASC method to compensate for over-exposed, under-exposed, or dark images.
of training images.Further, let B i,a,b be the decomposed image block of size K×K at location (a, b) of the i-th training image, a = 1, 2, …, M, b = 1, 2, …, N. Therefore, each image is decomposed into M×N image blocks.We then perform 2D-DFT on each image block to get the Fourier spectrum:

Fig. 1 .
Fig. 1.Block diagram of the proposed ASC method modified spectrum F a,b (u, v) can be derived from the modified magnitude spectrum | spectrum Φ a,b (u, v) of the input image: shadow compensated image block can be recovered by performing inverse 2D-DFT on the modified spectrum

Fig. 4 .
Fig. 4. Comparison of Recognition accuracy on the Yale B face database

Table 1 .
Recognition accuracy on the YALE B database (%) Data Subset 1 Subset 2 Subset 3 Subset 4 Subset 5 All