Training set of face images which is a subset of JNTU Face Database

ABSTRACT

Face recognition is an advanced statistical technology in computer systems that can assist recognition of human faces using principal component analysis (PCA) by comparing the given facial characteristics with the already available data (faces). Human faces are normally upright so they can be treated as 2-D images rather than 3-D. The 2-D facial image can be converted into a 1-D vector of pixels and projected into the principal components of the feature space called the eigenspace projection, which is calculated from the eigenvectors of the covariance matrix derived from a set of facial images. The projections of the given face are then compared with the available training set and the face is identified. In this paper, we introduce a novel approach of dimensionality reduction of the covariance matrix. Recognition rate versus Euclidean distance and cumulative eigenvalue variation is presented.

Keywords:Eigenvector,Eigenfaces,Eigenvalue,Dimensionality reduction, principal component analysis.

INTRODUCTION

Principal component analysis technique can be used to simplify a huge data set consisting of a number of correlated data values into a smaller set of uncorrelated values by reducing the dimension of the data set while still retaining most of the inherent data. The process begins with a collection of 2-D facial images into a database called the facespace. Using the eigenfeature method, the similar and dissimilar variations of the faces are obtained as a mean image of all the faces in the facespace. This gives the training set of images. The difference image is computed by subtracting the mean image from every image in the training set, which provides the covariance matrix. From the covariance matrix the eigenvectors and eigenvalues are obtained. The eigenvectors are treated as eigenfaces, which are projected into the eigenspace. Then weight vectors for all the eigenfaces in the eigenspace are calculated. The weight vector for the test image is also calculated and the distance between them determines the presence or absence of a similar face in the facespace.

Face recognition is a primary element of the many aspects of a society. If the population is limited as in small villages, identifying a particular person would not be a problem. But as the number of individuals increases, recognizing a specific face out of many becomes much difficult task. Giving a unique electronic identity to each one of them overcomes the problem to some extent but even they cannot work if the cards are stolen or passwords are forgotten. Instead, if physical aspects like facial features are taken as the basis for recognition, it would be easier to identify the faces. As much, face recognition techniques have gained a wide importance with the increase in the number of applications like security access control, criminal identification, human–computer interaction and surveillance systems.

Face recognition using principal component analysis is a simple, efficient and robust technique compared to any other matching approaches because direct data of any intensity can be employed without the need for complicated processing. Besides, PCA does not require any significant knowledge of geometry and reflectance of faces. PCA uses low-dimensional subspace representation to achieve data compression, which increases the speed of the system. However, the disadvantages are that the face recognition process works efficiently only when the number of face classes is larger than the dimensions of the facespace and updating the facespace is a complex task. If the input faces are deformed, PCA poses problems in terms of reduced efficiency. Research work towards face recognition has been intensified much during the past few years. Turk and Pentland et al [1] proposed that every image can be represented by a linear combination of eigenvectors of the covariance matrix of the set of face images and these projections of the new image are then compared with the training set to identify a particular image. An extension to the eigenface system is to include multiple viewing angles of a face thereby increasing the system’s performance in dealing with faces of various orientations as proposed by A. Pentland, B. Moghaddom [2]. But this work is tested for similar illumination conditions. An extensive study on human and machine recognition of faces was proposed by R. Chellappa [3] considering both static matching of photographs as well as real-time video images. The linear discriminant theory of Fisher to derive the similarities between multiple images and then deducting the variations due to lighting from the defined subspace is a fair extension to the standard eigenface approach as discussed by P. Belhumeur [4]. However, even PCA cannot capture the simplest invariance unless sufficient information is explicitly provided in the training data set as pointed out by L. Wiskott [5] discussed under the elastic bunch graph matching theory based on this fact.Relatively recent concepts called independent component analysis (ICA) and Kernel PCA were proposed by M.S. Bartlet, J.R. Movellan and B.A. Draper, K. Baek [6], [7] claiming that they were proven to be better than PCA when cosines were used as similarity measure. Yet, both ICA and Kernel PCA are computationally very expensive and complicated than PCA. Edge detection criteria could be used for feature extraction and identification to provide a good interpretation of face recognition as proposed by Pycock et al [8]. According to Huang et al [9], model-based hand gesture recognition system works in three steps – feature extraction which uses the spatial and temporal information to extract features, training, which uses principal component analysis and Hidden Markov Models to describe shape variations and recognition which uses the training images for gesture identification. Inoue et al [10] presented line algebra and three-dimensional feature points on the test face for face recognition. Two-dimensional Principal Component Analysis proposed by Jian et al [11] adopted 2-D vectors in place of 1-D vectors for covariance matrix. Rate of recognition is reported to be efficient in 2-D PCA than 1-D PCA. Bayesian discriminating features method for multiple front view face detection is proposed by Chengjun Liu [12]. This work uses training images from one facespace and test images from diverse sources. Generally super-resolution acts as a preprocessing step to derive a high-resolution image and then passed on to a face recognition system. Bahadir et al [13] proposed a transfer of the super-resolution reconstruction from pixel domain to a lower dimensional face space reducing the computational complexity of the super-resolution reconstruction. When compared with various techniques mentioned above, PCA provides a compromise between recognition rate and computational complexity. Photometric normalization algorithms can be used for face verification [14] to deal with illumination differences in facial images. Ch. Satyanarayana et al [15] proposed a method that describes the reduction of covariance matrix dimension in PCA for face recognition for which it can be used in eigenface approach to compute the variations in the similarities of the faces in the database and projecting them into a facespace. Then, when a new image is input for recognition, it extracts the prominent features of the face and projects them onto the facespace to classify and identify the new face. Another novel method [16] proposed by Ch. Satyanarayana et al deals with the concept of dimensionality reduction of the covariance matrix conforming with the fact that the number of eigenfaces increases the dimension of the covariance matrix thus increasing the difficulty of calculating the eigenvectors. The model proposes to remove all the redundant data from the facespace and hence decrease the size of the covariance matrix to make face recognition more feasible.

This work, though inspired by all the above researchers, introduces a new concept of dimensionality reduction of the covariance matrix, which was a prominent feature ignored in the previous related work. As the number of faces in the facespace increases, its dimensionality also increases, making it very difficult to calculate the eigenvectors. If all the redundant data from the facespace is removed, the dimension of the covariance matrix can be brought down to a more feasible size.

Rest of the paper is in the following manner – section 2 deals with the mathematical modeling and recognition in principal component analysis, section 3 describes the results and experiments and section 4 gives a conclusion of the paper.

MATHEMATICAL MODELING

Let f (x, y) be a two-dimensional function, where x and y are spatial coordinates and the amplitude of f at any pair of coordinates (x, y) is called the intensity of the image at that point. Every image I can be expressed as a matrix of f (x, y)s of dimension m X n. Let F be the set of M training images of same dimension expressed as a array of dimension (m X n) X M

F = (I₁ I₂ I₃………….I_M) (1)

These M images are converted as vectors X_i, 1 ? i ? M of dimension N (= m X n) where X_i is an N X 1 vector corresponding to the image I_i in the image space F. Now F becomes

F = (X₁ X₂ ………..X_M) (2)

The mean image is calculated by summing all the training images and dividing by the number of images with dimension (N X 1) as follows

= (3)

The difference image is obtained by subtracting the mean image from all the training images X_i is stored in a vector ?_i

(4)

The main idea behind the eigenface technique is to exploit the similarities between the different images. For this purpose the covariance matrix ?_v with dimension N X N is defined as

?_v = = ? ? (5)

Where ? = (?₁ ?₂ ?₃ ………….?_M ) is of dimension N X M. This covariance matrix dimension will normally be a huge matrix, and full eigenvector calculation is impractical.

Figure 1. Training set of face images which is a subset of JNTU Face Database

As a concrete Illustration, the training set, which is a subset of the JNTU Face Database, is shown in Figure 1.

2.1 DIMENSIONALITY REDUCTION IN COVARIANCE MATRIX

We assume that without loss of generality of the whole training set, we can reduce the dimensionality of the covariance matrix B with dimension M X M defined as

B = ? ? = (6)

The eigenvectors of the covariance matrix ?_v are computed by using the matrix B. Then the eigenvector ? _i and the eigenvalue ? of B are obtained by solving the characteristic eigenvalue problem | B - ? I | = 0

B. ? _i = ?. ? _i (7)

Substituting the value of B in eqn. (7)

? ?. ? _i = ?. ? _i (8)

Multiplying both sides with ? in eqn.(8), we obtain

?. ? ?. ? _i = ?. ?. ? _i (9)

Since ? is a scalar quantity,

? ? . ? ? _i = ?. ? ? _i (10)

Substituting the value of ?_v = ? ? in eqn.(10), we obtain ?_v. ? ? _i =?. ? ? _i (11)

Now, let ? = ? ? _i and ? are M eigenvectors and eigenvalues of ?_v. In practice, a smaller set of ((M – 1)

< M) eigenfaces is sufficient for face identification because the subspace is the basis for the facespace, i.e., we can represent the original face as a linear combination of thesevectors. Hence, only significant eigenvectors of ?_v, corresponding to the largest eigenvalues are selected for the eigenface computation thus resulting in a further data compression. The remaining (N –) eigenvectors would have associated eigenvalues very close or equal to zero.

2.2 RECOGNITION

Now, the training images are projected into the eigenface space and the weight of each eigenvector to represent the

Figure 2. The first 20 eigenfaces corresponding to the 20 largest eigenvalues

image in the eigenface space is calculated. The weight is simply the dot product of each image with each of the eigenvectors.

w_k = ?. ?_i = ?. (X_i - ) (12)

where k = 1,2,…….. All the weights are converted in the form of a matrix with dimension X 1

? = [w₁, w₂, w₃………..w_k] (13)

To test an unknown image ?, we evaluate its weight (w_i) by multiplying the eigenvector (?_i) of the covariance matrix (?_v) with difference image (? – )

w_i = ? (? – ) (14)

Now the weight matrix of the unknown image becomes

? = [w₁, w₂, w₃………..w_M’] (15)

The Euclidean distance ?_k between unknown image and each face class is defined by

?_k² = || ? - ?_k || ²; k = 1,2,…….,N_c(16)

Where N_c is the number of face classes. We obtain a reconstructed image by multiplying the weight matrix (?) of the unknown image with the eigenvector matrix (?) of the covariance matrix (?_v) and adding the mean face image () to it. ?_f = ? . ? + = [?₁ ?₂ ?₃……………..?_M¹] +

= + (17)

Where ? = [?₁ ?₂ ………?_M’]. Now the Euclidean distance (?) between the original unknown image and the reconstructed image is computed in order to distinguish between face images and non-face like images.

?² = || ? – ?_f || ²; (18)

Calculating the Euclidean distance between two data points involves computing the square root of the sum of the squares of the differences between corresponding values. These Euclidean distances are compared with an empirical threshold value. If the threshold value is greater than both the Euclidean distances, the system recognizesit as a face belonging to a known class. If the threshold value is less than the Euclidean distance (?_k) and greater than ?, then it recognizes the face as close as to the facespace but do not represent a known face class. If the threshold value is less than both the Euclidean distances, the system recognizes it as unknown object.

3. RESULTS AND DISCUSSIONS

The database used in this context contains a set of face images of 130 individuals each with 3 different expressions and poses under similar illumination. The dimension of each image is 200 X 150 pixels in gray mode. PCA technique is applied to the database with a test set of random images. Eigenfaces represent prominent features of face images of the training set. This plot corresponds to highest eigenvalues of the individual eigenfaces of the training set. The highest eigenvalues representing the eigenfaces of 80 individuals with a training set of one face per individual is plotted in figure 3. JNTU face database is used for the computation of the above plot. From the figure 3, it is observed that 20% of the faces are sufficient for recognition and the remaining 80% faces are insignificant. Even if they are discarded, there is no problem for recognition. In order to find out the proper value of M^’, the eigenvalues of the covariance matrix is ranked in descending order. Suppose there are M^’ eigenvalues, i.e., ?₁, ?_2, ?_3, …?_M’, (it is convenient if these are appropriately arranged such that ?_1??_2??_{3? …} ?_M’ )_.where ?₁ is the largest eigenvalue while ?_M’ is the smallest one. The eigenvalue range of the training data provides useful information for the choice of the reduced PCA space M’. It is also observed from Figure 3, that the eigenvalue variation of the training face images increases gradually as the facespace grows from 1 to 80 with 10. In the figure 3, the number of eigenfaces taken the x-axis and take the eigenvalue variation on the y-axis. The graph shows that how the eigenvalues changing from one face image to the other face image. As the number of the eigenfaces in the database increases, the eigenvalue is observed to have exponential decay till it reaches the value zero. For the first eigenface, the eigenvalue is the highest and for the second eigenface, it is the next highest and so on and for the last eigenface, it is zero. This means that the eigenvalue variation is inversely proportional to the number of eigenfaces with the Eq.(11) is presented in figure 3.

When the Euclidean distance increases, the recognition rate is found to be decreasing. That is, for the minimum distance, the recognition rate is the highest. From the figure 4,it is observed that as the distance goes on increasing, the recognition rate drops correspondingly until it reaches zero, which means that the recognition rate is inversely proportional to the Euclidean distance with the Eq.18 is presented in figure 4.

When experimented with the number of eigenvectors and recognition rate, results show that the recognition rate grows as a factor of individual number of eigenfaces. For a single set of individual eigenfaces, the recognition rate is lower but as the sets are added, the recognition rate increases correspondingly with it. From the figure 5, it is observed that the recognition rate is directly proportional to the individual eigenfaces.

Figure 3. Eigenvalue variation

Figure 4. Euclidean distance vs Recognition rate

For our experiments we have considered 200 images from the entire face database for which the timing diagram is illustrated in the figure 6. The figure shows timing plots for both the cases – with covariance matrix reduction and without covariance matrix reduction. It is observed from the graph that the timing for face recognition with covariance matrix reduction is around 34 seconds while without eduction of the covariance matrix, it is around 176 seconds.

Figure 5.Number of Eigenfaces vs. recognition Rate

Figure 6. Execution Time With and Without Covariance Matrix Dimension Reduction for 200 Images

This shows that without covariance matrix reduction, the execution time for the face recognition algorithm is almost five times that of with covariance matrix reduction. The graph from figure 6 deals with the static images of the front views.

4. Conclusion

We have presented an elaborate discussion about modeling and results of face recognition algorithm using principal component analysis. Dealing with eigenfaces, eigenvalues and recognition rates, we plotted the corresponding graphs that are obtained using MATLAB 6p5. By reducing the dimension of the covariance matrix, we could achieve faster implementation of the algorithm, which is brought down to a few seconds over a database of 200 images. This is far better than the results achieved by some experts in the field. The recognition rates also are found to be better than other algorithms presented in literature.

References:

[1] M. A. Turk and A. P. Pentland. Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1):71-96, 1991.

[2] A. Pentland, B. Moghaddom, T. Starner. “View-Based and Modular Eigenfaces for Face Recognition”, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR’94), 1994.

[3] R. Chellappa, C. L. Wilson, and S. Sirohey. Human and machine recognition of faces: A survey. In Proc. of IEEE, volume 83, pages 705-740, 1995.

[4] P. Belhumeur, J. Hespanha, D. Kriegman. “Eigenfaces vs. Fisher faces: Recognition Using Class Specific Linear Projection”, IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI19 (7), pp. 711-720, 1997.

[5] L. Wiskott, J.M. Fellous, N. Kruger and C. von der Malsburg, “Face Recognition by Elastic Bunch Graph Matching”, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 775-779, July 1997

[6] M.S. Bartlett, J.R. Movellan and T.J. Sejnowski, “Face Recognition by Independent Component Analysis”, IEEE Trans. Neural Networks, vol. 13, no.6, pp. 1450-1464, 2002.

[7] B.A. Draper, K. Baek, M.S. Bartlett, J.R. Beveridge, “Recognizing Faces with PCA and ICA”, Computer Vision and Image Understanding: special issue on face recognition, in press.

[8] Pycock, D, Pammu, S, Goode, AJ, and Harman, SA, "Robust model-based signal analysis and identification," PATTERN RECOGNITION, vol. 34, pp. 2181-2199, 2001.

[9] Huang, CL, and Jeng, SH, "A model-based hand gesture recognition system," MACHINE VISION AND APPLICATIONS, vol. 12, pp. 243-258, 2001

[10] Inoue, A, Drummond, T, and Cipolla, R, "Real time feature-based facial tracking using Lie algebras," IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, vol. E84D, pp. 1733-1738, 2001.

[11] Jian Yang, David Zhang, Alejandro F. Frangi and

Jing-yu Yang, “Two-Dimensional Principal Component Analysis :A wApproachto Appearance-Based Face epresentation and Recognition”,IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, No. 1, January 2004.

[12] Chengjun Liu, “A Bayesian Discriminaing Features Method for Face Detection”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, No. 6, June 2003.

[13] Bahadir K. Gunturk, Aziz U. Batur, Yucel Altunbasak, Monson H. Hayes, Russell M. Mersereau, “Eigenface – Domain Super Resolution for Face Recognition”, IEEE Transactions on Image Processing, vol. 12, No. 5, May 2003.

[14] James Short, Josef Kittler, Kieron Messer, “A Comparison of Photometric Normalization Algorithms for Face Verification”, Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition (FGR ’04), 2004.

[15] Ch. Satyanarayana and L. Pratap Reddy “Reduction of Covariance Matrix Dimension in PCA for Face Recognition”, Proceedings of an International Conference on Systemics, Cybernetics and Informatics (ICSCI 2006), Vol. 1, pp. 674-678, 2006, Hyderabad.

[16] Ch. Satyanarayana and L. Pratap Reddy “Dimensionality Reduction in PCA for Face Recognition”, Proceedings of an International Conference on Bioinformatics and Diabetes Mellitus (ICBDM 2006) vol.1, pp. 250-255, Visakhapatnam.

Attached Files:

• dip095561.doc (3165 KB)