UC Riverside

There has been enormous interest in developing automatic face recognition techniques. Be it for government use such as law enforcement, voter identification, surveillance and immigration, or for commercial use such as gaming industry, face tagging on internet, e-commerce, healthcare and banking, a large number of real world applications utilize face recognition. A variety of challenges are associated with a face recognition system. While modeling variations in facial expressions, age, pose and illumination is necessary in many applications, certain specific applications may also involve comparing face images taken over different media such as a facial sketch to a photo.

Selecting visually salient, i.e., highly informative and discriminative features, is critical to every face recognition task. Often, such features are selected based on expert knowledge and/or learned from training data. The choice of these features is largely governed by the application. While there has been extensive work on saliency-based feature selection strategies for object/activity recognition in general, the role of saliency in the context of face recognition is relatively unexplored. The primary focus of this work is to investigate the role of saliency for face recognition.

We discuss three face recognition applications illustrating the role of saliency in each of these problem domains. In the first, we propose a framework for identifying subjects (sitters) in ancient portraits belonging to the Renaissance era. Apart from the typical face recognition challenges, recognition in art works come with the additional challenges of having to deal with limited training data and the need to model variability in artistic renditions. In this direction, we propose a framework that is capable of learning salient characteristics of individual artists and subsequently perform identification based on statistical hypothesis testing.

We next discuss a related face recognition application of comparing an artistic sketch with a photograph. Here, we propose an unsupervised face recognition scheme based on computing saliency maps constructed from region covariance matrices of low level visual features. We also discuss the utility of such features for face recognition in unconstrained environments (often referred to as `recognition in the wild') and subject to artificial distortions such as Gaussian blur and white noise. We conduct experiments on the Chinese University of Hong Kong Photo-Sketch database and the Quality Labeled Faces in the Wild (QLFW) to demonstrate the advantages of the proposed method.

Taking cue from the scenario of face recognition in art works wherein we have limited authenticated portraits, we next investigate the general problem of face recognition from very limited training data. This problem is relevant to many forensic science applications. We show that by learning salient features characteristic of a style such as a facial expression, pose, etc., one can obtain better recognition accuracies between face image pairs than with the case where such style information is not used. In particular, we leverage upon statistical hypothesis testing frameworks which can learn and validate features specific to a style. We conduct experiments on the publicly available PubFig dataset wherein the annotated attributes such as smiling, frowning, etc. are used as style information. We show that as the number of training instances in a style class is reduced, the model performs better than state-of-the-art techniques.