UC San Diego

The goal of object recognition is to locate and identify instances of an object within an image. Examples of this task include recognition of faces, logos, scenes and landmarks. The use of this technology can be advantageous in guiding a blind user to recognize objects in real time and augmenting the ability of search engines to permit searches based on image content. Traditional approaches to object recognition use appearance features - e.g., color, edge responses, texture and shape cues - as the only source of information for recognizing objects in images. These features are often unable to fully capture variability in object classes, since objects may vary in scale, position, and viewpoint when presented in real world scenes. Moreover, they may introduce noisy signals when objects are occluded and surrounded by other objects in the scene, and obscured by poor image quality. As appearance features are insufficient to accurately discriminate objects in images, an object's identity can be disambiguated by modeling features obtained from other object properties, such as the surroundings and the composition of objects in real world scenes. Context, obtained from the object's nearby image data, image annotations and the presence and location of other objects, can help to disambiguate appearance inputs in recognition tasks. Recent context-based models have successfully improved recognition performance, however there exist several unanswered questions with respect to modeling contextual interactions at different levels of detail, integrating multiple contextual cues efficiently into a unified model and understanding the explicit contributions of contextual relationships. Motivated by these issues, this dissertation proposes novel approaches for investigating new types of contextual features and integrating this knowledge into appearance based object recognition models. We analyze the contributions and trade -offs of integrating context and investigate contextual interactions between pixels, regions and objects in the scene. Furthermore, we study context as (i) part of recognizing objects in images and (ii) as an advocate for label agreement to disambiguate object identity in recognition systems. Finally, we harness these discoveries to address other challenges in object recognition, such as discovering object categories in weakly labeled data