Skip to main content
eScholarship
Open Access Publications from the University of California

UC Berkeley

UC Berkeley Electronic Theses and Dissertations bannerUC Berkeley

Beyond Photo-Consistency: Shape, Reflectance, and Material Estimation Using Light-Field Cameras

Abstract

Light-field cameras have recently become easily accessible in the consumer market, making applications such as post-shot refocusing and viewpoint parallax possible. An important benefit of light-field cameras for computer vision is that multiple viewpoints or subapertures are available in a single light-field image, enabling passive depth estimation. However, most existing approaches are based on photo-consistency, i.e., all viewpoints exhibit the same color when focused to the correct depth. This assumption does not hold in a number of circumstances, e.g., in the presence of occlusions, and when the surface is not Lambertian.

In this thesis, we refrain from assuming photo-consistency, and explicitly deal with the situations where it fails to hold. First, we propose a novel framework that can handle occlusions when estimating depth, so we are able to get sharper occlusion boundaries in the obtained depth maps. Next, we extend traditional optical flow to the case of glossy surfaces, and derive a spatially-varying (SV) BRDF invariant equation. Using this equation, we can then recover both shape and reflectance simultaneously using light-field cameras. Finally, we show an application of recognizing materials in light-field images, based on the reflectance information extracted using learning-based methods. By looking beyond photo-consistency, we are able to estimate better depths and recover reflectance and material types, which can be useful for a variety of vision and graphics applications.

Main Content
For improved accessibility of PDF content, download the file to your device.
Current View