UC San Diego

Recovery of scene shape, reflectance, and illumination are of fundamental importance to computer vision. However, the image formation process involves complex interactions between all three components, making inference difficult or impossible in the absence of simplifying models or prior scene knowledge. Unfortunately, real-world scenes often violate these approximations, leading to biased or incorrect reconstructions. Thus, there is a constant struggle between model complexity and tractability. In this dissertation, new models and methods are presented for recovering shape, reflectance, and illumination which are valid for broader classes of scenes than competing techniques. Underlying all of the research presented is the ability to handle objects with complex reflectance. First, a novel approach is presented for resolving the generalized bas-relief ambiguity which arises in uncalibrated photometric stereo. Previous work showed that the ambiguity can be resolved for textureless objects; however, as shown in this dissertation, it is also possible to resolve the ambiguity for textured objects, provided there is statistical regularity in the distribution of albedo values across the surface. Next, a photometric stereo algorithm is presented that is capable of handling nearly arbitrary reflectance. The main contribution is the utilization of bilateral symmetry in the reflectance function, a property shared by most real- world materials. By explicitly utilizing symmetry, surface shape can be constrained without relying on parametric models; a significant advance over most photometric stereo algorithms which depend on simple parametric models of surface reflectance, such as the Lambertian model. Another photometric stereo algorithm is also presented that is capable of fully recovering the surface shape as well as the reflectance function across the surface. While a few additional constraints are necessary, this is one of only a handful of photometric stereo methods capable of simultaneously recovering shape and complex reflectance; of these, the reflectance model is by far the least restrictive. It is also shown that bilateral symmetry of the reflectance function can be exploited for multi-view shape reconstruction. The method presented handles both textured and textureless surfaces and is capable of recovering surface concavities. Finally, a novel technique for measuring illumination is presented which relies on spatially varying reflectance