UC Berkeley

This dissertation can be viewed as a collection of case studies in applying variational inferenceto analyze large data problems. The largest, most computationally difficult data problem we tackle is the task of cataloging light sources imaged by astronomical surveys (Chapter 2). For some surveys, the size of the raw data is on the scale of petabytes. To do inference, we employ two recent advances in variational inference: amortization and the wake-sleep algorithm.

The data analysis problems in Chapters 3, 4, and 5 are more tractable. The analyses inthese chapters are exploratory in nature. However, such exploratory data analyses often require fitting either several related models or fitting the same model on subsamples of the data. Repeatedly solving for variational optima after each model or data perturbation may be unnecessarily expensive, particularly for exploratory settings where approximate optima might suffice. In these chapters, we present the notion of local sensitivity which we use to quickly extrapolate, from an initial variational optimum, the posterior quantities that would be obtained after a model or data perturbation. Our approach is particularly apt for sensitivity analysis, where the goal is to understand how conclusions might change should a different model be specified or should different data be observed.

Finally, Chapter 6 considers probabilistic machine learning problems where the trainingobjective is an expectation over a discrete latent variable. The standard reparameterization and backpropagation method for computing stochastic gradients do not apply in this setting. Many alternative stochastic gradient estimators have been proposed specifically for this problem. Chapter 6 outlines a technique to lower the variance of any gradient estimator by employing a general statistical method called Rao-Blackwellization.