UCLA

Program evaluation plays a significant role in aiding our understanding of the effectiveness of interventions. Evaluations operate under the auspices of evaluation policies, which shape aspects of evaluation design including research questions, data collection and analysis procedures, and reporting of findings. Despite considerable speculation about the influence that evaluation policy has over evaluation practice, we do not yet have any empirical evidence to support these ideas.

Through document analysis, interviews, and an instrumental multiple case study, this study examines both the explicit and implicit policies that overarch the research and evaluation work commissioned by the Robert Wood Johnson Foundation (RWJF), the nation's largest philanthropy devoted to our public health. The primary objectives of the current study were to: 1) detail and describe RWJF's evaluation policies (including their development and communication to relevant parties); and 2) describe how the implementation of those evaluation policies happens in the field.

Findings suggest the existence of two main policy distinctions--policies that are goal statements (which describe the desired outcomes of evaluation activities) and operationalized policies (which describe the specific processes that take place during the course of evaluations). This dichotomy serves as the basis for an argument that evaluation policies function as a system, rather than existing in silos as previously described in the extant literature.

Furthermore, this study sheds light on the ways evaluation policies relate to organizational and shared learning within the Foundation. Finally, it is argued that the development of specific and directed policies to promote innovation are a necessary step to take to advance evaluation practice. Evaluation policy is one of the most critical issues facing the field of evaluation today. This study was designed to shed light on a Foundation's evaluation policies in an effort to draw connections about how evaluation policies influence evaluation practice. The investigation helped further understanding of these connections, and generated some insight into the components of evaluation policies, and the role they might play in shaping the future of evaluation practice.