UCLA

“Data-based decision-making” or “evidence-based decision-making” in education are now popularized phrases to describe the systematic collection and analysis of various types of data to help improve the success of students and schools (Marsh, Pane, & Hamilton, 2006). The theory of action underlying data use activities implies that education�practitioners who ground their decisions in evidence will more effectively deliver methodical improvements to teaching and learning. However, very little research has been conducted to test this hypothesis. In addition to the research community’s vague understanding of how schools − and the individuals comprising schools − interpret and implement data-based decision-making policies, it is difficult to determine whether data use practices are actually associated with improved instruction. As a result, school districts, as well as state and federal policy makers, have little understanding of how schools are actually using data, how differences in data use may affect school performance, and/or what kinds of measures could be used to indicate the effective use of data in schools.

This comparative case study of three high schools in Los Angeles Unified School District develops an illustrative understanding of how school decision-makers (i.e., teachers, principals, and district personnel) make meaning of directives to “use data for decision-making” and how the use of school-based data takes place in practical application. Drawing upon interview and observational data from principals, teachers, and district managers, it acknowledges that schools are inundated with multiple data sources and that teachers and administrators regularly rely on data use practices. The expectation that schools should more systematically, formally, and cooperatively review data to steer conversations around teaching and learning, however, implies paradigmatic shifts in the ways that data are currently understood and utilized.

Findings suggest that the effective use of school data in decision-making by school practitioners was not the product of an organized, rational process, nor one simply improved with the introduction of inputs and interventions. Rather, it suggests that culturally derived definitions of credible data, leadership, decision-making processes, accountability, organizational learning, and evaluation – and even whether data are relevant in teachers’ thinking in institutional contexts – shape stakeholder attitudes toward data use in classrooms and schools. In constant dialogue, stakeholders tacitly and explicitly negotiated what data were used in measuring school, teacher, and student performance, how data were collected and analyzed in ways that maintained credibility, who was involved in decision-making moments and in what ways, and how data could meaningfully inform programmatic student supports and instructional improvements. Data and data use processes intended to influence decision-making were, as a result, reliant on cultural, political, and subjective factors, and evolved in necessarily gradual cycles of establishment, revision, and refinement.