UCLA

Recent federal grant programs and legislative initiatives have focused on improving measures of effective teaching within comprehensive teacher evaluation systems. While emerging research is contributing important information about the reliability and validity of evaluative measures as they apply to general educators, relatively little work has been done on the use of these same measures with special education teachers. What is unknown at present is who is best qualified to perform evaluations of special education teachers, especially when using one of the most common classroom performance measures: observation protocols. This study examined the rater reliability of school administrators—who do not possess expertise in the area of special education, and are typically responsible for conducting evaluations of special education teachers in the actual school setting—as raters of special education teachers’ instructional practice. The study used a mixed method-design, which involved a quantitative analysis of administrator ratings using generalizability (G) theory. A qualitative analysis of participants’ perceptions of their rating experience, as well as special education teachers’ perceptions of the evaluation process, was performed using a phenomenological approach. Findings suggest that school administrators show promise as reliable raters even without formal training in special education, but school administrators need to engage in repeated classroom visits, be invested in the evaluation process, and be properly trained on any measure used for evaluative purposes.