The Wrong Fix to Real Problems
A new Brookings Institution study of teacher evaluation systems in four districts finds that teachers with initially higher performing students get, on average, higher observation scores. The implication, the authors argue, is that observations are biased in a way that benefits teachers of students with higher incoming achievement levels and unfair to teachers of students with initially lower incoming achievement.
This should give us all pause. Observations are essential for getting teachers the feedback they need to improve so they and their students benefit. If those observations are biased — either as a result of the rubric itself or the biases and differential judgments of individual observers — that undermines the integrity of evaluation systems. And it does a real disservice to the very students that the movement to new, more rigorous evaluations was intended to benefit most. Any teacher will tell you that some classes are just plain tougher than others, especially when they are full of students who haven’t been taught well in earlier grades. Every part of our human capital systems should be about attracting the strongest teachers to these classrooms and keeping them there, not creating disincentives to teach struggling students.
That said, the “straightforward fix” proposed by the authors — simply adjusting the observation data by classroom demographics — is not nearly as straightforward as they suggest.
We know from research and long experience that it’s all too common for low-performing students to be saddled with novice and otherwise weak teachers. So we must confront head-on how much of the finding that teachers of lower performing students get lower observation scores is a problem with the observation itself, and how much is one more piece of evidence that initially low-achieving students get lower quality teachers.
Certainly, the folks at Brookings provide evidence that some individual teachers get lower ratings when they are observed in classes where they are teaching lower achieving kids than in other classes filled with higher achieving kids. That points to real challenges with observations that need attention.
But addressing the challenge by adjusting for demographics? This solution would paper over whatever problems may exist with the observation systems, as well as the very real problems that we know exist with teacher assignment. Not to mention that, if we went this route, the AP teacher whose classroom of high achievers was half black would have her scores adjusted up. That, itself, is an ugly enough idea.
Rather than heading down this dangerous path, let’s work to understand how pervasive the trend of teachers of low-performing students getting lower observations scores is. If we looked in more districts and across entire states, would we see the same patterns? If the answer is yes, then let’s not try and work around the problem. Rather, let’s tackle it head-on by paying serious attention to the quality of our observation systems, especially the quality of training and certification for observers, so that we can have confidence in the information they’re generating. And, most important, let’s once and for all get serious about getting our strongest teachers to the students who most need them.
