In a nightmare scenario reminiscent of a Kafka novel, a public school teacher in Washington D.C. was fired, three teachers missed out on bonuses, and 40 others received inaccurate job evaluations, all because of a single missing suffix amongst a sea of programming.
How could this happen?
“Value-Added“ Teacher Evaluations
Blame the “value-added” method of evaluating teachers, and in particular, Michelle Rhee, the former chancellor of the D.C. public schools, who in 2009 announced a radical plan to rate teachers’ effectiveness on a numerical scale, then fire the worst and give the best huge pay hikes. (On a side note, not only has the system proven ineffective, but Rhee’s tenure was marred by allegations of cheating as scores at some schools shot up, only to plummet after more stringent security measures were put in place.)
Since that time, the idea of using a numerical scale to evaluate teachers has been spreading nationwide.
Now this miscalculation has raised alarms about the increasing reliance nationwide on complex “value-added” formulas that use student test scores to attempt to quantify precisely how much value teachers have added to their students’ academic performance. There’s good reason for alarm: these value-added metrics often carry high stakes like teachers’ employment, pay, professional licenses and overall job satisfaction.
What Exactly Happened in D.C.?
Apparently the problem stemmed from “a very small typo” inserted into complex programming code during an upgrade earlier this year, said Barbara Devaney, chief operating officer of Mathematica Policy Research, the private firm that holds the contract to calculate value-added scores for the district.
Devaney said the firm employs stringent quality control, which in this case included 40 hours of meetings to review the updated model and an analysis by independent programmers paid to comb through the code line by line. Yet no one noticed the missing suffix until yet another routine quality review took place this November — after the district had already distributed bonuses, layoff notices and evaluation scores based on the value-added data for the 2012-13 school year, Devaney said.
According to James Kamras, chief of human capital for the D.C. schools, the district has reached out to the teacher who was mistakenly fired with a job offer and has also promised salary payments retroactive to the start of the school year. In addition, the bonuses for the three highly effective teachers will be distributed immediately.
It’s great, of course, that the district has taken this action, but they should also use this opportunity to re-think the whole system of value-added and abandon it.
While it is true that no state evaluates its teachers on student test scores alone, states can allow student achievement on standardized tests to count for as much as 50 percent of a teacher’s evaluation.
It is Ridiculous to Evaluate Teachers Using a Mathematical Formula
* “You can‘t simply take a bunch of data, apply an algorithm and use whatever pops out of a black box to judge teachers, students and our schools,” Randi Weingarten, president of the American Federation of Teachers, said this week.
* Tests can be affected by human error, like the missing suffix in the programming code for D.C. schools.
* Test results fluctuate wildly from year to year; one high school where I taught had been deemed a “Blue Ribbon School” on the basis of the test results of one class, which happened to have a very strong group of students that year. If the school had been evaluated the following year, it would not have received that distinction.
* Teachers have to work with the hand they are dealt; every class varies from year to year. When I asked one of my current Spanish 3 students about his failing grades last semester, he smilingly informed me, “I don’t do homework.” Some students just aren’t into school, for a whole host of reasons, but that doesn’t mean teachers should be punished.
* It is educationally unsound to use one test as a main determinant of a teacher’s worth. I am lucky this year to have a wonderful evaluator, who will be looking for how engaged my students are, how my curriculum is developed, what my classroom looks like, my use of technology and a whole host of other indicators. Measuring teacher effectiveness is complicated.
* Test scores are only a snapshot. There have been any number of schools, for example, who will drill their students in the lower grades only in reading and math, because that’s what they are being tested on. Those kids basically do nothing else all year. What kind of education are they getting?
* Other countries place almost no emphasis on standardized test scores when evaluating teachers. These countries look at what children do every day in the classroom, rather than what they can repeat parrot-fashion on a piece of paper in test conditions; they also look at peer reviews, students scores overall throughout the year, parent surveys and many other measures.
Since I teach world languages, which are not considered “core” and therefore not subject to standardized tests, this issue doesn’t affect me directly. Nor does it affect at least 50 percent of public school teachers, whose fields don’t fall into the “core” category.
How ridiculous it that? Haven’t we gone too far with the data?
Photo Credit: Thinkstock