VAM-pire Metrics and the Cost of Using Students as Weapons of Mass Instruction

It is generally obnoxious to say “I told you so,” but since I am often accused of being obnoxious even when I am not intending to be, I’ll risk this: I told you so.

While speaking recently to an education law class of doctoral students, I revisited my argument that using value-added methods (VAM) to evaluate, retain/fire, and reward teachers has an incredibly negative consequence: Teachers are being incentivized to use their students as mechanisms for their own job security and pay and to hope their students outperform other teachers’ students.

Being perhaps too snarky two years ago I framed this as using students as weapons of mass instruction.

I have always been disappointed that my point has gained little traction and even prompted a great deal of push back, but I am now pleased to direct everyone to a new analysis by Susan Moore Johnson, Harvard Graduate School of Education: Will VAMS Reinforce the Walls of the Egg-Crate School?

Johnson, I believe, makes a comprehensive and compelling case against VAM that in many ways reinforces my argument.

The analysis begins by acknowledging the rise in concern about teacher quality as well as offering a detailed challenge to overly simplistic considerations of teacher quality and the role it plays in student achievement.

Johnson notes that the political and bureaucratic rise in embracing VAM has received considerable push back by teachers, building to her focus for the analysis:

In this article I bring an organizational perspective to the prospect of using VAMS to improve teacher quality. I suggest why, in addition to VAMS’ methodological limitations, reformers should be very cautious about relying on VAMS to make decisions that teachers view as important. Because the wide-scale use of VAMS is very recent, scant research exists with which to answer the organizational questions about the intended and unintended consequences of using VAMS in making consequential staffing decisions. However, relevant research about teachers and school improvement, coupled with my own experiences working with states and districts that are implementing new teacher evaluations, lead me to suggest that expanding the use of VAMS in teacher evaluations (even if it represents no more than 30% of the teacher’s total score) might compromise the school’s potential for improvement.

Instead of focusing on individual teacher quality, Johnson argues for a more comprehensive approach:

Recent studies have persuasively documented the benefits of systematic efforts to improve student learning through schoolwide improvement initiatives (Bryk, Sebring, Allensworth, Easton, & Luppescu, 2010; Little, 1982; McLaughlin & Talbert, 2001;Rosenholtz, 1989). Because students move through schools from class to class or grade to grade, they are better served when human resources are deliberately organized to draw on the strengths of all teachers on behalf of all students, rather than having students subjected to the luck of the draw in their classroom assignment. Successful schoolwide improvement increases norms of shared responsibility among teachers and creates structures and opportunities for learning that promote interdependence—rather than independence—among them. By contrast, a strategy for school improvement that focuses substantially on identifying, assigning, and rewarding or penalizing individual teachers for their effectiveness in raising students’ test scores depends primarily on the strengths of individual teachers.

Broadly, evidence shows that schools should function within a culture of collaboration and not competition (another “I told you so,” sorry).

The unintended consequences, then, far outweigh the claimed advantages of measuring, identifying, and retaining “good” teachers. Johnson identifies the following unintended consequences:

Therefore, heavy reliance on VAMS may lead effective teachers in high-need subjects and schools to seek safer assignments, where they can avoid the risk of low VAMS scores. Meanwhile, some of the most challenging teaching assignments would remain difficult to fill and likely be subject to repeated turnover, bringing steep costs for students….

Given that reality, it seems possible that when districts rely on VAMS for a substantial part of teachers’ evaluations, their teachers may pull back from sharing collegial responsibility for the students in a school….

As more states require that teacher evaluations include both classroom observations and data from standardized tests of student achievement, principals who are not instructional experts will be left to interpret discrepancies between what they see in the classroom and what they read on a VAMS score sheet. Will they doubt the validity of their observations or the accuracy of the VAMS score? If they are uncertain about judging instruction or believe VAMS to be more objective and precise than their own professional judgment, value-added scores may unduly influence their decisions….

In line with Coleman’s (1988) theory about social capital, this suggests that a school would do better to invest in promoting collaboration, learning, and professional accountability among teachers and administrators than to rely on VAMS scores in an effort to reward or penalize a relatively small number of teachers.

Ultimately, commitments to VAM prove to be based on ideology, including a conscious effort to ignore evidence—a great irony considering VAM advocates are calling for dat-driven teacher quality policies.

Addressing the inequity in every child’s life outside of school and also recalibrating the culture of school—toward collaboration and away from competition—would both reap great rewards for children and society.

Plowing forward with VAM will, I guarantee, continue to bleed our field and our schools dry.

For Further Reading

The Constitutional Challenge to Teacher Tenure, Derek W. Black

Merit pay could revive child labor, Stephanie Jones

One comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s