First, Do No Harm: That Includes the Media

Public education in the U.S. suffers under a powerful intersection of politics, the media, and the public. As I have too often documented, misinformation tends to be reinforced among all three of these forces.

The role of media, as Alfie Kohn has examined, is central to perpetuating not only misleading beliefs about school quality and education reform but also bad policy.

For example, in South Carolina, The Greenville News has posted an editorial position on reading legislation that misreads what is the best path forward for addressing literacy in this high-poverty and deeply segregated state, notably:

The core part of the legislation makes common sense and is widely supported. It would make it mandatory to retain any third-grader who is not proficient in reading by the end of the school year. It is an idea that has been implemented in Florida to promising results, and it simply makes sense. Promoting a child to fourth-grade if he or she lacks the needed reading skills dooms that child to failure. Although holding back a student can have negative effects on him or her, that student certainly will do better if educators ensure he or she first knows how to read before advancing past third grade.

While we may need some proof that retention of third graders based on high-stakes test scores is in fact “common sense,” it seems true that retention is “widely supported.” The problem with those justifications is that four decades of research strongly rejects retention and close analysis of Florida’s Just Read, Florida, policies discredits claims of its “promising results” (see a full analysis with extensive evidence of the research base on both here).

A point of flawed logic also drives this editorial position: Retention and promotion are not the only options available, and are thus a false dichotomy that likely sits beneath the reality that retention is “widely supported” by the public and then political leaders (see how narrow choices create a false narrative that an issue is supported in the fourth paragraph from the end here).

Further, embedded in this misunderstanding of the research base on retention are careless claims about Florida’s education reform success as well as the current understanding of reading/ literacy instruction and development.

Florida, in fact, is a lesson in what not to do in terms of education policy (see HERE, HERE, and HERE).

Reading/ literacy instruction has been eroded by the accountability era based on standards and high-stakes testing. Literacy is misrepresented by multiple-choice testing, and teaching to those tests greatly warps good literacy instruction.

Test-reading is almost nothing like reading in the real world.

Instead of misguided reading policies reinforced by uninformed media endorsements, a few important and grounding commitments should be guiding reading policy:

  • First, do no harm. Allowing reading policy to be linked to harmful retention policies is inexcusable.
  • Relieve literacy policy from the accountability machine. Authentic, rich, and holistic literacy is eroded by focusing on isolated and skills-based instruction and testing.
  • Recognize that literacy is deeply linked to social class. Unless some powerful efforts are made to address poverty and inequity, students from poverty will remain mislabeled as “bad students” in school and then mis-served in those schools by being funneled into skill-and-drill classes serving the mandates to raise test scores.
  • Set aside the crisis discourse and policies related to literacy. Treating third grade like an Emergency Room ensures that students in most need of patient and rich learning environments will continue to be offered emergency care, and thus once again will be cheated.
  • Embrace low-cost and evidence-based practices that will guarantee literacy growth by increasing student access to books in their lives and their schools: “Perhaps the most serious problem with current literacy campaigns is that they ignore, and even divert attention from, the real problem: Lack of access to books for children of poverty,” explains Stephen Krashen.

Literacy growth is a natural part of being human. Children in middle-class and affluent homes (and thus likely to be enrolled in schools and classes that reinforce rich and authentic literacy) enjoy the sort of experiences with literacy all children deserve.

Retaining third graders based on high-stakes testing will further perpetuate inequity and erode opportunities for children living in poverty to experience rich and authentic learning environments with texts that would result in the type of literacy growth associated with privileged children.

A final problem with the media’s endorsement of “common sense” and “widely supported” education policy is the recurring call for compromise (see Cindy Scoppe at The State), also expressed in The Greenville News editorial:

The important work this fall and heading into the next legislative session is for education leaders and lawmakers to get on the same page about how such an effort would be implemented. There is plenty of room for common ground on this issue….

When policy is evidence-based (as it should be), a compromise between positions that are not evidence based and positions that are evidence based results in flawed policy. In other words, compromise between wrong and right can only result in wrong.

Yes, I recognize that politics is the realm of compromise, but I also believe therein lies the great failure of politics for setting education policy.

In the end, then, much could be solved if we kept our focus on first, do no harm instead of seeking always compromise as the basis for decisions on education policy.

What We Know Now (and How It Doesn’t Matter)

Randy Olson’s Flock of Dodos (2006) explores the evolution and Intelligent Design (ID) debate that represents the newest attack on teaching evolution in U.S. public schools. The documentary is engaging, enlightening, and nearly too fair considering Olson admits upfront that he stands with scientists who support evolution as credible science and reject ID as something outside the realm of science.

Olson’s film, however, offers a powerful message that rises above the evolution debate. Particularly in the scenes depicting scientists discussing (during a poker game) why evolution remains a target of political and public interests, the documentary shows that evidence-based expertise often fails against clear and compelling messages (such as “teach the controversy”)—even when those clear and compelling messages are inaccurate.

In other words, ID advocacy has often won in the courts of political and public opinion despite having no credibility within the discipline it claims to inform—evolutionary biology.

With that sobering reality in mind, please identify what XYZ represents in the following statement about “What We Know Now”:

Is there a bottom line to all of this? If there is one, it would appear to be this: Despite media coverage, which has been exceedingly selective and misrepresentative, and despite the anecdotal meanderings of politicians, community members, educators, board members, parents, and students, XYZ have not been effective in achieving the outcomes they were assumed to aid….

This analysis is addressing school uniform policies, conducted by sociologist David L. Brunsma who examined evidence on school uniform effectiveness (did school uniform policies achieve stated goals of those policies) “from a variety of data gathered during eight years of rigorous research into this issue.”

This comprehensive analysis of research from Brunsma replicates the message in Flock of Dodos—political, public, and media messaging continues to trump evidence in the education reform debate. Making that reality more troubling is that a central element of No Child Left Behind was a call to usher in an era of scientifically based education research. As Sasha Zucker notes in a 2004 policy report for Pearson, “A significant aspect of the No Child Left Behind Act of 2001 (NCLB) is the use of the phrase ‘scientifically based research’ well over 100 times throughout the text of the law.”

Brunsma’s conclusion about school uniform policies, I regret to note, is not an outlier in education reform but a typical representation of education reform policy. Let’s consider what we know now about the major education reform agendas currently impacting out schools:

Well into the second decade of the twenty-first century, then, education reform continues a failed tradition of honoring messaging over evidence. Neither the claims made about educational failures, nor the solutions for education reform policy today are supported by large bodies of compelling research.

As the fate of NCLB continues to be debated, the evidence shows not only that NCLB has failed its stated goals, but also that politicians, the media, and the public have failed to embrace the one element of the legislation that held the most promise—scientifically based research—suggesting that dodos may in fact not be extinct.

* Santelices, M. V., & Wilson, M. (2010, Spring). Unfair treatment? The case of Freedle, the SAT, and the standardization approach to differential item functioning. Harvard Educational Review, 80(1), 106-133.; Spelke, E. S. (2005, December). Sex differences in intrinsic aptitude for mathematics and science? American Psychologist, 60(9), 950-958; See page 4 for 2012 SAT data: http://media.collegeboard.com/digitalServices/pdf/research/TotalGroup-2012.pdf

Just Say No to Just Read, Florida, South Carolina

“Nearly 3,000 of South Carolina’s third-graders who struggle with reading could be held back if the state adopts a plan that would require students to read at or near grade level,” Jamie Self reports in The State (“Education leaders: Florida holds key to SC literacy fix,” March 31, 2013), adding later,

Under Peeler’s bill, 2,886 SC third-graders scored low enough on a 2012 reading test to be held back a year — more than four times the 584 third-graders held back in 2012. The number reflects only the lowest-scoring readers, about 27 percent of the more than 10,500 third-graders who scored as not reading on grade level….

SC schools chief Mick Zais supports the plan to hold back struggling third-graders and would like to see a similar track for seventh graders, his spokesman Jay Ragley said.

SC Senate Majority Leader Harvey Peeler, R-Cherokee, has introduced the bill modeled on Just Read, Florida, part of a grade-retention policy widely promoted by Jeb Bush as one aspect of the larger so-called “Florida Miracle.”

SC political leadership must not follow Florida’s lead in reading policy or grade retention policy for several reasons, including the following: the “Florida Miracle” has been thoroughly discredited, grade retention has no support in the research that shows retention has no positive outcomes but many negative consequences for children and tax payers, and initiatives such as Just Read, Florida ignore and replace credible literacy policy desperately needed in high-poverty states such as SC.

First, SC political leadership and the public must acknowledge what low reading test scores represent and the negative consequences of basing policy on test data:

  • All standardized test scores, including reading scores, are overwhelmingly a reflection of home, community, and school inequities more so than direct and clear evidence of holistic reading ability by children. Low reading scores by third graders in SC are signals of a high-poverty state, first and foremost.
  • Reading test scores are often poor evidence of real-world and holistic reading ability. Standardized tests of reading are typically skills-based and/or complicated by other student skills imbedded in the test format (for example, having students write on a reading test blurs the evidence on reading and writing proficiencies).
  • High-stakes test-based decisions for grade retention, instructional programs, and instructional practices—especially in literacy—have a clear negative impact on the quality of instruction as well as the quality of learning opportunities children who need education the most are likely to receive:

In too many places, however, graduation and promotion tests are putting many students at sharply increased risk of suffering the serious, well-documented harm associated with grade retention and denial of high school diplomas. Those at greatest risk include the very populations—students of color, students with disabilities, English-language learners, and low-income students—whom standards-based reform could potentially help the most. (Heubert, 2002/2003)

Second, SC political leadership and the public must acknowledge that the “Florida Miracle”—like the “Texas Miracle,” the “Harlem Miracle,” and the “Chicago Miracle”—has been thoroughly discredited as incomplete data, misrepresented accomplishments, or outright failures.

A brief review of credible examinations of the “Florida formula” include strong cautions about both the claims of success and the use of Florida as a model of reform.

Matthew Di Carlo acknowledges mixed results, but cautions using the formula to drive policy:

That said, the available evidence on these policies, at least those for which some solid evidence exists, might be summarized as mixed but leaning toward modestly positive, with important (albeit common) caveats. A few of the reforms may have generated moderate but meaningful increases in test-based performance (with all the limitations that this implies) among the students and schools they affected. In a couple of other cases, there seems to have been little discernible impact on testing outcomes (and/or there is not yet sufficient basis to draw even highly tentative conclusions). It’s a good bet – or at least wishful thinking – that most of the evidence is still to come.

In the meantime, regardless of one’s opinion on whether the “Florida formula” is a success and/or should be exported to other states, the assertion that the reforms are responsible for the state’s increases in NAEP scores and FCAT proficiency rates during the late 1990s and 2000s not only violates basic principles of policy analysis, but it is also, at best, implausible. The reforms’ estimated effects, if any, tend to be quite small, and most of them are, by design, targeted at subgroups (e.g., the “lowest-performing” students and schools). Thus, even large impacts are no guarantee to show up at the aggregate statewide level (see the papers and reviews in the first footnote for more discussion)….

Whether we like it or not, real improvements at aggregate levels are almost always slow and incremental. There are no “miracles,” in Florida or anywhere else. The sooner we realize that, and start choosing and judging policies based on attainable expectations that accept the reality of the long haul, the better.

Julian Vasquez Heilig identifies the error of focusing on apparent increases in 3rd to 4th grade reading scores and associating them with policy:

Cloaking Inequity examined the purported test score miracle earlier here. In 4th grade, Florida improved over the last decade and was position in the top ten nationally, but as you move up the grade levels, the longer student stay in Florida schools, the worse their performance relative to the nation. I also discussed the official Florida scholarship evaluation in Florida that showed their scholarship (aka neovoucher) program had not increased the achievement of program participants.

Third, all evidence on grade retention reveals only negative consequences for children (academic and emotional) and tax payers, the public. [1]

Alfie Kohn notes that accountability “get tough” attitudes are masking the need for policy to be built on evidence:

The same get-tough sensibility that has loosed an avalanche of testing has led to a self-congratulatory war on “social promotion” that consists of forcing students to repeat a grade. The preponderance of evidence indicates that this is just about the worst course of action to take with struggling children in terms of both its academic and social-psychological effects. And the evidence uniformly demonstrates that retention increases the chance that a student will leave school; in fact, it’s an even stronger predictor of dropping out than is socioeconomic status.

Some of the well-documented effects of grade retention include the following:

Retained students are more likely to be male, younger than classmates, from a lower socio-economic class, black or Hispanic, a behavior problem and immature (Karweit, 1991).

Research shows a large correlation between dropouts and retention….

Controlled studies do not support the benefits claimed for extra-year programs (i.e., transitional first, pre-kindergarten) and negative side effects occur just as they do for retention in later grades….

Empirical research shows retention does not improve the achievement of children as measured by tests of basic skills. No significant positive long-term effect is evident. Studies indicate retention is either ineffective or harmful, with more negative than positive effects….

Retention imposes an economic burden of financing an extra year of schooling….

Children attach stigma, stress, and shame to retention.

Kevin Welner states directly and clearly that no policy decisions should include grade retention, specifically citing Florida’s policies:

Let’s use grade retention to illustrate. States across the U.S. are adopting mandates requiring that  third graders with low reading scores repeat the grade. The ‘leave the student back’ policy is being heavily marketed by the Foundation for Excellence in Education, an organization created by former Florida governor Jeb Bush. But retaining students is not a new idea. It’s an experiment that’s been tried on and off for generations, and it’s been studied for almost that long.

The overarching message from research in this area is that retaining a low-scoring third grader will not help her do better than a similar classmate with similar scores who is moved along to fourth grade, but she will be more likely to eventually drop out.

Viewed from a taxpayer perspective, retaining a student will likely have one of two outcomes:

1. She may drop out, meaning she will pay about $60,000 less in taxes over her lifetime, be more likely to commit crimes, and be more likely to depend on government assistance; or

2. She may complete high school, at a cost of an extra year of school – about $10,000. If retention had a substantial payoff, paying for an extra year of school would be worthwhile (although it nationally adds up to billions of dollars each year). But there’s no benefit. With grade retention, we are paying more and getting a worse outcome.

That’s the evidence. It’s what we have learned (or should have learned) from decades of  experience. Grade retention can be expected to have the same destructive results in 2012 as it did when it was tried ten or twenty or forty years ago – or any of the years in between. Yet our lawmakers do the same thing over and over again, each time expecting different results.

Allensworth (2005) found that retention based on test scores at the elementary level contributed to higher drop-out rates; this study is particularly important to evaluating the long-term impact of the Florida formula as well as any state considering implementing similar policies that focus on short-term test scores while ignoring long-term consequences.

Grade retention fails students retained, fails taxpayers, and as Gottfried (2013) has shown, fails the non-retained classmates of students retained:

This research has brought to the foreground a new dimension of the analysis relating to grade retention, as well as peer effects. In essence, by demonstrating a pervasive negative effect of retained classmates, as derived from multiple quasi-experimental methods on a panel data set of urban schoolchildren, this study corroborates the predominant conclusions of prior research on retention. Namely, this practice of grade retention is associated with negative outcomes (Allen et al. 2009; Holmes 1989; McCoy and Reynolds 1999; Meisels and Liaw 1993; Reynolds 1992; Roderick and Nagaoka 2005). The unique contribution of this study is that the negative outcomes of this schooling practice are not restricted to the retained students themselves; rather, this study has found that retention can affect the academic outcomes for other members of the classroom. Moreover, this study has also facilitated an opportunity for urban educational experiences to be further delineated and for policy implications of this practice to be more thoroughly discussed.

Finally, SC political leadership needs to address in authentic and effective ways the very real literacy challenges faced by our high-poverty students. Modeling SC educational policy on Florida, however, would achieve only what Kohn has identified as policy insuring more non-readers.

Credible reading policy in SC would include the following:

  • Create and implement social policy that addresses poverty and job creation/stability in the historically high-poverty areas of the state. Children living in high-poverty homes and communities are “doubly disadvantaged” in ways that cannot be overcome by schools alone and that are too often reflected in and perpetuated by community-based schools and failed educational policies such as Florida’s Just Read, Florida.
  • Reading and literacy achievement should be evaluated through holistic, classroom-based mechanisms, not high-stakes testing. Increasing high-stakes test scores in reading may have the unintended consequence of producing both misleading data and further eroding the literacy proficiency of high-poverty, minority, special needs, and ELL students. Test-based evaluations of students, teachers, schools, and policy tend to create conditions that ask less of everyone, not more.
  • Reading proficiency is of little value if students are non-readers due to punitive and skills-based school policy. Reading is a holistic and unpredictable human behavior that must be fostered over many years and through strategies that appear “too simple”—increase children’s access to books in their homes, increase children’s access to books in school, provide students extended opportunities to read by choice during the school day, address adult and community literacy
  • Reading proficiency and creating life-long, eager readers will never be achieved by reading policies or programs, particularly pre-packaged commercial programs. Teachers and parents can and should foster reading and all literacy, but ultimately children cannot be bribed, forced, or punished into being readers.

As Welner concludes in his consideration of education reform:

To be clear, “social promotion” – the movement of students from grade to grade with no meaningful intervention for those who fall behind – is also not supported by research evidence. Instead, as proven approaches to address the problem of early reading gaps, research supports high-quality early-childhood education, intensive early reading interventions, and smaller class sizes in early grades for at-risk students. These are all less costly and more effective than grade retention.

Evidence supports grade promotion combined with these sorts of interventions, and it clearly cautions against a systemic use of grade retention, even retention combined with additional academic support.

A reckless disregard of evidence is harmful. It leads to the waste of precious resources: our tax dollars and our children themselves. And grade retention is only one example of the larger problem.

SOURCES

“Florida Miracle” 

Review of Closing the Racial Achievement Gap, Madhabi Chatterji

Water into Wine?, Julian Vasquez Heilig

Lurking in the Bushes, Julian Vasquez Heilig

Parsing the Florida “Miracle,” Diane Ravitch

The Test-Based Evidence on the “Florida Formula,” Matthew Di Carlo

Editorial: Florida needs no advice from Jeb Bush on education policy, Jac Versteeg

Review of Getting Farther Ahead by Staying Behind, Derek C. Briggs

How Jeb Bush’s school reforms really played out in Florida, interview with Sherman Dorn

Grade Retention: U.S.

Third Grade Reading Policies, Rose (2012)

Education in Two Worlds: Follow Up to “50 Myths & Lies,” Gene V. Glass

A second myth we see as dangerous has that quality because of what it reveals about too many of America’s politicians and school leaders: it reveals both their ignorance and their cruelty! This is the myth that leaving a child back in grade who is not doing well academically is good for the child. It provides the child with “the gift of time” to catch up. We believe that only ignorant and cruel people would support such a policy, although it is law in about a dozen states, including Arizona and Florida. First of all, a large and quite consistent set of research studies, many of excellent quality, point out that for the vast majority of the children retention in grade has either no benefit, or is detrimental. Only rarely does retention benefit the child who was left back. So the research overwhelming suggests that those who recommend retention are likely to be ignorant. Second, an important piece of the rationale for retention policies is that if you cannot read well by third grade you are more likely to be a school failure. But reading expert Stephen Krashen disputes this, citing research on 12 young students with serious reading problems, dyslexics all. Eleven of the twelve did not learn to read well until they were between 10 and 12 years of age, and one did not learn to read until he was in 12th grade. Among these slow learners, all of whom would have been left back in Florida and Arizona, were nine who published creative scholarly works, and one who became a Nobel laureate. So not doing well by third grade does not determine one’s destiny. Third, the research informs us that retention policies are disproportionately directed at those who are poor, male, English language learners, and children of color. Middle class white children are rarely left back. Fourth, a retention decision changes family dynamics. Parents and siblings change in their treatment of, and aspirations for, the child identified by the school as having “flunked.” Of course, the schools do not say a child is dumb. Instead they offer the children and the families “the gift of time” to catch up. But the world interprets that gift more cruelly. Fifth, being left back is associated with much higher rates of dropping out before completion of high school. Thus, the social costs of this policy go way up since these children are more likely to need assistance in living because of poor wage earning capacity, and there is also the greater likelihood of a higher incarceration rate for people that do not finish school and cannot find decent work. Sixth, when surveyed, children left back say it feels as bad as losing a parent or going blind. It is an overwhelmingly negative event in the lives of the vast majority of the retained children, so leaving them back is cruel as well as a reflection of the ignorance of those who promote these policies. Seventh, and finally, the same costs expended for an extra year of education for the child who is held back, say eight thousand dollars, could more profitably be spent on a more beneficial treatment than repetition of a grade. A certified reading specialist, working twice a week as a tutor throughout the school year and for some part of the summer, would have greater success in improving a child’s academic performance. There is no more powerful treatment than tutoring, and in this case it is cheaper and more humane than is flunking a child. For the seven reasons given, we can think of no education policy that reflects worse on America’s politicians and educators than the policy of retaining students in grade.

Hold Back to Move Forward? Early Grade Retention and Student Misbehavior, Umut Özek

Test-based accountability has become the new norm in public education over the last decade. In many states and school districts nation-wide, student performance in standardized tests plays an important role in high-stakes decisions such as grade retention. This study examines the effects of grade retention on student misbehavior in Florida, which requires students with reading skills below grade level to be retained in the 3rd grade. The regression discontinuity estimates suggest that grade retention increases the likelihood of disciplinary incidents and suspensions in the years that follow. The findings also suggest that these adverse effects are concentrated among economically disadvantaged students

Retaining Students in Grade A Literature Review of the Effects of Retention on Students’ Academic and Nonacademic Outcomes, Nailing Xia, Sheila Nataraj Kirby (2009)

Our review of these 91 studies indicates that grade retention is associated with gender, race, SES, age for grade, student mobility, family and parental characteristics, prior academic achievement, prior behavioral and socioemotional development, and student health. Converging evidence suggests that grade retention alone is not an effective intervention strategy for improving academic and longer-term life outcomes. In general, retention does not appear to benefit students academically. Although some studies have found academic improvement in the immediate years after retention, these gains are usually short-lived and tend to fade over time. Past research has consistently shown that retained students are at significantly increased risk of dropping out of school. Although only a few studies have examined the effects of retention on postsecondary outcomes, the available evidence suggests negative effects on enrollment in postsecondary education and on employment outcomes in adulthood. Overall, the literature indicates mixed findings on attitudinal, socioemotional, and behavioral outcomes among the retained students….Our review found fruitful avenues of research, most notably the impact of supportive interventions (such as early identification of at-risk students, academic instructional services provided in and out of school, and different types of intervention strategies) on proximal and future student outcomes.

The Spillover Effects of Grade-Retained Classmates: Evidence from Urban Elementary Schools, Michael A. Gottfired, American Journal of Education 119 (May 2013)

Retention, Social Promotion, and Academic Redshirting: What Do We Know and Need to Know?, Nancy Frey, Remedial and Special Education, volume 26, number 6, November/December 2005, pages 332-346

The evidence gathered in the last 30 years on the practice of retention suggests that it is academically ineffective and is potentially detrimental to children’s social and emotional health. The seeds of failure may be sown early for students who are retained, as they are significantly more likely to drop out of high school. Furthermore, the trajectory of adverse outcomes appears to continue into young adulthood, when wages and postsecondary educational opportunities are depressed.

Dropout Rates after High-Stakes Testing in Elementary School: A Study of the Contradictory Effects of Chicago’s Efforts to End Social Promotion, Elaine M. Allensworth, Educational Evaluation and Policy Analysis, Vol. 27, No. 4 (Winter, 2005), pp. 341-364

Alternative to Grade Retention, Jimerson, Pletcher, and Kerr (2005)

Given the accumulating evidence that grade retention is an ineffective and possibly harmful intervention, it is imperative that school administrators advocate for “promotion plus” policies that depend on effective, evidence-based interventions. The issue for secondary school educators is twofold. Not only must educators determine whether retention is appropriate for a given student, they also need to address the negative academic, social, and emotional consequences for students who were retained in earlier grades. Very often the student’s original difficulties persist, or more likely worsen, as their school career progresses.

Winning the Battle and Losing the War, Jimerson, Anderson, and Whipple (Psychology in the Schools, Vol. 39(4), 2002)

Considering the results of this review of research examining the association between grade retention and high school dropout and other reviews of research addressing the efficacy of grade retention (Holmes, 1989; Jimerson, 2001a, 2001b; Smith & Shepard, 1987, 1988), we must move beyond the use of grade retention as an intervention strategy and attempt to implement those strategies research has demonstrated to be effective (Jimerson, 2001a). Educational professionals, researchers, parents, and policymakers would be remiss to overlook the implications of research that demonstrate the association between grade retention and school dropout. Furthermore, a new imperative has emerged, where the onus is on programs training future educational professionals to disseminate the results of the recent research presented in this review. It is crucial that we transcend limited solutions and begin to consider student developmental and achievement trajectories in order to reinforce and strengthen pathways that promote social and cognitive competence and lead to academic success.

Does Retention (Repeating a Grade) Help Struggling Learners?

Some stakeholders in Florida believe that the “hard line in the sand” created by mandatory, test-based retention created a motivational difference in teachers and parents…, since it is thought that many of the same learning supports were being provided to struggling students prior to the policy. This may be the case for test score gains close to the retention year, but given the well-known longer-term negative effect of retention on drop-out rates (e.g., Allensworth, 2005) as well as the assured delayed entry into the workforce, Florida’s evidence falls far short of even suggesting that retention is the only or best way to motivate a real positive difference for struggling students, nor has it contradicted the overwhelming evidence against retention prior and since.

What Doesn’t Work, Smith and Shepard (Phi Delta Kappan, October 1987)

The Lesson of the Cupcakes: Fixing Schools by Resisting Gimmicks and Heeding Evidence, Kevin Welner

Exploring the Association Between Grade Retention and Dropout, Jimerson, et al. (The California School Psychologist, Vol. 7, pp. 51-62, 2002)

Ultimately, the research is unequivocal in identifying that grade retention does not appear to address the needs of these students at risk of academic failure. Findings from this study should not be misinterpreted as an indication that retention was an effective intervention strategy for the retained students who did not drop out of high school. There is a need for further research comparing the retained students who completed high school with matched comparison groups of similarly low achieving but socially promoted students. This study highlights the association of early socio-emotional and behavioral adjustment and high school dropout among a group of retained students. These findings have direct implications for school psychologists and other educational professionals. In particular, rather than focusing on the unsupported academic intervention of grade retention, it is time to implement prevention and intervention programs that have been empirically demonstrated to meet the needs of these students in facilitating both positive academic success and socio-emotional adjustment.

Grade Retention: A Flawed Education Strategy, Xia and Glennia (part 1)

Decades of research suggest that grade retention does not work as a panacea for poor student performance. The majority of research fails to find compelling evidence that retention improves long-term student achievement. An overwhelmingly large body of studies have consistently demonstrated negative academic effects of retention. Contrary to popular belief, researchers have almost unanimously found that early retention during kindergarten to grade three is harmful, both academically and emotionally. [2] Many studies find that retention does not necessarily lead to increased work effort among students as predicted.

Cost-Benefit Analysis of Grade Retention, Xia and Glennia (part 2)

Grade Retention: The Gap Between Research and Practice, Xia and Glennia (part 3)

The majority of published studies and decades of research indicate that there is usually little to be gained, and much harm that may be done through retaining students in grade. Yet, many educators continue to use retention as a way to improve student achievement and claim that it produces positive results. The consequence is while a growing body of studies show that retention does not improve academic performance and has a number of negative side effects, more and more states and school districts have adopted retention policy in an effort to enhance the educational accountability.

Synthesis of Research on Grade Retention, Shepard and Smith (Educational Leadership, May 1990)

Grade Retention [a synthesis]

Social Promotion – In Comparison to Grade Retention, Advantages and Disadvantages, Different Perspectives, Jere Brophy

Meta-analysis of Grade Retention Research: Implications for Practice in the 21st Century, Shane R. Jimerson (School Psychology Review, 2001, Volume 30, No. 3, pp. 420-437)

A Synthesis of Grade Retention Research: Looking Backward and Moving Forward, Shane R. Jimerson (The California School Psychologist, Vol. 6, pp. 47-59, 2001)

In looking backwards at the retention research and previous reviews and meta-analyses, a consistent theme emerges—grade retention is not an empirically supported intervention. As reflected in the results of the three meta-analyses described above, the confluence of results from research during the past century fails to demonstrate achievement, socioemotional, or behavioral advantages of retaining students. Moreover, the research consistently demonstrates that students who are retained are more likely to drop out of high school.

Evaluating Kindergarten Retention Policy, Hong and Raudenbush (September 2006)

First, Do No Harm, Jay P. Heubert (Educational Leadership, December 2002/January 2003)

The Facts on Education: Should Students Be Allowed to Fail Grades?

Holding Kids Back Doesn’t Help Them, Deborah Stipek and Michael Lombardo

Grade Retention and Social Promotion, National Association of School Psychologists

For children experiencing academic, emotional, or behavioral difficulties, neither repeating the same instruction another year nor promoting the student to the next grade is an effective remedy. (p. 5)

Grade Retention: International

Early Academic Performance, Grade Repetition, and School Attainment in Senegal: A Panel Data Analysis (Senegal)

Grade retention and educational attainment (Belgium)

Grade retention and its association with school misconduct in adolescence (Flemish)

This study represents one of the few to investigate the effect of grade retention on students’ school-disruptive behavior in adolescence. It is unique in addressing multilevel issues in this line of research. First, it has shown that it is important to distinguish grade retention at different educational levels. While we find evidence that primary school retention may be associated with less misconduct in adolescence, we establish that secondary school retention may give rise to deviance in adolescence. Moreover, we address the important role of schools’ retention composition, finding that students attending schools with more retainees are more likely to be deviant, although this composition does moderate negative retention effects. Together with previous literature on the effectiveness of grade retention, we advocate the abandonment of this intervention, especially at the secondary level.

See Demanet and Van Houtte’s cites:

In many countries, the practice of grade retention is widespread (Switzerland: Bonvin, Bless, & Schuepbach, 2008; Germany:  Ehmke, Drechsel, & Carstensen, 2010; US:  Jimerson, 2001; Lorence & Dworkin, 2006; Canada:  Pagani, Tremblay, Vitaro, Boulerice, & McDuff, 2001; Belgium:  Juchtmans et al., 2011; Van Petegem & Schuermans, 2005). Proponents believe that giving students ‘‘the gift of time’’ will put them back on track for normal educational growth. Ensuing the popularity of this strategy, a rich body of research has developed to test its effectiveness. The practice has some positive effects on students’ cognitive growth (Alexander, Entwisle, & Dauber, 1994), but these remain mainly short term (Jimerson & Ferguson, 2007; Meisels & Liaw, 1993) and occur only when special help is provided to retained children. In fact, in recent decades many studies have condemned grade retention as an ineffective practice to improve student learning (see, e.g., Bonvin et al., 2008; Jimerson, 2001; McCoy & Reynolds, 1999; Pagani et al., 2001).

Relation of Academic History and Demographic Variables to Grade Retention in Lebanon

[1] See Krashen, S. (2013, May). Need Children Read ‘Proficiently’ by Grade Three? Language Magazine.

[2] Deborah A. Byrnes, and Kaoru Yamamoto, 1985, “Academic Retention of Elementary Pupils: An Inside Look,” Education, 106(2), 208-14; Peg Dawson, 1998, “A Primer on Student Grade Retention: What the Research Says,” NASP Communique, 26(8); Shane R. Jimerson et al., 1997, “A Prospective, Longitudinal Study of the Correlates and Consequences of Early Grade Retention,” Journal of School Psychology, 35(1), 3-25; Panayota Y. Mantzicopoulos, 1997, “Do Certain Groups of Children Profit from Early Retention? A Follow-Up Study of Kindergartners with Attention Problems,” Psychology in the Schools, 34(2), 115-27; Samuel J. Meisels and Fong-Ruey Liaw, 1993, “Failure in Grade: Do Retained Students Catch Up?” Journal of Educational Research, 87(2), 69-77; Judy Temple, Arthur Reynolds and Suh-Ruu Ou, 2001, “Grade Retention and School Dropout: Another Look at the Evidence,” The CEIC Review, 10(5), 5-6 & 21; Charles L. Thompson and Elizabeth K. Cunningham, 2000, “Retention and Social Promotion: Research and Implications for Policy,” Eric Clearinghouse on Urban Education Digest, 161, 1-5; Deneen M. Walters and Sherry B. Borgers, 1995, “Student Retention: Is It Effective?” School Counselor, 42(4).

“A Separate and Unequal Education System” 2013

The Education Trust-West has released At a Crossroads: A Comprehensive Picture of How African-American Youth Fare in Los Angeles County Schools (February 2013), highlighting:

Nearly 60 years after Brown v. Board of Education, too many of California’s African-American students languish in a separate and unequal education system. If current trends continue, only 1 in 20 of today’s African-American kindergartners will go on to graduate from high school and complete a degree at a four-year California university. Indeed, on nearly every measure of educational opportunity, the dream of equal access to a high-quality education is not a reality for African-American students and their families in California. (p. 1)

Despite almost 60 years since desegregation of schools and almost 50 years since the Civil Rights Era in the U.S., the racial and socioeconomic inequities confronted by Malcolm XJames Baldwin, and Martin Luther King Jr. remain persistent in our society and schools in 2013. While educational outcomes such as test scores, graduation rates, college attendance, and college completion present often cited achievement gaps that must not be ignored, much less attention is paid to the powerful and corrosive inequity of opportunity that still exists between African-American children and children of other races, as detailed in the ET-W report on Los Angeles.

African Americans have experienced a decline in their relative status as a minority race, as well as continued to experience socioeconomic inequity, but African-American students also disproportionately find themselves in either inequitable public school settings or charter schools, which also tend to segregate students:

African-American students used to be the third largest subgroup in L.A. County, making up about 12 percent of the student population in 1994. During the past two decades (from 1994 to 2011), however, the African-American population has been on the decline and is now only slightly larger than the Asian student population. Currently, 9 percent of students are African Americans and nearly three-quarters of these students are socioeconomically disadvantaged…. Of the African-American students enrolled in the public school system in L.A. County, the vast majority attend traditional public K-12 schools (94 percent), with the remaining 6 percent attending alternative schools of choice or continuation schools. Nearly 1 out of 6 (15 percent) attends one of L.A. County’s more than 300 charter schools, almost twice the rate of students overall. (p. 2)

One failure of the current education reform movement is focusing almost exclusively on in-school variables as well as school-related outcomes. For African-American students specifically, access to opportunities are a better place to look. Schools tend to mirror and replicate the inequity of the neighborhoods they serve; thus, “doubly disadvantaged” students from high-poverty homes and communities produce outcomes that represent the inequity of opportunity they face in the lives and schools—more so than their quality as students:

At the middle and high school levels, rates of participation and proficiency in math courses provide signals about college eligibility and readiness. Algebra I is a “gatekeeper” course for higher level math classes that students need to become eligible for admission to the University of California (UC) and California State University (CSU) systems. Yet Algebra I is effectively closed to many African-American middle school students in L.A. County. Only 60 percent of African-American students took Algebra I in the eighth grade in 2011-12. (p. 3)

For African-American students, separate-but-unequal persists, manifested in tracking and school-within-schools whereby race and class determine whether or not students enter Advanced Placement or International Baccalaureate courses as opposed to test-prep courses focusing on remediation and high-stakes accountability tests:

Unfortunately, African-American students in L.A. County graduate from high school at lower rates, are less likely to complete rigorous coursework while in high school, and are less ready for college-level coursework than their white peers. For every 100 African-American students who walk into a ninth-grade classroom in L.A. County, only 63 students leave high school four years later with a diploma in hand, and just 20 of them have completed the A-G course sequence that makes them eligible to attend a four-year public university in California. The outcomes are even worse for African-American male students: for every 100 African-American male students who enter ninth grade, just 58 graduate on time, and only 15 complete the A-G course sequence…. L.A. County high schools continue their practice of systematic tracking, whereby low-income students and students of color receive less rigorous coursework. For example, although African-American students make up 9 percent of L.A. County’s population, only 6 percent of students taking one or more Advanced Placement (AP) courses are African American….On the other hand, 22 percent of students taking at least one AP course are white, though they make up a smaller share of the overall student population. (p. 5)

If college readiness and college attendance/completion are genuine goals for all U.S. students regardless of background or race, then the gaps that remain in these goals must be traced back to the cumulative effect of access gaps existing in African-American children’s lives from birth and throughout their schooling:

The latest results reveal that the vast majority of African-American 11th-graders in L.A. County lack the skills necessary for college-level English and math work. In contrast, white students in L.A. County are three times more likely to be “ready for college-level work” in English and math…. 2 out of 5 African-American ninth-graders go to college five years later, lagging behind the rates of their white and Asian peers by 20 percentage points to more than 30 percentage points. (p. 6)

Inequity of educational opportunities for African-American students is paralleled by inequitable discipline policies and outcomes, including race-based inequities of the criminal justice system beyond the walls of schools. As Kathleen Nolan and Sarah Carr have shown, zero tolerance and no-excuses policies feed the school-to-prison pipeline and create schools-as-prisons:

Across California, nearly 1 out of every 5 African-American students (18 percent) was suspended at least one time, compared with 1 in 17 white students (6 percent). Suspension rates are slightly lower in L.A. County than the state average, but large gaps still exist: 15 percent of African-American students were suspended at least once, compared with 4 percent of white students…. The California Department of Justice reports that in L.A. County a much larger share of African-American students are arrested for felony charges than white students. Specifically, for every 1,000 youth ages 10-17, 38 African-American juveniles are arrested for felonies, as compared with 7 white youth. (p. 7)

While the education reform movement has argued that teacher quality drives student outcomes—an inaccurate claim—almost no attention has been paid to the inequitable distribution of teacher assignments that disadvantage students of color, ELL students, and special needs students:

These inequitable and often dismal outcomes are the result of many factors. In fact, this educational inequity is set in motion prior to elementary school. African-American children are more likely to grow up in poverty and enter school with critical educational disadvantages…. African-American children are less likely to access preschool than white children; and when they do, they are less likely to be taught by well-prepared teachers. In L.A. County, 59 percent of African-American three and four-year olds attend preschool, compared with 69 percent of white children. Across the state, just 13 percent of African-American children are estimated to be in preschool classrooms in which the lead teacher has at least an associate’s degree in early childhood education, compared with 41 percent for white and 42 percent for Asian children. (p. 8)

These inequities remain embedded in the rise of segregated schools in both traditional public schools and charter schools:

Although African Americans comprise a small percentage of the student population in L.A. County, they often attend schools where they are substantially overrepresented and that are intensely segregated (defined as schools where more than 90 percent of students come from underrepresented minority backgrounds)…. Research demonstrates that African-American students in high-poverty, high-minority schools receive less of everything we know matters most in education—from effective teachers and resources to sufficient interventions and supports. Students in intensely segregated schools are almost three times as likely to have a teacher lacking full qualifications than students attending majority white and Asian schools. And our own research finds that African-American students in LAUSD are less likely to be taught by highly effective teachers than their white or Asian peers. Such segregated schools often suffer from overcrowding, which creates unsafe and ineffective learning environments. (pp. 8-9)

Claims of a post-racial America, a meritocracy whereby each person’s success is the result of her or his “grit” are both factually untrue and terribly misleading as a message for children. The ET-W report ends with a sobering message:

More than 135,000 African-American students go to school in Los Angeles County, and far too many of these children and youth are underserved. Even before starting kindergarten, they are often disadvantaged by poverty, access to quality preschool, and a host of other factors. When they do enter the education system, they too frequently face school segregation, low academic expectations, insufficient resources, minimal educational and socioemotional supports that fail to leverage the assets they bring, and—dare we say it—racism that manifests itself in the form of over-identification for special education and more frequent suspension and expulsion, particularly among African-American male students. (p. 13)

Along with the ET-W report, I recommend some related reading:

“The Real Reason More Low-Income Students Don’t Go to College,” Sarah Carr

“The Fight for Accountability Continues for Trayvon Martin’s Family”

“Parents reflect: Trayvon Martin’s death is ‘lodged deep in our psyches'”

“School Police and Principals Forced to Undergo Trainings in Implicit Racism”

“Handcuffing and Interrogating a 7-Year-Old? The Police State Crashes Into America’s Schools”

“Black students’ learning gaps start early, report says”

Educators: “[N]ot the Time. . .to Follow the Line of Least Resistance”

In a major journal from the National Council of Teachers of English (NCTE), a teacher and scholar laments the current state of implementing the research in language: “A brief consideration will indicate reasons for the considerable gap between the research currently available and the utilization of that research in school programs and methods” (p. 87). [1]

And the discussion of that gap between research and classroom practices leads to this conclusion:

“Most thinking persons agree that the existence of civilized man is threatened today. While language is not food or drink, and will not satisfy the hungry and thirsty, it is the medium by which we must do much of our learning and panning, and by which we must think out solutions to our problems if we are not to solve them by the direct method of force. No sensible person believes that language will cure all difficulties; but the thoughtful person will certainly agree that language is a highly important factor in promoting understanding, and a most dangerous factor in promoting understanding between individuals and between the countries individuals represent. Moreover, language is a significant factor in the psychological adjustment of the individual. This is not the time for the teacher of any language to follow the line of least resistance, to teach without the fullest possible knowledge of the implications of his medium [emphasis added]. Before we, either as individuals or as a Council, experiment with methods of doing specific things or block out a curriculum, let us spend some time with the best scholars in the various fields of language study to discover what they know, what they believe uncertain and in need of study. Let us go to the best sources and study the answers thoughtfully. The game of Gossip is not for us.” (p. 94)

While those of us living our lives as teachers, especially teachers of literacy in K-12 settings or in teacher education, may recognize many points above in our current debates about education reform—including some of the debates that simmer below the surface of the workings of NCTE—this piece is by Lou LaBrant and was published in the January 1947 issue of Elementary English (now Language Arts).

More than six decades after LaBrant wrote about the gap between research and practice, More than six decades after she implores us that “[t]his is not the time for the teacher of any language to follow the line of least resistance,” educators across the U.S. are faced with the failure of leaders, the public, and professional organizations in the face of the promise of universal public education and its potential to drive the great hope we call democracy.

The Locus of Authority: Our Time for Resistance

At the 100th anniversary annual convention for NCTE in Chicago (November 2011), I presented during a panel on the Council’s century of leadership in the field of literacy—reading from the essay above by LaBrant and suggesting how she would have responded to the current calls for Common Core State Standards (CCSS), increased testing, intensified value-added methods (VAM) for teacher accountability linked to those tests, and accelerating mandates driving teacher preparation and accreditation of colleges and departments of education.

I know from my work as the biographer of LaBrant that she was a powerful voice for the professionalism, scholarship, and teacher autonomy—including herself and every teacher with whom she interacted. LaBrant, in fact, during the early 1930s when enrolled in her doctoral program at Northwestern University, faced pressure while teaching English to implement required reading lists, textbooks, and benchmark testing, all of which she knew to be flawed practices.

What did LaBrant do?

She fabricated lesson plans with her roommate, the foreign language teacher, and submitted them each week while practicing the pedagogy she embraced—student choice in what they read and wrote, holistic instruction and assessment of literacy. At the end of the year, LaBrant and her students (yes, in the early 1930s) faced end-of-course testing, and LaBrant’s students received top scores. Consequently, she was praised by the principal in front of the entire faculty for her dedication to the prescribed policies.

This tension between bureaucratic mandates that seek to shift the locus of authority (consider Freire’s distinction between “authoritarian” and “authoritative”) away from the teacher and within the standards and tests designed and prescribed by the state is not entirely new (except for the intensity), but neither is the need for teachers to own their autonomy, their professionalism—to be that resistance.

Also at the 2011 NCTE annual convention, a convention of celebration, Susan Ohanian, Stephen Krashen, Carol Mikoda, Bess Altwerger, Joanne Yatvin, and Richard J. Meyer proposed a resolution: NCTE will oppose common core standards and national tests. [2] This act of resistance, this act of teacher autonomy and professionalism resulted in what Catherine Gewertz in the Curriculum Matters blog at Education Week describes as: “The National Council of Teachers of English was asked by a group of its members to take a strong stand against the common standards, but it declined to do so.”

This is a time when political leaders, the public, and national organizations have abdicated their moral obligation to create and maintain universal public education for all children as a sacred trust between a free people and the promise of democracy.

As the faculty of Garfield High School (Seattle, WA) take a principled stand against MAP testing as a beacon of hope in the fog of corporate education reform, this is also a time for all educators to shine every light of our autonomy on what is right and what is wrong in the day-to-day pursuit of teaching children.

“This is not the time for the teacher of any [student] to follow the line of least resistance, to teach without the fullest possible knowledge of the implications of his medium.”

References

Freire, P. (2005). Teachers as cultural workers: Letters to those who dare to teach. Trans. D. Macedo, D. Koike, & A. Oliveira. Boulder, CO: Westview Press.

Freire, P. (1993). Pedagogy of the oppressed. New York: Continuum.

Freire, P. (1998). Pedagogy of freedom: Ethics, democracy, and civic courage. Trans. P. Clarke. Lanham, MD: Rowman and Littlefield Publishers, Inc.

LaBrant, L. (1947, January). Research in language. Elementary English, 24(1), 86-94.

[1] Originally posted at Daily Kos (November 21, 2011) and cross-posted at Truthout (November 28, 2011). Reposting here as a call for solidarity among educators inspired by the resistance of Garfield High School faculty (Seattle, WA) to the corrosive impact of MAP and other elements of high-stakes testing in U.S. education. The original piece has been revised.

[2] Revised resolution passed: Resolution Proposal to Support: No Confidence in United States Department of Education Secretary Arne Duncan

NFL again a Harbinger for Failed Education Reform?

During the impending NFL strike in 2011—the act of a union—I drew a comparison between how the public in the U.S. responds to unionization in different contexts:

“I am speaking about the possible NFL strike that hangs over this coming Super Bowl weekend: a struggle between billionaires and millionaires, which, indirectly, shines an important light on the rise of teacher and teacher union-bashing in the US. Adam Bessie, in Truthout, identifies how the myth of the bad teacher has evolved.”

Once again, the NFL is facing a situation that I believe and even hope is another harbinger of how education reform can be halted: A suit filed by the family of Junior Seau:

“The family said the league not only ‘propagated the false myth that collisions of all kinds, including brutal and ferocious collisions, many of which lead to short-term and long-term neurological damage to players, are an acceptable, desired and natural consequence of the game,’ but also that ‘the N.F.L. failed to disseminate to then-current and former N.F.L. players health information it possessed’ about the risks associated with brain trauma.”

This law suit has prompted a considerable amount of debate concerning whether or not the NFL as we currently know it could be dramatically reconfigured under the pressure of more law suits. In other words, the inherent but often ignored or concealed dangers of football are now being exposed by legal action, in much the same way as the tobacco industry was unmasked and thus the entire culture of smoking has radically changed in the last couple decades.

With the release of the Education Policy Analysis Archives (EPAA) Special Issue on “Value-Added Model (VAM) Research for Educational Policy,” a similar question should now be raised about the future of implementing high-stakes accountability policies that focus on teacher evaluation and retention through VAM-style metrics.

“High-Stakes Implementation of VAM,…Premature”

Two articles in the special issue from EPAA examines the validity and reliability of VAM-based teacher evaluation in high-stakes settings and then places these policies in the context of legal ramifications faced by districts and states for those policies.

“The Legal Consequences of Mandating High Stakes Decisions Based on Low Quality Information: Teacher Evaluation in the Race-to-the-Top Era” (Baker, Oluwole, & Green, 2013) identifies the current trend: “Spurred by the Race-to-the-Top program championed by the Obama administration and a changing political climate in favor of holding teachers accountable for the performance of their students, many states revamped their tenure laws and passed additional legislation designed to tie student performance to teacher evaluations” (p. 3). Because of the political and public momentum behind reforming teacher evaluation, Baker, Oluwole, and Green seek “to bring some urgency to the need to re-examine the current legislative models that put teachers at great risk of unfair evaluation, removal of tenure, and ultimately wrongful dismissal” (p. 5).

While Baker, Oluwole, and Green offer a detailed and evidence-based examination of the VAM-based and student growth model approaches to high-stakes teacher accountability, they ultimately place the weaknesses of reform policies in the context of potential challenges from teachers who believe they have been wrongfully evaluated or dismissed:

“In this section, we address the various legal challenges that might be brought by teachers dismissed under the rigid statutory structures outlined previously in this article. We also address how arguments on behalf of teachers might be framed differently in a context where value-added measures are used versus one where student growth percentiles are used. Where value-added measures are used, we suspect that teachers will have to show that while those measures were intended to attribute student achievement to their effectiveness, the measures failed to do so in a number of ways. That is, where value-added measures are used to assign effectiveness ratings, we suspect that the validity and reliability, as well as understandability of those measures would need to be deliberated at trial. However, where student growth percentiles are used, we would argue that the measures on their face are simply not designed for attributing responsibility to the teacher, and thus making such a leap would necessarily constitute a wrongful judgment. That is, one would not necessarily even have to vet the SGP measures for reliability or validity via any statistical analysis, because on their face they are invalid for this purpose.”

The analysis ultimately discredits both the use of narrow metrics to determine teacher quality and the high-stakes policies being implemented using those metrics, concluding with the ironic consequences of these policies: “Overly prescriptive, rigid teacher evaluation mandates, in our view, are likely to open the floodgates to new litigation over teacher due process rights. This is likely despite the fact that much of the policy impetus behind these new evaluation systems is the reduction of legal hassles involved in terminating ineffective teachers” (pp. 18-19).

In “Legal Issues in the Use of Student Test Scores and Value-added Models (VAM) to Determine Educational Quality” (Pullin, 2013), the rapid increase of VAM-based accountability is further examined in the context of “a wide array of potential legal issues [that] could arise from the implementation of these programs” (p. 2).

Pullin notes the motivation for reforming teacher evaluation:

“VAM initiatives are consistent with a highly publicized press from the business community and many politicians to make government services more like private business, data-driven to measure productivity and accountability (Kupermintz, 2003). VAM approaches are in part a response to concerns that the current system of selecting and compensating teachers based their education and credentials is insufficient for insuring teacher quality (Corcoran, 2011; Gordon, Kane & Staiger, 2006; Hanushek & Rivkin, 2012; Harris, 2011). There have been increasing expressions of concern that teacher evaluation practices are not robust and do not improve practice (Kennedy, 2010). In the contemporary public policy context, much of the support for the use of student test scores for educator evaluation comes from a concern that the current system for evaluation is ineffective and that the current legal protections for teachers are too cumbersome for schools seeking to terminate teachers (Harris, 2009, 2011).”

While a business model for addressing quality control of a work force may seem efficient, Pullin highlights that legal ramifications are likely with these new models.

Pullin’s analysis offers a detailed and useful examination of previous court cases involving the use of test scores to evaluate educators, including recent cases involving VAM, concluding that the picture is not clear on how the courts may rule in the future, but that a pattern exists of “heavy judicial deference to state and local education policymakers and the allure of using test scores to make decisions about education quality” (p. 5).

Further, Pullin notes “there are differences of perspective among social scientists about VAM and the defensibility of using it to make high-stakes decisions about educators,” further complicating the concerns of legal action (p. 9).

While raising many other complications, Pullin also notes that students and parents may enter legal battles using VAM metrics “to substantiate their own legal claims that schools are not meeting their obligations to provide education” (p. 14).

Pullin concludes with a sobering look at teacher quality reform built on VAM and implemented in high-stakes environments:

“In the broad contemporary public policy context for education reform, the desire for accountability and transparency in government, coupled with heavily financed criticisms of public school teachers and their unions, may mean that VAM initiatives will prevail. The concerns of education researchers about VAM, coupled with legal obligations for the validity and reliability of education and evaluation programs should require judges and education policymakers to take a closer look for future decision-making. At the same time, the social science research community should be generating substantial new and persuasive evidence about VAM and the validity and reliability of all of its potential uses. For public policymakers, there are strong reasons to suggest that high-stakes implementation of VAM is, at best, premature and, as a result, the potential for successful legal challenge to its use is high. The use of VAM as a policy tool for meaningful education improvement has considerable limitations, whether or not some judges might consider it legally defensible.” (p. 17)

Like the NFL, federal and state governments may soon be compelled to reform the reform movement under the threat of legal action from a variety of stakeholders since the science of teacher evaluation remains far behind the curve of implementation, particularly when teacher evaluation is high-stakes and based on VAM and other metrics linked to student test scores.

The special issue from EPAA is yet another call for political leadership to pause if not end wide-scale teacher evaluation and retention models that pose legal, statistical, and funding challenges that those leaders appear unwilling to acknowledge or address.

VAM: A Primer

Education reform has existed at some policy and public levels since at least the 1890s in the U.S. The current reform movement grounded in state-based accountability began in the early 1980s with a Nation at Risk, and then was nationalized in 2001 with No Child Left Behind.

In the first decades of the recent accountability era, standards and high-stakes testing were implemented and periodically revised at the state level with the primary focus being on student and school accountability. The current cycle, however, has seen an increase in policies and practices aimed at teacher accountability and increasing teacher quality—despite a solid research base showing that teacher quality constitutes only 10-15% of measurable student outcomes (test data).

The focus on increasing teacher quality and accountability has included both experiments and policies with value added methods (VAM) of determining teacher quality in order to label, rank, sort, and retain or dismiss teachers. VAM claims to isolate teacher quality through pre- and post-testing methods that seek to identify teacher quality and isolate that from the other factors reflected in test scores.

Before policy-makers and stakeholders in education commit to reforming teacher evaluation and retention, foundational questions must be addressed, and then the current facts about VAM must be acknowledged.

First, the foundational questions:

(1) What evidence exists identifying teacher quality as a primary or significant problem facing a school or district? Where, then, does teacher quality rank as a priority for a school, district, or state in terms of cost effectiveness in committing funding to the reform?

(2) Are all elements of implementing VAM at any percentage to the revised teacher evaluation process valid for determining teacher quality? In other words, what measures are taken to account for using student test scores (designed to reflect student learning, and not designed to reflect teacher quality) as data points for teacher quality?

(3) Have the teaching and learning environments for students’ home and schools been addressed to insure equitable teaching and learning environments in which determining teacher quality becomes valid?

Next, what is the current knowledge base about VAM*?:

(1) Including VAM at any percentage in reformed teacher evaluation models is currently in the experimental phase. Data and the validity of including VAM are being tested, but almost no researchers currently claim VAM (at any percentage but certainly at high percentages such as 40-50%) to be ready for widespread implementation.

(2) VAM models for labeling teacher quality are highly unstable. Teacher rankings tend to shift with new populations of students.

(3) Researchers agree that VAM is unlikely ever to be completely stable; thus, it is possible that VAM will never be a practical or fair element in teacher evaluation, particularly at the individual teacher level or in any single year.

(4) The statistical and practical requirements to isolate teacher quality in student test scores pose tremendous costs in time and funding that also may prove to be not cost effective in the context of most school systems’ priorities. In order to implement a fair and equitable teacher evaluation system that includes VAM at any percentage, states must create and implement pre- and post-tests to all students in all teachers’ courses, creating a new and costly commitment to education funding.

(5) Decades of research on high-stakes testing have shown many negative unintended consequences to accountability measures focusing on students and schools; including VAM in high-stakes accountability policies focusing on teachers is likely to have similar unintended negative consequences such as discouraging high-quality teachers from working with high-needs populations of students.

Thus, policy-makes and stakeholders are strongly cautioned to consider education reform priorities, the experimental nature of VAM, and the current knowledge base on VAM before committing tax dollars to either field testing or implementing new teacher evaluation policies built in any way on VAM.

* See a recent review of teacher evaluation reform for links and citations to numerous research studies on VAM.