First, Do No Harm: That Includes the Media

Public education in the U.S. suffers under a powerful intersection of politics, the media, and the public. As I have too often documented, misinformation tends to be reinforced among all three of these forces.

The role of media, as Alfie Kohn has examined, is central to perpetuating not only misleading beliefs about school quality and education reform but also bad policy.

For example, in South Carolina, The Greenville News has posted an editorial position on reading legislation that misreads what is the best path forward for addressing literacy in this high-poverty and deeply segregated state, notably:

The core part of the legislation makes common sense and is widely supported. It would make it mandatory to retain any third-grader who is not proficient in reading by the end of the school year. It is an idea that has been implemented in Florida to promising results, and it simply makes sense. Promoting a child to fourth-grade if he or she lacks the needed reading skills dooms that child to failure. Although holding back a student can have negative effects on him or her, that student certainly will do better if educators ensure he or she first knows how to read before advancing past third grade.

While we may need some proof that retention of third graders based on high-stakes test scores is in fact “common sense,” it seems true that retention is “widely supported.” The problem with those justifications is that four decades of research strongly rejects retention and close analysis of Florida’s Just Read, Florida, policies discredits claims of its “promising results” (see a full analysis with extensive evidence of the research base on both here).

A point of flawed logic also drives this editorial position: Retention and promotion are not the only options available, and are thus a false dichotomy that likely sits beneath the reality that retention is “widely supported” by the public and then political leaders (see how narrow choices create a false narrative that an issue is supported in the fourth paragraph from the end here).

Further, embedded in this misunderstanding of the research base on retention are careless claims about Florida’s education reform success as well as the current understanding of reading/ literacy instruction and development.

Florida, in fact, is a lesson in what not to do in terms of education policy (see HERE, HERE, and HERE).

Reading/ literacy instruction has been eroded by the accountability era based on standards and high-stakes testing. Literacy is misrepresented by multiple-choice testing, and teaching to those tests greatly warps good literacy instruction.

Test-reading is almost nothing like reading in the real world.

Instead of misguided reading policies reinforced by uninformed media endorsements, a few important and grounding commitments should be guiding reading policy:

  • First, do no harm. Allowing reading policy to be linked to harmful retention policies is inexcusable.
  • Relieve literacy policy from the accountability machine. Authentic, rich, and holistic literacy is eroded by focusing on isolated and skills-based instruction and testing.
  • Recognize that literacy is deeply linked to social class. Unless some powerful efforts are made to address poverty and inequity, students from poverty will remain mislabeled as “bad students” in school and then mis-served in those schools by being funneled into skill-and-drill classes serving the mandates to raise test scores.
  • Set aside the crisis discourse and policies related to literacy. Treating third grade like an Emergency Room ensures that students in most need of patient and rich learning environments will continue to be offered emergency care, and thus once again will be cheated.
  • Embrace low-cost and evidence-based practices that will guarantee literacy growth by increasing student access to books in their lives and their schools: “Perhaps the most serious problem with current literacy campaigns is that they ignore, and even divert attention from, the real problem: Lack of access to books for children of poverty,” explains Stephen Krashen.

Literacy growth is a natural part of being human. Children in middle-class and affluent homes (and thus likely to be enrolled in schools and classes that reinforce rich and authentic literacy) enjoy the sort of experiences with literacy all children deserve.

Retaining third graders based on high-stakes testing will further perpetuate inequity and erode opportunities for children living in poverty to experience rich and authentic learning environments with texts that would result in the type of literacy growth associated with privileged children.

A final problem with the media’s endorsement of “common sense” and “widely supported” education policy is the recurring call for compromise (see Cindy Scoppe at The State), also expressed in The Greenville News editorial:

The important work this fall and heading into the next legislative session is for education leaders and lawmakers to get on the same page about how such an effort would be implemented. There is plenty of room for common ground on this issue….

When policy is evidence-based (as it should be), a compromise between positions that are not evidence based and positions that are evidence based results in flawed policy. In other words, compromise between wrong and right can only result in wrong.

Yes, I recognize that politics is the realm of compromise, but I also believe therein lies the great failure of politics for setting education policy.

In the end, then, much could be solved if we kept our focus on first, do no harm instead of seeking always compromise as the basis for decisions on education policy.

What We Know Now (and How It Doesn’t Matter)

Randy Olson’s Flock of Dodos (2006) explores the evolution and Intelligent Design (ID) debate that represents the newest attack on teaching evolution in U.S. public schools. The documentary is engaging, enlightening, and nearly too fair considering Olson admits upfront that he stands with scientists who support evolution as credible science and reject ID as something outside the realm of science.

Olson’s film, however, offers a powerful message that rises above the evolution debate. Particularly in the scenes depicting scientists discussing (during a poker game) why evolution remains a target of political and public interests, the documentary shows that evidence-based expertise often fails against clear and compelling messages (such as “teach the controversy”)—even when those clear and compelling messages are inaccurate.

In other words, ID advocacy has often won in the courts of political and public opinion despite having no credibility within the discipline it claims to inform—evolutionary biology.

With that sobering reality in mind, please identify what XYZ represents in the following statement about “What We Know Now”:

Is there a bottom line to all of this? If there is one, it would appear to be this: Despite media coverage, which has been exceedingly selective and misrepresentative, and despite the anecdotal meanderings of politicians, community members, educators, board members, parents, and students, XYZ have not been effective in achieving the outcomes they were assumed to aid….

This analysis is addressing school uniform policies, conducted by sociologist David L. Brunsma who examined evidence on school uniform effectiveness (did school uniform policies achieve stated goals of those policies) “from a variety of data gathered during eight years of rigorous research into this issue.”

This comprehensive analysis of research from Brunsma replicates the message in Flock of Dodos—political, public, and media messaging continues to trump evidence in the education reform debate. Making that reality more troubling is that a central element of No Child Left Behind was a call to usher in an era of scientifically based education research. As Sasha Zucker notes in a 2004 policy report for Pearson, “A significant aspect of the No Child Left Behind Act of 2001 (NCLB) is the use of the phrase ‘scientifically based research’ well over 100 times throughout the text of the law.”

Brunsma’s conclusion about school uniform policies, I regret to note, is not an outlier in education reform but a typical representation of education reform policy. Let’s consider what we know now about the major education reform agendas currently impacting out schools:

Well into the second decade of the twenty-first century, then, education reform continues a failed tradition of honoring messaging over evidence. Neither the claims made about educational failures, nor the solutions for education reform policy today are supported by large bodies of compelling research.

As the fate of NCLB continues to be debated, the evidence shows not only that NCLB has failed its stated goals, but also that politicians, the media, and the public have failed to embrace the one element of the legislation that held the most promise—scientifically based research—suggesting that dodos may in fact not be extinct.

* Santelices, M. V., & Wilson, M. (2010, Spring). Unfair treatment? The case of Freedle, the SAT, and the standardization approach to differential item functioning. Harvard Educational Review, 80(1), 106-133.; Spelke, E. S. (2005, December). Sex differences in intrinsic aptitude for mathematics and science? American Psychologist, 60(9), 950-958; See page 4 for 2012 SAT data: http://media.collegeboard.com/digitalServices/pdf/research/TotalGroup-2012.pdf

Class Grades

Since I am quick to criticize the media for its role in the failures of the current education reform movement—such as PBS, The Charleston Post and Courier, and Education Week—I must also recognize when a media outlet provides much needed insight into education policy that has clearly run off the tracks, such as the so-called Florida miracle and the enduring practice of assigning letter grades to schools.

In “Low-income schools struggle under state’s grading system” (Miami Herald, August 10, 2013), Michael Vasquez and David Smiley offer a clear but disturbing picture of accountability in Florida:

With dozens of changes in just the past three years, the formula behind Florida’s A-to-F school grading system has been criticized as a confusing mess. But there’s been at least one constant in Miami-Dade and Broward results: The wealthiest schools never get Fs, and schools with high populations of poor students face an uphill battle to even get a C.

The trend is visible through a decade-plus of school grade results, dating back to the first grades issued in 1999.

Vasquez and Smiley, along with the Miami Herald, represent a needed aspect of journalism addressing education reform: Recognizing large and compelling patterns, and thus the consequences of education policy.

The analysis of assigning letter grades to schools in Florida exposes some important conclusions:

•  Although high poverty rates don’t necessarily doom a school to a subpar grade, D and F schools are overwhelmingly serving students from poor neighborhoods, and the few schools that do overcome poverty to achieve an A are outliers. (There were nine such schools this year, all in Miami-Dade).

•  Of the 209 schools in Miami-Dade and Broward with at least 90 percent of students receiving free or reduced lunch, 78 percent received a grade of C or worse. Roughly 39 percent of these high-poverty schools received a D or F.

•  Of the 43 local schools with much lower poverty rates (30 percent or fewer students receiving free or reduced lunch), 86 percent received an A, and none received a D or F.

Despite efforts to identify educational quality among schools by focusing on growth models, data used in accountability policies remain primarily a reflection of out-of-school factors. Further, the schools that sit outside the typical patterns are rightfully identified by Vasquez and Smiley as “outliers.”

This analytical report on letter grades for schools in Florida is a strong example of quality journalism that seeks out and presents complex and detailed evidence, placing that data in the broader context of the many factors that impact not only the evidence we gather on our schools but also what conclusions we draw as well as how we draw those conclusions.

In the article, Miami-Dade Superintendent Alberto Carvalho explains, “‘Just as much as poverty can’t be an excuse, the exclusion of poverty as a factor is immoral.’”

Rare is the news article that allows a perspective this complex.

No Child Left Behind (NCLB) ushered in several grand promises in 2001, such as closing the achievement gap, but one of the central requirements of the legislation—the use of scientifically based research—is now poised to dismantle the entire accountability movement, including policies such as labeling schools with letter grades based primarily on test scores.

The evidence is clear that thirty years of accountability built on standards and high-stakes testing has failed. The next step is composing and sharing a unified message of that fact, while also building a coalition to reset the reform agenda so that we address poverty, equity, and opportunity in the lives of children and their families as well as in the schools those children attend.

Charter Schools: A Primer

Stakeholders in education include virtually everyone in a democracy—students, parents, teachers, politicians, business leaders, the media, and more.

Historically, public education in the U.S. has experienced two continual popular narratives: (1) public schools are failing, and (2) [insert reform here] is needed to overhaul schools for (a) international competitiveness and (b) a stronger workforce.

Recently, charter schools have seen a significant rise in advocacy and implementation as a complex mechanism for reform. Along with that rise has come a new wave of research on the effectiveness of those charter schools, particularly as they compare with traditional public schools (TPS).

Most stakeholders in education receive their information about charter schools from the media; thus, when the media covers the charter school debate and research, the influence of those media accounts can be disproportional to the quality.

For example, The Post and Courier (Charleston, SC) has taken a strong position for charter schools in SC: “But there is one area where the state has taken bold steps to improve education: charter schools.” However, an analysis of charter schools in SC that compares state report card data between those charter schools and  TPS using the state metric of “Schools with Students Like Ours” revealed in 2012:

Charter schools in SC have produced outcomes below and occasionally typical of outcomes of public schools; thus, claims of exceptional outcomes for charter schools in SC are unsupported by the data (3/53 ABOVE Typical, 17/53 Typical, and 33/53 BELOW Typical).

Since the pattern of advocacy and implementation of charter schools includes a significant amount of support from political leaders, business leaders, the media, and advocates (such as charter-based organizations and think tanks), most stakeholders need a clear and accurate primer addressing what we currently know about charter school effectiveness, and that must be guided by this caution from Matthew Di Carlo:

There’s a constant barrage of data, reports and papers flying around, and sifting through it with a quality filter, as well as synthesizing large bodies of usually mixed evidence into policy conclusions, are massive challenges. Moreover, we all bring our pre-existing beliefs, as well as other differences, to the table. There are no easy solutions here.

But, one useful first step, at least in education, would be to stop pointing fingers and acknowledge two things. First, neither ‘side’ has anything resembling a monopoly on the misuse of evidence. And, second, such misuse has zero power if enough people can identify it as such.

One overarching point needs to be made about the charter school debate first. Charter advocacy and criticism both too often fail in their use of data, as Di Carlo warns, but both also make another mistake, ignoring the evidence base entirely.

What, then, is the current state of evidence on charter school effectiveness? [1] And, how do charter schools address, or not, clearly identified problems and goals of TPS—including what questions and concerns remain in the context of what the evidence suggests about charter school effectiveness?

• Research has repeatedly shown that measurable outcomes (test scores, graduation rates, college admissions rates, etc.) from charter schools produce about the same range of quality as TPS (and private schools) and that the type of school structure (charter v. TPS) appears not to be a determining factor in the outcomes with the demographics of the students and the community remaining powerful correlations with those outcomes.

• Claims of “miracle” schools fail to stand up under close scrutiny, but even if outliers exist in charter schools, outliers exist in TPS and private schools as well, and thus, outliers may prove to be ineffective models for scaling any success.

• Charter schools do not appear to address and often seem to mirror or increase key problems with TPS: (a) teacher assignment (high-needs students assigned to inexperienced and un-/under-certified teachers), (b) class and racial segregation, (c) selectivity and attrition of students, (d) teacher turnover and retention ["churn"], (e) concerns about excluding the most difficult sub-categories of high-needs students [English language learners, special needs students, highest-poverty students, students from home that cannot or will not pursue choices].

• Charter school student outcomes are often complicated by issues of selectivity, attrition, and scalability.

• Some charter school ideologies—notably “no excuses” policies—trigger concerns about classism and racism that are rarely weighed against data.

• Charter schools (along with school choice and home schooling) introduce problems concerning athletic participation as well as a wide range of extracurricular participation in TPS.

• Charter schools also complicate already stressed and controversial TPS funding policies and agendas.

The charter school debate seems to warrant a similar caution that many other reforms now deserve, including VAM-style teacher evaluation. As Di Carlo explains:

As discussed in a previous post, there is a fairly well-developed body of evidence showing that charter and regular public schools vary widely in their impacts on achievement growth. This research finds that, on the whole, there is usually not much of a difference between them, and when there are differences, they tend to be very modest. In other words, there is nothing about “charterness” that leads to strong results.

With commitments to charter schools, many policy makers are moving too quickly and failing to examine the evidence so far along with weighing that evidence against clearly defined problems with TPS and specifically identified goals for the reforms.

—–

[1] A number of studies inform the list above. Readers are invited to examine a wide array of research and reports listed below, but also urged to search for new evidence:

Charter Schools Not the Answer, Especially if We Fail to Identify the Question, P. L. Thomas

Comparing Teacher Turnover In Charter And Regular Public Schools, Matthew Di Carlo

Charter Schools posts at The Shanker Blog

Search “charter schools” at NEPC

Charter Schools posts at School Finance 101 (Bruce Baker)

Charter Schools research at NCSPE

Search “charter schools” at EPAA

Between Educational Research and the Public, a Cloud of Misinformation

Walt Gardner, blogging at Education Week, has posted “Esoteric Formulas and Educational Research,” concluding (with a focus on the complex formulas used in pursuit of value added methods of evaluating teachers):

“The point is that we are too accepting of research that relies heavily on esoteric formulas. I want evidence to support conclusions about educational issues. But the evidence has to be understandable. Just as legal contracts now are increasingly written with consumers in mind, I hope that educational studies will do the same in the future. Taxpayers are entitled to know if students are being well taught, but they can’t make that judgment when they are given incomprehensible data.”

I would suggest that the greatest problem related to educational research is that a cloud of misinformation exists between good educational research/data and the public; and that this cloud is created by political leaders, think tank advocacy groups, and the media [1] who all either do not understand stats or purposefully misuse stats. I also believe some see the world only through a technocratic lens (such as the pursuit of VAM)—also a huge failure of applying appropriate paradigms in different contexts. Larry Ferlazzo has recently cited Nate Silver, who recognizes VAM as misguided: “There are certainly cases where applying objective measures badly is worse than not applying them at all, and education may well be one of those.”

Democracy and the market both work best for the public good when the public and consumers are informed. Political leaders, think tanks, and the media do no one any good by continually being inept themselves (and dishonest) in the use and misuse of research to drive political agendas or advance their own brand.

Some excellent resources to confront how badly educational research is portrayed for the public see the following:

Bracey, G. W. (2006). Reading educational research: How to avoid getting statistically snookered. Portsmouth, NH: Heinemann.

Molnar, A. (2001, April 11). The media and educational research: What we know vs. what the public hears. Milwaukee, WI: Center for Education Research, Analysis, and Innovation. Retrieved from http://epsl.asu.edu/epru/documents/cerai-01-14.htm

Yettick, H. (2009). The research that reaches the public: Who produces the educational research mentioned in the news media? Boulder, CO and Tempe, AZ: Education and the Public Interest Center & Education Policy Research Unit. Retrieved from http://epicpolicy.org/publication/research-that-reaches

[1] See chapters on the media in two of my most recent books: Parental Choice? and Ignoring Poverty in the U.S.