Think Tank Advocacy Reports Not Credible for Education Policy: SC Edition

The Palmetto Promise Institute‘s report authored by Adam Crain, Money doesn’t translate into student results, is a follow-up to their 2013 report also comparing South Carolina education to Florida education reform.

Although this report offers several charts detailing an analysis of SC and FL National Assessment of Educational Progress (NAEP) tests data (some of which is aggregated by race, disabilities, and poverty, but focusing on 4th grade reading), the report proves to be overly simplistic and an incomplete picture of student achievement in both states—with the ham-fisted data analysis serving as a thin veneer for advocacy unsupported by valid research and a more nuanced analysis of data.

In short, this report proves to be significantly inadequate evidence to support the ideologically-driven recommendations offered at the end—recommendations this conservative think tank would make regardless of the evidence (mostly a mishmash of school choice policy). There simply is no credible link between the shallow analysis of SC/FL NAEP scores and the call for policy as solutions to the manufactured problems.

Let me outline here both the flaws of the data analysis and then the folly of the recommendations.

The foundational flaw of both reports is suggesting some sort of value in comparing SC to FL and the persistent but discredited claim that FL has successful education reform. In fact, the so-called Florida “miracle” has been strongly refuted, notably its grade-retention policy based on high-stakes test scores.

By comparison, SC is slightly more impoverished than FL, and SC (27%) has a higher percentage than FL (16%) of blacks (both metrics used in the report analysis). However, this report from PPI makes no effort to show how their raw comparisons are actually apples-to-apples, or valid.

Another analysis of NAEP data that adjusts for factors impacting test scores reveals a much more nuanced and important picture, one that exposes a huge flaw with the FL model of reform [1] and depending on test data.

While adjusted trend data on NAEP continues to show 4th grade FL reading scores better than SC scores, by 8th grade (see Table 6B1, 2013 data) SC (269.5) and FL (272.3) have nearly identical adjusted scores.

Here is a key point about FL’s retention policy: Retaining students can inflate short-term test data, but those gains erode over time. Further, grade retention [2] maintains a strong correlation with students dropping out of school and an inverse correlation with students receiving a diploma (see Jasper, 2016 [3]).

Ultimately, the data analysis and charts in this report are overly simplistic on purpose because PPI has an agenda: argue against increased school funding and promote school choice.

The report uses bold face, lazy math, and insufficient statistical methods to dramatize a baseless claim: “Simple funding comparisons indicate quite the opposite. Over the twelve year period between 1999 and 2011, South Carolina spent a total of $6,920 more per student, or an average of $692 per year.”

Without proper statistical analysis, using controls and making causal claims, this raw data approach, like the NAEP analysis, means almost nothing.

The body of educational research, in fact, shows that funding does matter (see Baker, 2016) [4].

Both, then, the NAEP analysis and the related argument that SC school funding is somehow excessive/wasteful are statistically inadequate and useless for making the recommendations at the end of the report.

Those recommendations fall into two broad categories: accountability and school choice.

SC and FL jumped on the accountability bandwagon early, about three decades ago, and remain completely unsatisfied with their educational outcomes, despite huge amounts of tax dollars and immeasurable time spent on ever-new standards and ever-new high-stakes tests.

Calling for accountability ignores the research base that shows accountability based on standards and testing has failed, will continue to fail:

There is, for example, no evidence that states within the U.S. score higher or lower on the NAEP based on the rigor of their state standards. Similarly, international test data show no pronounced tests core advantage on the basis of the presence or absence of national standards. Further, the wave of high-stakes testing associated with No Child Left Behind (NCLB) has resulted in the “dumbing down” and narrowing of the curriculum….

As the absence or presence of rigorous or national standards says nothing about equity, educational quality, or the provision of adequate educational services, there is no reason to expect CCSS or any other standards initiative to be an effective educational reform by itself. (Mathis, 2012)

The evidence on school choice also contradicts the report because choice fails to increase student achievement, but it is strongly associated with increasing segregation and inequity (see here and here).

Let’s summarize the major points of the report:

• The report claims SC lags FL in academic achievement and education reform while spending more per pupil. However, the analysis offered here is an incomplete picture and statistically flawed. None of the claims made in the report are proven, and more nuanced and longitudinal analyses of NAEP greatly erode the premise of PPI’s report (grounded also in the debunked Florida “miracle” claim).
• The report’s major recommendations about school funding, accountability, and school choice are all strongly contradicted by the research base, which the report fails to acknowledge.

Ultimately, as a colleague responded when I shared this report, PPI has published “a five page Op-Ed with bar graphs,” and I would add, not a very good one at that.

SC should in no way be influenced by this report when making education policy.

However, SC should heed a kernel the report’s conclusion: “The disparity between the stewardship of resources in Florida and our struggling education system in South Carolina is apparent.”

As I have detailed, while most educational rankings and comparisons prove to be hokum, what evidence from our schools and reform policies shows is that SC ranks first in political negligence.

Ironically, this report is calling for more negligence in the pursuit of market ideology.

---

[1] See evidence discrediting Florida “miracle” and FL’s reading policy; how SC could benefit from looking at Oklahoma, not FL; and why FL reform is harmful for students and literacy.

[2] See the National Council of Teachers of English’s Resolution on Mandatory Grade Retention and High-Stakes Testing:

Grade retention, the practice of holding students back to repeat a grade, does more harm than good:

• retaining students who have not met proficiency levels with the intent of repeating instruction is punitive, socially inappropriate, and educationally ineffective;
• basing retention on high-stakes tests will disproportionately and negatively impact children of color, impoverished children, English Language Learners, and special needs students; and
• retaining students is strongly correlated with behavior problems and increased drop-out rates.

[3] Jasper’s abstract captures the ultimate failure of FL’s reform:

In 2003-2004 approximately 23,000 third graders were retained in Florida under the third grade retention mandate outlined in the A+ Plan. Researchers in previous studies found students who were retained faced difficulty in catching up to their peers, achieving academically, and obtaining a high school diploma (Anderson, Jimerson, & Whipple, 2005; Andrew, 2014; Fine & Davis, 2003; Jimerson, 1999; Moser, West & Hughes, 2012; Nagaoka, 2005; and Ou & Reynolds, 2010). In this study I examined educational outcomes of students retained in a large southwest Florida school district under the A+ Plan in 2003-2004. I used a match control group, consisting of similarly nonretained students, who scored at level one on the Grade 3 Reading FCAT. I then compared the control group to the retained group. I also compared achievement levels on the Grade 10 Reading FCAT of the retained and non-retained group. I evaluated longitudinal data, for both the retained and non-retained students, and found 93% of the retained students continued to score below proficiency (below a level 3) seven years after retention on the Grade 10 Reading FCAT as compared with the 85.8% of the non-retained students. I also compared standard diploma acquisition of the retained and non-retained group. The non-retained group was 14.7% more likely to obtain a standard high school diploma than the retained group. Finally, I used data from previous studies to extrapolate economic outcomes.

[4] Baker’s analysis has key points detailed in the Executive Summary (p. i):

Today in “Don’t Believe It”

More often than not, mainstream media and think tanks produce claims about education that are without credibility.

Sometimes the source is also lacking credibility, but many times, the source has good intentions.

Today in “Don’t Believe It,” let’s consider both types.

First, NCTQ—a think tank entirely lacking in credibility—issued a report claiming that teacher education is lousy, basing their claims on a fumbled review of textbooks assigned and course syllabi.

Don’t believe it because NCTQ bases the claims on one weak study about what every teacher should know, and then did a review of textbooks and syllabi that wouldn’t be allowed in undergraduate research courses.

See the full review here.

Next, despite genuinely good intentions, Kecio Greenho, regional executive director of Reading Partners Charleston, claims in an Op-Ed for The Post and Courier (Charleston, SC) that South Carolina’s Read to Succeed, which includes provision for third-grade retention based on high-stakes test scores, “is a strong piece of legislation that gives support to struggling readers by identifying them as early as possible.”

Don’t believe it because Read to Succeed is a copy-cat of similar policies across the U.S. that remain trapped in high-stakes testing and grade retention, although decades of research have shown retention to be very harmful to children.

See this analysis of Read to Succeed, the research base on grade retention, and the National Council of Teachers of English’s resolution on grade retention and high-stakes testing.

When you are confronted with claims about education, too often the source and the claim are without merit, but you have to be aware that those with good intentions can make false claims as well.

Dear Florida: Mean People Suck

Dear Florida:

I know it is impolite to use harsh language and teachers often discourage students from resorting to cliches, but I am hard-pressed to find anything better suited to my concern about education policy in Florida (and Mississippi, and South Carolina, along with another 10+ states) than reminding political leaders and the public in your state: Mean people suck.

And as disturbing as it is when adults are mean to adults, there simply is no way to justify adults being mean to children—or in the case of misguided and uninformed education policy in Florida (and Mississippi, and South Carolina), adults being mean to some children (mostly black, brown, and poor).

I take this opportunity to reach out to you, Florida, because there is a way out of this mean streak: Fewer 3^rd-graders could be held back this year.

First, let me note that the avenue to a kind and equitable education system is not examining whether or not your (yet again) new high-stakes tests are valid, but recognizing that grade retention is discredited by a large body of research and grade retention is not a credible form of literacy policy.

Notably, the National Council of Teachers of English, the largest organization of English teachers in the U.S., has a clear position statement against grade retention based on high-stakes testing:

Resolved, that the National Council of Teachers of English strongly oppose legislation mandating that children, in any grade level, who do not meet criteria in reading be retained.

And be it further resolved that NCTE strongly oppose the use of high-stakes test performance in reading as the criterion for student retention.

As well, education policies such as grade retention linked to high-stakes testing disproportionately and negatively impact black, brown, and poor children.

Simply put, grade retention as a policy must be acknowledged as punishing children for the sake of punishing children.

None the less, Florida has become a flawed model for educational accountability across the U.S.

So, as a life-long educator in SC, where that model is now turning my home state into yet another place where mean people suck, I ask that Florida end grade retention and use your ill-got influence to start a new trend in education reform—one that rejects punitive education policy and chooses instead to treat all children with dignity, to provide all children an equitable opportunity to learn.

South Carolina and Education Reform: A Reader

Hartsville, South Carolina sits just north of I-20 and north-west of I-95 about a 30-minute drive from Florence.

Geography matters in SC when discussing education because the state has a long and tarnished history of pockets of poverty and educational inequity now commonly known as the Corridor of Shame [1], a name coined in a documentary addressing that inequity as it correlates with the I-95 corridor running mostly north and south paralleling the SC coast.

The town’s schools are part of Darlington County School District that serves approximately equal numbers of white and black students, although significantly skewed by poverty as reflected in the district’s 2014 report card detailing tested students:

Hartsville is the focus of an upcoming PBS documentary co-produced by Sam Chaltain, who writes about the community:

Take Hartsville. Until recently, no one there had ever asked Thompson or her colleagues what they noticed about their child passengers on the bus, or thought to connect their observations to the behavior teachers might witness in the classroom. Moreover, while Hartsville’s teachers were expected to be knowledgeable about their students’ academic standing, they were not expected to be attuned to their psychological states.

That began to change in 2011, when the community announced a five-year plan to transform its elementary schools. It partnered with Yale University’s School Development Program, which helps schools identify and meet the developmental needs of children. It began to evaluate its schools by a broader set of measurements – including the number of disciplinary referrals a bus driver had to write each morning. And it started to coordinate its social services to ensure a more equitable set of support structures for Hartsville’s poorest families.

This focus on Hartsville specifically and SC more broadly is important for understanding the entire education reform movement in the U.S. for several reasons.

The standards/testing issues in SC are complex (the anti-Common Core movement in SC is mostly ill-informed and falls along libertarian lines, for example) and represent well patterns found across the country during the mostly state-based accountability era (see below).

SC was one of the first accountability states, and we have had about 5-6 sets of standards and new tests over 30-plus years (see below). Throughout those years, almost no one in political leadership has acknowledged SC being in the bottom quartile of poverty in the U.S.—huge pockets of poverty and affluence—is the real educational crisis.

Like New Orleans, I think, SC is a perfect model of all that is wrong with the education reform debate.

I recommend viewing the 180 Days focus on Hartsville through the complicated and often jumbled politics and education reform over the past three decades in SC, much of which I have addressed in the reader offered below, organized by major accountability issues and policies:

Value-Added Methods of Teacher Evaluation:

South Carolina Officially Vamboozled

Review [UPDATED]: “How to Evaluate and Retain Effective Teachers” (League of Women Voters of SC)

SC and Common Core:

South Carolina and Common Core: A Next Step?

Death to Common Core! Long Live Failed Education Policy!

Should SC Ditch Common Core?

SC, Reading Policy, and Grade Retention:

Just Say No to Just Read, Florida, South Carolina

Retain to Impede: When Reading Legislation Fails (Again)

SC and Opt-Out:

SC Parents Warned: “no state provision…to opt-out of state-or district-wide testing”

SC, Oklahoma, and Florida:

GreenvilleOnline: SC should choose Oklahoma, not Florida

SC’s Former Superintendent Zais:

SC’s Zais Mistake

SC and Accountability:

Welcome to SC: A Heaping Stumbling-Bumbling Mess of Ineptitude

SC and Charter Schools:

Should SC Increase Charter School Investment?

Public School, Charter Choice: More Segregation by Design

SC and Exit Exams:

Ending Exit Exams a Start, But Not Enough

SC’s Conservative Leadership:

Conservative Leadership Poor Stewardship of Public Funds

—

[1] This documentary focuses on a court case in South Carolina initiated by high-poverty school districts surrounding primarily the I-95 corridor of the state, paralleling the east coast and stretching from the NE to the SE region. The documentary suffers from melodramatic production values (music, slow-motion panning of sad children’s faces), but the essential claim of the film is important for confronting the social inequity that is reflected in educational inequity, particularly in the South. Issues included in the film are school funding, community-based schools, access to high-quality educational opportunities and facilities, teacher assignments related to student characteristics, and state education accountability mechanisms. Some related resources (SC school report cards, poverty indices, related blog posts) to the documentary support examining the film in my educational documentary May experience course.

Media Fail, 10,000 hours, and Grit: The Great Media-Disciplines Divide, pt. 2

In his The Danger of Delegating Education to Journalists: Why the APS Observer Needs Peer Review When Summarizing New Scientific Developments [1], K. Anders Ericsson makes several key points about how the mainstream media present disciplinary knowledge to the public, focusing on Malcolm Gladwell’s misleading but popular 10,000 hour rule.

Ericsson’s key point includes:

Although I accept that the process of writing an engaging popular article requires considerable simplification, I think it is essential that the article does not contain incorrect statements and misinformation. My primary goal with this review is to describe several claims in Jaffe’s article that were simply false or clearly misleading and then discuss how APS might successfully develop successful methods for providing research summaries for non-specialists that are informative and accurately presents the major views of APS members and Fellows. At the very least they should not contain factually incorrect statements and avoid reinforcing existing misconceptions in the popular media.

Through the Gladwell/10,000 hour rule example, Ericsson provides an important argument relevant to the current (and historical) public debate about school quality, teaching and learning, and education reform.

Much in the same way Gladwell has misrepresented research (which is typical within the media), and how that has been uncritically embraced by the media and public (as well as many if not most practitioners), a wide array of issues have received the same fate: learning styles, “grit,” collaborative learning, progressive education, charter schools, school choice, language gap, and so on.

Even when a claim or practice has a kernel of research at its source, popular oversimplification (often by journalists, but practitioners as well) and then commercialization/politicizing (creating programs and policies through publishers, “star” advocates, and legislation) significantly distort that research.

Education Has Failed Research, Historically

John Dewey represents an odd paradox in that he is possibly the most mentioned educator in the U.S. (either as the source of all that is wrong in education or idealistically cited as all that is right about how school could be), despite the reality that Dewey is mostly misunderstood and misrepresented; and thus his philosophy, progressivism, remains mostly absent in U.S. public schools.

Dewey can be blamed, in part, for this reality because he refused on principle to allow his experiments in education to be carefully catalogued because he believed no educational practice should be come a template for others.

Throughout much of the twentieth century, Lou LaBrant, a vigilant progressive educator, spent much of her career practicing and advocating for progressive literacy instruction, but LaBrant also confronted the many instances of how progressivism was misrepresented.

Broadly, and early, LaBrant recognized the public confusion about progressivism:

Two adults speak of “progressive education.” One means a school where responsibility, critical thinking, and honest expression are emphasized; the other thinks of license, lack of plans, irresponsibility. They argue fruitlessly about being “for” or “against” progressive education. (LaBrant, 1944, pp. 477-478)

But she also confronted how progressivism was mostly distorted in its application. LaBrant’s criticisms still reflect why education has failed research, and why research has not failed education.

Credible educational research-based philosophy, theory, and pedagogy are often corrupted by oversimplification.

In 1931, LaBrant published a scathing criticism of the popularity of the project method, an oversimplification of Dewey that resulted in students doing crafts in English class instead of reading or writing:

The cause for my wrath is not new or single. It is of slow growth and has many characteristics. It is known to many as a variation of the project method; to me, as the soap performance. With the project, neatly defined by theorizing educators as “a purposeful activity carried to a successful conclusion,” I know better than to be at war. With what passes for purposeful activity and is unfortunately carried to a conclusion because it will kill time, I have much to complain. To be, for a moment, coherent: I am disturbed by the practice, much more common than our publications would indicate, of using the carving of little toy boats and castles, the dressing of quaint dolls, the pasting of advertising pictures, and the manipulation of clay and soap as the teaching of English literature. (p. 245)

Credible educational research is often corrupted by commercialization/politicizing, reducing that research to misguided programs/legislation.

“[L]anguage behavior can not be reduced to formula,” LaBrant (1947) argued (p. 20)—emphasizing that literacy growth was complicated but flourished when it was child-centered and practical (for example, in the ways many privileged children experience in their homes because one or more of the parents are afforded the conditions within which to foster their children’s literacy).

By mid-twentieth century, LaBrant (1949) had identified the central failure of teaching reading: “Our language programs have been set up as costume parties and not anything more basic than that” (p. 16).

For at least 80-plus years since LaBrant fought this fight, the same patterns of media, political, public, and practitioners failing educational research have continued

Oversimplification, Commercialization/Politicizing: Recovering the Evidence

The list is incredibly long, too long to be exhaustive here, but consider the following: sloganism (“Work hard. Be nice.”), silver-bullet ideologies (“grit,” 10,000-hour rule), miracle schools (KIPP), evidence-based programs (Dibbles, 4-block, 6-traits), common sense claims and policy absent evidence (Common Core), and trendy legislation (3rd-grade retention policies as reading policy, merit pay) as well as politicized government reports (National Reading Panel).

Each of these can be traced to some kernel of research (sometimes robust bodies of research, and sometimes cherry-picked research), but all of these represent a current and historical fact: Education has failed research, but research has not failed education.

When educational research is reduced to scripts or programs/legislation, that knowledge base is invariably distorted, corrupted—as Ericsson details well above.

Journalists, politicians, and commercial education entities have all played a fundamental and crippling role in this reality; thus, as Ericsson argues, educators, scholars and researchers must not allow the fate of educational research to remain primarily in the wrong hands.

We have a public and professional obligation to confront these oversimplifications as well as the commercialization/politicizing of educational research. And we must do this through our public work that speaks to those failures and the public simultaneously.

As LaBrant and Ericsson reveal, unless we take that call seriously, we too are part of the reason education continues to fail research.

References

LaBrant, L. (1949). A genetic approach to language. Unpublished manuscript, Institute of General Semantics, Lakeville, CT.

LaBrant, L. (1947). Um-brel-la has syllables three. The Packet, 2(1), 20-25.

LaBrant, L. (1944, November). The words they know. The English Journal, 33(9), 475-480.

LaBrant, L. (1931, March). Masquerading. The English Journal, 20(3), pp. 244-246.

For Further Reading

U.S. and Education Reform Need a Critical Free Press

My Open Letter to Journalists: A Critical Free Press, pt. 2

NPR Whitewashes “Grit” Narrative

Shiny Happy People: NPR, “Grit,” and “Myths that Deform” pt. 2

How I Learned to Distrust the Media (about Education)

My (Often Painful) Online Education

—

[1] See original and downloadable link to the paper here.

Misreading Florida

Misreading Florida

First, Do No Harm: That Includes the Media

Public education in the U.S. suffers under a powerful intersection of politics, the media, and the public. As I have too often documented, misinformation tends to be reinforced among all three of these forces.

The role of media, as Alfie Kohn has examined, is central to perpetuating not only misleading beliefs about school quality and education reform but also bad policy.

For example, in South Carolina, The Greenville News has posted an editorial position on reading legislation that misreads what is the best path forward for addressing literacy in this high-poverty and deeply segregated state, notably:

The core part of the legislation makes common sense and is widely supported. It would make it mandatory to retain any third-grader who is not proficient in reading by the end of the school year. It is an idea that has been implemented in Florida to promising results, and it simply makes sense. Promoting a child to fourth-grade if he or she lacks the needed reading skills dooms that child to failure. Although holding back a student can have negative effects on him or her, that student certainly will do better if educators ensure he or she first knows how to read before advancing past third grade.

While we may need some proof that retention of third graders based on high-stakes test scores is in fact “common sense,” it seems true that retention is “widely supported.” The problem with those justifications is that four decades of research strongly rejects retention and close analysis of Florida’s Just Read, Florida, policies discredits claims of its “promising results” (see a full analysis with extensive evidence of the research base on both here).

A point of flawed logic also drives this editorial position: Retention and promotion are not the only options available, and are thus a false dichotomy that likely sits beneath the reality that retention is “widely supported” by the public and then political leaders (see how narrow choices create a false narrative that an issue is supported in the fourth paragraph from the end here).

Further, embedded in this misunderstanding of the research base on retention are careless claims about Florida’s education reform success as well as the current understanding of reading/ literacy instruction and development.

Florida, in fact, is a lesson in what not to do in terms of education policy (see HERE, HERE, and HERE).

Reading/ literacy instruction has been eroded by the accountability era based on standards and high-stakes testing. Literacy is misrepresented by multiple-choice testing, and teaching to those tests greatly warps good literacy instruction.

Test-reading is almost nothing like reading in the real world.

Instead of misguided reading policies reinforced by uninformed media endorsements, a few important and grounding commitments should be guiding reading policy:

• First, do no harm. Allowing reading policy to be linked to harmful retention policies is inexcusable.
• Relieve literacy policy from the accountability machine. Authentic, rich, and holistic literacy is eroded by focusing on isolated and skills-based instruction and testing.
• Recognize that literacy is deeply linked to social class. Unless some powerful efforts are made to address poverty and inequity, students from poverty will remain mislabeled as “bad students” in school and then mis-served in those schools by being funneled into skill-and-drill classes serving the mandates to raise test scores.
• Set aside the crisis discourse and policies related to literacy. Treating third grade like an Emergency Room ensures that students in most need of patient and rich learning environments will continue to be offered emergency care, and thus once again will be cheated.
• Embrace low-cost and evidence-based practices that will guarantee literacy growth by increasing student access to books in their lives and their schools: “Perhaps the most serious problem with current literacy campaigns is that they ignore, and even divert attention from, the real problem: Lack of access to books for children of poverty,” explains Stephen Krashen.

Literacy growth is a natural part of being human. Children in middle-class and affluent homes (and thus likely to be enrolled in schools and classes that reinforce rich and authentic literacy) enjoy the sort of experiences with literacy all children deserve.

Retaining third graders based on high-stakes testing will further perpetuate inequity and erode opportunities for children living in poverty to experience rich and authentic learning environments with texts that would result in the type of literacy growth associated with privileged children.

A final problem with the media’s endorsement of “common sense” and “widely supported” education policy is the recurring call for compromise (see Cindy Scoppe at The State), also expressed in The Greenville News editorial:

The important work this fall and heading into the next legislative session is for education leaders and lawmakers to get on the same page about how such an effort would be implemented. There is plenty of room for common ground on this issue….

When policy is evidence-based (as it should be), a compromise between positions that are not evidence based and positions that are evidence based results in flawed policy. In other words, compromise between wrong and right can only result in wrong.

Yes, I recognize that politics is the realm of compromise, but I also believe therein lies the great failure of politics for setting education policy.

In the end, then, much could be solved if we kept our focus on first, do no harm instead of seeking always compromise as the basis for decisions on education policy.