Will Free-Pass Mainstream Media Clean Up Their Chetty Mess?

The New York Times couldn’t hold back: Big Study Links Good Teachers to Lasting Gain.

And for two years now, the so-called Chetty study that claims teacher quality directly causes higher lifetime earnings for their students has been re-released by Chetty and colleagues as well as rehashed by the free-pass mainstream media with a fervor that seems at least over-the-top.

When the Chetty numbers (has anyone noticed that the lifetime earning numbers appear to change?) are put into perspective, all the air should deflate from that misleading balloon: $50,000 gained over a lifetime (40 years) is only about 1.5-2 tanks of gas a month. But inflating that balloon to $1.4 million for a class of 28! Now you have something … (Hint: An overinflated balloon, possibly filled with poop.)

Initial scholarly responses to the Chetty balloon were cautious and critical (see notably Bruce Baker and Matthew Di Carlo here), but the free-pass mainstream press kept on keeping on.

Thus, since I have made a case for our needing a critical free press (in other words, “free press” and not “free pass” found among press-release journalists), we are at a key moment with the release of a thorough review of the Chetty study, a review that discredits the claims, Review of Measuring the Impacts of Teachers by Moshe Adler:

Can the quality of teachers be measured the way that a person’s weight or height is measured? Some economists have tried, but the “value-added” they have attempted to measure has proven elusive. The results have not been consistent over tests or over time. Nevertheless, a two-part report by Raj Chetty and his colleagues claims that higher value-added scores for teachers lead to greater economic success for their students later in life. This review of the methods of Chetty et al. focuses on their most important result: that teacher value-added affects income in adulthood. Five key problems with the research emerge. First, their own results show that the calculation of teacher value-added is unreliable. Second, their own research also generated a result that contradicts their main claim—but the report pushed that inconvenient result aside. Third, the trumpeted result is based on an erroneous calculation. Fourth, the report incorrectly assumes that the (miscalculated) result holds across students’ lifetimes despite the authors’ own research indicating otherwise. Fifth, the report cites studies as support for the authors’ methodology, even though they don’t provide that support. [emphasis added] Despite widespread references to this study in policy circles, the shortcomings and shaky extrapolations make this report misleading and unreliable for determining educational policy.

So my question now is: Will the free-pass mainstream media clean up their Chetty mess?

I suspect we will not have a NYT scorching headline, we will not even have a NYT article, we probably will not see interviews on NPR with Adler, and I am skeptical about Education Week‘s coverage (beyond some bloggers).

The fair and balanced mainstream media, alas, (those journalists who cannot judge the credibility of the research they cover) will not fall all over themselves to cover and then repeat for two years to come the popping of the Chetty balloon because that would mean admitting their own incompetence, which is the sweet allure of fairness that leaves us all misinformed.

Debating the Gates Moratorium, Or Life among the Roadbuilders

As a lifelong fan of science fiction (SF), I want to start by invoking a SF convention that never gets old: time travel (but I’ll spare you the whirlwind prestidigitation mastered by Kurt Vonnegut, who married time travel and non-linear narrative in Slaughterhouse-Five).

Once upon a time (well, I must confess, I enjoy a bit of genre-splicing also), Bill Gates, billionaire and burgeoning education hobbyist, began tossing his considerable expertise (read: money) at small schools projects. Low and behold, Gates eventually looked at the research his own foundation gathered, declared the project a failure, and scuttled away—only to decide that the greatest scourge on the planet was the enormous number of “bad” teachers failing our children everyday!

Since we are now back in time (and you may notice a pattern here about history repeating itself, and such), let’s look at that Gates/small school dynamic as I blogged about this in April and May 2011:

And, Gates’s small schools experiment? While Gates himself declared the experiment a failure, Marshak explains that Gates’ small school experiment actually exposes Gates’ own inability to understand the education dynamics he claims to reform. But ample evidence reveals both that Gates is inexpert and remains unsuccessful as an education reformer. See here, here and here – including his failure to understand statistics and the charts he enjoys using to make his points here. (Accountability? Start at the Top, OpEdNews 11 April 2011)

Let’s do some truly basic math.

First, consider that Bill Gates, a billionaire whose wealth and success have been built on computer innovation and entrepreneurship, has been an education reformer for many years now–stretching back to a small schools focus:

“Bill Gates used to believe that one of the solutions to failing schools was to create smaller ones with 500 students or fewer. His foundation spent $1 billion toward this; seeing the opportunity to bring in private dollars, districts started shifting to smaller schools. Small schools became the big new trend. But then the foundation conducted a study that found that, by itself, school size had little if any effect on achievement. The foundation dropped the project and moved on to teacher reform, but by then some urban districts throughout the nation had changed to small–and more expensive to operate–schools.”

So the first formula is:

Gates initiative + Gates funding = abandoned schools in the wake of failure (with no consequences for Gates)

As the Los Angeles Times reports above, Gates is now focusing on teacher quality–including calls for teacher evaluations tied to test scores measuring student achievement against the common core standards.

This suggests a new formula:

Gates money + common core standards + testing industry = profit for Gates and testing industry at the expense of students, learning, and public education. (If There Remains Any Question, Daily Kos 1 May 2011)

Let’s return now to the present where the Gates machine has called for a moratorium on all that Common Core and VAM stuff because his own people’s research appears to refute what Gates has been pontificating about in the compliant media gaze that only Gates seems worthy of receiving (I hate to beat this to death, but do any of you see a pattern here?).

I want to offer now that this call for a moratorium is another teachable moment—those who fail to learn from history are doomed to repeat it, and all that—and therein we have at least two important lessons:

First, Gates/small schools, Gates/VAM, and Gates/Common Core are all the same flawed dynamic in which political leaders, the media, and the public confuse wealth with expertise. Remove Gates’s billions and he would have zero credibility in any of these ventures—and I still maintain that the small school debacle is the most telling part of this story [1] because Gates misread his own research, declared erroneously the commitment a failure, and simply walked away leaving others to hold the bag (roadbuilder he has always been, roadbuilder he will always be).

And this leads to the debate and discussion that has blossomed from the Gates moratorium announcement.

So our second lesson is somewhat positive: If we could ever set aside confusing wealth with expertise, we may come to recognize that among educators, scholars, and researchers we already have a wealth of expertise that could better serve our goals of education reform.

And thus, I maintain my stance that the Gates moratorium is a sham, but I want to highlight here that within the credible responses to that moratorium call, we see how education reform should be debated: by those who know the field:

And while I enjoy coming back to and finding new SF again and again, in education reform I am eager to step off the “Déjà vu all over again” merry-go-round.

Bill Gates, about that moratorium? No thanks. And while we are at it, no thanks to all the rest either.

If you’d stopped building your roads we never asked for right through the middle of our villages, we would have time to take care of business here, instead of constantly staying one step behind underneath the rubble of the disasters you create.

[1] Please read carefully Good Doubt and Bad Doubt from 2007 and Why Did the Gates Small-High-Schools Program Fail?: Well, Actually It Didn’t from 2010 for some really powerful time travel. And this is just must-read: Bill Gates should hire a statistical advisor.

U.S. and Education Reform Need a Critical Free Press

Few things are worse than mainstream media coverage of education.

Except for that sentence above, which stretches hyperbole beyond credibility.

But that is exactly where the mainstream media finds itself when covering education. Journalists, in their quest to maintain the traditional commitment to “fair and balanced” journalism [1], consistently endorse and perpetuate organizations without credibility (such as NCTQ) and baseless claims (such as cries of “bad” teacher, “bad” teacher certification, and “bad” unions).

With yet another report released by NCTQ, that failure of the mainstream media has been highlighted once again—notably at NPR [2] and Education Week: Study Delivers Failing Grades For Many Programs Training Teachers, Claudio Sanchez and Juana Summers; Alternative Certification Deemed Weak by NCTQ in New Teacher-Prep Report, Stephen Sawchuk; Most Teacher Preparation Falls Short on Strategies for ELLs, NCTQ Finds, Lesli A. Maxwell.

First, the mainstream coverage of NCTQ’s reports remains trapped inside assumed crises that have no basis in fact; NCTQ’s reports and then the media begin with the givens that education suffers under the burden of “bad” teachers, “bad” teacher certification, and “bad” unions. However, at the very least, these claims are disproportional, if not outright erroneous:

  • If we maintain the current context that student achievement is accurately reflected in test scores (and it isn’t), then we must acknowledge that teacher quality (10-15%) and school quality account for only about 20% of that measurement, but “60 percent of achievement outcomes is explained by student and family background characteristics (most are unobserved, but likely pertain to income/poverty),” as Di Carlo details.
  • If we accept that value-added methods (VAM) can accurately and consistently identify “good” and “bad” teachers (and the evidence is that it cannot) and if we accept the much repeated claim by Chetty et al. that teacher quality can add $50,000 to the lifetime earning potential of a student (and that also is a significantly contested claim, as well as another example of advocacy and media hyperbole since that lifetime earning figure equates to about 1.5-2 tanks of gas per month), the enormity of the claims about “bad” teachers and the urgency expressed about creating and implementing huge and expensive test-based systems to address teacher quality are at best overstated. No rational person would endorse the cost-benefit analysis of such schemes.
  • Finally, claims that teachers unions are primary or significant negative influences on educational quality are powerfully refuted by the historical and current fact that the states in the U.S. with the lowest standardized test scores tend to be those that are right-to-work (non-union) states. Unionization correlates positively with measurable student achievement, in fact, while poverty is the greatest correlation with low measurable student outcomes (for the record, union bashing is a straw man because U.S. public education has a poverty problem, not a union problem).

Next, NCTQ has established a sort of immediate appearance of credibility through three strategies: partnering itself with U.S. News & World Report, garnering significant and influential sources of funding, and bombarding the mainstream media with a series of reports without vetting those reports as is common in traditional scholarship (which slows down and greatly harnesses higher-quality research from reaching the public [3]. But scholars don’t issue press releases, and apparently, journalists respond primarily to press releases instead of conducting investigative journalism [4].)

Finally, once I engaged Sawchuck (EdWeek) and then Summers (NPR) on Twitter, several key aspects of this phenomenon were highlighted. Both journalists argued that their pieces on NCTQ were fair, and even critical—which I will examine below—but even more significant is a comment on Twitter from Summers:

My two reactions to Summers deferring from examining the credibility of NCTQ are, first, to strongly disagree, and second, note that no journalists need to do any real investigative journalism to uncover that NCTQ has no credibility because all of that work has been done already by a number of scholars (see those critiques catalogued here and here).

As disturbing, however, as that stance is, examining carefully the coverage of NCTQ reveals that the mainstream media does in fact endorse NCTQ implicitly (despite claims of impartiality) and also marginalizes the credible critiques of NCTQ.

All three articles (see above) have headlines that establish immediately for any reader that NCTQ’s report is worthy of major media coverage. Next, all three articles have ledes that also present NCTQ positively:

The nation’s teacher-preparation programs have plenty of room for improvement, according to a new report. (Sanchez and Summers)

Alternative-certification programs for preparing teachers suffer from many of the same problems that the National Council on Teacher Quality has identified in traditional, university-based programs, the Washington-based group concludes in a new pilot study. (Sawchuk)

More than 75 percent of elementary teacher-preparation programs are failing when it comes to readying future teachers to work effectively with English-language learners, a new report from the National Council on Teacher Quality contends. (Maxwell)

Sanchez and Summers (again, note that Summers argues it isn’t her job to assign credibility to the study) certainly imply that the study is credible by using this language: “The study is a dismal read, given that the U.S. spends more than $6 billion each year to prepare teachers for the classroom.”

The NCTQ study is only a “dismal read” if it is accurate (and it isn’t). NCTQ has been carefully discredited in scholarship (for example, see Fuller here and here) for serious conflicts of interest (Teach For America and KIPP leaders sit on the Advisory Board, for example), for a flawed study design, and for shoddy methodology [5].

So how are credible academic critiques of NCTQ characterized in the journalism that claims not to take evaluative positions?:

When NCTQ released a version of this report last year, it was met with some skepticism among educators and those responsible for preparing teachers. Critics said the advocacy group should have visited individual teacher-prep programs and talked to graduates and students, rather than relying on syllabi. (Sanchez and Summers)

Last year’s inaugural teacher-prep review was immediately rejected by most teacher colleges and, especially, by their main membership body, the American Association of Colleges for Teacher Education.

Criticism focused on the NCTQ’s tack of reviewing syllabi and other course materials rather than visiting institutions; its use of open-records requests and current students to obtain documents; the complaint that its standards weren’t agreed to by the profession; and the fact that its research products aren’t peer reviewed. Additionally, critics have claimed that the project is ideologically driven, given NCTQ’s role as incubator of an alternative-certification group, the American Board for Certification of Teacher Excellence (ABCTE), which received federal funding from the George W. Bush administration.

The latter complaint seems less viable now that the NCTQ has turned its green eyeshade toward alternative-certification programs. (Sawchuk)

“Some skepticism” and “critics” clearly position credible scholarship negatively while maintaining the implied endorsement of NCTQ as an organization and NCTQ’s reports. And while Sawchuk appears to address more directly NCTQ’s lack of credibility, he still marginalizes scholars as “critics” and then in the last paragraph above, simply discounts the criticism [6].

Further in Sawchuk’s piece, the contrast between lacking credibility (NCTQ) and credibility (scholarship discrediting NCTQ) is reduced to a simple misunderstanding and a matter of tone (not substance):

Notably, the report’s introduction this year contains a number of mea culpas regarding the bad blood between the NCTQ and teacher colleges. And Walsh agreed that her group bore some of the blame.

“At times we were a bit arrogant about what it is we think teacher education should be doing,” she said. “Even if we agree to disagree, we can be more respectful.”

Again, this trivializes criticism of NCTQ and further equates NCTQ (an advocacy think tank) with scholarship—while also painting NCTQ as apologetic (despite the organization maintaining its threat of ranking programs whether they cooperate or not; a powerful tool afforded NCTQ because of its media partnership with U.S. News & World Report).

One of my strongest criticisms of teachers is that we far too often allow ourselves to be trapped within traditional calls that we take neutral stances; however, the U.S. needs critical teachers (political teachers) if our public schools are to be a foundation for our democracy.

What I have detailed above is that journalists in the U.S. have bowed to the same call for neutrality, one that cannot be accomplished but can serve as a shield for maintaining the status quo.

The U.S. needs critical journalists, ones who see their job as maintaining a commitment to seeking out and identifying the credibility of things they report. Only those in power benefit when the free press is mostly free of taking to task those in power.

Nowhere is that more apparent than in how the mainstream media fails the education reform debate.

[1] Journalists and teachers share the burden of traditional expectations that they should never be “political,” but taking a neutral stance is, in fact, taking a passive stance endorsing the status quo. In other words, taking a dispassionate pose is a political stance (see The Politics of Calling for No Politics).

[2] See a similar example with NPR’s coverage of “grit.”

[3] See the following in terms of how the mainstream media disproportionately reports on think tank (non-peer reviewed) reports as compared to peer-reviewed and university-based research:

Molnar, A. (2001, April 11). The media and educational research: What we know vs. what the public hears. Milwaukee, WI: Center for Education Research, Analysis, and Innovation. Retrieved from http://epsl.asu.edu/epru/documents/cerai-01-14.htm

Yettick, H. (2009). The research that reaches the public: Who produces the educational research mentioned in the news media? Boulder, CO and Tempe, AZ: Education and the Public Interest Center & Education Policy Research Unit. Retrieved from http://epicpolicy.org/publication/research-that-reaches

[4] I make this claim not as a direct attack on any journalists, but teachers and journalists now experience very similar and negative influences on their ability to conduct their professions. While education reform tends to impeded good teaching, the contracting media market has tended to overburden journalists. As a result of newspapers and magazines disappearing and contracting their staffs, many journalists resort to press-release journalism as a survival technique, similar to teachers teaching to the tests. The conditions of both professions, teaching and journalism, are stark reasons why both teachers and journalists must exert their political selves in their professional work.

[5] As a brief glimpse into NCTQ’s accidental admission of their methodology, in an effort to twist criticism of their practices, this post adds at the end “As one teacher candidate who is working on our office this summer said,” highlighting that anecdote is enough for NCTQ, as long as it matches their advocacy.

[6] Sawchuk fails to recognize that NCTQ is working within a scorched-earth policy as part of the large disaster capitalism driving education reform in the U.S. For a vivid example of how this works, and why NCTQ, TFA, and KIPP benefit once the traditional education system is dismantled, see the events that have occurred since Katrina in New Orleans where the public school system has been replaced by charters schools, many KIPP and many staffed by TFA recruits.

The Very Disappointing Teacher Impact Numbers from Chetty

It appears that Raj Chetty, of the very famous and mostly hypothetical Chetty et al. study addressing teacher quality’s impact on students’ lifetime earning potential, had a big influence on the Vergara case in California:

But one has to wonder how much impact that testimony would have had if the judge had considered that most reviews of the study find it to be poppy-cock (see Baker on the Chetty et al. molehill and Di Carlo) or if we simply displayed the numbers differently.

A 40-year “lifetime earnings” of $50,000 is only about $1250 a year, or about, after taxes, $75-80 per month, or about 1.5 to 2 tanks of gas a month.

Hmmmm. Compared to the $50,000 of hypothetical life-time earning or that insane $1.4 million for a class of 28, 1.5-2 tanks of gas a month appears to be much ado about nothing.

Chetty also fails to make a very important point about the economics of investing in measuring teacher quality: It will cost billions of dollars in public funding to create and implement a significant increase in high-stakes testing in order to hypothetically raise the monthly earning income of students enough for them to buy 1.5-2 tanks a gas each month.

A much more effective strategy would be to take those billions of dollars needed to create and implement VAM and give every worker in the U.S. $75 per month tax free. Simple, direct, and guaranteed.

Setting aside that this is all hypothetical, seems a ridiculously inefficient cost/benefit analysis of addressing a teacher quality issue that already is only about 10-15% of measurable student learning.

Gates Moratorium Another Scam: Beware the Roadbuilders pt. 2

The road to hell is not paved with good intentions. [1]

The road to hell in the U.S. of the 21st century is paved with the appearance of good intentions fostered by billionaires.

Billionaires are our roadbuilders, and in education reform the main roadbuilder is Bill Gates.

Gates is a billionaire education hobbyist who started a road to small schools, only to bail, but has since shifted his roadbuilding to value-added methods (VAM) for evaluating teachers and his tour de force superhighway, Common Core.

Now that Gates has issued a call for a moratorium on the intersecting roads to hell (VAM linked to next-generations high-stakes tests of the Common Core), we must return to two important points:

  1. I cannot stress or repeat often enough: If Bill Gates had no money, who would listen to him about education reform? No one–the same as who should listen to him now.
  2. Beware the roadbuilders.

To the first point, Gates has never had and does not now have any credibility as an authority on education or education reform. Zero. His commentaries linked to his huge bribes should be ignored when he advocates for policy, and his call for a moratorium should be ignored as well.

Delaying a road to hell still means we will have a road to hell.

To the second point, as Nettie and the Olinka learn in the Color Purple, the roadbuilders have an agenda to be done to those in their way and to benefit the roadbuilders. Words such as moratorium, philanthropy, and entrepreneur are thinly veiled code for not good intentions but the self-interests of the roadbuilders.

The roadbuilders are powerful because money speaks louder than words; however, the option before us is not a moratorium but a collective non-cooperation to end their roadbuilding.

[1] The best version of this cliche is in Ernest Hemingway’s The Sun Also Rises when Bill responds to Jake with the wonderfully ambiguous nod to the corrosive power of materialism: “‘Road to hell paved with unbought stuffed dogs’” (p. 78). In a capitalist society, the consumer’s hell is all the fake crap that consumer does not or cannot buy. The consumer doesn’t need the fake crap, of course, and there is never an end to the fake crap dangled before the consumer.

Death to Common Core! Long Live Failed Education Policy!

From the beginning of the call for and debate about Common Core, I have taken a clear stand against the promise of standards-based reform; for example, Why Common Standards Won’t Work:

A call for national standards ensures that we continue doing what is most wrong with our bureaucratic schools (establish-prescribe-measure) and that we persist in looking away from the largest cause of low student achievement: childhood poverty.

A call for national standards is a political veneer, a tragic waste of time and energy that would be better spent addressing real needs in the lives of children—safe homes, adequate and plentiful food, essential health care, and neighborhood schools that are not reflections of the neighborhoods where children live through no choice of their own.

Education is in no way short of a knowledge base. And even if it were, tinkering (yet again) at a standard core of knowledge while ignoring the dehumanizing practices in our schools, and the oppressive impact of poverty on the lives of children, is simply more fiddling while the futures of our children smolder over our shoulders and we look the other way.

So with the news of Oklahoma and my home state of South Carolina dropping Common Core, you’d think I would feel vindicated, but the truth is that if you dig beneath the (mostly partisan) policy moves to separate states from the Common Core movement, you find that those states maintain a robust and misguided commitment to all the reasons we should be dropping the Common Core.

For example, we must recognize that SC, specifically but as a typical example, is directly rejecting Common Core as a federal and thus flawed set of standards while continuing to develop and implement policy to design (yet again) new SC standards, new SC high-stakes testing, and new SC accountability—all of which are the essential structures that are ineffective.

The problem for education reform, then, is not specifically Common Core, but that the evidence base shows standards-based reform has not and will not address issues of equity or achievement.

As a parallel example of how most of the current education reform commitments simply do not and cannot address the fundamental problems facing education, consider that New Orleans has now replaced the entire public school system with charter schools; the result?:

White students disproportionately attend the best charter schools, while the worst are almost exclusively populated by African American students. Activists in New Orleans joined with others in Detroit and Newark last month to file a federal civil rights complaint, alleging that the city’s best-performing schools have admissions policies that exclude African American children. Those schools are overseen by the separate Orleans Parish School Board, and they don’t participate in OneApp, the city’s centralized school enrollment lottery.

The partisan political backlash against Common Core, then, is not reason to celebrate because the essential political commitments to misguided education reform policy (such as accountability built on standards and testing, charter schools, Teach For America, and value-added methods of teacher evaluation) remain robust below the partisan posturing against Common Core as a uniquely flawed set of standards.

Adopting and implementing Common Core and the related high-stakes testing and accountability mechanisms are tremendous wastes of time and money that we cannot afford. Yes, let’s stop Common Core, but as a key step to stopping the entire flawed education reform movement built on ever-new standards and tests.

Once Again, NAEP? Nope: “states and schools have lied about the rigor of their courses”

Some say the world will end in fire,
Some say in ice.

“Fire and Ice,” Robert Frost

While it appears I was right about Teacher Appreciation Week 2014, I was a tad bit off about the source of the Zombie Apocalypse or Armageddon: The world will not end because of PISA score rankings, but because of stagnant NAEP scores by high school students.

In fact, the U.S. Department of Education has just released a hot-off-the-press bumper sticker that celebrates Teacher Appreciation Week 2014 by acknowledging recent NAEP data:

USDOE TAW 2014

What happens when inept political leadership (Note: The 21st century prerequisite for holding the position of Secretary of Education appears to be a gentle blend of an absence of expertise and outright dishonesty related to NAEP reporting) collides with press-release journalism [1] (like an asteroid slamming into the Earth)?

Well, the claim made above by Schneider (“a vice president at the American Institutes of Research who previously led the government arm that administered NAEP”)—a truly ugly claim about education and teachers that appears to have been accepted without any request for evidence (Evidence? Secretary Duncan, You Can’t Handle the Evidence).

NAEP, then, once again prompts handwringing about stagnant scores and achievement gaps—and there are always charts and graphs to make the point along with the usual insincere nod to “the Civil Rights Issue of Our Time”:

U.S. Secretary of Education Arne Duncan said in a statement about the results, “We project that our nation’s public schools will become majority-minority this fall—making it even more urgent to put renewed attention into the academic rigor and equity of course offerings and into efforts to redesign high schools. We must reject educational stagnation in our high schools, and as [a] nation, we must do better for all students, especially for African-American and Latino students.”

Amongst the ugliness and baseless pontificating by political leaders are absent some key points that the media will fail (again) to uncover:

  • NAEP data are released and pronouncements made, but no one really knows the cause of the data concerns. Why scores appear stagnant and why racial/socioeconomic gaps persist are often complex (although a huge and evidence-based source of both is likely inequity and poverty). The initial reactions to NAEP this time in EdWeek and HuffPo are overwhelmingly speculation by people with political agendas. If we are genuinely interested in people who are likely telling lies, it appears we may want to look at the people cited in these articles.
  • “Achievement gap” is a misnomer for “opportunity gaps,” and using standardized tests to measure and examine that gap is inherently flawed since standardized testing remains biased by race, class, and gender; and thus, the tests themselves not only measure but create the gaps. Furthermore, for any gap to close, identified populations of students would need to be treated differently, but the current policy is a common core of what students experience in schools. And another dirty little secret is that the current era of accountability has damned high-poverty and minority students to test-prep course work that in fact asks less of them (thus, it is not “states and schools” that are telling lies, but politicians who shape accountability policy who are in fact telling lies).
  • Throughout the 20th and into the 21st centuries, we have found no correlation between how U.S. students do on test comparisons (among states or internationally) and claimed goals such as international competitiveness or the robustness of the U.S economy. None. And while we are at it, over the last three decades of accountability, we have found no correlation between the existence or quality of standards and measurable student outcomes. None. Again, it is a political lie to continue to cry “crisis” over test scores. A lie.

While I remain certain that accountability built on standards and high-stakes testing is a fundamental flaw in education reform, political leadership and the media are not doing us any favors either. This latest “high school achievement crisis” based on a rush to misread NAEP data is but more of the same—lamentably so as we certainly could do a better job even within the flawed test-based culture of U.S. education, as Matthew Di Carlo has outlined.

Childhood is steeped in a series of lies—what Kurt Vonnegut has labeled “foma,” although many of these lies are not so harmless: the Easter bunny, Santa Claus, work hard and be nice.

But one truism from our youths must be accepted as fact: Action speaks louder than words.

If we apply that to the USDOE, then we are likely to recognize just who is telling lies and about what:

  1. Lie: U.S. schools, teachers, and students are failing because of low standards and expectations.
  2. Lie: New standards and new tests will save public schools.
  3. Lie: State X is worse than State Y because NAEP (or SAT) scores say so; the U.S. is falling behind Country X because PISA scores say so.
  4. Lie: Poverty is not destiny.
  5. Lie: Arne Duncan (or Bill Gates or Michelle Rhee) knows what he is talking about.
  6. Lie: Education reform is the Civil Rights issue of our time.
  7. Lie: U.S. education is struggling because of “bad” teachers who are too hard to fire.
  8. Lie: Charter school X is a “miracle” school.

Truth: The USDOE is the embodiment of “lies, damned lies, and statistics.”

For Further Reading

I love the smell of NAEPalm in the morning

[1] See also Is It Journalism, or Just a Repackaged Press Release? Here’s a Tool to Help You Find Out.

 

PISA Brainwashing: Measure, Rank, Repeat

When Mary Catherine Bradshaw, a teacher since 1984 in Nashville, TN, announced her retirement from public schools, Bradshaw pointed her finger at one major reason, standardized testing:

[S]he says standardized testing is the reason….

Testing, she said, has taken away from instructional time and taken the joy out of learning.

Much has changed, she said, since she took her first job as a teacher at Hillsboro in 1984 when she said she was attracted to its diversity and commitment to academic reputation.

“There was more of a focus on the whole student, the joy of learning, building a community and finding one’s own passion in the midst of the K-12 experience,” she said.

“Now, with the focus on testing, data collection and closing a too narrowly defined gap among learners, I have found myself ready to retire from public education.”

Bradshaw’s concern about the loss of joy due to the central place of testing in education is echoed in a recent statement about PISA rankings [1], as Peter Wilby details in Academics warn international school league tables are killing ‘joy of learning’:

Now nearly 100 leading educational figures from around the world have issued an unprecedented challenge to Pisa – and what they call “the negative consequences” of its rankings – in a letter to its director, Andreas Schleicher….

“Education policy across the world is being driven by the single aim of pushing up national performance levels on Pisa,” says one signatory, Stephen Ball, professor at London university’s Institute of Education. “It’s having a tremendously distorting effect, right down to the level of classroom teaching.” Another signatory, Sally Tomlinson, research fellow at Oxford university’s education department, says that, though the Pisa league tables appear to be scientifically based, “you really can’t compare a country the size of Liechtenstein with one the size of China and nor can you compare education systems that developed over the years in different political, social and cultural contexts”.

The signatories are particularly concerned about the UK, the US and other countries imitating schools in Asian countries that come high in the Pisa rankings. They are suspicious of Shanghai’s success. “Shanghai’s approach is an incredibly strategic one,” says Ball. “Their students practise the tests. It’s difficult to see what their maths teachers can say to ours except ‘teach to the test’.”

While international rankings based on test scores have influenced public perception of U.S. public education for at least 60+ years (see Hyman Rickover’s books lamenting U.S. rankings, for example), state rankings based on NAEP and SAT/ACT scores have also been central to perception as well as policy, especially since the early 1980s.

While the open letter to Schleicher is a powerful and important challenge to the misleading influence of PISA, the essential problem is high-stakes testing coupled with ranking as well as a persistent misinterpretation of test data (see this excellent examination of how test scores are misunderstood and misused).

As I have addressed often about the SAT (see HERE and HERE), even when a comparison of states appears fair and accurate—South Carolina with Mississippi, for example, since the states share a similar high-poverty demographics of students—the reality is far more complex: MS has a higher SAT average score than SC because the test-taking populations of students are significantly different despite the overall student populations being similar:

Two Southern states, Mississippi and South Carolina, share both a long history of high poverty rates (Mississippi at over 30% and SC at over 25%) and reputations for poor schools systems. Yet, when we compare the SAT scores (pdf) from Mississippi in 2010 (CR 566, M 548, W 552 for a 1,666 total) to SAT scores in SC (CR 484, 495, 468 for a 1,447 total), we may be compelled to charge that Mississippi has overcome a higher poverty rate than South Carolina to achieve, on average, a score 219 points higher.

This conclusion, based on a “few data points”, is factually accurate, but ultimately misleading once we add just one more data point: the percentage of students taking the exam. Just 3% of Mississippi seniors took the exam, compared to 66% in South Carolina. A fact of statistics tells us that SC’s larger percentage taking the exam is much closer to the normal distribution of the all seniors in that state, thus the average must be lower than a uniquely elite population, such as in Mississippi. Here, the statistics determined by the populations taking the exam trump the raw data of test averages, even when placed in the context of poverty. (The truth about failure in US schools)

Even if the open letter about PISA prompts reform by the OECD, we have evidence that the problem will persist. For example, The College Board struggles with both the statistical complexity of SAT data (see here about the recentering) and the misleading use of SAT data to rank states:

Educators, the media and others should:

8.1 Not rank or rate teachers, educational institutions, districts or states solely on aggregate scores derived from tests
that are intended primarily as a measure of individual students. Do not use aggregate scores as the single measure to
rank or rate teachers, educational institutions, districts, or states.

And yet, each year when SAT data are released, the media, political leaders, and public school critics rank states and pronounce schools a failure.

The open letter about PISA implores, “Slow down the testing juggernaut,” adding:

OECD’s narrow focus on standardised testing risks turning learning into drudgery and killing the joy of learning. As Pisa has led many governments into an international competition for higher test scores, OECD has assumed the power to shape education policy around the world, with no debate about the necessity or limitations of OECD’s goals. We are deeply concerned that measuring a great diversity of educational traditions and cultures using a single, narrow, biased yardstick could, in the end, do irreparable harm to our schools and our students.

Once we apply the brakes, we must then take a close look at the fundamental policy errors—high-stakes standardized testing, labeling, sorting, and ranking—and then abandon those practices for alternatives that address inequity both outside and inside schools and that honor the essential dignity and humanity of students and their teachers.

For Further Reading

Among the Many Things Wrong With International Achievement Comparisons, Gene V. Glass

More Things Wrong with International Assessments Like PISA, Gene V. Glass

[1] As full disclosure, I am a signatory on the letter.

Welcome to SC: A Heaping Stumbling-Bumbling Mess of Ineptitude

This is my 53rd year of living in South Carolina, the totality of my life.

This is my 31st year as an educator in SC—18 years as a high school English teacher and 13 years now in higher education.

My teaching career, coincidentally, began the exact year SC officially stepped into the accountability/standards/testing arms race that grew out of the early 1980s.

Over the past 30 years, SC has created, implemented, revised, and changed a nearly mind-boggling array of standards and tests:

  • SC Frameworks
  • SC state standards (revised multiple times)
  • BSAP and exit exams
  • PACT
  • PASS
  • HSAP and EOC (end-of-course) tests
  • Common Core and high-stakes tests TBD
  • Two concurrent and competing sets of school report cards (the long-standing state version and the federal letter-grade based version)

Before I move on, let me add that SC is a high-poverty state (in the bottom quartile of poverty in the U.S.) that is historically and increasingly racially diverse, a right-to-work state that picks fights with unions that have no power, and a challenging environment for children of color (in the bottom third of the U.S. for African American and Latino/a children).

So now let’s return to the Accountability Hunger Games: SC Edition:

SC Senate approves replacing Common Core in 1 year

That’s right, before SC schools, teachers, and students can actually make the transition from the repeatedly revised (and obviously failed) standards-and-tests Merry-Go-Round of state-based accountability to the all-mighty Common Core gravy train of world-class and college-ready standards and next-generation high-stakes tests [insert trumpet]:

The Senate on Thursday unanimously approved a bill that replaces Common Core education standards with those developed in South Carolina by the 2015-16 school year.

The bill, which passed 42-0, is a compromise of legislation that initially sought to repeal the math and reading standards that have been rolled out in classrooms statewide since their adoption by two state boards in 2010. Testing aligned to those standards must start next year, using new tests that assess college and career readiness, or the state will lose its waiver from the all-or-nothing provisions of the federal No Child Left Behind law.

But the state won’t be able to use tests South Carolina officials helped create with 21 other states. A bid must go out by September for their replacement.

[Insert wah-wah-wah]

As ridiculous and muddled as that all is (and I think “heaping stumbling-bumbling mess of ineptitude” may be understating the level of ridiculousness), Seanna Adcox’s coverage of this likely next-move for SC is chock-full of even more ineptitude so let me counts the ways:

  1.  “‘We’re back on track,’ said Sen. Mike Fair, R-Greenville.” Fair has built a political career on his yearly efforts to dismantle SC’s science standards by inserting an endless series of not-so-clever Creationism edits to the evolution elements of those standards. [HINT: He may not be the best authority on SC's decisions about standards.]
  2. “Democrats say the bill forces the Legislature to spend money on technology in classrooms….” Ironically and sadly, SC is an equal-opportunity state in terms of political party ineptitude. SC is notorious for our Corridor of Shame, a swipe of high-poverty communities that roughly follows I-95 across the state. I will simply ask that you return to the Kids Count report on childhood opportunity and consider where tax dollars may be better spent than on technology investments for computer-based high-stakes testing that will further stigmatize the growing number of poor children of color in SC. [HINT: It ain't on more technology that will fail and become obsolete.]
  3. “Computer testing allows for a better assessment of both students’ abilities and teachers’ effectiveness….” And nothing like baseless and inaccurate claims to help! [HINT: Nope.]
  4. “‘This is about maintaining control,’ said Campsen, R-Isle of Palms. ‘We shouldn’t cede our authority over children’s education to an outside process.’” See above and consider the smashing good job SC has done on its own for three decades. [HINT: That last sentence is sarcasm.]
  5. “Both the House and Senate budget proposals would spend about $30 million on technology next school year, focusing on rural districts.” $30 million on technology. [HINT: $30 million on technology.]

Unless that technology plan includes a provision for turning the iPads purchased into food trays once they are obsolete in a few months, I would posit that this entire farce is beyond ineptitude.

And I must add: SC is not some looney example of ineptitude in the world of education reform (although we do tend to be on the outer edges of looney in many things); in fact, the series of fits and starts that constitute SC’s heaping stumbling-bumbling mess of ineptitude are being replicated all across the U.S. as political manipulation of education collides with Tea Party lunacy.

If we may pause, then, and consider the real problems and the likely solutions: SC has an equity problem in our state and thus in our schools, and accountability/standards/testing do not address those essential equity problems.

SC must step off the accountability Merry-Go-Round, but this latest effort suggests we are enjoying the circus too much to make any reasonable decisions.

Legalizing Marijuana Offers Lesson for Changing Course in Education Reform

The role of causality in educational research needs to be questioned on the basis that education is not the same as medicine. As Biesta says: “Being a student is not an illness, just as teaching is not a cure.” (2007, p8) We should never assume that education is a “push and pull” process of simply linear causal relationships.

Tait Coles, Take no heroes; only inspiration.

“Batman has officially been kicking the ass of Gotham’s villains for 75 years,” explains Ryan Kristobak, “and so to honor the Dark Knight, the Warner Bros. panel unveiled the ‘Batman Beyond’ animated short at this year’s WonderCon.”

For long-time and recent fans of Batman, however, the legends of the Dark Knight are complicated by the many versions that exist among the DC comic book and graphic novel universe, films, TV, animated series, and video games.

The Batman Myth has several foundational characteristics and common themes that are nested in the Caped Crusader’s first appearance in Detective Comics 27 in 1940: Batman’s essential nature as a detective and crime fighter, the ambiguous relationship between Batman and the Gotham police department and city officials, and the larger themes about justice that are contrasted by Batman’s vigilante tendencies.

In The Dark Knight Rises, the final installment of the film trilogy directed by Christopher Nolan and starring Christian Bale, the opening scene framing the film also highlights a central message reflecting how justice is traditionally characterized in the U.S. The mayor of Gotham and Commissioner Gordon preside over Harvey Dent Day, named for the district attorney who is killed as Two-Face in The Dark Knight:

[the Mayor is giving a speech being at hosted at Wayne Manor]

Mayor: Harvey Dent Day may not be our oldest public holiday, but we’re here tonight because it’s one of the most important. Harvey Dent’s uncompromising stand against organized crime has made Gotham a safer place than it was at the time of his death, eight years ago. This city has seen a historic turn around. No city is without crime, but this city is without organized crime because of Dent’s act gave law enforcement teeth in its fight against the mob. Now people are talking about repealing the Dent Act, and to them I say, not on my watch.

[the audience claps]

Mayor: I wanna thank the Wayne Foundation for hosting this event, and I’m told, Mr. Wayne couldn’t be here tonight. I’m sure he’s with us in spirit….

Mayor: Jim Gordon, can tell you the truth about Harvey Dent. He could…but I’ll let him tell you himself. Commissioner Gordon!

[the audience claps as Gordon makes his way to the stand, Gordon looks down at his prepared speech and says to himself as he remembers the real truth of what happened to Dent]

Commissioner Gordon: The truth…

[he addresses the audience]

Commissioner Gordon: I have a speech telling the truth about Harvey Dent. Maybe the time isn’t right.

[he puts the speech away in his jacket pocket]

Commissioner Gordon: Maybe right now, all you need to know is that there are one thousand inmates in Blackgate Prison as the direct result of the Dent Act. These are violent criminals, essential cogs in the organized crime machine. Maybe, for now, all I should say about the death of Harvey Dent is this; it has not been for nothing. (transcript found here)

Justice in Nolan’s Gotham reflects the central elements of justice found in the U.S.: the right laws, the right people to enforce those laws, and the evidence those laws are working represented by a growing prison population.

Reagan Era Mass Incarceration and Education Accountability

As I have detailed in Education Reform in the New Jim Crow Era, the 1980s and the Reagan administration planted the seeds of both an era of mass incarceration, labeled the New Jim Crow by Michelle Alexander, and the high-stakes accountability era in public education.

The most troubling aspects of both mass incarceration and high-stakes education accountability are that the policies have created, not ended, the claimed problems they were designed to address.

Over the past thirty years, the criminal justice system in the U.S. has filled prisons with a disproportionate number of African American men as part of our most recent war on drugs—despite whites and African Americans using recreational drugs at the same rates.

The current era of mass incarceration has unintended consequences similar to prohibition in the 1920s and 1930s:

Prohibition turned law-abiding citizens into criminals, made a mockery of the justice system, caused illicit drinking to seem glamorous and fun, encouraged neighborhood gangs to become national crime syndicates, permitted government officials to bend and sometimes even break the law, and fostered cynicism and hypocrisy that corroded the social contract all across the country. With Prohibition in place, but ineffectively enforced, one observer noted, America had hardly freed itself from the scourge of alcohol abuse – instead, the “drys” had their law, while the “wets” had their liquor.

The recent legalization of marijuana suggests a possible social recognition that traditional views of the right laws enforced by the right people and resulting in the right people sitting in prison is the wrong formula for either justice or a peaceful and equitable society.

Along with a growing number of states legalizing or decriminalizing marijuana is a concurrent discussion of releasing prior drug offenders from prison, again suggesting a social admission that the laws we establish create criminals, but rarely deter crime.

Seeking justice must not be separated from seeking equity. If the shift in how people in the U.S. view marijuana signals anything, I think, it shows a broader concern for equity: Just as changing inequitable laws surrounding powder cocaine and crack came to represent an inequitable criminal justice system, legalizing marijuana is yet another effort to move the pursuit of justice in the U.S. toward a pursuit of equity.

Legalizing Marijuana: A Lesson for Changing Course in Education Reform

The war on drugs and the resulting mass incarceration have proven to be the wrong policies for achieving justice or equity in the U.S. Directly, we know that mass incarceration negatively impacts children (see Holly Yettick and Children of the Prison Boom).

But the parallel era of high-stakes education accountability shares the central flaws now being recognized in mass incarceration: high-stakes accountability creates failure in schools, teachers, and students (see FairTest’s Reports: High Stakes Testing Hurts Education).

Under Barack Obama and Secretary of Education Arne Duncan, federal and state education policies have remained focused on identifying the right standards and the right tests, most recently Common Core standards and so-called “next generation” tests. Unlike the move toward legalizing marijuana, education reform remains trapped and unable to see the Bitter Lessons from Chasing Better Tests, as Duncan proclaimed in 2009:

Until states develop better assessments—which we will support and fund through Race to the Top—we must rely on standardized tests to monitor progress—but this is an important area for reform and an important conversation to have.

Debating the quality of Common Core and the related tests, however, are the wrong arguments because high-stakes accountability is the wrong policy paradigm just as the war on drugs and mass incarceration are the wrong policies for justice.

Adopting and implementing Common Core as yet another round of seeking the right standards and the right tests will not work. We have three decades of evidence on that approach revealing that there is no correlation between the existence or quality of standards and student achievement (see Mathis, 2012).

The war on drugs has proven to be finding ourselves in a hole and continuing to dig. Legalizing marijuana is dropping the shovel and choosing instead to acknowledge that failure and to try another approach, one more rightly attuned to equity.

This is a lesson high-stakes accountability advocates need to learn.

Common Core and the related high-stakes tests are the wrong approach to equity and high-quality education; they are finding ourselves in a hole we created and continuing to dig.

As legalizing marijuana signals a possible turn to the end of mass incarceration, we need also to end the era of high-stakes accountability in education.

Let’s choose instead An Alternative to Accountability-Based Education Reform.