Quality of So-Called “Education” Journalism Actually Low

In both my May Experience course on education documentaries and my foundations in education course, we view and discuss the 2008 HBO documentary Hard Times at Douglass High:

Shot in classic cinema verité style, the film captures the complex realities of life at Douglass, and provides a context for the national debate over the controversial No Child Left Behind Act, focusing on the brutal inequalities of American minority education, considered an American tragedy by many.

Although many scenes are powerful, one in particular remains disturbingly relevant in 2015: The camera captures with a voice-over students taking the standardized test that is being field-tested for students (no stakes), but will be high-stakes for the school and teachers; many of the students are shown with their heads down, essentially making no effort on the test.

Hard Times ends by noting that the administration has been replaced and Douglass High (Baltimore, MD) joins one of many narratives that too often we read about in the on-going era of high-stakes accountability: failed schools, schools “taken over” by the state, closed schools.

A few years ago, I was working on an Op-Ed for The State (Columbia, SC), but I was challenged about my outline of the accountability movement in South Carolina by the editor. Just for context, I began teaching in SC in 1984, when the first implementation of accountability began, linked to higher teacher pay, greater educational funding, and the start of the standards/high-stakes testing movement.

The editor insisted that accountability was a child of the late 1990s, but I was able to send her links to the first SC laws in the late 1970s and explained my own life as a teacher at a school where we were actively teaching to the exit exam in the early and mid-1980s (including double-tracking students in math and ELA courses as tenth graders to help them pass the tests to graduate).

What do these two topics above have to do with each other?

For thirty years, journalism addressing education and more specifically education reform has been inadequate to the point of being a huge part of the education reform problem.

Take for yet another example this piece from The Hechinger Report (and a repost in Education Week): Stakes for “high-stakes” tests are actually pretty low.

The maps, data, and serious tone are likely to have masked the flippant headline as well as terse “gotcha” lede: “It turns out that the stakes for this spring’s Common Core-aligned tests are not quite as high as they might seem.”

Seems all that opt-out nonsense and teacher caterwauling has been for naught, right?

Just as I suspected. As the article clarifies early, “both sides” are truly out-of-bounds:

“I think the stakes are either overstated or understated depending on which side of the argument you’re on,” said Chris Minnich, executive director of the Council of Chief State School Officers. “Both sides need to take a step back and just take a look at this map.”

As one point of concern, however, let’s consider another piece in the same publication: More than 5,000 Mississippi third-graders could be held back this year for low reading scores:

Results of the new third-grade reading test announced Thursday that aimed to make it tougher for students to advance if they don’t read at grade level could mean 15 percent of the test-takers will repeat third grade.

Some 38,000 public school students took the Third-Grade Reading Summative Assessment, widely known as the “third-grade gate,” created under state law to address lagging reading skills and prevent the practice of “social promotions.”

I wonder how these children being mis-served by callous legislation refuted by decades of research on grade retention and rejected by the National Council of Teachers of English feel about flippant and misleading journalism? [1]

Where has the mainstream press examined that grade retention doesn’t have “two sides,” but one very clear position supported by evidence?

Where has the mainstream press examined that standardized testing remains biased against racial minorities, the impoverished, English language learners, special needs students, and females?

Where has the mainstream press exposed that the entire accountability era has failed?

Don’t bother looking, the mainstream media is too busy being snarking, inadequate, and lead by the nose in the era of press-release journalism that has coincided with educational accountability.

The press is a willing participant in the “miracle” school lies, as long as they are about charter schools [2], but quick to vilify teachers who cheat.

Journalists serve as bridges between a more technical and complex world (political, academic, etc.) and the general public, many of whom spend little time beyond the headlines and a few sentences at the beginning and maybe the handy-dandy charts, graphs, and maps.

So let me return to the claim that “both sides” are misrepresenting the stakes surrounding the on-going accountability/standards/testing game that has now lingered for thirty years in the U.S.

Please, mainstream media, identify for me and your audience any states in which accountability/standards/testing are not ultimately geared toward high-stakes for students, teachers, and schools? (Note: That data point, by the way, would be 0).

And since all aspects of accountability are linked ultimately to high-stakes, in what way is this incomplete, misleading, and snarking “report” helping anyone—especially the children who have been and are now having their lives irrevocably changed due to inexcusable legislation with no basis in solid research?

The original breezy piece now includes an UPDATE, but even so, the essential problem remains that most people will see only the headline, maybe the lede, and then the maps. The conversation has been established by this piece even to its shoddy conclusion that includes a convenient Oliver North passive voice evasion:

All of which is to say, yes, the tests are important. Decisions will be made based on how students perform on them [emphasis added]. But the vast majority of states will use the scores only as one measure in a web of other factors when making staffing decisions. And most states have no plans to use the scores to make student advancement decisions.

Although the process would probably be pointless since journalists are trained to chase “both sides” (which tends to be one side that is credible and then another that is not), this piece could have been saved to some degree by talking with educators and assessment experts who could share that in the evidence around exit exams, grade retention, and teacher evaluations linked to test scores, a clear pattern has emerged: even when test scores are “one measure in a web of other factors,” those scores either distort that “web” or ultimately become the determining factor in that “web.”

As I have detailed before, at universities that use a “web” of factors to determine college admission, the SAT, even when weighted low, serves as a gatekeeper as those “other factors” cancel each other out. In other words, “one measure in a web of other factors” is a political scam being perpetuated by a non-critical press.

In the accountability game, this reality is even uglier since there is only one constant in the standards/testing movement: the standards and tests are constantly changing.

If anyone wants to begin to understand the dual disasters which are the accountability movement of recent history and the historical failure of providing children of color and impoverished children the educational opportunities they deserve, I suggest avoiding the mainstream press and simply spending some time with Hard Times at Douglass High.

The documentary is a hard watch, but its stark and complex examination rises above simplistic and breezy claims that trivialize children and educators in ways that occur daily in mainstream education journalism.

[1] See Retaining 3rd Graders: Child Abuse, Mississippi Style and Mississippi Reader.

[2] See Bruce Baker’s excellent The Willful Ignorance of the NJ Star Ledger.

What If Standardized Tests Were Biased Against Whites, Males, Affluent?

In my Marvel collecting days, I bought the first issue of What If?—begun in February of 1977 with “What If Spider-Man Joined the Fantastic Four?”

Marvel and DC have since then ventured into rewriting their comic book universes and even creating alternate universes for such thought experiments, but in the late 1970s, this was exciting stuff.

So much so, I want to apply this concept to standardized testing to ask the following:

1972 2014 SAT M v. F

  • What if standardized tests were biased against white, affluent males in ways that denied them access to college? Data from the 2005 SAT (the last year they separated data by gender and wealth) show that impoverished females disproportionately take the SAT while affluent males disproportionately take the SAT; see this chart:

2005 SAT M v. F plus wealth

  • What if standardized tests used to retain children in 3rd grade disproportionately retained while, affluent males? Grade retention impacts significantly impoverished and minority students, both pre-high-stakes accountability and then post-NCLB. From a 2000 research analysis of grade retention:

We review the policy context of school retention and show that age-grade retardation has been common and growing in American schools from the 1970s through the 1990s. Our analysis focuses on the period from 1972 to 1998 and on grade retardation at ages 6, 9, 12, 15, and 17. By age 9, the odds of graderetardation among African-American and Hispanic youth are 50 percent larger than among White youth, but these differentials are almost entirely explained by social and economic deprivation among minority youth, along with unfavorable geographic location. Because rates of age-grade retardation have increased at the same time that social background conditions have become more favorable to rapid progress through school, the observed trend toward more age-grade retardation substantially understates growth in the practice of holding students back in school. While there is presently little evidence of direct race-ethnic discrimination in progress through the elementary and secondary grades, the recent movement toward high stakes testing for promotion could magnify race-ethnic differentials in retention.

And then from a 2013-2014 position statement from the International Reading Association:

African American and Hispanic students and students living in poverty are most affected by grade retention practices that use the results of high-stakes assessments for decision making. Achievement patterns reveal wide disparities between the achievement of white students and that of African Americans and Hispanics (National Center for Education Statistics, 2011); thus, it follows that there would be similar differences in the number of students retained in each subgroup. In 2009–2010, African American students represented 49% and 56% of the third and fourth graders who were retained, respectively, which was disproportionate to their representation in those grades; Hispanic students were twice as likely to be retained than their white counterparts (Adams, Robelen, & Shah, 2012)….

  • What if standardized tests were biased against white, affluent males, denying them high school graduation? Also from the IRA position statement:

As with the outcomes of third-grade retention policies, African Americans, Hispanics, and students living in poverty are most affected by the use of high-stakes assessments for diploma decisions….

Policymakers may believe that linking grade retention and high school graduation to students’ results on high-stakes assessments will motivate students to perform better, but instead, evidence indicates that these practices have harsh and lasting consequences for students academically, psychologically, socially, and economically (Baker & Lang, 2012; Jimerson, 1999; Jimerson, Anderson, & Whipple, 2002; Norton, 2011; Walker & Madhere, 1987; Yamamoto & Byrnes, 1984).

I must, then, ask broadly, what if standardized tests were historically and then currently a powerful metric that closed doors for determining the educational and life opportunities of while, affluent males?

Would there be the same unyielding defense of the necessity for high-stakes tests?

Bonus What If…?

What if, instead of declaring race-based considerations for college admission illegal, we banned the use of legacy admissions? Historically and currently, for the outlier white, affluent male who does not score high on standardized tests of college admission another door is open wide, legacy admissions.

Yes, To Be Clear, I Am Anti-Testing, Anti-Grading

Since the early to mid-1990s, I have actively practiced and preached de-testing and de-grading as an educator.

So, to be clear and not as some ploy to be provocative or to slip into hyperbole, I am solidly anti-testing as well as anti-grading.

That stance is based on a very simple point of logic: Tests and grades have been central to formal education for over a century, and the stakes of those tests and grades have dramatically increased over the last three decades; yet, virtually no one is satisfied with our system or so-called “student achievement.”

In the colloquial parlance of my South, we cannot admit that weighing a pig doesn’t make it fatter.

However, virtually every time I speak publicly, write a public piece, or am interviewed by the media about testing and grades, I come against something like this from Jordan Shapiro:

If we consider standardized testing in schools, it is clear to me that many folks get caught up in the fire of the debate and lose the ability to see both sides of the story clearly. Those who take an extreme anti-testing position are well meaning. They want to protect children’s individuality. They want to shield them from unnecessary anxiety. They want to protect valuable learning time. They want to spare children the indignity of punching chads and filling in circles. And they want to empower young people by providing them with life-long experiential learning skills.

But some of these critics also seem to forget that those who advocate for measured accountability are also well meaning….

Ultimately, there’s no way for the Federal Department of Education to equitably serve the 50 million students who attend public schools in the United States without some sort of assessment data. But do the current tests provide meaningful data? The critics say no. The advocates point out that all data is ultimately incomplete, but that doesn’t make it worthless.

Typically, the reasonable position is that both sides have good and bad; as well, the final point always swing back to “OK, standardized tests (and even grades) are misleading, flawed, and all that, but we have to have something (which means just plowing ahead with flawed tests and grades).”

This sort of common sense journalistic approach (everything is reduced to “both sides” and then each side is treated as if equal) coupled with fatalism fueled by a refusal to back up far enough to reconsider norms is a false objectivity that can only reinforce the status quo.

Therefore, along with my appeal to logic and confronting a very long history of how tests and grades have failed our students and our formal education system, we have, ironically I think, a tremendous body of data: Standardized test data are overwhelmingly and persistently correlated to social class of students’ families and remain linked to race and gender biases. Those ugly roots of standardized testing (IQ, etc.) are not mere historical artifacts since all standardized testing continues to exhibit the worst elements of inequity exposed in those roots.

And if we genuinely investigate our commitment to data, the College Board’s own research on the predictive value of the SAT when compared to simple GPA is a powerful argument against standardized testing and common sense proposals like Shapiro’s above because GPA trumps the SAT as a valuable metric.

Even though I reject traditional classroom-based grading, hundreds of grades assigned among dozens of teachers over many years (logically again) serve our need to address accountability far better than a one-shot standardized test.

This leads me to suspect that advocates of standardized tests are not as enamored with tests as much as they simply distrust teachers, but again, the data refute that distrust.

And my additional recognition is that standardized test advocates do not love the tests as much as they love how standardized testing reinforces and perpetuates their privilege: high-stakes exit exams do not gatekeep the wealthy, college entrance exams do not gatekeep the wealthy, third-grade retention based on standardized tests do not hold back the wealthy.

Standardized tests have a false allure of objectivity, a bureaucratic allure of efficiency, and a traditional allure since they have always been central to formal schooling. But most significantly, standardized testing serves the interests of the privileged—at the expense of minority and disadvantaged populations.

In the context of equity and education, standardized tests have failed, repeatedly; they are a tragic drain on school funding and instructional time, and to what end?

Instead of tests or even grades, students need rich and engaging learning experiences that include high-quality feedback from their teachers and ample time to revisit those students’ demonstrations of learning.

One teacher or even one artifact of learning doesn’t mean much at any fixed point in time.

Education occurs in fits and starts over many, many years and within a complex matrix of influences (some “bad” experiences are “good” in terms of learning).

Tests and grades are inadequate for teaching and learning, and they simply do far more harm than good.

The evidence is overwhelming for that claim, and to argue otherwise is not simply “the other side,” and it is not reasonable or justifiable because test and grade advocates also want what is best for students.

Continuing to cling to tests and grades is clinging to very negative views of human nature (especially in children) and of teachers.

I am anti-testing and anti-grading because I have committed my life to children and young people, to the complicated and unpredictable art of teaching as an act of social justice, a pursuit of equity.

Testing and grading have not built an equitable system of formal education in the U.S. (in fact, testing and grading have labeled and then perpetuated inequity); therefore, to argue that we must continue both in order to reach that goal is a grand failure of understanding the very evidence advocates claim to understand.

What opportunities and experiences are we guaranteeing all students?—this is the thing to which we must be accountable, not simplistic metrics that serve only to quantify the very inequities we refuse to acknowledge or change.

For Further Reading

Email to My Students: “the luxury of being thankful”

To My Students at the End of the Semester

Grades Fail Student Engagement with Learning

Tests don’t improve learning. And PARCC will be no different

Co-authored with Schmidt, R. (2009). 21st century literacy: If we are scripted, are we literate? Heidelberg, Germany: Springer.

Education Needs a Collaboration (Non-Competitive) Pact

“The propagandist’s purpose is to make one set of people forget
that certain other sets of people are human.”
Huxley, Aldous. The Olive Tree. 1936.

While watching a documentary on schools recently, I felt that same uncomfortable feeling I do whenever I watch or read about this or that school “excelling”—notably the principal, but teachers as well, expressing how they have something different that is driving the school’s success.

Of course that claim caries the implication that other schools, teachers, and students are not doing that something different (hint: trying hard enough, demanding enough).

In this particular documentary, that something different included publicly identifying, labeling, and displaying students by test scores.

And while I have a great deal of compassion and collegial support for educators fighting the standardized testing craze corrupting U.S. public education, I feel compelled to note that many of those same educators turn right around and practice the same sort of tyranny with students—or quickly wave the testing data flag when their school seems to look good (although these claims of “miracles” are almost always mirages).

So here is a test we should all take.

Check all that apply: As a teacher or administrator in a school, do you …

[ ] use test scores to rank, compare, motivate, and/or shame students into working “harder”?

[ ] use test scores to rank, compare, motivate, and/or shame teachers within a department, grade level, or school into working “harder”?

[ ] use test scores to brag about your department, grade level, or school to parents or the media?

If any of these are checked, you have a decision: either stop complaining about high-stakes uses of test scores or stop doing all of the above.

If test scores are a flawed way to evaluate teachers and schools, they are a flawed way to evaluate teachers, schools, and students—and even when they work in your favor.

Thus, I recommend the latter choice above because education needs a collaboration (non-competitive) pact if we are to save the soul of our profession.

Recommended

Why competitive model fails schools. No one should lose in education, Alfredo Gaete and Stephanie Jones

De-Testing and De-Grading Schools: Authentic Alternatives to Accountability and Standardization, Bower and Thomas, eds.

Competition: A Multidisciplinary Analysis, Worthen, Henderson, Rasmussen, and Benson, eds.

To Catch a Cheat: More on the Pearson Problem as Our Problem

“Cheating by test takers is becoming more common in the United States and throughout the world,” explains T.J. Bliss, adding:

In the past year, multiple news agencies have reported several instances of cheating on high-stakes tests. Recently, news broke that doctors in a variety of specialties had cheated to pass certification exams (Zamost, Griffen, & Ansari, 2012). In another instance, high school students were arrested and charged with misdemeanors and felonies for cheating on the SAT (Anderson, 2011). At a university in Florida, over 200 students admitted to cheating on a midterm exam when faced with accusations based on statistical evidence (Good, 2010).

So what are teachers to do? Bliss offers evidence-based solutions:

There are many ways to detect cheating, some more useful and reliable than others (Cizek, 1999). Proctors and invigilators can walk the exam room and directly observe some forms of cheating, like answer copying. This method will not work, though, if a person cheats by gaining pre-knowledge of exam items (Good, 2010), is taking the exam for someone else (Anderson, 2011), or is trying to memorize items to share with others (Zamost et al. 2012). Some cheating is detected through whistle-blowers, manual comparison of answer and seating charts, and other qualitative approaches (Cizek, 2006). However, in both large-scale and classroom testing situations, statistical approaches have also been used to identify suspected cheaters. Such methods have been successfully utilized to detect several different kinds of cheating, including answer copying, collusion, pre-knowledge, and attempts to memorize items.

And why the increased cheating? It seems legislation, competition, and technology have roles in that:

With the passage of legislation requiring increased school accountability (e.g. No Child Left Behind Act, 2001) and increased compet[ti]iveness for jobs requiring certification (United States Bureau of Labor Statistics, 2010-2011) the stakes for passing standardized and licensure exams have increased dramatically. At the same time, technologies to enable cheating have also increased. For instance, some examinees have begun using smart phones, digital recorders, and other personal electronic devices to cheat during exams. Fortunately, the advent of new methods for administering exams (like Computer Adaptive Testing) and analyzing test results (like Item Response Theory) have led to the development of more complex and sensitive statistical methods to detect cheating.

For classroom teachers seeking ways to prevent cheating and catch students who cheat, the Internet offers a nearly endless supply of strategies:

To be honest, I could continue that list for dozens and dozens of bullets, all of which have about the same strategies.

How many of us as classroom teachers at all grade levels have given traditional tests such as multiple-choice, fill-in-the-blank, matching, etc., formats? How many of us have implemented some or many of the cheating prevention strategies commonly taught in methods courses such as creating several versions of the tests, asking students to cover their work, re-arranging desks and seating assignments, walking around the room during the tests?

And how many of us are outraged at Pearson and other testing corporations for monitoring social media in order to detect cheating and thus to protect the credibility of their product (as any business would do in a consumer society)?

How many of us as classroom teachers are rightfully angry about the misuse of value-added methods (VAM) for teacher evaluation and pay, and then hold our students accountable for tests in traditional formats in our classes?

How many of us help select, purchase, and then implement commercial programs for teaching and assessing our students?

I teach and co-teach several methods courses for candidates seeking certification to teach in high school. As a critical educator, my classes go something like this: For Topic A, here is what traditional/progressive educators say you should do, but here is what critical educators suggest.

Yes, this is a bit tedious, for me and, I suppose, my students.

In the methods course I co-teach, I am responsible for assessment, so the topics of tests and cheating are addressed.

In order to help my future teachers develop a critical lens, we often talk about their experiences with assessment and cheating while college students—by focusing on writing essays and being monitored for plagiarism.

Most of these students are familiar with directly or indirectly faculty using TurnItIn.com to evaluate plagiarism in student essays. And like most of the faculty, my students see nothing problematic about using that technology both to discourage and detect cheating.

As high-achieving students, my students tend to be harsh about cheating—until we start investigating their own behavior as students. Until we start unpacking what counts as cheating (for example, TUrnItIn.com allows each professor to manipulate the threshold for plagiarism, thus changing what counts as plagiarism from course to course).

Is sharing homework cheating? Is collaborating while writing an essay cheating? Is peer-editing an essay cheating?

And then we go further by asking why students cheat, and considering why students plagiarize.

Certainly all college students know not to plagiarize.

My larger point is about the conditions created in the classroom that foster cheating.

I explain to my students why I don’t give grades and don’t use traditional tests. My students submit multiple drafts of all essays and often spend a good deal of time in class drafting with me there to help.

I also detail how I incorporate a group midterm exam in my foundations of education course, the testing format being both small-group and whole class discussion with a focus on learning being a social construct.

I have consciously over 30-plus years sought ways to end the sort of testing culture that fosters cheating and instead implemented approaches to assessment that encourage full engagement that makes cheating (typically) not even an option.

Unable to avoid my English teacher Self, I tend to add that in Margaret Atwood’s The Handmaid’s Tale, a powerful theme addresses the consequences of “reduced circumstances”; the main character Offred/June, who appears to be a decent person before the world crumbles, expresses murderous cravings—fantasizing about stabbing someone with a knitting needle and feeling the blood run warm over her hand.

This dystopian novel forces readers to consider the sources of the violent urges—are they inherent in Offred/June or prompted by her reduced circumstances?

That same sort of critical inspection must be a part of both the wider education reform movement and our own classrooms.

Under high-stakes accountability, why the cheating scandals in Atlanta, DC, and elsewhere?

But also, why are students cheating in our classes?

Why do students not read assigned works? Why do students claim they hate to read?

As teachers and public school advocates, to maintain our gaze only on outcomes is to miss the reasons for those outcomes, and to avoid our own culpability.

And then we come back to Pearson and the commercial boom connected directly to the accountability era.

A Software & Information Industry Association report reveals, “testing and assessment products–which include software, digital content and related digital services–now make up the largest single category of educational technology sales,” increasing by 57% since 2012-2013:

testing and assessment 57 percent

That tax-payer’s cash grab combined with what Anthony Cody calls “necessary surveillance” for national high-stakes testing is a cancer on the educational body—one that must be not only cured, but also eradicated and then prevented.

However, we must also admit that this cancer is the result of cancer-causing behavior.

As teachers and public education advocates, we must continue to fight the profits and surveillance of Pearson and other commercial interests, but we also must face our own bad habits.

While we raise our voices against misguided and harmful policy (including taking a professional and political stance), we should practice what we preach by creating classrooms that reflect our obligations and commitments.

So I completely agree when Peter Greene makes this pointed and accurate claim connected to Pearson: “You know what kind of test need this sort of extreme security? A crappy one.”

I also support Jersey Jazzman concluding:

But even more than that: a good teacher gives assessments that are largely cheat-proof. So if the PARCC people really think their exam can be gamed by students over social media, they are admitting they have created an inferior product. …

Further: if the assessment is any good, and is really measuring higher-order thinking, it likely can’t be gamed. It’s easy to cheat on a multiple choice exam; it’s much harder to cheat on a chemistry lab. And it’s nearly impossible to cheat on a choir concert, or a personal response to a novel, or number line manipulative. …

But if the PARCC is so vulnerable that a tweet by a student after the test compromises the entire exam, it must be useless — particularly as a measure of student learning.

However, I feel obligated to raise a concurrent point related to the opening of this post: If our own assessment practices need the amount of surveillance and diligence detailed for preventing cheating and catching cheaters, there is ample evidence some pretty suspect testing is happening in our classrooms as well.

Let’s end the tyranny of high-stakes standardized testing that has spawned Pearson et al., but let’s make sure we address that tyranny in our own practice as well.

For Further Reading

Who’s Cheating Whom?, Alfie Kohn

Email to My Students: “the luxury of being thankful”

Grades Fail Student Engagement with Learning

“Students Today…”: On Writing, Plagiarism, and Teaching

De-Testing and De-Grading Schools: Authentic Alternatives to Accountability and Standardization, Bower, Joe / Thomas, P.L. (eds.)

The Fatal Flaw of Teacher Education: “We have met the enemy and he is us.”

The Problem with Pearson (and Hyperbole)

Passive Progressivism

The Problem with Pearson (and Hyperbole)

The education reform debate—one often derailed by both the tone taken and then the debate about that tone (and not the issues)—appears determined to have an Evil Person (or Evil Corporation) to slay.

There was Michelle Rhee (whose Time cover really was just begging for it), and then Bill Gates. But to be honest, the list of candidates for Evil Person has been pretty long.

If you mention a person directly, you are immediately discounted as using ad hominem (although in many if not most of those cases, legitimate issues have been raised about expertise, experience, and even intentions—and not ad hominem at all).

Now, Pearson has entered the picture and prompted a new round (and maybe a new level) of hyperbole.

Data security about children/students, surveillance of Twitter and social media—Pearson has become the manifestation of Big Brother for those skeptical of technology and high-stakes testing.

To that, I’d say that we can have a reasonable debate about whether the comparison is hyperbole or an apt literary analogy, but the larger point I’d like to make is that in ether the debate or the comparison, we are likely missing the important issue.

Pearson has earned 8 billion—4 billion in the U.S.—in annual sales as a consequence of the accountability policies adopted by those elected to office within a democratic process.

I’m sorry to have to note this fact, but Pearson is not a Evil Demigod or some such.

Pearson and its profits are a consequence of very clear and consistent decisions by people in power and the people who put them in power.

Pearson is the logical conclusion of democracy and capitalism—not some totalitarian monster.

When you look at the Pearson phenomenon, and its relationship with education policy and the drain on tax dollars, you must admit this: “We have met the enemy and he is us.”

And what hath we wrought?

If Pearson makes you angry, be sure to consider just who is to blame.

More on Evidence-Based Practice: The Tyranny of Technocrats

Depending on your historical and literary preferences, spend a bit of time with Franz Kafka or Dilbert and you should understand the great failure of the standards movement in both how we teach and how we certify teachers—bureaucracy.

Bureaucracy tends to be inadequate because bureaucrats themselves are often lacking professional or disciplinary credibility or experience, depending, however, on the status of their authority to impose mandates. For education, Arne Duncan serves well as the face of the bureaucrat, an appointee who has only the bully pulpit of his appointment to hold forth on policy.

However, as corrosive to education—and ultimately to evidence-based practice—is the technocrat.

Technocrats, unlike bureaucrats, are themselves credible, although narrowly so. For technocrats, “evidence” is only that which can be measured, and data serve to draw generalizations from randomized samples.

In short, technocrats have no interest in the real world, but in the powerful narcotic of the bell-shaped curve.

As a result, a technocrat’s view often fails human decency (think Charles Murray and Richard Herrnstein) and certainly erases the very human reality of individual outliers.

The face of the technocrat—in fact, the technocrat’s technocrat—is Daniel Willingham, whose work is often invoked as if handed down by the hand on God, chiseled on tablets. [1] [Note: If you sense snark here, I am not suggesting Willingham’s work is flawed or unimportant (I would say important but narrow), but am being snarky about how others wield the technocratic hammer in his name.]

And it is here I want to return to a few points I have made recently:

  • Even the gold standard of experimental research fails the teacher in her day-to-day work because her classroom is not a random sampling of students, because her work is mostly with outliers.
  • And in the teaching moment, what counts as evidence becomes that teacher’s experience couched in that teacher’s content and teaching knowledge as all of that happens against the on-going evidence of the act of teaching.

Stewart Riddle, offering yet another effort in the reading war, is essentially speaking for evidence-based practice while raising a red flag against the tyranny of the technocrat, embodied by the systematic phonics crowd (those who wave the Willingham flag, for example).

On Twitter, in response to my piece on evidence-based policy and practice, Nick Kilstein raised a great point:

My ultimate response (prompting this blog):

My thoughts here, building on the bullet points above, are that having our practice informed by a wide range of evidence (including important evidence from technocrats, but also from other types of evidence, especially qualitative research [2] that can account for outliers, nuance, and the unexpected) is much different than having our practice mandated by evidence (think intensive, systematic phonics for all children regardless of needs or fluency because that is the program the school has adopted).

For day-to-day teaching, the tensions of the disciplines remain important: what we can measure against what measuring cannot address.

When Willingham proclaims that a certain type of research does not support the existence of learning styles, for example, teachers should use that to be very skeptical of the huge amount of oversimplified and misguided “teacher guides” and programs that espouse learning styles templates, practices, and models. [3]

But day-to-day teaching certainly reveals that each of our students is different, demanding from us some recognition of those differences in both what and how we teach them.

It is in the face of a single child that technocrats fail us—as Simon P. Walker notes:

Some educational researchers retreat to empiricist methods. Quantitative studies are commissioned on huge sample sizes. Claims are made, but how valid are those claims to the real-life of the classroom? For example, what if one study examines 5,000 students to see if they turn right rather than left after being shown more red left signs. Yes, we now with confidence know students turn left when shown red signs. But so what?  What can we extrapolate from that?  How much weight can that finding bear when predicting human behaviour in complex real world situations where students make hundreds of decisions to turn left and right moment by moment? The finding is valid but is it useful?

If that child needs direct phonics or grammar instruction, then I must offer them. If that child is beyond direct phonics and grammar instruction or if that direct instruction inhibits her/his learning to read and write, then I must know other strategies (again, this is essentially what whole language supports).

The tyranny of the bureaucrats is easy to refute, but the tyranny of the technocrat is much more complicated since that evidence is important, it does matter—but again, evidence of all sorts must inform the daily work of teaching, not mandate it.

Professional and scholarly teachers are obligated to resist the mandates by being fully informed; neither compliance nor ignorance serves us well as a profession.

[1] For more on worshipping technocrats, explore this, notably the cult of John Hattie and that those who cite his work never acknowledge the serious concerns raised about that work (see the bottom of the post).

[2] Full disclosure, I wrote a biography for my EdD dissertation (published here), and also have written a critical consideration of quantitative data.

[3] See, for example, how evidence (Hart and Risley) functions to limit and distort practice in the context of the “word gap.” The incessant drumbeat of the “Hart and Risley” refrain is the poster child of the tyranny of technocrats.