More on Evidence-Based Practice: The Tyranny of Technocrats

Depending on your historical and literary preferences, spend a bit of time with Franz Kafka or Dilbert and you should understand the great failure of the standards movement in both how we teach and how we certify teachers—bureaucracy.

Bureaucracy tends to be inadequate because bureaucrats themselves are often lacking professional or disciplinary credibility or experience, depending, however, on the status of their authority to impose mandates. For education, Arne Duncan serves well as the face of the bureaucrat, an appointee who has only the bully pulpit of his appointment to hold forth on policy.

However, as corrosive to education—and ultimately to evidence-based practice—is the technocrat.

Technocrats, unlike bureaucrats, are themselves credible, although narrowly so. For technocrats, “evidence” is only that which can be measured, and data serve to draw generalizations from randomized samples.

In short, technocrats have no interest in the real world, but in the powerful narcotic of the bell-shaped curve.

As a result, a technocrat’s view often fails human decency (think Charles Murray and Richard Herrnstein) and certainly erases the very human reality of individual outliers.

The face of the technocrat—in fact, the technocrat’s technocrat—is Daniel Willingham, whose work is often invoked as if handed down by the hand on God, chiseled on tablets. [1] [Note: If you sense snark here, I am not suggesting Willingham’s work is flawed or unimportant (I would say important but narrow), but am being snarky about how others wield the technocratic hammer in his name.]

And it is here I want to return to a few points I have made recently:

  • Even the gold standard of experimental research fails the teacher in her day-to-day work because her classroom is not a random sampling of students, because her work is mostly with outliers.
  • And in the teaching moment, what counts as evidence becomes that teacher’s experience couched in that teacher’s content and teaching knowledge as all of that happens against the on-going evidence of the act of teaching.

Stewart Riddle, offering yet another effort in the reading war, is essentially speaking for evidence-based practice while raising a red flag against the tyranny of the technocrat, embodied by the systematic phonics crowd (those who wave the Willingham flag, for example).

On Twitter, in response to my piece on evidence-based policy and practice, Nick Kilstein raised a great point:

My ultimate response (prompting this blog):

My thoughts here, building on the bullet points above, are that having our practice informed by a wide range of evidence (including important evidence from technocrats, but also from other types of evidence, especially qualitative research [2] that can account for outliers, nuance, and the unexpected) is much different than having our practice mandated by evidence (think intensive, systematic phonics for all children regardless of needs or fluency because that is the program the school has adopted).

For day-to-day teaching, the tensions of the disciplines remain important: what we can measure against what measuring cannot address.

When Willingham proclaims that a certain type of research does not support the existence of learning styles, for example, teachers should use that to be very skeptical of the huge amount of oversimplified and misguided “teacher guides” and programs that espouse learning styles templates, practices, and models. [3]

But day-to-day teaching certainly reveals that each of our students is different, demanding from us some recognition of those differences in both what and how we teach them.

It is in the face of a single child that technocrats fail us—as Simon P. Walker notes:

Some educational researchers retreat to empiricist methods. Quantitative studies are commissioned on huge sample sizes. Claims are made, but how valid are those claims to the real-life of the classroom? For example, what if one study examines 5,000 students to see if they turn right rather than left after being shown more red left signs. Yes, we now with confidence know students turn left when shown red signs. But so what?  What can we extrapolate from that?  How much weight can that finding bear when predicting human behaviour in complex real world situations where students make hundreds of decisions to turn left and right moment by moment? The finding is valid but is it useful?

If that child needs direct phonics or grammar instruction, then I must offer them. If that child is beyond direct phonics and grammar instruction or if that direct instruction inhibits her/his learning to read and write, then I must know other strategies (again, this is essentially what whole language supports).

The tyranny of the bureaucrats is easy to refute, but the tyranny of the technocrat is much more complicated since that evidence is important, it does matter—but again, evidence of all sorts must inform the daily work of teaching, not mandate it.

Professional and scholarly teachers are obligated to resist the mandates by being fully informed; neither compliance nor ignorance serves us well as a profession.

[1] For more on worshipping technocrats, explore this, notably the cult of John Hattie and that those who cite his work never acknowledge the serious concerns raised about that work (see the bottom of the post).

[2] Full disclosure, I wrote a biography for my EdD dissertation (published here), and also have written a critical consideration of quantitative data.

[3] See, for example, how evidence (Hart and Risley) functions to limit and distort practice in the context of the “word gap.” The incessant drumbeat of the “Hart and Risley” refrain is the poster child of the tyranny of technocrats.

2 comments

  1. Jakob

    You’ll find no argument from me about the broad strokes of this post, but I do wonder just how far we should go in criticizing data-oriented policy. I think I understand what you’re saying when you claim in that first bulleted point that teachers aren’t dealing with random samples, which will be true on some level. But it seems to me that there’s a relative question of randomness in samples. Relative to the national student population no sample can ever be random (because students live where they attend school, for now), but that says more about the unit of analysis than classroom dynamics. Relative to a local community it’s possible to have randomly assigned classes, even if there are many pressures away from randomness. Additionally, I’m very confused when I hear you say that most students are outliers because that would seem to contradict the definition of the term. I think there’s a lot of promise to a perspective that’s data-informed rather than data-driven (not my own phrasing), even if accountability in its current incarnation is hogwash.

  2. Steven

    The one thing that I appreciate the most about Dan Willingham is that he is the first to acknowledge that “research” doesn’t translate cleanly into the classroom. If you look further at his body of work you see that he often gives presentations on the danger of thinking that research (particularly neuroscience) can neatly and quickly find it’s way into instruction. Roughly paraphrasing, he says research shows repetition is excellent for memory, but any teacher can tell you it kills motivation.

    I think the point of attack falls to leaders who “dabble” in the work of “technocrats” and attempt to establish policy based on an incomplete understanding– we see people do this with Marzano, Dufour, you mention Hattie, and we could probably name even more. Another aspiring “technocrat” who I hope gains traction is Dr. Stephen Chew– my favorite comment from him: buzzwords don’t help students learn. Teachers need an informed understanding of how students learn and the time and tools to effectively apply this understanding in the classroom.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s