Category Archives for Teacher Evaluation

Teacher v. Professor: On Why Anyone Would Be an Educator

While cycling with a new acquaintance, I navigated through the usual questions about how long I have been a professor, and then, after I mentioned that I was a high school English teacher for 18 years before moving to higher education, the follow up about which was easier, or which I preferred.

On this ride and during the conversation, I realized I am quickly approaching the tipping point in my career since I am starting my 17th year as a professor, just one year away from having been a professor for as long as I was a high school teacher—the identity I remain strongly associated with about myself professionally.

Being a writer has held both aspects of my being an educator together, but K-12 teaching and being a professor in higher education (especially my role as a teacher educator) are far more distinct than alike.

However, and most disturbing, K-12 teachers and college professors are increasingly sharing disillusionment with being educators as K-12 teachers are fleeing public education and fewer are majoring/certifying to teach while quit lit has become a phenomenon throughout higher education.

My journey as an educator offers some unique insight since I have nearly two decades in each contexts, K-12 and higher education. As well, my work in higher education remains directly connected to K-12 classroom teaching (I am a teacher educator and spend time observing in public schools as well as having professional and personal relationships with public school teachers).

For a majority of my time as a high school teacher, I was department chair, and several of those years included being a coach. My days were often very long, starting at 7:30 AM and ending on match days as late as 11 PM.

As a high school English teacher, I taught around 100 students or more at a time and had five classes a day, usually 2-3 preps. Since I focused on teaching my students to write, I responded to about 4000 essays and 6000 journals a year.

In 1995, I entered a doctoral program, an EdD in curriculum and instruction; I continued to work full-time and even maintained my adjunct work at local colleges. The doctorate experience was sobering since every other candidate I encountered was seeking a doctorate to leave the K-12 classroom—except me.

I completed my degree and was still resolute I would teach high school until I retired, and then maybe seek something at the university level. My pay bump for the advanced degree was, in fact, quite good.

The summer of 2002 was not something I planned, but when a position opened at a nearby university (a position held by my former high school teacher and mentor), I applied with no real expectations about making the transition so soon.

After a flurry of on-campus interviewing, and then a disheartening negotiation about salary (the opening offer was a $17,500 pay cut), I agreed to leave my high school job for higher education. The final pay was still $6000 below my public school salary, but the university promised I would have overloads and summer work to make up the difference.

So I sit here this summer about to start my 17th year as a college professor, a full professor with tenure. I have learned a great deal.

First, the prestige and respect shift from K-12 teaching to higher education was stunning, especially since I taught high school for 4 years while I had a doctorate; the degree was not the key factor in how people viewed and treated me.

Professors receive immediate respect and assumptions about our expertise that K-12 teachers never experience. My ability to publish, for example, in local, state, and national newspaper magically appeared once I could list my university instead if my high school when querying.

Next, and related, that respect divide cannot be disassociated from the impact of gender: More than 3 of 4 K-12 teachers are women, but the largest group of professors is men, and that imbalance is even greater at the higher ranks, where men are the majority. The university where I teach, for example, is well over 60% male faculty.

Possibly the greatest differences, however, between K-12 teaching and being a professor are expectations for labor and what counts as your professional obligations.

By the time I left K-12 teaching, I was wearing a wrist brace; my right hand was nearly immobile from marking essays, and to be honest, teaching English as I knew I should [1] was nearly unmanageable against the rest of my responsibilities and having a family or any sort of recreational life.

Burn out is a common term associated with K-12 teaching, but since teaching doesn’t appear to be manual labor (such as construction) or isn’t associated with production (most of us balk at seeing our students as widgets), those who have not taught fail to recognize the physical and psychological wear that comes with teaching.

I joke, though it isn’t funny, that being stared at by 100 or so students per day is stunningly exhausting. But most K-12 teachers have no real time to eat alone (or with only other adults), to go to the restroom, or to do with their work day anything other than grade, respond to student work, plan, or address the never-ending minutia of bureaucracy that is teaching (standards, meetings, paperwork, etc.).

Teaching—even just lasting past the first 3-5 years—two or three decades is a herculean task in surviving a career; too many teachers out of self-preservation learn to work in auto-mode, mailing in a profession because it has simply erased your humanity.

Along with professional respect, I gained a great deal of professional autonomy (which K-12 teachers have almost none) and, most of all, time. A heavy semester for me is teaching several courses three days a week, usually M, W, F and from about early morning to mid-afternoon.

Except for meetings (and higher education has an ugly committee and departmental meetings problem), I have multiple days a week to devote to my professional commitments other than teaching, for me, being a writer.

And as a professor, I have never fretted about going to the bathroom, and making sure I eat, calmly, is nearly never a struggle.

I also teach with almost no direct evaluative surveillance or oversight (which can be a bad thing, of course); this I note because it reduces the unnecessary stress of teaching in a high-stakes accountability environment that allows you no professional autonomy (what it means to be a K-12 teacher).

I must stress that a great deal of pettiness and an inordinate amount of unhealthy practices still plague higher education—the tenure and promotion process along with the faculty evaluation process are steeped in sexism and inequity, for example.

And the cancer that is high-stakes accountability and reducing education to work-preparation is creeping, no galloping, toward and eventually over higher education.

When I first took my university position, I was surprised at how out of touch professors were with K-12 teaching and the negative impact of the standards and high-stakes testing movement. I, in fact, warned my colleagues that the accountability movement would some day come to colleges and universities so it was in their own self-interest to begin fighting the movement in K-12 schools.

But they didn’t listen.

Higher education isn’t called the Ivory Tower for nothing.

So this brings me to why anyone would be an educator—especially in 2018 when the consequences weighed against the rewards for being a K-12 teacher or a college professor are tipping mightily in the wrong direction.

To teach, at any level, for many of us is something like a calling. Just as one day in my first year of college I recognized I am a writer (I did not choose that), I know myself to be a teacher.

Despite my introversion, and my discomfort with people, crowds, I am never more relaxed than in a classroom with students. We are there with common purpose and we mostly are seeking ways to be a community.

These are things I believe in, things I trust about the possibility of humans being better than we have been so far.

To be an educator, then, is not the problem in that the profession itself, whether K-12 or in higher education, is compelling and deeply fulfilling.

The problem is that to be a teacher in the U.S. is colored by the cultural negative attitude toward labor, being a worker, and the power of collective workers against the wishes of corporatism.

Teaching at all levels has continually been corrupted by the urge to reduce public institutions to private entities driven by corporate paradigms.

K-12 teachers have always worked in environments that isolate us, overwork us so that we cannot resist, and have gradually become less and less unionized. Much of higher education (because of the tenure and promotion process as well as departmental politics) has also allowed competition to trump collaboration.

It is not so much why anyone would be an educator, but why those of us who teach at any level have allowed our profession to be dismantled, devalued, and dehumanizing.

And finally, teachers and professors are regularly policed for being political, admonished for being activists. And to that we must ask, in whose interest is this political call for teachers and professors to not be political?

Isolated, silenced, and depoliticized, we educators are failing a profession that deserves better.

In solidarity, raising our voices, and actively exercising our politics, we educators can resurrect one of the most valuable acts of labor humans can embrace.

The latter is why anyone would be an educator.

[1] The martyr/missionary dilemma.

 

Advertisements

The Unbearable Lightness of Being a Practitioner in Education

In the final days of my Summer I graduate course, my department chair asked me to switch from the Summer II graduate course I was assigned (a new preparation I had worked on diligently to teach for the first time) to a literacy course in which the instructor could no longer teach the class.

This course was one I have never taught, and thus, would have to prepare in just a few days to take on—scrambling as I did to understand the other instructor’s syllabus and schedule while also facing the herculean task of teaching from four assigned books that I have never read.

This afternoon, then, when I face these graduate students, I will confess that I have never taught elementary literacy (the course is a graduate literacy methods class)—having all my experience and expertise in teaching high school and college literacy, primarily writing—and thus, I will be relying on their practitioner expertise (the students are mostly practicing elementary teachers) while performing the role myself of facilitator.

In my Summer I class, as well, one assignment required students to read a professional book on literacy, and share with the class. Part of our discussion revolved around professional books in education emphasizing classroom practice (over theory and philosophy) while being written by education personalities.

The credibility of these books are often grounded in the assumption these personalities are credible; issues of validity and reliability—and even thorough citation—are ignored or de-emphasized. So I cautioned them that professional books (and education personalities) are not to be viewed as scripture, not as sacred directives, but as opportunities to think along with these education personalities in order to develop and sharpen their own practitioner expertise.

My journey as an educator has included 18 years as a classroom high school English teacher followed by 16 years as a teacher educator, and concurrently, an education scholar and public intellectual.

I lived, then, nearly two decades of sitting in mandated workshops and presentations where education consultants spoke down to us practitioners while earning in a few hours what no practicing teacher earned. These consultants and speakers may have had some classroom experience, but it was vividly clear to us they had all eagerly jumped ship to talk to lowly practitioners because the hours and the pay were much better.

K-12 teachers tend to loath this traditional aspect of being a teacher—the torture of being treated unprofessionally and the waste of our precious time that we could all better use to do the stuff of teaching, planning and responding to student work.

The edu-guru market is an ugly beast that perpetuates the notion that K-12 teachers are not professional or experts themselves, that practice is somehow just a mechanical thing that can be imposed onto a passive and compliant workforce (let us hasten to add, a passive and compliant workforce in which 3 of 4 teachers are women with undergraduate and graduate degrees and years of experience).

So when I teach or provide in-service for teachers, I emphasize my own classroom experience above all else, and couch my scholarly expertise in that practice now edging toward 40 years.

The accountability era has ratcheted up this divide, in part perpetuated by authoritarian structures (prescriptive legislation and top-down managerial styles of administrators) and in part by the market.

This latter influence must not be ignored. Publishers depend heavily on the cult of personality to drive textbook and professional book sales as well as the related consultant appearances.

Too often, however, what is being mandated and sold proves to be mostly hokum beneath the shimmer and shine of well-formatted books and over-confident edu-gurus.

Paul Murphy’s Teachers Are Tired of Robert Marzano highlights nearly everything that is wrong with this cult of personality that de-professionalizes teachers while also blaming them for the outcomes driven by the practices they are mandated and coerced to implement.

Murphy stresses: “For years, teachers were asked (or, more often, told) to swallow a lot of crap. More and more of us are done eating it” (emphasis in original).

In a powerful and thorough interrogation of this dynamic, Benjamin Doxtdator challenges Doug Lemov and Dave Burgess:

Both Lemov and Burgess construct masculine, individualistic heroes. Champion teachers, according to Lemov, “routinely do what a thousand hand-wringing social programs have found impossible: close the achievement gap between rich and poor, transform students at risk of failure into achievers and believers, and rewrite the equation of opportunity.” For Burgess, Pirates are “entrepreneurs”, “daring, adventurous, and willing to set forth into uncharted territories with no guarantee of success. They reject the status quo and refuse to conform to any society that stifles creativity and independence.”

I have spent a great deal of my work as a scholar and public intellectual raising the same concerns about Angela Duckworth’s grit and Carol Dweck’s growth mindset.

Scholars of poverty and social class began lining up more than a decade ago to refute the popular but invalid training provided by Ruby Payne, who continues to profit greatly off the uncritical edu-guru poverty circuit funded mostly by tax dollars.

There are patterns to all this madness:

  • Practitioners are framed as or assumed to be unprofessional and inexpert.
  • Experts are, then, the consultants themselves, who are beyond reproach (criticize the work of Duckworth, or John Hattie, and expect to be accused of attacking the people themselves, to be shamed for the criticism).
  • Both educational research and teacher practices are trivialized as secondary to the gimmick (grit, teaching like a champion, visible learning, etc.) and the edu-guru who peddles the gimmick.
  • Teaching and learning are necessarily narrowed and over-simplified. Marzano and Hattie direct a laser focus on the impact of teachers; Duckworth and Dweck keep the accusatory eye on weaknesses and flaws in the children/students themselves.
  • Teacher and student voices are muted or entirely ignored.
  • Teachers are conditioned to behave in unprofessional ways that are used to justify treating them unprofessionally.
  • Divisions of labor and compensation for labor are disturbingly skewed so that practitioners are underpaid and under-appreciated while consultants and administrators (farthest from the day-to-day experiences of students) are overpaid and overvalued.

When I met with a colleague who designed the course I will be teaching for the first time this afternoon, she empathized with the abrupt change in course assignments and then helped me tremendously by noting that when she taught the course, she used elements of the National Writing Project (NWP) model for summer institutes.

I was co- and lead instructor of a writing project in South Carolina at the end of my high school teaching career just before entering higher education in 2002. Being a participant in and then facilitating for a NWP site were by far the greatest experiences for me as an educator and a professional.

Why? The sacred elements of these summer seminars were the professionalism of the teachers and the community of scholars that was fostered and developed.

One of the most important refrains of these communities was the call to check ourselves regularly against the allure of edu-gurus and gimmicks (we at first embraced the term “best practice” and then quickly felt it had become a mandate and not a healthy generalization for how any teacher works from a toolbox of practices with the needs of the learner guiding those fluid decisions).

K-12 practitioners remain trapped in a hellish contradiction created by the cult of personality driving edu-gurus and gimmicks: Teachers are simultaneously posed as the singular and most important factor in student learning (a verifiable lie) and then treated as incompetent technicians.

Teachers need to be relieved of edu-gurus and gimmicks; they deserve professional experiences that include the time, support, and conditions that are conducive to what is best for each student taking a seat in any of their classrooms.

Teachers must not be reduced to technocrats, must not be compelled to be martyrs and missionaries.

If we can resist the allure of celebrity and cashing in, we must ultimately acknowledge the humanity of teachers and their students, while admitting the ugly influences of sexism and consumerism that too often trump our stated goals of democracy and equity.

Does Your Academic Institution Value Diversity, Equity? (Probably Not)

Several years ago, my university was forced to acknowledge it has a gender problem. As a selective liberal arts university, the institution had already begun addressing its race and diversity problems among students admitted and faculty hired.

Two gender concerns could not be ignored: Women were paid less than men at the same ranks, and faculty attrition was overwhelmingly among women professors, who constitute only about 30% of the faculty.

A gender equity study was commissioned, but when the report was issued, a group of male faculty circulated an open letter challenging the methodology of the report, raising concerns about a lack of empirical data and expressing the need for quantitative versus qualitative methods.

This response certainly had an image problem—white male faculty calling into question a gender equity study—and the concerned faculty did eventually withdraw the letter in deference to the good of the university community.

However, this study and the response illustrate a serious problem in academia, the pervasive power of traditional structures (expectations about what data matter, what types of research matter, and a lingering argument that objectivity can be achieved) to serve as a veneer for entrenched, and thus rendered invisible, sexism, racism, and classism.

A parallel example is when my university seeks to increase the diversity of the faculty, that effort is always contested with “Let’s just hire the best candidate,” again often voiced by white male faculty [1].

“Best,” of course, like quantitative methods and empirical data is a veneer for the embedded biases that have been normalized (and thus seemingly invisible to the power structure itself and those who benefit from the bias).

White and male privilege, then, are institutionalized in higher education (see here and here for ways those privileges exist, again, invisibly to white men). Despite the popular claim that higher education is some liberal indoctrination factory, higher education is incredibly traditional and conservative at its core; only the edges appear liberal.

But, I can feel many wanting to interject, how can calling for high-quality research to address gender equity on campus and expecting candidates for open faculty positions to be the best constitute flawed practices in academia?

Let me often another example, one that calls into question the grounding of those arguments themselves, the claims of fidelity to high standards.

Another traditional practice in higher education is the use of Student Evaluations of Teaching (SET), feedback gathered from students and then used in various ways to evaluate faculty for tenure and promotion.

Notably, a significant body of research [2] has revealed that SET lack validity and negatively impact women, faculty of color, and international faculty (in the U.S.).

Concurrently, the use of SET positively impact the existing and skewed white male faculty at most universities, who disproportionately dominate higher ranks and salaries.

Guess what happens when concerns are raised about SET based on high-quality empirical data and quantitative studies? The same faculty crying foul over gender equity reports and hiring practices toss up their hands and say, “O, well, we have to have something.”

As Colleen Flaherty explains:

While some institutions have acknowledged the biases inherent in SETs, many cling to them as a primary teaching evaluation tool because they’re easy — almost irresistibly so. That is, it takes a few minutes to look at professors’ student ratings on, say, a 1-5 scale, and label them strong or weak teachers. It takes hours to visit their classrooms and read over their syllabi to get a more nuanced, and ultimately more accurate, picture.

For example, my university’s self-evaluation form and the connected chair evaluation directly instructs in the teaching evaluation section: “Give particular emphasis to evidence of teaching quality, which could include numerical results from student opinion survey forms, written comments from student opinion survey forms, and comments from faculty or other consultants visiting your classes.”

“Evidence” is bolded and then the first example is “numerical results from student opinion survey forms.” There are clear biases here that privilege an instrument invalidated by a body of high-quality research—exactly what some faculty deemed missing in our gender equity study.

Junior faculty explain, often in private, that they are aware numerical data from the SET are the most important element of their case for tenure and promotion. As well, our Faculty Status Committee has provided workshops directly detailing which data from those forms are most influential, providing, as the committee claims, ways to distinguish faculty from each other.

Virtually every college and university has a diversity and equity statement and a perpetual formation and reformation of diversity and equity committees.

No statement or committee can make existing institutional sexism, racism, and classism disappear—especially if those words and that work are forced to work within existing biased structures.

“Research is reviewed in a rigorous manner, by expert peers,” writes Flaherty. “Yet teaching is often reviewed only or mostly by pedagogical non-experts: students. There’s also mounting evidence of bias in student evaluations of teaching, or SETs — against female and minority instructors in particular. And teacher ratings aren’t necessarily correlated with learning outcomes.”

As long as calls for “high-quality” and “best” to guide policies and practices remain selective—and clearly in the service of the existing inequities and lack of diversity—we must admit the real commitment is not to”high-quality” or “best,” but to the status quo.

While not the only litmus test, a powerful way to determine if your academic institution values diversity and equity is if it continues to implement SET. Almost all do, so the answer remains, probably not.

See Also

Is Your University Racist? Bedelia Nicola Richards

[1] See how “merit” can work in the service of privilege in this reconsideration on Jordan Peterson:

I met Jordan Peterson when he came to the University of Toronto to be interviewed for an assistant professorship in the department of psychology. His CV was impeccable, with terrific references and a pedigree that included a PhD from McGill and a five-year stint at Harvard as an assistant professor.

We did not share research interests but it was clear that his work was solid. My colleagues on the search committee were skeptical — they felt he was too eccentric — but somehow I prevailed. (Several committee members now remind me that they agreed to hire him because they were “tired of hearing me shout over them.”) I pushed for him because he was a divergent thinker, self-educated in the humanities, intellectually flamboyant, bold, energetic and confident, bordering on arrogant. I thought he would bring a new excitement, along with new ideas, to our department.

[2] See:

28 November 2017 Education Reader

I am the teacher South Carolina wants to retain, and I am barely hanging on, Rachel Caulder

Teachers need autonomy. Thankfully, I teach in a school that does not use curriculum alignment documents or strict pacing guides, and my administration values the judgment of teachers within our classrooms. Teachers in districts that are solely focused on numbers are restricted, and students suffer because no allowance is made for differentiation or reteaching for content-mastery. In districts with strict pacing guides, teachers are left with no option but to stay the course — even when they know they are failing their students.

Why do schools use grades that teach nothing? Jonathan Lash

At the college where I serve as president, we do evaluate student work; we just use a higher-quality method. Our students receive written evaluations not only on every assignment, but also for every course and learning activity. These evaluations are designed to be formative teaching tools.

For similar reasons, we completely stopped accepting SAT and ACT for admissions two years ago, after an internal study revealed standardized test scores are poor predictors of student success at Hampshire. We also recognized the bias of standardized tests against low-income students, and the negative influence of standardized testing on education.

A North Carolina Teacher’s Guest Post on His/Her EVAAS Scores

NEPC Review: Tackling Gaps in Access to Strong Teachers: What State Leaders Can Do (The Education Trust, October 2017)

The Every Students Succeeds Act (ESSA) directs states and districts to identify equity gaps in students’ access to excellent educators and transformative school leaders. States are encouraged to use Title II funds strategically in order to identify and remedy these gaps. A new report from The Education Trust draws on ESSA documents and state teacher equity plans to provide guidance to state leaders, including some sound advice—but with significant omissions. The report does not engage with thorny issues around alternative pathways into teaching, and it largely skirts issues around incentives for supporting teacher recruitment and retention in hard-to-staff schools. The report also does not consider what attracts teachers into the profession and into particular school environments. Likewise, the report fails to draw on the explicit remedies sought by ESSA to link high-quality leadership with strong teacher recruitment and retention. Instead, the report casts the teacher equity problem primarily in terms of labor supply shortages and treats teachers like interchangeable widgets. Relying heavily on advocacy sources, it misses an opportunity to unpack the root causes of the teacher retention problem, particularly the corrosive impact of past federal and state policies on the teaching profession. The report does not help state leaders understand how they might build incentives and cultures that draw strong teachers into high-need schools, and they will thus be left with an incomplete and insufficient set of tools for ensuring that all students have equitable access to excellent educators.

Go public and perish? Supporting the engaged scholar, Jennifer Ditchburn

Despite the fact that university presidents and the people who run university communication departments are only too happy to have their scholars out building a profile, the academic system is not set up to help them connect with the public. Writing a piece for Maclean’s or appearing on CBC’s The National doesn’t count toward tenure or get you a promotion: publish or perish is about peer-reviewed journals and books.

Time for public engagement is not often budgeted into a professor’s employment – scholars do this on top of their personal and academic responsibilities (I always feel a bit sheepish when I approach a busy prof to write something for me). The challenges are arguably tougher for some women in academia, whose pursuit of tenure or awards is already interrupted by maternity leave or childcare responsibilities.

The Missing Link In Student Writing

Don’t Buy Bluster from Teacher Quality VAM-pires

The responses are predictable online and through social media any time I address teacher quality and policy focusing on teacher evaluation such as my recent commentary on Charleston adopting value-added methods (VAM).

How dare I, some respond, suggest that teacher quality does not matter!

The pattern is exhausting because most responding in indignation first misrepresent what I have claimed and then make the most extreme arguments themselves in order to derail the conversation along their own agenda, usually linked to the charter school movement grounded in teacher bashing and making unobtainable promises.

So let me state here that the central elements of what we know about teacher quality and efforts such as VAM-based teacher evaluation is that teacher quality is not an independent variable (any teacher may be effective for one student and ineffective for another, for example) and, since student high-stakes testing is not designed to measure teacher quality and is more strongly linked to out-of-school factors, VAM is both a horrible technique for identifying teacher quality and, ironically, a guaranteed process for devaluing the importance of teachers.

Teacher quality is unparalleled in importance in terms of student learning, but it is also nearly impossible to measure, quantify—especially through student scores on high-stakes standardized tests.

Teacher quality VAM-pires, then, often have agendas [1] that are masked by their bluster about teacher quality.

Trying to measure and quantify teacher quality is a mistake; linking any evaluation of teacher quality to student test scores lacks validity and reliability—and VAM discourages teachers from teaching the most challenging populations of students (high-poverty, special needs, English language learners).

Focusing on simplistic and inappropriate measures reduces teacher impact to 10-15% of what high-stakes standardized testing measures; in other words, VAM itself devalues teacher quality.

My informed argument, based on 18 years as a public school classroom teacher and 15 years as a teacher educator and scholar, then, is that we must recognize teacher quality is impacted by teacher preparation, teaching/learning conditions, student characteristics, and dozens of other factors inside and outside of schools—many of which are beyond the control of teachers or students.

As well, we must address the teacher quality issues that political and administrative leaders can control: class size, school funding, and most important of all, teacher assignment.

Just as decades of research have revealed that teacher quality accounts for no more than 10-15% of student test scores, decades of research show that affluent and white students are assigned the most experienced and certified teachers while poor and black/brown students are assigned new/inexperienced and un-/under-certified teachers.

The charter school crowd’s bluster about teacher quality is pure hokum because charter schools increase that inequity of teacher assignment by depending on new and uncertified teachers such as candidates from Teach For America.

No one is saying teacher quality does not matter—I clearly am not saying that—but dishonesty about teacher quality does lay at the feet of the edu-reformers and the VAM-pires who wave their collective arms any time we call them on their failed policies and their political agendas.

[1] See the evangelical urge of Broad-trained acolytes, the resume building and cut-and-run patterns of edu-reformers, and the post-truth practices of turn-around and charter advocacy.

Post and Courier (Charleston, SC): CCSD plan for teachers won’t work

Post and Courier (Charleston, SC): CCSD plan for teachers won’t work

[see full submission with hyperlinks below]

Charleston School District Arriving Late to a Very Bad Party

P.L. Thomas, Professor, Furman University

The most telling aspect of Charleston County School District’s announcement about holding teachers accountable for student test scores may be in the second paragraph of Paul Bowers’s coverage: “District leaders say they don’t want to fire anyone — particularly not in the midst of a statewide teacher shortage that’s only getting worse.”

While it appears some are aware of the unintended consequences of new education policy, we must be concerned that such awareness has not helped better inform this recent decision to move forward with using value-added methods (VAM) in teacher evaluations.

Early research warned and current studies confirm that VAM fails to fulfill political promises, but also feeds existing problems. This pattern has been seen with school choice increasing segregation, exit exams causing higher drop-out rates, and high-stakes tests driving teaching to the test, asking far less of students.

First, we should acknowledge the flawed logic driving the use of VAM to increase teacher accountability. The recent concern about teacher quality is grounded in several false assumptions.

One is the “bad” teacher myth strongly associated with the stereotypical unionized teacher as portrayed in Waiting for Superman.

However, across the U.S. teachers in unionized states produce students with higher test scores that teachers in non-union (called right-to-work) states such as South Carolina. The problem with associating teacher quality and low student outcomes with union protection is that student test scores are more powerfully associated with poverty than any other factor.

This leads to the other false assumption that teacher quality is the most or one of the most important causes of student learning. In fact, teacher quality accounts for only about 10-15% of standardized test scores, the basis of VAM, while out-of-school factors remain the greatest influence, at 60%.

A final assumption is that using test data to evaluate teachers is objective and valid, and thus fair. Yet, many teachers are in content areas without standardized tests, and then, ironically, making any set of data high-stakes causes that data to be less credible (see Campbell’s Law).

Student test scores are poor evidence about teacher quality, and making those scores central to evaluating teachers is guaranteed to further erode that evidence’s usefulness and the process.

Matthew Di Carlo, blogging for the Albert Shanker Institute, details the unintended consequences of VAM as implemented in Houston.

While assessing the Houston teacher evaluation system, Julie Berry Cullen, Cory Koedel, and Eric Parsons discovered that teachers identified as weaker by their student test scores were already leaving at high rates before the new system was implemented in order to identify weak teachers.

In other words, Houston adopted a policy without investigating whether or not teacher quality was the key problem, without acknowledging that with high teacher attrition among so-called weaker teachers, schools were not achieving in ways they envisioned.

Here is one powerful lesson of education reform: Do not adopt a policy until you gather evidence of the problems; and assumptions about those problems are not enough.

After Houston adopted VAM-based evaluation of teacher, as Di Carlo explains, teachers identified by test scores as weak were even more likely to leave, but:

On the other hand, all exits increased under the new evaluations — including among teachers who were rated as average and high performers. The extent to which this spike is attributable to the new evaluation system per se is unclear, but it served to “dilute” the impact on student achievement of the increase in exits among low performers. There is also some indication that higher-rated teachers were more likely to switch out of schools with low-performing students after ETI (versus before the policy), which would also attenuate the impact of the policy.

Historically and currently, Charleston has a teacher problem, yet Houston is a powerful example of how VAM is not the solution to those problems.

The remaining challenge is recruiting and maintaining experienced and certified teachers in the schools that serve the most vulnerable students from high-poverty homes and communities.

The new policy linking teacher evaluations with student test scores will not address those challenges, and we can expect it will actually increase challenges to recruit new teachers and stem teacher attrition.

By adopting VAM and investing in critically discredited system, EVAAS, Charleston County School District is late to a very bad party. The students and community would be better served by making sure we know what the teacher quality problems are and then seeking ways to address those instead of choosing political expediency and wasting tax dollars on policies and programs already shown to fail.

VAM, Teacher Bashing, and Unintended Outcomes: “[A]ll [teacher] exits increased under the new evaluations”

Research analysis at Shanker Blog is among the very best available online, notably the work of Matthew Di Carlo.

The posts there are predictably nuanced and careful, dispassionate—to a fault. As a critical educator, I am on edge when I read these careful explications of educational research because they tend to stand so far back from drawing critical conclusions that they leave a great deal of room for forgiving awful and baseless policy.

Teacher Evaluations And Turnover In Houston is an extremely important post as it unpacks new research on the current era of teacher evaluations spawned during the Obama years, notably the increased use of value-added methods (VAM) that link teacher quality to student test scores.

I highly recommend reading the post in full, but here I want to add a few annotations to address both my concerns the analysis tip-toes when it should stomp and to emphasize a few key takeaways well addressed by Di Carlo.

Let me share a few passages, and I will boldface what I want to address; first, the opening:

We are now entering a time period in which we might start to see a lot of studies released about the impact of new teacher evaluations. This incredibly rapid policy shift, perhaps the centerpiece of the Obama Administration’s education efforts, was sold based on illustrations of the importance of teacher quality.

The basic argument was that teacher effectiveness is perhaps the most important factor under schools’ control, and the best way to improve that effectiveness was to identify and remove ineffective teachers via new teacher evaluations. Without question, there was a logic to this approach, but dismissing or compelling the exits of low performing teachers does not occur in a vacuum. Even if a given policy causes more low performers to exit, the effects of this shift can be attenuated by turnover among higher performers, not to mention other important factors, such as the quality of applicants (Adnot et al. 2016).

To address incredibly flawed educational policy, I believe we must be much more careful about distinguishing between political/public claims and then how the research community poses the same issues.

As Adam Bessie has outlined, the “bad” teacher myth was never “sold” in the ways Di Carlo notes above. The film Waiting for Superman is a powerful example of how political and public discourse about “bad” teachers was primarily an argument that teacher quality was the singular or most important factor in student learning, period; politicians and the public almost never added the caveat “most important in-school factor.”

And we must acknowledge that the “bad” teacher movement driving new teacher evaluations including VAM was significantly grounded in anti-union sentiment and union-busting objectives—not about teacher quality or student learning.

The nuanced argument about teacher quality, in fact, was most often expressed among some researchers, while mostly absent from the media or political discourse, such as Di Carlo (from 2010):

But in the big picture, roughly 60 percent of achievement outcomes is explained by student and family background characteristics (most are unobserved, but likely pertain to income/poverty). Observable and unobservable schooling factors explain roughly 20 percent, most of this (10-15 percent) being teacher effects. The rest of the variation (about 20 percent) is unexplained (error). In other words, though precise estimates vary, the preponderance of evidence shows that achievement differences between students are overwhelmingly attributable to factors outside of schools and classrooms (see Hanushek et al. 1998; Rockoff 2003; Goldhaber et al. 1999; Rowan et al. 2002; Nye et al. 2004).

Next, further into the recent post:

Prior to ETI, there was a negative relationship between teacher effectiveness and exits – i.e., less effective teachers were more likely to exit than their more effective colleagues, with effectiveness here defined in terms of validated measures of teachers’ ability to raise students’ test scores (in part because the original value-added scores, unlike the other components of the system, are available both before and after the new evaluations were implemented).

A strong footnote for this important point—so-called weaker teachers were already leaving—is that the real teacher quality problems facing schools, notably high-poverty schools serving vulnerable populations of students, are a lack of equity in terms of teacher assignment (poor students, black/brown students, ELL students, and special needs students disproportionately are assigned year after year to new or inexperienced and un-/under-certified teachers; white and affluent students are gifted the most experienced and certified teachers) and the debilitating grind of high teacher attrition, turnover, in high-poverty and majority-minority schools.

Just as school choices increases educational problems such as segregation, VAM-based teacher evaluation does not address the real problems—equitable access to experienced and qualified teacher, and teacher turnover in high-poverty schools—while also increasing those problems.

And finally:

The big finding of Cullen et al. is that the relationship was stronger after the onset of the new evaluation system, with the estimated effects concentrated among low-performing teachers in schools serving low-performing students, who were more likely to exit the district than they were before ETI.

On the one hand, this suggests that the new evaluations worked as intended. Under a system in which principals were armed with better information about their teachers’ performance (full evaluation results instead of single year value-added scores), teachers who were less effective in raising test scores were more likely to exit the district (or be dismissed) post-ETI than they were prior to ETI, particularly in schools serving lower performing students. On the other hand, all exits increased under the new evaluations — including among teachers who were rated as average and high performers. The extent to which this spike is attributable to the new evaluation system per se is unclear, but it served to “dilute” the impact on student achievement of the increase in exits among low performers. There is also some indication that higher-rated teachers were more likely to switch out of schools with low-performing students after ETI (versus before the policy), which would also attenuate the impact of the policy.

The Big Caveat, of course, is that this evaluation process and concurrent analysis remain trapped in the efficient (read: lazy) use of test scores by students to determine teacher quality and effectiveness. That said, this study seems to show that VAM-type evaluations may actually push out so-called weak teachers—while also pushing out so-called effective and experienced teachers.

This preliminary evidence supports what many of us have been warning about during the Obama era of education reform: The “bad” teacher approach to education reform causes more harm than good because it misrepresents teacher quality and further de-professionalizes teaching, such as eroding the current teacher work force and discouraging the so-called “best and brightest” from choosing education as their career.

Di Carlo continues to offer incredibly important education research analysis, and I highly recommend anyone interested in education reform to return to this blog regularly. There you will find careful and crisp analysis—although I will continue to hope for the sort of analysis that will critically confront what lies beneath the political and public discourse about schools, education, teachers, and students.

The story inside the story of the research analyzed above is that beneath the “bad” teacher approach to education reform is a great deal of bad politics and bad media; and we must stop tip-toeing around those facts.