NAEP? Nope: Why (Almost) Everyone Will Misread (Again) Data on Gaps

Let the data orgy begin!

NAEP data have been released and I anticipate almost as much time and money will be wasted on the data as has been wasted on administering the tests, scoring the tests, and creating the handy web link to all that data—notably the predictable link to gaps. [For the record, most of these data charts can be prepared without any child ever taking tests; just use the socioeconomic data on each child and extrapolate.]

Take a moment and scroll through the gray space between myriad groups in both math and reading.

…

There, enjoy it?

While you’re at it, look at the historical gaps between males and females in the SAT.

Males on average outscore females in reading and math (though females outscore males in writing, the one section of the SAT that doesn’t count for anything anywhere, hmmmm).

The problem, of course, is that standardized test data are simply metrics for social conditions that we pretend are measures of learning and teaching.

It is a particularly nasty game, but it seems few are going to stop playing any time soon. “Achievement gap”* has now ascended to the point of being classified as a subset of Tourette syndrome among politicians and education reformers.

The problems with persisting to lament achievement gaps and then address those gaps with new standards and more testing are that the solutions both primarily measure those gaps and contribute to them:

• Standardized testing remains biased by class, race, and gender.
• Standardized test scores remain mostly a reflection of any child’s home (from about 60% to as much as 86%).
• School and classes students take are more often than not a reflection of the community and homes children are born into; thus, school/learning quality is determined by a child’s socioeconomic status, but those schools do not change that status.
• If affluent children and impoverished children are provided equal learning opportunities (which they are not), the gap cannot close (go back and look at the handy NAEP charts on gaps, by the way).

The short point is something different has to be done in both the lives and schools of children in poverty (as well as racial and language subgroups overrepresented in poverty) if those data-point gaps are ever going to be reduced.

David Berliner (2013) is illustrative of what those differences should entail, using PISA data often instrumental in ranking educational quality of countries:

Let me look at inequality and schooling internationally: Do countries with greater income inequality generally do worse on achievement tests than countries where income inequality and poverty is lower? The answer is yes (Condron, 2011). Larger income disparities within a nation are associated with lower scores on international tests of achievement. For example, on the 2006 mathematics tests of the Program on International Student Achievement, with a mean score near 500, Finland scored above all other nations (548), and substantially beat the United States of America (474). But Finland is a country with low inequality and a very low childhood poverty rate. But suppose that Finland had the same rate of childhood poverty as the United States of America, and the United States of America had the same rate of childhood poverty as Finland. What might the scores of these two nations be like then? If one statistically adjusted each nation’s scores using the poverty rate of the other, then Finland’s score is predicted to be 487, a long way from the top position it had attained. The score for the United States of America would have been 509, quite a bit better than it actually did. Clearly, inequality within a nation matters. If large numbers of youth in a nation are poor, then achievement test scores are likely to be lower. If there were a reduction in the poverty rate of a nations’ youth, achievement scores are likely to go up….

To those who say that poverty will always exist, it is important to remember that many Northern European countries such as Norway and Finland have virtually wiped out childhood poverty. (pp. 205, 208)

Thus, if we are bound and determined to persist in our fetish for test scores and remain committed to raising test scores (instead of actually alleviating inequity or providing all children with wonderful and rich school days that would end in learning and happiness), guess what?

We need to do something different than what we have been doing for thirty-plus years!

First, end the standards-testing rat race.

Second, end childhood poverty.

Reference

David C. Berliner (2013) Inequality, Poverty, and the Socialization of America’s Youth for the Responsibilities of Citizenship, Theory Into Practice, 52:3, 203-209, DOI: 10.1080/00405841.2013.804314

* Please see my series on “achievement gaps”:

Achievement Gap Misnomer for Equity Gap, pt. 1

Achievement Gap Misnomer for Equity Gap, pt. 2