GUEST POST: The Danger of Delegating Education to Journalists, K. Anders Ericsson

The Danger of Delegating Education to Journalists: Why the APS Observer Needs Peer Review When Summarizing New Scientific Developments

K. Anders Ericsson, Department of Psychology, Florida State University

Tallahassee, Florida, October 28^th, 2012

I have always viewed the APS Observer as a source for up-to-date summaries of advances in the wide research domains of the Association for Psychological Science (APS). Receiving summarized information about research outside my domain of expertise is only helpful if it accurately represents the consensus views of the active APS members and Fellows. The cover article by Eric Jaffe on “Piecing together performance” in the September 2012 issue of the APS Observer focused on my own area of expertise, namely expert performance and deliberate practice, so I feel that I am able to assess the accuracy of its presentation [1].

Although I accept that the process of writing an engaging popular article requires considerable simplification, I think it is essential that the article does not contain incorrect statements and misinformation. My primary goal with this review is to describe several claims in Jaffe’s article that were simply false or clearly misleading and then discuss how APS might successfully develop successful methods for providing research summaries for non-specialists that are informative and accurately presents the major views of APS members and Fellows.  At the very least they should not contain factually incorrect statements and avoid reinforcing existing misconceptions in the popular media.

Jaffe’s  (2012, p. 13) article discusses recent research on one of the longstanding issues in Psychology, namely  “the roles of natural endowment and hard work in human performance”(Jaffe, 2012, p. 13). It goes on to state that “Ericsson and his colleagues found in a 1993 study that professional musicians had accumulated about 10,000 hours of deliberate practice over the course of a decade. The results became the basis of Ericsson’s deliberate practice theory of elite performance, also called the 10,000 hour rule” (Jaffe, 2012, p. 13). With these two sentences Jaffe reinforces misconceptions in some popularized books and internet blogs that incorrectly infer a close connection between deliberate practice and the “10,000 hour rule”.  In fact, the 10,000 hour rule was invented by Malcolm Gladwell (2008, p. 40) who stated that “researchers have settled on what they believe is the magic number for true expertise: ten thousand hours.” Gladwell cited our research on expert musicians as a stimulus for his provocative generalization to a magical number. Our research found that the best violinists reported having spent a remarkably large number of hours engaged in solitary practice when, in fact, 10,000 hours was the average of the best group; indeed most of the best musicians had accumulated substantially fewer  hours of practice at age 20. Our paper found that the attained level of expert music performance of students at an international level music academy showed a positive correlation with the number of solitary practice hours accumulated in their careers and the gradual improvement due to goal-directed deliberate practice. In contrast, Gladwell (2008) does not even mention the concept of deliberate practice.

The focus of Jaffe’s (2012) paper is to report on empirical evidence to reject a straw man primarily advocated by bloggers on the internet. The Jaffe strawman argues that individuals would only display clearly superior performance after 10,000 hours or ten years of practice. This strawman is even rejected by our research on deliberate practice in music and other domains. The music performance of our violinists developed gradually by engaging in solitary practice under the guidance of teachers and coaches. In our study, the best violinists had practiced significantly more than the two less skilled groups of violinists (Ericsson, Krampe, & Tesch-Römer, 1993). After only a few thousands of hours of solitary practice, the music performance of our musicians was clearly superior to other groups of amateur musicians at the time they were accepted to the music academy.

Jaffe’s most surprising and potentially most damaging criticism of deliberate practice was made in the section with the interview with Joanne Ruthsatz, where she is described as having “reanalyzed Ericsson’s data from the 1993 study and found that the top violinists had begun to distance themselves well before they’d put in 10,000 hours“ (Jaffe, 2012, p. 13) [2]. Most remarkably, Jaffe’s APS article goes on to quote Ruthsatz as saying that, “when they [the top violinists] were eight years old, for instance, they won two-thirds of their competitions. Compare that to the “good” students, who only won about half at that age, and the lowest group, who won about a fifth of the time.” (p. 13). This statement is puzzling in light of our 1993 paper, which says that “The biographic histories of the four groups of subjects with respect to violin playing are remarkably similar and show no systematic differences between groups. The age when they began practice was 7.9 years old and essentially coincided with the age of starting systematic lessons, which was 8.0-years-old” (Ericsson et al., 1993, p. 374). Taken together, these two statements would imply thatthe violinists in all groups won competitions even before they started playing the violin or took systematic lessons—if confirmed, it would be the most compelling evidence for innate talent reported to date in a journal of scientific organization! However, it should be noted that in their only peer-reviewed published report Ruthsatz, Detterman,  Griscom, and Cirullo (2008) make a different claim than that reported by Jaffe (2012).  Namely, that “the group of elite violinists won more open competitions from the time they were 8 years” (p. 331). In contrast, this published claim is consistent with our 1993 paper, which reported that the top violinists had won significantly more open competitions (competitions without any age constraints) by the time that the top violinists were 23 years old on the average (their age at the time of our study). In our original 1993 paper we interpreted this finding as confirmation that the top violinists were exhibiting a higher level of performance at the time of the study as a result of their gradually gained improvements from more extensive training (e.g., a larger number of hours of solitary practice).

Jaffe (2012, p. 13) focuses his article around a statement made by APS Fellow Hambrick, who is reported to say that “it seems clear to me that there’s something else [than deliberate practice]”, as if these other factors were not already considered in the 1993 Psychological Review paper. The 1993 paper reviewed evidence for a wide range of immutable genetic factors, such as height and body size, and heritable individual differences in motivation: “We believe that a more careful analysis of the lives of future elite performers will tell us how motivation is promoted and sustained. It is also entirely plausible that such a detailed analysis will reveal environmental conditions as well as heritable individual differences that predispose individuals to engage in deliberate practice during extended periods and facilitate motivating them.” (Ericsson et al., 1993, p. 400).  More generally, our 1993 paper offered accounts of cognitive abilities, innate abilities, and personality differences in terms of their development from sustained practice activities. Ericsson, Roring, Nandagopal (2007a, 2007b) has a more comprehensive proposal for the account of giftedness. For example, mathematically gifted students would have had years of activities related to mathematics even before they attained a high score on the mathematical part of the SAT test at age 13; scientists would have decades of focused scientific study before they produced Nobel-prize winning research, and so on. Only future research will uncover the role of deliberate practice activities and associated motivational factors in accounting for their reproducibly superior achievement.

The most objectionable omission of a deliberate practice account in the evidence reviewed by Jaffe (2012) is presented under the heading “personality influences performance.” In this section, Jaffe (2012) describes the work of Angela Duckworth on grit and how “elite performers also tend to have a lot of endurance when it comes to achieving their goals” (p. 15). To argue that deliberate practice needs to be augmented, he explicitly cites an article, which includes deliberate practice in its title, “Deliberate practice spells success.” In that study we (as I was also one of the co-authors) collected data on “deliberate practice.” We found that “Grittier spellers engaged in deliberate practice more so than their less gritty counterparts, and hours of deliberate practice fully mediated the prospective association between grit and spelling performance”, (Duckworth,  Kirby, Tsukayama, Berstein, & Ericsson, 2011, p. 178).  In direct contrast to Jaffe’s (2012) suggestion that personality characteristics (grit) have a relation to performance completely independent of deliberate practice, the cited article shows that this relation is fully explained by the differential engagement in deliberate practice.

The more generalizable issues concern  how factually incorrect statements and misrepresentations were not captured prior to publication of this article in the APS Observer. Unlike regular newspapers, such as the New York Times, Jaffe could not have engaged in fact checking or even interviewed researchers with different theoretical orientations. From contacts with a number of researchers active in the study of expert performance Jaffe (2012) did not interview them—including researchers explicitly named in the article, such as Angela Duckworth and myself . Even Dean Simonton, who was interviewed felt that his arguments were not accurately described.

In response to the problems with Jaffe’s (2012) article, I hope that the APS Observer will not continue to hire professional writers to publish articles independently. I think that the APS Observer should have procedures to ensure that its articles are examples of engaging, yet accurate, summaries of research development.  If the professional writers are invited in the future to write on controversial topics, I recommend that the APS Observer rely on an APS member as an invited action editor that evaluates accuracy of the article by sending it out for peer review. It should be possible to develop procedures and standards for popularized articles that incorporate the checks and balances of the traditional journal procedure. In an interview the popular author MalcolmGladwell said the following: “There is an important distinction to be made between popular works and academic works. My book is intended to be a popular work based on academic research, but it is not the same as an academic work. It is a different kind of literary enterprise. And it has to be judged by different criteria.” (Gladwell cited in Gruber, 2006, p. 398). The procedures for assuring that popularized articles meet reasonable standards of scientific rigor will be different from the ones from those used with scientific journal articles, and should focus on avoiding factual mistakes and biased simplified accounts that misinform its readers. Hopefully the problems with the Jaffe (2012) article might lead to a discussion about how summary articles on research developments pertaining to controversial topics can be produced so that they exemplify how popularized articles on psychological topics can be made appropriate for the APS Observer, one of the flagship journals representing the Association for Psychological Science.

References

Duckworth, A. L., Kirby, T., Tsukayama, E., Berstein, K., & Ericsson, K. A. (2011). Deliberate practice spells success: Why grittier competitors triumph at the National Spelling Bee. Social Psychological and Personality Science, 2, 174-181.

Ericsson, K. A. (in press). Training history, deliberate practice and elite sports performance: An analysis in response to Tucker and Collins Review – What makes champions? British Journal of Sport Medicine.

Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993).  The role of deliberate practice in the acquisition of expert performance.  Psychological Review, 100(3), 363-406.

Ericsson, K. A. Roring, R. W., & Nandagopal, K. (2007a). Giftedness and evidence for reproducibly superior performance: An account based on the expert-performance framework. High Ability Studies, 18, 3-56.

Ericsson, K. A. Roring, R. W., & Nandagopal, K. (2007b). Misunderstandings, agreements, and disagreements: Toward a cumulative science of reproducibly superior aspects of giftedness. High Ability Studies, 18, 97-115.

Gladwell, M. (2008). Outliers: The story of success. New York: Little, Brown and Company.

Gruber, D. A. (2006). The craft of translation: An interview with Malcolm Gladwell. Journal  of Management Inquiry, 15, 397-403

Jaffe, E. (2012). Piecing together performance. APS Observer, 25(7), 13-16.

Ruthsatz, J., Detterman, D., Griscom, W.S., & Cirullo, B.A. (2008). Becoming an expert in the musical domain: It takes more than just practice. Intelligence, 36, 330–338.

Footnote

1. In the last month I (Ericsson, in press) responded to a similar misrepresentation of my collaborators’ and my work in British Journal of Sports Medicine.
2. I have never given Dr. Ruthsatz access to our original data.