How the Fade-out Myth Gets Perpetuated

Last week, the Associated Press (AP) reported on an evaluation of Tennessee’s state-funded pre-K program commissioned by that state’s Comptroller’s office (See, for example, Memphis Daily News, “Report: Tenn. Pre-K Not Effective After Second Grade”). As the headline indicates, the report is being widely cited as finding that pre-K has no lasting impact. This would be dismaying if true, because Tennessee has relatively high standards for its pre-K program, as indicated in NIEER’s 2008 State Preschool Yearbook. However, those claiming that Tennessee pre-K has no lasting effect might want to actually read the full report carefully, because it does not substantiate that claim.

Anyone reading beyond the executive summary will find that the authors themselves conclude, “Although the effects of Pre-K on long-term academic achievement are not evident in the present study, the lack of a statistically significant difference in measures of student achievement in the long term can not logically be attributed to an ineffective Pre-K intervention.” In other words, this study can’t really answer the question of whether Tennessee’s pre-K has lasting effects.

Readers can be excused for being confused by misleading statements in the executive summary that combine the conclusions of a rather one-sided literature review with an unjustified interpretation of this study’s findings. To wit: “Consistent with the results of the present evaluation, many studies find improved language or math skills in Kindergarten following Pre-K, but these effects have often dissipated by the First or Second Grade.” The literature review in this instance fails to acknowledge recent meta-analyses that find lasting effects and a number of rigorous evaluations that have reported lasting effects from higher quality early childhood programs in the last several years. They help perpetuate a fade-out myth that is no longer supportable by the research. (Stay tuned for a second installment on this topic.)

My own take is that just like the last report on this evaluation, this one provides clear evidence that the analyses have not been able to overcome serious design problems including selection bias. As my grandmother who taught school in Tennessee might have said, the stitching is mighty fancy, but it’s still a sow’s ear. The real question here is why the Comptroller spends taxpayer money on this flawed study year after year.

Steve Barnett
Co-Director, NIEER