Fact Checker–Quis custodiet ipsos custodes?

Apparently oblivious to irony, Chester “Checker” Finn attacked the National Center for Education Statistics (NCES), NIEER, and the State of Preschool Yearbook last week in a blog titled “Now you’re entitled to your own facts, too.” His blog relies heavily on innuendo and insinuation, topped off by outright error, to try to kill the messenger in advance of the message. It seems Checker has lost his way when it comes to evaluating research and really does believe that he is entitled to his own facts. In this post, I review the facts to set the record straight.

The key facts are these: NIEER is a research unit of Rutgers–the State University of New Jersey; the NCES contract with Rutgers/NIEER for data collection on state pre-K is perfectly legitimate and the contracting process was completely above board; the data we collected for NCES are entirely trustworthy (see the report); and, NCES did not pay for NIEER to put out an independent report of our own. Checker cites not one fact that supports his claims to the contrary. He does, however, engage in considerable name-calling–his blog is replete with terms like “Chicken Little,” and “hotbeds of passionate advocacy” (slightly racy that one: students interested in a career in survey research should know it is not all that exciting). Let’s look at his claims and the facts in more detail.

When attacking NIEER as an advocacy organization, Checker recommends that readers read NIEER’s vision statement to see if NIEER “looks like a neutral source of factual data.” Please do. You will find that NIEER seeks to provide “independent, research-based advice and technical assistance to policymakers, journalists, researchers, and educators.” We “monitor and evaluate national and state progress toward early educational excellence.” NIEER also “conducts and communicates research to support high-quality, effective early childhood education for all young children.” Keeping in mind that parents are a child’s first and most important teachers, the only specific features of early education endorsed by our vision statement are that it be “high-quality,” “effective,” and informed by research. Beyond the shared vision that our research should support good early learning experiences for children from birth through age 8, NIEER does not as an organization have “a” view. As NIEER is a unit of Rutgers, every researcher at NIEER has the academic freedom to express her or his own views, and responsibility for the views in each publication ultimately resides with the authors.

Why did NCES contract with NIEER to collect survey data on state pre-K?  The straightforward answer is because these data are valuable and would not have been collected otherwise. Data collection efforts that seek to make their results available to everyone at no cost encounter a classic free-rider problem that typically is solved through government support. So when NCES approached us about funding the data collection, we were delighted. NCES not only provides a disinterested funding source, their independent review process also strengthens the survey, and NCES funding enabled us for the first time to prepare a public use data set that will be accessible to anyone through the NCES website soon.

Given the capacity we had developed over a decade to collect the data, NCES concluded that it would be more efficient to fund NIEER to collect the data than to conduct the survey themselves, and that our readiness to do so would prevent an interruption in the data series. The contracting process was public, and followed standard procedures, including an extensive public comment period. I encourage everyone to read the official notice as well as all of the public comments and NCES responses, rather than just accept what Checker says.

The 2012-2013 state pre-K survey, data, and report for NCES meet all standards and guidelines set forth in the most currently available publication of the NCES Statistical Standards, and amendments implemented since that time. These products were subject to an extensive and rigorous NCES review process. In addition, it is noteworthy that the source of the survey data is state government, and the data collection process requires that states verify the accuracy of their responses as we have reported them before they are published.

Checker can’t point to any errors in the NCES report.  He faults the NCES report because it “doesn’t say anything about how many [statewide pre-school programs]are skimpy offerings … with little or no curriculum and scant evidence of learning outcomes” and “doesn’t say anything about whether whatever short-term gains they manage to produce are sustained….”   And, rightly so, as such NCES reports are supposed to focus on the data collected without commentary, and the survey data do not support Checker’s claims. In sum, he is complaining that the NCES report is “reprehensible” because it sticks to the facts and does not promote his advocacy agenda by making claims that cannot be substantiated with the data in the report.

Finally, I turn to NIEER’s own The State of Preschool Yearbook 2013, which Checker falsely claims was funded by NCES. It was not. If he had read it, he would know that, but he chose to criticize the report before it was published. As it is ours, we can go beyond the tightly limited NCES report. We still do not make unsubstantiated claims, but we can cite other data to contextualize the results, list states in rank order for enrollment and funding, compare state policies to benchmarks and each other, examine change over an entire decade, raise questions, and offer policy recommendations.

If Checker had read our Yearbook, he also would know that our report does not (as he suggests) give states an easy pass, equate their programs with the Perry Preschool or other exemplary models, or fail to express a concern for observable classroom quality and effectiveness. Here are 3 examples from the Executive Summary:

• More than half a million children, or 41 percent of nationwide enrollment, were served in programs that met fewer than half of the quality standards benchmarks.
• It is possible that most children served by state pre-K attend programs where funding per child is inadequate to fund a quality early education.
• States should collect data on the quality of teaching practices in their pre-K classrooms from a sufficient sample to assess how frequently good quality is provided, and to assess the strengths and weaknesses of current services.

As John Adams observed, “facts are stubborn things.” No one is entitled to mislead others or make up their own facts, but as we have found to our dismay some will try. That is precisely why NIEER collects and disseminates data to bring greater transparency and accountability to public policy, and to provide a common factual basis for policy debates regardless of one’s position on the issues. – Steve Barnett, NIEER Director