Federal Pre-K Curriculum Study Shows Few Stand-Outs

Experts Ponder Why Most Curricula Showed Little or No Gains Over Local Practice

Six years after the U.S. Department of Education undertook to evaluate preschool curricula there are as many questions as answers about what makes for an effective curriculum--or at least more effective than what prevails in many pre-K programs today.

In the Department's recently released report of the Preschool Curriculum Evaluation Research (PCER), only two of the 14 off-the-shelf curricula evaluated were found to have statistically significant positive impacts on student outcomes for the prekindergarten year. Only three had positive impacts on students in the kindergarten year.

That comes as something of a surprise to those expecting the curricula represented by this study to show more impacts on children's progress. It is less of a surprise to those familiar with past large-scale curriculum studies, like Planned Variation Head Start, and to those who work with teachers regularly and know how long it takes for teachers to fully learn any truly sophisticated new approach to teaching.

When it comes to measures of classroom quality and instruction, there was a stronger showing with eight of the curricula having at least one positive effect. This aspect, however, wasn't the primary focus of the evaluation since it's the impacts on children's learning and development that matter most.

The PCER initiative engaged 12 research teams around the country to implement the curricula in the evaluation. Classrooms or programs were randomly assigned to use either one of the 14 treatment curricula or the "control condition," consisting of the local "curriculum-as-usual," which ranged from programs developed by teachers to well-established products like the High/Scope curriculum.

About 2,900 children, 300 preschool classrooms, and 200 preschools were involved in the initiative. In general, the children were from disadvantaged backgrounds, one-third being white, a little less than half black, and about one-fifth Hispanic. Fewer than half of the kids' mothers were married and fewer than half the moms had attended college. Most of the pre-K programs in the study were publicly funded. Two-thirds of the teachers had at least a four-year college degree.

Children were observed and data gathered at the beginning of the pre-K year, at the end of the pre-K year, and in the spring of the kindergarten year. Data were also gathered to measure the impact of the curricula on preschool classroom quality, teacher-child interactions, and instructional practice.

With so few curricula showing positive effects on children's skills, questions arise as to whether something in the study design or execution might account for the tepid showing or if in fact most curricula simply aren't likely to show gains in studies of this type. Ohio State University professor Laura Justice, who implemented a curriculum in the study while at the University of Virginia, says while it's useful to look into the way the study was conducted, it's also important to consider the findings at face value. "They are telling us these curricula don't produce demonstrable effects over prevailing practice...that most don't produce value-added benefits over widely used curricula like High/Scope," she says.

The curriculum showing the most positive effects over the control condition was DLM Early Childhood Express, published by The Wright Group, a McGraw-Hill Company. It was supplemented in the study by Open Court Reading, a phonological awareness program published by SRA that's designed for pre-K through sixth grade. The combination positively affected reading, phonological awareness, and language in both the pre-K and kindergarten years. Jeff Livingston, who markets both products at McGraw-Hill, credits the research that went into developing the curriculum for its showing. He says selection and sequencing of learning activities are designed to follow learning trajectories and build on what children already know.

Of course, selection and sequencing of material isn't unique to any one developer. Bette Chambers of Johns Hopkins University offers a case in point: Curiosity Corner, a curriculum in the study from the Success for All Foundation, has more than three dozen thematic units sequenced into an integrated approach to literacy. Curiosity Corner showed no positive effects over the control condition in the pre-K year but did show a positive effect on reading in the kindergarten year.

Chambers credits the gain to the curriculum's emphasis on oral language development. She says children in Curiosity Corner classrooms had more opportunities for book reading than the control group did, making it more likely they would understand words they were exposed to at reading time in kindergarten.

One curriculum in the study focused on math skills--Pre-K Mathematics with DLM Early Childhood Express Math Software. It showed gains in children's math skills in the pre-K year. University of Buffalo professor Douglas Clements, who developed the math software with colleague Julie Sarama, says the math gains likely came about because pre-K curricula, including those in the control group classrooms, have emphasized literacy. "What we're finding is that sequencing more math into the pre-K years may be beneficial," he says.

Asked why more curricula didn't show gains over the control group, Clements points to implementation as a possible reason. Initial training for the treatment group teachers varied from one day to six days, depending on the research team and curriculum involved. DLM Early Childhood Express with Open Court Reading Pre-K provided the most initial training and showed the most positive effects in children's skills. Teacher support also varied substantially. As Clements puts it, "Curriculum matters but it matters a heck of a lot more when teachers are trained to implement it." Chambers agrees, saying that curricula with open-ended approaches rely on the teacher creating instruction from scratch and might not have as standard an implementation, something that could lead to variability in outcomes.

Justice says the measures chosen to assess children's learning also affect study results. "I would expect that in the case of language-focused curricula there would be gains in certain areas of language that weren't measured in the study," she says.

NIEER Co-Director Ellen Frede, who has extensive experience in curriculum implementation, says the two-year duration of the study may favor scripted curricula such as the DLM approach over open curricula requiring more self-determination on the part of the teacher. "Anyone basing their curriculum decisions on this one study should exercise caution," she says. "The fidelity of implementation of the curricula was only medium for the treatment group and lower for the control group. What that tells us is high-quality implementation takes longer than two years for many of these curricula.

"Had the study run another year or two, we might see some different results," Frede says. In addition, she cautions that the control condition (or counterfactual) was different in each study so that results cannot really be compared across the studies within PCER.

Diane Trister Dodge, president of Teaching Strategies, Inc., developer of Creative Curriculum, agrees. "It's unfortunate the study ended before researchers had the opportunity to test the impact of a well-implemented Creative Curriculum classroom on child outcomes," she says. She cites the positive effects her curriculum had on teacher behaviors and overall classroom quality as proof the curriculum was reaching a more desirable level of implementation when the study ended.

The consensus among researchers Preschool Matters talked to was that it would be good to have more questions answered for the $36 million invested in the study. Clements says it's important to note one positive thing coming out of it, however: "Most of the schools reached a fairly acceptable base level of early education where all the kids, whether in control classrooms or treatment classrooms, were learning. On the whole, that's pretty good news."

Curriculum Decision-Making: What to Consider

Preschool curriculum models vary a great deal. Some are highly scripted, spelling out in detail what to teach and how to teach it. Others are more open to variation, providing guidance but not necessarily going through lessons step-by-step. Some emphasize one area, such as literacy, while others encompass all domains of child development. Choosing which curriculum is best for a program depends in part on the program's philosophy and approach as well as the skill levels of the teachers.

Guidance for curriculum decision-makers is available in a policy brief written by NIEER's Co-Director Ellen Frede and Associate Director of Research Debra J. Ackerman. They explain what goes into making a curriculum, discuss various curriculum approaches, and provide a checklist of questions any curriculum decision-maker should be asking when choosing a product. Preschool Curriculum Decision-Making: Dimensions to Consider is available at http://nieer.org/docs/index.php?DocID=142.