What the new OCR early childhood data do and do not tell us

Recently released to great interest is the Office for Civil Rights (OCR) Early Childhood Data Snapshot. I want to call additional attention to this document and the survey behind it for two reasons. First, these new data identify serious educational problems that deserve more than one day in the sun. Second, these OCR data have significant limitations that policy makers, the media, and others should understand when using them. Public preschool education is delivered by a complex, interagency, mixed-delivery system that makes it more difficult to measure than K-12. Unless key limitations of the OCR survey are taken into account, users of the data can reach incorrect conclusions. For example, it was widely reported that 40 percent of school districts do not offer preschool. This is untrue: at the very least, every preschooler with a disability is offered a free appropriate education. The OCR survey also undercounts the provision of preschool education nationally, and its accuracy varies by state, which makes cross-state comparisons particularly perilous. Finally, definitions of such key terms as “suspension” are not what most people would assume, which complicates the interpretation of some high-profile findings.

Data from this OCR survey point to problems with access to preschool education and with policies regarding suspensions from preschool programs and retention (grade repetition) in kindergarten.

• Every child should have access to high-quality preschool education. Yet, nearly half of all 3- and 4-year-olds do not attend any preschool program, public or private, and even at age 4, when attendance is more common, just 64% of 4-year-olds not yet in kindergarten attend preschool, according the 2012 Current Population Survey.
• The only “zero tolerance” policy that should apply in preschool is that there should be no preschool suspensions. Yet, a substantial number of preschoolers are suspended each year, with boys and African-American children more likely to be suspended than others. States and LEAs should examine their data, practices, and policies closely to prevent this problem.
• States should look closely at their policies regarding kindergarten grade retention. Does it really make sense to pay for more than 1 in 10, or even 1 in 20, children to attend kindergarten twice? Better access to high-quality preschools, and added services in kindergarten such as tutoring for children who are behind, could be much more cost-effective. States with high kindergarten retention rates should be looking into why they are retaining so many children and what can be done to reduce these rates.

Universal access to high-quality public preschool addresses all of these problems. Better teachers, smaller classes, and more support from coaches and others would reduce suspensions. Such preschools would have more appropriate expectations for behavior, and teachers who can support the development of executive functions that minimize behavior problems. The lower quality of preschools attended by African-American children may partly explain their higher rates of preschool suspension. Finally, good preschool programs have been shown to reduce grade repetition, though bad policies are likely behind many of the high rates of kindergarten retention.

The importance of the problems identified by the OCR data raises another key issue to which most of this article is devoted: to use the data appropriately we must understand the limitations of the data and make sure we interpret them correctly.

Access is Complicated

Let us begin with the finding that “40 percent of school districts do not offer preschool.”  Federal and state laws require that every child with a disability be offered a free, appropriate education from ages three to five. Yet OCR data do not seem to consistently include these children when reporting preschool special education at either the LEA or school level. One reason is that some “school districts” include only older children, e.g., high school districts and vocational school districts. (About 1 percent of high school districts also provide preschool, typically to serve children of teen parents or as a vocational training program.) Limiting the analysis to districts with kindergarten, 70 percent report that they provide preschool, which still seems low. This is partly because some agencies other than LEAs are responsible for preschool special education services. It is also possible that some LEAs mistakenly stated that preschool was not provided. Turning to the number of children reported served, rather than the number of districts serving them, we find a similar problem. School reports undercount the numbers of preschool children receiving services, and the undercount is a bigger problem in some states than others. (A complete copy of the questionnaire can be downloaded here.)

The most obvious explanation for these undercounts is that the OCR survey respondents interpret the questions asking about children served in public school buildings.  At the district level, the OCR survey asks LEAs to first report the number of schools and then to report on their provision of preschool services. This may have led some districts to respond positively only when they served preschool children in public school buildings. At the school level, the OCR survey asks individual schools to report on whether they offer preschool programs and services “at this school” and the enrollment count table specifies “only for schools with these programs/services.”  Whether or not this has any influence on LEA interpretation of the survey, it seems likely that each school reports only preschool offered physically in that school.

Different Data Sources Yield Different Counts

Just how different are the OCR numbers on enrollment from estimates of total enrollment in preschool education offered by states and local education agencies derived from other data sets?  The OCR survey reports 1.4 million enrolled. Data from the Current Population Survey, minus Head Start enrollment, leads to an estimate of about 1.8 million children attending state and local preschool education programs, indicating that the OCR survey is low by about 400,000 children or 22% of the total. In terms of preschool special education services, the OCR data report about 300,000 children, but the Office of Special Education Programs reports 430,000 3- and 4-year-olds receiving special education services under IDEA, and there are additional preschoolers served who are older (while younger children are included in the OCR data). Preschool special education may account for a substantial portion of the undercount, but it seems unlikely to account for the majority of the problem. In sum, the OCR survey undercounts of numbers of children receiving public preschool education from states and LEAs when those served outside public schools are included.

State Approaches Vary

As states differ in how they fund and operate preschool education, the extent to which the OCR data comprehensively capture preschool enrollment varies greatly by state. Looking state by state, it appears that the OCR survey performed fairly well in measuring regular preschool enrollment in most states. However, it grossly undercounted preschool provision in Arkansas, California, Florida, Georgia, New York, Oregon, Pennsylvania, and Vermont. These states make extensive use of private providers for public preschool education. In addition, the OCR figures diverge significantly from the IDEA counts for 10 other states. There are a number of possible reasons for more widespread “undercounting” of preschool special education including: contracting with private providers for special education, responsibility for preschool special education in agencies other than LEAs, and service delivery in homes and other nonpublic school settings. Some preschoolers receive only individualized therapy or other services under IDEA, rather than a publicly provided classroom experience, but neither the OCR nor other data sets allow for the determination of how many children receiving IDEA services are in classrooms funded by public education.

For some states, the data appear to be reasonably accurate when compared to data for the same year from NIEER.[1] Data from the NIEER Yearbook as well as the OCR report are compared below for select states. For states like Georgia and Florida, where many programs are not funded through LEAs, this comparison indicates that the OCR numbers are very incomplete measures of the number of children provided with public preschool education. Relative to total enrollment in state-funded preschool education (which does not include all LEA provision or all preschool special education), Florida is undercounted by about 120,000 and Georgia by more than 30,000. Even in states where funding flows through districts, many children seem likely to have been unreported because they are not served in public schools, which seems to be the case in New York. Also interesting is the case of Wyoming which served 2,207 preschoolers aged 3 and 4 under IDEA, yet the OCR report has Wyoming serving just 13 children under IDEA. While the discrepancies could result primarily from OCR school level respondents counting only children served in public school buildings, this may not be a complete explanation.

New Jersey allows us to conduct a more fine-grained comparison of OCR data with data from LEAs that include children served by private providers. A simple statewide comparison might suggest reasonably full reporting for New Jersey. New Jersey enrolled about 51,000 children in state-funded pre-K which is not very different from the OCR number. However, about half of the 51,000 in state-funded programs attended private providers (including Head Starts) contracted with districts. New Jersey’s districts vary greatly in the extent to which they serve preschoolers through private providers.  When we look at the numbers district by district, we find that the OCR and district totals closely correspond for districts serving children only or overwhelmingly in public school buildings, but  for districts relying heavily on contracted private providers the OCR numbers correspond closely only to the numbers in public school buildings. The OCR report identifies more than 20,000 preschoolers served in New Jersey public schools who are not funded through the state pre-K programs, which just happens to be close to the number served under contract who are not in the OCR data. This strengthens our conclusion that the OCR data represent only children in public school buildings. This is not to fault the OCR survey in the sense that this is what it is designed to do, but this is not how the OCR data have been widely interpreted, nor is it adequate as a survey of preschool education offered through the public schools (and not just in their own facilities).

Suspension and Retention Data

Given the limitations of the OCR data on numbers of children served, the total numbers should not be used as estimates of all children provided preschool education by the states and LEAs. They much more closely approximate the numbers served in public school buildings. Comparisons across states, LEAs, and schools, should be approached with great caution. It is unclear exactly how this might affect the percentage of children reported as suspended, but it seems unlikely to overturn either the general conclusion that suspensions occur at a disturbing rate or that they are higher for African American children and boys. However, comparisons of suspensions across states or districts might be distorted by limitations of the data.

Another aspect of the survey with the potential for misunderstanding is presented by the definition of “suspensions.”  In the OCR survey the definition includes not just children who have been sent home, but also those temporarily served in other programs offering special services for children with behavior problems. Such placements are not necessarily bad for children or to be avoided. However, the data do not allow for any division between children sent home and children sent to more appropriate placements. Nevertheless, the high rate at which children are temporarily removed from their regular classrooms for behavior problems is cause for concern.

The accuracy of the kindergarten retention data also deserves scrutiny. Earlier this year, NIEER collected state data on grade repetition by grade level from state sources of information, though not all for the 2011-12 year. Across all 27 states for which we obtained data, our figures averaged 8/10 of a percentage point lower. Comparing only those for which we had 2011-12 data, our figures averaged ½ of one percent lower. At least judged relative to the only other source we have, the OCR retention data seem reasonably accurate. That the OCR data are slightly higher might reflect efforts to minimize the appearance of a problem.  There are some large discrepancies for a few states. Arkansas had 12 percent kindergarten retention in the OCR data and 6 percent in the state data we obtained; Michigan had 7 percent kindergarten retention in the OCR data and 12 percent in the state data we obtained. For such states, it may be useful to review the data on a district-by-district or school-by-school basis to identify reasons for the discrepancies. Even with kindergarten retention there can be differences due to interpretation. For example, should children who enter a transitional kindergarten program after kindergarten be considered retained?  What about children who enter kindergarten after a year of transitional K?  Any problems with the data would not negate the conclusion that some states have very high rates compared to others and that this deserves consideration by policy makers.

Overall, OCR has provided a valuable service by collecting these early childhood data. Without the OCR data, there would be no basis for raising the issue of preschool suspensions and no way to track progress on this issue in the future. Similarly, without the OCR data there would be no basis for comprehensive state-by-state comparisons on grade retention at kindergarten. Nevertheless, great care must to be taken to recognize the limitations of the OCR data, and the federal government should do more to reduce those limitations. OCR is already working to improve the next survey. Ultimately, they may have to go beyond a school-based survey, because much of public education for preschool children takes place outside of public school buildings even when it is under the auspices of the state education agency (SEA). And, in some states public preschool education is not entirely under the SEA. Possibly, states could supplement LEA data by providing the same basic information for preschoolers they serve outside public school buildings. In addition, procedures might be added to verify that respondents properly understand all questions, especially for states where the responses seem at odds with data from other sources. Some data might be collected in more detail: preschoolers suspended at home with no services separated from those in alternative placements; preschool education children in classrooms separated from those served elsewhere; and, transitional K separated from repetition in regular K.  If you have additional suggestions, particularly based on knowledge of your state’s preschool services systems, OCR would undoubtedly welcome them.

– Steve Barnett, NIEER Director

[1] Though NIEER data report on enrollment in state-funded pre-K enrollments, they do not include LEA preschool services that are not part of state-funded pre-K or IDEA; NIEER data will not capture the full undercount.