Data Literacy is More Than Just Test Results

Data have been used in education for many years. Good teachers and administrators have been using data to inform their practice and make decisions. Why is data use important? Because it is no longer acceptable for educators to solely use anecdotes and gut feelings to make decisions. Educators need hard evidence. To that end, there has been a growing emphasis for the past 15 years to make education a more evidence-based and data-driven discipline.

Data have ranged from accountability and compliance data to data for continuous improvement at all levels of education, but one issue that has loomed large is the conflation of data literacy with assessment literacy. The two constructs have been confounded for many years (Mandinach & Gummer, 2016a; Mandinach & Kahl, 2014). When educators and the general public think about data, they typically think about test results and student performance. They fail to think about all the other sources of data that help educators to inform their practice. Until fairly recently, there has been no clear definition of data literacy and certainly no analyses of the skills, knowledge, and dispositions that are needed to use data effectively and responsibly (Data Quality Campaign, 2014; Gummer & Mandinach, 2015; Mandinach & Gummer 2016a, 2016b). The work of Mandinach and Gummer, based on several years of research, has attempted to lay out what it means for educators to be data literate.

A foundation of data literacy is the consideration and use of diverse sources of data, not just the limitation to only student performance data. For educators to have a comprehensive understanding of their students, they must look to behavioral, attitudinal, motivational, medical, attendance, home context, and other kinds of data. Even though measures of teacher readiness such as the edTPA (SCALE, 2013) contain an assessment rubric, it makes clear that teacher candidates must be able use contextual information, “assets,” to inform their understanding of student performance.

Diverse sources of data are particularly important in early childhood education where teachers often must look beyond student performance results to understand a child. As Dwyer (2015) notes, some of the data most relevant in early childhood settings other than assessments (formative, summative, and diagnostic) include screening results, informal check-ins, child characteristics and experiences, attendance, health information, family language/education experiences, family conditions for learning, classroom observations, participation, walkthroughs, and staff experience and education. In workshops conducted across Pennsylvania for early childhood educators, Dwyer, Mandinach, Nunnaley, and Saylor (2015) noted several purposes for data use in early learning:

• Improve child outcomes
• Improve teachers’ skills
• Identify gaps in achievement
• Realign resources
• Facilitate parental engagement
• Improve program quality
• Increase access to high quality programs
• Change adult behavior

Dwyer and colleagues (2015) reflected on why data use is important in early learning settings, recognizing that evidence is important. They noted that there needs to be realistic expectations for how data use can inform and improve daily practice. Through effective data use, educators can:

• Reflect on practice
• Check assumptions
• Get others’ views
• Commit to new actions
• Attend to the effects of changes in practice
• Make practice public

But how does this happen? More than eight years ago, the Institute for Education Sciences (IES) commissioned a comprehensive review of the literature that existed at the time, recognizing that data-driven decision making was only then emerging as a hot topic in educational research (Hamilton, Halverson, Jackson, Mandinach, Supovitz, & Wayman, 2009). Some 3,000 research and implementation studies were identified with only a handful meeting the strict criteria for rigorous research laid out by the What Works Clearinghouse. Five recommendations were noted. For there to be effective data use at any level of education, schools and districts must:

• Make data part of an ongoing cycle of instructional improvement
• Teach students to examine their own data and set learning goals
• Establish a clear vision for schoolwide data use
• Provide supports that foster a data-driven culture within the school
• Develop and maintain a districtwide data system (p. iii)

These five recommendations have stood the test of time. The growing research in data use further confirms the recommendations. Much of the work firmly espouses the need for the introduction of data teaming, leadership, and the creation of data cultures within schools and districts. Data systems have morphed from data warehouses to dashboards and apps that provide real-time data for instructional decision making.

Yet despite having much of the infrastructure in place, particularly the billions of dollars spent on technology at the federal, state, and local levels, attention to the human infrastructure remains problematic. Fulfilling all of the five recommendations from the IES practice guide is an important step forward. However, if educators do not know how to use data both effectively and responsibly, the investment in attaining the recommendations will go for naught. Even though the field recognizes the importance of data use, the delivery of consistent and comprehensive professional development is often lacking and falls below other competing priorities.

As Means, Padilla, and Gallagher (2010) noted, professional development for data use must be ongoing, not sporadic. As Mandinach and Gummer (2016a) note, waiting until educators are in practice to acquire data literacy skills is too late. They must begin to acquire such skills at the earliest stages of their professional careers, that is, during pre-service preparation. Because of this growing need, WestEd and its collaborator, Using Data Solutions, is working toward the development of curriculum materials that can be used in teacher preparation programs to teach data literacy. The objective is for teacher preparation programs across the country to begin to integrate the construct, data literacy for teachers (Mandinach & Gummer, 2016a, 2016b) into their curricula. The ultimate objective is to create a teaching corps that knows how to use data.

Ellen Mandinach is a Senior Research Scientist and the Director of the Data for Decisions Initiative at WestEd. Dr. Mandinach is a leading expert in the area of data-driven decision making at the classroom, district, and state levels; her current focus is on data literacy for teachers. Dr. Mandinach has authored a number of publications for academic journals, technical reports, and five books. She received a Ph.D. in educational psychology from Stanford University.