Schools and districts have come a long way in gathering and analyzing data to help boost student achievement, but according to a new report from the Data Quality Campaign (DQC), what school data initiatives are still missing is the human element.
The DQC’s eighth annual state analysis, Data for Action 2012, found that although states are making progress in supporting “effective data use” and have enacted data-based policy changes, they have “not yet focused on helping people—especially parents, teachers, and students—effectively use data.” The organization issued the findings of its analysis in a report titled “Focus on People to Change Data Culture [1].”
“States should be commended for their hard work building robust data systems,” said Aimee Rogstad Guidera, executive director of the DQC. “But it’s time to focus on the people side of the data equation—how this benefits teachers and students. State policy makers must actively support a culture in which all education stakeholders are actually using and learning from this crucial information to improve student achievement.”
The DQC is the creator of 10 State Actions to Ensure Effective Data Use [2], and according to the campaign, no state has taken all of these steps yet. For example:
- States have laid the foundation to link P-20/workforce (P-20W) data systems but lack governance structures with the authority necessary to share appropriate data among agencies and departments. “This deficiency makes it nearly impossible to provide people the data they need to ensure that students stay on track for success in college and careers,” says the report.
- States are producing reports and dashboards using longitudinal data but are lagging in ensuring data access by stakeholders such as parents.
- States are increasingly providing training to help stakeholders use data but have not done enough to build the capacity of all education stakeholders to effectively use data, especially teachers.
To change the culture of education data use, the report says, states not only must create enabling state conditions, “but also determine their role in creating enabling local conditions.”
What’s also critical in proper data dissemination is promoting data ownership and trust by “building end users’ capacity to use data responsibly, and focusing on using data for continuous improvement, not to shame or blame.”
According to the report, three state-level conditions must be present to enable a data culture change: P-20W leadership, policy, and resources.
“These three conditions are necessary but insufficient to ensure a culture of effective data use,” explains the report. “Local conditions are equally important to realizing this culture change … [and states must ensure] that stakeholders have the time to review data and the authority to act on them.”
Linking P-20W data
Only six states (Alaska, Arkansas, Delaware, Maine, Oregon, and Texas) have met the criteria to link state longitudinal data systems across the P-20W pipeline and across state agencies, the report notes.
One challenge in completing this goal is that more cross-agency data governance bodies are currently authorized to make decisions based on voluntary or charter agreements than on legislation or executive order, which “hinders sustainability and continuity over time,” says the report.
Another challenge is that P-20W governance bodies in 22 states are not chaired by policy makers, making it difficult to garner the political will necessary to work across agencies.
Another challenge is linking K-12 data systems to special education and Head Start/Early Start programs. What’s more, match rates between K-12 and postsecondary data are below 95 percent, and only 14 states link K-12 and workforce data systems.
However, the report highlights some key questions to ensure effective data use:
- What is the quality of the data being shares across state agencies?
- Did the governance bodies start their data planning efforts with the most pressing questions from the state’s stakeholders?
- Have the data governance bodies developed all of the policies and procedures they need to guide data collection, sharing, and use?
Ensuring data access
Only five states (Arkansas, D.C., Delaware, Indiana, and New Hampshire) have met the criteria to ensure that data can be accessed, analyzed, and used by stakeholders.
One challenge to meeting this goal is that aggregate reports do not include individual student information. Also, 47 states produce high school feedback reports, but only 38 states make these reports publicly available; and states do not have benchmarks for report quality, and many reports are produced to meet compliance, rather than stakeholder, needs.
Finally, only nine states are ensuring access to student-level data for parents, as well as teachers and counselors; and many states are unclear about their role in ensuring that local stakeholders have access to data.
The report highlights these key questions to ensure effective data use:
- Do reports meet stakeholders’ needs?
- How can the state ensure that local stakeholders have access to the data they need?
- Are all stakeholders aware of the data and reports that are available?
Building capacity to use data
According to the report, only four states (Florida, Louisiana, North Carolina, and Ohio) have met the criteria to build the capacity of all stakeholders to use longitudinal data.
One challenge is that only 16 states require data literacy for both educator certification and education preparation program approval.
Also, there is no consensus around the definition of data literacy, and states are not clear about how to differentiate data literacy among various types of educators.
And while addressing data literacy “is critical,” says the report, “conditions in schools and districts are generally not conducive to the effective use of data (e.g. principal support for data-informed decision-making, and sufficient time in the school day for collaboration around data).”
Finally, only eight states share teacher performance data with educator preparation programs, and 28 states do not share any data about educators with educators preparation programs.
The report highlights these key questions to ensure effective data use:
- What is the quality of the training that the state is providing around effective data use?
- What evidence shows the impact of educators’ data use in the classroom on student achievement?
- What is the state doing to ensure enabling conditions to promote the use of data, include but not limited to, changing the use of time and how educators work together to process and use data, identifying who has the authority to act on the data analysis, and building trust and a safe environment in which the data can be discussed and shared honestly, without fear of being blamed or shamed?
“The hardest work remains,” concludes the report, “because changing the culture in education is more difficult than building data systems. One thing is certain: we will not change the culture of education data use by focusing solely on systems or even policy. It is only by strengthening our focus on people and what they need that we will reach our goal of improving outcomes for the most important stakeholder: students.”
Many positive examples of state data achievement can be found in the report [1], as well as definitions of the different types of data, policy suggestions, and thought-leader questions for states.