The nonprofit Consortium for School Networking (CoSN) has released a new report as part of its multi-year “Data-driven Decision Making: Vision to Know and Do” initiative, a cross-industry effort to help educators use data to improve instruction.
The report, entitled “From Vision to Action: How School Districts Use Data to Improve Performance,” builds on the developing body of knowledge about data-driven decision making (DDDM). It follows up on the consortium’s 2003 report, “Vision to Know and Do: The Power of Data as a Tool in Educational Decision Making,” which highlighted the need for educators to collect, analyze, report, and share data as required by the No Child Left Behind Act (NCLB).
“From Vision to Action” details best practices from school districts that have used data on student achievement to effect change within their schools and better meet state and national standards for student achievement. It profiles examples of successful leadership in a data-driven school environment and includes examples of school districts that are successfully implementing improvement strategies based on the data their schools have collected and studied.
“I’m continually struck that all sorts of districts are involved in this kind of effort,” said Irene Spero, CoSN vice president and “Vision to Know and Do” project director. “Everybody realizes the importance of it.”
Spero said interest in DDDM has increased considerably since the last report came out. More than 1,000 people have signed up for CoSN’s newsletter on the subject, and 950 school districts have used CoSN’s free self-assessment tool that helps districts determine their readiness to use DDDM for accountability and continuous school improvement, she said.
“Data can play such an important role in improving the … instruction delivered in schools,” said Karen Henke, author of both reports and a consultant in systems technology for school districts.
“By the time we wrote the second report, people knew” about DDDM, Henke said. “We wanted to get them to a point where they could use it.”
“From Vision to Action” finds there are three key elements to successful DDDM in schools, Spero said: (1) the linchpin of effective DDDM is leadership and teamwork; (2) the collection of data must be a cyclical process; and (3) there needs to be technology to support the process.
The first sentence in the body of the report addresses the educational culture in which DDDM can be deployed most successfully. “Leaders establish a climate of acceptance based on a shared responsibility for change rather than on a system of punishment and reward,” it says. The report notes this cultural distinction is critical in getting buy-in at all district levels–from teachers and staff members, to chief information officers (CIOs), to the general public.
Successful leadership also requires a great deal of invisible “stage management” in terms of public relations, according to the report.
“No district or organization gets data-driven decision making right the first time,” the report says. A key challenge is that the evaluation of district data is designed to assess weak points in school curricula and teaching methods–weaknesses it can be difficult for district stakeholders to acknowledge.
“Leadership has an external focus as well–there has to be a leadership within the community,” said Henke. “If the district and principals don’t set the agenda with the [news] media, the data are going to be problematic in the first couple of years–people don’t want to hear what’s going wrong. It’s important to show the public why the data show what they show and what you’re doing to improve [them].”
“Participants experience a j-curve in productivity,” the report says. “Mastering a new discipline generally takes time and effort, with disappointing or limited results at first.”
Henke said administrators must be ready to target easily achieved goals to release publicly along with “the bad news.” She also said that the quick attainment of early goals would help ensure teacher buy-in.
“From Vision to Action” includes an example of the j-curve in action. In the first two years of implementing a DDDM process in Georgia’s Fulton County School District, principals and teachers were highly resistant.
“The district data warehouse generates customized reports for specific teachers and populations to support their planning process,” the report says. At first, the principals and teachers felt “criticized and frustrated” because the district in the past had not returned school reports with suggestions for improvement.
The report says that district teams went to each school to help principals, teachers, and site specialists understand the data and their role in using this information. The district, in turn, addressed teachers’ concerns about assessment tools, and teachers helped to refine the reporting process. As teachers became more comfortable with the reporting and feedback, they began to make more suggestions. Principal Steve Curry said, “Everything we do, the plan reflects or is reflected by the plan.”
Integral to the entire DDDM process is technology, as Spero noted.
Teachers and administrators in the small Lemon Grove School District in Lemon Grove, Calif., had used paper reports to identify achievement gaps and areas of need since 1999. Officials used highlighters to go through paper reports and identify problems. They spent no time actually analyzing data and all their time organizing and reviewing the information.
In 2002, the report says, the district introduced online reports that were easy to understand and query. The new electronic tools gave teachers “easy access with cross-tab options, historical data, and custom data views.” The data then could be easily analyzed.
Before Lemon Grove launched its multiple measures matrix, teachers were asked to test the system. Those involved found it difficult to use. Once administrators realized teachers would never use the system, the district switched to one that was more user-friendly. The teachers now have ubiquitous access to all the data resources.
Access to student achievement data has helped Lemon Grove schools improve dramatically. The district was able to design and install an effective new literacy program as a result of data analysis made possible by the system. According to the report, seven out of the eight schools in Lemon Grove were performing poorly by California state standards before the data analysis technology was put into place. Now, three out of four Title 1 schools in Lemon Grove reportedly have been declared high-achieving by those same state standards.
“The culture of our district is that we can get better in our classrooms,” said Jeri McInerney, director of educational services at Lemon Grove.
CoSN’s Data Driven Decision Making Initiative: Vision to Do and Know