How do state education agencies gather research to influence school and district policy?
School improvement is on every education agenda across the country, but with guidance from a first-of-its kind study, schools and districts can better understand how state education agencies find and implement research to inform practice.
According to the report, “How State Education Agencies Acquire and Use Research in School Improvement Strategies,”conducted by the Consortium for Policy Research in Education (CPRE), over the last 20 years, state and federal laws and grant programs have given state education agencies (SEAs) more responsibilities for improving low-performing schools.
“At the same time, they have pressed SEAs and school districts to incorporate research-based school improvement policies and practices in their statewide systems of support for low-performing schools, technical assistance for districts, professional development for teachers, and schools improvement programs,” according to the report.
However, the most recent research on how SEAs operate was conducted more than 20 years ago, which begs the question: “Do SEAs effectively incorporate good research to inform policy?”
(Do you think SEAs do a good job of incorporating research into state policy and practice? Take our poll on Page 4. Next page: Types of research)
Types of research
After doing an in-depth study of three SEAs located in different regions of the country and varying in size, organizational structure, and school improvement strategies, the CPRE separated research into four types:
- Research-based knowledge: Research findings that provide empirical or theoretical insights.
- Designed for use in practice: Models, programs, protocol, or other tools that embed research or research-based practices in guides to action.
- Evidence-based knowledge: Data, facts, and other information relevant to the problem of school improvement, such as formative feedback loops on implementation.
- Practitioner knowledge: The information, beliefs, and understanding of context that practitioners acquire through experience, along with research in their decision-making processes.
The study found that SEAs mainly use evidence-based and practitioner knowledge for their research, which the study notes are good options, because research-based knowledge is not sufficient to meet the needs of professionals using it.
“Integrating contextual, local, and practitioner knowledge with research knowledge is critical to developing ‘useable’ knowledge to guide action,” the report emphasizes.
SEA leaders in charge of finding and incorporating research into state education policy used “local practitioner feedback, state professionals’ experience, and external partners’ knowledge of relevant research to contextualize various research findings in light of their states’ school improvement needs,” according to the report.
SEA leaders also worked collaboratively to adapt research to address particular problems and, in some cases, to co-construct new useable knowledge for guiding action.
For example, one SEA used the Center on Innovation and Improvement’s (CII) Handbook on Restructuring and Substantial School Improvement when redesigning its school improvement plans. The SEA took a proactive stance with CII to recreate its checklist of indicators for schools, districts, and teachers to identify areas for improvement, into a format more tailored and relevant to the SEA’s individual needs.
(Next page: Where does the research come from?)
Where does the research come from?
According to the study, the most commonly mentioned source of external research information was the federal government; specifically, offices within the U.S. Department of Education and federally funded centers.
The second-largest external source of research advice was professional membership associations. These included:
- Associations focused on specific subject matter, or teaching and learning more generally, such as state and national reading, mathematics, and technology organizations, and ASCD (formerly the Association for Supervision and Curriculum Development).
- Occupationally-focused organizations, such as the Council of Chief State School Officers (CCSSO) or state-level associations representing superintendents, principals, and federal program administrators.
- Regional organizations, such as the Southern Regional Education Board (SREB).
Staff in all three SEAs, however, reached out primarily to national, not state, professional membership associations for research advice.
Perhaps surprisingly, “fewer SEA staff sought research from institutions of higher education or research organizations,” notes the report.
However, the report emphasizes that variation in the states’ external research networks reflects the SEAs’ differing stages of policy development, internal capacity, and prior partnership histories.
For instance, State A’s school improvement system has been in place for many years and is more likely to pull in a range of external partners on an as-needed basis to co-develop very ‘discrete and specific’ tools and resources.”
In contrast, State C was “bracing for more and more schools to come within its purview for not meeting state or federal accountability standards in the midst of a very spare and declining SEA workforce,” explains the report. “With limited research capacity and expertise in school improvement, SEA staff turned to the CII, as well as their own regional assistance center and a state professional membership association, to help them redesign their supports and create a research-based infrastructure of tools to monitor and assist schools.”
Outside of external organizations, the study found that much of the research came from personal connections and prior work histories.
“While a few of the SEA staff who we interviewed suggested they used a range of internet and academic resources, many turned to their existing network of academics…to access research and new ideas,” says the report. One of the SEAs “sought much of its practitioner advice from its districts and district networks.”
All three SEAs created their own original research by undertaking both formal and informal evaluations of their school improvement policies and programs.
(Next page: Who collects research, and how is it used?)
Who collects and uses this research?
In all three SEAs, a “core group” of SEA leaders were chosen to aggregate and review research, leaders as in directors of school improvement, research departments, and accountability departments.
All three SEAs also said it was critical to have a group of people working collaboratively on choosing and disseminating research.
“There is no one individual that holds all the information, which is why we have a group,” said one office director. “All of those different people hold enough pieces that we can have conversations and share information across the table that can…push us along…to that ideal goal at the end.”
When asked why she trusted the research produced by the core group within her SEA, one “influential” office director from State B said: “Because we digest it together. And people challenge each other…We solve problems. What are we going to do about this? And people bring in research and we’ll table things and [then come back to them with the research, and then we’ll challenge the research.”
Do SEAs do anything with this research?
The report notes that all three SEAs sought research to help districts “be more successful in managing the problems of their low-performing schools. And each used research to develop tools and processes that would aid the growing number of schools coming under the purview of accountability mandates.”
For example, State C used its research to overhaul its district and school improvement planning processes. It also encouraged the CII to create a web-based platform using the Handbook’s indicators to help more “efficiently and cost-effectively monitor progress and interact with local educators,” says the report.
State C also developed “change maps,” thanks to research produced by the CII, a process that the state could use to differentiate their technical assistance to sites.
For more examples on how each SEA incorporated their research into state education policy and tools, read the report.
Even though the report notes that all three SEAs use good practices to select and implement research, there is still a need for researchers to “assess the quality of research acquired by SEA staff and underlying research designed for use. While [much] of the [research] identified in our study were written by or cited national experts, sometimes research was added in a fairly superficial manner.”
The report also highlighted the “major” need to strengthen the knowledge base, or create more usable research, which include supporting more varied types of research on policy implementation and effects in understudied areas of education policy.
[poll id=”38″]
- #4: 25 education trends for 2018 - December 26, 2018
- Video of the Week: Dealing with digital distraction in the classroom - February 23, 2018
- Secrets from the library lines: 5 ways schools can boost digital engagement - January 2, 2018
