Growth models and peer evaluations

If pay-for-performance programs should be based on multiple measures and not simple test scores, what should those measures be, and how can they be implemented properly?

According to CAP, programs should be designed to discourage teachers from “overly focusing on test-taking strategies, or repetitious drill on a narrow band of curricular material believed to be heavily represented on state exams.”

CAP suggests that more inclusive approaches can be derived from high-quality evaluation systems that incorporate a variety of student outcomes and observations of teacher performance, in addition to value-added estimates. Also, the group suggests that evaluation systems should be correlated with student achievement data–teachers whom observers rate more highly should garner better results in terms of student achievement.

Joe Kitchens, superintendent of the Western Heights Public Schools in Oklahoma City, Okla., believes test scores must be a basic consideration, but there should be other types of measures as well, such as “other types of engagement with students that would matter greatly.”

Based on Kitchens’ observations within his district, he thinks student achievement depends on a number of factors, some of which are outside the control of teachers–such as student mobility. “We have pretty strong evidence … that we must become more effective in dealing with mobile families, and any positive solution regarding merit pay would have to take the issue of mobility into consideration,” he said.

Districts that are experimenting with pay-for-performance systems include the Plano, Texas, Independent School District, which has come up with an innovative “growth model” to help analyze how much of a student’s achievement can be attributed to his or her teacher–and how much can be attributed to other factors. Twenty-eight of Plano’s 68 schools are participating in this state-funded program as part of the Texas Educator Excellence Awards, or TEEG.

“We developed a measure for use, [called] the ‘Plano Effect Score,’ that provides an accurate look at student achievement over the course of a school year using multiple measures and not simply the state assessment,” said Jim Hirsch, associate superintendent for technology at Plano ISD. “Using a growth measure based on individual starting points eliminates the reliance on other factors. Our research shows that a minimum of 70 percent of students’ growth is determined by their starting point; other factors are minimal if the analytics are modeled correctly.”

Plano uses the program to target its schools that are most in need of intervention, and teachers at the selected campuses who teach the core subjects of reading, English, math, and science are eligible to receive a minimum award of $1,400 based on the academic growth of the students instructed by their grade-level or departmental team.

Other examples of pay-for-performance models that include many measures of success include Arkansas’ Achievement Challenge Project, Denver’s Pro Comp Program, North Carolina’s Mission Possible Program, and Vanderbilt University’s Teacher Advancement Program.