Teachers and unions are upset over an online ratings database.

Teachers and unions are upset over an online ratings database.

In a move that has many local educators seething, the Los Angeles Times has published an online database comparing more than 6,000 elementary school teachers based on a controversial statistical method that relies on test-score data to determine their effectiveness.

The database, and its resulting fallout, marks the latest chapter in a national debate over how best to measure teacher quality–one that pits members of the Obama administration against many teachers’ unions. It also raises important questions about what kinds of teacher and school district information should be made publicly available.

The Times “has produced an analysis of how effective Los Angeles Unified School District teachers have been at improving their students’ performance on standardized tests,” the newspaper wrote in explaining its actions. “The Times has decided to make the ratings available because they bear on the performance of public employees who provide an important service, and in the belief that parents and the public have a right to the information.”

The Times rated the city’s third- through fifth-grade teachers using an approach called the value-added model, which is gaining popularity nationwide but remains controversial.

The value-added model seeks to determine the effectiveness of a teacher by looking at the test scores of his or her students. Each student’s past test performance is used to project his or her performance in the future. The difference between the child’s actual and projected results is the estimated “value” that the teacher has added or subtracted during the year. The Times says its ratings of teachers reflect their average results “after teaching a statistically reliable number of students.”

Using test-score data covering seven years, the Times analyzed the effects of more than 6,000 elementary school teachers on their students’ learning in English and math. The analysis reportedly found huge disparities among teachers, some of whom work just down the hall from one another.

“After a single year with teachers who ranked in the top 10 percent in effectiveness, students scored an average of 17 percentile points higher in English and 25 points higher in math than students whose teachers ranked in the bottom 10 percent,” the newspaper reported. “Students often backslid significantly in the classrooms of ineffective teachers, and thousands of students in the study had two or more ineffective teachers in a row.”

The Los Angeles school district has had the ability to analyze these differences among teachers for years “but opted not to do so, in large part because of anticipated union resistance,” the Times reported.

The newspaper acknowledges that its methods aren’t perfect.

“Scholars continue to debate the reliability of various statistical models used for value-added estimates,” it notes. “Each has an inherent error rate that is difficult to measure. Value-added estimates may be influenced by students not being randomly assigned to classes, or by students moving from class to class during a single year. Likewise, they could be misleading for teachers who team-teach. Even many critics of the approach, however, say value-added is a vast improvement on the current evaluation system, in which principals make subjective judgments based on brief pre-announced classroom visits every few years.”

In the days leading up to the Times’ Aug. 29 publication of its database, the president of the Los Angeles teachers union said he was organizing a “massive boycott” of the newspaper.

“You’re leading people in a dangerous direction, making it seem like you can judge the quality of a teacher by … a test,” said A.J. Duffy, president of United Teachers Los Angeles, which has more than 40,000 members. Duffy said he would urge other labor groups to ask their members to cancel their subscriptions.

The American Federation of Teachers (AFT) also weighed in on the newspaper’s move.

The head of the AFT, Randi Weingarten, said she believes parents have a right to know how well their children’s teachers are rated on employee evaluations–but she disagreed with the newspaper’s decision to publish data from its value-added analysis. Such data should be considered only as part of a broader evaluation of a teacher’s performance, she said, and they should be available only to the teacher, his or her principal, and individual parents.

“Today, the Los Angeles Times chose to ignore experts from across the country who have pointed out both the limitations and dangers of using, in isolation, the value-added method to rate a teacher’s performance. We are extremely disappointed that the Times gave no weight to these opinions, but we are more disturbed that teachers will now be unfairly judged by incomplete data masked as comprehensive evaluations,” Weingarten said in an Aug. 29 statement.

“Leading researchers, including the Educational Testing Service, RAND Corp., the National Science Foundation, the U.S. Department of Education, and the Economic Policy Institute, have concluded that value-added models, which deal with predictions and assumptions, are inherently undependable and imprecise. All have concluded that value-added models should never be used in isolation–without other relevant factors–to judge a teacher’s performance. All have found value-added models to be inappropriate for high-stakes decisions about individual teachers, students, and schools. Even one of the pioneers of value-added research, William Sanders, considers the publication of these value-added scores, by themselves, to be wrong.”

She added: “The Los Angeles Times ignores the growing number of districts around the country that are building comprehensive development and evaluation systems that inform teaching and learning. We must use good and meaningful data–but the real value of data is to inform instruction. Ironically, most of the teachers cited in the article were surprised to learn of their ranking and couldn’t explain why their particular method of teaching works. That is why we should follow the experts’ recommendation that this data should not be published.”

Comments left at the Times web site revealed a heated debate within the district itself about the newspaper’s actions.

“Although the value-added measure may not be the most effective tool at measuring teacher effectiveness, it is a tool,” wrote one reader. “Many teachers who are upset over these ratings need to read what is value-added; it does not judge the teacher by how many students are at proficient or advanced level, it looks at student improvement–regardless of where they scored. Teachers should be held responsible for teaching and stop pointing out the injustices of the students they had to work with.”

The reader noted that the teacher deemed most effective according to the Times’ analysis was from a low-income school, suggesting that the value-added model doesn’t penalize teachers for the environment in which they teach.

Others were highly critical of the approach.

“Given the complexity of the issue, a public newspaper isn’t the proper forum for this discussion,” a second reader wrote. “Value-added evaluation and, for that matter, standardized testing are very limited indicators. What about personal evaluations? School funding? One-parent families who don’t read to their kids? The [Times] tossed about a bunch of highly limited if not misleading statistics in an inflammatory way to gin up hatred of teacher’s unions. This was the most slanted, sickening thing I’ve seen from a mainstream newspaper.”

Yet another reader wrote: “I think it was wrong to provide this information. My own daughter has taught third, fourth, and fifth grade. She has a huge class size, and she is not provided a teacher’s aide. … I know my daughter has extreme passion for teaching and helping her students do their best, [and yet] she is expected to pay for many supplies out of her own pocket. Next time, LA Times, why not provide the whole picture and not just a fraction of what the facts are?”

The Los Angeles controversy mirrors that experienced in the Houston Independent School District when district leaders earlier this year led an initiative to make the value-added model a formal part of the city’s teacher evaluation system.

Prompted by the Times’ analysis, Education Secretary Arne Duncan last week called on schools across the country to disclose more information about student achievement and teacher effectiveness, saying too much information that would help teachers and parents is being kept out of public view.

Duncan said schools too often aren’t disclosing years of data on student achievement that could not only help parents measure a teacher’s effectiveness, but also would help teachers gain better feedback.

“Too often, our systems keep all of our teachers in the dark about the quality of their own work,” Duncan told an audience in Little Rock, Ark., on Aug. 25. “In other fields, we talk about success constantly, with statistics and other measures to prove it. Why, in education, are we scared to talk about what success looks like?”

Duncan, who was speaking at a lecture hosted by the University of Arkansas Clinton School of Public Service and the Clinton Presidential Library, said he’s not advocating posting the results online, like the Times did–but he noted that some teachers said their districts had never provided them with such data. He said more than 2,000 teachers asked the newspaper for their scores before the full database was published.

“The fact that teachers did not have information like this for all those years is ridiculous,” he said. “Local school districts, in real partnership and collaboration with their teachers, must decide how to share this information, how to put it in context, and how to use it in order to get better.”