In a move that has many local educators seething, the Los Angeles Times has published an online database comparing more than 6,000 elementary school teachers based on a controversial statistical method that relies on test-score data to determine their effectiveness.
The database, and its resulting fallout, marks the latest chapter in a national debate over how best to measure teacher quality–one that pits members of the Obama administration against many teachers’ unions. It also raises important questions about what kinds of teacher and school district information should be made publicly available.
The Times “has produced an analysis of how effective Los Angeles Unified School District teachers have been at improving their students’ performance on standardized tests,” the newspaper wrote in explaining its actions. “The Times has decided to make the ratings available because they bear on the performance of public employees who provide an important service, and in the belief that parents and the public have a right to the information.”
The Times rated the city’s third- through fifth-grade teachers using an approach called the value-added model, which is gaining popularity nationwide but remains controversial.
The value-added model seeks to determine the effectiveness of a teacher by looking at the test scores of his or her students. Each student’s past test performance is used to project his or her performance in the future. The difference between the child’s actual and projected results is the estimated “value” that the teacher has added or subtracted during the year. The Times says its ratings of teachers reflect their average results “after teaching a statistically reliable number of students.”
Using test-score data covering seven years, the Times analyzed the effects of more than 6,000 elementary school teachers on their students’ learning in English and math. The analysis reportedly found huge disparities among teachers, some of whom work just down the hall from one another.
“After a single year with teachers who ranked in the top 10 percent in effectiveness, students scored an average of 17 percentile points higher in English and 25 points higher in math than students whose teachers ranked in the bottom 10 percent,” the newspaper reported. “Students often backslid significantly in the classrooms of ineffective teachers, and thousands of students in the study had two or more ineffective teachers in a row.”
The Los Angeles school district has had the ability to analyze these differences among teachers for years “but opted not to do so, in large part because of anticipated union resistance,” the Times reported.
The newspaper acknowledges that its methods aren’t perfect.
“Scholars continue to debate the reliability of various statistical models used for value-added estimates,” it notes. “Each has an inherent error rate that is difficult to measure. Value-added estimates may be influenced by students not being randomly assigned to classes, or by students moving from class to class during a single year. Likewise, they could be misleading for teachers who team-teach. Even many critics of the approach, however, say value-added is a vast improvement on the current evaluation system, in which principals make subjective judgments based on brief pre-announced classroom visits every few years.”
In the days leading up to the Times’ Aug. 29 publication of its database, the president of the Los Angeles teachers union said he was organizing a “massive boycott” of the newspaper.
“You’re leading people in a dangerous direction, making it seem like you can judge the quality of a teacher by … a test,” said A.J. Duffy, president of United Teachers Los Angeles, which has more than 40,000 members. Duffy said he would urge other labor groups to ask their members to cancel their subscriptions.
The American Federation of Teachers (AFT) also weighed in on the newspaper’s move.
The head of the AFT, Randi Weingarten, said she believes parents have a right to know how well their children’s teachers are rated on employee evaluations–but she disagreed with the newspaper’s decision to publish data from its value-added analysis. Such data should be considered only as part of a broader evaluation of a teacher’s performance, she said, and they should be available only to the teacher, his or her principal, and individual parents.
“Today, the Los Angeles Times chose to ignore experts from across the country who have pointed out both the limitations and dangers of using, in isolation, the value-added method to rate a teacher’s performance. We are extremely disappointed that the Times gave no weight to these opinions, but we are more disturbed that teachers will now be unfairly judged by incomplete data masked as comprehensive evaluations,” Weingarten said in an Aug. 29 statement.
“Leading researchers, including the Educational Testing Service, RAND Corp., the National Science Foundation, the U.S. Department of Education, and the Economic Policy Institute, have concluded that value-added models, which deal with predictions and assumptions, are inherently undependable and imprecise. All have concluded that value-added models should never be used in isolation–without other relevant factors–to judge a teacher’s performance. All have found value-added models to be inappropriate for high-stakes decisions about individual teachers, students, and schools. Even one of the pioneers of value-added research, William Sanders, considers the publication of these value-added scores, by themselves, to be wrong.”