Grading teachers is proving difficult for principals

Teachers in school districts where Michigan’s proposed teacher evaluation system is being piloted are generally receptive to the changes. But inconsistent scores and a lawsuit in one district are among the challenges schools are likely to face when the system is rolled out across the state as soon as next school year.

Brian Rowan, professor in the School of Education at the University of Michigan, is part of a team at U-M’s Institute for Social Research assessing pilot programs at 14 school districts across Michigan. That assessment isn’t complete, but Rowan offered insights into the team’s findings at a conference of education journalists in Chicago in September, and later to Bridge Magazine.

Under the state’s new evaluation system, teachers’ performance will be judged by a number of criteria, with two of the most significant being: more extensive classroom observations by trained administrators, and measurement of student academic growth during the school year.

Pilot programs tested during the 2012-2013 school year included four different classroom observation models, as well as a first stab at using student test scores to evaluate teacher performance.

The student growth data is still being assessed, but reviews of the observation portion of the pilot programs were mostly positive. “Most of the district personnel were really happy to get a standardized evaluation protocol,” Rowan said. “The teachers were generally positive (and) most teachers in these districts were observed.”

The scores produced from those observations, however, tell Rowan that a lot more training needs to be done.

The number of times teachers were observed “varied tremendously” from school to school, Rowan said, noting that “reliability increases with multiple observations.”

About 10 percent of the evaluations weren’t filled out completely. And teacher ratings were all over the map; in some cases, the scores given by principals varied widely from those of the U-M observers who were conducting parallel evaluations.

“There was very little agreement between principals and our observers about what is relevant,” Rowan said. “It makes you wonder if people are scoring it consistently.”

Consistent scoring, based on common definitions of performance, is critical to the system’s reliability, and credibility, and will help ensure that teachers get the training and support they need to improve. It’s important, for instance, that a highly effective rating means the same thing from one school to the next. Proponents of this reform say better evaluations can also identify truly high-performing teachers so they can be recognized for their work and assigned to the students who need help most.

Rowan said the state needs to invest the time and money for thorough training of school principals, who are likely to conduct the majority of evaluations, to avoid “tons of observation error.”

In Port Huron, teachers are suing the school district over the pilot program evaluation system. In the lawsuit, the Port Huron Education Association claims teachers at one school were all graded "highly effective" because performance data was missing. The criteria used to measure student growth varied from building to building, and observations varied in frequency and scoring method.

“There are severe inconsistencies in how the district is doing the evaluation system,” said attorney Jeff Donahue, representing the teachers union. “And when you have such high stakes, we believe it has to be done correctly.”

Farmington Public Schools had a more positive experience with the pilot program, said Associate Superintendent Michele Harmala. Teachers were involved in the implementation of the system. “It created buy-in from the teachers,” Harmala said. “They set the bar high. They are professionals.

“After a year in the pilot, we realize we haven’t done enough training yet,” Harmala said. “We’re going through another round of training to deepen the practices. I think the training relative to the protocols will be ongoing for a number of years.”

Harmala said the state needs to understand the complexities of scaling up a massive reform like this, particularly in large districts.

“We’ve got over 800 teachers,” Harmala said. “For us it’s a big deal to scale it up. That’s why the training is so critical. Even going in to a year of training, we’re still just reaching the tip of the iceberg.”