Spotlight Shines On Pittsburgh’s Bumpy Ride to Teacher Evaluation Reform

It’s a Saturday night in early November. A jazz band plays near a rocket simulator in the Carnegie Science Center where a line snakes toward the bar. People in cocktail attire chat over hors d’oeuvres. It’s not often that city public school teachers are at the center of a gala celebrating their work.

But these are uncommon times in the Pittsburgh Public Schools. The teachers honored that night for excellence were identified through a rigorous new performance evaluation process—one that administrators and union officials have collaborated and clashed over for years as educators and scholars across the nation watched.

The new evaluation system has attracted its share of controversy, particularly over the issue of how to use student outcomes to rate teacher performance. But when the first official scores were posted last year, teachers across the district shined.

Teachers matter

A growing body of research shows that a child’s education is strongly related to the quality of his or her teachers. Researchers find, for example, that Pittsburgh Public Schools students who have a teacher rated in the top 10 percent of the staff gain an extra year of learning over students taught by teachers whose work is rated in the bottom 10 percent, according to a 2012 report on teacher effectiveness by Mathematica Policy Research.

“The bottom line is that being an effective teacher means that the students in your charge learn and grow,” says Jerry Longo, a clinical associate professor in the School of Education at the University of Pittsburgh “Having a good teacher makes a difference, but there’s also a lot of research that shows that having consistently good teachers makes a difference. If you have a bad teacher one year, you can actually lose some ground.”

Effective teachers make a big difference, including being critical to closing the achievement gap among races, a 2010 Mathematica study reports. But determining who the quality teachers are and determining the factors that distinguish great teachers from good ones, as well as fair teachers from the weakest teachers has long been problematic for schools.

Traditional evaluation methods have largely failed to help as schools struggle to recognize the individual competencies of teachers and realize that they’re not interchangeable parts, according to a report by the New Teachers Project, a national nonprofit focused on education inequities. As a result, excellence often goes unrecognized, chronic poor performers are kept on staff, and appropriate professional development fails to reach all teachers who could use it to improve their classroom skills.

Such reports have led to policies and strategies to reform the way teachers are evaluated and to promote their professional growth.

Sweeping change

In 2012, Pennsylvania adopted Act 82, requiring all school districts to evaluate teachers based half on observation and half on student performance measures using a model drawn from research on practices that promote professional development and improvement.

While most districts began making the changes only last year, the Pittsburgh Public Schools have been working on sweeping evaluation reform since 2009 supported by federal dollars and a $40 million grant from the Bill and Melinda Gates Foundation.

The model emerged from the collaboration of school administrators and the Pittsburgh Federation of Teachers, the union representing the district’s teachers. It’s an attempt to fashion a system of teacher evaluation aimed at creating opportunities for professional growth, a modified career ladder for teachers, financial incentives for distinguished teachers and multiple measures for determining effective classroom teaching.

The Pittsburgh work on teacher effectiveness has been applauded for developing and implementing best practices related to teacher evaluation in national studies, including a 2014 study by the National Council on Teacher Quality.

“What we can do with this system is learn over time to support teachers in different ways and make sure that the teachers who are doing well on all these things, have the opportunity to lead and share their practice, and feel valued and recognized,” says Sam Franklin, executive director of the Pittsburgh Public Schools’ Office of Teacher Effectiveness.

The model uses teacher observation, student surveys and measures of student outcomes to rate teacher quality as distinguished, proficient, needs improvement, or failing.

Teachers are observed by principals and other teachers under the district’s “Research-based Inclusive System of Evaluation” (RISE), which evaluates them on 24 teaching components to help inform their professional development. Student perceptions are solicited through surveys that ask them about their teachers’ control of the classroom, whether teachers challenge them, whether they feel teachers care about them, and other aspects of teaching.

The third and perhaps the most controversial measure links teacher effectiveness to student outcomes. To do so, the district uses “value-added measures” (VAM) based on a formula developed by Mathematica Policy Research to assess growth in student standardized test scores while taking into account factors like Individual Education Plans and attendance. The student outcome measure, implemented during the 2011–12 school year, was the last component to be added to the evaluation process.

A work in progress

For Donna Ervin, a kindergarten teacher at Pittsburgh King K–8 school, the new evaluation model has meant balancing concerns about how it will affect her work with its potential to promote professional growth—something she’s experienced as one of the district’s teachers recruited to help train and support classroom colleagues.

“Before this evaluation system, you couldn’t say ‘This is good teaching,’” says Ervin, who has taught for 11 years and was recognized for distinguished work at the Carnegie Science Center gala in November. “I couldn’t go into a classroom and say, ‘Here’s where I see a weakness, let me help you.’ RISE has been a tremendous asset to teachers and supporting teachers.”

But the system is not without controversy. From the beginning, adding student outcomes to teacher evaluations raised concern among teachers. Student performance hinges on many factors, including some which teachers have little or no control over, such as family stability, household income and health.

Moreover, the formula for accommodating student outcomes in teacher evaluations has grown complex.

“How we developed RISE in the beginning was so close to my heart,” says Nina Esposito-Visgitis, president of Pittsburgh Federation of Teachers, “It was true collaboration. Our evaluation system had been broken for so long. It was beautiful to watch this grow from the ground up. But now the system has become too overwhelming. It’s become so huge. The transparency isn’t always there. With something like VAM, people say, “Let me see the formula.’ The formula is on the website but who understands it? It’s difficult to be rated on something when you don’t understand how they did it.”

Tensions also have risen between the union and school district over the benchmark scores that determine whether teachers need improvement or are failing. These so-called “cut scores” are generally considered more rigorous than those used in the statewide system. The dispute peaked after a dry run of the evaluation system in 2013 showed that more than 9 percent of the district’s teachers fell into those categories.

But data released late last year told a surprisingly different story.

Nearly 97 percent of the more than 1,700 teachers evaluated earned scores high enough to rate their performance as either distinguished or proficient during 2013–14, the first school year the district used multiple measures in the report that informed what, if any, actions would be taken to improve teacher performance.

Of the top performing teachers, 22.5 percent were identified as distinguished— the highest rating they can receive. As such, they are eligible to be promoted to leadership positions in the district that include financial incentives.

Only 1.6 percent of all teachers were given unsatisfactory ratings, a much smaller share than was identified as unsatisfactory during the dry run the previous year. The evaluation system offers teachers getting such poor ratings the chance to participate in an individualized improvement plan during which their progress is monitored and they are subject to additional observation and feedback from principals and others.

High-performing teachers were identified in schools throughout the city, in all grades, in schools with low-income students and in schools with the most lowachieving students. How the evaluations and emphasis on quality teaching will affect such high-need schools remains unclear. Like distinguished teachers, those who rated less than proficient were spread across the district. But schools with the most students needing academic improvement also had a greater proportion of teachers with unsatisfactory ratings.

Perhaps the final test of the new system of teacher evaluation and professional development unfolding in Pittsburgh is whether it can be sustained.

“It takes an infrastructure to maintain these tools with integrity, and it takes time for principals and teachers to invest in this,” says Franklin, “The evaluation system has components that are deeply focused on professional growth. What we do with that information and how the information is used will go a long way to determine the sustainability.”