Of all school-level factors related to student learning and achievement, the quality of the student’s teacher is the most important. Yet the teacher evaluation systems in use in American school districts historically have been unable to differentiate teachers who improve student learning from lower-performing educators. Many have failed to differentiate teachers at all. A 2009 study by The New Teacher Project found that “satisfactory” or “unsatisfactory” were the only ratings available to school administrators in many districts, and that more than 99 percent of teachers in those districts were deemed satisfactory.

Improving methods for evaluating teacher performance and using the resulting information to change teaching practice has been a focus of recent reform efforts. According to the National Council on Teacher Quality, 32 states and the District of Columbia altered their teacher-evaluation policies in recent years to incorporate multiple methods of assessing and evaluating teachers, spurred in part by the federal Race to the Top competition. And each of the 43 states to which the Obama administration has granted a waiver from No Child Left Behind is now in the process of implementing evaluation systems that employ multiple measures of classroom performance, including student achievement data. These systems differentiate among three or more performance levels and are used to inform personnel decisions.

While much of the debate over these new evaluation systems centers on their use of student test-score data to measure a teacher’s “value added” to student learning, classroom observations remain critically important. Most teachers work in grades or subjects in which standardized tests are not administered and therefore will not have a value-added score. Even when students’ test scores are available, classroom observations may capture dimensions of teachers’ performance that are important but not reflected in those scores. Finally, value-added scores on their own do not tell teachers how they might improve their practice and thereby raise student achievement.

We examine a unique intervention in Chicago Public Schools (CPS) to uncover the causal impact on school performance of an evaluation system based on highly structured classroom observations of teacher practice. An iterative process of observation and conferencing focused on improving lesson planning and preparation, the classroom environment, and instructional techniques should drive positive changes in teacher practice. As teachers refine their skills and learn how best to respond to their students’ learning needs, student performance should improve. Recent evidence from Cincinnati Public Schools confirms that providing midcareer teachers with evaluative feedback based on the Danielson Framework for Teaching observation system can promote student-achievement growth in math, both during the school year in which the teacher is evaluated and in the years after evaluation (see “Can Teacher Evaluation Improve Teaching?” research, Fall 2012).

The Excellence in Teaching Project (EITP), a teacher evaluation system also based on the Danielson framework, was piloted in Chicago Public Schools beginning in the fall of 2008. Leveraging the random assignment of schools to the EITP intervention, we find large effects of the intervention on school reading performance. The program had the largest impact in low-poverty and high-achieving schools but little or no impact in less-advantaged schools. These effects seem to be a consequence not only of the design and focus of the EITP pilot but also of the extent to which CPS supported the implementation of the new evaluation process. Similar benefits were not observed in schools implementing the same program the following year with less support from the central office, suggesting the importance of sustained support for teacher evaluation reform to translate into improved student performance.

Teacher Evaluation in Chicago Public Schools 

For nearly four decades prior to the introduction of the EITP, CPS teachers were observed and evaluated based on a checklist of 19 classroom practices. During a classroom observation of a teacher’s lesson, the observer (usually the principal, but sometimes an assistant principal) would check one of three boxes (Strength, Weakness, Does Not Apply) next to each of the practices. The checklist approach was unpopular among both teachers and principals. High-performing teachers believed that the system did not provide meaningful feedback on their instruction, and only 39 percent of veteran principals agreed that the checklist allowed them to adequately address teacher underperformance. The system provided no formal guidance or rubric to either party on what constituted strong or weak performance on any of the checklist practices.

Moreover, there was no direct correspondence between a teacher’s ratings on the checklist and the overall evaluation rating, which determined teacher tenure. Overall evaluations also showed little differentiation among teachers. Nearly all teachers (93 percent) received ratings of “Superior” or “Excellent” (the top-two categories in a four-tier rating system). Meanwhile, two-thirds of CPS schools failed to meet state proficiency standards under Illinois’s accountability system, and Chicago remained among the nation’s lowest-performing urban districts on the National Assessment of Educational Progress.

Dissatisfaction with the evaluation system led CPS leadership under then CEO Arne Duncan to develop the EITP in partnership with the Chicago Teachers Union (CTU), beginning in 2006. A joint CPS–CTU committee met together over two years to negotiate the details of the evaluation pilot. In the summer of 2008, just prior to implementation, the district and union disagreed on whether the ratings teachers received under the EITP would be used for teacher accountability purposes, such as tenure decisions. The district nonetheless moved forward with the pilot to implement formative, ongoing assessments for teachers that would provide them with structured feedback on their instructional practices.

The classroom observation process had occurred formally (if superficially) twice a year for all teachers, irrespective of tenure status, as part of the district–union teacher contract. While maintaining this schedule, the EITP changed the process significantly. First, principals and teachers engaged in a brief (15- to 20-minute) pre-observation conference during which they reviewed the rubric. The conference also gave the teacher an opportunity to share any information about the classroom with the principal, such as issues with individual students or specific areas of practice about which the teacher wanted feedback. During the 30- to 60-minute lesson that followed, the principal was to take detailed notes about what the teacher and students were doing. After the observation, the principal was expected to match classroom observation notes to the Danielson framework rubric in order to rate teacher performance in 10 areas of instructional practice.

The Danielson framework delineates four levels of performance (Unsatisfactory, Basic, Proficient, and Distinguished) across four domains, of which the EITP focused on two: Classroom Environment and Instruction. Within a week of the observation, the principal and teacher conducted a postobservation conference. During the conference, the principal shared evidence from the classroom observation, as well as the Danielson ratings, with the teacher. Principals and teachers were expected to discuss any areas of disagreement in the ratings, with a specific focus on ways to improve the teacher’s instructional practice and, ultimately, student achievement.

The EITP represented a dramatic shift in the way teacher evaluation had occurred in CPS, and central-office staff sought to develop principals’ capacity to conduct these classroom observations and conferences. In 2008–09, the first year of implementation, 44 participating principals received approximately 50 hours of training and support, with three days of initial training during the summer and follow-up sessions throughout the school year. The initial training covered the use of the Danielson framework to rate teaching practice, methods for collecting evidence, and best practices for conducting classroom observations. The follow-up sessions consisted of seven monthly meetings in which principals brought materials from classroom observations that they had conducted and engaged in small-group discussion with their colleagues. Four additional half-day trainings during the school year provided an opportunity for principals to update their understanding and use of the rubric for evaluating teachers.

Principals also received additional one-on-one support from the CPS central office. During this first year of implementation, central-office administrators responsible for EITP engaged with principals through weekly e-mails, providing consistent reminders to principals about observation deadlines and other EITP requirements. Principals could request time with EITP central-office staff to review their teacher ratings as a means of calibrating their observation sessions to EITP central office expectations. Finally, principals received individualized ratings reports from the University of Chicago Consortium on Chicago School Research (CCSR). The CCSR reports provided principals with a comparison of their own teacher ratings to ratings generated by trained external observers of the same teachers. These reports supported principals in making adjustments to their own ratings of teacher performance.

Forty-four schools participated in EITP in the first year. These 44 Cohort 1 schools continued to take part in the second year, and an additional 48 schools (Cohort 2) implemented EITP for the first time. The extent of principal training and support for the 48 new schools differed dramatically from Cohort 1, however. In their first year, Cohort 2 principals received just two days of initial training on how to collect evidence on teaching practices during classroom observations and how to rate these practices using the Danielson framework.

Cohort 2 principals also received significantly less district-level support throughout the school year than Cohort 1 principals had in their first year of implementation. Although Cohort 2 principals could request technical assistance from EITP central-office staff, these principals did not have access to the ongoing technical support and oversight that Cohort 1 principals received. Indeed, Cohort 1 principals received the same level of support and ongoing training in their second year of implementation as did the Cohort 2 principals in their first year.

Data

Data for this study consist of CPS administrative, personnel, and test-score information from the 2005–06 school year to the 2010–11 school year. As the intervention occurred at the school level, we used school-level averages of all student-level and teacher-level data records. Administrative data collected on students include basic demographic information, such as gender and race/ethnicity as well as information on poverty level and students with special education needs. We also use school-level characteristics such as student enrollment levels and the distribution of race/ethnicity, gender, students qualifying for free or reduced-price lunch, and special education students, which were generated from student-level CPS data files. Teacher personnel data include teacher-level data about tenure status, years of experience in the district, demographic information, level of education attained, and certification status.

Our primary outcome variable is student achievement as measured by performance on standardized tests. Students in Illinois take the Illinois Standards Achievement Test (ISAT) in reading and mathematics in grades 3 through 8, usually in March of each school year. We use a school-level measure that has been standardized across the sample of schools included in our analysis, taking into account the various grade configurations in different schools.

Methodology

We take advantage of a unique randomized control trial design. CPS, in partnership with CCSR, selected four elementary-school instructional areas (of the 17 elementary zones in the city at the time) that would implement the EITP. These areas are located in different parts of the city, and they serve different populations of CPS students with varying needs. Within each of the four instructional areas, elementary schools were randomly selected to participate in the first year of EITP (Cohort 1). Schools with first-year principals and those slated for closure in the spring of 2009 were excluded from the sample prior to randomization.

Schools that were not selected to participate in the first year implemented the program the following school year (Cohort 2). The randomization process resulted in 44 Cohort 1 schools and 49 Cohort 2 schools (the latter number fell to 48 due to the unexpected closure of one school). Our data indicate that the randomization procedure worked as desired. On average, the Cohort 1 and Cohort 2 schools were very similar in terms of both student and teacher characteristics as well as school working conditions.

We measure the initial impact of the EITP on a school’s math and reading achievement by comparing student achievement between the Cohort 1 and Cohort 2 schools at the end of the 2008–09 school year, during which Cohort 1 schools implemented the EITP but Cohort 2 schools did not. To increase the precision of our results, we control for student enrollment, the proportion of female students, the proportion of students by race/ethnicity, the proportion of special education students, the proportion of students receiving free or reduced-price lunch, and average prior achievement.

Results

In its first year, the EITP increased student achievement in the Cohort 1 schools by 5.4 percent of a standard deviation in math and 9.9 percent of a standard deviation in reading, relative to the Cohort 2 schools. The effect on reading scores is statistically significant, but the effect on math scores is not. The reading effect is significant not just statistically but also in size. A 10 percent of a standard deviation effect size is equivalent to closing one-quarter to one-half of the performance gap between weak schools (those at the 10th percentile of the achievement distribution) and average schools (those at the 50th percentile) in large urban districts like Chicago.

In the second year, as Cohort 2 schools implemented EITP, we might have expected the difference between the two groups of schools to shrink or even disappear as the Cohort 2 schools benefited from the same program that had a positive impact on Cohort 1 schools the prior year. We find, however, that the difference in student achievement between the two groups of schools persisted over time. Figure 1 shows that the math effect of 5.4 percent increased to 8 percent in year two and was 6.6 percent in year three. For reading, the first-year effect of 9.9 percent grew to 11.5 and 12 percent in the second and third years.

More-advantaged schools—those with fewer students eligible for free or reduced-price lunch and those with higher initial student achievement—benefited the most from the program. On average across all schools, 83 percent of students received free or reduced-price lunch. The effect of EITP at lower-poverty schools—those with just 60 percent of students receiving free or reduced-price lunch—was double the effect for the full sample, at more than 20 percent of a standard deviation (see Figure 2). On the other end of the distribution, there was no detectable EITP effect at higher-poverty schools. This differential effect persisted into the second and third years of the intervention, after Cohort 2 schools implemented the program.

We find similar differential effects on math by school poverty level, with a statistically significant positive effect for lower-poverty schools, even though the average effect across all schools was not distinguishable from zero. We also find evidence that schools with higher student achievement before the start of the EITP benefited the most from the program. We do not, however, find any consistent evidence that the effect of the program was related to the racial composition or share of special education students in the school.

Explaining Differential Impacts

Why did the EITP only improve achievement in certain schools and only in the first year? The EITP represented a dramatic departure from the existing teacher-evaluation system in Chicago and relied on the human capital that already existed in the schools to generate improvements in school performance. Its efficacy depended on principals’ capacity to provide targeted instructional guidance, teachers’ ability to respond to the instructional feedback in a manner that generated improvements in student achievement, and the extent of district-level support and training for principals who were primarily responsible for implementing the new system.

The pilot forced principals to make significant changes to how they conducted classroom observations and conferences with teachers. The intervention itself was time-intensive for the principals, who were required to participate in extensive training pre-intervention. Principals also had to rate teachers on the new evaluation framework, and work with them in pre- and postobservation conferences to develop strategies to improve their instructional practice. On average, CPS principals reported that they spend about six hours per teacher during each formal observation cycle.

The principals’ role evolved from pure evaluation to a dual role in which, by incorporating instructional coaching, the principal served as both evaluator and formative assessor of a teacher’s instructional practice. It seems reasonable to expect that more-able principals could make this transition more effectively than less-able principals. A very similar argument can be made for the demands that the new evaluation process placed on teachers. More-capable teachers are likely more able to incorporate principal feedback and assessment into their instructional practice.

Our results indicate that while the pilot evaluation system led to large short-term, positive effects on school reading performance, these effects were concentrated in schools that, on average, served higher-achieving and less-disadvantaged students. For high-poverty schools, the effect of the pilot is basically zero.

We suspect that this finding is the result of the unequal allocation of principals and teachers across schools as well as additional demands placed on teachers and principals in more disadvantaged schools, which may impede their abilities to implement these types of reforms. For example, if higher-quality principals and teachers are concentrated in higher-achieving, lower-poverty schools, it should not be surprising that a program that relies on high-quality principals and teachers has larger effects in these schools. In addition, less-advantaged schools with, on average, harder-to-serve student populations, may require additional supports for these kinds of interventions to generate improvements in student learning similar to those of more-advantaged schools.

Varied Implementation

School-level implementation is critically important for the success of any new educational intervention. As discussed above, the extent of principal training and district-level support varied dramatically for Cohort 1 and Cohort 2 schools. We speculate that district support also played an important role in explaining the large positive effect for Cohort 1 and the null effect for Cohort 2.

Leadership turnover in CPS led to a decline in institutional and district support for EITP between the first and second years of the pilot program. When the pilot started in Chicago in 2008, few people were paying attention to teacher evaluation issues. Through its two years of planning work with the teachers union, the district leadership demonstrated its commitment to the program and to evaluating teachers in a way that was systematic and fair. When introducing the pilot program for the first time to principals, the chief education officer, Barbara Eason-Watkins, herself a former principal, personally delivered the message that the EITP pilot would be the district’s cornerstone in improving the quality of teaching and instruction and increasing student learning.

Not long into the pilot’s first year of implementation, however, CEO Arne Duncan left CPS to serve as U.S. secretary of education. While Duncan’s arrival in Washington in early 2009 was followed by a national emphasis on refining teacher evaluation systems, his departure from Chicago marked a move away from the rigorous year one implementation of the EITP pilot. The incoming administration deemphasized the teacher evaluation pilot and instead focused on performance monitoring, data usage, and accountability.

When the EITP expanded to include the Cohort 2 schools in 2009, doubling the number of schools implementing the pilot, the budget for district support of the program did not increase. This limited the amount of support the central office could provide to principals, which we suspect reduced the fidelity with which the pilot was implemented and in turn weakened the intervention. CPS central-office staff responsible for EITP oversight and school-level implementation indicated that there was a significant decrease in both CPS staff and budgetary resources dedicated to Cohort 2 principals in comparison to the level of support Cohort 1 principals received during their first year of program participation.

As a result, Cohort 2 principals received fewer hours of training as well as different types of training than Cohort 1 principals did in their first year of system implementation. Finally, in the summer of 2010, prior to the third year of implementation, CPS ended EITP. Just before this announcement, half of the principals in the district were set to receive Danielson framework training, but the district canceled it. As a result, there is little evidence that the Danielson framework was used in any systematic way in year three. Our results are consistent with strong implementation in year one and weak or no implementation in subsequent years.

Conclusion 

The implementation of the EITP pilot in Chicago occurred prior to the nationwide shift toward more rigorous teacher-evaluation systems. These new teacher-evaluation systems incorporate multiple measures of teacher performance, including value-added metrics based on standardized tests or teacher-designed assessments and, in some cases, student feedback on teacher performance and peer evaluations. Unlike these systems, the EITP was focused solely on classroom observation. What is notable about the version of teacher evaluation systems currently evolving in districts throughout the nation, however, is the continued emphasis on classroom observations, with many systems employing the same observation tool used in CPS under the EITP initiative.

A number of important issues remain unexamined. Specifically, what are the mechanisms through which the evaluation pilot produced improvements in school performance? For example, did the teacher evaluation pilot produce changes in instructional climate or alter the nature of within-school teacher collaboration? To what extent does a performance evaluation system alter teacher mobility and turnover patterns? Answers to these and other questions will shed light on how teacher evaluation systems might improve instructional practice as well as their
implications for the teacher labor market.

Chicago’s decision to abandon the EITP pilot, after supporting it fully for just one year, illustrates the difficulty urban school districts have in sustaining large-scale policy changes that require ongoing support from the central office and significant investment on the part of educators in specific schools. In this case, the program had considerable promise. In the fall of 2012, CPS launched a new teacher-evaluation program in order to comply with the Illinois Performance Evaluation Reform Act, which requires that indicators of student growth be a “significant factor” in teacher evaluation. Called REACH (Recognizing Educators Advancing Chicago Students), the new program also uses the Danielson framework for the classroom observation component.

Matthew Steinberg is assistant professor of education at the University of Pennsylvania Graduate School of Education. Lauren Sartain is research analyst at the Consortium on Chicago School Research at the University of Chicago. This article is based on a forthcoming study in Education Finance and Policy.

In June, a California court ruled, in Vergara v. State of California, that the state’s tenure and seniority laws are unconstitutional. Minority students have filed a similar case in New York, with more to come elsewhere.

The powerful California Teachers Association has appealed the judge’s decision, and teachers unions across the country are arguing that teachers deserve job protection because principals rate almost all of them as effective. Their point is well taken. In October 2014, 94 percent of teachers in the State of New York were identified by school district officials as either effective or highly effective. (The percentage does not include New York City teachers, as no data are available for the district.)

But how do teachers rate their own colleagues when given a chance to do so anonymously?

An answer to that question is to be found in the eighth annual Education Next survey of public and teacher opinion discussed in this issue of the journal (see “No Common Opinion on the Common Core,” features, Winter 2015).

Survey respondents were asked to state the percentage of teachers in their local school district they think deserve one of the five grades on the traditional A-to-F scale. To make sure respondents provided a consistent set of responses, the survey forced the five grading categories to add up to 100 percent before the respondent was able to move on to the next question.

Teachers awarded 69 percent of their colleagues in the local school district an A or B. But teacher enthusiasm for their co-workers does not extend to all of them. Teachers report that 8 percent of their colleagues deserve a D and 5 percent deserve an F.

Parents grade teachers more harshly. They give 56 percent of the teachers in the local schools one of the two top grades, and hand out a D to 11 percent and an F to 10 percent.

Union leaders could argue that the parents are blaming teachers for their own children’s faults, but they may find it difficult to explain away the fact that even teachers consider well over 10 percent of their colleagues to be woefully inadequate.

You can’t have good schools without good teachers across the board, but improving the lowest-performing segment of teachers would go a long way in this regard. Good teachers are so important to student learning that if the lowest-performing 5 to 7 percent of all teachers were replaced with just average teachers, the long-term benefits to the nation’s human capital would be enough to increase annual economic growth rates by as much as 1 percent annually, according to estimates from Stanford economist Eric Hanushek.

The public seems to agree that something needs to be done, and that is where tenure laws come in. Survey respondents favor ending tenure by a ratio of 2:1. By about the same ratio, the public also thinks that if tenure is awarded, it should be based in part on how well the teacher’s students perform in the classroom.

Interestingly enough, a majority of teachers agree that change is needed. Only 41 percent of teachers both favor tenure and think that it should be unrelated to student performance.

Courts have yet to reach a final verdict on teacher tenure and seniority rights, but the court of public opinion has already made a clear determination.

—Paul E. Peterson

Detroit is a classic story of a once-thriving city that has lost its employment base, its upper and middle classes, and much of its hope for the future. The city has been on a long, slow decline for decades. It’s difficult to convey the postapocalyptic nature of Detroit. Miles upon miles of abandoned houses are in piles of rot and ashes. Unemployment, violent crime, and decades of underinvestment have led to a near-complete breakdown of civic infrastructure: the roads are terrible, the police are understaffed, and there is a deeply insufficient social safety net.

There are new federal funds and private investment being directed to Detroit’s renewal. Bankruptcy proceedings are finally under way, and a new mayor wants to make a fresh start. But it’s hard to see how a renaissance can occur without making headway on the public schools. Detroit parents still have very few high-quality options, despite a number of different reform interventions, including putting a state-appointed emergency manager in charge of the district, pulling the lowest-performing schools into a statewide turnaround district, and allowing a significant number of charter schools to operate.

In January 2014, as part of a multicity study, researchers from the Center on Reinventing Public Education (CRPE) met with a dozen parents in Detroit to learn about their experiences with education in the city. What follows is one of many similar stories we heard.

Ms. Gordon (not her real name) is a lifelong Detroit resident. Her 11-year-old son will enter middle school in the fall of 2014, and she is anxious about how to find and choose his next school. He has not had an easy time in elementary school; he struggled academically and was often in trouble for his behavior. Over the years, she has tried to talk to the principal and her son’s teachers, but it always felt as though no one was listening to her concerns or willing to work with her to address them. Now, as she’s looking for a middle school, she wants her son to have a fresh start and a chance to get the academic and social support that he needs. A friend suggested she look at the charter school that her daughter attends. The school sounded interesting, but Ms. Gordon decided it was too far away for her son to travel there safely on his own. Even if she could find a safe route, she was disappointed to read in a parent guide [published by Excellent Schools Detroit] that the kids at the charter school weren’t doing any better than those at the low-performing neighborhood middle school. In fact, few schools looked like good options, even though there were many to choose from. As she faced spring enrollment decisions at the time we talked to her, she felt she was no closer to finding a school that would be a good fit for her son. She expressed frustration and despair, recounting her efforts: “It just feels like you have to fight for your kids every day in this city, because no one else will.” 

Today, Detroit is a “high-choice” city. Families choose from among charter schools, magnet schools, district schools, and schools in nearby districts. In Michigan, public universities, community colleges, intermediate school districts, and all traditional K–12 districts, called “sponsors,” can authorize an unlimited number of charter schools in Detroit and elsewhere in the state. The city’s charter sector expanded rapidly between 2010 and 2013; 32 new schools opened, a 42 percent increase in just those three years, bringing the total number to 109. By percentage of total enrollment, Detroit has the third-largest charter sector in the country, after New Orleans and Washington, D.C. As of the 2012–13 school year, nearly as many children attend charters (39,353) as attend Detroit’s district schools (49,172) (see Figure 1). Other Detroit families, around 14 percent, take advantage of Michigan’s interdistrict choice law that allows them to send their children to nearby suburban schools. Both Detroit’s charter and traditional public-school sectors serve predominantly African American families (roughly 85 percent) with limited economic resources (in charters, 84.5 percent qualify for free or reduced-price lunch versus 81.6 percent in district schools).

In Detroit, as was the case in the other cities we studied, parents struggle to navigate the city’s complex education marketplace and find quality options for their children. For parents like Ms. Gordon, a lack of information, confusing paperwork, and transportation gaps all make it hard to find a school that will work for their child. Parents told us you have to be a “fighter” if you want to find a good school in Detroit—no one is there to help. If parents like Ms. Gordon have a tough time, imagine those who do not speak English or have a student with disabilities. “There are no watchdogs in Detroit to make sure parents [of children with special needs] get what they need from schools,” said a charter school leader. “They’re on their own.”

School Choice with Few Options

Between 2005 and 2012, Detroit Public Schools (DPS) lost two-thirds of its enrollment or more than 84,000 students (see Figure 1). Most of these enrollment losses are a result of the city’s larger economic decline and the corresponding departure of residents. With a dwindling student population and an expanding array of education options, Detroit’s schools are in an all-out battle for students. The challenges of navigating choice in Detroit are made more complicated by this hypercompetitive environment. Some estimate there are currently 20,000 to 30,000 more seats than students in the city’s traditional and charter schools. A parent advocate called the competition for students a “snatch and grab.” A district official likened it to “guerrilla warfare,” with door-to-door battle plans for student recruitment. Competition between schools is so fierce that charter schools in the same charter network say they sometimes fight over students. Other school leaders say they worry about fending off schools that might open nearby.

Poor performance plagues schools in both DPS and the city’s large charter sector. Excellent Schools Detroit (ESD), a coalition of philanthropic, education, and community leaders, began grading publicly funded schools based on school climate and academic performance data in 2012. ESD gave just 16 percent of the city’s public schools (district or charter) a C+ or better in 2014, based on a combination of academic status, progress, and school climate measures. While the city has more than 250 schools, the city’s Eastside and Westside neighborhoods have just 10 high-quality K–8 programs between them; some neighborhoods have no schools with a passing grade.

Scores on the National Assessment for Educational Progress have been impossibly low since 2009; just 4 percent of 4th-grade students were proficient in math and 7 percent in reading in 2013. The results put the district far behind other urban school districts, and behind even other midwestern industrial cities like Cleveland and Chicago (see Figure 2).

Charter schools offer slightly more hope than traditional district schools. According to a 2013 report by the Center for Research on Education Outcomes (CREDO) at Stanford University, based on 2011 data, nearly half of Detroit’s charter schools outperform district schools, but given the very low bar set by the district, that’s not too difficult. The other half of Detroit’s charter schools have outcomes that are statistically indistinguishable from the city’s poorly performing traditional district institutions. It bears noting that these charter results are significantly better than the national average CREDO reported in 2009, in which just 17 percent of charter schools in the 16 states they studied performed better than their district counterparts.

The CREDO analysis also shows that Michigan’s low-income students, who comprise the vast majority of charter students in Detroit, make modest achievement gains (less than a month of additional learning in math each year) compared to district schools, as do black and Hispanic students.

Another study, by Michigan’s Mackinac Center for Public Policy, found positive, but by their admission “not great,” results: Detroit charter high schools performed somewhat better than predicted based on their socioeconomic makeup, while Detroit Public Schools performed worse than predicted.

Even Michigan charter advocates have trouble defending the overall state of charter quality. Many told us there were too many low-performing charter schools in Detroit and struggled to name any Detroit charter schools that are getting results like those achieved by national nonprofit charter management organizations. No high-performing national CMOs operate in the city. The incoming head of one of Detroit’s most well-regarded charter school networks told us that he thought his schools had a long way to go academically and had trouble attracting students.

The dearth of high-quality options is evident to parents. Nearly half of Detroit parents surveyed in March 2014 (48 percent) believe they would have a hard time finding another good school if their current school closed. Nearly half (45 percent) said that it was difficult to find a good fit for their child.

Navigation Trials

Parents in Detroit confronted more barriers to choice than those in any other city in our sample: they cite safety issues, lack of transportation, and lack of information as serious barriers to finding a good school. Nearly 40 percent of surveyed parents say they have trouble knowing whether their child is eligible for different schools, compared to 33 percent for all eight cities in which we administered the survey. About one-third of parents say that finding transportation and information about schools made choosing harder, which is again worse than we found in the average city. In Detroit, as elsewhere, information barriers are higher for parents with little education and for those whose children have special needs.

Though many disadvantaged parents in Detroit are actively choosing a school for their child (55 percent of parents with a high school education), parents with the least education are much less likely than parents with college degrees to say their child is in a school that was their first or second choice (57 percent of parents with a high school education or less were in their top choice compared to 80 percent of those with a bachelor’s degree or more). Parents whose children have special needs are much less likely than parents of students in regular education to say their child is in a school that was their first or second choice (58 percent versus 74 percent).

Students with special needs are less likely than their peers to avail themselves of school choice options. These students are much more likely to attend Detroit’s traditional public schools than charters: 18 percent of DPS students have IEPs compared to 10 percent in charter schools. And while there are a variety of reasons this gap may exist, parents and others we interviewed told us that the proportion of IEP-eligible students in DPS is growing rapidly in large part because a number of Detroit charter schools simply don’t offer many special-education supports.

One community leader summed up the situation this way:

Detroit’s marketplace is as unregulated and unmanaged as any in the country, and tilts strongly toward favoring the supply side. It’s like a flea market…anyone can set up a table to sell their magic, and anyone can come shopping and make a deal, but buyer beware. In Detroit, more parents exercising choice has not resulted in better schools, and more charter schools has not resulted in better choices.

“The market is saturated,” a charter school leader confirmed, “but they keep on coming, and no one is shutting down the bad ones.”

Wanted: An Education Leader

Whose job is it to fix the problems facing parents in Detroit? Our interviews with leaders in the city suggest that no one knows the answer. It is not the state, which defers oversight to local education agencies and charter authorizers. It is not DPS, which views charters as a threat to its survival. It is not charter school authorizers, who are only responsible for ensuring that the schools they sponsor comply with the state’s charter-school law. It is not the mayor, who thus far sees education as beyond his purview. And it is not the schools themselves, which only want to fill their seats and serve the children they enroll.

No one in Detroit is responsible for ensuring that all neighborhoods and students have high-quality options or that parents have the information and resources they need to choose a school.

“It’s a free-for-all,” one observer said. “We have all these crummy schools around, and nobody can figure out how to get quality back under control…. Detroit hasn’t set the conditions to make school choice work for families and kids.”

Only one of Detroit’s charter authorizers is local—DPS, which authorizes 13 schools; the rest are colleges and universities with headquarters outside of the city. Each sponsor has different standards for approving and closing schools, and the quality of their schools varies widely. Unfortunately, there are few incentives and little capacity for improving the city’s charter schools. State law allows charter authorizers to retain 3 percent of the state funding for the schools they sponsor, creating a strong disincentive for closing schools. The state has the authority to close schools and revoke an authorizer’s license but has never used it. According to data from the National Alliance for Public Charter Schools, 12 of Detroit’s charter schools closed between 2010 and 2013. Among these closures, just five were related to academic performance.

Authorizers are free to open schools wherever they choose, regardless of need, and to allow poor-performing schools to remain open. In 2012, the legislature seemingly weakened its oversight of the charter sector by eliminating a requirement that the state education agency report on charter school quality each year. At the local level, few schools or authorizers are willing to do anything that might threaten their ability to attract and retain families.

Poor quality and a lack of incentives for improvement extend to the traditional district schools as well. Our interviews with DPS officials make clear they view the district’s problems in terms of marketing and customer service, rather than school quality. A visit to the district’s web site for enrollment reveals numerous materials touting the quality of district schools. But if parents want to find information on school performance or climate, they need to look elsewhere.

The Education Achievement Authority (EAA) of Michigan, a state agency, in 2011 took over 12 schools and now oversees 15 schools in Detroit, including 3 charter schools. The most intensive EAA schools run 210 days a year and employ a blended learning model. Results reported thus far have been mixed: an analysis of 2013 cohort data by Wayne State University professor Thomas C. Pedroni found that the majority of EAA students failed to demonstrate progress toward proficiency on the state’s assessments in reading and math, and some students’ performance (approximately one-third) declined. Yet according to Excellent Schools Detroit, in its first year, six of EAA’s K–8 schools scored in the top 20 for growth out of 127 schools rated.

Detroit Needs a Plan 

Detroit is a powerful illustration of what happens when no one takes responsibility for the entire system of publicly supported schools in a city. Parents struggle to navigate their many, mostly low-performing options, and providers face at best weak incentives to improve academic quality. As a result, large numbers of failing district and charter schools continue to operate.

Still, hope is not lost. Many parent groups, nonprofits, and foundations in the city are working to step in where government has failed. Civic groups and leaders are helping parents learn what qualities to look for in schools, working to create high-quality schools in neighborhoods with the greatest concentrations of school-age children, asking schools to voluntarily agree to common enrollment deadlines and application processes, and putting pressure on charter school authorizers to close low-performing schools.

But to move forward, Detroit will need more than dedicated advocates and motivated parents. It will need strong civic leadership, a plan for investment and action, and creative problem solving. It will need to be strategic about what’s required to solve these complex problems, but also opportunistic about when and how they are solved. And if it’s to last, the plan needs to be owned and acted on by the community.

That plan will have to address negligent charter authorizers and persistently low-performing charter schools, and identify novel ways to build and attract high-quality school-management organizations. It will also have to come up with strategies for restructuring or replacing most of the schools run by the school district and the state-run EAA.

We heard from many thoughtful advocates and civic leaders in Detroit who are trying to develop creative solutions for renewing Detroit’s schools. Their ideas tended to coalesce around five strategies:

1. Develop a strong core of high-quality schools in the charter sector by working with the best charter authorizers to develop quality benchmarks and close low-performing charters in a targeted set of neighborhoods. Local leaders also told us that they believe the governor is the only official who has the needed credibility and authority to weigh in on negligent charter authorizers.

2. Leverage change from the bottom up by helping parents and communities to push authorizers and the district to increase performance accountability. Community groups such as the Detroit Parent Network, Excellent Schools Detroit, and the Skillman Foundation are leading efforts to inform parents about their options and how to identify a high-quality school.

3. Double down on recruiting talented school leaders and teachers to Detroit. There have been some investments in Teach for America and other talent-recruitment strategies, but many observers believe they need an even stronger focus on human capital to bolster nascent high-quality local school providers. While through 2011, Detroit’s school spending was on a par with similar  cities (see Figure 3), charter schools in the city and statewide have received considerably less funding per pupil than district schools. Equalized, student-based funding, many say, would help to attract high-quality charter providers. DPS is also challenged to attract talent thanks to a 10 percent salary concession the emergency manager has put in place as a result of the district’s extreme financial deficit.

4. Engage Detroit city leaders, like the mayor and local developers, in addressing safety, transit, and social-service support to help families and schools develop a strong choice infrastructure. These efforts should be leveraged along with other urban-renewal strategies in the city.

5. Recognize that DPS is at risk for financial collapse and develop a plan to replace DPS with a community “portfolio manager” board and superintendent who will see their role as overseeing a citywide system of high-quality schools rather than operating schools directly. This would likely mean sharing district facilities and special education services with charter schools, and coordinated information and enrollment systems.

Given that there seems to be little appetite from the state legislature and governor for legislative action on these fronts, much of these efforts have to be driven by local leaders. One strategy is for a group of charter authorizers, district leaders, and school and school association leaders to come together to take a stand for quality to build on the existing success stories in Detroit. A public statement followed by a series of activities to promote more high-quality schools could drive improvement from the ground up if state leaders continue to fail to act. The group could form a powerful lobby to rally needed state and federal investment and regulations.

Another solution is to create a Detroit-based nonprofit organization that has sufficient funding and authority to be the citywide coordinating body for all public school buildings, special services for families, transportation, enrollment, and parent information systems. The mayor and foundation leaders could help tremendously by investing in citywide safe transit routes for students and new solutions for choice-based special education and mental health services, as well as counselors and consultants to help families navigate the choice process.

There are no simple solutions for Detroit, but it is clear that no progress will be made until state and local leaders stop trying to defend their turf and start solving the very real problems that parents face. As one Detroit community leader opined,

We will need to centralize and coordinate school opening and closure decisions and focus strictly on achievement and what’s best for kids/neighborhoods, and stop worrying about what’s best for all the competing institutions…. If we could make school choice work for families again instead of for institutions by publicly supporting a common transportation, enrollment, scorecard, special education, and data system, I think school choice could be a positive force for school quality.

Detroit parents made it clear to us that they don’t care whether their child’s school is called a charter school, district, EAA, or private school. What they want and need is for some one to take responsibility for making sure that when their child heads to school each day, he or she will be safe, cared for, and well educated so that Detroit can rise again.

Robin Lake is director of the Center on Reinventing Public Education, where Ashley Jochim is research analyst and Michael DeArmond is senior research analyst. This article is adapted from Michael DeArmond, Ashley Jochim, and Robin Lake, “Making School Choice Work,” Center on Reinventing Public Education, 2014.

When I tell people that I spent my summer creating a curriculum aligned with the Common Core State Standards, I invariably get a quizzical look. In the often heated national debate over the Common Core, opponents have cast the standards as a threat to teacher autonomy and students’ intellectual creativity. The result is a public perception that there is very little wiggle room for teachers in choosing what to present in their classrooms. My experience as a lead lesson planner reveals that perception to be a false one.

During my summer planning, I kept the Common Core standards next to me while I dove deeply into the novels and nonfiction works we would be reading in 7th-grade English the next year. The texts themselves were chosen by the leadership of my charter school network, Uncommon Schools, with guidance from both the Common Core text-selection criteria and the network’s own curricular team The lesson plan sequence, questioning, activities, close reading passages, schema, and focuses were up to me and my co-teacher.

To teach works ranging from Julia Alvarez’s In the Time of the Butterflies about the dictatorship of Trujillo in the Dominican Republic to Shakespeare’s infamous tragedy Romeo and Juliet, we created literature units with supplemental nonfiction readings, as the Common Core standards suggest. We chose key vocabulary words from each work  and included discussions of broader concepts such as imperialism and internal oppression. We created lengthy writing assignments that asked students to compare and contrast nonfiction and fiction texts about the same topic, such as Julius Lester’s To Be a Slave and Walter Dean Myers’s The Glory Field. For the end of the year, we wrote an extensive sonnet unit, as the Common Core suggests for 7th-grade students, in which students analyze the impact a sonnet’s form has on its meaning.

The Common Core standards served as a helpful resource. The New York State Department of Education online resource EngageNY lays out the standards by subject and grade level and offers additional resources for educators and families. Along with the standards, my co-teacher and I looked at essay questions from the English literature Advanced Placement tests to see where students would need to be in four or five short years.

Once our lesson plans were finalized, all the grade-level teachers were asked to compose, using key vocabulary and concepts from each unit, “ideal student responses” to serve as measures of student comprehension based on participation in class discussions. Such tools ensure that students are not only being taught according to the Common Core standards, but that they are learning according to them, too.

When the curriculum was completed, I felt confident about the lessons we had created, but knew this meant nothing if they did not resonate with the students. When preparing to teach 7th graders about dramatic irony and iambic pentameter, a teacher will naturally wonder, will this be too hard for them? A teacher’s worst nightmare is to look out across a room to see the blank faces of students who are completely perplexed.

Happily, I found the answer to be no; it’s not too hard. For our final class session devoted to The Pearl by John Steinbeck, students were asked to evaluate Steinbeck’s characterization of Juana as weak. They first wrote their responses. Then “Daphne,” a student who often struggled in English class, raised her hand. Daphne explained how Steinbeck depicts Juana as physically weak because she doesn’t stand up to Kino’s violence, but mentally strong because she refuses to “submit” to the power of the pearl. She went on to explain that Steinbeck’s portrayal of Juana implies that she is stronger than Kino since the power of the pearl is what leads to his “destruction.” The sheer fact that Daphne described Steinbeck’s purpose with such precise vocabulary is, for me, proof that our students are more than ready for the challenge.

Moments like these by no means prove that the Common Core standards are perfect, nor do they account for other influences on students’ learning. But as a teacher, I have found the Common Core standards to be an instrumental guide for constructing lessons that will challenge and engage my students.

Lucy Boyd taught for three years at North Star Academy Vailsburg Middle School, an Uncommon School, and is now pursuing her master’s degree in public policy at the Harvard Kennedy School.

Blended: Using Disruptive Innovation to Improve Schools
By Michael B. Horn and Heather Staker
Jossey-Bass, 2014, $32.95; 336 pages.

As reviewed by Mark Bauerlein

When in the 1990s computers became big business for schools and technology providers, a few skeptics such as Todd Oppenheimer (author of The Flickering Mind) urged caution and thrift, marshaling evidence of hype and waste in the purchase and use of tools in the classroom. Enthusiasts of digital instruction usually ignored them and proceeded with statewide laptop programs, lessons based on blogs and wikis, and other novelties in the digital coming. They had the momentum of history on their side, plus parents who wanted the latest enhancements for their kids and politicians who saw in cutting-edge technology a political winner, so why bother with naysayers?

Wiser minds, however, took the skeptics more seriously and identified practical hindrances, such as inadequate coordination throughout a school. Recently, a more sophisticated explanation has emerged, one that casts those deficiencies not as signs of failure but as unavoidable steps in the course of reform. It derives from “disruptive innovation,” the concept in education at the present time. Among other things, the theory states that when a new technology or method comes along, it is always less productive and less economical than existing ones. In the early 19th century, for instance, steamships performed worse than sailing vessels, and their advantage appeared limited to lakes with no wind and rivers with strong currents. Steam power improved, though, until it surpassed wind power everywhere, and those invested in sail-based commerce who didn’t adapt disappeared.

The example typifies disruptive innovation, and it applies to digital learning as well. When a breakthrough arrives, it doesn’t evoke universal adoption. Initial uses are “down-market,” that is, geared to simple needs and impoverished settings, such as a computer in a classroom that lacks reference books. But as the innovation advances, it moves “upmarket,” its uses expanding to complex tasks in resource-rich milieus. Some pathways succeed, others falter. The proper approach is to correct and refine, to experiment and revise, develop the good and discard the bad.

Michael B. Horn and Heather Staker’s Blended: Using Disruptive Innovation to Improve Schools is a primer in this second phase of disruption. (Horn is co-author, with Clayton M. Christensen and Curtis W. Johnson, of Disrupting Class: How Disruptive Innovation Will Change the Way the World Learns, the cardinal expression of disruption theory in education; Staker is senior research fellow at the Christensen Institute.) The primary advent has already transpired, and as countless teachers and school officials across the country try different technologies and deliveries, a filtering process should follow. We no longer need debate the merits of digital vs. nondigital instruction: “Schools have no choice but to confront technology.” Instead, we must determine how best to integrate it, when and where and in what proportions.

The first problem, Horn and Staker write, stems from people “with an appetite for the dazzling technology” who end up merely “cramming more devices, screens, gadgets, and software into students’ and teachers’ already noisy lives.” Time and again, educators have tended to “get excited about a product” instead of defining specific educational and logistical goals they aim to meet. In fact, Horn and Staker report in their review of more than 150 programs across the states,

Nearly without exception those leading the most successful programs avoid the trap of “technology for technology’s sake” by beginning with a clearly articulated problem or goal that does not reference technology [emphasis in original].

The best programs, they continue, generally follow a hybrid practice, one that combines online with face-to-face instruction, and teacher control with student control over the learning process. Hence the title Blended, a term marking genuine integration of human and technological resources, not just the layering of one on the other.

In blended learning, some study takes place at home and some on a campus, be it a school or a learning center. Digital and traditional methods complement one another, for instance, students leaving a teacher-led classroom for an hour in the computer lab devoted to exercises that determine how well students assimilated the classroom content. Blending may happen in a technology-rich school or in one with few tools, as long as the tools are a significant part of instruction. All the advantages of personalization are pursued, too, as when students slow down or speed up an online presentation, but a critical mass of teacher supervision
is maintained.

In successive chapters, the authors distinguish the features of blended learning, highlight strong programs, and explain the keys to success. We have a taxonomy of models (“Enriched Virtual,” “Flex,” etc.), strategies (“Flipped Classroom,” “Individual Rotation,” etc.), and structures (“blended-learning teams” divided into “Functional,” “Lightweight,” “Heavyweight,” and “Autonomous”). Tactical questions are posed—“Should leaders focus on sustaining or disruptive rally cries?”—and diagrams provided (one on “Station Rotation” with “Teacher-led instruction,” “Collaborative activities and stations,” and “Online instruction” components). The authors give advice to school leaders, such as urging them to adopt digital programs first for “nonconsumption opportunities,” that is, for activities a school cannot otherwise provide (good results may convince everyone that they may benefit existing activities as well). One chapter ranges from large issues such as the shift of school architecture to modular spaces to small instructions such as ‘use one outside provider’ per course or subject.

As you can tell, Blended is a book for school officials in the process of change. They feel the pressures of digital innovation, uncertain funding, and stalled student performance, but they haven’t the techno-knowledge and managerial expertise to proceed sagely. Horn and Staker guide them forward—Chapter 10 bears the title “Discover Your Way to Success”—outlining how to create the right team, match relevant models to different student populations, and arrange proper physical spaces for various programs. Troubling questions such as those raised when students take control of their own learning go unaddressed, and at times their version of “factory-model” schools slides into caricature (“Top-down, teacher-centered, monolithic instruction is an uninspired match for generating the entrepreneurial, inquisitive problem solvers that today’s employers are paying top dollar to recruit”). But in the authors’ view, the current phase of disruption calls for implementation, not deeper reflection. Let’s get on with the process, they imply, and if we find that teachers aren’t blending the way we expected or students aren’t motivated as we predicted, let’s self-correct and try again. That’s what shrewd disrupters do.

Mark Bauerlein is professor of English at Emory University.

It is widely understood that there are vast differences in the quality of teachers: we’ve all had really good, really bad, and decidedly mediocre ones. Until recently, teachers were deemed qualified, and were compensated, solely according to academic credentials and years of experience. Classroom performance was not considered. In the last decade, researchers have used student achievement data to quantify teacher performance and thereby measure differences in teacher quality. Among the recent findings is evidence that having a better teacher not only has a substantial impact on students’ test scores at the end of the school year, but also increases their chances of attending college and their earnings as adults (see “Great Teaching,” research, Summer 2012).

In response to these findings, federal policy goals have shifted from ensuring that all teachers have traditional credentials and are fully certified to creating incentives for states to evaluate and retain teachers based on their classroom performance. We contribute to the body of knowledge on teacher evaluation systems by examining the actual design and performance of new teacher-evaluation systems in four school districts that are at the forefront of the effort to evaluate teachers meaningfully.

We find, first, that the ratings assigned teachers by the districts’ evaluation systems are sufficiently predictive of a teacher’s future performance to be used by administrators for high-stakes decisions. While evaluation systems that make use of student test scores, such as value-added methods, have been the focus of much recent debate, only a small fraction of teachers, just one-fifth in our four study districts, can be evaluated based on gains in their students’ test scores. The other four-fifths of teachers, who are responsible for classes not covered by standardized tests, have to be evaluated some other way, including, in our districts, by basing the teacher’s evaluation score on classroom observations, achievement test gains for the whole school, performance on nonstandardized tests chosen and administered by each teacher to her own students, and by some form of “team spirit” rating handed out by administrators. In the four districts in our study, classroom observations carry the bulk of the weight, comprising between 50 and 75 percent of the overall evaluation scores for teachers in non-tested grades and subjects.

As a result, most of the action and nearly all the opportunities for improving teacher evaluations lie in the area of classroom observations rather than in test-score gains. Based on our analysis of system design and practices in our four study districts, we make the following recommendations:

1) Teacher evaluations should include two to three annual classroom observations, with at least one of those observations being conducted by a trained observer from outside the teacher’s school.

2) Classroom observations that make meaningful distinctions among teachers should carry at least as much weight as test-score gains in determining a teacher’s overall evaluation score when both are available.

3) Most important, districts should adjust teachers’ classroom-observation scores for the background characteristics of their students, a factor that can have a substantial and unfair influence on a teacher’s evaluation rating. Considerable technical attention has been given to wringing the bias out of value-added scores that arises because student ability is not evenly distributed across classrooms (see “Choosing the Right Growth Measure,” research, Spring 2014). Similar attention has not been paid to the impact of student background characteristics on classroom-observation scores.

Observations vs. Value-Added

The four urban districts we study are scattered across the country. Their enrollments range from about 25,000 to 110,000 students, and the number of schools ranges from roughly 70 to 220. We have from one to three years of individual-level data on students and teachers, provided to us by the districts and drawn from one or more of the years from 2009 to 2012. We begin our analysis by examining the extent to which the overall ratings assigned to teachers by the districts’ evaluation systems are predictive of the teacher’s ability to raise test scores and the extent to which they are stable from one year to the next. The former analysis can be conducted only for the subset of teachers with value-added ratings, that is, teachers in tested grades and subjects. In contrast, we can examine the stability of overall ratings for all teachers included in the districts’ evaluation systems.

We find that the overall evaluation scores in one year are correlated with the same teachers’ value-added scores in an adjacent year at levels ranging from 0.33 to 0.38. In other words, teacher-evaluation scores based on a number of components, including teacher- and school-level value-added scores, classroom-observation scores, and other student and administrator ratings, are quite predictive of a teacher’s ability to raise student test scores the following (or previous) year. The year-to-year correlation is in keeping with the findings from prior research on value-added measures when used on their own as a teacher-performance metric. The degree of correlation confirms that these systems perform substantially better in predicting future teacher performance than traditional systems based on paper credentials and years of experience. These correlations are also in the range that is typical of systems for evaluating and predicting future performance in other fields of human endeavor, including, for example, those used to make management decisions on player contracts in professional sports.

We calculate the year-to-year stability of the evaluation scores as the correlation between the overall scores of the same teachers in adjacent years. The stability generated by the districts’ evaluation systems ranges from a bit more than 0.50 for teachers with value-added scores to about 0.65 when value-added is not a component of the score. Evaluation scores that do not include value-added are more stable because they assign more weight to observation scores, which are more stable over time than value-added scores.

Why are observation scores more stable? The difference may be due, in part, to observations typically being conducted by school administrators who have preconceived ideas about a teacher’s effectiveness. If a principal is positively disposed toward a particular teacher because of prior knowledge, the teacher may receive a higher observation score than the teacher would have received if the principal were unfamiliar with her or had a prior negative disposition. If the administrator’s impression of individual teachers is relatively sticky from year to year, then it will be less reflective of true teacher performance as observed at a particular point of time. For this reason, maximizing stability may not increase the effectiveness of the evaluation system.

This leaves districts with important decisions to make regarding the tradeoff between the weights they assign to value-added versus observational components for teachers in tested grades and subjects. Our data show that there is a tradeoff between predicting observation scores and predicting value-added scores of teachers in a subsequent year. Figure 1 plots the ability of an overall evaluation score, computed based on a continuum of different weighting schemes, to predict teachers’ observation and value-added scores in the following year. The optimal ratio of weights to maximize predictive power for value-added in the next year is about two to one (value-added to observations), whereas maximizing the ability to predict observations requires putting the vast majority of weight on observations.

We do not believe there is an empirical solution for the ideal weights to assign to observation versus value-added scores. The assignment of those weights depends on the a priori value the district assigns to raising student test scores, the confidence it has in its classroom-observation system as a tool for both evaluation and professional development, and the political and practical realities it faces in negotiating and implementing a teacher-evaluation system.

At the same time, there are ranges of relative weighting—namely between 50 and 100 percent value-added—where significant increases in the ability to predict observation scores can be obtained by increasing the weight assigned to observations with relatively little decrease in the ability to predict value-added. Consequently, most districts considering only these two measures should assign a weight on observations of at least 50 percent.

Classroom Observation

The structure, frequency, and quality of the classroom observation component are also important. The observation system in place should make meaningful distinctions among teachers. An observation system that provides only two choices, satisfactory and unsatisfactory, for example, will result in similar ratings being given to most teachers. As a result, the observation component will not carry much weight in the overall evaluation score, regardless of how much weight is officially assigned to it. An evaluation system with little variation in observation scores would also make it very difficult for teachers in nontested grades and subjects to obtain very high or low total evaluation scores; they would all tend to end up in the middle of the pack, relative to teachers for whom value-added scores are available.

In all of our study districts, the quality of information garnered from classroom observations depends on how many are conducted. Figure 2 shows that moving from one to two observations (independent of who conducts them) increases both the stability of observation scores and their predictive power for value-added scores in the next year. Adding additional observations continues to increase the stability of observation scores but has no further effect on their predictive power for future value-added scores.

In districts that use a mix of building leaders and central administration staff to conduct classroom observations, the quality of the information also depends on who conducts the observations. Observations conducted by in-building administrators, e.g., the principal, are more stable (0.61) than those done by central administration staff (0.49), but observations conducted by evaluators from outside the building have higher predictive power for value-added scores in the next year (0.21) than those done by administrators in the building (0.15). The higher year-to-year stability of observations conducted by the principal or assistant principal compared to out-of-building observers is consistent with our hypothesis that a principal’s observation is influenced by both her preexisting opinion about a given teacher and the information that is derived from the classroom observation itself.

Classroom observations are expensive, and, for the majority of teachers, they are the most heavily weighted contributor to their individual evaluation score. Observations also have a critical role to play for principals who intend to be instructional leaders, as they present a primary point of contact between the school leader and classroom teaching and learning. It is important to balance what are, in part, the competing demands of empowering the school leader to lead, spending no more than necessary in staff time and money to achieve an effective observation system, and ensuring that observation scores are based on what is observed rather than on extraneous knowledge and prior relationships between the observer and the teacher.

Our data suggest that three observations provide about as much value to administrators as five. We recommend that districts conduct two to three annual classroom observations for each teacher, with at least one of those being conducted by a trained observer from outside the teacher’s school without substantial prior knowledge of, or conflict of interest with respect to, the teacher being observed. Districts should arrange for an additional classroom observation by another independent observer in cases in which there are substantial and potentially consequential differences between the observation scores generated by the primary observers.

Bias in Observation Scores

A teacher-evaluation system would clearly be intolerable if it identified teachers in the gifted and talented program as superior to other teachers because students in the gifted and talented program got higher scores on end-of-year tests. Value-added metrics mitigate this bias by measuring test-score gains from one school year to the next, rather than absolute scores at the end of the year, and by including statistical controls for characteristics of students and classrooms that are known to be associated with student test scores, such as students’ eligibility for free or reduced-price lunch.

But as noted above, classroom observations, not test-score gains, are the major factor in the evaluation scores of most teachers in the districts we examined, ranging from 40 to 75 percent of the total score, depending on the district and whether the teacher is responsible for a classroom in a tested grade and subject. Neither our four districts, nor others of which we are aware, have processes in place to address the possible biases in observation scores that arise from some teachers being assigned a more-able group of students than other teachers.

Imagine a teacher who, through the luck of the draw or administrative decision, gets an above-average share of students who are challenging to teach because they are less well prepared academically, aren’t fluent in English, or have behavioral problems. Now think about what a classroom observer is asked to judge when rating a teacher’s ability. For example, in a widely used classroom-observation system created by Charlotte Danielson, a rating of “distinguished” on questioning and discussion techniques requires the teacher’s questions to consistently provide high cognitive challenge with adequate time for students to respond, and requires that students formulate many questions during discussion. Intuitively, the teacher with a greater share of students who are challenging to teach is going to have a tougher time performing well under this rubric than the teacher in the gifted and talented classroom.

This intuition is borne out in our data: teachers with students with higher incoming achievement levels receive classroom- observation scores that are higher on average than those received by teachers whose incoming students are at lower achievement levels. This finding holds when comparing the observation scores of the same teacher with different classes of students. The latter finding is important because it indicates that the association between student incoming achievement levels and teacher-observation scores is not due, primarily, to better teachers being assigned better students. Rather, it is consistent with bias in the observation system; when observers see a teacher leading a class with higher-ability students, they judge the teacher to be better than when they see that same teacher leading a class of lower-ability students.

Figure 3 depicts this relationship using data from teachers in tested grades and subjects for whom it is possible to examine the association between the achievement levels of students that teachers are assigned and the teachers’ classroom-observation scores. Notice that only about 9 percent of teachers assigned a classroom of students who are “lowest achieving” (in the lowest fifth of academic performance based on their incoming test scores) are identified as top-performing based on classroom observations, whereas the expected outcome would be 20 percent if there were no association between students’ incoming ability and a teacher’s observation score. In contrast, four times as many teachers (37 percent) whose incoming students are “highest achieving” (in the top fifth of achievement based on incoming test scores) are identified as top performers according to classroom observations.

This represents a serious problem for any teacher-evaluation system that places a heavy emphasis on classroom observations, as nearly all current systems are forced to do because of the lack of measures of student learning in most grades and subjects. Fortunately, there is a straightforward fix to this problem: adjust teacher-observation scores based on student background characteristics, which, unlike prior test scores, are available for all teachers. We implement this adjustment using a regression analysis that calculates each teacher’s observation score relative to her predicted score based on the composition of her class, measured as the percentages of students who are white, black, Hispanic, special education, eligible for free or reduced-price lunch, English language learners, and male. These background variables are all associated with entering achievement levels, but we do not adjust for prior test scores directly because doing so is only possible for the minority of teachers in tested grades and subjects.

Such an adjustment for the makeup of the class is already factored in for teachers for whom value-added is calculated, because student gains are adjusted for background characteristics. But the adjustment is only applied to the value-added portion of the evaluation score. For the teachers in nontested grades and subjects, whose overall evaluation score does not include value-added data, there is no adjustment for the makeup of their class in any portion of their evaluation score.

When classroom-observation scores are adjusted for student background characteristics, the pattern of observation scores is much less strongly related to the incoming achievement level of students than is the case when raw classroom-observations scores are used. A statistical association remains between incoming student achievement test scores and teacher ratings based on classroom observations, but it is reduced substantially.

States have an important role to play in helping local districts make these statistical adjustments. In small districts, small numbers of students and teachers will make these kinds of adjustments very imprecise. We estimate the magnitude of this issue by creating simulated small districts from the data on our relatively large districts, and find that the number of observations in the adjustment model can have a large impact on the stability of the resulting evaluation measures.

The solution to this problem is also straightforward. States should conduct the statistical analysis used to make adjustments using data from the entire state, or subgroups of demographically similar districts, and provide the information necessary to calculate adjusted observation scores back to the individual districts. The small number of states that already have evaluation systems in place address this issue by calculating value-added scores centrally and providing them to local school districts. This should remain the norm and be expanded to include observation scores, given the important role they play in the evaluations of all teachers.

Conclusions

A new generation of teacher-evaluation systems seeks to make performance measurement and feedback more rigorous and useful. These systems incorporate multiple sources of information, including such metrics as systematic classroom observations, student and parent surveys, measures of professionalism and commitment to the school community, more differentiated principal ratings, and test-score gains for students in each teacher’s classrooms.

Although much of the impetus for new approaches to teacher evaluation comes from policymakers at the state and national levels, the design of any particular teacher-evaluation system in most states falls to individual school districts and charter schools. Because of the immaturity of the knowledge base on the design of teacher-evaluation systems, and the local politics of school management, we are likely to see considerable variability among school districts in how they go about evaluating teachers.

That variability is a double-edged sword. It offers the opportunity to study and learn from natural variation in the design of evaluation systems, but it also threatens to undermine public support for new teacher-evaluation systems to the extent that the natural variation suggests chaos, and is used by opponents of systematic teacher evaluation to highlight the failures of the worst-performing systems. The way out of this conundrum is to accelerate the process by which we learn from the initial round of district experiences and to create leverage points around that learning that will lift up the weakest evaluation systems.

Our examination of the design and performance of the teacher-evaluation systems in four districts provides reasons for optimism that new, meaningful evaluation systems can be designed and implemented by individual districts. At the same time, we find that our districts share, to one degree or another, design decisions that limit their systems’ performance, and that will probably be seen as mistakes by stakeholders as more experience with the systems accrues. We focus our recommendations on improving the quality of data derived from classroom observations. Our most important recommendation is that districts adjust classroom observation scores for the degree to which the students assigned to a teacher create challenging conditions for the teacher. Put simply, the current observation systems are patently unfair to teachers who are assigned less-able and -prepared students. The result is an unintended but strong incentive for good teachers to avoid teaching low-performing students and to avoid teaching in low-performing schools.

A prime motive behind the move toward meaningful teacher evaluation is to assure greater equity in students’ access to good teachers. A teacher-evaluation system design that inadvertently pushes in the opposite direction is clearly undesirable. We have demonstrated that these design errors can be corrected with tools in hand.

Grover J. “Russ” Whitehurst is director of the Brown Center on Education Policy at the Brookings Institution, where Matthew M. Chingos is a senior fellow, and Katharine M. Lindquist is a research analyst.

The media and observers across the ideological spectrum were surprised and, in some cases, disconcerted in July 2014 when at the annual American Federation of Teachers (AFT) convention in Los Angeles, the union’s leadership team announced that its Innovation Fund grants of $20,000 to $30,000 were going to be made available to state and local affiliates to critique the Common Core State Standards (CCSS), the massive multistate effort to improve student achievement.

“It’s a sign that teachers are frustrated and fed up—and they’re making their anger heard, loud and clear,” opined a July Politico story about the new initiative.

“This is a huge step because this time last year, they were gung-ho for Common Core,” said Fordham University’s Mark Naison, a critic of the standards, also in Politico.

David Menefee-Libey, a political scientist at Pomona College, went even further: “It’s all blowing up.”

The AFT’s announcement, combined with the much-publicized rise of Chicago’s Karen Lewis and the election of Barbara Madeloni as president of the Massachusetts Teachers Association, as well as the emergence of the Badass Teachers Alliance and other social-justice factions within the unions, have added to the impression that union opposition to the standards is large—and growing. And indeed, there’s little argument that the unions’ rhetoric and tone have changed.

But have the AFT and its larger counterpart, the National Education Association (NEA), really turned their backs on the Common Core in concrete, substantive ways—and if so, how much does it matter?

A Mixed Message

The unions have no direct authority over the Common Core implementation or the assessments. They’re not directly responsible for preparing teachers to use the new standards or for administering the new tests. They have no formal governance role in the process.

Union pronouncements about the success or failure of the process, however, and unions’ work with states and districts and outside partners on the standards, do influence the materials and supports that are being provided to teachers, and also help shape media and public perceptions of the initiative, and in theory could shape lawmakers’ positions on whether to continue, pause, or reengineer the effort.

Concern that the AFT, and to some degree the NEA, was flip-flopping on the Common Core, which could encourage classroom teachers’ resistance to the changes and endanger the effort’s ultimate success, has become a common one among standards supporters and union critics.

“It seemed like they signed on to do this [Common Core development] three years ago, banging the door down saying they needed to be part of it, and then little by little they’ve peeled off,” says Democrats for Education Reform’s Charlie Barone.

According to this line of reasoning, the unions expressed their support for the standards during the early stages, when they were being developed and then adopted by states during competition for Race to the Top funds. It was only when the development of assessments began, and the U.S. Department of Education’s (ED’s) No Child Left Behind waiver process included clear requirements for evaluating teachers based partly on student test scores, that the unions began to balk.

Not everyone goes that far. “They’re trying to walk a fine line in which they still support the standards but don’t like the way they’ve been implemented,” says Bob Rothman, a Common Core supporter at the Alliance for Excellent Education. “But they haven’t reversed themselves” (see “The Common Core Takes Hold,” features, Summer 2014).

To say that the unions had flip-flopped on the Common Core “would be an absolute mischaracterization,” insists Sandra Alberti, field director for the nonprofit Student Achievement Partners (SAP), who has been working with union leadership on implementation.

“The local affiliates doing this work and teachers have remained really committed, and both unions have been committed to providing resources to members despite some of the political controversy,” attests Gates Foundation staffer Lynn Olson, who works with the unions on their Common Core endeavors.

Common Core critics within the AFT aren’t buying the notion that the AFT has reversed itself, either. The AFT-passed Common Core resolution “tries to have it both ways,” according to the Chicago Teachers Union’s (CTU) Jennifer Johnson. She says the new AFT grant program accepts on its face that the standards were a positive step and only encourages minor changes.

“That’s not the right focus at all,” according to Johnson, who believes that Chicago teachers would find their time and energy better spent on working to stop the standards rather than tweaking the standards already in place.

As the controversy grows, it becomes much less clear what union leadership should do. There are dangers on both sides: rebellion from the rank and file on one, and political marginalization on the other.

“If the standards go down the tubes because of fear-mongering and misinformation, the NEA is going to look really bad,” one union official explained to Education Week. “Why would anyone take us seriously if we had a seat at the table, and then we turned our backs on the standards?”

There is also the very real danger of confusing teachers. “Most people don’t live and breathe the nuances of Common Core, so I’m not sure where they are in terms of understanding the AFT position,” admits Marla Ucelli-Kashyap, director of AFT’s Educational Issues Department. “But our support is still on the books. It’s “not a yes/no kind of issue.”

The Backstory

The Common Core State Standards were officially launched in June 2009. Drafts were released in March 2010 and finalized in June of the same year. By August 2010, 33 states and Washington, D.C., had adopted them. Two years later, 12 additional states had joined. Teams of teachers from both the NEA and AFT reviewed the standards at key points and signed letters of support along the way. Union involvement in the Common Core process continued in the following months and years, largely through partnerships with outside funders and nonprofits.

The standards developers at SAP have met with union representatives like the NEA’s director of education policy and practice Donna Harris-Aikens and AFT’s Ucelli-Kashyap every couple of months, according to Alberti, to share activities, get feedback, and take a look at what needed to be done next.

The Gates Foundation grant for the NEA’s Master Teacher project, which has thus far supported 95 teachers developing Common Core lessons, included the creation of 24,000 resources or 3,500 lessons, delivered through Better Lesson, according to the foundation. AFT locals, including those in Albuquerque, Boston, Chicago, Cleveland, and Jefferson County (Alabama), received Gates-funded grants to produce Common Core lessons, some of which are available through Share My Lesson.

The unions have worked closely with SAP, which received an $18 million grant from the GE Foundation to create “immersion institutes” to familiarize teachers with the standards and to create a storehouse of materials for them to use in their instruction. The two national unions and SAP were part of a group that received an $11 million three-year grant from the Helmsley Charitable Trust in 2012, which closes out in 2016. Helmsley also provided the two unions and the two assessment consortia, Partnership for Assessment of Readiness for College and Careers (PARCC) and the Smarter Balanced Assessment Consortium (SBAC), a $1.6 million grant in 2013, specifically aimed at engaging teachers in the development and implementation of end-of-year assessments.

Nor has the unions’ engagement with the Common Core been limited to initiatives with outside funding. Noting the AFT’s focus on the standards in its education journal American Educator and the NEA’s $60 million investment in teacher training projects, a February 2014 Education Week article described the two national unions as “among the initiative’s biggest boosters.” Dubbed the Great Public Schools initiative, the NEA projects were funded mostly with union dues rather than outside grants, and focused primarily on helping teachers prepare for the Common Core.

To be sure, the unions and others expressed concerns about the Common Core along the way. The AFT produced a report in 2011 that outlined an action plan and issued recommendations for what the union and others needed to do. The report expressed the AFT’s apprehensions about the design, content, and potential for punitive use of assessments and vowed to ensure that educators continue to have a “significant voice” in implementation of the standards.

Such concerns spread as states began to implement the standards with varying success. In some places—states that won Race to the Top funding in particular—there was time and there were resources to familiarize educators with the new standards and prepare materials to go along with them. In others, things were hurried and incomplete. There wasn’t a new curriculum to go along with the new standards, or it wasn’t very good, or there wasn’t any time to become familiar with it and how to teach with it. In some places, the new standards were in place, but with the old curricula and tests. In others—New York, especially—the new tests were being tried out even before the new curricula were fully in place.

The Pushback

In April 2013, AFT president Randi Weingarten floated the idea of a moratorium on high-stakes uses of the results from the new assessments, and the idea quickly gained steam. Eventually, those who supported some version of a slowdown included the accountability hawks at the Education Trust and the Gates Foundation. The U.S. Department of Education began giving out waivers to states allowing them to push back high-stakes aspects of the new system, and a number of other states took steps on their own to delay full implementation (see Figure 1).

In July 2013, the NEA affirmed its support for the Common Core standards by a voice vote but it did not endorse the tests.

Then NEA state leaders met in Washington in January 2014 to consider how best to proceed. In February, the NEA issued its first formal statement expressing concern about the implementation process.

Serious questions about the unions’ support for the CCSS became even more public in March 2014 at the Council of Chief State School Officers’ annual legislative conference, where AFT and NEA leaders “squabbled” with state education chiefs over public perception and implementation issues, according to an Education Week story. At the event, Weingarten and then NEA president Dennis Van Roekel critiqued various aspects of the implementation process, which led Massachusetts K–12 chief Mitchell D. Chester to remark that the leaders were “condoning” opposition to the standards, creating a “totally adversarial conversation.”

Things soon got worse. In May, the CTU passed a tough resolution opposing the Common Core in its entirety, which it followed with calls for parents to boycott tests; a handful of teachers refused to administer the tests (see sidebar, opposite page).

In June, the New York state legislature went along with a union request and determined that teachers wouldn’t be penalized for low student-achievement scores. The union in New York also encouraged districts and parents to boycott the 2014–15 field tests of the new assessments.

Under pressure from members, the AFT stopped taking new money from the Gates Foundation for its Innovation Fund, replacing at least some of the lost funding with member dues and grants from less objectionable funders.

A Fine Line

Some Common Core advocates consider union criticism of the new tests, which replace multiple-choice, fill-in-the-bubble exams that reflect low expectations, to be deeply hypocritical. Opposing the assessment undermines the credibility of teachers unions, argued The New Teacher Project (TNTP) president Tim Daly in Education Week in February 2014.

The unions themselves see little or no conflict between their support for the standards and their concerns about their implementation and use. According to their leaders, the unions were and still are in favor of high, uniform standards for all students. It’s just the implementation of the standards, the new tests developed to measure student achievement, and the planned uses of the results that are problematic.

“We’ve been very supportive and involved in development of the standards,” says Van Roekel in a recent interview. “Then it came time to implement those. We’re still in that phase. Adoption was an event; the implementation is far more of a process.”

And even if the case could be made inarguably that this represents a reversal, unions wouldn’t be the only ones who have flip-flopped on the Common Core. Republican presidential aspirants like Bobby Jindal have taken reversal to an extreme.

At least some of the confusion and concern has come from the fact that the AFT’s Weingarten is a prolific and somewhat peripatetic speaker who is notoriously hard to pin down (or inconsistent in her positions, depending on your point of view). Her position on issues such as measuring teachers using student test scores has changed over the years. Once supportive of the approach, she later called it “a sham.”

Behind the Scenes

Through it all, both unions have continued working on implementation and continued to collaborate with test developers SBAC and PARCC, which received additional funding from ED to help schools get ready for the tests that they were developing and field testing.

SAP’s Alberti insists that teachers and the unions have supported the standards despite reservations about testing and how agencies were rolling them out. “The strength of the partnership has remained fairly steady,” she says.

And if one takes a close look at the resolutions debated and passed at the 2014 AFT convention, there really was no “dizzying about-face on the Common Core,” as described by Politico. The AFT voted down a resolution calling for a full-throated rejection of the Common Core, put forth by the CTU. Proposals to limit the union’s ability to participate in or even fund Common Core–related activities were dropped. The AFT was careful not to say anything against the standards themselves, and spent time before and after the convention making sure that its external partners, funders and nonprofits, knew the nuances behind its statements.

“I think that most people got it,” says the AFT’s Ucelli-Kashyap, though she admits that in some cases outside partners wanted reassurance. “They asked, ‘Here’s how I’m reading this. Is that right, or should we be concerned?’”

Less Action Than Rhetoric

How to tell what the union’s real position is, when the rhetoric is so complicated? One way is to look at the unions’ direct actions. For all the concern and caution that has crept into union leaders’ rhetoric, neither one has joined conservative Republicans’ efforts to roll back the standards in state legislatures around the country.

“In those places where we saw pushback, the teachers unions did not seem to play a large role,” says one insider who did not want to be named. “Where they did make their voices known, they were not supporting bills to have the standards taken away but rather working on issues they cared most about, implementation or evaluation.”

An exception might be Tennessee, which dropped out of PARCC, at least temporarily, in June. “The people who actually have the power to block the Common Core and are exercising that power come from the right,” says Rothman. While they may have criticized the standards, the implementation process, or the testing timeline, unions in Illinois and New York “have not done anything to stop implementation of the standards.”

The most aggressive resistance to the Common Core isn’t taking the form of direct action against the process but is coming from “social justice” liberals and progressives, who may be union members but are not by and large in charge of state or local affiliates. These small but vocal factions within the unions have exerted as much pressure as possible against the standards process, which they see as unwise and destructive. But their impact thus far has been limited.

Union leaders like Weingarten may have picked up some of the rhetoric of these advocates’ views and given them time and space to be heard, but thus far at least do not seem to have adopted their views wholesale.

Coming out against the testing timeline while continuing to work on implementation is a way for a union to “divert the anger and frustration” of these activists and possible challengers, according to union watchdog Mike Antonucci, without having to reverse itself entirely.

“The problem is,” explains Antonucci, “as with any education reform you come up with, when it’s instituted there’s a lot of pushback, and it is coming from people in the union, and so union leaders find themselves on the opposite side of a lot of their members” (see “Teachers Unions and the War Within,” features, Winter 2015).

What Happens Now?

The current course may well continue: union officials express concern while continuing to work on assessments and implementation behind the scenes.

Under this scenario, the unions will continue to take foundation funding and spend down whatever remaining Gates Foundation resources that have already been received. (As of July, the Gates funding had still not all been spent.)

At least some of those who’ve been involved with the implementation process believe that things will continue this way. “I looked through the new grant opportunities language last night,” says SAP’s Alberti, referring to the revamped AFT Innovation Fund criteria. “The language is all about moving the standards forward.”

Alberti also notes that incoming NEA president Lily Eskelsen García is a staunch standards supporter, not a “shut it down” kind of president like some social-justice members would probably have liked (see “Teacher of the Year to Union President,” features, Summer 2014).

Another possibility is that the standards and assessments continue to rankle teachers and that pushes the unions into taking tougher positions and actions than they have thus far. According to the 2014 Education Next poll, teachers’ support for the Common Core slipped from 76 percent to 46 percent in just a year (see “No Common Opinion on the Common Core,” features, Winter 2015). An increase in negative publicity from states already using the new assessments (like New York), or (most likely) their expanded use this spring, could prompt union leaders to change their views, as well.

“Their position has to get stronger as accountability and assessment triggers take place,” notes Alberti. That’s not all bad, she says. “We need them to become increasingly strong about demanding the resources that their teachers need to do the work well. I don’t think it’s about not doing the work. I don’t think that’s a threat to the effort.”

“The biggest threat to the Common Core is not that states will pull out” under union pressures, argues Rothman. “The biggest threat is states that stay in but don’t do much to implement the standards.”

“What happens next?” responds Van Roekel. “I don’t think the process will be reversed. Too many states have gone too far, many with positive experiences, so they’ll correct any flaws rather than start over. Are you going to start a whole new process? It took three to five years to get this far.”

“The unions so far, when they have stood up, it’s been for the Common Core,” explains another pro-standards insider, “but they haven’t stood up for the assessments.”

And where the timelines on implementation and uses of assessments have been relaxed, they haven’t yet played a more constructive, supportive role, either.

Alexander Russo is a freelance education writer who edits two blogs (“This Week In Education” and “District 299:
The Inside Scoop on CPS”) and tweets from @alexanderrusso.

The conventional wisdom is that the news business is in a death spiral, taking with it serious reporting about all manner of public policy issues. But all hope is not lost, for there’s been a recent, and in my view, surprising revival of education reporting, a resurrection driven by a new breed of journalism (see Figure 1).

Some of these new outlets produce so much education content—much of it quite wonky—that they should now be considered part of the trade press. That’s certainly the case for both Politico Pro and Chalkbeat. Politico Pro, which launched its education coverage in the summer of 2013, pumps out loads of ministories, and at least a handful of meaty ones, almost every day, plus its “Morning Education” newsletter, which lands in the in-boxes of the powerful and influential. Chalkbeat, the result of a merger between New York City’s GothamSchools and EdNews Colorado (with outlets in Indiana and Tennessee, too), is a grant-funded entity committed to doing regular, in-depth reporting in a given location—a geographically based Education Week, if you will. The Hechinger Report and StateImpact are cousins of these news outlets. Hechinger places national, long-form stories in prominent publications across the country; StateImpact taps into the infrastructure of National Public Radio affiliates in Florida, Indiana, and Ohio to go deep in their home states (on education and other issues).

Then there are the more ideological outlets: Huffington Post on the left and the Daily Caller on the right. The Daily Caller, in particular, has come on strong of late; among Washington-based media outlets, only the Washington Post gets more readers. The Daily Caller’s education coverage is hyperkinetic (writers are rumored to be paid based on clicks), not to mention schizophrenic, veering between hot policy issues (mostly Common Core) and the schoolhouse scandal of the day (“Drunk Teacher Pulls Knife on Students, Demands Fast-Food”).

The other entrants—BuzzFeed, FiveThirtyEight, and Vox—are just getting started with their education coverage. BuzzFeed has a focus on the “business of education” and to date its articles center mostly on higher education. FiveThirtyEight, created by über–data geek Nate Silver, shows a lot of potential but has only produced two education stories so far (understandable since it doesn’t have a dedicated education reporter). Of the three, Vox has shown the most interest in K–12 happenings, with a mix of the serious (“How Racial Inequality in Education Persists 60 Years after Brown v. Board of Education”) and the silly (“Why a Florida Lawmaker Thinks Common Core Tests Will Make Kids Gay”).

Whether this education-reporting revival is a renaissance or regrettable certainly depends on your point of view. It’s easy to look down one’s nose at the short, “shareable,” and sometimes salacious coverage of some of these upstarts. Then again, some of the best in-depth coverage, and the most knowledgeable writers, call these outlets home, too. And it’s rarely smart to bet against Nate Silver.

Michael J. Petrilli is president of the Thomas B. Fordham Institute.

As schools narrow their focus on improving performance on math and reading standardized tests, they have greater difficulty justifying taking students out of the classroom for experiences that are not related to improving those test scores. Schools are either attending fewer field trips or shifting toward field trips to places they know students already enjoy. When testing is over, schools are often inclined to take students on “reward” field trips to places like amusement parks, bowling alleys, and movie theaters.

The nature of culturally enriching field trips is that they are often to places that students don’t yet know they might enjoy. In a previous study, we examined the impact of field trips to an art museum. We found significant benefits in the form of knowledge, future cultural consumption, tolerance, historical empathy, and critical thinking for students assigned by lottery to visit Crystal Bridges Museum of American Art (see “The Educational Value of Field Trips,” research, Winter 2014). In the current study, we examine the impact of assigning student groups by lottery to see high-quality theater productions of Hamlet or A Christmas Carol. This is the first randomized experiment to discover what students get out of seeing live theater. Our results are generally similar to those found in the previous study. Culturally enriching field trips have significant educational benefits for students whether they are to see an art museum or live theater. Among students assigned by lottery to see live theater, we find enhanced knowledge of the plot and vocabulary in those plays, greater tolerance, and improved ability to read the emotions of others.

Our goal in pursuing research on the effects of culturally enriching field trips is to broaden the types of measures
that education researchers, and in turn policymakers and practitioners, consider when judging the educational success or failure of schools. It requires significantly greater effort to collect new measures than to rely solely on state-provided math and reading tests, but we believe that this effort is worthwhile. By broadening the measures used to assess educational outcomes, we can also learn what role, if any, cultural institutions may play in producing those outcomes.

Research Design

The opportunity to study the effects on students of seeing live theater arose as part of a collaboration with TheatreSquared, an award-winning professional theater in Fayetteville, Arkansas. TheatreSquared agreed to add matinee performances of A Christmas Carol and Hamlet, and school groups in grades 7 through 12 were offered the opportunity to receive free tickets to one of those performances. A total of 49 school groups, with 670 students, completed the application process and participated in the study. Twenty-four of those groups applied to see A Christmas Carol, and 25 applied to see Hamlet. Applicants were organized into 24 matched groupings based on their similarity in terms of grade level, demographics, and whether they comprised a drama, English, or some other type of class. Lotteries were held within each of those matched groupings to determine which groups would receive the free tickets to see a play and which would serve as the control group. A total of 22 school groups attended one of the performances, and 27 were in the control group. As one would hope from a lottery-based research design, the resulting treatment- and control-group students are generally alike in terms of gender, race, and the grade in which they are enrolled.

During the winter and spring of the 2013–14 school year, surveys were administered by members of the research team to 330 treatment- and 340 control-group students. The information was collected from each matched grouping on average 47 days after the treatment students in that matched grouping saw the play. Of the students who completed the consent forms necessary to participate in the study, we collected surveys from 78 percent in the treatment group and from 79 percent of students in the control group.

The survey collected background information on students as well as a number of important outcomes. Each survey contained a series of items to assess student knowledge of the plot and vocabulary used in the plays. There were also several items to measure student tolerance as well as student interest in viewing and participating in live theater.

We also employed a measure known as the Reading the Mind in the Eyes Test (RMET), which captures the ability to infer what other people are thinking or feeling by looking at their eyes. The test was developed by British psychologist Simon Baron-Cohen and his colleagues as a tool for studying theory of mind, particularly for people with autism. It is now widely used by researchers interested in studying theory of mind and empathy for people developing typically, as well as for those with autism. Researchers using RMET have found that reading literary fiction or engaging in theatrical role-playing enhances people’s ability to read the emotions of others. We suspected that watching live theater might have a similar effect and decided to include RMET in our survey. The version of RMET we employed was developed for use with adolescents and has 28 photographs cropped to show only people’s eyes. Subjects are asked to pick one of four words that best describes what the photographed person is thinking or feeling.

All of the scales used to measure student knowledge about the plays, tolerance, ability to read the emotions of others, as well as interest in watching or participating in live theater, are either established or were validated with conventional tests of scale construction. All of the results reported below are based on analyses that control for student grade level, gender, and minority status, and compare students only within each matched grouping, while taking into account the fact that students within a given group are likely to be similar in ways that we are unable to observe.

Knowledge

Among students who are assigned by lottery to see live theater, knowledge of the plots of those plays as well as the vocabulary used in those productions is significantly enhanced, above and beyond what they learn by reading those works or by seeing film versions (see Figure 1).

 

For each play we asked students six questions about the plot and five questions about the vocabulary used. We combined those 11 items into a single scale measuring a student’s knowledge of the plays. Students assigned by lottery to see the live productions improved their knowledge of those plays by 63 percent of a standard deviation, a dramatic increase.

It may be easier for readers to grasp the nature and magnitude of this knowledge result by describing some of the changes produced on some of the individual plot and vocabulary questions we asked. For example, we asked Hamlet students, “Who are Rosencrantz and Guildenstern?” and 83 percent of the students who were assigned by lottery to see the play could correctly identify them as Hamlet’s friends, compared to 45 percent of the control group. Of the students who saw A Christmas Carol, 88 percent could correctly identify Jacob Marley as Ebenezer Scrooge’s deceased business partner, compared to 66 percent of the control group. More than 94 percent of the treatment group knew that Ophelia drowns in Hamlet, compared to 62 percent of the control group. Of those who saw A Christmas Carol, 93 percent knew that humbug meant “nonsense or a trick” compared to 62 percent of the control group, and 66 percent knew that destitute meant “very poor,” compared to 50 percent of the control group. And 71 percent of students who saw Hamlet knew that idle meant “not working, active, or being used,” compared to 61 percent of the control group.

Even the control-group students did much better than chance in picking the correct answer out of four multiple-choice options for each question. Our sample contained a large number of drama and Advanced Placement (AP) English students, whose knowledge of the plots and vocabulary from these plays may have been relatively high even without seeing a live production. But another possible explanation for the relatively high amount of knowledge among the control group is that almost one-quarter of them had read Hamlet or A Christmas Carol or watched movie versions of those stories for school that year. An even higher rate of the treatment groups, 52 percent, reported having read or watched movies of Hamlet or A Christmas Carol for school that year. Teachers who knew their students would attend the play were almost twice as likely to assign students to read or watch movies to prepare for the theater field trip.

This raises the question of whether treatment-group students acquired so much knowledge about the plays because they saw the live theater performances or because they had been prepared by reading and watching movies. Of course, student groups were not randomly assigned to read or watch the movies, so we can’t have the same confidence in identifying causal relationships, but we can use information about reading and watching movies to try to separate the extent to which the benefits we observed were produced by seeing a live theater production, or by having read and watched movies of those same works in school.

It is very clear that reading or watching movies of Hamlet and A Christmas Carol cannot account for the increase in knowledge students experienced by winning the lottery to see the plays. Even when we control for watching the movie or reading the material for school, the estimated effect of winning the lottery to see the plays remains basically unchanged, producing an effect size of 58 percent of a standard deviation for the treatment group on knowledge of the plot and vocabulary of the plays. Students who were assigned to read Hamlet or A Christmas Carol did no better on tests of their knowledge of the plays than did other students. Watching the movies for school is associated with about half of the benefit (30 percent of a standard deviation) as seeing the live theater performance. If teachers want students to learn plays, it is much better for them to take students to a live theater performance than to have them read the material or watch a movie. Plays are taught best by seeing them performed live.

Tolerance

We hypothesized that culturally enriching field trips are broadening experiences that expose students to a diverse world populated with different people and ideas, making them more aware and accepting of those differences. As part of the previous study in which we randomly assigned school groups to tour an art museum on a field trip, we found a significant increase in tolerance among students who toured the museum. To test whether field trips to see live theater have a similar effect, we utilized a scale that measures tolerance of others.

Students assigned by lottery to see A Christmas Carol or Hamlet scored significantly higher on our tolerance measure than did the control-group students. The difference is a little more than one-quarter of a standard deviation (26 percent). To put this effect in context, consider that students were asked the extent to which they agreed or disagreed with the following statement: “Plays critical of America should not be allowed to be performed in our community.” If students won the lottery and went on the field trip to see the plays, only 9 percent agreed that plays critical of America should be forbidden, compared to 21 percent of the control group. Students were similarly asked the extent to which they agreed or disagreed with the statement: “People who disagree with my point of view bother me.” Only 22 percent of the treatment-group students agreed with that statement, compared to 30 percent of the control group.

There is something about cultural experiences that seems to promote tolerance. Is it possible that reading or watching movies of these stories account for the benefit we observed? Again, we reanalyzed our data controlling for students having been assigned to watch or read these works for school. Students who read A Christmas Carol or Hamlet for school were no more tolerant than students who had not done so, and students who watched the movies were actually somewhat less tolerant. Controlling for reading and watching movies strengthens slightly the estimated benefit of seeing live theater on tolerance to an effect of 31 percent of a standard deviation.

If it is true that seeing live theater increases tolerance, then past theater exposure should also be positively related to tolerance. We randomly assigned students to only this theater experience, not past ones, so our confidence in making any causal claims about the effects of past experiences should be lower. Nevertheless, it would support our finding that being randomly assigned to see this show improved tolerance if past theater experiences were also positively related.

We do not have a direct measure of the cumulative exposure students have had to theater, but it is reasonable to assume that students who are more interested in live theater have also had more exposure to it. We did ask a series of items about students’ interest in seeing live theater. As we discuss below, attending a play had no impact on interest in seeing live theater, so this measure is telling us about differences in students’ interest in live theater independent of our experiment. When we modify our analysis to control for interest in theater, the benefit of seeing A Christmas Carol or Hamlet does not change much (22 percent of a standard deviation), but we do find that interest in seeing theater (our proxy for past exposure) is strongly related to tolerance. A one-standard-deviation increase in theater interest is associated with an increase of 37 percent of a standard deviation in tolerance. This evidence suggests the plausibility of our finding that being randomly assigned to go on a field trip to see live theater increases tolerance. We had a similar finding in our earlier study of field trips to an art museum.

Reading Other People’s Emotions

Seeing live theater may be particularly beneficial in teaching students to recognize the emotions of others. Theater works best when the actors effectively convey to the audience what their characters are thinking and feeling. The intensity of that experience may provide the audience with practice in reading emotions that is not normally found in everyday experience. Some earlier research also suggested that literary and acting interventions are effective at increasing people’s ability to read the emotions of others. We expected the same might be true of the students seeing live theater in our experiment, so we administered the youth version of the Reading the Mind in the Eyes Test (RMET).

Students who won the lottery to go on a field trip to the theater scored significantly higher on RMET than did the control group. Even the control group scored fairly well on the test, correctly identifying the emotions portrayed in the photographs of eyes 71 percent of the time. But the students who saw A Christmas Carol or Hamlet could correctly identify emotions 73.4 percent of the time. That translates into an increase of almost one-quarter of a standard deviation (23 percent).

Having read or watched movies of these works, in contrast, was not associated with students’ ability to read the emotions of others. And adding controls for reading or watching films of these works did not substantially change the estimated effect of seeing the live performances. The intensity and immediacy of live performance appears to have conveyed this ability to recognize what other people are thinking and feeling in a way that watching a movie or reading a text could not.

We collected the RMET measure, on average, 47 days after treatment, which suggests more than a fleeting benefit. To test whether past theater experiences are also positively related to the ability to read the emotions of others, we ran another model controlling for interest in seeing live theater as a proxy for the cumulative effect of past theater exposure. Doing so does not change the estimated benefit of seeing A Christmas Carol or Hamlet, but it does show that past theater experience is also significantly and positively related to the ability to read the emotions of others. A one-standard-deviation increase in interest in live theater (as a proxy for past theater exposure) is associated with an increase of 9 percent of a standard deviation in the RMET score.

While our proxy for past theater experiences was not randomly assigned and cannot be used to make causal claims, the fact that past theater is positively related to the ability to recognize emotions helps confirm the plausibility of our finding. Seeing live theater seems to teach students how to be better at reading other people’s emotions, a quite useful skill.

Participation and Consumption

It is also important to note outcomes that we thought might be affected by winning the lottery to see live theater but that produced no significant effects. In particular, we suspected that seeing live theater might inspire students to become more interested in either participating in future theater productions or going to view them. We found no evidence of this. Students who saw A Christmas Carol or Hamlet were no different in their desire to participate in or view theater than those who did not see a live performance.

The lack of an effect on student interest in viewing theater differs from the findings from the art museum study. In the earlier study, students who toured an art museum expressed significantly stronger interest in future museum attendance and actually returned to the museum at higher rates than did the control group. But on closer examination, the results are not actually so different. The benefit in the earlier field-trip study was concentrated in students who lacked previous cultural experiences, specifically younger, rural, minority, and low-income students as well as those who had not previously been to the art museum. More-advantaged students showed no significant benefit in the art museum study in terms of their interest in future museum consumption. The subjects in the TheatreSquared study are similar to the more-advantaged students in the art museum study. They are older, on average in 9th grade, and many were in drama or AP English classes (particularly for Hamlet, since that play is often read for AP English). These students already had a fairly high prior exposure to live theater, so it is possible that the marginal benefit of this one experience on interest in theater consumption is not strong enough to be detectable.

Limitations

Several groups had to cancel their attendance of A Christmas Carol because their schools were closed by snowstorms on the days of the performances. In the results presented above, we have treated school closures caused by weather as random events and reassigned those schools that did not see the play to the control group. An alternative way to handle these missed performances is to leave those students in the treatment group, but to adjust the results for the fact that not all treatment-group students actually attended the play as intended. Doing so produces basically the same estimated effects, with the exception that the effect of live theater on tolerance falls short of being statistically significant. We provide these results and a more detailed explanation for why they are not our preferred analysis in the methodological appendix available here.

Because the statistical power of this experiment is driven by the number of school groups, not the number of individual students, we are unable to conduct subgroup analyses to reveal how seeing a play may differently affect subsets of students. For example, with our limited number of school groups we cannot know whether minority students, female students, younger students, low-income students, or rural students receive different benefits from seeing live theater. Our ability to conduct these subgroup analyses is further constrained by the relative homogeneity of the students in our sample, with most being white and in advanced classes. However, this homogeneity does improve the similarity of students within our matched groupings, strengthening the overall results.

We are unable to say much about how A Christmas Carol and Hamlet may have differed in their effects. Again, we have too few school groups. Also, school groups chose the play for which they applied, so we do not have the benefit of random assignment to make causal claims about how the two plays may have differed in their effects.

Conclusion

Culturally enriching field trips matter. They produce significant benefits for students on a variety of educational outcomes that schools and communities care about. This experiment on the effects of field trips to see live theater demonstrates that seeing plays is an effective way to teach academic content; increases student tolerance by providing exposure to a broader, more diverse world; and improves the ability of students to recognize what other people are thinking or feeling. These are significant benefits for students on specific educational outcomes that schools pursue and communities respect. Especially when considered alongside our previous experiment on field trips to art museums, this research shows that schools can draw upon the cultural institutions in their communities to assist in producing important educational outcomes. Not all learning occurs most effectively within the walls of a school building. Going on enriching field trips to cultural institutions makes effective use of all of a community’s resources for teaching children.

Finally, this research helps demonstrate that schools produce important educational outcomes other than those captured by math and reading test scores, and that it is possible for researchers to collect measures of those other outcomes. If what’s measured is what matters, then we need to measure more outcomes to expand the definition of what matters in education.

Jay P. Greene is professor of education reform at the University of Arkansas, where Collin Hitt and Anne Kraybill are doctoral students and Cari A. Bogulski is a researcher.

Read the methodological appendix to this study

Seventeen years and a host of education reforms separate public declarations by its highest-ranking officials that the nation’s largest labor union should become a leader of education reform. Children who were just entering the public school system when National Education Association (NEA) president Bob Chase addressed the National Press Club in 1997 are adults now, perhaps with children of their own. NEA executive director John Stocks issued the same call to arms in 2014.

The notion was not a new one, even in 1997. In that same speech, Chase admitted he was not the first to call for the union to be an agent of change. “In 1983, after the A Nation at Risk report came out, NEA president Mary Hatwood Futrell tried to mobilize our union to lead the reform movement in American public education,” he said.

Futrell failed at that task, as did Chase, as did his successors, as will future NEA presidents. The failure is the inevitable result of the difference between what teachers unions are and what they would like others to think they are. This difference manifests itself as two messages: an internal one, meant for the unions’ leaders and activists, and an external one, meant for education policymakers and the public at large. In the good old days, the two audiences were always separate. But in today’s world, where everyone with a phone or Internet access can act as a reporter, the two messages can overlap, causing confusion and contradiction.

The teachers unions now face an environment in which their traditional enemies are emboldened, their traditional allies are deserting, and some of their most devoted activists are questioning the leadership of their own officers.

Moment of Truth

The events of the last five years have led the two national teachers unions to what normally is referred to as “the moment of truth.” But truth is tricky to define when perceptions are an integral part of the unions’ influence. Even weakened, together the NEA and the American Federation of Teachers (AFT) constitute the single most powerful force in American education policy. Nothing moves forward without an answer to one question: What will the union do?

Will the NEA and the AFT continue to exert veto power over education reform? Are their recent setbacks the beginning of an irreversible decline? Will they become more militant or less as the years go on?

Predicting the future is a hazardous business, but if what’s past is prologue, we can at least make a reasonable estimation that assumes no unprecedented, revolutionary change in direction. Considering the quotes above, that seems to be a safe assumption.

A Nation at Risk may have jolted the education world, but it had no effect on the growth and power of teachers unions. They enjoyed substantial boosts in membership each year, as the hiring of teachers and education support employees grew at historic rates. There were periodic national efforts at education reform, such as Goals 2000, but the unions weathered these storms, and the winds eventually died down.

The first sign that the world was changing around them was the passage of the No Child Behind Act (NCLB) in 2001. Staunch union allies such as Representative George Miller of California and Senator Ted Kennedy of Massachusetts co-authored the bill, and it passed Congress by wide margins, with more Democratic votes than Republican ones.

The accountability provisions of the law bedevil the unions to this day, but the large increase in federal education spending ended up helping the unions’ bottom line, as still more teachers and support workers were hired.

By 2008, both unions hit their high-water mark, with a combined membership approaching 4 million and annual revenues at all levels estimated at nearly $2 billion. But all was not well.

The 2008 Democratic presidential primaries were a series of mishaps for the teachers unions. The AFT, with its large plurality of members from New York State, inevitably endorsed Hillary Clinton. There was strong sentiment within the NEA to do likewise, but NEA president Reg Weaver hailed from Illinois and was a strong supporter of Barack Obama. Still, he could not sway a sufficient number of the union’s decisionmakers, and the NEA was as divided on whom to endorse as was the Democratic Party.

The situation was further complicated by Obama’s embrace of education reforms the NEA found anathema. He gave moderate praise to performance pay. As a U.S. senator, he was the only Democrat to introduce an NCLB-related bill that the union opposed. The NEA analysis of the proposed legislation claimed it favored “1) establishing a teacher evaluation system using gains in student test scores; 2) allowing ‘community stakeholders’ to have a role in designing teacher evaluation systems; and 3) providing merit pay for teachers based upon gains in student test scores.”

As a result, the NEA did not get around to endorsing Obama until after he had clinched the nomination. They worked hard to help elect him in 2008, but it was clear that he felt no special obligation to the unions when he named Arne Duncan as U.S. secretary of education, instead of union favorite Linda Darling-Hammond of Stanford.

Duncan has since become the bête noire of the teachers unions, but his selection and subsequent actions signified a clear continuity in education policy from Obama the candidate to Obama the president. Coupled with the growing influence of Democrats for Education Reform, it was now OK to be a Democrat without kowtowing to the unions on every education issue.

But let’s not go crazy. When the recession hit, the Obama administration’s first instinct was to protect the jobs of educators. Secretary Duncan claimed the stimulus package of 2009 funded more than 300,000 education jobs. Follow-on legislation, the so-called “edujobs” bill of 2010, was purported to save 160,000 more.

As mammoth as the spending was, these were short-term fixes. Education hiring actually grew 2.3 percent during the recession, but then fell off a cliff when the money ran out. Educators experienced the Great Recession about two years after everyone else.

Reductions in force meant fewer teachers-union members, which meant reduced revenue for the unions. Budgetary concerns also provided the impetus for Republicans seeking to curb the power of teachers unions, the primary example being the passage in 2011 of Act 10 in Wisconsin, which greatly restricted the bargaining power of NEA and AFT affiliates in the state, and ultimately reduced their dues-paying membership by more than one-third.

Right-to-work legislation followed in Indiana and Michigan, and in places where teachers unions were already struggling—like Arizona, Idaho, Louisiana, North Carolina, and Tennessee—membership fell by 20 percent or more. The 3,000-member University of Hawaii Professional Assembly left the NEA and became independent.

Today, the NEA’s membership is down more than 9 percent over the last four years. The AFT claims its membership is steady, though it has maintained it by affiliating unions outside the field of education, and not by recruiting a horde of new teachers. The percentage of teachers who are union members has dropped (see Figure 1).

A House Divided

The question dogging both national unions and their affiliates is how to turn this state of affairs around. There seem to be two alternatives, roughly analogous to a choice between war and diplomacy.

One faction, existing in both unions, wants to man the barricades, fight over every inch of territory, and take no prisoners. It sees education reformers outside of the union sphere as either corporate privatizers seeking to grasp some of the $640 billion this country spends annually on public schools, or their tools.

The most identifiable leaders of this militant faction are Karen Lewis, president of the Chicago Teachers Union, Alex Caputo-Pearl of United Teachers Los Angeles, Bob Peterson of the Milwaukee Teachers’ Education Association, and Barbara Madeloni of the Massachusetts Teachers Association.

The establishment faction of the unions includes the leadership of both national teachers unions. Both the NEA and AFT have severely restricted paths to power, making it unlikely that any of the militants will rise to the national presidency. When union challengers upset incumbents, however, it is almost always because the challenger successfully painted the incumbent as too accommodating to the education powers that be.

To avoid becoming losers in the game of “more teacher-protective than thou,” the leaders of the national teachers unions have to co-opt the militant message without alienating the education world at large, or the general public. This is a tricky dance, and it’s not uncommon for NEA and AFT executive officers to make conflicting, if not contrary, statements depending on which ears are listening.

When union officers address an audience of union activists, the world is described in Manichaean terms. Standardized testing is not just misused, it is “toxic.” Opponents are not just opponents, they are adversaries “who want to destroy our democracy and our public schools”—for money. These enemies are identified by name: the Koch Brothers, the Cato Institute, Americans for Prosperity, Pearson, Inc., Democrats for Education Reform, Michelle Rhee, and Arne Duncan.

The only force standing in their way is the teachers union—“the champions of equity,” who “define solutions that drive excellence and success for all students,” as described by former NEA president Dennis Van Roekel in his keynote address to the Representative Assembly in July 2014. Union activists, in the words of John Stocks, spoken two years earlier, are “social justice patriots” who “put the power of our soul to work to defend democracy, to fight for equal opportunity, and to create a more just society.”

That plays well with the troops, whose enthusiasm and commitment are needed to advance the agenda. Unfortunately for the teachers unions, the wider world is not an echo chamber of their beliefs. To the general public, many of whom have little idea what the NEA and the AFT actually do, it sounds more than a little hyperbolic and self-congratulatory.

The external message cannot be so bellicose. Both the NEA and the AFT need allies, including those who might not sign on to the totality of the unions’ vision for public education and American politics. Even with their opponents, they cannot escalate every confrontation to Armageddon. Compromises occur.

Union officers are also aware that it is detrimental to their cause to be constantly saying “no” to so many proposals for school reform. Thus the external message is devoted to depicting an organization that is forward-thinking and innovative when it comes to operating the nation’s schools.

The problem for the unions’ establishment wing is that the internal message leads their devotees to believe that such compromises, collaborations, and accommodations are selling out the movement. They are not always wrong about that.

While both national unions decry the corporate influence on education, they have partnerships with large corporations on many levels: sponsorships of union events, discount arrangements and credit cards as part of member benefits packages, funding for joint projects, etc. The NEA even went so far as to team up with Walden Media on a book-buying initiative for needy children. Walden Media produced Waiting for Superman, a documentary about families trying to get their kids into charter schools. It was especially critical of teachers unions.

Union activists often depict the Gates Foundation as the mastermind behind corporate education reform. But in 2009, when the foundation announced it would award $335 million to a number of school districts and charter schools to promote teacher effectiveness, the union response was a far cry from the anticorporate rhetoric it regularly delivers to its internal audience.

“These districts, working with their unions and parents, were willing to think out of the box, and were awarded millions of dollars to create transparent, fair, and sustainable teacher effectiveness models,” said AFT president Randi Weingarten.

“Collaboration and multilevel integration are important when it comes to transforming the teaching profession,” said then NEA president Van Roekel. “These grants will go far in providing resources to help raise student achievement and improve teacher effectiveness.”

The NEA’s own foundation received $550,000 from the Gates Foundation to “improve labor-management collaboration.” The AFT accrued more than $10 million from the Gates Foundation, until internal pressures forced the union to end some of the grants. And of course, the Gates Foundation helped bankroll the development of the Common Core State Standards (CCSS), which both unions continue to officially support (see “Teachers Unions and the Common Core,” features, Winter 2015).

The militant wing is mostly hostile to CCSS, seeing the standards as part and parcel of the corporate education-reform agenda. The establishment wing has been forced to triangulate by defending the standards but attacking the way they have been implemented.

The split between the two factions was illustrated at the 2014 AFT Convention. The delegation from Chicago introduced a resolution to place the AFT in full opposition to CCSS, but it was handily defeated in committee, a committee dominated by New York City’s United Federation of Teachers, the backbone of the AFT’s establishment wing.

Instead, AFT delegates passed a resolution stating the union would “continue to support the promise of CCSS, provided that a set of essential conditions, structures and resources are in place.”

The edTPA Battle

The disconnect between the two messages of unions as leaders of education reform and unions as defenders against education reform can sometimes reach comical levels.

The edTPA is a teacher-candidate performance assessment that has met with withering criticism from the unions’ militants. Barbara Madeloni of the Massachusetts Teachers Association made her reputation and her ascent to power based on her opposition to and boycott of the evaluation. Their biggest bone of contention is the fact that the videotaped lessons delivered by teacher candidates to their students are contracted out to be scored by Pearson, Inc.

If that’s all there were to it, it would simply be a standard union response to corporate influence over teacher education. But edTPA was developed by the Stanford Center for Assessment, Learning, and Equity (SCALE), whose adviser is the illustrious and union-beloved Linda Darling-Hammond. SCALE partnered with the American Association of Colleges for Teacher Education (AACTE), another union-beloved (and -financially supported) organization to develop edTPA.

The edTPA is apparently scored entirely by classroom teachers and university professors, and is similar to the assessment used by the union-beloved National Board for Professional Teaching Standards. NEA vice president Becky Pringle and AFT president Randi Weingarten sit on the edTPA Policy Advisory Board. The NEA promoted the assessment on its website. As with CCSS, the AFT passed a resolution praising the design of edTPA while lamenting its implementation and the role of Pearson in scoring the assessment.

At a time when the unions need all the allies they can muster, they are looking askance at the ones they already have. “If we are attentive to the broader assault on public education by the forces of neoliberalism,” said Madeloni, “we have to critique Darling-Hammond at the very least for naïveté about these forces when she allows herself and her work to be taken up by them, when she allows the discourses they use to bash teacher education to become part of her rationales.”

These are amusing little infights unless you have some responsibility as an education policymaker and you need to know if the unions favor or despise CCSS or edTPA. It is little help to hear that the answer is both.

The edTPA reaction also illustrates why the unions maintain a long and growing enemies list. The national-union officers are especially adept at co-opting the anger of the militants and focusing it on the approved targets—“toxic tests,” corporate reformers, and a host of nefarious individuals. Otherwise, the more radical activists might spend time examining their own leaders’ complicity in the current state of affairs.

The Long Gamble

The teachers unions are under duress. Their efforts to simultaneously exist as the champions and opponents of education reform are meeting resistance even internally. Their public image has never been worse. How do they plan to turn this around?

The clues are in the address made by John Stocks to the NEA Representative Assembly delegates: “play the long game.”

Times are bad, but bad times don’t last forever. As long as the unions can head off any more reversals in state collective-bargaining laws, and stem the bleeding in right-to-work states, they can await a more positive correlation of forces. As the economy rebounds, teacher hiring will increase, though probably not to previous levels. A few key governorships, a few key legislature flips, a couple of new U.S. Supreme Court justices, and the momentum could shift quickly.

Renewed membership growth and an increased revenue stream will not only improve the unions’ bottom line, but it will quiet the internal dissent. Unions are well aware that no one wants change when things are going well.

There are two problems with playing the long game. The first is that if you keep getting trounced in the short game, you don’t last long enough to win the long game. The unions need a major victory in the short term to stop the death spiral. The second problem is assuming that your strategy is the correct one and only needs time and better circumstances to work. Such thinking stifles internal reforms just as effectively as its stifles school reform.

People have been waiting patiently for the teachers unions to champion school reform since the days of Al Shanker, who served as AFT president from 1974 to 1997. But the reality never seems to match the rhetoric.

In 2008, the NEA unveiled the “Great Public Schools for Every Student by 2020” project, in which the union committed to “creating models for state-based educational improvement,” “developing a new framework for accountability systems that support authentic student learning,” and “fostering a constructive relationship with U.S. Department of Education leadership.”

The last goal has clearly been a failure and no one has mentioned the GPS 2020 project by name in public in more than five years.

In 2011, the NEA put together a Commission on Effective Teachers and Teaching, which actually delivered several levelheaded and worthwhile recommendations. The commission inspired the union to release a “three-part action agenda to strengthen the teaching profession and improve student learning.”

One of the commission’s recommendations was to “address internal barriers to organizational engagement about teaching quality and student learning.” It called on the NEA to “transform the UniServ Program, making UniServ directors advocates for educational issues to advance NEA’s professional agenda.”

UniServ directors are the union staffers who are collective bargaining specialists, political operatives, and experts in the finer points of school finances. A proposal to find a way to make them “advocates for educational issues” was presented to the delegates at the 2012 Representative Assembly. It was overwhelmingly defeated on a voice vote.

The lesson is that while many union members, particularly younger ones, might join the public in hoping for teachers unions that embrace change, the folks who run the unions are comfortable with their traditional mission: the protection of teachers in the workplace and of union prerogatives everywhere.

What we are likely to see in the future is simply more of what we have seen in the past and are still seeing in the present. The NEA and the AFT will be the obstacles in the road, blocking the way of new ideas, even those generated from within. Their adherence to a mission designed for the world of the 1960s will cause them to follow in the footsteps of the private-sector industrial unions, albeit several decades later.

The International Brotherhood of Teamsters was a massive political and social force in the 1950s and 1960s. Today, the union still exists; it still can deliver PAC money and campaign workers, but no one worries about the power and influence of the Teamsters anymore.

Even if their current difficulties continue, the NEA and the AFT will never disappear. But their days of dominating the education environment are on the wane. In the future, we will look upon them as we now do the Teamsters, as remnants of an earlier age.

Mike Antonucci is the director of the Education Intelligence Agency, which specializes in education labor issues.