Teachers often participate in professional development programs to stay on top of technology they could use to teach their students. Rarely, however, do they look at potential roles for technology their students are already using. The cell phone is one such device. Its value as an educational tool is vast and virtually untapped.

Cell phones are a significant feature in kids’ daily lives. According to Generation M², a 2010 Kaiser Foundation media study, nearly two-thirds of 8- to 18-year-olds have cell phones. Among 8- to 10-year-olds, 31 percent have their own phone, as do 69 percent of those ages 11 to 14. Eighty-five percent of teenagers 15 to 18 have them. A study by Mediamark Research & Intelligence found that most of the younger kids use the phone to contact their parents. Girls are more likely to use the phones for social uses, while boys are more inclined to play games or access the Internet.

The Pew Research Center in April 2010 released results from a survey that confirmed the ubiquity of cell phones among teenagers, some of whom manage to send text messages from class, even when the technology is banned in their school. While the Pew survey focused on texting, kids use their cell phones for all kinds of things. Along with brief calls to their parents and hours spent texting their friends, kids use their cell phones to listen to music, play games, and watch videos. Kids whose cell phones have cameras take pictures and send them to their friends. Older teens use smartphones like iPhones and Blackberrys to check Facebook and e-mail, get directions, and to obtain any other information they might need during the day.

Businesses have certainly caught on. Phone manufactur­ers and wireless carriers target their advertisements to young people. (Nearly all backpacks have cell-phone pockets.)

So have other groups. The New York Times has reported a rise in education apps, as they’re called. At a summer camp held at the New York Hall of Science in Queens, kids used smartphones and probes with Bluetooth capabilities to test and record levels of air pollution, part of a project run by New Youth City Learning Network. With other new mobile applications, students can take a picture of an insect or his­torical site, send it off, and receive a message back with full identification of the image.

Surely schools could make productive use of a technology that is relatively cheap, por­table, and already in the hands of the majority of U.S. schoolchildren.

The simplest use for students’ cell phones is keeping track of assignments. Rather than carrying around an assignment notebook, stu­dents could use their phones. The calendar and reminder functions can easily handle home­work and tests. Kids are much less likely to leave the phone at home, at school, or some­where else than they are a notebook.

A pilot program in North Carolina extends the cell phone’s reach far beyond keeping track of deadlines. Project K-Nect, a pilot program in Onslow County, uses smartphones as a learning tool in math classes, supplementing traditional math instruction with alternative teaching strat­egies. The project provides at-risk high-school students who lack computer or Internet access at home with smartphones. Teachers assign math problems for students to solve on the smart­phone. If students need help, they can connect with their classmates through instant messaging and dedicated blogs. If they still can’t solve the problem, they can access digital content through the phone. Project Tomorrow, which has evaluated the pro­gram, found improvement in student test scores, engagement and participation in class, and collaboration among students.

Cell phone use is typically forbidden in public school class­rooms. Teachers rightly object to phones ringing and students updating their Facebook profiles or texting during class. But educators could view cell phones differently. For adults, they are engaging, interactive tools—for communicating and for storing and accessing useful information. The same could be true for kids in school.

Rebecca Fortner teaches special education in Livingston County, Kentucky.

Podcast: Peter Meyer reports from Chicago, where two public schools have been launched by a Roman Catholic religious order.

Additional photographs of the Catalyst Schools are available here.

---

Train a child in the way he should go, and when he is old he will not turn from it.
—Proverbs 22:6

It wasn’t exactly a marriage made in heaven. In fact, the idea that one of the Catholic Church’s most respected religious orders might run a public school sounded odd, maybe even, as Francis Cardinal George, head of the Archdiocese of Chicago, conjectured, illegal.

But a decade ago several trends in American education, and in the Catholic Church, made a Catholic-operated public school seem increasingly possible: 1) the traditional, parish-based Catholic school system, especially in the inner cities, was crumbling; 2) equally troubled urban public-school systems were failing to educate most of their students; and 3) a burgeoning charter school movement, born in the early 1990s, was beginning to turn heads among educators in both the private and public sectors.

The various currents merged in the Windy City in 2006 and 2007 when the Christian Brothers helped open two charter schools in impoverished neighborhoods on Chicago’s west side, embarking on a unique experiment in public education. Could Catholics run a school without mentioning Jesus, Mary, or Joseph? Without prayer, Mass, the rosary, the sacraments? Without God? And could the high wall between church and state be kept intact?

Back to Their Roots

The Christian Brothers—known in France, where the Catholic order was founded in 1680, as Frères des écoles chrétiennes or Brothers of the Christian Schools—have had some experience in education. The order’s founder, Jean-Baptist de La Salle, is the church’s patron saint of teachers and today the order serves nearly 1 million students in more than 80 countries, including some 20,000, mostly middle-class, students in 90 Catholic middle and high schools and education centers in the United States.

But what caught the eye of Arne Duncan, while he directed Chicago Public Schools (CPS), was the success the brothers were having with an initiative the order had launched in the early 1990s. They had opened San Miguel schools, named after a Christian Brother saint from Ecuador, in American pockets of poverty, including an Indian reservation in Montana, the Tenderloin district of San Francisco, and inner-city Camden, New Jersey. (In 2006 San Miguel merged with the Jesuits’ Nativity Schools; today the NativityMiguel Network operates over 70 schools for the poor in 26 states and the District of Columbia.)

The first Chicago San Miguel school opened in 1995, behind the infamous (now gone) stockyards. The goal was simple enough: bring to poor children, tuition-free, what the brothers were delivering to middle- and upper-class students in their other American schools, including small class sizes and a college-prep academic program.

It worked. Within a few years of opening, San Miguel Back of the Yards School’s low-income students were outperforming their Chicago Public Schools counterparts. The school’s success prompted Lands’ End company founder Gary Comer to donate $1.2 million to open a second school, now known as the Gary Comer Campus, in the blighted Austin neighborhood.

The schools employ a year-round academic calendar, have a 9:1 student-to-teacher ratio, and a core academic curriculum. They put heavy emphasis on reading—80 minutes a day, an average of 165 books read per year—and individualized instruction.

“Our model is not rocket science,” says Mike Anderer-McClelland, a former brother who is now president of the San Miguel organization in Chicago. “It is a lot of reading, writing, and arithmetic.”

The schools also have a Family and Graduate Support Program that not only tracks students through high school but helps them and their parents with tutoring and counseling, long after they leave left San Miguel at the end of 8th grade.

“We graduate 85 percent of our kids from high school in a neighborhood that traditionally graduates less than 40 percent,” says Anderer-McClelland. “Sixteen percent of our kids graduate from four-year colleges, compared to less than 5 percent of public school kids in our neighborhoods; and it’s only 3 percent of CPS Latinos and 4 percent of CPS blacks who graduate from college.”

 

Getting from No to Yes

A faith-based or church-sponsored charter school had been the subject of some discussion among Catholics in Chicago almost from the moment that Illinois passed a charter school law in 1996. Though the law initially allowed just 20 charters statewide, 15 of the slots were assigned to Chicago.

Coincidentally, the Archdiocese of Chicago, which ran the nation’s largest parochial school system with more than 130,000 students, was in the midst of a demographic and financial crisis. The archdiocese had closed 55 of its schools in the previous 10 years. (See my story, “Can Catholic Schools Be Saved?” features, Spring 2007.) At the time, CPS CEO Paul Vallas and others encouraged the church to consider converting their closed and closing schools to charters. But in 1999 Cardinal George said “No.” It was “a square circle,” he remarked, “not because of archdiocesan protocols but because of the nature of the beast.”

That view was shared by others, including the National Catholic Educational Association (NCEA). “Are charter schools another way to keep Catholic schools alive, as some proponents suggest?” the NCEA asked in a 2009 press release. “Absolutely not.”

Catholic leaders could not support schools that were not grounded in religious instruction.

However, in what was perhaps an unintended consequence of the church’s crisis, a former priest became one of the first to open a public charter school in Chicago. John Horan, director of the Archdiocese’s Catholic Youth Organization when he was a priest, opened the public North Lawndale College Prep charter in 1998 as a layperson. “Catholic schools were terrific,” says Horan today, “but there just wasn’t enough financial support to send all of our poor kids to Catholic schools. So we thought, we…have to make public schools work.”

This is what Paul Vallas and Arne Duncan were thinking, too. Vallas had floated the idea of the Christian Brothers running a public school as early as 1997, recalls Ed Siderewicz, a young Christian Brother who helped start the San Miguel Back of the Yards School, “but we shrugged it off as compromising our mission.”

The question would keep coming up.

“The [San Miguel] board really wrestled with this,” recalls Sister Margaret Farley, director of personnel for the archdiocesan schools of Chicago and a founding board member of San Miguel. “We thought we could do a values-based school, but many members of the board were superparanoid about anyone thinking it would be a Catholic school.”

Early on in the discussions, Brother Gordon Hannon, a Chicago native and cofounder of San Miguel Back of the Yards School, researched the question in a paper for a graduate course at DePaul University. He concluded that though the San Miguel model “meets a clear and urgent secular need,” it was an open question whether “a faith-based group of competent, licensed educators” could run a publicly funded school without crossing the church/state line.

Brother Mike Fehrenbach, head of the Christian Brothers’ Midwest District, saw an opening in the “values-based” charter idea (instead of faith- or religious-based) and convinced the San Miguel board that the question was worth exploring. And in July of 2004 they formed a special charter-school planning committee.

Money was a significant consideration.

“We are probably near the limit of our ability to raise the dollars necessary to expand effectively,” wrote Fehrenbach in a one-page memo to the group, referring to the cost of running the two private San Miguel schools. He suggested that a charter school, which was publicly financed, might be a way of bringing education to the poor without having to spend so much time fundraising.

“This sounded like a growth company,” smiled Terry Toth, recalling his initial reaction to the idea. Toth, a lifelong Catholic, was a member of the San Miguel board as well as head of the world’s 12th largest investment bank, Northern Trust. “We would get 85 percent of our costs paid for—versus the need to fundraise 100 percent,” he laughs. “Win-win.” Toth also recognized that it was an opportunity to have an impact on more people. “Obviously, there was a church/state issue. But it was worth trying.”

“We Can Do This”

That is what Arne Duncan said, Ed Siderewicz recalls, when Gary Comer brought him to visit the San Miguel Austin campus in 2004. “You get your team behind it and I’ll make it work.”

“We asked a few questions,” recalls Siderewicz, “but I remember thinking that if our starting point is how to make it work rather than what we have to give up, then we should continue talking.”

And they did continue talking, with more and more detailed attention given to the question of San Miguel’s core mission. God or no God? For his part, Mike Fehrenbach didn’t see the charter undertaking as a challenge to the brothers’ mission, which he believed was “about offering people an opportunity for a future worth living.” They could do this by exemplifying Christian and Catholic values; they didn’t have to preach them. (This was not unknown territory for the Christian Brothers. In many countries they ran secular schools; in Indonesia, in fact, they operated a Muslim school.)

At the end of January 2005, the order’s district council voted to launch the charter school. And a month later, Brother John Johnston, then the order’s Superior General in Rome, weighed in: he saw “no important reasons for saying no” and “important reasons why we should say yes.”

 

Too Good to Be True

But the Christian Brothers were suddenly given another challenge. In the middle of writing the application for the charter school they wanted to open, Arne Duncan asked them to take over a public school he was closing in the North Lawndale neighborhood. This was part of the “turnaround” strategy that Duncan initiated in Chicago and would bring to his job as secretary of education in the Obama administration: improve some schools by closing them, then reopening under new management.

This was fine except that the brothers had already found the neighborhood for their new charter school: Austin, near the Gary Comer Campus San Miguel School. Austin was clearly needy; it had the highest number of homicides in Chicago in 2003 and nearly 30 percent of families with children under 18 lived in poverty. The neighborhood high school had the second-highest dropout rate in the state of Illinois. And the brothers had good connections in Austin, including with Circle Urban Ministries and the Rock of Our Salvation Church, an active evangelical Baptist congregation that owned a former Catholic school there.

“It was like manna from heaven,” recalls Rev. Abraham Lincoln Washington, pastor of the congregation, remembering Brother Ed Siderewicz’s request to open a charter school in his facility. Circle Rock had been struggling to keep open its 185-student religious school for the poor. “We had a vacant building and they had 300 years of experience educating kids.”

Reverend Washington’s congregation was also a strong one, providing a rich mix of social services, including a food pantry, legal and medical services, transitional housing, and education. “Education is like preventive medicine,” he says. “It’s so important.”

But the perfect union between Christian Brothers and Baptists would have to wait.

“At the last moment, Duncan asked us to take over the Howland School in North Lawndale,” recalls Siderewicz, “before we opened Austin Circle Rock. Naively, we said ‘Okay.’”

Although the neighborhoods are, technically, adjacent, Austin and North Lawndale are among Chicago’s largest neighborhoods and so the Howland School was more than five miles south of Austin Circle Rock—a world away. Especially for the Christian Brothers, who had no presence there.

Worse, North Lawndale was even needier than Austin. At the time of the 2000 U.S. Census, approximately 10,000 adult males from the neighborhood were in prison. Says Father Lawrence Dowling, pastor of St. Agatha Parish, which is just four blocks from the Howland School, “We have the highest incidence of HIV in the state, the highest rate of asthma for kids in the state, and the highest percentage of grandparents raising grandchildren in the nation.”

“As poor a census tract as you can find,” says John Horan, who had established his North Lawndale charter high school in one part of the sprawling Howland building. “Do I know the neighborhood?” he chuckles. “I am completely grey because of it.”

But the Christian Brothers didn’t know the neighborhood, and the neighborhood didn’t know them. Recalls Fehrenbach, “The community was in an uproar over the closing of their school—and then we came in.”

At an initial public meeting, a group calling itself “The Voice of the ExCon” sent dozens of people, who shouted and screamed. “It was a bit scary,” recalled Siderewicz. “But we made it through.”

The Christian Brothers eventually won the group over by giving them a tour of their San Miguel schools, but the initial animosity was a sign of things to come.

 

A Tale of Two Schools

The brothers opened Howland Catalyst Charter School, on schedule, in the fall of 2006 and Austin’s Circle Rock Catalyst in 2007. The Howland plan called for starting with 4th and 5th grades, with two classes of 15 students in each grade; it would add 3rd and 6th grades in year two, 2nd and 7th grades in year three, 1st and 8th grades in year four, and kindergarten in year five, growing to 540 students in grades K through 8 by 2010. The Austin plan was to start with 5th and 6th grades beginning in 2007, add 7th and 8th grades in the second year, then build up from kindergarten to 4th grade in the next two years.

Both plans reflected the brothers’ belief in the importance of middle school, which was the focus of their San Miguel initiative. And true to the San Miguel model, they brought to the Catalyst charters their 9:1 student-to-teacher ratio, a belief in academic excellence, graduate and family support, and the goal of having every child reading at or above grade level before graduation (8th grade). They also brought modern assessment tools, including standardized testing several times a year, teacher-designed testing on a weekly basis, nightly homework that was checked each day by the teacher, and daily testing/assessment in core subject areas.

Despite the model, the polished floors, new banners, and students outfitted in spiffy olive and khaki uniforms, the staff at Howland was quickly overwhelmed by the outsized needs of its student population, which was 100 percent African American and 98 percent eligible for free or reduced-price lunch.

“These children live in a world where you can’t appear weak—it can be deadly,” says assistant principal Igbazenda Moses, a Nigerian native and former Christian Brother who came to Chicago in 2002 and joined the Catalyst staff in 2006. “All of this makes the kids very uptight and creates a huge barrier to learning.”

Howland’s was a student population suffering from enormous environmental, economic, and social trauma—a kind of permanent Post-Traumatic Stress Disorder.

“Three weeks ago we had a child whose dad was shot and killed,” Moses says. “We have two kids like that here now. There is so much violence in their daily lives. They hit you—and they’re used to getting hit back. Self-control is a big issue for these kids.”

Howland’s leaders were caught off guard by the severity of the social and environmental ills, and the effort it would take to address them. As Brother Ed Siderewicz recalls, “We had every problem in the book and more. I had forgotten how hard it was to start a school.”

 

God Help Us!

Assistant Principal Moses believes that the inability to teach religion was a significant part of the problem. “Talking about how to be respectful and trustworthy is not the same as talking about a person’s life being grounded in the knowledge of God,” he says. “And that’s what these children and their families need. ‘Character Counts’ just doesn’t do the trick.”

Brother Mike Fehrenbach smiles at this in-house dissent. “We’re doing fine.”

Both Catalyst schools use the popular youth ethics program developed in the early 1990s by the Josephson Institute of Ethics. Character Counts includes “six pillars”: Trustworthiness, Respect, Responsibility, Fairness, Caring, and Citizenship. Banners with those words are hanging throughout the school.

“Yes, we’re doing character education,” says Gordon Hannon, no longer a Christian Brother, but who was brought back in as Catalyst CEO at the end of the 2009 school year to help right the Howland ship. “But we have to go beyond that. To capture the essence of Catholic education, one of our core values is reverence. We must instill a sense of reverence for each other and show how that is different than respect.”

“You can get lost in debates about whether you should have a crucifix on the wall or not,” says John Horan. “It’s more about violence and drugs and poverty than the decorations.”

And leadership.

“I had a bad feeling in the pit of my stomach,” says Megan Dougherty, who arrived in August 2007, at the beginning of Howland’s second year, and was assigned a 5th-grade classroom. “There was no new teacher training,” she recalls. “And there was a new principal and a brand-new administrative team. The kids were being bad and teachers didn’t feel they had any support.”

Five teachers quit that year; another five didn’t return the next year. Dougherty would have been gone early except that when she came back from Christmas holiday, she realized that “my kids really missed me. I had to stay for them.” But she did eventually leave. “No one was happy with their job,” says Dougherty. “I loved the school. I loved the kids. But I couldn’t stay another year.”

Fourth-grade teacher Tina Corsby calls herself “the last of the Mohicans.” A veteran of CPS, she started at Howland Catalyst when it opened in 2006 and is still there in 2010. “It’s better than a regular public school,” she says. “Here, they really do care about the kids.”

But Corsby and her teacher colleagues were caught in the classic bind: trying to do a good job, they seemed to get no help from the top. “Job one in a school like this is to establish a culture of peace and high academic expectations and do rigorous social supports,” says John Horan. “Stable leadership makes all the difference in the world. You can’t end-run this one. The principal makes it all happen.” Howland has had three principals in three years.

“We’ve struggled from the get-go,” says Catalyst CEO Hannon. “We had everything we had at San Miguel, but we weren’t clear enough or deliberate enough about who we are. We were hesitant. We were not tough enough about the teaching, about embracing reading as the number one priority…. The bottom line is that the leadership wasn’t there.”

While these lessons were being learned, the San Miguel team opened its second Catalyst charter school, at Austin Circle Rock, in 2007. Perhaps it was just luck that Sala Sims, a seven-year veteran of CPS classrooms, applied for the job of principal. “I knew I wanted to start a school,” she recalls. The Chicago native met Brothers Mike and Ed in early 2007 “and never looked back.”

“She gets it,” says Brother Mike Fehrenbach.

And it showed. After just two years, Austin Circle Rock had an air of order that eluded Howland after three. From the front desk to the back offices and faculty lunchrooms, students and adults at Austin Circle Rock were both more relaxed and more disciplined. And the test scores proved it.

In 2009, Austin Circle Rock students outperformed the other Austin neighborhood schools on the composite (reading, math, and science combined) Illinois Standards Assessment Test (ISAT), with from 68 to 76 percent of students in grades 4 through 8 meeting or exceeding the state standard.

“It’s part of the Resurrection before our very eyes,” says Brother Ed Siderewicz.

How was Howland doing on its test scores?

“Flat would be generous,” says Hannon. In fact, Howland ISAT scores in 2009, with just 49.8 percent of its students meeting or exceeding the state standard, tied for last place among five public schools in North Lawndale.

How could the “formula for success” have gone so far wrong in Howland?

In many respects, it is as simple as what Donnell Harrison, the safety manager at Austin Circle Rock, calls, “following the model.” Donnell has manned the front desk at Austin since it opened. “At Howland they’re conforming to the community instead of to the model.”

This point has not been lost on Catalyst leaders, especially as they work on turning Howland around.

With Hannon as the new Catalyst CEO came a new principal, Chaun Johnson, and the two have become a veritable tag team. Johnson grew up in the Austin neighborhood, where Circle Rock is located and where his wife Natalie now teaches, and he attended Our Lady of Lourdes Catholic School (now closed) in Lawndale. Johnson is articulate, self-assured, and, at over six feet tall, a commanding presence in a room full of teachers and students. (The photograph on page 40 shows him leading the “principal’s choir.”)

“This is a community of potential,” Johnson says. “It would appear that nothing can grow here. I’m here to change that.”

And it would appear, at the beginning of his second year, that he has done that.

Fehrenbach noticed the change “just after Christmas,” he says. “The tone shifted radically. Kids in the classrooms were actually smiling; there was less shouting.”

There is an air of discipline—in the old sense of the word, order—in the school that had not been there the previous two years. All children are now wearing uniforms with shirts tucked in. Test scores have improved markedly. Among Chicago’s 91 charters, Howland showed the fourth-best improvement on ISAT composite test scores in 2010, jumping from a 49.8 percent to a 60.8 percent passing rate.

With Hannon’s help, Johnson began conducting weekly teacher and staff training sessions. They brought in a curriculum instructor, and he brought in trainers for grade-level teacher meetings. The eventual goal, says Hannon, is to send teachers to a Lasallian Leadership Institute. Run by the Christian Brothers and named for the founder of the order, these three-year programs, including a one-week intensive training session during the summer and several weekend sessions, are meant to introduce teachers and administrators to the Lasallian mission and show them how to implement it in their classrooms and schools.

Though the church/state question will no doubt live on, for now the leaders of the Catalyst charter experiment are convinced that the essence of their San Miguel schools can work in a public school setting. “We will live the values and virtues of the Christian Brothers without speaking the words,” says Chaun Johnson. “And that will open the doors so that our education plan can work.”

Johnson represents what it is that the Catalyst backers believe is the point of their charter: they bring a Catholic ethos, not the catechism, to children. And they educate them in reading, writing, and ’rithmetic.

While admitting that Howland still has challenges, Catalyst leaders believe they have turned the corner. And they have learned some hard lessons.

First, they did inadequate relationship building in the community. As many of those involved in the early discussions with the city have said, the Christian Brothers were “naive” or “stupid” to have taken on the Howland project before getting to know the neighborhood.

“A second mistake was starting Howland with grades 4 and 5,” says Ed Siderewicz. Taking on older kids, the brothers now know, requires the kind of knowledge of and relationships with a community that they did not have in North Lawndale.

Third, leadership and staff must understand the model. “Reforming the academic leadership team was our biggest challenge,” says Hannon.

And finally, “we worried too much about the church/state issue,” says Terry Toth. “We run a public school. We don’t have crucifixes on the wall. We don’t teach religion. We teach truth and honesty.”

Is there any one ingredient of success in these matters?

Says North Lawndale College Prep charter school’s John Horan, “You have to have a community of full-grown adults who understand the culture piece and the academic expectation piece. The kids will come around.”

“Hard work,” adds Hannon, who has all but lived at Howland for the last year and shows no signs of slowing down.

And Terry Toth remains bullish on Catalyst’s future. “We’re working on a strategic plan,” he says. “We’re trying to button down the academics and get more consistency there. We’ve taken faith-based and made it values-based. We’re even exploring opportunities to add more schools. The Circle Rock campus got more traction because of Rock of Our Salvation Baptist church and there was community support. So, we’ll be looking for similar things as we expand: a pastor, a supportive community, etc. We’ve learned that you can’t just plop a school down in a neighborhood and expect it to work.”

Brother Mike Fehrenbach would agree. “Be good citizens,” he told a Circle Rock graduation class. “We pledge to stand by you. Failure is not an option.”

Peter Meyer, former news editor at Life Magazine, is currently senior policy fellow with the Thomas B. Fordham Institute and contributing editor at Education Next.

Additional photographs of the Catalyst Schools are available here.

As education policy churns through fad after fad, merit pay is really hot right now. The U. S. Department of Education asked states to include proposals for implementing teacher merit pay—pay based on classroom performance—in their 2010 applications for Race to the Top (RttT) monies, and many applicants promised action on this front. In Washington, D.C., former schools chancellor Michelle Rhee negotiated a strikingly original merit-pay plan, despite strong union opposition. According to the latest Education Next poll, public support for merit pay gained significant ground over the past year and now outdistances opposition by a 2:1 margin.

Replacing the standardized salary schedule, where the only factors that determine teacher salaries are the number of years on the job and academic credentials, seems a worthwhile goal. In theory, pay-for-performance plans both provide a clear monetary incentive to teachers to find the best way to motivate and instruct their students and, over the longer term, attract and retain those more-effective teachers who wish to work in a field that rewards professionals for the quality of their efforts. But enacting high-quality performance pay plans in the United States is easier said than done. Last year, the Florida legislature enacted one of the more stringent proposals any state has ever attempted—only to have the bill vetoed by Governor Charlie Crist as a way of jump-starting his ultimately doomed bid to become Florida’s first independent U.S. senator. That is not the only time a merit pay bill has seemed on the verge of success, only to founder or be undermined by the need to compromise. In general, merit pay plans are more likely to be symbolic than substantive and more likely to be promised than delivered.

Most often, they are not even promised. Even if one counts the most token of performance pay plans, they are to be found in no more than 500 school districts out of some 14,000 districts nationwide, a mere 3.5 percent of the total.

When new merit-pay plans are proposed, teachers unions often block their enactment or water down their provisions. In Cincinnati and Philadelphia, for example, merit pay policies were blocked just before they were about to be implemented. Denver’s Professional Compensation for Teachers (ProComp) plan, widely heralded as the leading national example of performance pay, awards more money for earning another degree than for demonstrated performance in the classroom. In Houston, merit was defined so broadly that it included an overwhelming majority of the teachers. In Florida, Iowa, and Texas, the legislatures have encouraged local districts to enact performance pay plans. But unions have been able to dissuade local districts from participating in the state-authorized programs. Only a handful of Florida districts participate in merit pay, for example, even though state funds cover the cost of the initiative.

A strong, well-designed merit-pay plan requires more than offering a bonus to high-performing teachers while paying the remainder according to the standard schedule. To be truly effective, pay for performance must mean in education what it does in other industries—salary increases for the successful, and salary reductions, even dismissals, for poor performers. State laws governing teacher tenure in most states make implementation of such plans unlikely.

All of this leads us to measured skepticism about the merit of merit pay, unless it is coupled with school choice innovations hefty enough to instigate sustained competition among schools and school sectors. Only then would local districts have the incentive to both lobby states for changes in state laws and to negotiate tough contracts with teacher unions. Only then would they find it important, if merely to retain their student enrollments, to structure their pay systems so as to attract top-notch employees and give them strong incentives to strive for excellent performances.

But we have covered a lot of ground very quickly. Let’s step back and consider carefully the propositions we have set forth.

Does It Work?

High-quality research on this topic within the United States is sparse and results are mixed. Matt Springer and his colleagues at Vanderbilt released a study recently on a well-designed randomized trial of a merit pay experiment in Nashville. The program involved bonuses of up to $15,000, which would presumably be large enough to affect individual incentives. Yet virtually no effect was seen on test scores (outside of 5th-grade math, an effect that disappeared for those same children the next year). That said, the Nashville study did not examine long-term effects on the composition of the teacher workforce.

The Bloomberg administration in New York City made headlines in late 2007 by announcing a pilot merit-pay initiative, the School-Wide Performance Bonus Program. The New York City Department of Education randomly assigned eligible schools to treatment or control groups, which has enabled scholars to conduct rigorous evaluations. Early results with respect to student achievement are not promising overall, although the program appears to have had a positive impact in schools with fewer teachers (see “Does Whole-School Performance Pay Improve Student Learning?” research). The researchers theorize that the group benefit feature of the merit pay program made it unlikely that it would have an impact on teacher behavior in any but the smallest schools.

The international evidence on performance pay is more encouraging, including a recent worldwide look that indicates that students learn more in countries with performance pay plans, all other known factors held constant. Ludger Woessman (see “Merit Pay International,” research) looked at 27 Organisation for Economic Cooperation and Development (OECD) countries and found that students in countries with some form of performance pay for teachers score about 25 percent of a standard deviation higher on the international math test than do their peers in countries without teacher performance pay.

Union Roadblocks

If merit pay seems promising (and certainly not harmful), convincing tests of its performance are difficult to undertake within the United States, simply because merit pay proposals typically end up being blocked, co-opted, or diluted by established interests. Admittedly, it is not easy to identify the various instances where merit pay has been proposed but then blocked from enactment, and therefore we cannot provide an explicit enumeration. In all likelihood, most potential proposals are never articulated, simply because likely sponsors regard the cause as hopeless. When 96.5 percent of all districts rigidly follow a standard salary schedule, it takes an energetic and devoted innovator to brave the odds and try to break from tradition nonetheless.

Still, there are several telling examples of established interests blocking merit pay proposals. Governor Mitt Romney proposed merit pay in Massachusetts back in 2005–06, as part of an education budget that included tens of millions in new spending. That proposal went down to defeat; as the Lowell Sun reported, “the Massachusetts Teachers Association [MTA] and United Teachers of Lowell opposed the idea. Catherine Boudreau, president of the MTA, called teacher bonuses ‘inequitable and divisive.’”

Philadelphia tried to institute a pilot merit-pay program in 2000, but later ditched the initiative, “calling it too expensive, too difficult to administer, and a failure at giving teachers useful feedback” according to the Philadelphia Inquirer. Then, in 2006, Philadelphia received a $20.5 million grant from the U.S. government to develop a merit pay program. Said the Inquirer, “At the time, the federal grant was announced with much fanfare—the union would be the district’s partner, officials said, ensuring the plan would succeed where others failed. But the deal fell apart.” The local union abandoned the program in the face of a “surprise $180 million budget deficit,” and the district gave the money to charter schools instead.

Another example comes from Cincinnati. That city’s merit-pay plan proposed in 2002 was overwhelmingly voted down by teachers (1892 to 73), even though it did not base bonuses on student test scores. As Education Week noted, the plan “was based on an extensive evaluation system, which determines whether teachers advance in five career categories…. The evaluations entail multiple classroom observations by fellow teachers and administrators and portfolios that include logs of parent contacts, lesson plans, student work, and more.” Education Week quoted a former associate superintendent of the Cincinnati schools, who blamed the proposal’s failure on the fact that it “would have applied to nearly all teachers, rather than allowing veterans the choice of opting into the new system.”

In Alabama, the state’s “Race to the Top” application originally proposed merit pay and a “new salary schedule that would give more money to math, science and special-education teachers,” but that portion of the application was deleted, reported the Press-Register (Mobile), “after Alabama Education Association leader Paul Hubbert wrote state Superintendent Joe Morton a letter…opposing them”

If special interests fail to block a merit pay program, they may still be able to make it temporary. A Little Rock, Arkansas, performance-pay program lasted only three years and was not renewed by the local school board, despite evidence of positive effects on student achievement in math, reading, and language. Similarly, the Alaska School Performance Incentive Program was canceled after three years.

Special interests are also able to repeal merit pay based on putative budgetary constraints. The state of North Carolina suspended incentive awards to high-performing schools in 2008–09 due to budget problems. Winston-Salem/Forsyth County, North Carolina, suspended its bonus program due to budget difficulties as well.

Unions have been similarly successful at preventing local districts from participating in statewide programs, as the experience in Florida, Iowa, and Texas shows. Florida’s “Merit Award Program” provides state money to local school districts. According to the Florida Department of Education, “Each district will determine an amount equal to at least 5% and no more than 10% of that district’s average teacher salary to be awarded to all of the top performing personnel in the district, regardless of years of experience.” Even though this program involves free money from the state for districts to hand out to teachers, the political forces opposing merit pay were able to prevent 88 percent of Florida districts from participating in 2009.

Similarly, Iowa’s statewide Career Ladder and Pay-for-Performance grant program was passed in 2007, but only 3 Iowa districts, out of 360, bothered to apply. Only 20 percent of Texas districts opted into the District Awards for Teacher Excellence program in 2009–10. In other words, as a result of political opposition, the vast majority of school districts, even in conservative Texas, turned down extra money from the state rather than adopt merit pay.

Merit Pay in Name Only

When interest groups succeed in diluting or co-opting a merit pay plan, the plan ends up rewarding teachers mostly or entirely for inputs (e.g., professional development, graduate degrees, national certification) rather than for outputs (test scores, graduation rates, or even supervisor assessments).

One example is Arizona’s Classroom Site Fund (CSF), a mandatory statewide program that involved a couple of new taxes. Districts had to “allocate forty per cent of the monies for teacher compensation increases based on performance and employment related expenses, twenty per cent of the monies for teacher base salary increases and employment related expenses and forty per cent of the monies for maintenance and operation purposes.”

According to a 2010 report from the Arizona Auditor General, out of 222 districts receiving CSF funding, the auditor could identify only 29 “with strong performance pay plans that did a good job of linking teacher performance pay to student achievement.” The report noted that “allowing districts the freedom to determine performance pay goals can help gain district and teacher buy-in,” but that such freedom “has also led to inconsistent performance pay plans and to situations in which teachers receive similar performance pay for significantly different levels of effort and related performance results.”

One example from the auditor’s report deserves to be highlighted:

One district awarded performance pay to eligible employees if freshman students’ algebra test scores increased by at least 10 percent between a pre- and post-test. The actual increase in test scores was almost 90 percent. Since the pre-test is given to freshman students who have never been exposed to algebra and the post-test is given to them after receiving a full year of algebra instruction, it should be expected that scores would increase significantly more than 10 percent.

In other words, algebra teachers were being rewarded merely for getting students to learn 10 percent more about algebra than they knew before studying that subject at all. This is not a high hurdle to clear.

Denver’s ProComp program has been heralded as a political and policy success. Then Senator Barack Obama said, “Cities like Denver have already proven that by working with teachers, this can work, that we can find new ways to increase pay that are developed with teachers, not imposed on them and not just based on an arbitrary test score.” But the Denver ProComp program may be less than meets the eye. For one thing, it exempts teachers hired before January 1, 2006, from having to join, which means that the vast majority of teachers whose pay depends on seniority rather than on merit are able to keep their old pay structure in place. And if older teachers opt to enter the ProComp program, they keep their old base salary; the ProComp program merely offers them a chance for bonuses on top of that old salary.

The ProComp program also rewards the old definition of “merit” more immediately and to a greater extent than it does anything that improves student achievement. The largest monetary award is for earning a graduate degree: a $3,300 permanent salary increase plus a tuition or student loan subsidy of $1,000 per year for up to four years. By comparison, teachers receive a one-time award, not a bump up in base salary, of up to $2,403.26 if their students exceed “district expectations” for student growth.

Moreover, as Paul Teske, a principal evaluator of the ProComp program, noted in the Christian Science Monitor, bad teachers face no penalty under the ProComp or similar merit-pay programs: “I guess your salary stays low, and maybe that sends the message that you should look at another career. But ProComp doesn’t directly address that.”

The federal Teacher Incentive Fund (TIF) provides grants to school districts that promise to develop merit pay programs “for raising student achievement and for taking positions in high-need schools.” Currently, the Department of Education’s website lists 33 TIF grantees, including some small districts and a few major city districts. But these programs may also end up being diluted or co-opted.

For example, the TIF program in Charlotte-Mecklenburg (North Carolina) includes substantial bonuses for professional development, working at hard-to-staff schools or in hard-to-staff subjects, and for taking on leadership roles. To the extent the program involves student achievement, it bases awards on “student learning objectives” as “created by individual teachers, with the approval of site-based administrators”; these objectives “will be measured by a combination of existing assessment instruments, and teacher designed tools,” as well as by state standardized tests. The superintendent of Charlotte-Mecklenburg schools recently announced a plan to bring performance pay to the entire district.

Perhaps it is desirable to have teachers receive more professional development, work in hard-to-staff schools or subjects, and assume leadership roles, but these are inputs, not student outcomes. The bulk of the Charlotte-Mecklenburg TIF program, like many such programs, is MPINO—merit pay in name only.

Some locales have diluted the merit pay concept by making the bonuses to teachers small and setting the bar for receiving the bonuses low, thereby converting merit pay into something approximating an across-the board pay raise.

For example, the Texas Educator Excellence Grant (TEEG) program began in 2006–07 and ended after the 2008–09 year; it was funded at approximately $100 million per year. After analysts at the National Center on Performance Incentives (NCPI) reported no positive effects on student test scores, the Dallas Morning News declared the program a failure. NCPI report coauthor Lori Taylor speculated that “one possible cause of the program’s failure was that bonuses were relatively small and were given to most teachers at each school—about 70 percent—so that the incentive for individual teachers to push for higher scores was ‘relatively weak.’ In addition, campuses that qualified already had to be higher performers, so it was difficult to register much improvement.”

The same thing seems to be happening in Houston, where a merit pay program has existed since January 2007. The district announced financial awards totaling $40.4 million in 2010. The district’s webpage notes, “in all, 15,688 HISD employees received performance pay [in 2010], ranging from $25 to $15,530. That’s 88 percent of eligible HISD employees.”

Minnesota’s oft-heralded “Q Comp” program offers yet another example of a “merit pay” program that ends up as an across-the-board pay raise. As the Minneapolis Star Tribune recently reported, “In 22 school districts whose Q Comp practices were analyzed by the Star Tribune in 2009, more than 99 percent of teachers in the program received merit raises during the preceding school year. Only 27 of the roughly 4,200 teachers eligible did not get a pay raise.”

The New York City School-Wide Performance Bonus Program mentioned above may also have been undermined by its structure. Some 180 schools were eligible in the 2007–08 school year for a collective $14 million in bonuses, or $3,000 per union teacher, if they met test score goals established by the district. In a key factor that enabled the plan to draw union support, committees composed of a principal, a person of the principal’s choosing, and two union representatives were allowed to decide how the bonuses should be distributed at any given school. Researchers identified a number of drawbacks to the program design, including the possibility that bonuses based on school-wide improvements weaken the incentives for individual teachers to increase their efforts.

Making Merit Pay Work

The prospects for merit pay are not promising, despite both theoretical and empirical reasons for expecting that the programs would produce positive results for students. Our findings are consistent with the theory that school districts are not primarily educational institutions where policies are organized around maximizing student achievement. Instead, they are best understood, at least when it comes to compensation policies, as political entities shaped by powerful interest groups, including organized groups of employees.

Viewed in that light, it is unsurprising that public school systems have relatively little interest in authentic merit-pay programs. If some teachers could earn improvements in their wages and working conditions from their own efforts rather than from the efforts of their organized representatives or affiliated politicians, then more-effective teachers would have little reason to support the unions financially or politically. Their interests would be at odds with those of less-effective teachers. In short, the single salary schedule by which almost all public school teachers are paid is essential to the financial and political power of established interests.

One way to diminish the power of established interests and permit the adoption and implementation of merit pay is to expand choice and competition in education. If students choose their school, those schools have incentives to adopt and implement policies and practices that will improve their quality and attract students as well as the resources they generate. If merit pay systems help attract and motivate effective teachers, schools in a more competitive environment will have incentives to adopt those systems. They are more likely to design and maintain merit pay systems in a sensible way, since their revenue depends on it.

Schools that already compete for students appear more open to including merit pay in their personnel policies. According to University of Washington’s Daniel Goldhaber and his colleagues, charter schools are more likely than traditional public schools to use merit pay. Michael Podgursky, professor of economics at the University of Missouri, looked at data from the 1999–2000 Schools and Staffing Survey and found that when school administrators were asked whether they used salaries to reward “excellence,” only 6 percent of traditional public school administrators answered yes, while “the rates for charter (36 percent) and private schools (22 percent) were much higher.” Even those charter and private schools without a formal performance-pay plan are typically able to offer higher salaries to teachers they hope to retain and, as important, to readily dismiss teachers deemed ineffective.

Attaching continued employment and level of compensation to job performance is something that frequently occurs among private enterprises in competitive markets. The difficulty with merit pay in education is that it attempts to simulate a market-based practice in a nonmarket environment. None of the forces that cause organizations to seek effective merit pay systems, or to maintain and alter them effectively over time, exist in public education.

Imposing merit pay on an unwilling education system is like trying to get kids to eat their vegetables when the kids are 25 years old and stronger than their parents. No matter how nutritious green beans may be, powerful adults who don’t want to eat them can usually keep them off their plates and can almost always keep them out of their mouths.

Stuart Buck is a doctoral fellow in education reform at the University of Arkansas. Jay P. Greene is professor of education reform at the University of Arkansas and a fellow at the George W. Bush Institute.

Five years after Hurricane Katrina, the New Orleans public schools bear little resemblance to the disintegrating system that was further undone by the catastrophic flood. Two-thirds of city schools in 2004 were rated “Academically Unacceptable” under Louisiana’s accountability standards; in 2010, about 4 in 10 rate that designation, and the percentage of students attending a low-performing school has fallen by half, from 67 percent to 34 percent. Most striking of all, nearly three-quarters of public school students attend charter schools, proportionally more than in any other U.S. city.

Just weeks after the storm, officials turned the city’s failing schools over to the state-run Recovery School District (RSD) and gave the RSD five years to turn them around. That deadline was reached last December, and a vote by the state school board has extended the RSD’s reform effort, albeit with modifications that promise greater autonomy to schools that meet performance targets and create a process for qualified operators to take over failing schools. The December vote was a victory for charter schools and the RSD, one that boldy advances a school reform model as innovative as it is controversial.

District in Recovery

For decades, the deterioration of the New Orleans public school system had been shocking and seemingly inexorable. Students graduating with honors were sometimes incapable of elementary mathematics and some were barely able to read. One high-school valedictorian failed the graduate exit exam and then failed it some more—five times all told—and this was the school’s top student. Deferred maintenance and contract fraud ensured that the system’s physical infrastructure was as degraded as its instructional capacity. The system was bankrupt and the payroll so padded with no-shows—some of them deceased—that the FBI had set up a satellite branch within the school board’s central office. The hurricane was the coup de grâce. Some 110 of 127 schoolhouses were completely destroyed.

But ruin so extreme bred opportunity.

The RSD had been established in 2003 to manage “recovery” from academic failure, not from Hurricane Katrina, as the name is sometimes taken to imply, but had seized only five New Orleans schools before Katrina. After the storm, the RSD took control of an additional 63 deemed in need of radical intervention. The elected Orleans Parish School Board (OPSB) retained authority over the system’s 16 still-viable schools, an administrative domain that shrank further as several of the best schools fled central control for the greater autonomy that comes with charter status. Today, the majority of OPSB schools are charters (see Figure 1). Further erosion of the board’s legitimacy came with the jailing of its former president for bribery.

In a similarly pivotal blow to the old order, with teachers scattered to 50 states and schools shuttered for the 2005 fall term, the OPSB discharged the 7,000 employees who had answered to it prior to Katrina, effectively nullifying the system’s contract with United Teachers of New Orleans. When the collective bargaining agreement formally expired at the end of the 2005–06 school year, it was not renewed.

Freed from union rules and OPSB central-office control, the RSD was able to act on its conviction that improved performance lay in spinning off as many schools as possible and chartering them as independent institutions with open-enrollment admissions policies and citywide catchment areas. Critics on the left accused Louisiana of implementing a version of the “shock doctrine,” whereby disaster is exploited to rescind worker protections and other strands of the social safety net. Critics on the right lamented that the Bush administration and its allies within the parochial school establishment failed to go even further and make private school vouchers a bigger part of the new regime.

Five years later, the city’s bet on charter schools had begun to pay off. The average rate of improvement in the New Orleans public schools stood at three to four times the statewide rate, despite persistent poor performance by several schools. For a change, extraordinarily good things could be said about New Orleans’s traditionally atrocious public school system.Wake-up Call

Forced to compete for students and rank, the New Orleans schools were jolted from a decades-long coma. The awakening coincided with efforts in reform-minded cities like New York, Long Beach, California, and Washington, D.C. But what  was  distinctive about New Orleans was that the dynamic tension among schools was built into the system’s new polycentric administrative structure. The old apparatus of central control had not, as in other cities, merely been tweaked in the name of reform; it had been scrapped. Under the old order, the all-powerful school board and central office had seemed to view the district more as an adult jobs program and dispenser of patronage-based contracts than as a source of education for young people. Now, by design, no single apparatus of power—not OPSB, RSD, or the charter schools and charter management organizations that answered to them and to the Louisiana Board of Elementary and Secondary Education (BESE)—could assert hegemony and dominate the others.

That made New Orleans a test not just of cutting-edge instructional practices but of variant administrative models as well. The city became a laboratory for the reinvention of its school system and, as was attested to by the enthusiasm of major foundations and the Obama administration, a crucible for ideas that might well be replicable in other cities.

As reformers hoped, the opportunity attracted a raft of independent school service providers ranging from charter management organizations to firms that aligned curricula with state standards and then developed metrics for measuring individual student achievement on a monthly or even weekly basis. Teach For America and the New Teacher Project saw opportunity and beefed up their presence in New Orleans, as did a homegrown organization called Teach NOLA. The Louisiana Association of Public Charter Schools gained prominence as a deft legislative advocate for what was being called the New Orleans reform model. The largest of the independent reform groups, the nonprofit New Schools for New Orleans (NSNO), developed an array of services, subsidies, and other forms of support. To plug the human capital deficit in a city still depopulated by Katrina, NSNO began training prospective school leaders and directors as well as teachers. It also sponsored a small nonprofit to engage and inform parents about student choices in the new landscape. By 2010, NSNO had incubated 10 citywide, open-admission charter schools, the basic integer of local reform, and provided key personnel and services for dozens more.

Katrina spawned a gamut of visionary ideas for the transformation of New Orleans. They ranged from land-use plans to flood protection to the development of neighborhood health-care clinics to economic development and governance proposals. Many died at inception, undone by the impulse to re-create the old order before attempting its improvement. School reform was the exception. A sense of moral obligation combined with hard work and sheer exasperation to make it the most far-reaching achievement of the post-Katrina era. The decent public education long denied New Orleans youth was framed as a civil right at least as fundamental as the access to jobs, public accommodations, and polling places that had been milestones in an earlier generation’s fight to overcome segregation. The numbers show that charter schools were the barricades from which a new struggle was being waged successfully (see Figure 2). Parents, initially skeptical about school reform efforts, or accustomed to thinking of them as concessions aimed largely at luring parochial and private school students back into a low-income, black-majority system, flocked to the new schools, even lining up in pre-dawn hours to assure a child’s admission. Alone among American cities, New Orleans was actually beginning to close the much-discussed “performance gap” among students of different racial, ethnic, and economic backgrounds. A poll in late autumn 2010 by Tulane University’s Cowen Institute for Public Education Initiatives found that 60 percent of New Orleans residents opposed returning the schools to OPSB. Small wonder then that many politicians had loosened their ties to teachers unions and school system contractors. Change was in the air and the implications were revolutionary.

Sustaining Momentum

Now, the question, as keenly studied by chartering’s foes as by its friends, is this: Can the early success be sustained? The challenges remain numerous and daunting. There is concern that school reform’s bountiful harvest in the half decade since Katrina has been low-hanging fruit and that further gains—even with sharp improvement, the system remains subpar—will be much more difficult. Looking ahead, Neerav Kingsland, a Yale Law graduate and strategist for NSNO, talks about “Charter Issues 2.0,” the problems that arise on the way from being 10 percent of the system to being 80 percent of the system, the next and far more demanding phase of work.

For the nation’s foremost experiment in charter schools to rest even briefly on its laurels would be to risk setbacks, Kingsland and like-minded reformers contend. Loss of momentum would be pounced on by now-disenfranchised partisans of the old regime eager to buttress their claim that the rising test scores are somehow bogus or, in any event, temporary, merely a blip. That argument has been made by Larry Carter, president of the United Teachers of New Orleans. Like other skeptics, Carter seized on a 2010 report from Stanford University’s Center for Research on Education Outcomes that portrayed many charter schools as doing no better, and indeed sometimes worse, than traditional schools nationwide. Carter rushed into print in New Orleans’s daily newspaper, The Times-Picayune, with an editorial saluting the Stanford study as proof of failure, but without mentioning the parts of the report that identified charters in New Orleans as a sharp exception to the national numbers and particularly successful with low-income students. In light of rearguard attacks of this sort, the only way to ensure that the system remains performance-driven, many of reform’s proponents believe, is to push the New Orleans model—predicated on open-admission, citywide charter schools—all the way to scale. That means encouraging the RSD to complete the chartering of its entire portfolio of schools; it also means resisting return of a still-shaky school system to OPSB, with or without a collective bargaining agreement. Above all, sustaining charter-based school reform means taking very seriously the criticisms that have been lodged against it.

The Search for a Governance Model

The 2005 legislation that designated New Orleans a district in crisis and placed more of its failing schools under state control gave the RSD five years to achieve recovery. The timetable guaranteed that school governance would emerge as a focus of debate on Katrina’s fifth anniversary. Eli Broad, whose foundation has committed millions to the reform effort, put the governance question at the top of the agenda as schools reopened for the 2010–11 school year:

The most important areas in which we think the city should focus going forward are putting in place a sustainable governance structure, continuing to develop and support teachers and leaders to become long-term, high-performing employees and continuing to improve the lowest-performing schools.

Last December BESE decided to extend the RSD’s shelf life rather than return the schools to OPSB control. In the run-up to the December decision, public interest swelled and rhetoric heated up. Opponents of the state’s post-Katrina intervention rallied to the cry of “local control,” which usually meant restoring power to the school board or something like it. The argument carried a racial subtext, sometimes explicit, more often coded. The bureaucrats in a white-majority state were cast as having usurped administrative power over a district in which 9 out of 10 students were African American, as were many teachers, politicians, and contractors.

Another theme popular among advocates of local control was the contention that RSD’s school performance gains were somehow illusory or rigged: students with special needs were being turned away from schools and those with disciplinary problems were being expelled to keep performance scores high, critics insinuated. The argument lost some of its political punch when 2009–10 enrollment figures revealed that the schools overseen by the OPSB, not the RSD, have the lowest proportion of special needs and behaviorally challenged students.

State Superintendent of Education Paul Pastorek, shortly after Labor Day, pointed the way for BESE’s December decision. The gist of his recommendation was that RSD would retain control of its current portfolio of schools for at least another two school years. At that point, schools that had met or surpassed minimum standards could return to local governance, if—the big if—they chose to do so. Pastorek’s further proviso was that local capacity to administer the schools would be reviewed before such transfers were approved. Many, if not most, eligible schools are expected to resist a return to OPSB control. In a late amendment to his plan calculated to impose greater accountability on the RSD, Pastorek advocated giving OPSB and others a crack at taking over not just successful schools, but also those that are still failing after five years in the RSD portfolio.

The December vote was not a foregone conclusion. Some board members were inclined to override Pastorek’s recommendation and restore the entire city system to OPSB control. But former OPSB and BESE board member Leslie Jacobs, widely regarded as the founder of Louisiana’s school reform movement, correctly predicted that BESE did not have the votes to oppose Pastorek.

Darryl Kilbert, the superintendent hired by OPSB to manage its small portfolio of schools, portrays the current transitional arrangement as an erosion of democracy itself and espouses restoration of “community control.” He tactfully makes clear that community control need not necessarily mean OPSB control, but clearly assumes that it will.

The countervailing observation is that the locus of democratic control has merely shifted, from an elected school board to an elected governor and a partly elected, partly appointed BESE. The mantralike criticism that a diminished OPSB means control is less “local” ignores the fact that the once all-powerful seven-member school board has been augmented by a growing cohort of charter school board members numbering in the hundreds. (The Left counters by deploring the charter schools as “privatized,” notwithstanding that most of them observe an open-enrollment admissions policy and that they, like all public schools in Louisiana,  are publicly authorized, funded, and evaluated. By statute, their meetings must also be open to the public, though critics say access is sometimes grudging.)

While its argument for regaining control of the schools rested on the principle of local control, a chastened OPSB also pointed out that it had instituted financial reforms since the system’s bankruptcy prior to Katrina.

But the broader political context was aligned in ways that favored continuing the reform effort, at least for now. Under the New Orleans city charter, the school system is a separate entity that does not answer to the mayor, but the incumbent administration, like the state education bureaucracy in Baton Rouge, was and remains vehemently opposed to cutting it short.

“There will be no turning back,” New Orleans mayor Mitch Landrieu said to cheers in his inaugural address in May 2010. He was reiterating a slogan that had been embedded in his campaign platform. If reform were to fail, he asserted in a network TV appearance in late September, it would be precisely because politics, perhaps especially racial politics, had eclipsed the commitment to improve the education of children. Landrieu is white and a Democrat, the first white mayor of New Orleans since his father served in that capacity in the 1970s, but he was elected with overwhelming black support. Governor Bobby Jindal, a conservative Republican and a devout Catholic, is even less disposed to resurrect the old regime. Indeed, he is a proponent not only of charter autonomy but of vouchers, which though ardently desired by the parochial system, are so far only a token presence in the New Orleans schools landscape (see “In the Wake of the Storm,” features, Spring 2010).Paul Vallas, who as superintendent of the RSD since 2007 has lengthened both the school day and the school year, sees technical as well as political reasons why charters are here to stay. “You can’t turn back. Charters are authorized by the state,” Vallas told PBS Newshour during a July 2010 appearance. “The state would have to not renew them. The great thing about this system is, it’s really going to be hard to dismantle what’s been created.”

The influential Jacobs agrees. OPSB couldn’t roll back the clock even if it wanted to, Jacobs contends; the charter school constituencies—the families who use them—won’t let it happen.

And yet Jacobs, like many others, including Eli Broad, sees eventual return to an upgraded form of local control as both inevitable and wise. In the interim, every governmental entity with a management role in local schools, and that would include BESE, must maintain a local presence to facilitate citizen access, she told an independent citizens forum on school governance that met throughout the summer. Longer term, she believes any resolution of the governance question must observe two categorical imperatives: One is that any and all decisions must be based on whether they measurably improve the quality of the education being provided to children. The other is that the management of schools must be cleanly separated from the business of authorizing and evaluating them.

Extrapolating from these core values, Jacobs envisions a school board–like body, perhaps the OPSB itself, eventually recovering the power to authorize charters, reorganize failing schools, set policy consistent with state mandates, and provide systemwide services. Actual management of schools would be left to autonomous charter boards, each of which comprises a school “district” under the current arrangement.

Building Anew

As the December vote was approaching, Jacobs was also grappling with the question of whether central administrative functions should include facilities management, or whether that responsibility should lie with the schools that occupy assigned campuses. The real estate is owned by OPSB and is subject to reconstruction or replacement now that the city has finally settled with the Federal Emergency Management Agency (FEMA) for a post-Katrina allocation of construction funds totaling $1.8 billion—big, big money in a relatively small city like New Orleans (see sidebar).

Where those schools should be placed and what they should look like has long stirred debate. Some factions have clamored for a return to “neighborhood schools.” To some, this is code for an antireform agenda, given that citywide open access is one of the hallmarks of the new generation of charter schools since Katrina. That open access is a deliberate and effective assault on racial inequity associated with the segregation era is an irony not lost on reform advocates. In debating the issue, they point out that charters with open-access admission policies are an option already available to neighborhood residents; for admission to most they need only show up on time and enroll. Moreover, reform advocates note, basing admissions on geographical boundaries is an exclusionary practice, all too redolent of the days when low-income students of minority background desperately sought to escape from “slum” or “ghetto” schools and gain access to the generally superior schools in “good” neighborhoods from which they had been barred.

The neighborhood schools movement has found friends among some of the city’s more progressive urban planners. The master plan for reconstruction of the school system after Katrina envisions schools as centers of the adjacent community. At a time when budgets are tight, obesity epidemic, and fuel costs likely to rise, schools at the center of walkable communities are seen as both healthful and thrifty. School-centered communities also further neighborhood cohesiveness, the argument goes. To that end, the Orleans schools master plan calls for bundling several community services within or adjacent to new and reconstructed schools—a library branch, a wellness clinic, a community garden, and a senior center, for example.

The Money Factor

No discussion of school reform in New Orleans is complete without acknowledging that notable gains have occurred at a time of unusually high levels of government financial support, chiefly drawn from special funds set up in the aftermath of the Katrina disaster. Those dollars nearly doubled per-student allocations in New Orleans, lifting the figure above $12,000, even without factoring in support from foundations and individual donors (see Figure 3). That tide of money has now begun to ebb. It was hoped that reversion to more normal funding levels would be mitigated by federal Race to the Top (RttT) money, but Louisiana was not selected in the program’s first two rounds, in part, it was assumed, because upstate districts and teachers unions were not wholehearted in their support for RttT goals.

New Orleans has, however, secured $28.5 million in federal “i3” funds for educational innovation. The award, announced over the summer of 2010, will go to the RSD and to NSNO primarily to lubricate reorganization of failing schools. To test the replicability of the New Orleans model, some of the money will be used to help launch charter schools in Memphis. On the home front, NSNO is committed to implementing i3’s goal of reorganizing the lowest-performing 5 percent of failing schools. The intended uses of the i3 money align with an evolving vision of philanthropy’s role. As Broad put it,

Foundations can continue to play an important role in enabling school districts and states around the country to understand how and why New Orleans has made better relative academic gains in such a short period of time, and to encourage them to adopt similar approaches. We’ve only begun to unlock the lessons this city holds for education reform nationwide.

In early December 2010, notwithstanding a lawsuit threatened by OPSB, BESE accepted Pastorek’s recommendation to extend the current reform paradigm. The vote was preceded by histrionics at times reminiscent of pre-Katrina meetings of the Orleans school board at its most chaotic and dysfunctional. From the speaker’s rostrum, one OPSB member warned that a vote for Pastorek’s plan would be an act of criminal malfeasance that would trigger “civil war,” an indication that regardless of the board’s decision, the political battle was far from over.

Where to Go from Here

Amid changes as exciting as they are fragile, this much seems clear to the reform community: Even briefly settling for today’s improved performance levels is to avail critics of the opportunity to say that school reform has stalled after early gains that were easy and perhaps unsustainable (see Figure 4). It would be to settle for schools that are, not excellent, but merely “good enough.”

Efforts by the old order to claw back power are portrayed as only the most obvious threat to the gains achieved in New Orleans. The more insidious threat, reformers contend, is for schools and the communities of students and parents they serve to get comfortable with a still-inadequate status quo. A stubborn loyalty to the school they know, and indeed may have helped build, can abort the wrenching changes that may be required for a school to become truly excellent.

Reform advocates call it “churn,” the business of aggressively and systematically zeroing in on the least successful schools, ousting failed managers, and reorganizing the schools as open-enrollment, citywide charter schools. Churn is “disruptive,” a term of approbation in the school reform lexicon. But disruption breeds resistance. Even badly failing school administrations sometimes secure the affection of parents and students uncertain that striving for a truly excellent school will necessarily lead to improvement of the mediocre institution with which they have grown comfortable. That psychology is what for a time bedeviled the process of replacing a popular principal at the International High School of New Orleans, a BESE charter, with a controversial but dynamic former superintendent of the New Orleans system. In other school settings the resistance is communitarian or racial. The delicate and sometimes unpleasant politics of churn are the reason many reformers question whether an elected body, such as a traditional school board, has the gumption to handle tasks as potentially unpopular as declaring schools to be failures and handing them over to more capable managers, or shuttering them altogether.

Resolving the issue of governance will be the biggest test ahead for cities engaged in Charter Issues 2.0. At stake is not just the credibility of the reform movement but the prospect, at last, of convincing America that an excellent education is a civil right worth the kind of struggle that so far is exhilarating New Orleans with the possibility of transformational change.

Jed Horne educated two sons in Orleans Parish public schools. He is the author of Breach of Faith: Hurricane Katrina and the Near Death of a Great American City.

Teach Like a Champion: 49 Techniques That Put Students on the Path to College
By Doug Lemov
Jossey-Bass, 2010, $27.95; 352 pages.

The first five words of Doug Lemov’s book Teach Like a Champion are “Great teaching is an art.” This is not a promising start.

Over 3 million women and men stand in front of classrooms every day in the U.S. It is too much to hope for, and always will be, that more than a small percentage of them will be artists, great, bad or mediocre. The degree to which we pin our hopes for large-scale school improvement on attracting artists and rock stars to the classroom is the degree to which we plan to fail.

Yet all is not lost. After dispensing with five poorly chosen words, Lemov spends the next 300 pages completely contradicting them, demonstrating in convincing detail that teaching is not an art at all, but a craft, a series of techniques that can be identified, learned, practiced, and perfected. His focused, obsessively practical study of what makes teachers effective could—and should—shift the terms of our increasingly vitriolic national debate from “teacher quality” to “quality teaching.” This is no mere semantic distinction. The difference is not who is in the front of the room. The difference is what that person does. Lemov’s achievement is to examine teaching at the molecular level. By doing so, he may have rescued education reform from its implicit dependence on classroom saints and superheroes. It is an indispensable shift. If teaching effectively is something for the best and the brightest, rather than the merely dedicated and diligent, education reform is finished, now and forever.

“Many of the techniques you will read about in this book at first may seem mundane, unremarkable, even disappointing,” Lemov begins apologetically. Don’t be fooled by his modesty. The managing director of Uncommon Schools, a network of successful charter schools in New York and New Jersey, Lemov has invested thousands of hours in classroom observations, work that has made him a sought-after consultant to ed reform icons such as KIPP, New Leaders for New Schools, Teach For America, and others. Under Lemov’s watchful eye, the subtle magic of solid craftwork is revealed: Stand still when giving directions; ask students who have not raised their hands to answer a question; use “wait time,” a few seconds’ pause after asking a question before calling on a student to answer it. Each technique is intended to improve classroom management, enhance student engagement, raise expectations, and to do so briskly.

Lemov is obsessed with time and the amount of it wasted moving from one place to another, putting materials away, or transitioning to a new lesson or activity. “There isn’t a school of education in the country that would stoop to teach its aspiring teachers how to train their students to pass out papers,” he writes, “even though it is one of the most valuable things they could possibly do.” As Lemov calculates, the time saved on such mundane tasks quickly adds up to days of extra instructional time over a school year.

If Teach Like a Champion fails to become a standard text in our schools of education, however, it will not be a function of the utilitarian thrust of Lemov’s observations, but his refusal to pay even lip service to the standard homilies of effective practice. Guide on the side? Self-directed learning? Lemov favors students in rows as the default classroom structure. Culturally relevant pedagogy? Substitute rap lyrics for lyric poetry? “Content is one of the places that teaching is most vulnerable to assumptions and stereotypes. What does it say,” Lemov asks, “if we assume that students won’t be inspired by books written by authors of other races?” Volumes have been written on differentiated instruction. Lemov gives it a single, not very deferential paragraph. Group work is “as likely to yield discussions of last night’s episode of American Idol as it is higher-order discussions of content.” Asking frequent, targeted, rigorous questions of students, he believes, “is a powerful and much simpler tool for differentiating.”

“One of the biggest ironies I hope you will take away from reading this book is that many of the tools likely to yield the strongest classroom results remain essentially beneath the notice of our theories and theorists of education,” he writes. It is hard not to agree.

At no point in Teach Like a Champion does Lemov explicitly state that his goal is to wipe out the beau ideal of the rock star teacher. He doesn’t need to. If we’re fortunate, it will wither away once Lemov’s taxonomy takes root. In the popular imagination, fueled by Hollywood, hero teachers are charismatic figures, endowed with an unshakable will, and a deep, abiding belief in the untapped genius of their (inevitably) unruly students. Miracle-working mavericks, they defy the forces of mediocrity arrayed against them and magically transform a class full of those kids—the ones the “system” and every other adult in the building—nay, the world—has long since given up on. Good-bye, poverty, gangs, and mean streets. Hello, Harvard. And if one teacher can work such wonders, surely it is not too much to expect them all to do it. The exceptions prove what the rule should be. No excuses. Whatever it takes. Relentless pursuit. What part of “every child can succeed at the highest level” do you not understand?

Readers hoping for stock tales of hero teachers will be disappointed. Teach Like a Champion features many teachers, but they are master craftsmen, not superheroes. Describing grammatically correct complete sentences as “the battering ram that knocks down the door to college” is as close to hyperbole as Lemov gets. He sees high expectations as essential to student achievement, but not an occasion for grandstanding and motivational speeches. Instead, it’s about running a classroom where participation is not optional, standards are clear, and no apologies are made for rigor. Mastering sound teaching techniques “will be far more productive than being firm of convictions, committed to a strategy, and, in the end, beaten by the reality of what lies inside the classroom door in the toughest neighborhoods of our cities and towns,” says Lemov.

Teach Like a Champion is not a perfect book. Its advice is broadly applicable, but its sweet spot is elementary and middle-school teachers in low-performing urban schools. Veteran teachers who work in other settings may wonder what all the fuss is about, and Lemov’s definition of effective teaching as getting big test gains in low-income schools may be too narrow for some.

Still, Uncommon Schools runs 17 charter schools in New York, which as a group outperform state averages in both math and English language arts (ELA). Two of its Brooklyn schools have posted math scores that were the best in the state, Excellence Boys Charter School (6th grade) and Kings Collegiate Charter School (7th grade); ELA test scores of 8th graders at True North Rochester Preparatory Charter School in Rochester placed that school at number 6 out of 1,450 schools tested. While that’s not proof positive of effectiveness, there is an intuitive, even visceral appeal to Lemov’s techniques. Keeping all students engaged and attentive is surely better than failing to do so. There is no conceivable downside to lengthening learning time by wasting less of the school day. Teachers seem not to be waiting for gold-standard research to validate the 49 techniques in Lemov’s taxonomy as effective. Six months after publication, Teach Like a Champion was still a top education book and overall best seller on Amazon.com. Doug Lemov has struck gold, and a nerve.

Seldom has a book been better timed or more urgently needed. Walk into a struggling urban school and you will mostly find well-intentioned people working hard and failing. More often than not, they’re failing despite doing precisely what they’ve been trained to do. The proper question is not how do we get rid of bad teachers, but how can we make our existing teacher corps more effective? Thus, perhaps the highest praise that one can heap on Mr. Lemov’s book is that, for the first time, it makes helping teachers improve their craft on a broad scale seem not merely sensible, but achievable.

A former 5th-grade teacher, Robert Pondiscio writes about education at the Core Knowledge Blog. For his take on Steve Farr’s Teaching as Leadership, see our blog.

An unabridged version of this article is available here.

---

American 15-year-olds continue to perform no better than at the industrial-world average in reading and science, and below that in mathematics. According to the results of the 2009 Program for International Student Assessment (PISA) tests, released in December 2010 by the Organisation for Economic Co-operation and Development (OECD), the United States performed only at the international average in reading, and trailed 18 and 23 other countries in science and math, respectively. Students in China’s Shanghai province outscored everyone.

Many have identified variations in teacher quality as a key factor in international differences in student performance and have urged policies that will lift the quality of the U.S. teaching force. To that end, President Barack Obama has called for a national effort to improve the quality of classroom teaching and repeatedly indicated his support for policies that would provide financial rewards for outstanding teachers.

In a March 2009 speech to the Hispanic Chamber of Commerce, he explained,

Good teachers will be rewarded with more money for improved student achievement, and asked to accept more responsibilities for lifting up their schools. Teachers throughout a school will benefit from guidance and support to help them improve.

In the administration’s Race to the Top initiative, the U.S. Department of Education encouraged states to devise performance pay plans for teachers in the hope that such an intervention could have a significant impact on student performance.

But is there anything in the data the OECD has accumulated to give policymakers reason to believe that merit pay works? Do the countries that pay teachers based on their performance score higher on PISA tests? Based on my new analysis, the answer is yes. A little-used survey conducted by the OECD in 2005 makes it possible to identify the developed countries participating in PISA that appear to have some kind of performance pay plan. Linking that information to a country’s test performance, one finds that students in countries with performance pay perform at higher levels in math, science, and reading. Specifically, students in countries that permit teacher salaries to be adjusted for outstanding performance score approximately one-quarter of a standard deviation higher on the international math and reading tests, and about 15 percent higher on the science test, than students in countries without performance pay. These findings are obtained after adjustments for levels of economic development across countries, student background characteristics, and features of national school systems.

I draw these conclusions cautiously, as my study is based on information on students in just 27 countries, and the available information on the extent of performance pay in a country is far from perfect. Further, the analysis is based on what researchers refer to as observational rather than experimental data, making it more difficult to make confident statements regarding causality.

It is possible that what I have observed is the opposite of what it seems: countries with high student achievement may find it easier to persuade teachers to accept pay for performance, thereby making it appear that merit pay is lifting achievement. More generally, both performance pay and higher levels of achievement could be produced by some set of factors other than all of those taken into account in the analysis. For example, performance pay could be more widely used in places where, as in Asia, cultural expectations for student performance are high, making it appear that performance pay systems are effective, when in fact both performance pay plans and student achievement are the result of underlying cultural characteristics. But even if my findings are not indisputable, I did carry out a variety of checks to see if any observable factor, such as Asian-European differences, could account for the conclusion. Thus far, I have been unable to find any convincing evidence that the findings are incorrect. Given that, let us take a closer look at what can be learned about the impact of performance pay from PISA data.

Prior Research

Standard economic theory predicts that workers will exert more effort when monetary rewards are tied to the amount of the product they produce. Not only does performance pay stimulate individual effort on the job, it is theorized, but jobs where rewards are tied to effort attract energetic, risk-taking employees who are likely to be more productive. This latter consideration, says Stanford economist Edward Lazear, “is perhaps the most important” way in which a merit pay plan can influence worker performance. But if economists expect positive results from merit pay, many educators believe that teachers are motivated primarily by the substantive mission of the teaching profession and that they do not respond to—indeed, they may resent and resist—monetary incentives that tie salary levels to performance indicators.

To see whether the education sector is an exception to general economic theory, a number of performance pay experiments have been carried out, and in Israel and India such studies have shown positive impacts on student achievement. Experimental studies have tracked only the short-term impact of merit pay, however, and so have not identified any long-term effects that might come from changes in the kinds of people who choose to go into this line of work. Conceivably, a merit pay system could discourage entry into the profession of potentially excellent teachers reluctant to subject themselves to the requirements of a pay-for-performance scheme. Alternatively, if performance pay makes teaching more attractive to talented workers, short-term evaluations could understate its benefits.

One way to capture the long-term effects of teacher performance pay, including changes in the characteristics of those choosing to become a teacher, is to compare countries with performance pay systems to those without. This is now possible because the OECD in 2005 administered a separate survey to each of its member countries concerning the teacher compensation systems in place during the 2003–04 school year, when the 2003 PISA study was conducted. The 2003 PISA provides test score results in math, reading, and science for representative samples of 15-year-olds within each country, or nearly 200,000 students altogether. (The relative performance of countries on the PISA changed only slightly between the 2003 and 2009 tests. For example, in no subject did the scores for the United States differ significantly between 2003 and 2009.)

The PISA study is particularly useful, because it also includes information on a wide variety of family, school, and institutional factors that are likely determinants of student achievement. My analysis adjusts, at the level of the individual student, for such characteristics as the student’s gender and age, preprimary education, immigration status, household composition, parent occupation, and parent employment status. Nine measures of school resources and location are available, including class size, availability of materials, instruction time, teacher education, and size of community. Country-level variables included in the analysis were per capita GDP, teacher salary levels, average expenditure per student, external exit exams, school autonomy in budget and staffing decisions, the share of privately operated schools, and the portion of government funding for schools.

The PISA sampling procedure ensured that a representative sample of 15-year-old students was tested in each country. The student sample sizes in the OECD countries range from 3,350 students in 129 schools in Iceland to 29,983 students in 1,124 schools in Mexico. I therefore use weights when conducting my analysis so that each country contributes equally to the estimated effect of performance pay on student achievement.

Measuring Teacher Performance Pay

The measure of performance pay available from the OECD survey is less precise than one would prefer. It simply asks officials in participating countries whether the base salary for public-school teachers could be adjusted to reward teachers who had an “outstanding performance in teaching.” While the survey asked about many other forms of salary adjustments, the study protocol reports that this was the only one that “could be classified as a performance incentive.”Among the 27 OECD countries for which the necessary PISA data are also available, 12 countries reported having adjustments of teacher salaries based on outstanding performance in teaching. The form of the monetary incentive and the method for identifying outstanding performance varies across countries. For example, in Finland, according to the national labor agreement for teachers, local authorities and education providers have an opportunity to encourage individual teachers in their work by personal cash bonuses on the basis of professional proficiency and performance at work. Outstanding performance may also be measured based on the assessment of the head teacher (Portugal), assessments performed by education administrators (Turkey), or the measured learning achievements of students (Mexico). Unfortunately, the coding of the measure does not allow my analysis to consider variation in the scope, structure, and incentives of performance-related pay schemes.

As an example of the limitation of this measure, note that the United States is coded as a country where teacher salaries can be adjusted for outstanding performance in teaching on the grounds that salary adjustments are possible for achieving the National Board for Professional Teaching Standards certification or for increases in student achievement test scores. That policy, however, affects only a few teachers in selected parts of the country. Given such weaknesses in the survey measure, it is all the more remarkable that I was able to detect impacts on student achievement.

Main Results

As noted above, my main analysis indicates that student achievement is significantly higher in countries that make use of teacher performance pay than in countries that do not use it. On average, students in countries with performance-related pay score 24.8 percent of a standard deviation higher on the PISA math test; in reading the effect is 24.3 percent of a standard deviation; and in science it is 15.4 percent (see Figure 1). These effects are similar to the impact identified in the experimental study conducted in India and about twice as large as the one found in a similarly designed Israeli study.

Figure 2 depicts the math result graphically. The figure’s vertical axis displays the average math test scores of students in each country after adjusting for all of the control variables in the model, with the exception of the variable measuring the use of performance pay. The horizontal axis in turn shows the performance pay variable, also after adjusting for those same control variables. The solid line on the figure shows the estimated relationship between these two variables across the 27 countries included in the analysis. It shows a clear positive association between the variation in country-average test scores and the variation in teacher performance pay that cannot be attributed to the other factors included in the analysis.

A lingering concern, however, is that the analysis may be contaminated by the fact that the very cultures that introduce merit pay are those that set high expectations for student achievement. The countries represent widely different cultures, including Asian ones, where expectations for students are often much higher than in Europe and North America. The best way to account for cultural differences among the continents of the world is to control in the analysis for the average effect of living on a particular continent, a strategy known to statisticians as continental fixed effects. Figure 1 thus also shows results based on models that include a fixed effect for each of the four continents with OECD countries: Europe, North America, Oceania, and East Asia. In these models, the effects of pay for performance are shown to be even larger than the results based on comparisons across continents. In other words, the findings cannot be attributed to cultural differences among the major regions of the world, because they are even larger when one looks only at patterns within these regions.

As a further test, I estimated the impact of performance pay for only the 21 participating European countries. Once again, the results showed even larger positive effects than those obtained for the full sample.

Other Sensitivity Tests

When findings are based on small samples, it is important to ascertain whether a conclusion is sensitive to the particular analysis being conducted. Even after conducting a preferred analysis that maximizes use of the information available and best conforms to underlying economic theory, it is important to make sure that the pattern that one has identified is not a statistical accident that readily disappears if a slightly different analysis is conducted. For this reason, I performed a variety of sensitivity tests for math achievement because the reliability of the math test across countries and cultures is usually considered higher than it is for reading or science. Remarkably, the relationship between performance pay and math achievement remained essentially unchanged, regardless of the sensitivity test that I ran.

My first sensitivity check focused on cultural differences among countries that were not captured by the continental fixed effects analysis. In this sensitivity check, I excluded two countries, Mexico and Turkey, which have particularly low levels of GDP per capita. Since it is known that the level of GDP is strongly correlated with educational performance, it may be that the inclusion of these two countries is producing misleading results. But dropping these countries hardly affects results.

The second sensitivity test excluded the level of educational attainment of the teachers, on the grounds that teacher quality might itself be affected by a country’s performance pay policies and therefore should not be used as a control variable. Excluding this variable did not materially change the results from those reported in Figure 1. In a third series of sensitivity tests, I excluded from the analysis one country at a time to make sure that the situation in no one country was driving the overall pattern of results. I found no evidence that that was happening.

The incidence of performance pay is, to some extent, clustered in two regions: Scandinavia (Denmark, Finland, Norway, and Sweden) and Eastern Europe (Czech Republic and Hungary). In a fourth set of sensitivity tests, I separately excluded the countries from these two regions from the analysis to see whether results were highly dependent on one or the other cluster. The results remained unchanged, indicating that neither of these regional clusters is solely responsible for the main result.

A fifth set of sensitivity tests was possible because I have information on other policies that lead to differential pay among teachers. Salaries may vary depending on 1) the teaching conditions and responsibilities (such as taking on management responsibilities, teaching additional classes, and teaching in particular areas or subjects), 2) teacher qualifications and training, or 3) a teacher’s family status and/or age. Since it is possible that student achievement is higher whenever pay schedules are flexible, regardless of the connection to teacher classroom effectiveness, I estimated the impact of each of these three sets of factors on math achievement. None showed a significant impact on performance, and the effect of performance pay remained large and significant, even when these other possible salary adjustments were included in the analysis.

In sum, the main results shown in Figure 1 survive a wide variety of sensitivity tests. That the results are robust to multiple model specifications provides strong evidence that performance pay helps to explain the variation in student performance on the PISA tests.

Differential Effects

With one exception (immigrants benefited less than native-born students from a performance pay regime), I found only small differences in the impact of performance pay on the math achievement of subgroups in the population. Since important differential effects were identified for only one subgroup, one cannot infer that the impact of performance pay on student math learning is concentrated on any particular group of students.

I did, however, find a surprising difference in the way in which a teacher’s education background affects math learning, depending on the presence of a pay-for-performance system. In countries with performance pay, teachers who have an advanced degree in pedagogy do not outperform those without such a degree (the only measure of a teacher’s education available in the PISA data base). However, in countries without performance pay, students learn more in math if they have a pedagogically trained teacher. Perhaps an incentive system washes out any differences that may be caused by variations in teacher training.

Conclusions

The analysis presented above represents the first evidence that, all other observable things equal, students in countries with teacher performance pay plans perform at a higher level in math, reading, and science. The differences in performance are large, ranging from 15 percent (in science) to 25 percent (in math and reading) of a standard deviation. Since one-quarter of a standard deviation is roughly a year’s worth of learning, it might reasonably be concluded that by the age of 15, students taught under a policy regime that includes a performance pay plan will learn an additional year of math and reading and over half a year more in science. However, this conclusion depends on the many assumptions underlying an analysis based on observational data.

Although these are impressive results, before drawing strong policy conclusions it is important to confirm the results through experimental or quasi-experimental studies carried out in advanced industrialized countries. Nothing in the PISA data allows us to identify crucial aspects of performance pay schemes, such as the way in which teacher performance is measured, the size of the incremental earnings received by higher-performing teachers, or very much about the level of government at which or the manner in which decisions on merit pay are made. Studies of such matters are probably better performed within countries, taking advantage of variation in policies within those countries. The study design also does not allow one to tease out the relative importance of the incentive to existing teachers of a performance pay plan as compared to the changes that may take place in teacher recruitment when compensation depends in part on merit rather than just on a standardized pay schedule. Since much more work needs to be done on all of these questions, a wit might insist that performance pay apply to scholars as well.

Ludger Woessmann is professor of economics at the University of Munich and head of the department of Human Capital and Innovation at the Ifo Institute for Economic Research.

An unabridged version of this article is available here.

An unabridged version of this article is available here.

---

Merit pay proponents argue that monetary incentives for better teaching can improve the quality of instruction in our nation’s classrooms. Yet only a handful of studies have evaluated the impact of teacher merit pay on student achievement. These studies offer no conclusive recommendations regarding the optimal role of merit pay in U.S. school systems, leaving policymakers largely dependent on studies on other countries for information about how best to implement merit pay programs.

Recently, the New York City Department of Education (DOE) conducted a policy experiment to test whether merit pay given to all teachers at an effective school could increase student achievement. The city’s School-Wide Performance Bonus Program, launched in 2007 and endorsed by both the DOE and the teachers union, was implemented in a randomly selected subset of the city’s most disadvantaged schools. The randomized design of school selection makes it possible to separate out the causal effect of this form of merit pay from myriad other influences on student learning.

Our analysis is based on data from the first two years of the bonus program. In interpreting our findings, it is important to appreciate the key features of the program’s structure. Teachers received bonuses based on the overall performance of all tested students in their school, rather than just on the performance of students in their own classrooms. According to proponents of group incentives, this design can minimize conflicts and foster a spirit of cooperation among teachers at participating schools. However, under group incentive schemes, individual teachers may not have sufficient motivation to improve their own performance if they know that their success in attaining a bonus depends heavily on the efforts made by other teachers. Especially in schools with a large number of teachers, it may be difficult to sustain a school-wide push to mobilize the efforts of most teachers. The New York City bonus program thus provides valuable information on the effects of a school-wide bonus plan.

Other specific characteristics of the bonus plan and the New York City context may also have influenced its effectiveness. If a school won a bonus, money was distributed among teachers and other school personnel by a committee consisting of two administrators and two teachers union representatives at the school. The bonus program was implemented alongside a new citywide accountability system that provided strong incentives to improve student achievement, regardless of whether a school was participating in the bonus program. Also, over the period we examine, all schools experienced increases in student achievement on the New York state test, leading some to suggest that the exam had grown easier (or at least easier to teach to). Roughly 90 percent of participating schools received a bonus in the second year of the program.

Did the group bonus program operating in this policy environment have an impact on student achievement? We find very little effect overall, positive or negative. There is some evidence, however, that the program had a positive impact in schools where teachers were few in number, an environment in which it may be easier for teachers to cooperate in pursuit of a common reward. This study leaves open the question of whether a bonus program that rewards teachers for their own specific effectiveness would be more successful.

The Program

In November 2007, the New York City DOE launched the School-Wide Performance Bonus Program, randomly selecting 181 schools serving kindergarten through 8th grade to participate from a group of 309 high-need schools. (Disadvantaged high schools were also randomly selected into the program; we focus only on elementary and middle schools since these are the grades for which we can measure math and reading achievement.) The remaining 128 schools that were not selected serve as the control group for the purposes of our evaluation. The 309 schools included in the study differed from other city schools in the following ways: They had a higher proportion of English Language Learners (ELL), special education, minority students, and students eligible for the Title I free or reduced-price lunch program, as well as lower average math and reading scores. Teachers in these schools had slightly less experience and slightly more absences than teachers in other schools. The schools were smaller and had fewer teaching staff than other New York City schools.

The bonus program was the product of lengthy negotiations between district administrators and the teachers union. As a result of these negotiations, schools had to gain the support of 55 percent of their full-time United Federation of Teachers (UFT) staff each year in order to participate. Out of the 181 schools selected for the program, 25 schools voted not to participate in the first year of implementation or withdrew from the program following an initial vote of approval, and three more schools pulled out before the second year. Additionally, at the discretion of the DOE, two schools initially assigned to the treatment group were moved to the control group, and four schools initially designated as control schools were moved to the treatment group and subsequently voted to participate in the program. Of course, the schools that elected not to take part in the program and those moved by the DOE may differ in important ways from schools that chose to participate. We therefore consider the treatment group to include all 181 schools originally deemed eligible for bonus payments and take into account the fact that not all of them were actually participating in the program when interpreting our results.

Schools that implemented the program could earn a lump-sum bonus for meeting school-wide goals. These goals were tied to the New York City accountability system and were mainly determined by student performance on state math and reading exams. Under this accountability system, schools receive scores and grades that summarize their overall performance on three sets of measures: school environment, student performance, and student progress. The school environment measure incorporates student attendance and the results from surveys of parents, teachers, and students. Student performance measures include average student achievement on reading and math exams, along with median proficiency and the percentage of students achieving proficiency. The student progress measure considers the average change in test scores from year to year and the percentage of students who made progress from one year to the next. The accountability system also gives “extra credit” for exemplary progress among high-need students. Schools received target scores based on their accountability grades, and schools with lower accountability grades needed to make larger improvements to reach their targets.

Schools participating in the bonus program received awards based on their progress toward meeting target scores. Schools that achieved their goals received bonuses equal to $3,000 per union teacher. Schools that fell short but manage to meet 75 percent of their goal received $1,500 per union teacher. Schools that did not achieve their target faced no consequences from the bonus program beyond the absence of incentive pay. For a sense of the strength of the incentive provided by the bonuses, the full $3,000 award represents a 7 percent increase in the salary of teachers at the bottom of the pay scale and a 3 percent increase for the most experienced teachers. In other words, these bonuses provided a substantial monetary benefit to most recipients.

Each participating school was required to develop a plan for distributing any lump-sum bonus awarded to the school. In the first year of the program, plans had to be submitted to the DOE after students took the state math and reading exams but before exam results were released and, thus, before schools knew whether they would receive a bonus. In every school, a four-member compensation committee, consisting of the principal, a second administrator, and two teachers elected by the school’s UFT members, determined how bonuses would be distributed. The DOE program guidelines placed only two restrictions on the schools’ bonus distribution plans: all union teachers had to receive a portion of the bonus payment and bonuses could not be distributed based on seniority. Otherwise, the committees had full discretion over bonus amounts and over whether other school employees would also receive funds. About half of the school committees chose to divide the award roughly equally among all recipients. In these schools, the difference between the highest and lowest bonus payment was less than $100. In the rest of the schools, the difference between the highest and the lowest bonus ranged from a low of $200 to a high of $5,000.

Of the 158 schools that voted to participate in the first year of the program, 87 (55 percent) received bonus payments. The bonus pool totaled $14.0 million in the first year and averaged $160,500 per school. In the second year of the program, the 2008–09 school year, 139 participating schools (91 percent) earned bonus awards, averaging $195,100 per school and totaling $27.1 million.

Little Difference for Students

Before we get to the detailed findings of our study, it is important to make clear the nature of the incentives NYC teachers and administrators faced over the period we examine. First, the 2007–08 school year was the first year of both the bonus program and a new citywide accountability system. The accountability system provided strong incentives to improve student achievement, regardless of whether a school was participating in the bonus program. For example, schools that earned A or B accountability grades were eligible for principal bonuses and additional funds when students transferred from schools receiving a poor grade. Schools that received D and F grades faced potential consequences, including principal removal and school closure. With this in mind, we see the results of our study as representing the effect of group-based teacher merit pay for schools that are already under accountability pressure. However, given that all school districts in the United States are subject to No Child Left Behind and many states have implemented their own accountability systems, this may be the most appropriate context in which to study the consequences of merit pay.

The second thing to keep in mind is that the power of the bonus program incentives was likely muted in the first year because of the timing of the program announcement. Eligible schools were notified in November of 2007, leaving relatively little time for teachers and administrators to alter their educational plans before accountability exams were administered in January for reading and March for math. As noted above, the percentage of schools that hit their achievement targets increased between the first, truncated year of the program and the second, when schools had more time to respond to the program incentives. But we caution readers to remember that this leap in bonus payouts is not, by itself, evidence that merit pay worked. It may instead reflect citywide performance improvements or, more pessimistically, that the New York state tests decreased in difficulty over this period. The most important comparison to make is between the treatment group schools eligible for the bonus program (most of which actually participated in the program) and the schools in the control group. Treatment-group schools need to at least outpace their counterparts in the control group over these two years for us to say that merit pay made a real difference for student achievement. It is this comparison that is at the heart of our analysis.

How did bonus program schools fare compared to schools in the control group? Both groups of schools saw an increase in the average math and reading scores during the first two years of the bonus program; treatment-group schools, however, did not experience a statistically significant improvement in average test scores relative to the schools in the control group. Nor did these results change notably when we 1) made adjustments for the small differences in treatment and control school characteristics that existed despite randomization between treatment-group and control-group schools, or 2) took into account whether treatment-group schools elected to participate in the bonus program. It is possible, of course, that looking at average student achievement could divert our attention from changes for particular groups of students. Were teachers, we wanted to know, focusing their attention on either high-achieving or low-achieving students in an effort to meet target scores? We used statistical techniques similar to the one we employed to examine changes in average scores to assess the effect of the bonus program on the percentage of students achieving proficiency on math and reading exams. Once again, we found no evidence that the bonus program led to changes in this measure of student achievement. Participation in the bonus program did not, for example, boost the percentage of students who scored at or above the level designated as “proficient” under New York state accountability standards. Bonus-program schools fared no better than schools in the control group, and in the second year of the program, treatment schools experienced a statistically significant, although quite small, decrease in math proficiency.

On a related note, the New York City accountability system and, as a byproduct, the bonus program, contain incentives to focus on particular groups of students, since improvements for some student groups matter more in the calculations of a school’s accountability grade. In addition to calculating overall achievement for all students in a school, components of the New York City accountability system take into account changes in the achievement of students who were in the lowest third of their grade in the prior year, those on the cusp of proficiency, and those close to the school’s median score, along with students who are designated as ELL and students who are enrolled in special education programs. Again, we found no evidence to suggest that the bonus program led to achievement gains for any of these groups of students. On average, students in these groups fared just as well whether they attended a school that was participating in the bonus program or one in the control group.

Limitations of Group Bonuses

Does evidence that the New York City bonus program did not lead to marked gains in student achievement, at least in the program’s first two years, mean that merit pay for teachers in general does not work? That is certainly one possible conclusion to be drawn from our findings. Another possibility is that this particular type of merit pay program, where bonuses are based on school-wide performance and teachers expect to receive bonus payments regardless of their effort, does not work in all schools. Group bonuses may weaken the incentives for individual teachers to increase effort devoted to raising student achievement to the point that the programs become ineffective. And perhaps this problem would be mitigated in programs in which rewards are more tightly coupled to the effort an individual makes in the classroom.

Think about two schools, one with many more teachers than the other, both participating in a school-wide merit pay program. In each school, the impact of an individual teacher’s effort on the expected bonus is determined by the number of other teachers with tested students, since bonus receipt is primarily based on student performance on math and reading exams. Because of this, a very good teacher with a large number of teaching colleagues can do less to raise school-wide student performance than a teacher of the same quality in a school with fewer teachers. In the school with more teachers, the diffusion of responsibility for test-score gains across many teachers may erode the incentive that any individual teacher has to increase effort in the classroom. Some teachers may conclude that exerting additional effort will produce little difference in the overall performance of the school. The central idea here is that teachers could face relatively strong or weak incentives under the same merit pay program as a result of the number of teachers at their school. With this logic in mind, we examined the effect of the New York City school-wide merit pay program at schools with different numbers of teachers with test-taking students. Did schools with fewer teachers show signs that teachers were responding to merit pay incentives?

We conducted a statistical analysis similar to our method for estimating the average effect of the bonus program across all New York City schools in the experiment. But this time, we looked for different effects on math scores in schools with more and fewer math teachers and different effects on reading scores on schools with larger and smaller cohorts of reading teachers.

It turns out that the effectiveness of school-wide bonus programs may, in fact, depend on the number of teachers with tested students in a school (see Figure 1). For schools in the bottom quartile of the number of teachers with tested students, that is, schools with approximately 10 or fewer such teachers in elementary and K–8 schools and five or fewer in middle schools, school-wide merit pay did lead to improved student achievement. We estimate that the New York City bonus program had a positive effect on student math achievement in these schools in both program years, although the estimated effect in the second year fell just short of conventional levels of statistical significance. Conversely, this analysis also indicates that the program may have slightly lowered student achievement in schools with larger teaching staffs. Math achievement gains attributable to the bonus program in schools with smaller teaching staffs were modest in size but meaningful. In the first year of the program, the bonus program boost to math scores was, by our estimates, 3.2 points on the New York state test, or 0.08 student-level standard deviations. To benchmark this effect against the magnitude of other familiar results, it is slightly smaller than the estimated 0.1 standard deviation gain in achievement that results from being assigned to a teacher at the 85th percentile of the effectiveness distribution rather than a teacher at the median.

The Devil in the Details

The New York City bonus-pay program provides us with a valuable opportunity to study the effect of merit pay for teachers in an experimental setting. We are a long way from amassing a convincing body of research on either side of the debate over merit pay in education, but what this experiment makes frustratingly clear for merit pay proponents is that the structure of the payment scheme can make a large difference. For merit pay to improve student outcomes, teachers must face strong incentives to improve their performance. Our study indicates that school-wide bonus programs may be able to provide those incentives in schools with relatively small teaching staffs. They may also be appropriate for schools characterized by a high degree of staff cohesion, in which teachers work collaboratively to improve student learning and it is difficult to isolate the performance of a single teacher. The early experience with the New York City School-Wide Performance Bonus Program suggests, however, that a heavy reliance on school-wide rewards may hamper the effectiveness of merit pay programs in schools with large teaching staffs that are not highly collaborative.

Sarena Goodman and Lesley Turner are PhD candidates in Columbia University’s Department of Economics. The randomization of schools participating in the School-Wide Performance Bonus Program was designed and conducted by Harvard University economist Roland Fryer.

An unabridged version of this article is available here.