At the turn of the 21st century, the United States was trying to come to grips with a serious education crisis. The country was lagging behind its international peers, and a half-century effort to erode racial disparities in school achievement had made little headway. Many people expected action from the federal government.

George W. Bush and Barack Obama, the century’s first two presidents, took up the challenge. For all their differences on how best to stimulate economic growth, secure the national defense, and fix the health-care conundrum, the two presidents shared a surprisingly common approach to school reform: both preferred the regulatory strategy. In 2001, Bush persuaded Congress to pass a new law, No Child Left Behind (NCLB), which created the nation’s first reform-minded federal regulatory regime in education. When NCLB ran into trouble, Obama invented new ways of extending the top-down approach. Unfortunately, neither president came close to closing racial gaps or lifting student achievement to international levels.

The Obama administration is now packing up and heading home, leaving the regulatory machine in ruins. A new federal law, the Every Student Succeeds Act (ESSA), has unraveled most of the federal red tape. Although the mandate for student testing continues, the use of the tests is now a state and local matter. School districts and teachers unions are rubbing their hands at the prospect of reasserting local control.

With districts beset by collective bargaining agreements, organized special interests, and state requirements, choice and competition are the main levers of reform that remain. Vouchers and tax credits are slowly broadening their legal footing. Charter schools are growing in number, improving in quality, and beginning to pose genuine competition to public schools, especially within big cities. Introducing such competition is the best hope for American schools, because today’s public schools are showing little capacity to improve on their own.

From Dreary to Dismal

Two gaping gorges have been cleaved through the dreary education landscape. The black-white divide, obvious in 1966 when James Coleman analyzed equality of opportunity in the nation’s schools, remains virtually intact. Today’s racial gap is nearly one full standard deviation—approximately the difference between the performances of 4th and 8th graders. Only the South has done much to bridge racial differences between black and white high-school students, allowing the region to pride itself today on having no worse a record than the rest of the country (see Figure 1). The international gap is no less distressing. On the tests administered by the Organization for Economic Cooperation and Development, known as the Program for International Student Assessment (PISA), the math performance of U.S. 15-year-olds trails that of their peers in most other industrialized nations. The gaps between U.S. students and those in Japan, Korea, and the Netherlands are nearly as large as the ones between blacks and whites in the United States. Even students from households where a parent has been to college are not performing anywhere near the level attained abroad (Figure 2).

>U.S. 15-year-olds who scored at the 25th percentile on the PISA showed some signs of improvement between 2003 and 2012. Their scores climbed a modest 7 points in reading over this period of time, with no change in the math scores. Those at the very bottom—the lowest 10 percent—showed an even sharper gain of 17 points in reading and 12 points in math. Arguably, these results indicate that NCLB was proving helpful for the lowest-performing students, who were the focus of the regulations. But those gains were offset by a decline of 7 points in both reading and math among students scoring at the 75th percentile, with similar declines appearing among those scoring at the 90th percentile. Those who claimed that the schools were ignoring the educational needs of the better-prepared student in order to avoid leaving any child behind may have had a point. Overall, average student performance remained essentially unchanged over this time period (Figure 3). Whatever equalization was occurring did nothing for the overall quality of American schools.

Bully Pulpit

Regulation is not the only reform tool available to presidents. There is also persuasion. Since the days of Theodore Roosevelt, presidents have used the “bully pulpit” to bend the recalcitrant to their desires. Two years into the presidency of Ronald Reagan the National Commission on Excellence in Education issued a report, “A Nation at Risk,” to highlight the low, declining performance of U.S. schools. Though lacking any regulatory components, the report proved to be a blockbuster, mobilizing reform efforts in states and school districts across the country. SAT scores that had trended downward now reversed themselves. The reading scores of African American 17-year-olds on the National Assessment of Educational Progress (NAEP) leaped dramatically by no fewer than 31 points, a gain equivalent to roughly two to three years of learning (Figure 5). However, it was during this period—and only this period—that the racial gap in reading narrowed significantly.

Unlike Reagan, Bush was no rhetorician. His “compassionate conservative” messages asking that no child be “left behind” inspired little enthusiasm among either educators or students, perhaps because the close presidential election in 2000 had left the country hopelessly divided.

Eight years later, expectations for Obama were greater. The young had flocked to the Obama bandwagon, and the enthusiasm in the African American community was particularly dramatic. But the president saved his bully pulpit for Wall Street profiteers and health-care redesign, leaving him with little leverage for K–12 education reform. Early in his tenure, Obama pointed out that “leadership tomorrow depends on how we educate our students today, especially in math, science, technology, and engineering.” But even that restrained language disappeared as Obama’s term wore on. By the time of his final State of the Union address in January 2016, he had nothing to say about K–12 other than to mislead the public into a false sense of well-being: “Today, our younger students have earned the highest math and reading scores on record. Our high-school graduation rate has hit an all-time high,” he said. The rosy proclamation obscured the fact that racial achievement gaps were nearly as wide as when he entered the White House. Obama’s main education issue had become student loans and the rising cost of a university education, a topic far distant from school reform but one of great interest to present and future members of the upper middle class.

The Money Pit

Just as the bully pulpit was left vacant, the billfold was barely opened. The Bush administration was reluctant to “throw more money at the problem” of educational disparities, though it agreed to some additional spending as the price for securing NCLB’s enactment. During Bush’s administration, expenditures from federal coffers edged upward from 10 percent to 11 percent of total spending on K–12 education (with the remainder of the costs shared about equally by state and local governments). When President Obama took office, it seemed at first he would dramatically alter the federal fiscal role. With an overwhelming Democratic majority in Congress, he secured passage of a trillion-dollar tax-and-expenditure package that included more than $100 billion for K–12 education. The new money was to be spent over a two-year period, with some of it devoted to compensatory education or special education, the rest to district priorities. Federal aid to K–12 and preschool education jumped from $39 billion in 2008 to a high of $73 billion in fiscal year 2010 (0.49 percent of GDP). The following year, $66 billion in federal funding continued to flow. Much of the aid targeted urban districts with heavy concentrations of low-income and special-education populations. For two years, local school districts enjoyed a generous flow of federal cash.

<Ironically, the federal dollars arrived before the recession-induced fiscal crunch hit local revenues from local property taxes, as it takes a year or two, sometimes longer, for depressed property to be assessed at its new, lower value. But the federal dollars had to be spent immediately, in the administration’s view, and local expenditures could not be reduced. As a result, total per-pupil expenditures on education reached a near all-time high in the recession school year of 2009–10, climbing (in constant dollars) to $13,154 from a $12,520 level in 2005–06.

The education industry hoped and expected the stimulus package would set a new floor for federal expenditure. But the 2010 election knocked that plan into a cocked hat, as the newly elected Republican majority in the House of Representatives pushed federal expenditures downward. Federal aid to preschool and K–12 education dropped steadily—to just $41 billion in 2014, 0.24 percent of GDP, less than in the last year of the Bush administration. State and local governments could not—or would not—make up the difference. Expenditures per pupil (in constant dollars) slid to $11,012 in 2011–12, the latest school year for which data are available, a 4.5 percent decline. For big cities, the cuts were much larger. In Philadelphia, for example, revenue per pupil (in constant dollars) dropped from more than $15,400 at the height of the stimulus package to just $13,660 in 2013, a free fall of 12 percent, which forced deficit financing, personnel cuts, and shortened school years (see “The Philadelphia School District’s Ongoing Financial Crisis,” features, Fall 2014).

Regulation to the Rescue

Short on both rhetoric and ready cash, the presidents turned to regulation. But instead of trying to knock schools into shape with a regulatory hammer, they employed measuring sticks and magnifying glasses: each year states were asked to release information for every school on the performance of its students in math, reading, and science. This strategy rested on the assumption that the public would notice deficiencies and demand corrective actions, giving regulators the power to impose solutions.

Admittedly, regulatory reform was not invented in Washington. Calls for higher standards, minimum competency tests, and school accountability had surfaced at the state level as early as the 1970s. Southern governors—James Hunt in North Carolina, Bill Clinton in Arkansas, Jeb Bush in Florida, Ann Richardson in Texas, and others—played major roles. Outside the South, Massachusetts took the lead.

That set the stage for a strange alliance between Texas and Massachusetts in the months following the fall of the twin towers in 2001. Putting aside partisan feuding sparked by the nail-biting presidential contest in 2000, Senator Edward Kennedy and President Bush worked together to persuade Democrats and Republicans to pass NCLB, which was signed into law in January 2002. Every state was henceforth expected to set proficiency standards toward which students had to make adequate progress each year until all students had crossed that bar in 2014. The law also required annual statewide tests in grades 3 through 8, and again in high school, and states had to publish the performances of students on these tests for every school, breaking out the results by ethnicity, eligibility for a subsidized lunch, and a variety of other categories. If students were not making the requisite progress, families would have the option of picking another public school within the district. If that didn’t work, students were to have access to afterschool study programs. And if that failed, schools were to be reconstituted under new leadership.

All these steps required a vast number of regulations. But school districts still found ways of undermining federal objectives. They instituted byzantine procedures that parents had to navigate before they could exercise choice. Afterschool programs offered by private providers were frequently denied space at local schools. Reconstitution of low-performing schools often consisted mostly of window dressing.

Nonetheless, NCLB did shine a spotlight on the public schools. Every school had to report to the public the percentage of students at each grade level who performed at “proficient” or above in reading, math, and, later, science. If schools failed to make adequate progress, officials had to explain themselves to reporters, parents, and the public at large. As the goal was to make all students proficient by 2014, the explanations proliferated with each passing year.

The utopian goal set for 2014 was never meant to be taken seriously. After all, NCLB, like many other federal laws, had a five-year expiration date, and it was generally assumed that a new piece of legislation would be on the books by 2007, long before the full-proficiency deadline was reached. But when 2007 arrived, the two houses of Congress were caught in a deadlock and could not agree on new reform legislation. Instead, Congress simply extended NCLB from one year to the next (a necessary step if federal funds were to continue flowing to the states). Not until December 2015—eight years past the deadline for new legislation—did the legislators replace NCLB with ESSA.

In the meantime, the absurdities in NCLB were becoming increasingly apparent. With nearly every school failing to bring all of its students up to full proficiency, nearly every school was theoretically at risk of reconstitution. Criticisms of NCLB escalated, and many were justified. For instance, the definition of “failing schools” unfairly picked on those serving disadvantaged students. But the critiques of NCLB quickly degenerated into blanket attacks on all standardized tests: “The tide on testing is turning,” said Randi Weingarten, president of the American Federation of Teachers, who then called for NCLB revisions that would “address the root cause of test fixation.” Secretary of Education Arne Duncan, averring that testing was “sucking the oxygen out of the room,” promised to do something about it.

Race to the Top

Even before criticisms had escalated to this peak, the Obama administration, anticipating the growing opposition to NCLB, had invented an alternative way of perpetuating regulatory reform. Duncan announced Race to the Top (RttT), a competitive grants program that had been authorized and funded by the education stimulus package. At $4 billion, the money for RttT was but a minor component of the stimulus package and, overall, it amounted to less than two-tenths of 1 percent of school expenditures in the United States. Yet the idea of a competition among states for a fixed sum of money captured media attention. RttT’s purpose, the president said, was to “incentivize excellence and spur reform and launch a race to the top in America’s public schools.” In an analysis of the program, political scientist William Howell wrote that RttT encouraged applicants to develop “common core state standards,” design a teacher evaluation plan based in part on the performance of their students, ensure “successful conditions for high-performing charter schools,” and numerous other reforms (see “Results of President Obama’s Race to the Top,” research, Fall 2015). Eighteen states and the District of Columbia won RttT awards that ranged between $17 million and $700 million.

Frederick Hess at the American Enterprise Institute attacked the grant program for “its emphasis on promises rather than accomplishments,” but Joanne Weiss, who helped direct the program, later defended it by claiming that “it moved away from the notion that federal policy is designed chiefly to prevent bad actors from doing harm, and it set its sights on excellence. It urged idea-rich, capable states to define and navigate paths to educational excellence, and in so doing, to blaze trails that could show the way for other states” (see “What Did Race to the Top Accomplish?” forum, Fall 2015). Howell found that most states made efforts to follow the federal directives to which they had agreed, though he was not able to determine whether plans were effectively implemented at the local level.

The RttT competition proved so successful the Department of Education relied upon its framework for an even bolder policy: states could seek a waiver of the most onerous NCLB requirements by submitting alternative reform plans broadly similar to ones encouraged by RttT. The list included two significant proposals—the Common Core State Standards and test-based teacher evaluations. Eventually, 43 states, the District of Columbia, and Puerto Rico were granted waivers from NCLB, in effect gutting the federal law.

RttT and the waiver policies it engendered must therefore be counted as extraordinary political successes if only because they allowed the Obama administration to substitute its priorities for those of its predecessor. Although nothing in either program specifically ordered states to adopt Common Core, the standards were enacted into law by 46 states and the District of Columbia, and state expectations for student performance shifted upward (see “After Common Core, States Set Rigorous Standards,” features, Summer 2016). Teacher evaluations were instituted in the District of Columbia, and, in the following few years, student performance in the district improved by a surprisingly large margin. Obama had burnished his reform credentials every bit as bright as Bush had.

But as Martha Derthick wrote at the time, waivers “undermine the rule of law,” raising “a concern that extends well beyond the field of education.” Secretary Duncan had left himself badly exposed by constructing an education policy on a series of questionable administrative maneuvers rather than a solid piece of congressional legislation. Political opposition began to arise to two of the waivers’ key recommendations—establishing higher state standards and tightening teacher evaluations. Tea Party activists attacked Common Core, objecting to what the Heritage Foundation called the Obama administration’s intent to nationalize “the content taught in every public school across America.” And teachers unions tightened the screws by balking at unfair evaluations of teacher performance. “Old tests are being given, but new and different standards are being taught,” National Education Association president Dennis Van Roekel declared. “This is not ‘accountability’—it’s malpractice.” Meanwhile, student progress on NAEP tests came to a virtual standstill (Figure 4).

Caught in the maelstrom, the Obama administration was unable to defend against a bipartisan move on Capitol Hill to end waivers altogether by enacting, for the first time since 2002, a new federal education law, ESSA. The law requires annual testing but leaves it to the states to decide how the results will be used. Most of the other regulations have been removed, shifting authority over schools back to states and localities. As an education reform strategy, federal regulation is dead. The regulations had little long-term effect, and the political opposition crescendoed. The regulated captured the regulators. Nor is there much appetite for new accountability rules at the state level. If reform is to take place as the rest of the 21st century unfolds, it will happen because more competition is being introduced into the American education system.

Competition

Introducing competition is slow, arduous, disruptive, upsetting, and politically divisive. Benefits come slowly. Losses are painful. The politics is messy at best, disastrous at worst. Winners are ingrates who feel they deserve any benefits they enjoy. Losers blame not themselves but changes in the rules of the game. But the long-term consequences of competition for consumers and society as a whole can be amazingly beneficial, as deregulation of the airlines and telecommunications industries has shown. Comparable gains have yet to appear throughout American K–12 education, but to see how it might happen, let us reflect on the slow growth of choice and competition via vouchers and charters that has taken place over the past quarter of a century.

Vouchers. Milton Friedman made the case for school choice in his seminal 1955 article on school vouchers, writing:

[School choice] would bring a healthy increase in the variety of educational institutions available and in competition among them. Private initiative and enterprise would quicken the pace of progress in this area as it has in so many others. Government would serve its proper function of improving the operation of the invisible hand without substituting the dead hand of bureaucracy.

Thirty-five years after these words were penned, Wisconsin enacted a voucher program for the city of Milwaukee. Since then, another 28 state legislatures have passed some kind of voucher program, tax credit, education savings account, or other intervention that provides government aid to students attending private schools. None of these programs are at scale, however. Nationwide, less than 1 percent of the school-age population is participating. But studies show that voucher students of minority background, even if they do not perform much better on standardized tests than their peers in public school, are more likely to graduate from high school and go on to college (see “The Impact of School Vouchers on College Enrollment,” research, Summer 2013). Apparently, private schools seem to do better at fostering character and grit than at academic instruction per se (see “Schools of Choice,” features, Spring 2016).

Perhaps this is why vouchers are popular within the minority community. According to Education Next’s annual public-opinion poll (see “The 2015 EdNext Poll on School Reform,” features, Winter 2016), 48 percent of African Americans support universal vouchers that any family could access, and 65 percent favor a voucher plan limited to those of low income, a feature of most current voucher plans. Support among Hispanic adults is comparable, but the public as a whole is less enthusiastic. Only a third of the public supports targeted vouchers, though opinion with respect to universal vouchers is more or less evenly divided between supporters and opponents.

The strongest opposition comes from teachers unions. Al Shanker, the brilliant (if controversial) leader who led union efforts to win collective bargaining rights in New York City, denounced the idea of vouchers: “Without public education, there would be no America as we know it,” he cried. Vouchers for the poor would be “merely the nose of the camel in the tent.” School boards and teachers themselves could not have agreed more.

Charters. Union opposition to vouchers was so intense it opened the way for another choice reform—charter schools. When first enacted in Minnesota in 1989, charters appeared to be nothing more than a safe place for teachers to try out new ideas that public schools could adopt. Shanker himself initially endorsed charters, making it difficult for subsequent union leaders to express unconditional opposition. Unions nonetheless resist charter growth because the schools are run by nonprofit organizations rather than the government; they are free of many state regulations; and they are usually not subject to collective bargaining agreements. Yet charters can claim that they are in fact public schools. They are authorized by a government agency (a state department of education, state university, mayor’s office, or local school district). Their operating funds come primarily from government sources. Their educational mission is secular. When parental demand for a charter school exceeds available space, the school typically holds a lottery in order to choose impartially among the applicants. Because of these characteristics, charters regularly win endorsements from Democratic and Republican leaders alike, and the movement has enjoyed steady, if unremarkable, growth. Forty-three states allow the authorization of charters, more than 6,000 charter schools have been established, and nearly three million children now attend them (Figure 6).

As their numbers grow, charters are beginning to disrupt the status quo more than vouchers are. In places where these schools of choice are allowed, students are no longer limited to attending their neighborhood public school but can apply to a charter school elsewhere in the community. In other words, charter schools compete with district schools for students. The greater the number of charter schools, the more intense the competition.

Admittedly, charter schools have had difficulty penetrating rural communities. There, a public school, no matter its quality, is perceived as a valuable community institution. Sports events enliven Thanksgiving mornings and Friday nights, school auditoriums are a favorite spot for community events and town meetings, and successful high-school graduates are celebrated in hometown newspapers. The school district is also one of a community’s major employers.

In suburban communities, charters appeal to families who reject the rigid, routinized forms of instruction thought to be prevalent in district schools. The pedagogy often emphasizes project learning, experiential learning, and other student-determined exercises. However beneficial such approaches may be in some respects, they do not seem to translate into higher performances on state-required exams. Even more importantly, school choice among traditional public schools already exists for those who have the resources and resourcefulness to purchase homes or rent apartments in neighborhoods that offer the best educational opportunities. The affluent already have the options they need.

The story is different within central cities. Big-city public schools are in big-time trouble, and many families send their children to their local school more out of necessity than choice. For these families, the charter school option often holds strong appeal. Such schools are generally perceived to be smaller, safer, friendlier, and, more often than not, a better place to learn than district schools. In contrast to progressive charters in suburban areas, central-city charters typically embrace the “no-excuses” model of teaching and learning, emphasizing strict dress codes, rigorous discipline, extended school days and school years, and high expectations for performance on standardized tests. In general, urban charters are outperforming their traditional public-school counterparts. The charter advantage seems to be particularly striking for African American students from low-income families.

The charter school movement has benefited from the spectacular results achieved by the Harlem Children’s Zone Promise Academies, Success Academy, BASIS Schools, KIPP Schools, Uncommon Schools, and others in New York City, Boston, Los Angeles, New Orleans, Washington, D.C., and other prominent cities. Led by strong entrepreneurs, staffed by high-quality teachers from selective colleges, financed by local donors and major foundations, these institutions are providing rigorous instruction over an extended school day and school year. Their ability to lift students who come from low-income, single-parent families to a high level of performance that prepares them for college has shed a warm glow over the entire charter-school undertaking. According to Education Next’s 2015 poll, supporters of charters outnumber opponents by a two-to-one margin, both among the public at large and in minority communities. Partisan differences remain, however, as nearly a third of all Democrats express opposition to charters, as compared to less than one in five Republicans.

Charters serve about 5 percent of the public-school population, less than the 11 percent attending private schools. But the market share of private schools has been stable for decades, while charter school enrollments are growing. In 16 cities, more than 25 percent of public school students are enrolled in charter schools. In New Orleans, the percentage is no less than 79 percent; in Detroit, 51 percent; in the District of Columbia, 43 percent; and in Philadelphia, 28 percent. In many other cities—Los Angeles, Boston, New York City, Chicago, and elsewhere—charter enrollments would be much higher if supply were not artificially constrained by state laws limiting charter growth.

Will the competition between charters and standard, district-operated public schools intensify over the next decade? Is this competition the new reform wave that will sweep over American education? Is there a tipping point at which the demand for charters will force a reconstruction of the educational system more generally? Several factors point in that direction:

· Many charters in urban areas are oversubscribed.

· Big-city school districts must spend a large share of their budget for employee health-care benefits and pensions, a problem charters have escaped thus far.

· Charter school parents can be mobilized in numbers when political confrontations occur.

· Student performance at charter schools is showing signs of improvement over time (mainly because of the closing of weak charter schools).

· Some charters are using new technologies that personalize and customize the learning experience.

· Competition, once introduced, is difficult to reverse.

· Big-city school systems are fighting charters by giving parents a wider array of choices among their public schools, suggesting that the choice genie has escaped from the bottle.

These are straws in the wind, but it is still too soon to predict confidently the degree to which choice will be introduced into American education over the next decade. Teachers unions are mobilizing to block charter expansion in state legislatures and through collective bargaining agreements with local districts. Also, one finds little support for charters in suburbia, small towns, or rural America. If charters achieve a breakthrough, it will be in the country’s largest cities. Spreading out from that base will be a slow, arduous process, achieved only if charters demonstrate that they can deliver a superior educational experience.

If the future of charter schools remains uncertain, the same cannot be said for top-down regulation. Unless teachers surprise us all by embracing a new curriculum generated by Common Core standards, and that curriculum motivates students to make a greater commitment to their learning, reforming the system from within is unlikely to succeed in the years ahead. If school reform is to move forward, it will occur via new forms of competition—whether they be vouchers, charters, home schooling, digital learning, or the transformation of district schools into decentralized, autonomous units. And if student testing has an impact on reform, it will be due to the better information parents receive about the amount of learning taking place at each school. The Bush-Obama era of reform via federal regulation has come to an end.

The essay was the Friedman Lecture presented in January 2016 before the Centre for the Study of Market Reform of Education (CMRE), London, U. K.

Paul E. Peterson, editor-in-chief of Education Next, is professor of government and director of the Program on Education Policy and Governance at the Harvard Kennedy School.

This article appeared in the Summer 2016 issue of Education Next. Suggested citation format:

Peterson, P.E. (2016). The End of the Bush-Obama Regulatory Approach to School Reform: Choice and competition remain the country’s best hope. Education Next, 16(3), 22-32.