In a recent New York Times op-ed, I argued that the case for Betsy DeVos’s Secretary of Education appointment rests on a very weak track record—in particular, the evidence does not support her free market approach to school reform that relies, first and foremost, on school vouchers for private schools, as well as unregulated forms of charter schooling.
To support my case, I presented three categories of evidence: (1) the fact that national reform groups seem deeply concerned about Detroit; (2) the similarity in performance between the city’s charter and traditional public schools; and (3) the large negative effects of two statewide voucher programs on student outcomes. Given that DeVos was a key architect behind Michigan’s policies and a devotee of the free market ideology behind them, this collection of facts is highly problematic. Each deserves more attention than I had room to provide in the original piece. Here, I will take a closer look at the evidence and what it says about DeVos’s approach:
(1) Concerns from within the charter movement.
The fact that a broad swath of the national charter reform community is quite critical of Detroit charters does not seem to be in dispute. In addition to the article I initially cited, the Center for Reinventing Public Education (CRPE) and the National Association of Charter School Authorizers (NACSA), groups that support charter-based reforms, have also expressed concern about the city’s efforts. This is worrisome because it tells us that key leaders who share DeVos’s concerns about public education and even support some of her preferred solutions—people who know a lot about what’s happening in Detroit and nationally—believe she is off track.
The reasons for concern are clear. One of the main arguments for charter schools is that the government can write performance-based contracts, then close or take over schools that fall short on performance. In Detroit, it doesn’t work that way. A NACSA report indicates that Michigan is well below average in the number of charter schools that close, and there is no indication that these decisions are based on performance. One local organization, sympathetic to charter schools, recently lamented that when four low-performing schools came up for renewal, they dodged closure by shopping around for a charter authorizer who would take them. The authorizers themselves have financial incentives to keep schools open and may know little about what is happening in the schools themselves if their offices are located hundreds of miles away.
In addition to running against the basic logic behind charter school reform, allowing low-performers to keep operating is bad for students. In our research on New Orleans, we found very large improvements in student outcomes when charter schools were taken over based on low performance and replaced with others. While it can be hard for authorizers to predict which charter schools will be effective, this only reinforces the importance of following up with accountability after approved schools open, to ensure that students are being well served. Authorizers also need to ensure transparency in the use of funds and ensure that charter managers and board members do not have conflicts of interest, which has been a real problem in Detroit.
A second basic principle of charters is that they allow families to choose schools. However, one of the main concerns with charters is that some act like private schools, picking their own students and pushing out those they don’t want. In other cities, charters have been found to have secret lists of students they are trying to get rid of. Unfortunately, the lack of oversight in Detroit makes the extent of cherry-picking hard to measure. We only learn about it when there are inside whistleblowers. The key point is that the design of the city’s charter system creates incentives for cherry-picking and does nothing to prevent it.
A third issue is that there are certain aspects of school systems that have to be coordinated. For example, if families are going to choose schools, they need good information from third parties that are working on a citywide basis. This lack of coordination is partly why Detroit’s charter system received a D on the Brookings Choice and Competition Index.
(2) The failure of Detroit charter schools to improve student outcomes.
At first glance, it might seem that, despite all this criticism, Detroit charter schools are producing decent results. If you look at Figures 1 and 2 in the report on Detroit that I cited from Stanford’s CREDO research center, you will see that the city’s charter schools do look somewhat better than the comparison traditional public schools, but there are four problems with taking these results literally.
First, given the lack of oversight in Detroit and evidence from other cities that some charter schools cherry-pick their preferred students, these results may make Detroit’s charter schools look better than they are. There is no way the CREDO analysis, or really almost any analysis, could account for this. If it’s happening, then the charter effects on achievement are inflated.
Second, given the potential concerns about schools cherry-picking students and other concerns with high-stakes testing, it’s worth looking at other evidence on academic achievement. Among the 21 mostly low-performing urban districts participating in the urban NAEP test in recent years, Detroit experienced growth that was below the group average growth, even though many of these districts were not undergoing any major governance reforms. This reinforces concerns that the CREDO results may reflect cherry-picking.
The third issue really gets us into the research weeds. Estimates of policy effects are based on comparisons between control and treatment groups. Normally, we researchers look only at the difference—the effect estimate—but we are also coming to realize that we have to pay more and separate attention to the control group itself. In this case, Detroit is the lowest-performing school district in the country. Incredibly, based on the federal urban NAEP test, it is 0.3 standard deviations (approximately 12 percentile points) worse than the next best available, Cleveland. For reformers, the first reaction to this might be, “Yes, and this is why we need to take radical action in Detroit!” I agree, but the point here is quite different: the extraordinarily low standing of the city as a whole, to the degree it is caused by low performance of traditional public schools, should make it easier to improve student outcomes when trying something new. The worse your comparison group, the better you look, and it probably doesn’t get worse than Detroit. Put differently, if we could put Detroit charters on a national scale, they would likely be well below average.
For the same reason, comparing Detroit and New Orleans in the CREDO framework is questionable. In New Orleans, where essentially all schools are charters, the comparison schools have to come either from a handful of district schools (which aren’t really traditional public schools) or from the suburbs—whereas, in Detroit, the comparison schools are apparently within the city. The broader point is that it’s not just that the charter schools in Detroit and New Orleans differ; the comparison groups are also very different in ways that make it very hard to compare these two cities in the CREDO analysis. (To be clear, none of this is a critique of CREDO. They have taken on a herculean task, and their work is extremely important, even if sometimes hard to interpret by itself.) If you want to understand the much more positive effects of the New Orleans reforms, take a look at a summary of evidence on a wide range of metrics.
Fourth, and perhaps most important of all, whatever effect we think occurred for a given city, we can only interpret it in comparison with the other options. To take a generic example, if all programs have positive effects, but Program A has effects that are twice as large as Program B, then why choose B? There does not seem to be much dispute that we can do much better than DeVos’s free market approach.
(3) Recent convincing evidence of the negative effects of school vouchers.
There is no better place to look for evidence about DeVos’s free market approach to education than the most unregulated strategy available: private school vouchers. This is also an important topic because DeVos founded and now directs a national organization advocating for vouchers and other private school choice programs. In the group’s own words: “The American Federation for Children is the leading national advocacy organization promoting school choice, with a specific focus on advocating for school vouchers, scholarship tax credit programs and Education Savings Accounts.”
I have written about voucher results from Louisiana extensively before, especially the large negative effects that the state’s program, and a similar program in Ohio, have had on the achievement of students using them to move to private schools. We don’t know for sure why these programs had negative effects. Some argue (not persuasively in my view) that the negative effects suggest that the programs are actually over-regulated. Also, while some might point to the fact that both programs show signs of helping lift achievement in traditional public schools a bit by increasing competition between schools, I don’t think anyone would argue that we should sacrifice the achievement of students using vouchers in order to help others. “Do no harm” are words to live by.
One likely reason the Louisiana and Ohio voucher results were disappointing is that the market-based approach to school reform seems to work better in urban areas—and the voucher programs were statewide. This reflects the fact that the right approach to school improvement varies by community. Some areas need a new form of governance, while, for others, the traditional public school district no doubt makes the most sense. Given the advocacy of her organization, the American Federation for Children, for statewide vouchers and tax credit programs, it seems unlikely that DeVos would attempt a more focused approach to reform.
Negative effects are also common for virtual (online) schools, which DeVos has also supported. It’s important to emphasize that it’s rare to find negative effects of any educational program, especially ones this large. The fact that we are finding them in DeVos’s signature programs is worrisome.
A Triumph of Ideology
I am an economist and certainly support free markets in general, but it is widely recognized in my field that markets sometimes fail and that external oversight and accountability can go a long way to making them work better. This is especially true of education where there are all sorts of reasons to expect market failures and where equal educational opportunity is a key objective.
DeVos clearly rejects this view. After years of wrangling in Michigan, the Republican-led state was finally able to pass a law that added some of the needed accountability and oversight provisions for the state’s charter schools. Though these provisions fall far short of real accountability, they are a step in the right direction. But contrary to what some have said in defense of DeVos, the law’s passage was not a sign of her lack of influence or of her support for charter accountability. Her level of influence is well established, and after the Michigan law passed the state House of Representatives, a press release from DeVos’s group, the Great Lakes Education Project, noted that lawmakers “were successful tonight in preserving choices and options for [students] and their families […] Despite efforts by some of the elites and big city bosses who believe they know what is best for children.” In other words, in passing this bill, Devos’s group had prevented stronger accountability measures, thus “preserving” the system essentially intact. By all accounts, including their own press release, they fought against an accountability plan developed by a broad-based, bipartisan group of educators and policymakers, and on top of it, wrote the group off as “big city bosses.”
Perhaps the strongest indication of DeVos’s beliefs on accountability comes from the “model legislation” on her group’s web site. I read through two of these in full, one for a tax credit and another for a universal voucher program. In both cases, the only provision for academic accountability would require states to publicly report students’ academic results and satisfaction surveys. That is a start, though all of these data would be self-reported by the schools, apparently without verification. Moreover, there are no provisions in the model legislation for the states to actually do anything if schools are failing. This hands-off approach is consistent with the group’s failed efforts in Detroit.
The evidence simply doesn’t support the free market approach DeVos has championed in Michigan and in her national advocacy efforts. While each piece of evidence I’ve discussed has some limitations, collectively, they present a strong case against DeVos’s ideas. To argue that she has been even moderately successful with her approach, we would have to ignore the legitimate concerns of local and national charter reformers who know the city well, and ignore the possibility that Detroit charters are taking advantage of loose oversight by cherry-picking students, and ignore the very low test score growth in Detroit compared with other cities on the urban NAEP, and ignore the policy alternatives that seem to work better (for example, closing low-performing charter schools), and ignore the very low scores to which Detroit charters are being compared, and ignore the negative effects of virtual schools, and ignore the negative effects of the only statewide voucher programs that provide the best comparisons with DeVos’s national agenda.
That’s a lot to stomach. I don’t usually step out so strongly for or against a given idea—or in this case, a nominee. I only do so when I believe the evidence is overwhelming. This is one of those times.
In addition to evidence, there is an important place for ideology. I agree, for example, with many charter and voucher advocates that freedom of choice is an important value. But that is not a justification to confirm DeVos. We can have choice and government oversight, too—in fact, as we have found in New Orleans, oversight can lead to more and better choices. The best evidence suggests that, at least in urban areas, a regulated charter sector can substantially improve results, much more than we have seen in Detroit. Moreover, maintaining a significant role for government will help ensure that we meet crucial public ends that markets alone are unlikely to support—especially equal opportunity for all students. At the very least, the government has a fiduciary responsibility to oversee the use of public revenues and ensure that schools are using them well.
This brings us full circle back to my New York Times piece. I hope it’s clear now why I concluded that the DeVos nomination was a “triumph of ideology over evidence.” Where school reform is needed, choice with accountability works better to achieve the wide range of goals we have for education than a free market ideology that relies on choice alone.
There are several criteria we might use to judge’s DeVos’s candidacy, but if the goal is to improve measureable student results, the evidence votes no.
— Doug Harris
Douglas N. Harris, a professor of economics at Tulane University, is the founding director of the Education Research Alliance for New Orleans and a nonresident senior fellow at the Brookings Institution.
