The Indiana Choice Scholarship Program, launched in 2011, offers a rich opportunity to study how a large-scale tuition-voucher program works and to analyze the results it has produced in its first few years. As we consider the merits of private-school choice and what it would take to make it succeed, this initiative deserves particular attention: it is the nation’s largest voucher program, accounting for nearly 20 percent of all voucher students nationwide, with 34,299 students receiving vouchers and 313 private schools participating during the 2016–17 academic year. And in terms of student eligibility, Indiana’s program is the broadest, with its vouchers aimed at both low- and moderate-income families and no cap on the number of students who can take part. The average scholarship amount, based on the public-school district in which students live, ranges from nearly $4,500 in kindergarten to $5,600 in high school.
Our four-year evaluation of the Indiana program is one of a few recent studies that finds statistically significant negative effects on student achievement of using a voucher to switch from a public to a private school in the first years after a choice program’s launch. But that is only part of the story. Our research also shows that voucher students begin to recoup their academic losses in their third and fourth years of attending a private school. Students transitioning to a private school may need time to acclimate to what are usually more-rigorous academic standards and higher expectations for homework and schoolwork. Our findings also speak only to the achievement gains of students using vouchers to switch to a private school in grades 5–8. Starting students in private schools in earlier grade levels, and thus giving them more time to adjust, might produce better outcomes.
Given that many state and federal policymakers support the expansion of private-school choice, Indiana’s experience can offer lessons for the design of future voucher programs.
Who Participates?
About 76 percent of Indiana’s private schools—and almost 100 percent of its Catholic schools—participate in the voucher program. Not long after the program began, we conducted a survey of all Indiana private schools to learn why they did or did not decide to participate. Those that chose to join most frequently specified their mission, service orientation, and school finances as reasons (see Figure 1). Schools that opted not to participate cited reasons such as the desire to maintain autonomy from government, their lack of required accreditation (and unwillingness to become accredited), and concern that enrolling voucher students would require the schools to lower their academic standards.
When examining student participation, we looked at those who joined the program early and those who are participating now, because the program has changed over time. Initially, students could receive a voucher only if they had attended a public school for at least one year, or if they had attended a private school with the help of the state’s less-generous tax credit for some parents paying private-school tuition. In 2013, the program dropped the public-school attendance requirement. Thus, in its first year, 90 percent of Indiana Scholarship students had previously attended a public school, but by 2016–17, over half—55 percent—had never attended one. In other words, the program started out serving students who wanted to leave public schools, but it now serves a majority of students who have attended private schools from day one.
As the program changed, so too did the demographics of participating students (see Figure 2). In the first year, 24 percent were African American, but this number declined to 12 percent in 2016–17. Conversely, the percentage of white students receiving vouchers increased from 46 percent in the first year to 60 percent in 2016–17. The shares of Hispanic students (20 percent) and multiracial students (6–7 percent) remained consistent over time. Statewide in 2016–17, the K–12 student population was 69 percent white, 12 percent African American, 11 percent Hispanic, and 5 percent multiracial.
In the program’s first year, 69 percent of its students lived in metropolitan areas, 16 percent in suburbs, 7 percent in rural areas, and 7 percent in small towns. By 2016–17, the proportion of students from metropolitan areas decreased to 61 percent, while the share of those from suburban areas increased to 23 percent. Statewide in 2016, about 31 percent of students in grades 3–8 lived in metropolitan areas, 27 percent in suburbs, 14 percent in small towns, and 28 percent in rural areas. (Corresponding figures for the full K–12 enrollment were not readily available.)
How the Program Works
When the voucher program first began, we conducted about 100 interviews with principals, teachers, parents, and students in 13 participating private schools. We sought to learn how schools were providing educational opportunities to students using vouchers; how schools adapted their curriculum and instruction; and how students were integrating into their new environments academically, behaviorally, and religiously.
Statewide, students receiving vouchers were low-achieving before entering private schools (on average, performing at the 42nd percentile compared to public- and private-school students statewide). Principals, teachers, and students we interviewed said that students who transferred into private schools using a voucher had not been required to do much homework in the public schools. These students moved to private schools whose students were performing, on average, at the 53rd to 57th percentile in mathematics and English language arts (ELA), respectively. Schools responded to their new students by providing more individualized instruction and in some cases, by adding an “ability” group. One teacher, for example, observed a much larger gap between her high-achieving and low-achieving students than in previous years. In response, she provided more differentiated instruction and worked with smaller groups when possible.
Voucher students themselves also had to adapt to the higher academic and behavioral expectations in their new private schools. A principal’s comment typifies what we heard in our interviews: while the school did not change its expectations, he said, teachers worked to help the new students adjust to the expectations and understand that such standards applied not just to them but to all the students.
Effects on Achievement
Because Indiana does not cap the number of vouchers awarded, it has no lottery process to determine who’s in and who’s out. Without the benefit of random assignment, we used a variety of statistical approaches to determine the program’s impact on student achievement immediately after its launch. We focused on students using a voucher to switch from a public to a private school in grades 5–8 during the program’s first four years (2011–12 through 2014–15). Because Indiana public and private schools use the same assessment in grades 3–8, we could identify public-school students who shared similar achievement trajectories and demographic characteristics with these voucher students at baseline (the year prior to a student switching from a public to a private school) and track both groups’ academic progress for up to four subsequent years.
Overall, we found an average loss in mathematics of 0.12 standard deviations (roughly 3–4 percentile points) from baseline for students who used a voucher to transfer from public to private schools (see Figure 3a). The largest losses occurred during years one and two. However, voucher students began to show signs of improvement by their fourth year in a private school, and in that year there was no statistically significant difference between them and their public-school peers in terms of total achievement gains from baseline. The negative math effects in the early years are similar to recent findings for students participating in new statewide voucher programs in Louisiana and Ohio, though smaller in magnitude. In ELA, we find no statistically significant average difference in the performance of voucher and public-school students across all four years (see Figure 3b).
Our estimates of the effects of voucher use after three and four years are based on a relatively small number of students: fewer than 200 in year four, as compared to roughly 3,000 in year one and 1,700 in year two. That’s mainly because state test scores are not available for voucher students who had reached high school by that time. Nearly 15 percent of voucher students also return to a public school within one or two years, so our longer-term estimates represent the most persistent students. Furthermore, our main results are averages across all participating private schools, and estimates of the effects of using a voucher to attend specific private schools vary widely (that is, voucher students excel in some private schools and perform poorly in others). Even so, our analysis provides the most complete picture to date of the early effects on student achievement of a voucher program operating at scale.
Implications for Program Design
Although further research is necessary to understand the variation in the Indiana program’s effects—including the organizational and instructional environments of voucher schools—our research to date offers some clear policy implications:
Allow enough time for preparation. Implementing voucher programs incrementally might be more prudent than scaling up quickly. The state legislature passed the Indiana Choice Scholarship Program in the spring of 2011, and students began using vouchers that fall. Our interviews reveal that, despite their strong support for the program, private schools felt rushed in their communication with families and enrollment of students. Giving schools, families, and students more time to prepare for change might ease adjustment all the way around.
Start smaller. Before expanding voucher programs statewide, it might be better to start with smaller programs using a lottery process. This way, the randomization can allow policymakers, educators, and researchers to assess more definitively the programs’ effects.
Measure broadly, and begin in early grades. States should rely on a wider set of measures to evaluate voucher programs and begin this appraisal at earlier grade levels. Certainly, test scores are important proxies for what students are learning, but currently there is no standardized assessment taken by both public- and private-school students in grades K–2 in Indiana. What’s more, researchers know that other outcomes (for instance, social-emotional skills, engagement, motivation, and progression through K–12 schooling) are critical when considering the impact of education reforms on students of all ages. Going forward, our study will examine the impacts of vouchers on high-school students’ achievement and attainment.
Start students early. As we mentioned at the outset, it may be that implementing voucher programs in early grade levels would better acclimate students to private schools. Voucher students in upper grade levels appear to need time to adjust to the more demanding homework and high expectations in their new schools. In this regard, it is worth noting that a majority of students currently using vouchers in Indiana have enrolled in private schools from the start.
Make it easy for good schools to participate. States implementing voucher programs should ensure the quality and number of private schools willing to take part. About three quarters of those in Indiana are participating. This contrasts with Louisiana, where the supply of private schools has posed a challenge, likely due to the regulatory burden they would face if they signed on.
Consider teacher training. Private-school educators may need additional training to prepare for serving voucher students. Our findings suggest that teachers would benefit from professional development in mathematics curriculum and instruction and in learning to lead more-diverse classrooms.
Although we have much to discover about the impacts of statewide voucher programs, we are beginning to understand what policymakers and educators should consider when implementing new ones. Until research can show the conditions under which voucher programs succeed, the policy debates will rage on. In the meantime, Indiana’s experience to date provides useful guidance for states intent on moving ahead now.
This is part of a forum on private school choice. For alternate takes, see “Programs Benefit Disadvantaged Students,” by Patrick J. Wolf, or “Still Waiting for Convincing Evidence,” Douglas N. Harris.
This article appeared in the Spring 2018 issue of Education Next. Suggested citation format:
Wolf, P.J., Harris, D.N., Berends, M., Waddington, R.J., and Austin, M. (2018). Taking Stock of Private-School Choice. Education Next, 18(2), 46-59.
