In a stereotypical image of a high-school debate tournament, straight-A students compete to see which renowned prep school team comes out on top. Increasingly, this is no longer the case: in recent decades, nonprofit organizations have been working to expand access to debate in public school systems that serve large concentrations of low-income students and students of color. More than 10,000 students from 20 cities participated in debate tournaments last year, according to the National Association for Urban Debate Leagues.

That includes the Boston Debate League, which was founded in 2005 to “develop critical thinkers ready for college, career, and engagement with the world around them.” The league supports teachers to launch and coach debate teams and runs monthly after-school debates for middle- and high-school students, among other initiatives (see “Making the Case for Student Debate Leagues,” features). While the immediate virtues of debate are easy to spot—teenagers research real-world topics, practice public speaking, and use evidence in support of their arguments—we wanted to know whether that translates into better academic achievement and attainment. Does participation in formal debate programs improve student outcomes?

First, we look at individual debaters’ reading and math test scores over time and compare students to themselves in years when they do and do not participate in debate. When students are on a debate team, their reading scores improve by 13 percent of a standard deviation, or about two-thirds of a typical year of learning. We find the biggest gains are for students with the lowest elementary-school test scores and reflect improvements in literacy skills related to critical thinking and reading comprehension. The impacts on math scores are minimal.

We also examine how debate affects high-school graduation and postsecondary enrollment by comparing debaters to similar peers who attended schools that did not offer debate. We find positive impacts on graduation and postsecondary enrollment, mainly driven by increased enrollment in four-year colleges. Debaters are 17 percent more likely to graduate high school within five years and 29 percent more likely to enroll in a postsecondary institution.

While many reading interventions target younger students, our results reveal a high-impact strategy to boost literacy skills and post-secondary outcomes for teenagers—particularly those whose low test scores and socioeconomic status typically pose high barriers to college success. Our results provide policymakers with a rare promising strategy for reducing inequality in reading achievement, analytical thinking skills, and educational attainment during students’ high-school years.

Potential Benefits of Policy Debate

Policy debate is an interscholastic, competitive, extracurricular activity for which teams of students engage in structured argumentation about public policy issues. Participants focus on a single topic for an entire academic year, such as arms sales, criminal-justice reform, or immigration policy, and work in two-person teams to research and develop policy proposals and arguments that support them. In tournaments, teams take on affirmative or negative positions, present their proposals, and cross-examine one another in a fast-moving sequence lasting 75 to 90 minutes. Policy debate students rely on their knowledge, effective use of evidence, ability to speak persuasively, and how well they can think on their feet.

Why might we expect all of this to pay off academically? First, successful debaters construct and deliver compelling arguments that are well-supported by both reasoning and evidence. In addition, the research aspect of policy debate includes reading and interpreting advanced non-fiction texts and social science research, while competitive debating includes quickly reading, analyzing, and refuting unfamiliar texts that opponents submit as evidence. Debaters are trained to consider both the content and relative credibility and objectivity of source materials. These skills are assessed on state reading tests and support advanced coursework in high school, including writing papers and participating in class discussions.

Debate also may provide a mechanism for motivating academic engagement. Rather than passively listening to an adult deliver a lecture, debaters are at the front of the room, creatively engaging with content they have mastered. The topics are directly related to high-interest current events and invite students to pair academic work with questioning authority, by recommending what the government should and should not do. And because timed tournament play moves quickly, is designed to engage the audience, and involves competition with other schools, debate teams and leagues can energize a school population as a whole, much like interscholastic sports. These events call on an array of softer skills, such as time management, independent organization, and teamwork. Competition also exposes students to a college-going culture, as tournaments are often held on college campuses and judged by current or former college-level debaters.

Assessing Impacts in Boston

Our study focuses on the Boston Debate League, which supports 40 school-based teams in public middle and high schools in Boston, Chelsea, and Somerville, MA. We look at 10 years of individual students’ league participation data, from 2007–08 to 2016–17, and match that with demographic and academic-achievement data from the Boston Public Schools. We also use data from the National Student Clearinghouse, which shows students’ high school graduation status, postsecondary enrollment status, and whether they enrolled in a two-year, four-year, public, or private institution.

Our sample includes 3,515 students who ever participated in a debate team. These students attend schools that serve disproportionate shares of low-income families and where students’ average elementary-school reading and math scores are more than one-quarter of a standard deviation lower than schools not in the league. Some 82 percent of students at debate schools qualify for free or reduced-price school lunch and 36 percent are English language learners compared to 68 percent and 26 percent of students, respectively, at non-debate schools. The group of debaters we study is 42 percent Black, 39 percent Hispanic, 9 percent white, and 8 percent Asian. The typical debater began in the ninth grade, and a large majority only participated for a single academic year. Twenty eight percent participated in middle school.

Debaters are a self-selected group—the team is a voluntary, after-school activity, and tournaments are held offsite on evenings and weekends. We examine baseline characteristics of debaters and students at debate schools who never join a team and find notable differences. Debaters have higher elementary-school reading scores, better attendance rates, and are less likely to receive special-education services than their classmates who choose not to join the team. They also are more likely to be female, Black, and economically disadvantaged.

Because of these non-trivial contrasts and the opt-in nature of the teams, it is likely that debaters and their non-debating classmates differ from one another in ways unrelated to debate. Therefore, for part of our analysis, we look to another group of students to serve as a comparison group: students attending schools that were not in the league and therefore could not choose whether to join a team. These students are more similar to debaters in terms of baseline test scores and are likely non-debaters because the program was not available to them.

Effects on Academic Performance

First, to assess the impact of debate on academics, we compare debaters to themselves over time. Our analysis looks at individual students’ test scores, attendance, and suspension records to test whether performance is different in years when students did and did not participate in debate.

Debaters earn higher scores on reading tests in the years when they participate in debate, and those benefits increase the longer students spend on the debate team (see Figure 1). Among all students who ever debated in school—who spend an average of 1.4 years on the team—reading scores increase by 13 percent of a standard deviation in the years they participate. Scores for students who spend just one year on the team increase by 10 percent of a standard deviation compared to 14 percent for students who spend four years on the team. Among the very small group of students who start in middle school and debate for five years, reading scores are 36 percent of a standard deviation higher.

In math, we do not find strong evidence that debate has a positive impact, although we see no evidence of harm. However, the math results do provide another insight: the much smaller math impacts relative to reading gives us confidence that our reading impact estimates are not simply an artifact of selection.

We also investigate which literacy skill gains drive the increase in debaters’ reading scores by looking at which test items exhibit the biggest differences in student performance. We compare performance on “language” items, which test grammar, vocabulary, and punctuation knowledge, with performance on “reading” tasks, which focus on comprehension and analysis, such as identifying the main idea of a passage or supporting evidence for a claim. The positive impacts for debaters are nearly twice as large in more sophisticated reading tasks, at 10 percent of a standard deviation, than in language, at 6 percent of a standard deviation.

Interestingly, although debaters are generally higher performing than students in the same schools who never join debate, our analysis shows that the largest gains from debate are among students who had the lowest reading scores at the start of sixth grade (see Figure 2). When they participate on a debate team, students who were in the bottom quartile in elementary-school reading experience gains of 24 percent of a standard deviation compared to 10 percent of a standard deviation for students with the best elementary-school performance.

Finally, we also assess the impacts of debate participation on student attendance and behavior, as measured by how many days students are suspended from school. Overall, students have slightly better attendance in years they participate in debate, with an increase of 1.7 percent in days present. The impact on suspensions is minimal. However, in looking at the small group of students who start in middle school and spend five years on a debate team, we find the number of days present grows by 4 percent and the number of days suspended falls by about one-fifth.

Most likely, these comparisons produce conservative estimates of the impacts of debate because every student in our sample has participated at least once. Even after a student leaves a debate team, they may carry those experiences and learning gains with them for some unknown length of time. Therefore, our comparison between participating and non-participating years may understate the true impact of debate participation on academic achievement, since our non-participant group includes students who have already benefitted from debate.

On the other hand, these estimates may camouflage other factors contributing to the impacts of debate, such as students choosing a high school in order to join the debate team. Therefore, we also analyze our data by excluding students who debate for multiple years and by excluding students who started debate in grade 9. We do not see meaningful changes to our results, indicating that our preferred estimates capture the impact of debate participation itself.

Effects on Graduation and College Enrollment

To study the impacts of policy debate on students’ postsecondary outcomes, we use a different comparison group: demographically similar students at schools that do not offer debate. We find that debate has substantial effects on both high-school graduation and college enrollment (see Figure 3). Some 80 percent of debaters graduate high school in five years compared to 68 percent of non-debaters, an increase of 17 percent. In addition, 53 percent of debaters enroll in a postsecondary institution within two years of their expected high-school graduation date compared to 41 percent of non-debaters, an increase of 29 percent. As with the impacts on academic outcomes, we find large differences when comparing debaters by their baseline reading performance at the start of middle school. Debaters with low elementary-school reading scores experience the greatest gains in post-secondary outcomes: they are 25 percent more likely to graduate high school in five years and 55 percent more likely to enroll in a postsecondary institution, based on gains of 16.4 and 20.5 percentage points, respectively.

We also find big increases in the share of students enrolling in four-year institutions after graduating high school, with the largest gains for students with the lowest elementary-school reading scores (see Figure 4). Overall, debaters are 38 percent more likely to enroll in a four-year school and 28 percent more likely to enroll in a two-year school, based on gains of 12 and 4 percentage points, respectively. Students in the lowest quartile are 16 percentage points more likely to enroll in a four-year college after graduation compared to 9 percentage points for students with the highest baselines scores.

Policy Implications for Policy Debate

Most reading interventions are focused on the early elementary years, and third grade reading proficiency is viewed as a bellwether for success in adulthood. But what about the nine years of school that follow? We find substantial positive impacts for teenage students, the majority of whom are low-income students of color, when they participate in a competitive high-school policy debate team. Debaters make outsized progress in mastering sophisticated literacy skills and are more likely to graduate high school and enroll in college—and the biggest gains are among the students the farthest behind at the end of fifth grade. It’s never too late to accelerate student progress.

The average improvement in debaters’ reading scores is comparable to two-thirds of a year of learning and about 20 percent of the gap in 8th-grade reading between students who do and do not qualify for subsidized school lunch. Prior research has uncovered few interventions that generate literacy impacts of this magnitude for secondary school students.

Further, the positive impacts on reading scores from participating in debate are twice as large for students with the lowest baseline levels of proficiency than for students with average scores, and we find a similar pattern of results for postsecondary outcomes. Debate programs therefore have the potential to reduce educational inequality by accelerating improvement most dramatically for the students who struggle most.

These programs also are inexpensive relative to other interventions. For example, the current per-pupil cost of the Boston Debate League is about $1,360 compared to about $2,800­ for high-dosage tutoring, such as the well-regarded Match Education program. Prior research has found that students’ reading performance improves by 15­ percent to 25 percent of a standard deviation after tutoring. Therefore, policy debate programs appear to generate up to double the impact on reading test scores per dollar compared to state-of-the-art high-dosage tutoring.

Our study is not without limitations. Only a small subset of Boston students, all of them volunteers, participate in debate, and we can’t speak to what would happen if students were required to join. We also can’t fully rule out the possibility that some or all of the estimated effects on postsecondary outcomes are driven by selection bias, particularly because the postsecondary impact estimates are quite large.

However, our finding that the gains in reading scores are concentrated on analytical thinking competencies rather than foundational language rules and conventions strengthens our confidence that our results reflect the impact of debate participation, not some other unobserved factor. This finding also suggests that policy debate develops students’ critical thinking skills, another goal for which evidence-based strategies are in short supply. Future research should probe this finding further with better measures of critical thinking, argumentation skills, and other competencies needed for academic and civic participation such as social perspective taking, media literacy, the ability to distinguish fact from opinion, and engagement with the policy process.

Beyond highlighting the value of formal debate programs, we believe these findings also have implications for classroom instruction. A handful of organizations, including the Boston Debate League, have developed and implemented professional development programs to help teachers infuse debate pedagogy into regular classrooms. Often called “debate-centered instruction,” the goal is to give more students the opportunity to benefit from debate-like learning opportunities, not just those who can choose to take part in an intensive out-of-school program. The potential for such instruction to accelerate reading development, particularly for students far behind grade level, is an important subject for future research. While our study demonstrates exciting results for extracurricular debate participants, there may be even greater dividends to incorporating some of these practices into regular classroom-based instruction, to reach all students.

Beth E. Schueler is an assistant professor at the University of Virginia. Katherine E. Larned is a doctoral candidate at the Harvard Graduate School of Education.

This article appeared in the Summer 2024 issue of Education Next. Suggested citation format:

Schueler, B.E., and Larned, K.E. (2024). Resolved: Debate Programs Boost Literacy and College Enrollment: How debaters become better students. Education Next, 24(3), 52-59.

Americans with more education vote at higher rates. In the 2020 presidential election, 77 percent of eligible voters who had attended or graduated from college and 90 percent with post-graduate studies cast a ballot compared to 54 percent of voters with only a high-school diploma and 36 percent of dropouts. These trends in turnout rates have persisted for more than three decades, suggesting a link between years of schooling and voting. But does achieving higher levels of education cause citizens to show up and vote on election day? Or do education and voting simply go hand-in-hand, because some other variable contributes to them both?

The research to date is mixed. Some studies have found evidence of a causal relationship, while others have not. The available data also tell us little about why and how education increases voting.

We take on these questions by looking at the educational trajectories and adult voting records of students who attend charter schools in Boston. We focus on Boston because prior research has found that students who attend a city charter are more likely to pass high-school exit exams, have higher test scores, and are more likely to attend a four-year college than their non-charter peers. Further, because Boston charters are oversubscribed and enroll students based on random admissions lotteries, we can compare charter students, who receive more education, with similar students who did not win a lottery and therefore receive less education. If education is a causal factor in voting, we’d expect to find that the students who experience these academic gains are also more likely to vote as adults.

That is, in fact, what we find—but only for girls. We look at the voting records of charter and non-charter students and find substantial differences. While similar shares of charter and non-charter students are registered to vote by age 21, charter-school students are slightly more likely to vote in any election and substantially more likely to vote in the first presidential election for which they are eligible. Specifically, 41 percent of all charter-school students vote in their first presidential election compared to 35 percent of students who did not attend a charter, an increase of 17 percent.

When we look more closely at the data, we see that the charter effect is a female phenomenon. Female high-school students are 11 percentage points more likely to vote in adulthood if they attended a charter school, while the impact for males is nil. We investigate multiple explanations for these differences and find that increased civic participation is likely due to gains in noncognitive attributes like grit and self-control, which we measure by looking at student behaviors, such as school attendance and taking the SAT.

These findings are in line with widening gender gaps in educational attainment and political participation. In 2020, 82 percent of eligible women voted in the presidential election compared to 73 percent of eligible men. Meanwhile, in 2021 some 39 percent of women ages 25 and older had a bachelor’s degree compared to 37 percent of men, and males currently account for just 42 percent of all students at four-year colleges. Our research sheds new light on these patterns and points to a critical question for future study. What can schools do to enhance non-cognitive skills development in boys, and what intervention could boost civic participation in young men after graduation?

Academic Success at Boston Charter Schools

Charter schools are public schools, funded with public money, but managed by private organizations. In Massachusetts, the state board of elementary and secondary education authorizes charter schools for five-year terms, and for-profit charter operators are not permitted. State law caps the share of district funds that can be used for charter tuition, with limited flexibility. If a school cannot enroll all interested students, they conduct a random admissions lottery, enroll the winners, and place students who did not win on a waitlist. For the 2023-24 school year, some 76 charter schools statewide enrolled about 46,000 students, and 66 of those schools had waitlists with another 21,270 unique students.

Boston has the highest concentration of charter schools in the state. Most use policies associated with the “No Excuses” charter school movement: longer school days and years, a focus on academic achievement and behavior management, in-school tutoring, frequent teacher feedback, and data-driven instruction. Prior research has found that attending a Boston charter school for one year boosts student scores on standardized tests by about one-third of a standard deviation in math and one-fifth of a standard deviation in reading. These findings are generally in line with studies of similar charter schools in Chicago, Denver, Los Angeles, New York City, Newark, New Orleans, and the national non-profit KIPP network.

Our study looks at the voting behavior of young adults who applied to a randomized admissions lottery for a Boston charter high school. We include all charter middle and high schools that kept lottery records and enrolled students who were at least 18 by the 2016 general election. In all, that includes 12 charter schools and 9,562 lottery applicants who were scheduled to graduate between 2006 to 2017. The applicant pool is 58 percent Black, 27 percent Hispanic, and 10 percent white. About 20 percent receive special-education services and 74 percent qualify for free or reduced-price school lunch. Females account for 52 percent of applicants.

Through the lotteries, about two-thirds of applicants are offered a charter seat. This creates a natural experiment that we use to explore the potential causal link between charter-school attendance, which boosts academic scores and access to college, and voting. We use state education and voting records to compare academic outcomes and election turnout for students who are and are not offered a charter seat and adjust our estimates based on who actually attends a charter school. We do not include siblings of current students or other applicants who receive lottery preferences. Of course, not all students offered seats attend the charter; however, state data show that applicants who win the lottery are 46 percentage points more likely to attend a charter during their time in Massachusetts public schools. We also see that boys and girls are equally as likely to enroll in a charter school if offered a seat.

Linking Learning with Voting

First, we benchmark the impact of charter attendance on academic outcomes against results from prior research. As in other analyses, we find that students who enroll in a charter school experience large gains in AP test-taking and scores, SAT scores, and four-year college enrollment. On state tests, scores increase by about half of a standard deviation in math and one-third of a standard deviation in reading two years after winning an admissions lottery. Charter students take longer to graduate high school, with a decline of 9 percentage points in the four-year graduation rate, but there are no statistically significant differences in five- or six-year high school graduation rates. Boston charters boost enrollment in four-year colleges by 7.2 percentage points.

We then investigate whether these educational gains extend beyond the classroom to civic participation. We find no impact on voter registration—about 78 percent of students in both groups are registered to vote by age 21, with about 45 percent of students registered by their 19th birthday. However, we do find differences in voter turnout. We focus on the first possible presidential election after students turn 18 to leave less time for them to leave Massachusetts or the region, and thus our sample. Additionally, the first possible presidential election is the election closest to the charter school treatment, which we believe is most likely to show the influence of attendance.

Charter-school students are more likely to vote than non-charter students, with the biggest difference in the first presidential election in which they are eligible to vote (see Figure 1). Some 41 percent of charter students vote in the first presidential election after they turn 18 compared to 35 percent of non-charter students, a difference of 17 percent. Charter students are also more likely to vote in any presidential election, with turnout at 65 percent compared to 61 percent for non-charter students. In looking at all opportunities to vote, including off-cycle elections where turnout is generally very low, we find a difference of 3 percentage points, with 67 percent of charter students voting compared to 64 percent of non-charter students, though the difference is not statistically significant.

We also look at voting by student subgroups and find that female charter students experience outsized gains (see Figure 2). In terms of voting in the first possible presidential election, the charter impact is 11 percentage points for girls and zero for boys. We also find meaningful effects for other student subgroups. Voting increases by 7.5 percentage points for students who receive free or reduced-price school lunch, 12.1 percentage points for English language learners, and 11.3 percentage points for students who earn relatively higher scores on state tests.

“Soft Skills” and the Ballot Box

Our findings show that charter schools boost academic outcomes and civic participation. That raises a second question: how? What aspects of education contribute to students’ likelihood to vote as adults?

We look at five possible explanations of why education may increase voting: development of cognitive skills, civic skills, social networks, the degree to which charter attendance politicizes students, and noncognitive skills. Our finding of a gender gap in voting allows us to identify proxies for these mechanisms and test the impact of each one. If the gender gap we find in voting is also present on a proxy measure, that mechanism is the most likely to explain increased civic participation among female charter school graduates.

For example, to assess whether increased cognitive skills help explain why citizens with more education are more likely to vote, we compare the impact of charter attendance on average test scores in reading and math for the males and females in our sample. Both genders experience the same large increase in math scores, while the positive impact in reading is slightly bigger for males. Since these impacts do not mirror the female-only effect of attending a charter school on voting, cognitive skill development does not appear to influence civic participation. More knowledge doesn’t necessarily beget more voting.

We conduct similar analyses of proxies for the other four mechanisms and find evidence that development in one area appears to explain charters’ impact on voting: noncognitive skills. While our data do not include a direct measure of noncognitive skills, such as a survey-based measure of self-control or grit, we use high-school attendance and taking the SAT as a proxies, since they are related to persistence and follow-through. This approach builds on prior research and captures some of the attitudes and behaviors students would draw on in order to vote, as voting in the U.S. often involves navigating sign-up processes, planning ahead, and following through.

Overall, students at charter schools attend 12 additional days of school from grades 9-12 compared to non-charter students. However, this effect is driven entirely by girls. Female charter students attend 22 additional days of school compared to non-charter females, while charter males do not attend school more regularly than their non-charter counterparts. We find similar, but not statistically significant, differences in SAT taking: charter females are 8 percentage points more likely to take the SAT than non-charter females, while the effect of charter attendance for males is just 2 percentage points.

This evidence cannot prove that stronger noncognitive skills cause a boost in voting. But taken together, we see that charters appear to shift noncognitive skills more for girls than boys, and that these differences align with the observed pattern in voting gains. Further, the gender gap in noncognitive skill gains we observe is consistent with prior research. Studies have shown that girls enter kindergarten with greater noncognitive skills than boys, maintain their advantage through elementary school, and have greater self-discipline than boys in 8th grade. Other research has found that these differences explain 40 percent of the gender gap in college attendance. There is also research showing that girls may gain more noncognitive skills from educational interventions, and that conscientiousness and emotional stability increase voter turnout for women, but not men. Thus, girls—perhaps because of socialization—are more likely to turn gains in noncognitive skills into voting.

Although our study finds the main beneficiaries of civic gains are young women, education’s contribution to voting need not operate solely through girls. Interventions that increase noncognitive skills for boys may have similar effects, though we do not observe them in this context. It is also possible that U.S. schools, and charter schools specifically, are set up in such a way that they particularly develop the skills of girls but not boys. Research to date has mainly focused on the overall impact of noncognitive skill development through social and emotional learning programs or documented longstanding gender gaps in this arena. Interventions that boost noncognitive skill development and other lagging outcomes in boys (see “Give Boys an Extra Year of School,” reviews, Spring 2023) or school curricula that specifically target civic engagement (see “A Life Lesson in Civics,” research, Summer 2019) are areas ripe for further study.

Sarah R. Cohodes is associate professor at the Gerald R. Ford School of Public Policy at the University of Michigan. James J. Feigenbaum is assistant professor at Boston University.

This article appeared in the Summer 2024 issue of Education Next. Suggested citation format:

Cohodes, S.R., and Feigenbaum, J.J. (2024). Why Education Increases Voting: Evidence from Boston charter schools. Education Next, 24(3), 60-65.

Educators and researchers have been fighting the reading wars for the last century, with battles see-sawing literacy instruction in American schools from phonics to whole language and, most recently, back to phonics again. Policymakers have entered the fray, after more than a quarter-century of stagnant reading scores in the United States. Over the last decade, 32 states and the District of Columbia have adopted new “science of reading” laws that require schools to use curricula and instructional techniques that are deemed “evidence-based.”

Such reading programs include direct instruction in phonics and reading comprehension skills, such as finding the main idea of a paragraph, and efforts to accelerate learning tend to double down on more of the same skill-building practice. But research increasingly points to another critical aspect of literacy: the role of student knowledge. For example, prior research by two of us found that a young child’s knowledge of the social and physical world is a strong predictor of their academic success in elementary school. And advocates for knowledge-based education often cite the so-called “baseball study” where students reading a passage about baseball who knew about the sport were far better at understanding and summarizing the story than students who didn’t, regardless of their general reading skills.

Knowledge-building reading curricula are rooted in these insights, and use materials and activities based on a sequence of integrated science and social studies topics, texts, and vocabulary. Yet the potential value of this approach is often an afterthought in state and district efforts to strengthen reading instruction, and the benefits to students of combining evidence-based curriculum with systematic efforts to build student knowledge have yet to be rigorously documented.

We conduct the first-ever experimental study of this topic, based on randomized kindergarten-enrollment lotteries in nine Colorado charter schools that use an interdisciplinary knowledge-based curriculum called Core Knowledge. To assess the long-term impact of experiencing a knowledge-building curriulum on student learning, we compares performance on statewide tests in grades 3–6 between kindergarten lottery winners who attended a Core Knowledge charter school with lottery losers who could not enroll.

We find that winning an enrollment lottery and enrolling in a Core Knowledge charter school boosted long-term reading achievement in 3rd to 6th grade by 16 percentile points, as compared to comparable applicants who did not win their enrollment lottery. The size of this gain is approximately equivalent to the difference between the mediocre performance of U.S. 13-year-olds on the 2016 Progress in International Reading Literacy Study and that of top-scoring countries like Singapore and Finland. Our results are also notable in their contrast with other studies of reading interventions, which typically find small, short-term effects.

Students and teachers in many public elementary schools spend up to two hours each day on reading instruction. While the component skills of literacy are critical to student development and learning, our findings point to a missed opportunity to accelerate literacy by building knowledge at the same time. Skill building and knowledge accumulation are separate but complementary cognitive processes, and while the adage “skill begets skill” may be true, a fuller description of cognitive development could be “skill begets skill, knowledge begets knowledge, and skill combined with knowledge begets them both.”

Kindergarten Lotteries for “Core Knowledge” Charters

The Core Knowledge curriculum was created in the 1980s by E.D. Hirsch, Jr., a researcher and advocate of knowledge-building education. Its content and activities follow a planned sequence of the knowledge and skills students should accumulate and master in grades K–8 in all academic subjects and the arts. This “knowledge-based schooling” approach is rooted in the belief that a common base of shared knowledge is foundational for not just individual students’ reading comprehension abilities but also for our ability as a society to communicate and promote equal opportunity. An estimated 1,700 schools across the U.S. use the curriculum today, including more than 50 in Colorado.

To assess the impact of the Core Knowledge curriculum on student achievement, we look at nine oversubscribed Colorado charter schools that all use the curriculum, had been open for at least four years, and held random enrollment lotteries to register kindergarten students in either or both of the 2009–10 and 2010–11 school years. Our study includes 14 separate lotteries with 2,310 students, almost all of whom are from high- or middle-income families.

These families generally have a range of schooling options, including private schools, other charter schools, and public schools outside their district under Colorado’s open-enrollment law. About one in five students in our sample applied to multiple charter lotteries—usually two instead of one. Some 41 percent won at least one lottery, and 47 percent of winners enrolled in that school. In all, 475 lottery winners went on to attend a Core Knowledge charter, while 1,356 students did not win the lottery and attended school elsewhere. In analyzing the effects of attending a Core Knowledge charter, we take into account the fact that not all lottery winners actually enrolled.

Attrition and Family Choice

We base our analysis on the performance of lottery applicants on the Partnership for Assessment of Readiness for College and Careers (PARRC) reading and math tests in grades 3, 4, 5, and 6, as well as the 5th-grade science PARRC test. By looking at these scores, we can compare the performance of students who did and did not experience a knowledge-building curriculum over up to seven years of their schooling.

However, roughly 36 percent of students in our sample did not complete all scheduled PARCC tests through grade 6, and the attrition rate for students who did not win the enrollment lottery is 5 percentage points higher than for lottery winners. Detailed student data reveals three major factors at play. First, some students stop participating in Colorado’s PARCC testing because they move out of state, transfer to a different school, or are homeschooled. A second group of students don’t have test-score data because they are exempted as language learners or special-education students. Third, other students are off-track with their expected kindergarten cohort in later years because of delayed kindergarten entry (“redshirting”) or due to having skipped or repeated a grade.

To ensure that this attrition does not skew our results, we exclude from our analysis both the four lotteries with the highest rates of differential attrition between lottery winners and losers and the youngest applicants, who are more likely to be redshirted by their parents regardless of their lottery outcome. We also adjust our results for students’ gender, race or ethnicity, and eligibility for a free or reduced-price school lunch to ensure that any demographic differences between lottery winners and losers do not introduce bias.

Accelerated Achievement

We find positive long-term effects on reading performance for students who are randomly selected by a kindergarten enrollment lottery and attend a Core Knowledge charter school. Across grades 3–6, these students score 47 percent of a standard deviation higher in reading than comparable lottery applicants who did not have a chance to enroll. This is equivalent to a gain of 16 percentile points for a typical student (see Figure 1). Students who attend a Core Knowledge charter also make outsized gains in science of 30 percent of a standard deviation, which is equivalent to a gain of 10 percentile points. Effects in math are positive, at about 16 percent of a standard deviation, but fall short of statistical significance.

The effects are slightly larger for female students than males (see Figure 2). In reading, female Core Knowledge charter students score 50 percent of a standard deviation higher compared to 44 percent for males, for a gain 17 of percentile points compared to 15 percentile points for males. Females gain about 12 percentile points in science and 9 percentile points in math, while males gain 6 percentile points in science and experience no gains in math. We also look at effects by student grade level and find no upward or downward trend, suggesting the effects may have stabilized by 4th grade (see Figure 3).

While prior non-experimental research has documented stronger reading performance among students who already have knowledge about a topic, our analysis shows positive long-term impacts in reading from systematically building student knowledge over time. In our view, these results suggest that the “procedural skills” approach that has dominated reading comprehension instruction over the last 30 years in public schools is less effective than a “knowledge-based” approach that teaches skills and also is designed to build a body of knowledge as the main mechanism for increasing comprehension.

These findings also build on the body of evidence linking students’ levels of general knowledge to achievement in reading, science, and math. Research also shows that levels of general knowledge are strongly correlated with socio-economic status and parental levels of education. However, unlike these factors, knowledge is malleable through curricular choices. The intervention we study, where students experience seven years of a knowledge-building curriculum, appears to set off a long-term, compounding process whereby improved reading comprehension leads to increased knowledge, and increased knowledge leads to even better comprehension.

A Call to Build Knowledge About “Knowledge”

In addition to informing current-day decision-making, we believe these results should inspire a new research and policy agenda to measure and track students’ knowledge development and understand the mechanisms involved in knowledge-building curricula. The effects our study finds are similar in pattern and magnitude to earlier non-experimental evidence, which suggests that gains in students’ general knowledge could have a larger effect on future achievement than similar gains in more widely studied non-cognitive domains, such as executive function, visual-spatial and fine motor skills, and social and emotional development.

The potential benefits of knowledge-building curricula could be far-reaching. The compounding process our analysis reveals would occur not only in reading, but also across all subjects to the extent that they depend primarily on reading comprehension for learning. Moreover, these achievement gains across all subjects would likely extend into future years, as increased comprehension in one year leads to increased knowledge and comprehension in the next, and so on. We believe that these curricula could also increase students’ educational attainment and future labor market success.

However, elevating student knowledge to a more central place and higher priority in research and policy will require a significant conceptual shift—the term “building knowledge” does not readily trigger a conceptual map linking the intervention to higher achievement, unlike common interventions like reducing class size, extending the school day, and raising teacher pay.

Well-designed measures of student knowledge should be considered as an important addition to other national measures for students in elementary grades. To be sure, they will carry an additional challenge. Any definition and measures of “general knowledge” will need both scientific validity and political viability at a moment when attempts to ban library books and shape course content are on the rise. Attempting to define what all public-school students should know will undoubtably trigger debates and a variety of viewpoints. However, the evidence points to building knowledge as a critical foundation of student literacy with potentially lifelong effects. The benefits of skillful reading and broad knowledge should be a shared starting point, from which a stronger approach to reading instruction can grow.

David Grissmer is research professor in the School of Education & Human Development at the University of Virginia, where Richard Buddin is education consultant, Jamie DeCoster and Tanya Evans are research assistant professors, and Chris S. Hulleman is research professor. Thomas G. White is a former senior researcher at the School of Education & Human Development. Daniel T. Willingham is professor of psychology at the University of Virginia. Chelsea A.K. Duran is a postdoctoral fellow at the Curry School of Education at the University of Virginia. Mark Berends is professor at the University of Notre Dame. William M. Murrah is associate professor at Auburn University.

This article appeared in the Spring 2024 issue of Education Next. Suggested citation format:

Grissmer, D., Buddin, R., Berends, M., White, T.G., Willingham, D.T., DeCoster, J., Duran, C.A.K., Hulleman, C.S., Murrah, W.M., and Evans, T. (2024). How Building Knowledge Boosts Literacy and Learning: First causal study finds outsized impacts at “Core Knowledge” schools. Education Next, 24(2), 52-57.

Since its founding 34 years ago, Teach For America has prepared nearly 70,000 recent college graduates and career-changers to teach in high-poverty schools across the United States. The selective program recruits mission-driven applicants, with annual admissions rates as low as 12 percent. Unlike traditional teacher-preparation programs, it places “corps members” in short-staffed public schools after an intensive summer seminar, before they are fully certified to teach, and provides additional support and training during a two-year stint in the classroom.

The program, which is part of Americorps and known broadly as TFA, has been both successful and controversial. Its theory of action is that bringing highly educated, mission-motivated “future leaders” into public schools in low-income communities will positively impact student learning, influence professionals to commit themselves to the issue of educational equity, and give educational equity a firmer foothold in the nation’s agenda. Prior research has found encouraging evidence on all counts. TFA teachers perform as well or better than their most likely alternative during their first two years in the classroom, and compelling quasi-experimental evidence suggests participating in TFA shapes alumni’s worldviews. As one example, TFA participants are more likely to view student achievement gaps as a symptom of systemic social injustices rather than a symptom of individual choices or behaviors (see “How Teach For America Affects Beliefs About Education,” research, Winter 2020). In addition, TFA alumni are well represented in state departments of education and have founded some of the nation’s largest charter-school networks (see “Creating a Core of Change Agents,” feature, Summer 2011).

The program remains controversial, however. TFA teachers are usually new to education, and critics worry that the program provides students who arguably have the greatest need for excellent instruction with the least-prepared novice teachers. In addition, TFA teachers commit to just two years in the classroom. In general, teachers improve rapidly during their first five years, which means that TFA teachers are scheduled to move on before they fully get the hang of the job.

These concerns, and the evidence base to date, overlook TFA teachers who continue teaching for five years or more. To learn how these teachers perform over the longer term, I look at estimates of value-added, or the estimated contribution to students’ test scores, of TFA teachers in New York City between 2012 and 2019, when about 25 percent remained teaching after year five. I compare that to the same estimates of non-TFA teachers working in TFA-participating schools, 43 percent of whom taught for at least five years.

TFA alumni who choose to keep teaching improve more rapidly over the first five years of their career than non-TFA teachers. The average TFA teacher who stays in the classroom for five years or more improves at more than double the rate of non-TFA teachers, at 13 percent of a standard deviation by year six compared to 6.3 percent of a standard deviation.

I then explore the question of whether regular turnover within the TFA workforce neutralizes or offsets these effects. To examine this issue, I compare student performance under two hypothetical scenarios: a district policy to continually hire TFA teachers to replace exiting corps members, or a district policy to never hire any TFA teachers at all. My results show a district policy of continuous TFA hiring would boost student achievement by an estimated 5 percent of a standard deviation relative to a policy of no TFA hiring. So long as exiting TFA teachers are replaced by new TFA recruits, student achievement will be higher, on average, than if the district did not hire any TFA corps members.

To be clear, hiring is complex, and teachers’ projected effects on student achievement should not be the only criterion driving staffing decisions. This research addresses a singular concern—that high turnover among TFA teachers drives down student achievement over the long run. Overall, I find that the data does not substantiate this concern, given the rapid rate at which TFA teachers improve on the job.

Assessing an Alternative Teacher Pipeline

TFA was launched in 1990 and rooted in founder Wendy Kopp’s undergraduate senior thesis at Princeton University. The thesis called for the creation of a new public-service program to train college seniors who did not major in education to teach in high-poverty schools for two years, building on a wave of new alternative-route certification laws that had been adopted in 20 states. From TFA’s earliest cohort of 489 recruits, the non-profit has expanded to 50 cities and regions nationwide. Over that same time, the share of U.S. teachers trained through alternative-certification programs has grown to nearly one in five. By 2023, every state except Alaska, Oregon, and Wyoming had adopted an alternative-route certification law.

Kopp was inspired by the recruiters from investment banks and consulting firms, who aggressively scouted talent and offered promising students an impressive launching pad after graduation. TFA recruiters used a similar approach, and the program quickly became a sought-after post-collegiate option; in 2011, fully 18 percent of Harvard’s graduating class applied. The competitive nature of the short-term program and demographics of its idealistic, highly educated corps members, who were predominantly wealthy and white at that time, attracted skepticism and critiques. Over the last decade, the organization has shifted its recruitment and training practices to achieve a more diverse corps of educators whose background or ethnicity matches that of their students. Today, about 60 percent of TFA corps members are Black, Indigenous, or people of color, compared to 20 percent of all U.S. teachers. In addition, 45 percent of TFA corps members are first-generation college graduates.

Another important change is the growing number of TFA alumni who have continued to work in the classroom—nearly one in four, or 14,600 teachers nationwide. To fully understand the impact of TFA on student learning, we need to assess the performance of these alumni over the longer term.

The New York City Department of Education, the nation’s largest school district, has hired nearly 6,000 TFA corps members since 1990. My analysis is based on New York City public schools that hired both TFA and non-TFA teachers between 2012–13 and 2018–19. These schools tend to have relatively lower standardized test scores, greater shares of students of color, and more students from low-income families. On average, student performance is 36 percent of a standard deviation below the citywide mean.

The data include teacher salary data, which I use to identify teachers’ years of experience. Because the district does not maintain the same data for charter-school teachers, those teaching at charters are excluded from this analysis. I further restrict the sample to teachers working in grades and subjects participating in state tests: grades 4 through 8 in math and English Language Arts. In all, my analysis includes 308 TFA teachers and approximately 40,000 non-TFA teachers.

I first look at retention rates. Overall, first-year TFA teachers are more likely than non-TFA teachers working in similar schools to keep teaching a second year, but less likely to remain in the classroom by year five (see Figure 1). About 88 percent of first-year TFA teachers persist to a second year compared to 82 percent of non-TFA teachers. However, non-TFA teachers are retained at higher rates in years three and beyond. Over the study period, 43 percent of non-TFA teachers kept teaching for five years or more, compared to 25 percent of TFA teachers.

Impacts on Student Performance

To estimate returns to experience for TFA and non-TFA teachers, I use a measure of value-added, or how much their students improve on annual tests in math and reading relative to expectations based on the students’ prior achievement and demographic characteristics. My calculations control for teacher and student characteristics and assume a “steady state” of the world, with no major changes in overall teacher supply or average teacher performance over time.

First, I look at teachers’ initial performance and find no average difference in first-year performance for TFA and non-TFA teachers. Then I look at teacher value-added by years of experience. Over the first five years of their careers, TFA teachers improve by 13 percent of a standard deviation compared to 6.3 percent for non-TFA teachers (see Figure 2). In particular, between their first and second year in the classroom, TFA teachers improve by 6.1 percent of a standard deviation, more than twice the rate of non-TFA teachers at 2.8 percent of a standard deviation. Non-TFA teachers catch up somewhat in year three, but TFA teachers continue to improve more quickly in the subsequent early-career years.

What do these results imply regarding the effects of TFA hiring on the teaching workforce over time? Because of their higher turnover rates, TFA teachers are about 1.5 times more likely to be in their first year of teaching than non-TFA teachers and only half as likely to have five years of experience or more. Nevertheless, I find that hiring continually hiring TFA teachers to replace exiting corps members, as compared to hiring non-TFA teachers, would increase student achievement by an estimated 5 percent of a standard deviation. This estimate assumes a constant turnover rate after year five and, consistent with my findings, assumes no average difference in performance for TFA and non-TFA teachers in year one.

This analysis may well understate the advantages for schools of relying on TFA. Given the staffing challenges common at schools that hire from the program, it could be the case that the best alternative to a TFA teacher is not the average first-year traditionally trained teacher, but rather a long-term substitute or a college graduate with an emergency certification who may or may not have any training in education or in the subject they teach. In that case, TFA teachers could perform better in year one than these likely alternatives and the estimates reported here would understate the true benefits of TFA hiring for student achievement.

Changing Demographics

While prior research has shown that TFA teachers perform as well or better than their most-likely alternative during their first two years in the classroom, my analysis additionally suggests that TFA teachers who continue teaching beyond year two continue to outperform their non-TFA counterparts after five years in the classroom.

This is a somewhat different result than that of a 2007 study of TFA teachers in New York City, which did not find meaningful differences in performance between teachers based on their certification status or preparation path. It also showed TFA teachers underperforming traditionally certified teachers in their first year before quickly catching up (see “Photo Finish,” research, Winter 2007). I find no difference between TFA and non-TFA teachers in year one, and faster performance gains for TFA teachers in years two through five. What could account for this difference?

One plausible explanation could be changes in TFA’s recruiting, workforce, and organizational strategy from its earliest days. These evolved dramatically between the two study periods to include an explicit focus on corps-member diversity. The TFA teacher workforce of the late 1990s and early 2000s skewed young, wealthy, and white; in New York City, TFA teachers were 82 percent white, and the median age was 23. By contrast, as of 2014, one-third of TFA teachers identified as first-generation college graduates, one-half as teachers of color, and 43 percent as growing up with a low household income. Research points to this increase in representation and instruction by teachers with backgrounds similar to their students as a powerful learning accelerator for students of color (see “What We Know About Teacher Race and Student Outcomes,” feature, Winter 2024).

TFA and other alternative-certification programs have grown rapidly over the past three decades. There are now more than 200 alternative providers, which prepare about one in five newly hired U.S. public-school teachers. While these teachers have higher rates of turnover after they are hired, they also are dramatically more racially diverse than teachers trained in traditional college-based programs. A review by the Center for American Progress found that in 2018–19, aspiring Black and Latino teachers accounted for 37 percent of all graduates of alternative-route programs based outside universities, compared to 17 percent at traditional teacher-preparation degree programs.

TFA, though a smaller provider, has influenced the national conversation about who should teach American students and how best to prepare those recruits. After several years of shrinking its workforce, the organization grew its corps members by 40 percent this year and is planning to grow by another 25 percent next year, with “an exceptional and diverse cohort” of 2,000 new teachers nationwide. A ripe area for future research is to examine whether improvements in the performance of the TFA workforce since the early 2000s continue and if they are a direct result of the organization’s efforts to recruit teachers who more closely resemble the students they serve.

Virginia S. Lovison is an Associate Director of Research & Policy at Deloitte Access Economics and a former postdoctoral research associate at the Annenberg Institute for School Reform at Brown University. This analysis is the author’s own, independent of Deloitte Access Economics.

This article appeared in the Spring 2024 issue of Education Next. Suggested citation format:

Lovison, V.S. (2024). Teach For America’s Fast Path to the Classroom Accelerates Performance Over the Longer Haul. Education Next, 24(2), 46-51.

Rankings and Results

The strongest academic performance from charter-school students is in No. 1-ranked Alaska, at 32 percent of a standard deviation above the average charter score nationwide, followed by Colorado and Massachusetts, then by New Hampshire, New York, Oklahoma, and New Jersey (see Figure 1). The lowest-ranked charter performance is in Hawaii, at 54 percent of a standard deviation below the national average, followed by Tennessee, Michigan, Oregon, and Pennsylvania.

Alaska’s high ranking for charter-school student achievement may seem surprising given its low ranking for NAEP performance by all public-school students. In a 2019 analysis by the Urban Institute, Alaska ranked at or near the bottom in both reading and math in grades 4 and 8. It is possible that results are skewed in some way by the challenge of controlling for Alaska’s distinctive indigenous population, which makes up about 20 percent of K–12 students. However, Stanford economist Caroline Hoxby found Alaska among the top three states in an analysis conducted on scores in 2003. Further, Alaska’s charter achievement ranks seventh when no adjustments are made for background characteristics. Charter student performance in Alaska seems to deserve its ranking in the top tier.

In looking at the five lowest-ranking states, Hawaii’s very poor performance is skewed downward by NAEP’s incorporation of indigenous Hawaiian population and other Pacific Islanders into the broad “Asian” category, a sizeable share of the charter student population (see “Does Hawaii Make the Case for Religious Charters?,” features, Winter 2024). If the analysis is limited to the years 2011 to 2019, indigenous Hawaiians and Pacific Islanders can be classified separately. When this is done for those years, Hawaii’s performance shifts to –35 percent of a standard deviation, and the state’s score resembles that of Tennessee.

We then estimate differences in test-score performance between students of various racial and ethnic groups in each state, while still adjusting for other background characteristics. States vary in the degree to which the performance of white charter students exceeds that of Black and Hispanic ones (see Figures 2 and 3). The gap between Black and white charter-school students’ test scores is more than a full standard deviation, or roughly equivalent to three-and-one-half years of learning, in D.C. and five states: Missouri, Wisconsin, Delaware, Michigan, and Maryland. By comparison, that gap is equivalent to about two-and-one-half years of learning in Oklahoma, Arizona, New York, Florida, and Illinois.

We find the largest score differences between white and Hispanic students in D.C., Pennsylvania, Delaware, Georgia, Idaho, and Massachusetts. States with the least divergence in white-Hispanic scores are Oklahoma, Louisiana, Illinois, Florida, and Ohio, where scores differ by roughly one to one-and-a-third years of learning.

Oklahoma and Florida have among the smallest disparities between white charter students and both Black and Hispanic charter students. By contrast, D.C. and Delaware have exceptionally large differences between those student groups. These differences may be a function of which students opt to enroll in charter schools or some other mechanism not captured by observed student characteristics. Or they may reflect divergent charter practices.

Comparison to Statewide Rankings

How closely do the PEPG state rankings mirror similar efforts to rank states based on student achievement across all public schools? We might expect strong correlations, as charter student performance could be affected by a state’s educational climate, including family and community support for schools and students as well as the talents and training of its teachers.

To explore this possibility, we calculate the relationship between PEPG rankings for charter students with state rankings made by the Urban Institute for student achievement at all public schools. Importantly, the comparison is for performance on the same tests for the same period, and the adjustments for family background characteristics are virtually identical.

The rankings for charters and for all public-school students are only modestly correlated (see Figure 4). Massachusetts, New Jersey, Colorado, and Florida have similarly high rankings on both. At the other end of the distribution, California sits at the 24th position in both standings. But the rankings for other states differ sharply. Texas, Pennsylvania, and Indiana are ranked 2, 10, and 12 on the Urban Institute list but land at 15, 31, and 20, respectively, in the PEPG ranking. Conversely, Oklahoma is ranked 6th and Utah is ranked 9th in the PEPG rankings, but these states rank 21st and 32nd, respectively, on the Urban Institute’s list. In short, charter-school performance is not simply a function of the educational environment of the state as a whole.

A Close Look at CREDO

Another state-level ranking of charter schools warrants detailed discussion. In a June 2023 report, the Center for Research on Education Outcomes (CREDO) at Stanford University ranked 29 states by the academic performance of their charter schools from 2014 to 2019. This ranking is based on state test results and compares charter students’ performance, adjusted for prior-year test scores and student background characteristics, to that of students at nearby district schools. This average difference approach to assessing charter performance diverges significantly from the PEPG yardstick, which ranks states by the average level of charter performance, adjusted for student background.

CREDO rankings would nonetheless resemble the ones reported by PEPG if average student achievement were identical at all district schools throughout a state and the country as a whole. Since that is not the case, CREDO rankings are affected as much by scores at district schools as by scores at charters. This is not a mere hypothetical possibility. CREDO finds that test scores for Black students at charter schools showed they “had 35 days more growth in a school year in reading and 29 days in math” relative to comparable students in nearby district schools, and Hispanic students “grew an extra 30 days in reading and 19 additional days in math.”

Meanwhile, white charter students do no better in reading than white students at district schools, and they perform worse in math by 24 days of learning. CREDO also finds better outcomes for charter schools in cities than suburbs—test scores for students at urban charters showed 29 additional days of growth per year in reading and 28 additional days in math. Suburban charters did not perform significantly better than district schools in math but had “stronger growth in reading” amounting to 14 additional days of learning.

These findings could indicate that Black, Hispanic, and urban students attend higher-quality charter schools than those available to white and suburban students. But an alternative interpretation is more likely: White and suburban students have access to higher-quality district schools than those available to Blacks, Hispanics, and city residents. CREDO’s state ranking is useful in considering how the presence of charters affects the choices available to students in each state, but it does not order states by the performance levels of charter students, as the PEPG rankings do.

Impacts of Innovations

The specifics of each state’s charter law and regulations differ substantially, helping the charter sector live up to the “laboratory” principle. This sets the stage for a variety of comparisons looking at which aspects of charter school governance might contribute to student success.

For example, the type of agency granted the power to authorize charters ranges from the state board of education to local school districts to mayoral offices. Accountability requirements vary from tight, ongoing monitoring to nearly none. The saturation of the charter sector is similarly diverse—in states like Arizona, California, and Florida, 12 percent or more students attend a charter compared to 3 percent or less in Maryland, Mississippi, and New Hampshire. Charter funding differs as well, both among and within states, based on revenues and regulations set by federal, state, and local agencies and authorizers. In 2019, charter-school revenues per pupil ranged from $27,825 in D.C. to $6,890 in Oklahoma.

On some widely debated topics, we find little support for either side of the dialogue. For example, we find no higher levels of achievement in states with a larger percentage of public-school students attending charters. Nor do we find a correlation between charter student achievement and the age of the charter school, whether a state permits collective bargaining, or the level of per-pupil funding charter schools receive within a state.

We do find differences when looking at some of the innovative features of charter schools, including authorizing agencies, management structures, and whether schools have an academic or programmatic specialization.

For example, charter student performance varies with the type of authorizer that granted its charter. Students whose charter schools are authorized by a state education agency earn higher scores on NAEP than students whose schools were authorized by school districts and comparable local agencies. Compared to charter schools authorized by a state education agency, student achievement is 9 percent of a standard deviation lower at charter schools authorized by local education agencies like school districts, 10 percent lower at charter schools authorized by independent statewide agencies, 15 percent lower at schools authorized by non-education entities like a mayor’s office, and 19 percent lower at charter schools authorized by higher education institutions.

These results should not be interpreted as showing a causal connection between type of authorizer and student outcomes. Still, it might be noted that state education agencies have decades of experience at overseeing educational systems, an advantage not matched by any other type of authorizer. Local school districts do not authorize as effective charters as do state offices, but they outperform agencies that have had no prior experience in the field of education. Perhaps Helen Keller was right when she said, “Only through experience of trial and suffering can the soul be strengthened . . . and success achieved.”

We also find notable differences in student achievement between schools based on their management model. These fall into three categories: freestanding or standalone schools; schools run by nonprofit charter management organizations or networks like KIPP Foundation and BASIS Charter Schools; and schools run by for-profit education management organizations, such as Academia and ACCEL Schools.

Some 55 percent of the students in our sample attend freestanding, independent charter schools—the classic charter type, led by a small team, that is one of the thousand flowers expected to bloom. Another 23 percent of students attend charters that are part of nonprofit networks or management organizations, and 22 percent of the sample are at schools run by for-profit entities.

Compared to students at for-profit and freestanding, independent charters, students at charters that are part of a nonprofit network score 11 to 16 percent of a standard deviation higher on NAEP. This may be because networked charters benefit from an association with a larger entity, or perhaps because successful charters expand beyond a single school.

For-profit schools are arguably the most controversial component of the charter sector. Charter critic Diane Ravitch has argued that “our schools will not improve if we expect them to act like private, profit-seeking enterprises,” and in 2020, the Democratic Party platform proposed a ban on charter schools run by for-profit entities (see “Ban For-Profit Charters? Campaign issue collides with Covid-era classroom reality”, feature, Winter 2021).

Why do students at for-profit schools earn relatively lower scores on NAEP than at networked charters? For-profit organizations may launch charters where circumstances are more problematic, or they may find operations more challenging when faced with heavy political criticism and threats of closure and government regulation. Or possibly the profit motive is indeed inconsistent with higher student performance, as critics have alleged.

Our main purpose in ranking states by the performance of their charter students is to focus public and policymaker attention on the provision of high-quality schools, the purpose of charter legislation from its very beginning. Our second purpose is to supplement current state-level rankings of the charter-school environment and focus attention on outcomes, not simply state policies and procedures. Although previous rankings document the variety of environments in which charter schools operate, they do not report student achievement measured by a national test common to public schools across the country.

However, the PEPG rankings are not the last word on charter-school quality. We are not able to track year-by-year trends in charter quality within states, as the number of charter student test scores for any given year are too few for precise estimation. We have no information on student performance at virtual charters, as NAEP only monitors student performance at brick-and-mortar school sites. Also, these rankings are based on assessments of student performances in 4th and 8th grade, which excludes any insights as to charter contributions to early childhood and preschool education or high school or career and technical training programs. Finally, NAEP data are observational, not experimental, so causal inferences are not warranted.

It should also be kept in mind that these data are based upon an 11-year period ending in 2019, the eve of a pandemic that closed many charter and district schools for more than a year. Student performance was dramatically affected by the event, and charter enrollment appears to have increased substantially since then. The data reported here stand as a baseline against which future measurement of charter performance in the aftermath of that event may be compared—an especially important measure given the continued growth of the sector.

Paul E. Peterson is a professor of government at Harvard University, director of its Program on Education Policy and Governance, and senior editor at Education Next. M. Danish Shakeel is professor and the director of the E. G. West Centre for Education Policy at the University of Buckingham, U.K. An unabridged version of this paper has been published by the Journal of School Choice (2023).

This article appeared in the Winter 2024 issue of Education Next. Suggested citation format:

Peterson, P.E., and Shakeel, M.D. (2024). The Nation’s Charter Report Card: First-ever state ranking of charter student performance on the National Assessment of Educational Progress. Education Next, 24(1), 24-33.

For more than a century, children in the United States have been enrolled in public schools based on where they live, and pressure to improve public education has been mainly channeled through school board elections, inter-district housing decisions, and test-based accountability. Over the past 30 years, however, charter schools, vouchers, and public-school choice programs have challenged this model. Rather than voting at the ballot box, market-based accountability allows families to vote with their feet and select the schools they prefer without moving households. In theory, this alternative also increases competition that promotes educational improvement systemwide.

How choice and competition affect the market for education depends on the characteristics of schools that families prefer. To the extent that families value school effectiveness with respect to academics, there is the potential for schooling choices and competition to lead to improved school quality along this dimension and better learning outcomes. However, if families prefer characteristics that are unrelated (or negatively related) to academic effectiveness, there is a possibility of reduced academic learning. Understanding family preferences is thus crucial in understanding the potential consequences of school-choice policies.

We study family preferences in one of the most competitive school markets ever developed in the United States: New Orleans, where virtually all district students attend a charter school. The vast majority provide transportation from anywhere in the city, and none can charge tuition. Admission is based on parental preferences expressed through a common application system. For many years, an advocacy group also published detailed school guides to inform families’ choices. Not only do parents have more freedom to choose, but they have a ready source of information and a wide variety of options to choose from.

What are the school characteristics that drive family choices, and how do family resources influence these decisions? New Orleans presents a unique opportunity to answer this, with its combination of ranked-ordered preferences within an extensive choice system with detailed data about school characteristics and common enrollment forms. We look at school characteristics such as academic outcomes and extracurricular activities, as well as practical considerations such as the school’s proximity, hours, and availability of after-school care. We report the influence of these factors in miles to illustrate the distance families would be willing to travel to enroll their child in a preferred school.

Our analysis finds that New Orleans families do indeed value academic performance, but they also value many other things at least as much. Improving a school’s performance score by one letter grade is equivalent to reducing its distance by 0.8 miles for elementary schools and 2.1 miles for high-school families. An improvement of one standard deviation in a school’s measure of value-added is equivalent to reducing distance by two miles for elementary schools and 6.4 miles for high schools.

But other factors are at least as important. Families prefer schools with more extracurricular activities, and the availability of a football or band program is especially influential in choosing high schools. Practical considerations also figure prominently. Families generally prefer schools that are close by, and we find some evidence that after-school care is important to elementary-school families. In looking at the preferences of low-income families, after-school care, distance, and extracurriculars seem especially important relative to academic factors, which has important implications for achievement gaps.

These findings confirm that New Orleans families of all income levels place substantial weight on academic quality when choosing schools, including measures of schools’ value-added to student achievement that are not available in published guides. Yet families also value a broader range of school characteristics. And low-income families face constraints on their ability to choose schools based on academic considerations alone.

A District of Choice

Two major factors sparked the growth of charter schools in New Orleans. First, in the 1990s and early 2000s, Louisiana state lawmakers passed a series of laws allowing charter schools and creating the state Recovery School District to turn around low-performing schools. Then, in 2005, Hurricane Katrina devastated the city of New Orleans, with severe flooding that claimed 1,800 lives, caused more than $160 billion in damage, and displaced 250,000 people.

The state quickly took over almost all public schools. These schools were gradually turned over to charter-school organizations, while attendance zones were abolished. During the period of our study, New Orleans Public Schools comprised about 75 charter elementary, middle, and high schools, which were authorized and operated by a diverse group of charter-management organizations, nonprofits, and state and local agencies. More recently, governing authority has shifted back to the locally elected board under an unusual arrangement that has largely left the reforms in place.

Any student living in Orleans Parish is eligible to attend any of these schools under the district’s all-charter, open-enrollment system. However, families are not guaranteed their first choice. Instead, since 2012, they have been required to fill out a common school-enrollment form and rank their chosen schools in order of preference.

In 2008, local advocates launched the New Orleans Parents’ Guide to Public Schools in order to provide detailed information about this new landscape, as part of a larger effort to organize local parents in the pursuit of excellent public schools. The guide was available online and in print in schools, libraries, post offices, and other public locations throughout the city. It described schools’ locations, offerings, hours, and characteristics based on what parents and community members expressed as most important to them. Over time, it grew to include more than 100 unique attributes for each school.

This detailed information forms the basis of our analysis, which aims to identify the academic and non-academic characteristics that families value when choosing schools. We narrowed the set of characteristics to consider based on discussions with local education stakeholders, survey evidence about New Orleans parent preferences from the Cowen Institute at Tulane University, and prior research about parental preferences. Our main characteristics of focus include school-performance scores, which are calculated by the Louisiana Department of Education based on student achievement and expressed as numbers or letter grades; estimates of each school’s value-added to student achievement, which we calculate using standard methods (and are not published in Parents’ Guide); the distance between each school and the family’s home address; the availability of football, band, and other extracurricular activities; and whether a school offers extended school days, weekend classes, and after-school care.

Our analysis also includes several other characteristics that local parents and advocates indicated would be important to New Orleans families. We use the Parents’ Guide to identify whether a school is “in flux,” meaning that it has recently moved locations or would be moving soon. This is relevant for understanding the role of distance, as well as a general desire for certainty and stability. Relatedly, we also include an indicator for a “legacy” school, which denotes whether a school’s name was in use prior to Katrina. Parents and grandparents may prefer to enroll a child in the school that they themselves attended so their child or grandchild could potentially play on the same sports team. While not part of the Parents’ Guide, we worked with local officials to identify schools that would meet this criterion. We also created a variable to capture the relative quality of a school building. This can vary considerably, due to building ages, storm damage, and whether a building was part of a major school construction and renovation initiative following Katrina.

We include this large number of factors because the characteristics of schools are correlated with each other. Therefore, if an important factor were excluded, it would distort our estimates of parent preferences for the characteristics we included. We did, however, have to exclude two potentially important factors: data on school safety, which were not available; and student demographics, because the city’s publicly funded schools have little variation on these measures. While the precise shares can vary from year to year, at least 80 percent of city students are Black, and a similar share are from low-income households.

Data and Method

We focus on the 2013–14 school year, which was the second year that the common school-enrollment form was in use. The form, then called OneApp, allowed families to rank up to eight schools in order of preference. Our sample includes roughly 31,000 students in 2013, which is about two-thirds of the district’s total enrollment of about 45,000. The OneApp process that year excluded the 19 schools run directly by the Orleans Parish School Board, including the city’s selective-admissions schools, which means that the average academic achievement of students in this analysis is below the city average.

We first look at the characteristics of the schools in our sample, which we separate into two groups: elementary schools, which serve some combination of grades K–8, and high schools, which serve the upper grades. Most of our school characteristics comes from the Parents’ Guide.

The average New Orleans elementary school offers about three different sports and six extracurricular programs and has a school performance score of 78.7, which is below the state average of 93.9. Nearly 70 percent of elementary schools have an extended school day, 24 percent offer free aftercare, and 20 percent offer paid aftercare. The average high school has a school performance score of 80 and offers six sports and seven extracurriculars. Nearly 90 percent of high schools offer band and football, and only one school offers one without the other (band, but not football). Two-thirds of high schools are “legacy” schools, with names the same as or similar to schools that existed before Katrina.

These characteristics show considerable variation in program offerings between schools, which is key to our study. If all the schools had the same offerings, then parent rankings would tell us little about what they prefer. Also critical is that we have data on parents’ rankings of schools. We therefore can combine these data and study the relationship between the rankings and each school characteristic, accounting for the other characteristics at the same time. This includes both the weight families place on academic and nonacademic factors and how practical considerations influence their choices.

We begin by reporting average preferences across all families choosing among elementary schools and among high schools, and then look for any differences in preferences based on family income. To quantify our findings, we take advantage of a consistent finding in research on school choice in New Orleans and elsewhere: all else being equal, families strongly prefer a school that is close to their home. We first measure the extent to which proximity to school influences the choices of families in our sample. We then express our findings for other school characteristics in terms of relative distance from home to school.

Results

New Orleans families have a clear preference for schools with stronger academic performance. For families choosing elementary schools, a one-letter-grade improvement in a school-performance score is equivalent to reducing distance to school by 0.8 miles (see Figure 1). Differences in school-performance scores among highly rated schools appear to matter more for family choices than similar differences among schools with low scores. That is, earning an A grade from the state rather than a B has more influence on how families rank a school than earning an D rather than a F.

Value-added to student achievement is also positively related to elementary-school rankings, even after controlling for the school-performance scores assigned by the state. For elementary schools, increasing school value-added by one standard deviation is equivalent to reducing distance by two miles. Apparently, families have access to and are influenced by information on schools’ academic quality beyond what is published in the Parents’ Guide.

The academic school characteristics also play a large role in shaping family preferences about high schools, with measures of academic performance again playing a leading role. A one-letter-grade improvement in school-performance score is equivalent to reducing distance to school by 2.1 miles, and improving value-added by one standard deviation is equivalent to a school being 6.4 miles closer to home (see Figure 2).

In addition to a general preference for schools that are close by, we find that families are more likely to assign a high ranking to the specific elementary or high school that is nearest to their home. This suggests that some families view the nearest school as the default choice, even when there is a viable option only slightly farther away.

After-school care is another practical consideration that is important to elementary school families. The availability of a free after-school program is equivalent to reducing distance by about 0.8 miles, and a paid program is equivalent to a 0.7-mile reduction. In addition, we find that extended school days and weekend sessions have slightly negative effects on rankings. The seeming conflict between these results may be because after-school care is specifically designed to help parents work; extended school days are not.

The role of extracurriculars has also received relatively little attention in prior research. Football and band, for example, are particularly popular in New Orleans, so it is not surprising that families prefer high schools with these programs. Having either band or football is equivalent to reducing distance by two miles. However, the total number of sports and other extracurricular programs have negligible effects, and the presence of other music programs in addition to band are associated with a school’s being 2.5 miles further away. Families seem to pay little attention to extracurricular programs outside of football and band in high school.

Families also appear to value high schools with a long tradition or “legacy” in the city, dating to the pre-Katrina years, which is equivalent to a reduction in distance of 4.8 miles. This could be because families want to continue traditions, sending children to the schools that parents or other family members attended. Alternatively, this could reflect established reputations; although the schools now have new operators in the post-Katrina period, families may perceive that having the same name means that it has programs and qualities similar to prior years. The fact that legacy status seems especially important in high school might be because adults in New Orleans tend to identify themselves by the high school they attended.

Fewer families choose elementary schools that are “in flux,” although the role of this factor, equivalent to 0.2 additional travel miles, seems small in comparison to other school characteristics. While attending school in a newer building would seem appealing, the estimates of the role of new and refurbished school buildings are erratic for both elementary and high schools, perhaps because many “in flux” schools also are in new buildings.

Differences by Family Income

Prior research has suggested that low-income families often place relatively little emphasis on academic quality in their schooling choices, and several theories take a deficit perspective on the topic. Some studies have focused on a lack of information among low-income groups, while others even suggest that groups with lower test scores might prefer schools where other students’ academic performance is similarly low.

We explore an alternative explanation for why researchers may see lower-income families choosing schools of lower academic quality. Even among families with the same schooling preferences, there are reasons to expect lower-income families to place less emphasis on academics in their choices due to resource constraints. Any financial expenditures involved in schooling choices (e.g., childcare and transportation) yield proportionately greater losses in personal well-being for low-income families. Compounding this effect, some of the family resources that are necessary for education are also important for other household purposes. In particular, lower-income families are less likely to own automobiles that are used for many purposes, and the absence of a car increases the marginal cost to families of sending their children to schools further away.

To better understand how these income-related constraints play out in practice, we divide families into three groups based on the median income in their immediate neighborhood. Our data do not include families’ household incomes, so these are based on Census median block group incomes from the 2007–2011 American Community Survey. The simple average of these median incomes is $16,174 in the lowest group, $28,461 in the middle group, $48,337 in the highest.

In the case of elementary schools, we find that the lowest-income families express somewhat weaker demand than the highest-income families for both school performance scores and value-added. School characteristics related to income constraints also seem to affect their choices more: Low-income families rank schools with free after-school care, extended days, and weekend classes higher than higher-income families. The lowest-income families also have weaker preferences than higher-income groups for paid after-school care, presumably because they cannot afford to pay for it.

The patterns differ somewhat in high school. Compared to the highest-income families, families in the two lower-income groups actually place greater importance on school value-added. Estimates of the influence of school-performance scores are similar across all three groups. Football and band are more important to lower-income families, as is the availability of other sports programs, but at the high school level this preference does not lead them to place less emphasis on academic quality.

In short, our results for elementary schools generally align with those of prior studies that have found weaker preferences for academic quality among lower-income families. However, our analysis points to a different explanation—one that is related to income itself and the way in which schooling choices intersect with household budgets. This role for cost factors reinforces the importance of considering a wide range of school characteristics when studying family preferences.

A Question of Competition

Identifying how families view and rank school characteristics is a difficult task. We rarely have data on how families rank schools in real choice settings, and even when we do, we see little variation about the schools they are choosing among and little information about those options. New Orleans’s school-choice market and efforts to help parents make informed choices enable us to provide unique insights into what school characteristics drive parental choices and how family income influences family preferences. We show that, in addition to academic factors, practical considerations such as the availability of after-school care are also important to families—especially those from low-income neighborhoods. And while academic performance is important to families across grade levels and income groups, extracurriculars and especially football and band programs are highly valued overall and are particularly important considerations for the lowest-income families. This, too, could be related to cost, as wealthier families may be able to afford these experiences through other paid organizations if they are not offered by their school.

Our findings have important implications for school-choice policies, whose ultimate effects on educational quality will depend on what families value in schools. Even when schools do compete, it is not based solely on academics. When parents choose a school, they consider a wide range of characteristics as well as logistical factors related to their household budgets. To attract families—and particularly lower-income families—school leaders may have to reallocate resources away from academics to pay for after-school care and other nonacademic services, for example. This could help explain why studies of school-choice programs to date find only modest effects of competition on student test scores.

The share of U.S. families with access to school-choice programs has expanded rapidly in recent years, with about 7.5 percent of students nationwide attending charter schools and nearly 311,000 using publicly funded vouchers to attend a private school. More than a dozen states introduced legislation to enact or expand school-choice programs in the last year. While the ultimate impact of these efforts on educational quality is not yet clear, our findings indicate that the context of family choices is complex and includes more than just academic quality. Policymakers seeking to harness the power of competition to drive improvements in academic achievement would do well to keep this complexity in mind.

Douglas N. Harris is professor of economics at Tulane University and Matthew F. Larsen is associate professor of economics at Lafayette College.

This article appeared in the Winter 2024 issue of Education Next. Suggested citation format:

Harris, D.N., and Larsen, M.F. (2024). Going-to-School Shopping: Investigating family preferences in New Orleans. Education Next, 24(1), 62-69.

During the latter half of the 20th Century, the early blooms of spring were also a signal to the nation’s teenagers: it’s time to find a job. About half of all Americans between 16 and 19 years old spent part of their summer break bagging groceries or slinging ice cream until the early 2000s. Then, the youth employment rate fell sharply and stayed low for the next two decades and through the Covid-19 pandemic. Teenage employment has since rebounded, with about one in three young people employed in July 2023.

Black and Hispanic teens are less likely to be employed than white students, both during the summer and the school year. They also are less likely to graduate high school, enroll in college, and earn a degree. The sort of community-based learning that teenagers’ jobs can impart, such as gaining employable skills and learning to meet professional expectations for responsibility, punctuality, and collaboration, has attracted the interest of policymakers looking to improve outcomes for at-risk students.

How do early workplace experiences affect academic outcomes? We provide experimental evidence from the Boston Summer Youth Employment Program, which has matched high-school students from low-income neighborhoods with summer jobs since the early 1980s. For much of that time, students were enrolled in the program via random lottery to work in local city agencies, businesses, and nonprofits, as seasonal workers in parks, day camps, and other local organizations. By matching academic records with teenagers who are and are not offered the chance to take part, we estimate the program’s causal impact on high school graduation rates, grades, and attendance.

We find broad benefits for students selected by the program lottery. Students who receive job offers are 7 percent more likely to graduate high school on time and 22 percent less likely to drop out within a year of the program. We also find that students’ school attendance and grade-point averages improve, as do their work habits, soft skills, and aspirations to attend college. In looking at the program’s costs, the evidence suggests that its long-term benefits outweigh its costs by more than 2 to 1.

A Summer Jobs Lottery in Boston

The Boston Summer Youth Employment Program began in the 1980s and now connects about 10,000 young people with jobs at roughly 900 local employers each summer. It is part of the city’s workforce development efforts and is intended to connect young people with meaningful job opportunities that offer professional experience, resume fodder, and a paycheck.

The six-week program is available to all Boston city residents aged 14 to 24 who apply through local nonprofits or other intermediaries. Participants are paid the Massachusetts minimum wage (currently $15 per hour) and work up to 25 hours per week in either a subsidized position (e.g., with a local community-based organization or city agency) or a job with a private-sector employer. The program also offers 20 hours of job-readiness training, which includes an evaluation of learning strengths and interests; practical instruction in resume preparation, job-searching, and interviewing; and opportunities to develop soft skills like time management, effective communication, persistence, and conflict resolution. In 2015, the program cost about $2,000 per participant—including $600 in administrative expenses and $1,400 in wages earned—or approximately $10 million total from municipal, state, and private funding.

Our study focuses on Action for Boston Community Development, a large and established nonprofit that works in all of Boston’s 18 neighborhoods and serves a predominately young, school-aged, and low-income population. Prior to the pandemic, the organization used a computerized lottery system to select applicants to participate in the summer jobs program based on ID numbers and the number of available slots, which is determined by the amount of funding each year. This system effectively assigned the offer to participate at random.

We focus on the summer of 2015, when 4,235 young people applied. We match applicant names with data from the Massachusetts Department of Elementary and Secondary Education to review the demographic and academic characteristics of youth who were and were not offered a program spot and to track and compare their outcomes over time. About 80 percent of applicants, or 3,372, were in grades 8–11 when they applied.

The average applicant was between 15 and 16 years old. About 53 percent were Black, 32 percent were identified as mixed race or “other,” 9 percent were white, and 6 percent were Asian. About 54 percent were female. Seven percent identified as having limited English ability, 7 percent reported being homeless, and upwards of 18 percent reported receiving cash public assistance in some form. Nearly 10 percent had switched schools during the academic year, and 15 percent attended a charter school. Applicants’ mean grade-point averages were 1.9, and nearly 30 percent were chronically absent from school. More than a quarter of applicants had failed a class.

Job offers were randomly granted to 28 percent of applicants, or 1,186 young people. The other 3,049 applicants did not receive an offer. We look at the demographics and academic performance of youth in these two groups and find no substantial differences—not surprising given that offers were awarded by a lottery. However, workforce participation rates were starkly different in the months that followed: 83.6 percent of lottery winners accepted the summer job offer, while just 28.2 percent of applicants who were not offered a job through the lottery worked between July and September, data from the Massachusetts Division of Unemployment Assistance shows.

We compare school outcomes for students who were and were not offered a summer job during the four-year period after the summer of 2015. We focus on the full group of 1,186 students who were offered a job rather than the 990 teenagers who accepted the placement and participated to measure the impact of receiving an offer. In many cases, that is the policy-relevant estimate, because while program administrators can offer an intervention, they cannot control who agrees to take part.

We theorize that the Boston summer-jobs program could have both direct and indirect effects on graduation. The program could directly increase career and academic aspirations that motivate students to graduate on time. It also could have two potential indirect effects that positively influence graduation. First, it is designed to develop good work habits like showing up on time, which could help students improve their school attendance and the likelihood of high-school graduation. Second, it provides youth with an opportunity to practice existing skills on the job and develop new ones, which may lead to better course performance and, ultimately, increase the probability of graduating.

Therefore, our primary outcomes of interest during the four-year post-intervention period are high-school graduation and dropout rates. We also examine more proximate outcomes that serve as potential mediators for longer-term effects: school attendance, course performance, and standardized test scores. Because offers are distributed by a random lottery, we obtain causal estimates simply by comparing the average outcomes of lottery winners and losers. Finally, we look at exploratory mechanisms from our survey data, which describes changes in students’ aspirations, work habits, and soft skills. We also look at effects by subgroups of students.

Impacts on Graduation, Attendance, and Academic Performance

Students who win the lottery and are offered a summer job are more likely to graduate high school on time and less likely to drop out compared to students who are not offered a job. Some 67.8 percent of students offered a summer job graduate high school on time compared to 63.4 percent of students who don’t receive an offer, a difference of 4.4 percentage points, or 7 percent (see Figure 1). In addition, dropout rates are higher among students who are not offered a summer job compared to those who are: 12.7 percent drop out within four years compared to 10.1 percent of lottery winners. Most of that difference occurs within the first year of participating in the program, when the dropout rate is 10.7 percent for students without job offers compared to 8.8 percent for students offered a summer job—a difference of 1.9 percentage points, or 22 percent.

We next examine outcomes that could help to explain the program’s impact on high-school graduation. In looking at attendance in the year after the program lottery, we see that students with job offers attended 3.4 additional school days compared to students who were not offered a summer job. This difference is due mainly to their having fewer unexcused absences during the next school year. Students offered a summer job are truant 2.1 fewer days compared to students not offered a summer job, suggesting a behavioral shift.

In fact, the overall difference in absenteeism is driven largely by lottery winners maintaining their attendance rates from the previous school year while attendance for non-winners falls. Since school attendance rates typically decline as youth age, this suggests that the summer jobs program could contribute to higher graduation rates by preventing chronic absenteeism. Indeed, we look at the relationship between these outcomes and find better attendance is positively correlated with a greater likelihood of graduating from high school.

In terms of academic achievement, we find a small positive impact on overall grade-point averages for lottery winners in the first year but no impact on course failures. Grade-point averages are 6.8 percent higher for students offered jobs than for students not offered jobs in the first year after the program. While the difference is relatively modest, with a grade-point average of 1.94 for lottery winners compared to 1.75 for non-winners, further analysis indicates that this small increase in course performance contributes significantly to boosting on-time high-school graduation. However, we find that the program’s effect on grade-point average disappears by the second year.

We also look at impacts across different groups of students and find outsized impacts on school attendance and academic performance. The positive impact from a job offer on school attendance is three times as great for males, applicants of legal dropout age, and students who were chronically absent before applying to the lottery. For students of legal dropout age, the program’s boost in grade-point average is also three times as large as that for younger youth. The program also appears to increase the likelihood of high-school graduation more for students with limited English proficiency and low socioeconomic status. However, the results for those students are less precise as the subgroups are relatively small.

Shifts in Attitudes and Aspirations

What might be driving the reduction in chronic absenteeism and subsequent increase in on-time high-school graduation rates? To learn more about students’ experiences and behaviors, we worked with Boston city officials and the Action for Boston Community Development to administer a survey that included questions related to job readiness, post-secondary aspirations, work habits, and socio-emotional learning. This survey was completed by 1,327 participants, split equally between students who participated in the program and students who were not offered a summer job. While response rates differed between these groups, given a lack of data and evidence on potential reasons why a summer jobs program boosts important school-based outcomes months and years later, we feel that there are still some key insights to be gained. While the first part of our analysis establishes causal impacts, the goal here is to provide a glimpse into how the program achieves those outcomes.

After working a summer job, students experience significant improvements across a variety of short-term behaviors and skills that could plausibly contribute to the improvements in school outcomes our causal estimates show. For example, 67.7 percent of students who participated in the jobs program report having gained a mentor over the summer compared to 52.4 percent of students who were not offered a spot. They also are significantly more likely to report having developed good work habits, such as being on time and keeping a schedule, as well as essential soft skills, such as managing emotions and asking for help. Notably, 11.4 percent of program participants report that they are saving for college tuition compared to 7.1 percent of applicants who were not offered a spot—an indication that the participants are not only exposed to experiences that might boost academic aspirations but are also motivated to act on those ambitions.

An Annual Opportunity

To our knowledge, this is the first study to document an improvement in high-school dropout and graduation rates associated with a summer jobs program. Young people who were randomly selected to receive a job offer are 7 percent more likely to graduate high school on time compared to students who do not receive an offer—an impact that is similar in size to the gap in on-time graduation rates between economically disadvantaged students and their wealthier peers in the Boston Public Schools. Within the first year of the program, students with job offers are 22 percent less likely to drop out of school than students who were not offered a job. These effect sizes are meaningful in terms of closing achievement and attainment gaps. They also are on par with low-cost educational interventions, such as reminding parents about the importance of attending school.

When assessing the value of any program, benefits should be considered relative to their costs. By some estimates, each new high-school graduate confers a net benefit to taxpayers of roughly $127,000 over the graduate’s lifetime. In 2015, the Boston Summer Youth Employment Program cost roughly $2,000 per participant, resulting in a total cost of about $2.4 million for the 1,200 youth who were offered jobs that summer through the nonprofit we study. We find that the program increases the likelihood of high-school graduation by 4 percentage points, which would yield an additional 48 graduates. Over their lifetimes, these graduates would collectively confer a benefit of $6 million—for a benefit-to-cost ratio of more than 2 to 1.

While these positive impacts are notable, they are likely not the only benefits. Students who participate in a supervised and development-oriented summer jobs program gain new experiences and professional connections that may yield additional advantages in terms of future employment, career pathways, or postsecondary education. Insights from survey data show students seem to benefit from mentorship and developing work habits and soft skills that promote success in a variety of settings, including high school. Finally, summer jobs programs also can help families at or near the poverty line by providing income to young people. Our survey found that half of participants use their earnings to help pay one or more household bills, and one in five report saving for college tuition.

While most students and families often look forward to summer vacations, seasonal jobs programs present a clear opportunity to benefit young people and their families, particularly those from low-income neighborhoods with few job opportunities nearby. Supervised work experiences improve high-school graduation rates and boost students’ employability, work habits, and family finances. With clear and positive benefits that last beyond the summer, seasonal youth jobs programs have an important role to play in the landscape of extracurricular activities.

Alicia Sasser Modestino is associate professor of economics and the research director for the Dukakis Center for Urban and Regional Policy at Northeastern University. Richard Paulsen is assistant professor of economics at Bloomsburg University.

This article appeared in the Fall 2023 issue of Education Next. Suggested citation format:

Modestino, A.S., and Paulsen, R. (2023). Year-Round Benefits from Summer Jobs: How work programs impact student outcomes. Education Next, 23(4), 60-65.

Public K–12 education in the United States is distinctively a local affair: school districts are governed by local boards of education, composed of lay members typically elected in non-partisan elections. These boards have decision-making power over hundreds of billions of public dollars and oversee complex agencies that, in addition to preparing a community’s children for the future, can be the biggest employer in town. Yet we know very little about what factors influence a board’s governance and impact, including the professional backgrounds of elected members.

One profession would seem to have particularly relevant effects: educators. Organizations like the National Education Association and Leadership for Educational Equity, the political arm of Teach for America, are training and supporting their educator members and alumni to run for elected offices. What might be the impacts of such efforts on school board elections, district governance, and student outcomes?

Research focused on boards of directors, which play a similar role in the corporate world, has found that adding members with more industry expertise increases a firm’s value. It stands to reason that electing educators to school boards could have similarly beneficial effects. For example, former classroom teachers or school leaders with firsthand knowledge of common challenges could theoretically make better decisions about teachers’ working conditions and positively influence student performance.

On the other hand, 70 percent of U.S. teachers are members of teachers unions. This raises the possibility that educators serving on school boards could be influenced not only by expertise but also allegiance to union priorities. That could theoretically influence collective bargaining, which is one of the major responsibilities of a school board. Union allegiance could shift bargaining agreements toward union goals, such as increasing teacher salaries or limiting charter-school growth, which may not necessarily benefit students.

We investigate these possibilities in California. State election rules randomize the order of candidates’ names on the ballot, which allows us to estimate the causal effects of an educator serving on a school board. By looking at randomized ballot order, candidate filings, election records, and school district data, we provide the first evidence on how the composition of local school boards affects district resource allocation and student performance.

Our analysis finds no impact on student achievement from an educator serving on a school board; neither average test scores nor high-school graduation rates improve. However, outcomes relevant to union priorities advance. Relative to a district without an educator on the school board, charter-school enrollment declines and the number of charter schools shrinks by about one school on average during an elected educator’s four-year board term.

In addition, each educator elected to a board leads to an increase of approximately 2 percent in teacher pay, while non-instructional salaries remain flat. Benefits spending is stable, while the share of district spending on ancillary services and capital outlays shrinks. We also find that educators are 40 percent more likely than non-educators to report being endorsed by teachers unions.

Despite raising teachers’ salaries, electing an educator to a school board does not translate into improved outcomes for students and has negative impacts on charter schools. We believe this shows that school boards are an important causal channel through which teachers unions can exert influence.

Electing Educators in California

Nationwide, nearly 90,000 members serve on about 14,000 local school boards. These boards have several general responsibilities, which include strategic planning for the district, curricular decisions, community engagement, budgeting, hiring senior administrators, and implementing federal and state programs and court orders. In addition, in nearly all states, school boards determine contracts for instructional staff through collective-bargaining agreements with teachers unions. These negotiations set salary schedules, benefits, work hours, and school calendars. Local school boards also set attendance zone boundaries and, in about three dozen states, authorize and monitor charter schools. In 2020–21, local education agencies accounted for 90 percent of all charter-school authorizers in the U.S. and enrolled 48 percent of the nation’s charter-school students.

While typical in most respects, school district governance in California has several unique characteristics. First, teachers unions are especially influential: 90 percent of California teachers are full voting union members. Second, school boards effectively do not have the power to tax. Under Proposition 13, property-tax collections are capped at 1 percent of assessed value, and assessments are adjusted only when a property is sold. Finally, charter authorization is overwhelmingly a local issue, with about 87 percent of California charters authorized by local school districts. Los Angeles Unified School District is the single biggest local authorizer in the U.S. and enrolls 4 percent of all charter-school students nationwide.

Our analysis is based on records from the California Elections Data Archive for all contested school board elections from 1996 to 2005. The data include each candidate’s vote share, ballot position, electoral outcome, and occupational background. We identify as educators candidates who describe their primary occupation or profession as a teacher, educator, principal, superintendent, or school administrator. Educators account for 16 percent of all 14,150 candidates in contested races and 19 percent of all 7,268 winners during this period.

Almost all school-board members serve four-year terms with staggered contests occurring every two years. The average tenure is seven years, and the average school board has five members. We use candidate-level records to construct yearly measures of school-board composition in each district, including the share of members who are educators. On the average school board, educators account for 18 percent of members. We link school-board rosters with district-level characteristics and charter-school campus and enrollment counts from the federal Common Core of Data, as well as negotiated salary schedules and district finance information from the state Department of Education. To look at impacts on student outcomes, we include average test scores in elementary and middle schools along with high-school graduation rates, also from the state education department.

Investigating Educator Impacts

To estimate the causal effects of an educator being elected to a school board, we need to compare two sets of circumstances: what happens after an elected educator joins the board and what would have happened if the educator had not won. While the effects could appear immediately and persist over time, it is also possible that they only become apparent in the longer run. Our approach therefore must examine the profile of effects over time.

The key challenge we face in making these comparisons is that the school districts that elect educators likely differ from those that do not—and these other differences could be responsible for any policy outcomes that change after an educator’s election. To overcome this challenge, we take advantage of the fact that, under California law, the order in which candidates for elected office appear on the ballot is randomly determined. Our data confirm that candidates who have the good fortune of being listed first on the ballot gain an advantage of 10.3 percentage points of the votes cast in their election. When an educator is listed first, this advantage translates into a 2.3 percentage point increase in the share of the board’s members who are educators. In short, the random assignment of an educator to the top of a ballot will shift a board’s composition.

Armed with this insight, we compare the policy choices of districts where educators are and are not listed first to isolate the causal effects of adding an educator to a school board on student outcomes, district spending, and charter schools. We first look at elementary- and middle-school scores on reading and math tests, as well as high-school graduation rates, and find no impacts.

We then consider teachers’ working conditions and find limited evidence of effects on service days, benefits, or class size. However, when an educator is elected to a school board, teachers’ salaries increase by 2 percent more than they would have otherwise four years after election. These increases apply across the board, for teachers at all levels of education and experience.

Because California school boards cannot raise the tax rate, boards decrease spending on building repairs and services like professional development in order to pay teachers more (see Figure 1). Four years after an educator is elected, a school board has increased the share of spending on certified salaries by 1.3 percentage points and decreased spending on capital outlays and services by 0.6 and 0.7 percentage points, respectively. We do not find evidence for impacts on superintendents’ salaries.

In looking at effects on charter schools, the share of district students enrolled in charters declines by three percentage points (see Figure 2). By the end of an elected educator’s four-year term, there are 1.3 fewer charter schools in the district. In a state with an active charter sector serving at least one out of every 10 public-school students, these are sizeable impacts.

What if a school board includes multiple educators? That could shift the identity of the median board “voter” for a given issue and influence board decisions through deliberations and agenda-setting. To examine these possibilities, we estimate the effects of electing an educator to a school board if it already has a sitting member who is an educator. Our results suggest that this is of limited importance. There are slightly larger negative effects on charter school enrollment, but these are not statistically significant.

We also investigate whether electing an educator to a school board has consequences for subsequent elections and find evidence that it does. In this analysis, we look again at the effect of ballot order. An educator being listed first increases the number of elected educators in that election by 13 percent but decreases the number of elected educators by 9 percent in the next election. Interestingly, educators are no less likely to run in these subsequent elections; those who do run are just less likely to win. The long-term causal effects of electing an additional educator would be even larger in the absence of this electoral dynamic.

The Influence of Teachers Unions

Our findings suggest that educators’ professional expertise on boards does not translate into improvements in student learning. The results are consistent with a rent-seeking framework, in which representation of union interests predicts higher teachers’ salaries and potentially negative effects on student performance. Our own data reveal that educators are 40 percent more likely than non-educators to be endorsed by a teachers union. School board member survey data also indicate a strong positive association between professional experience in education and alignment with union priorities.

We conclude that school boards may be an important causal mechanism for the influence of teachers unions on local education, which points to several avenues for future research. Our ballot-order-based strategy provides a new approach to inferring how the characteristics of candidates causally affect outcomes. A valuable next step would be to analyze candidate-level records of union endorsement. This would facilitate separating out the influence of educators on education production from their possible alignment with teachers unions. Likewise, shifting from aggregate school-level to administrative student records would enable disentangling impacts on student sorting from their effects on education quality. Future work should also focus on broader dimensions of students’ skills and behavior, such as social-emotional attributes and civic engagement.

In summary, the election of an educator to a local school board shifts spending priorities on K–12 public schools, which collectively cost about $800 billion in federal, state, and local tax dollars a year. Yet voter turnout in school-board elections is typically between 5 and 10 percent. While more research is needed, voters don’t need to wait. Our results show just how much these races matter.

Ying Shi is assistant professor at Syracuse University and John G. Singleton is assistant professor at the University of Rochester.

This article appeared in the Summer 2023 issue of Education Next. Suggested citation format:

Shi, Y., and Singleton, J.D. (2023). Putting Teachers on the Ballot: Raises for teachers, fewer charters when educators join the school board. Education Next, 23(3), 56-60.

From their earliest years, children use art for learning and self-expression. Preschoolers draw, paint, and build to understand and depict their surroundings. They learn their letters by singing the alphabet song. And they immerse themselves in stories to learn about their natural and social worlds, from books read by caregivers and during dress-up and imaginative play.

Yet the arts maintain a precarious position in K–12 public education. After a steady increase throughout the middle of the 20th century, arts education has been in decline since the 1980s. In a 2012 national survey, roughly half of public-school teachers reported declines in instructional time and resources for art and music over the previous decade, while only about one in 10 reported similar declines for reading or math. Teachers attributed the declines to test-score pressures, budget cuts, or both.

These trends have been most pronounced for students of color, who are more likely than white students to attend under-resourced schools and about half as likely to experience any arts education, on average. In a survey by the National Endowment for the Arts, the percentage of Black adults reporting any arts education during childhood fell by nearly half in 2008 compared to 1982, to 26 percent from 51 percent. Hispanics experienced similar declines to 28 percent from 47 percent, while the share of white adults who experienced arts education remained relatively flat, at around 58 percent.

How are these changes affecting American students? To begin with, an education without the arts is insufficient and fails to provide what federal education law defines as a “well-rounded education.” The arts have intrinsic value as a foundational form of human expression, providing ways of learning and experiencing different perspectives on the human condition. Moreover, theory and emerging research suggest arts education may have positive effects on student behavior, school engagement, and social-emotional development, all of which contribute to success in school.

We investigate the causal effects of arts education by looking at the Arts Access Initiative in Houston, which brings teaching artists, performances, and workshops to under-resourced public elementary and middle schools from the city’s ballet, symphony, and fine-arts museum, among many others. Our analysis compares schools that were enrolled by a random lottery to schools that applied to participate but were not chosen, in the first large-scale randomized control trial of an arts education program in an authentic school setting.

We find that arts learning has positive effects on empathy, school engagement, student discipline, and writing achievement. Students’ emotional and cognitive empathy increase by 7.2 percent and 3.9 percent of a standard deviation, respectively. At schools with expanded arts education, students are 20.7 percent less likely to have a disciplinary infraction. School engagement increases by 8 percent of a standard deviation. Arts learning improves writing test scores by 13 percent of a standard deviation but does not have significant effects on reading, math, or science test scores. The positive effects are especially pronounced among English language learners, whose writing scores improve by 27 percent of a standard deviation. These results demonstrate that the arts positively affect meaningful educational outcomes and can inform strategies to restore and retain arts education in under-resourced schools.

Art for More Than Art’s Sake

The benefits of arts education are rich in theory and testimony, but little rigorous evidence supports most claims. In a recent report co-written by one of us (Brian Kisida), the American Academy of Arts and Sciences took stock of the many theories and claims surrounding arts education and identified several areas of educational benefits that are supported by research. First, there is the primary claim that learning about the arts is good for its own sake, both because the arts are a fundamental mode of human expression and because familiarity with the arts helps students acquire cultural capital. In addition, there are intrinsic benefits to learning about and engaging with the arts. These include broadening students’ understanding of other cultures and history, supporting their social-emotional development and interpersonal skills, and providing opportunities for career exploration and creativity.

In terms of academic outcomes, there has been little causal research to date examining how arts education in school settings affects academic achievement. Some research has found that integrating arts experiences with instruction can boost student interest and content knowledge, such as by pairing a history unit with a live theater performance about the topic, particularly for English language learners and students with low test scores. Other studies focusing on arts education through field trips have found increases in students’ empathy and tolerance of others, as well as improvements in school measures like attendance and behavior—outcomes that contribute substantially to long-term success. For example, students’ attendance and disciplinary records are better predictors of their eventual on-time graduation and college enrollment than their grades (see “The Full Measure of a Teacher,” research, Winter 2019). And students who attend schools that improve social-emotional development have fewer absences and disciplinary infractions and are more likely to graduate and persist in a four-year college (see “Linking Social-Emotional Learning to Long-Term Success,” research, Winter 2021).

In many areas, school districts have formed broad-based coalitions with arts and community partners to restore and expand in-school arts education. According to the U.S. Department of Education, 42 percent of U.S. public schools partner or collaborate with cultural or community organizations, 31 percent with individual artists, 29 percent with museums, and 26 percent with performing arts centers. These arrangements take various forms, such as in-school teaching-artist residencies, workshops for students and teachers, professional artist performances, and after-school programs.

Our analysis focuses on one such program, the Houston Arts Access Initiative. The initiative was created by the Houston Independent School District, city government leaders, and local arts institutions and philanthropists with the goal of equitably advancing student access to the arts. It began in 2013 with a district-wide campus inventory of arts educational offerings, which found that 29 percent of K–8 schools had no full-time arts specialist, and 39 percent had either one or zero arts partnerships with community arts organizations. Meanwhile, 98 percent of surveyed principals and teachers agreed that “students benefit from access to the arts in school.”

The initiative focused on expanding arts education in schools with the fewest resources and raised funds to support expanded partnerships with local arts organizations. School participation was voluntary, but principals had to commit to spending between $1 and $10 per student on the program, with foundation support and in-kind donations from cultural institutions contributing a dollar-for-dollar match. During the first two years of the program, 60 eligible schools applied, and 42 were enrolled through a random lottery. More than 50 local arts organizations provided a diverse array of programs, including theater (54 percent), music (18 percent), visual arts (16 percent), and dance (12 percent). Nearly two-thirds of schools had either teaching artist residencies or on-campus performances during the school day, while about one in four schools went on field trips and one in 10 offered arts education after school.

The mission of the Arts Access Initiative was familiar to the participating organizations, virtually all of which already had well-articulated educational philosophies and had been providing educational services. These organizations also had designed their programs to be culturally representative and meet the needs of underserved students. Arts offerings included classical music and fine art, as well as African dance and drumming, Asian dance, Aztec dance, Brazilian music and dance, Chinese art, Mexican folklórico, hip-hop music and dance, and Hispanic literature.

In addition to touting their programs’ impact on students’ social-emotional development, many organizations also had made deliberate efforts to align their work to state educational standards or content from tested subjects. For example, Writers in the Schools described its workshops as aligned to state tests and core content, while the Mercury Chamber Orchestra offered workshops that integrated science with classical music “to introduce the science of Galileo, Sir Isaac Newton, and Einstein,” or civics by “hearing the favorite tunes of Ben Franklin… while learning about democracy and the people who helped create our nation.”

Assessing the Impact of Art in School

We designed our study to identify the causal impact of community-based arts partnerships and programs in school, including whether a substantial increase in arts education improves student engagement and academic achievement. The study’s central feature is the random assignment of eligible applicant schools to participate (or not) in the initiative. This approach ensures—and our data confirm—that participating and non-participating schools are similar based on their grade levels, student demographics, preexisting arts resources, and percentages of students earning scores of at least “proficient” on statewide math and reading assessments. These schools also had equivalent numbers of school-community partnerships before the initiative began: an average of 2.80 partnerships at schools that did not take part compared to 2.76 at participating schools. After the program, participating schools gained 7.10 more partnerships, and students experienced 5.03 more arts educational experiences over the course of a school year compared to students at non-participating schools.

Our analysis is based on data from 2016–17 and 2017–18 for 15,886 students in grades 3 through 8. These students attended 42 schools, 36 of which were elementary schools. In all, 86 percent of students in the sample qualified for free or reduced-price school lunch, and 33 percent were English language learners. In terms of race and ethnicity, 68 percent of students identified as Hispanic, 25 percent as Black, and 3 percent as white.

We consider individual student attendance and enrollment records, disciplinary records, and test scores on the State of Texas Assessments of Academic Readiness (STAAR), which include reading and math tests in grades 3 through 8, writing tests in grades 4 and 7, and science tests in grades 5 and 8. In addition, we conducted an original survey in 2017–18. We successfully collected and linked outcome survey data to the district’s administrative data for 10,066 eligible 3rd–8th grade students (79 percent), and 7,640 eligible 4th–8th grade students (78 percent of the sample with prior year test scores). We use the latter sample when examining test-score outcomes so that we can control for any minor differences in students’ academic achievement before the start of the intervention.

The survey items are intended to capture levels of college aspirations, opinions about the value of the arts, indicators of social-emotional learning, and school engagement. The baseline survey was administered at the beginning of the fall semester (late September through early October) and the outcome survey at the end of the school year (late April through May).

We group student responses to create measures of school engagement and empathy. Our school engagement measure captures how students rate their agreement with statements like, “School work is interesting” and “This school is a happy place for me to be.” Our emotional empathy measure is based on a single survey item: “I want to help people who are treated badly.” Our cognitive empathy measure assesses the degree to which students can understand and learn from someone else’s perspective, through survey items like, “I can learn about my classmates by listening to them talk about works of art.” and “Works of art… help me understand what life was like in another time or place.” Students’ college aspirations were captured by a single item (“I plan to go to college”) and are indicated by a binary measure of whether students strongly agreed or not.

Results

Increasing students’ arts educational experiences has positive effects on student discipline, writing achievement, school engagement, and empathy. At participating schools, 13.8 percent of students received disciplinary infractions compared to 17.4 percent at non-participating schools—a difference of 20.1 percent. Students’ writing scores are 13 percent of a standard deviation higher than at similar schools with less arts education. School engagement increases by 8 percent of a standard deviation, and students’ emotional and cognitive empathy grow by 7.2 percent and 3.9 percent of a standard deviation, respectively (see Figure 1).

Our analysis does not find effects on students’ math, reading, or science achievement, contrary to popular claims that arts education has a transfer effect on other subjects. However, the positive effects on writing achievement on statewide standardized tests are noteworthy. Many of the arts programs offered opportunities for self-expression and reflection, and some included student writing exercises, either through a specific focus on literary arts or arts-integrated writing activities. The STAAR writing test features open-response expository essays to assess composition skills as well as multiple-choice items on mechanical skills. When we disaggregate student scores on this assessment, we find significant increases on both sections. But the effects are twice as large for the written compositions than for the mechanics sections, at 18 percent and 9 percent of a standard deviation, respectively. This finding aligns with the theory that participation in arts experiences improves students’ ability to express themselves and articulate their own ideas.

The positive effects on students’ writing achievement are especially large for English language learners, whose scores increase by 27.1 percent of a standard deviation overall (see Figure 2). For elementary-school English language learners, the effect is 34.8 percent of a standard deviation. English language learners also experience greater-than-average gains in school engagement, at 14.3 percent of a standard deviation, and emotional empathy, at 15.7 percent of a standard deviation. They are 6.5 percentage points more likely to plan to attend college, despite the fact that the program did not increase college aspirations significantly for students overall.

These findings reinforce earlier research showing the benefits of using arts-learning techniques to deliver core content to English language learners, including increases in written and oral language skills and student engagement and decreases in absences. Researchers have suggested that arts learning increases verbal interactions between students and teachers and offers multiple pathways to connect with educational content. Moreover, the arts programs in Houston tended to have a strong emphasis on art from a diverse array of cultures, which may be especially engaging for students whose first language is not English.

Our analysis also finds notable differences in the experiences of elementary and middle-school students. Writing achievement improves by 19.7 percent of a standard deviation for elementary-school students compared to 5 percent of a standard deviation for middle-schoolers. There is no improvement in school discipline at elementary schools, whereas middle-school students are 6.8 percentage points less likely to experience an infraction. We find opposite trends in school engagement: it grows by 21 percent of a standard deviation for elementary-school students but declines by 12.5 percent of a standard deviation among middle-school students. We see a similar split in students’ college aspirations: an increase of 5.2 percent at elementary schools and a decrease of 4.6 percent in middle schools.

One possible explanation is the implementation of the program, which was primarily focused on elementary schools. Programming in middle schools tended to be more piecemeal, one-off experiences, whereas elementary schools were more likely to opt for artist residencies where teaching-artists provided arts instruction to entire grades on a weekly or semi-weekly basis for a semester or full school year. As a result, smaller proportions of middle-school students participated in arts programming, and those that did were exposed to a diluted dosage—limitations that may have compromised students’ enjoyment or engagement with the arts. It could also be the case that younger students are more receptive to arts education experiences, since educational interventions tend to have greater effects in early years.

A Well-Rounded Education

Our investigation, the first large-scale randomized control trial of an arts education program implemented in an authentic school setting, finds significant and policy-relevant benefits for students across a diverse array of elementary and middle schools in the nation’s 7th largest school district. When young people engage with the arts, they gain unique opportunities for self-discovery, social development, and community connections. When arts education is part of the school day, students experience greater school engagement, fewer disciplinary infractions, enhanced social-emotional development, and stronger academic achievement in writing. Arts education is a promising option for policymakers interested in improving social-emotional learning outcomes and student behavior.

Our study is not without limitations. Our results may not be generalizable to schools where leaders are not dedicated to supporting the arts. The results also may not translate easily to communities without sufficient arts resources and institutions. The findings also reflect the severely deficient arts resources that participating schools had at the outset of the program. A similar program in schools with higher initial levels of arts resources may not produce the same effects.

Still, our analysis provides evidence that arts education can support student success above and beyond its intrinsic benefits. We also show that expanding arts education does not harm student achievement on standardized tests—and actually benefits writing performance. As education policymakers seek reforms that improve school engagement, school climate, and other social-emotional and behavioral outcomes to restore student progress and mental health after pandemic-related disruptions, they should weigh the opportunity costs when arts education is decreased or eliminated.

Daniel H. Bowen is associate professor at Texas A&M University. Brian Kisida is associate professor at the University of Missouri. They co-direct the Arts, Humanities & Civic Engagement Lab, supported in part by the National Endowment for the Arts.

This article appeared in the Summer 2023 issue of Education Next. Suggested citation format:

Bowen, D.H., and Kisida, B. (2023). The Fine Art of School Engagement: How expanding arts education affects learning, behavior, and social-emotional growth. Education Next, 23(3), 48-54.

In education policy and public debate, we often talk about students from “low-income” families. That descriptor is typically based on data from the National School Lunch Program, which provides qualified students with school meals for free or at a reduced price. Enrollment in the program, which is operated by the U.S. Department of Agriculture, plays a central role in identifying low-income students in U.S. schools and thus a central role in consequential education funding and accountability policies at the federal, state, and local levels.

For example, the federal Every Student Succeeds Act requires states to track gaps in student achievement by poverty status. Among the 50 states, 44 use free and reduced-price lunch enrollment to identify low-income students. These data are also commonly used to allocate federal, state, and local funding to schools serving low-income children. School and district poverty rates, as determined by free and reduced-price lunch enrollment, additionally feature prominently in social science research, school-funding lawsuits, state laws and regulations, and philanthropic investment.

Yet a close look shows that free and reduced-price meal designations in the National School Lunch Program are grossly inaccurate indicators of family income. Using administrative data from Missouri, we find that student enrollment in the program is oversubscribed by about 40 to 50 percent relative to stated income-eligibility rules. This finding is not unique to Missouri. We see the same basic pattern in an extended sample of 27 states. Moreover, this is not a recent phenomenon. Enrollment in the school lunch program was oversubscribed even before 2014–15 when the “Community Eligibility Provision” was rolled out nationally, which permits sufficiently high-poverty schools and districts to enroll all their students to receive free meals.

While it has been understood for some time that school lunch enrollment as a poverty indicator is blunt and prone to error, the magnitude of the problem has not yet been fully appreciated. In exploring the rules, features, and processes of the National School Lunch Program, we find that the program’s design, incentives, and lack of income-verification enforcement likely contribute to the oversubscription. These findings raise important questions about the administration of a program that supports the nutrition of American schoolchildren as well as key datasets driving policy and funding decisions across the country.

More Than Just Lunch

For three quarters of a century, schoolchildren from low-income families have received low- or no-cost meals under the National School Lunch Program. In 2019–20, the program provided subsidized lunches to nearly 22 million students at about 94,000 public and nonprofit private schools across the United States.

Students are enrolled in the program in two ways. Through direct certification, students are automatically enrolled for free school meals if their families receive benefits such as food assistance, Medicaid, and Temporary Assistance for Needy Families, or if they are migrant, in foster care, or homeless. Alternately, school districts administer income surveys to parents to determine eligibility. Students from families with incomes at or below 130 percent of the federal poverty line qualify for free meals, and those from families with incomes between 130 and 185 percent of the poverty line qualify for reduced-priced meals. State and federal aid programs like Medicaid and food stamps verify incomes reported by participants, while school districts usually do not. In addition, if an attempt to verify eligibility fails, a student’s enrollment in the lunch program ends, but there are no other repercussions.

Enrollment in the school lunch program is a commonly used proxy for student poverty in policy, research, and for other purposes. But is it an accurate indicator of family income? To answer this question, we analyze school meal enrollment and two alternative measures of poverty in Missouri schools during the 2016–17 school year to determine how closely they are aligned. Our data on meal enrollment are from the state education department and show students’ National School Lunch Program designations. Each student is coded as enrolled for free meals, reduced-price meals, or neither. Our alternative poverty measures are based on students’ direct certification data and estimates of school-neighborhood poverty from the National Center for Education Statistics, which are based on the incomes of households located near schools as reported in the Census Bureau’s American Community Survey.

Assessing Accuracy

A case could be made for using either direct certification or school neighborhood poverty data for the purpose of assessing the accuracy of students’ meal designations. In Missouri, direct certification applies to students from families living at or below 130 percent of the poverty line, which is the same income threshold for free-meal enrollment by the National School Lunch Program. And while school-neighborhood poverty estimates are reported as the average family income associated with a school, with some basic adjustments they also can be used to estimate the share of students living at or below 130 percent of the poverty line.

However, there is no guarantee that either alternative metric is itself accurate. Therefore, in a two-way comparison of free lunch data to either direct certification or school neighborhood poverty data, it would be difficult to know the source of any discrepancy. Given this, we first compare the two alternative data sources to each other. When we do this, we find that the estimated shares of students in a school living at or below 130 percent of the poverty line are very closely aligned, which gives us confidence that both alternative measures are accurate, on average.

We then conduct similar tests to assess the accuracy of the shares of students who receive free or reduced-price lunch. That is, we test whether the school lunch program enrollment share in a school matches the share of students living at or below 130 percent of the poverty line (free) or 185 percent of the poverty line (free and reduced price) as measured by the direct certification and school neighborhood poverty data. If schools and districts are following the National School Lunch Program rules, these numbers should line up.

This is not what we find. We conduct a series of alignment tests where a value of 1.0 indicates one-to-one correspondence; that is, 1.0 means the poverty data from the measures being compared match each other across Missouri schools, on average. While the direct certification and school neighborhood poverty measures are aligned with each other, neither of them lines up with free or reduced-price lunch enrollment (see Figure 1). Compared to direct certification, free lunch enrollment flags 39 percent more children in a school as living in households with incomes at or below 130 percent of the poverty line, on average. Similarly, compared with school neighborhood poverty, 47 percent more children are flagged as living in households with incomes at or below 130 percent of the poverty line in the free lunch data.

We then assess alignment at the free-and-reduced-price enrollment threshold of 185 percent of the poverty line. Unfortunately, we cannot conduct the alignment test at this threshold using direct certification data because that threshold is 130 percent in Missouri. However, we can use the school neighborhood poverty data, which show that enrollment for free and reduced-price meals is also substantially oversubscribed, by about 40 percent.

Next, we explore how the Community Eligibility Provision figures into our findings. This provision, which was included in the 2010 reauthorization of the program and was rolled out nationally during the 2014–15 school year, subsidizes free meals for every student at participating schools and districts. To be eligible, a school or district must have at least 40 percent of students qualify for direct certification. For districts and schools in Missouri (and many other states) that adopt community eligibility, all of their students are reported as being enrolled for free meals. Community eligibility for free meals will certainly contribute to our finding that the poverty rate is overstated by data from the National School Lunch Program, but the magnitude of the effect is unclear.

To disentangle the effect of the Community Eligibility Provision, we incorporate data from the 2013–14 school year, just before the provision was implemented. Specifically, for schools that adopted community eligibility in our data from 2016–17, we use their free and reduced-price meal enrollment rates to the 2013–14 values. We leave the enrollment rates for non-participating schools unchanged. This exercise shows that while the Community Eligibility Provision has contributed to oversubscription in the free lunch category in recent years—as expected—it is not the primary driver. The provision explains 15 percentage points of the free lunch oversubscription and 9 percentage points of the free-or-reduced-price-lunch oversubscription in our data, or about one third to one fourth of the total oversubscription rates. The implication is that even before the Community Eligibility Provision, enrollment was greatly inflated relative to the National School Lunch Program’s stated income thresholds.

Finally, we consider school lunch and neighborhood poverty data from a larger 27-state sample to show that our findings are not unique to Missouri. This exercise involves different datasets and some additional assumptions, the details of which we provide in another publication. Suffice it to say here that the average oversubscription rates in this larger sample are close to the rates in Missouri. We conclude that the oversubscription of free and reduced-price lunch is likely endemic in the United States.

Why Is Enrollment Oversubscribed?

The degree to which enrollment in the National School Lunch Program is oversubscribed is not well understood—but perhaps it should not be surprising. Outside of direct certification, free and reduced-price lunch enrollment is based on mostly unverified surveys. These are administered by school districts to parents, and both groups have incentives that encourage oversubscription.

Districts may be motivated by concerns about child welfare and academic performance; healthy, ample lunches during school contribute to both. But districts also may be incentivized to encourage and approve parent applications in order to gain access to additional federal, state, and local funding to support low-income students. Meanwhile, parents are incentivized to enroll their children because participation lowers their food costs.

In addition, the United States Department of Agriculture does not seem particularly interested in enforcing income eligibility rules. As noted by David N. Bass, only a very small number of applications go through an income-verification process (see “Fraud in the Lunchroom,” feature, Winter 2010). In fact, according to the department’s Eligibility Manual for School Meals in 2017, attempting to verify more than 3 percent of applications without special cause is prohibited. When eligibility is checked and cannot be verified, the student’s meal subsidies are discontinued, but there are no other consequences. The incentive structure clearly favors districts and parents stretching the boundaries of eligibility.

We do not want to go too far down the path of wondering why the federal agriculture department does not enforce its income-eligibility policies more strictly, much less whether it should. The most obvious explanation is that lax enforcement is a strategy to increase meal access for students in public schools, especially considering other initiatives to promote broader access to subsidized school meals, like community eligibility. This may be an appropriate approach to policy implementation given evidence that children benefit from expanded access to free and subsidized meals.

However, this highlights a fundamental problem with using these data to inform other consequential education policies: enrollment for free or reduced-price school lunch is not a reliable measure of family income. Rather, it is a measure that can be, and seemingly is, manipulated by administrators to promote their own objectives related to meal access and program participation. The end result is that school lunch data are a poor proxy for student poverty counts. The problem is not with the National School Lunch Program’s administration of its own program, but rather the education system’s reliance on enrollment data to achieve objectives for which the program and data were never designed—and are not maintained—to support.

A Policy Problem

The use of data from the National School Lunch Program in consequential education policies is ubiquitous. The most prominent example is in state funding formulas, which use free and reduced-price lunch enrollment as the basis for distributing billions of dollars to school districts every year. While states’ allocations of federal Title I aid to support low-income students are based on Census data, not National School Lunch Program data, school lunch data can affect the allocation of federal aid within states and school districts. State accountability policies that track achievement gaps by poverty status also commonly use free and reduced-price lunch enrollment to identify students in the “low-income” group.

The substance and scope of these policies suggest that the consequences of inaccurate school lunch data are significant. For example, consider a funding formula designed to allocate resources to students living at or below 185 percent of the poverty line. If oversubscribed school lunch data are used to proxy for this condition, our estimates from Missouri indicate that the number of students identified as low-income would be overstated by about 40 percent. If the resources to support low-income students are from a fixed budget set aside to support the true target population, the inflated count due to oversubscription would greatly dilute the resources available for each targeted pupil. And that would shift funding away from the most severely disadvantaged students.

Our findings support the position that the education system should move away from relying on National School Lunch Program meal designations as consequential measures of income status. In addition to showing that lunch program participation is greatly oversubscribed, we also note the possibility of substantial variation in oversubscription among school districts. This can lead to a situation where districts receive funding support that is more related to their success in soliciting applications that show eligibility rather than the actual number of low-income families they serve. To the extent that this variation exists, school districts that are more aggressive in signing up students or where parents are more engaged in the application process stand to gain more than districts that are less aggressive, even when their underlying levels of true poverty are the same.

Where do we go from here? Consternation among policymakers caused by the Community Eligibility Provision has led some states to change from using school lunch data to using direct certification data to count low-income students. Our findings in Missouri suggest this shift improves accuracy. However, a caveat to this result is that different states implement federal social-assistance programs differently, and those particularities make it difficult to project how broadly our findings will generalize outside of Missouri. The primary concern is how state policies differ regarding eligibility for food assistance through the Supplemental Nutrition Assistance Program, the primary program that leads to direct certification. Some states allow families with incomes of up to 200 percent of the poverty line to qualify for the program through Broad Based Categorical Eligibility. Missouri is one of a handful of states that does not have Broad Based Categorical Eligibility.

Variation across states in their policies regarding Broad Based Categorical Eligibility has two implications for poverty measurement using direct certification data. First, it means that direct certification status conveys different information about the level of poverty in different states. This has implications both for individual state policies and federal policies that affect multiple states. Second, it is unclear whether direct certification status will accurately measure the income thresholds intended by state rules, given differences among states in program participation and how income rules are enforced. For example, in some states, participation in Medicaid can lead to direct certification, but research shows that many Medicaid-eligible families do not participate in Medicaid. Concerns have also been raised about the fidelity with which Broad Based Categorical Eligibility criteria are enforced.

A larger conceptual concern is that income metrics based on direct certification share a critical flaw with metrics based on free and reduced-price lunch: they are not policy invariant. Like with data from the National School Lunch Program, the criteria that determine direct certification status are subject to continued change as policymakers target evolving policy objectives outside of the education system, not accurate poverty measurement.

Looking Ahead

Despite these concerns, direct certification data are likely the most feasible alternative to school lunch data to identify low-income students and implement education policies to support those students—at least in the short run. Over a longer horizon, we hope for more comprehensive solutions. One aspirational alternative would be to merge education data with tax data from the Internal Revenue Service, state tax agencies, or both, which could capture family income more accurately. This merge is technically feasible, and proof of concept has been established by recent research and at least one state policy. However, to adopt this as common practice would require overcoming political barriers and establishing new avenues of data sharing between agencies in most states.

In the more immediate term, it is worth considering policies that lessen the emphasis on flawed measures of family income in favor of broader indicators of student need. For instance, we could develop generalized measures of student disadvantage to inform education funding and accountability policies. Such measures could incorporate imperfect information on poverty from subsidized meal data and direct certification data but also include information about geographic mobility, attendance patterns, test and other school performance measures, and participation in remedial programs, among other factors. By considering these many facets of disadvantage together, we can improve measurement and expand our understanding of the broad range of need among students.

Ishtiaque Fazlul is a clinical assistant professor at Kennesaw State University. Cory Koedel is a professor at the University of Missouri, where Eric Parsons is an associate teaching professor.

This article appeared in the Spring 2023 issue of Education Next. Suggested citation format:

Fazlul, I., Koedel, C., and Parsons, E. (2023). A Poor Poverty Measure: To identify children in need, look beyond free lunch data. Education Next, 23(2), 48-53.