Many private colleges and some public institutions no longer require prospective students to submit their scores on standardized tests such as the SAT and the ACT. Now the coronavirus pandemic is hastening that trend, as additional institutions announced this spring that they would waive the testing requirement for students applying for fall 2021 admission. With testing dates canceled at least until June 2020, more colleges may consider going “test optional.” Downplaying standardized tests, some education leaders say, can help increase the racial, ethnic, and socioeconomic diversity of a campus population. Is that claim true? And might this policy have any negative consequences? Increasingly, state university systems are facing these questions as they consider the use of standardized testing in the admissions process. Should these public institutions, which often serve thousands of students, go test optional? Jack Buckley, president of the testing company Imbellus, Inc., says no, that test scores have a legitimate place in a holistic admissions approach. Dominique Baker of Southern Methodist University and Kelly Rosinger of Penn State College of Education argue that test-optional policies can enhance equity in college admissions.

 

Standardized Tests Can Serve as a Neutral Yardstick

By Jack Buckley

 

 

 

Test Optional Offers Benefits but It’s Not Enough

By Dominique Baker and Kelly Rosinger

 

 

Although many postsecondary students in the United States attend local or regional nonselective institutions, the selective-college admissions process nevertheless captures the imagination of the media and policymakers year in and year out. One frequently returning story has centered on the growing number of institutions changing their policies on the use of standardized testing in admissions. These reforms—typically grouped together under the label “test optional”—are touted as fostering an increase in campus racial, ethnic, and socioeconomic diversity.

More than 1,000 institutions, mostly liberal-arts colleges, have jumped on the test-optional bandwagon over the past five decades. Recent years have seen an acceleration of the trend—and an expansion to a much broader group of colleges and universities, including a small handful of large public institutions (such as the University of Delaware) and the elite, private University of Chicago. The most closely watched case is the University of California system, whose board of regents is set to decide this year whether or not to make the submission of SAT or ACT scores optional for applicants.

With some 226,000 undergraduates statewide, the University of California could become the largest public university system to eliminate or de-emphasize standardized tests in admissions. That possibility prompts a closer look at test-optional reform and how it could affect the nation’s state-university systems.

What Is “Test Optional”?

Admissions policies that downplay or do away with standardized test scores come in several flavors. The simplest and purest is a “test-blind” policy, under which admissions officers do not consider applicants’ test scores under any circumstances. Virtually no U.S. institutions are test blind. (While Hampshire College in Massachusetts has a published test-blind policy, that institution will look at students’ submitted International Baccalaureate, AP, and SAT Subject Test scores, just not their ACT and SAT I scores.)

The most common form of test-optional policy encourages applicants to submit their college-entrance examination (usually ACT or SAT) scores only if they believe that so doing will help their chances of admission.

“Test-flexible” policies are another, less common variation, generally meaning that applicants are not required to submit ACT or SAT scores but that they are required to submit some sort of standardized-testing results—often AP, IB, or SAT Subject Test scores.

In the case of the University of California, media accounts have reported that the regents are considering either a truly test-optional policy or an approach that requires students to submit scores from the state’s Smarter Balanced high-school accountability tests.

Why Avoid Standardized Tests?

Colleges and universities that drop the SAT and ACT from their admissions requirements generally say they do so because they view the exams as biased against disadvantaged minority and low-income students or they consider high-school GPA to be at least as predictive of college success, if not more so.

On the question of bias, test-optional proponents often cite several kinds of evidence, including the undeniably racist early history of standardized intelligence measurement; the gaps that persist between the average scores of students of different races, ethnicities, or socioeconomic backgrounds; and the ability of wealthy parents to secure advantages for their children, such as private “test-prep” courses.

While the worst examples of past, misguided efforts to measure human intelligence and aptitude are indefensible, today’s standardized admissions tests are developed explicitly to measure the mastery of academic content that students are expected to learn in school—not IQ or general intelligence or some other notion of aptitude. Both the ACT and SAT are aligned with state content standards in mathematics and English language arts—which is one reason some states use college-entrance exams as their state accountability test in place of longer, state-specific assessments.

Citing the existence of score gaps as evidence of test bias is particularly puzzling. Results from the National Assessment of Educational Progress and many other standardized tests consistently show, as do countless research studies, that Asian and white students, on average, outperform their black and Hispanic counterparts and that wealth and socioeconomic status confer a compounding advantage to academic performance and life outcomes. How, then, could one expect a standardized measurement at the end of high school not to reflect the unfortunate educational inequities inherent in American society?

If high-priced test prep or coaching is exacerbating these score gaps, that is a valid case against the SAT and ACT. As the Varsity Blues bribery scandal demonstrates, some parents will pay or do almost anything to secure an admissions advantage for their children. Yet evidence fails to show that the ecosystem of test-prep providers, consultants, and coaches does much more than profit from parental anxiety. Moreover, both major testing companies that oversee the ACT and SAT have introduced free test-preparation materials in an attempt to offset any advantage of test prep, although the efficacy of such offerings is unknown. Finally, given the amount of parental anxiety over the elite selective-admissions process, it’s likely that the widespread adoption of test-flexible and -optional policies would simply cause a shift in emphasis to test prep for any new required tests (IB, AP, Smarter Balanced) and to pricey tutoring for boosting high-school GPA.

Perhaps the strongest argument against the use of the SAT and ACT in admissions is that high-school grades offer similar value in predicting college success. While both major testing programs have produced decades of research showing a small overall advantage when institutions use both a standardized test and GPA, the relative predictive power of tests varies across institutions. Indeed, both ACT and the College Board work confidentially with colleges and universities to conduct local validity studies meant to shed light on the utility of testing relative to GPA and how the different sources of data should be weighted in support of decision making. Larger institutions that conduct their own such studies often report that GPA alone is a sufficient predictor. Yet here a caveat is in order: a small but growing body of evidence finds that high-school grades are “inflating” over time, and that they are rising at a faster rate for affluent, advantaged students. It’s possible that standardized tests act as a partial check on grade inflation; if so, then reducing or eliminating their role in college admissions could worsen the problem. What’s more, the predictive value of the GPA could diminish if too many institutions stop using test scores.

Test Optional in State Universities

While most state systems comprise colleges and universities of varying levels of selectivity, all of their individual institutions must review a large number of applicant files compared to small, private colleges. For example, the University of California system in 2018 reviewed files from over 180,000 applicants to fill about 46,000 seats. Conducting an in-depth, “holistic” review of every single application would not be feasible, so large institutions often prescreen students based on factors such as test scores and GPA to winnow down the pool. In this context, the scores can help admissions officers put students’ grades (and different high schools) in perspective. This is presumably one of the reasons the University of California system is considering the use of the state’s Smarter Balanced high-school test scores in place of the SAT and ACT rather than going test optional or test blind, especially since recent research suggests that the Smarter Balanced test exhibits similar levels of predictive validity and smaller differences between relatively advantaged and disadvantaged groups of students.

Adopting a test-flexible policy or requiring a single test other than the ACT and SAT appears to be a viable strategy for a state system. The university gets to continue using standardized-test scores to screen applicants while also getting credit for ending the tyrannical reign of the despised college-entrance exams. There are several considerations, however, that state-university administrators and governing bodies should keep in mind.

First, although alternative tests may appear more equitable than the SAT or the ACT, this advantage may not necessarily arise from any specific properties of the assessments, but simply because they have never been subjected to the corrupting pressures that high-stakes admissions tests must withstand. Smarter Balanced, for example, which California uses as part of its public-school accountability system, holds high stakes for educators and leaders but has little impact on individual high-school students. Affluent parents don’t pay thousands of dollars for “ringers” to take the Smarter Balanced test for their kids, and organized criminal rings do not supply those ringers, attempt to steal test forms, or sell questions online—at least not yet. State systems will need to constantly monitor the performance and security of the various tests they permit. Other possible assessments in a test-flexible menu, like AP or IB, might better resist these pressures than state-accountability tests, but that is unknown until the pressures become real.

Next, it’s important to note that there is little evidence that test-optional policies succeed in increasing campus diversity. Indeed, what little scientific research exists has produced mixed findings at best. When an institution goes test optional, applications go up, average test scores rise (since applicants with lower scores choose not to send them), but little else seems to change, at least among the liberal-arts colleges that have implemented the policy for the longest amounts of time.

Finally, and as noted earlier, it stands to reason that standardized test scores will reflect the achievement gaps endemic to American education, but colleges and universities are free to consider this reality when developing their holistic application-review procedures. Indeed, within the University of California system there are examples of such adjustments. UC San Diego, for one, accepts disadvantaged minority and lower-income students with average SAT scores that are more than a standard deviation below those of their more-advantaged counterparts.

Indeed, the UC Academic Council’s standardized-testing task force recently concluded, after a review of the evidence around admissions testing, that UC institutions on the whole are using testing responsibly and that the practice has contributed to campus diversity. Moreover, the task force recommended against switching to the state’s Smarter Balanced assessment, in part because such a shift would make it harder to compare in-state applicants to those from states using different testing systems. In its report, issued in February 2020, the task force recommended that the UC system continue to use the SAT and ACT while also working to develop a new standardized-admissions test, tailored to the UC system, that would measure a “broader array of student learning and capabilities” and possibly “enable UC to admit classes of students more representative of the diversity of the state.”

Testing in Perspective

State systems are under enormous pressure to provide access to low-cost, high-quality postsecondary education in a way that is equitable and fair. It is understandable, given the controversy that besets college-entrance testing, that they should want to scrutinize the role of these tests in admissions. When used thoughtfully, as part of a holistic process, well-designed standardized assessments do not have to be a barrier for disadvantaged students—they can serve as a neutral yardstick that helps put students’ academic performance in context. If state systems elect to shift their policies around the use of such tests, they should do so with a clear eye and an active program of research to avoid unintended consequences.

This is part of the forum, “Should State Universities Downplay the SAT?” For an alternate take, see “Test Optional Offers Benefits but It’s Not Enough,” by Dominique Baker and Kelly Rosinger.

A growing number of colleges and universities—including state systems of higher education such as the University of California and Indiana University—are weighing the role of standardized-test scores in the admissions process, typically citing concerns that the tests disadvantage low-income students and students of color and that the scores add little beyond high-school grades to predict a student’s ability to succeed in college.

Since 1969, when Bowdoin College announced that the SAT and ACT would be optional for applicants, dozens of selective liberal-arts colleges have followed suit. While the test-optional movement began in in that sector, over the past five to ten years it has expanded to highly selective private universities such as the University of Chicago and George Washington University, and to public universities such as Temple and the University of Delaware (for in-state residents only). The test-optional movement has also extended to graduate education, with a growing number of programs eliminating the GRE requirement for applicants and nearly 40 law schools allowing applicants to submit GRE scores in place of the traditional LSAT.

The National Center for Fair and Open Testing reports that more than 1,000 institutions have made test scores optional for undergraduate applicants or have de-emphasized the use of standardized-test scores in the admissions process. Among them are 370 colleges and universities that receive top rankings from U.S. News & World Report in their respective categories. The question is, do test-optional policies do what they are intended to do—increase racial and income diversity on campus?

The Research 

Generally speaking, selective admissions offices across the United States use college-entrance-exam scores as one factor among many in evaluating applicants. When an institution goes test optional, this usually means that applicants are welcome to send their test scores if they so choose, but they are not required to. When students elect not to send their scores, admissions professionals rely on other criteria, such as grades, personal essays, and extracurricular activities, to make their decisions. Some institutions offer “test-flexible” admissions, still requiring students to submit a standardized-test score but allowing them to choose an SAT subject test or another assessment in place of the SAT or ACT.

A 2015 study by Andrew Belasco, Kelly Rosinger, and James Hearn examined test-optional admissions among selective liberal-arts colleges and found that these initiatives did not expand enrollment among recipients of the federal Pell grant, which is directed toward low-income students, or among black, Latinx, or Native American students. The study did find that the policies helped the colleges themselves by increasing the number of applications and the average standardized-test scores of those who submitted them, both of which boost an institution’s perceived selectivity. A study by Kyle Sweitzer and colleagues, included in the 2018 volume Measuring Success (co-edited by Jack Buckley, our companion essayist in this forum) also examined the effects of test-optional policies at selective liberal-arts colleges and reported similar results. More recently, a 2019 study by Matt Saboe and Sabrina Terrizzi failed to find effects of SAT-optional policies on measures of student quality, selectivity, or institutional diversity.

These studies, however, either focused on private, highly selective liberal-arts colleges; analyzed a short span of years, which restricted the outcomes that could be measured; or compared the test-optional institutions to all four-year institutions in the United States (a problematic comparison from which to draw conclusions). The body of prior research therefore makes it difficult to hypothesize about what would occur if large public-university systems such as the University of California system adopted test-optional policies. Private, highly selective liberal-arts colleges from which the test-optional movement emerged differ from more recent adopters that tend to be larger, more research-focused institutions and even some selective public universities.

A 2019 study by Christopher Bennett,* presented at the annual meeting of the American Educational Research Association, looked at the effects of test-optional policies on campus diversity at private institutions, and his findings diverged from those of previous investigations. Bennett examined more recent adopters of test-optional policies, comparing them to several groups of peer institutions with similar characteristics. The study found that implementing these policies increased the enrollment of black, Latinx, Native American, and, to a lesser extent, Pell-grant students. Because this research focused on the most recent test-optional adopters—which include a wider range of institution types—we might expect similar effects to occur at public institutions.

But are there alternatives to test-optional admissions that could expand access? One strategy several states have employed is requiring high-school students to take one of the college-entrance exams, potentially increasing the likelihood that they will pursue college studies. As of 2017, 11 states had adopted policies requiring all high-school juniors to take the SAT or the ACT, and they also pay for the students’ testing fees. Recent studies by Joshua Hyman and by Michael Hurwitz and colleagues indicate that these policies boost college enrollment by 0.5 to 3 percentage points. While required testing appears to induce low-income students to enroll at an increased rate, there is little evidence that the policy has a particular effect for black, Latinx, or Native American students. Therefore, while mandatory college-entrance exams might have some salutary effect, we are not confident this policy can close both economic and racial gaps in enrollment at selective institutions. Additionally, given the troubling and persistent racial and economic gaps in standardized-test scores, we worry that these policies will not do enough to level the playing field and expand access among underserved students.

Recommendations

In light of recent evidence that test-optional policies at some institutions appear to have expanded campus diversity, we think there could be benefits for students if public institutions elected to adopt such a policy. However, test scores are not the only source of bias in the selective admissions process. Race and class inequalities are baked into many of the metrics that selective colleges use to evaluate applicants. For instance, there are decades of research demonstrating that low-income students and students of color have less access to the advanced high-school coursework that selective colleges view as a measure of a rigorous curriculum. While selective colleges try to evaluate applicants in the context of their individual high schools and communities—that is, taking into account whether students took advantage of the most difficult coursework available to them—other common metrics used to evaluate students may also reflect racial and class privilege.

Factors that vary by race and class can influence the perceived quality of a student’s credentials. Affluent families can afford to hire admissions consultants to prep their children for the college interview or to critique their personal essays. Similarly, underserved students may not have the same opportunity to participate in extracurricular activities as more advantaged peers, potentially because certain pursuits are not available or because they have outside responsibilities, such as having to earn money to support their families. Perhaps this is why a study by Kelly Rosinger and colleagues found that consideration of extracurricular activities during the admissions process did not move the needle substantially when it came to expanding access to selective colleges.

Other barriers stand in the way of low-income students and students of color gaining access. For instance, studies show selective colleges tend to recruit from high schools whose students are largely white and well-to-do. Less-advantaged students also lack the means to pay for college, may not have help navigating the admissions process, and may wonder if they will develop a sense of belonging on an “elite” campus. Without a comprehensive approach to recruiting, admitting, enrolling, and supporting underserved students on the way to and through college, higher education will fail to serve not only these students, but also the entire country.

It is tempting to view test-optional policies as a silver bullet that can expand access to selective colleges for all qualified students, regardless of race or class, but large systemic issues rarely have simple solutions. To a limited extent, such policies may contribute to racial and socioeconomic diversity on college campuses, but until the higher-education sector addresses the many other hurdles that block the way, access to and success in college will continue to elude many of the country’s qualified young people.

This is part of the forum, “Should State Universities Downplay the SAT?” For an alternate take, see “Standardized Tests Can Serve as a Neutral Yardstick,” by Jack Buckley.

The Supreme Court heard oral argument Monday on two of this term’s most highly anticipated cases: Our Lady of Guadalupe School v. Morrissey-Berru and St. James School v. Biel. Both involve teachers who were fired from Catholic Schools (For background, please see “Justices Will Hear Cases About L.A. Catholic Schools”). The question was whether their firings were covered by the “ministerial exception,” which allows religious institutions to select their own ministers and, thus, exempts ministers from statutory employment protections.

When the Supreme Court first recognized the ministerial exception in 2012’s Hosanna-Tabor Evangelical Lutheran Church and School v. Equal Employment Opportunity Commission it held that the First Amendment’s “free exercise” clause must allow religious institutions to hire and fire ministers—otherwise the government could compel them to retain individuals that the institution believes are no longer suited to teach their faith. But it left the scope of the exception broadly undefined. Monday’s argument focused on how far it should go. While the court’s usual ideological divisions were apparent, the balance of oral argument seemed to favor the schools.

On one side, the liberal bloc of justices expressed concern that if these teachers were considered ministers then nurses or janitors in religious hospitals could be fired because they were considered ministers. On the other, the court’s conservatives were more concerned about protecting the rights of religious institutions to determine who should be considered a minister. Justice Gorsuch took perhaps the hardest line, indicating that he felt uncomfortable having a “secular court” second-guessing a religious institution and declaring that some employees’ activities were insufficiently related to religion to count them as ministers.

Perhaps the most interesting question came from the normally taciturn Justice Thomas. Others have noticed that the court’s move to a remote format has made Thomas almost loquacious. His ability to get to the core of the issue was illustrated by a question for the employees’ attorney, Jeffrey Fisher. Fisher was trying to argue that the employees’ job duties, which included teaching weekly religion classes and supervising students during Mass,  were insufficiently religious for them to be considered ministers. This led Thomas to ask if the employees performed the same religious functions in a public school that they did in the Catholic schools, would it “be a violation of the Establishment Clause?” Fisher responded that it would be. Thomas responded, “don’t you think it’s a bit odd that – that things that violate the Establishment Clause, when done in a public school, are not considered religious enough for Free Exercise protection when done in a parochial school?”

The trouble did not end there for Fisher. Justice Alito asked him if a teacher in a Catholic school only taught religion classes, whether the ministerial exception would apply. Fisher responded that they “would not be a minister in that case” because the teacher would not be in a position of “spiritual leadership of the congregation.” At that point, Fisher might have also lost the vote of Justice Kagan. In response, she said she thought that such a teacher “is protected by the exemption” and that she understands that sometimes teachers might have to teach other subjects making it difficult to “draw the line” between a full-time religious teacher and a “half” or “quarter” time religious teacher.

The teachers had placed great emphasis on the fact that they were not in significant positions of leadership and were not required to be Catholic. Neither argument appeared to gain any traction with the court’s conservatives or to be compelling to either Justices Breyer or Kagan. On the question of leadership, Breyer expressed concerned that in some religions “everyone” has a position of authority. Who should count as minister for those faiths and how could the court provide guidance that wouldn’t require courts to “meddle” too much in religion? Likewise, Justice Kagan wondered why a yeshiva couldn’t hire a non-Jewish expert on the Talmud to instruct on what the Talmud teaches and not have that person qualify under the exception.

In the end, four justices—Thomas, Alito, Gorsuch, and Kavanaugh—appeared willing to grant religious institutions broad discretion to decide for themselves who counts as a minister. Chief Justice Roberts also seemed to favor the schools. He, for instance, was worried about judicial entanglement and requiring titles common in one faith, Protestant Christianity, to be used by others to count employees as ministers. Justice Alito even said he would jettison the phrase “ministerial exception” because it was discriminatory.  However, given Robert’s distaste for 5-4 opinions, it would not be surprising to see him work to construct a 7-2 majority by bringing Breyer and Kagan alongside to craft a decision in favor of the schools but not going as far as the court’s conservatives would prefer. Regardless, you would rather be the schools than the teachers after Monday’s oral argument.

Imagine a 6th-grade math teacher with high hopes for her students. Let’s call her Ms. Rodriguez. She wants her students to find joy in the beauty and complexity of math, make connections to the world around them, and master the skills and content they’ll need to succeed in middle and high school, college, and beyond. She believes that each one is capable of rigorous study and is committed to doing all she can to prepare them.

But in a typical class of 25 students, she’s finding that as few as five can keep up with 6th-grade work. One day, after her students struggled with adding and subtracting decimals, she sensed that most of them hadn’t quite mastered decimal place value in the 5th grade. So she found a suitable lesson online and retaught place value—after all, you can’t hope to add decimals accurately if you can’t keep your tenths and hundredths straight. Her students caught on quickly and were relieved by the refresher.

By chance, the principal dropped in for an observation that day, and the feedback wasn’t good. Stick to the 6th-grade curriculum, he said. That’s what will be covered on the end-of-course exam, so that’s what students need to be taught. There won’t be time enough to cover much else.

Ms. Rodriguez believes in keeping expectations high. But she doesn’t see how students will ever meet 6th-grade standards if she can’t help them address unfinished learning from elementary school that is foundational to middle-school math. Gaps in learning lurk beneath the surface, and math instruction builds on itself each year. By only looking at the current year’s curriculum, how will her students ever master all they need to know?

This is what the organization I lead has identified as the “Iceberg Problem,” and it’s what inspired us to create a personalized-learning model for middle-grade math eight years ago. That program, Teach to One, provides students with individualized instructional programs each day that integrate teacher-led instruction, collaborative learning with peers, and virtual online study. Our goal is the same as Ms. Rodriguez’s: for students to access the right lesson, in the right way, based on where they are starting and where they need to go.

We expected to run into a number of barriers that we would need to overcome if we were to be successful. We would need to raise capital, hire top-notch academicians and technologists, find interested schools, and navigate the myriad of operational complexities that can arise in any school environment—unfilled staff positions, uneven teacher quality, frequent leadership transitions, politicized decisionmaking, and unreliable technology infrastructure, to name a few.

Then came another barrier that we did not anticipate, which has emerged as one of our most formidable challenges.

It is that in math, today’s assessment and accountability policies may be inadvertently working against the interests of the very students they were designed to help.

High Expectations  Matter, And…

The shift to more rigorous college- and career-ready standards has been one of the biggest policy developments in recent decades. Federal education legislation adopted in 2001 under No Child Left Behind and amended in 2015 under the Every Student Succeeds Act requires each state to administer annual math and reading tests aligned with grade-level standards for grades 3 through 8 and at least once in high school. The cumulative impact has set more consistent expectations for students based on benchmarks pegged to a college- and career-readiness trajectory and yielded progress in several areas, including greater transparency into achievement gaps between student subgroups, increased clarity for teachers on what students need to stay on a college- and career-ready path, and more objective information for families on whether students are reaching key educational milestones.

But in math, the subject our organization knows best, these well-intended standards and measures don’t always consider the diverse needs of individual learners to encounter material and progress at an optimal pace. To be sure, keeping expectations high for all students is critical—and for students from historically disadvantaged communities, there is ample evidence that many schools do not expect nearly enough. But within an individual class, an on-grade lesson for one student may be far out of reach, while that same lesson for another student may be too easy. Assuming that grade-level content is what’s best for everyone fails to acknowledge the reality that individual students have different levels of background knowledge.

When students miss key steps along the prescribed grade-level path, or learn at a pace that is faster or slower than state standards anticipate, the standards alone do not provide guidance to teachers on where to focus instruction. Instead, policies signal to a 7th-grade teacher, for example, that all 7th-grade students should be taught 7th-grade content—whether students happen to be performing at grade level or not.

For students who never fell behind, 7th-grade materials may be perfectly appropriate.  But for those who may have missed key concepts along the way, trying to cover 7th-grade material when key concepts from 5th or 6th grade weren’t quite mastered can cause learning gaps to accumulate. The pattern repeats, year after year, as they fall farther away from the college- and career-ready track.

Few can credibly argue that a policy orientation focused on annual grade-level mastery is working. Only 34 percent of American students in 8th grade met the proficiency level or above in math based on the 2019 administration of the National Assessment of Educational Progress, with historically disadvantaged student groups hitting this target at less than half the rate of white students. These middle-grade math woes are often rooted in the elementary years, where one in five 4th graders fell into the lowest tier of math performance, well below grade level.

What are middle-school math teachers like Ms. Rodriguez to do when the grade-level material they must teach depends on foundational knowledge students do not have? How exactly are teachers supposed to deliver an impactful lesson on quadratic equations when many of their students do not understand exponents?

There is an acute tension between an instructional program that is best for each student to ensure they are ready for college and career and an underlying policy context rooted in grade-level expectations. The mathematical skills required for students to engage with grade-level material in middle school and high school are built on a deep, conceptual understanding from previous years. And although many students arrive at middle school without these foundational skills, state and federal policy systems incentivize teaching to grade-level standards to curtail low expectations and inequitable outcomes.

We do not see strong evidence in the field or in research that, in math, strict adherence to grade-level content is best for all students.

There is a path for far more students to achieve college and career readiness, but it requires systematically addressing students’ unfinished learning from prior years. Simply demanding that teachers somehow figure out both how to address students’ unfinished learning from previous years and how to cover all grade-level material may be causing some students to fall farther behind.

 

 

Math: The Jenga Game of Subjects

Math is cumulative, with new learning resting atop earlier mastery. Year by year, students learn interconnected concepts in a coherent progression, building a foundational body of knowledge that undergirds new understanding in the grades ahead.

For example, 7th-grade math typically includes performing basic operations with rational numbers. For students to know how to do that, they need to have mastered several key skills and concepts from 5th and 6th grade, including understanding integers and rational numbers and performing basic operations with decimals and fractions (see Figure 1).

When a student starts the school year with unfinished learning from prior years, the challenge of both covering grade-level material and addressing unfinished learning can be daunting. For example, 8th graders are expected to learn about multistep equations during the course of the school year, even though some students begin the year not having mastered critical predecessor skills such as solving simple equations, operations on rational numbers, or adding and subtracting algebraic expressions.

Imagine being asked to solve 2(x + 1) – x = 5 without understanding the order of operations or how to work with x. It can’t be done. And lessons focused on multistep equations would be lost—alongside the foundational lessons missed the first time around.

So would precious instructional time.

Of course, high-performing math teachers and thoughtful curriculum materials build in opportunities to revisit important concepts, including through “spiraling” questions that reinforce recent skills and topics throughout the school year. But what happens when the missing knowledge is from several grade levels ago? In the example above, each of these component skills could take three to four days to cover sufficiently—and they are only the predecessors for one grade-level skill. For many students, the learning gaps they are expected to bridge in one year in order to attain grade-level proficiency are so substantial that meeting that benchmark in that time period is highly improbable.

Longitudinal studies of individual students over time can show more precisely how students who fall behind are likely to stay behind. In a 2012 study conducted by ACT, researchers tracked math test results from tens of thousands of students to calculate their math skills and relative chances of reaching grade-level expectations in the 8th and 12th grades. Students below grade level in math in 4th grade had just a 46 percent chance of reaching grade-level expectations in 8th grade; those below expectations in 8th grade had a 19 percent chance of reaching 12th-grade expectations (see Figure 2). These figures were even more daunting for the lowest-scoring students, whose chances of meeting expectations for the 8th and 12th grades were 10 percent and 3 percent, respectively. This same study also found that the lowest-scoring students were much more likely to attend high-poverty schools.

 

 

There are multiple reasons why it can be so hard for lower-performing students to catch up in math. In some communities, there are particular challenges in recruiting, developing, and retaining high-quality math teachers, many of whom have more attractive employment opportunities in other sectors. In other communities, ongoing leadership transitions at the school or district level can lead to continual shifts in organizational direction. Poverty-related issues such as trauma, violence, and nutrition are all highly relevant to student academic performance. So, too, are the expectations that adults have for students.

While the impact of these and other factors are well documented, an underlying policy landscape that incentivizes grade-level instruction may also be playing a key role in preventing students from getting back on track.

 

When Policy Meets Practice 

Under federal law, all students in grades 3 through 8 must take a statewide summative assessment in math that is aligned to their enrolled grade level to form the basis of states’ accountability systems. All 6th graders take the 6th-grade test, all 7th graders take the 7th-grade test, and so forth, with only a narrow set of exceptions.

These exams are used as key components across many evaluation and decisionmaking activities. All states are required to set goals based on test results, including to increase the share of students who meet standards in reading and mathematics, accelerate progress of underperforming subgroups, improve graduation rates, and identify low-performing schools for varying levels of support and intervention. Some states have opted to include tests among measures of student growth in teacher-evaluation systems. Local communities review their test performance closely, and many district and school administrators believe their career success is dependent upon success on annual assessments. For many charter schools, meeting test-based goals for proficiency and student growth can mean the difference between closure and continued existence.

States have discretion to choose which test to use. All the tests, though, are required to be aligned to grade-level standards that reflect a college- and career-ready academic trajectory. Few, if any, of the actual test questions relate to anything other than the standards that match each student’s enrolled grade level. The idea is to have the equivalent of an educational “dipstick” so that decisionmakers and the public can have an objective, comparable view of where each student performs in relation to state expectations.

The combination of grade-level tests and high-stakes accountability interjects reliable data into the decisionmaking process of administrators, teachers, parents, and policymakers. It ensures that school communities remain focused on measurable student outcomes and highlights profound inequities across and within schools and districts that can require public response. But as currently constructed, there’s a significant tradeoff: Teachers face pressure to focus instruction on the grade-level material appearing on the end-of-year test, regardless of students’ background knowledge. And in math, for those students whose incoming preparation is poorly matched to standard grade-level expectations, these incentives can produce the opposite of their intended effect.

When a 6th-grade student is taught 6th-grade material, some of those skills will be learned and some will go “unlearned” for a variety of reasons (e.g., lack of predecessor knowledge, uneven teacher quality, student absences). The next year, as the focus of accountability shifts to the 7th-grade assessment, the unlearned skills from 6th grade remain unaddressed, even though those skills may be essential to mastering 7th-grade content. By 8th grade, even more learning gaps accumulate so that by the time a student enters high school, the student is simply unprepared for more advanced mathematical topics.

In middle-grade math, while the gaze of policymakers is focused on how students are performing relative to grade-level assessments, learning gaps continue to accumulate below the surface, making longer-term success harder to achieve. This is the “Iceberg Problem” illustrated in Figure 3.

The authors of the Every Student Succeeds Act likely foresaw the value in measuring growth outside of grade-level standards, as the law permits states to adopt assessments that also “measure academic proficiency and growth using items above or below the student’s grade level.” However, subsequent guidance from the U.S. Department of Education specifies that if states design tests to include additional measures of off-grade performance, they must still measure and score students’ on-grade performance accurately. Any off-grade measures would not satisfy the law’s accountability provisions for student performance. As a practical matter, due to the broad set of standards at each grade level and pressures to reduce test time and length, most summative state assessments are almost exclusively focused on grade-level content.

The federal policies that undergird statewide assessment and accountability systems send an unmistakable signal to middle-grade math teachers: focus your instruction on the grade-level standards.

 

 

Putting Personalization to the Test

Teach to One is just one approach schools use to meet the unique needs of each student. It is designed around 300 mathematical skills and concepts that connect basic numerical understanding to college-readiness benchmarks.

The program uses ongoing assessments of students’ competencies to tailor instruction. A customized software program assesses individual skill levels and creates customized skill libraries, units of study, and tests and quizzes. Day to day, students learn in extended class periods that include instruction from teachers, group work, individual practice, and a brief daily assessment of progress called an “exit slip.” Up to three times a year, they take the Measures of Academic Progress, a computer-based adaptive test that measures learning growth over time.

The idea is to ensure that students are continually learning within their “zone of proximal development,” where the material they are expected to learn is appropriate given what they already understand and where they need to go. With the right support, students working in the “zone” can master content that was previously out of reach. This approach can fill knowledge gaps and catch up students who are lagging, as well as accelerate learning and engage students ready to advance beyond grade-level material. Of the students in grades 6 though 8 who have participated in Teach to One over the past three years, about two thirds started the school year at least two grade levels behind, and 9 percent were four grade levels behind. Just 2 percent were ahead of their grade level.

In its eight years, Teach to One has operated in districts and schools with different philosophies regarding the role of grade-level material, notwithstanding federal signals. In schools where accountability systems imposed at the district or school level are based on growth measures that cross multiple grades, the program can tailor a personalized curriculum for each student that includes a mix of pre-grade, on-grade, and post-grade material, depending on his or her unique starting point. By contrast, in schools focused on students’ performance on annual state assessments, leaders have often asked us to weigh students’ personalized curricula more heavily toward grade-level content for the year. This can mean leaving important pre-grade gaps unaddressed.

Our preference would be to help schools fill pre-grade gaps and cover grade-level material. But a 180-day school year is often not enough to make up for multiple years of unfinished learning. Absent clarity on how to reconcile this tension, schools can struggle to make hard choices about what to prioritize.

This challenge was highlighted in an experimental study of Teach to One at five schools in New Jersey published in 2018. During the study period of 2015 to 2018, different schools requested a variety of program adjustments that either emphasized or deemphasized grade-level content. As a result, researchers were unable to draw any conclusions on the overall impact of the program as it was designed.

What can happen when schools make a clear choice to prioritize learning growth over grade-level exposure?  A 2019 study looking at progress on the Measures of Academic Progress assessment compared the growth of Teach to One students at 14 schools over three years to national averages. It found that Teach to One schools whose accountability systems focused on growth made stronger gains than schools whose accountability systems focused on proficiency (see Figure 4). The study also found suggestive evidence that schools tended to see stronger gains when the math content presented to students matched their tested grade level from the beginning of the year.

 

 

While firm causal evidence of the impact of grade-level assessment and accountability has yet be established, these findings should be viewed as an important contrast to the evidence base for what policy currently promotes: providing all students with grade-level content, regardless of their starting point.

Researchers explored the effects of giving students content far outside their current skill level after a policy push in the early 2000s placed many 8th-grade students in Algebra who would have otherwise taken a pre-algebra 8th-grade math course. In 2008, Tom Loveless found that very low-achieving math students enrolled in Algebra courses performed about seven grade levels below their peers on the National Assessment of Educational Progress and struggled with questions that tested elementary-level understanding. Another study found that low-achieving students pushed into algebra did less well in subsequent math courses throughout high school, especially in geometry (see “Solving America’s Math Problem,” research, Winter 2013).

There is little evidence to suggest that in math, low-performing students pushed into grade-level content without appropriate support and attention to prerequisite skills will be better off in the long run. Yet today’s policy landscape incentivizes just that.

Fixing the Iceberg Problem

This is not a call to reverse the principles of standards, accountability, rigor, transparency, and equity that form the basis of education policy and practice today. These are essential elements for building a school system worthy of the students it serves. The historical roots of our inequitable education system run deep, and the need for guardrails within federal policy to mitigate and, ultimately, reverse this pernicious legacy is essential.

But it is a candid acknowledgement of the tradeoffs and costs that a focus on annual grade-level expectations creates.  In math, preparing students for college or a career requires honestly confronting any gaps in learning that lie beneath the surface. That does not mean lowering expectations for students; rather, given the inherently sequential nature of mathematics, it means designing viable paths that can connect students from where they are starting from to where they need to be.

Today’s policy landscape is constraining these kinds of innovations.

I have seen in our work that individualized instruction and high expectations can go hand in hand, and that if we are able to identify and address unfinished learning from prior years, students can advance more quickly and successfully toward their goals. I hope our experience can help to spur the development of more innovative learning models that seek to address individual students’ learning needs. Teach to One is by no means the only way for schools to do that. Ensuring all students can access a tailored path to college and career readiness will require the development of a variety of innovative learning models with different philosophies and pedagogical approaches underpinning them.

There are several steps states and districts can take to address students’ unfinished learning while working within the current framework of federal law. They can measure learning growth more comprehensively by using adaptive assessments that adjust in difficulty based on student responses. They can also adjust their accountability systems to emphasize proficiency at key grade levels, as opposed to annually. States might also create space for innovation (as Texas has done with its Math Innovation Zones program) so that new learning models can be properly implemented and tested in ways that operate largely outside of the current grade-centric system.

But in the longer term, policymakers and advocates will need to develop and advance a shared vision for what one day might be a new assessment and accountability system. It will need to be one that preserves our current values and commitments without promoting instructional practices that prevent students from accessing academic paths that might better enable their longterm success.

Joel Rose is co-founder and chief executive officer at New Classrooms, which published The Iceberg Problem, from which this essay is adapted.

The corner of my suburban Boston bedroom, where I write this letter, has by necessity become a hotbed of education innovation since mid-March, when the Covid-19 pandemic closed Harvard’s campus and confined most Massachusetts residents to their homes.

It was the venue for the first defense of a doctoral dissertation at the Harvard Graduate School of Education to be conducted virtually, with the student, my fellow committee members, and me in separate locations. (The student was brilliant—and the fact that dozens of her friends and family from around the world were able to “attend” more than offset our inability to congratulate her in person.) And it is where I’ve worked to convert my spring 2020 courses, designed for in-person consumption, into equally productive online learning experiences—an aspiration as yet unfulfilled.

The pandemic has affected every last corner of the American education system, forcing system leaders to attend to an unprecedented array of logistical and technological challenges, educators to teach in new and unfamiliar ways, and parents to adjust their routines to supervise their children’s learning at home.

The closure of schools and much of the economy, however necessary it may be, will have enduring consequences—in learning lost during the crisis and in diminished resources to devote to education in the years ahead. The task facing policymakers, and the rest of us, will be to do what’s needed to ensure that the tragic loss of life the coronavirus has caused does not bring with it a loss of opportunity for the next generation. It won’t be easy.

On the Education Next website, we have already sought to provide a forum to anticipate those consequences and how education leaders might best respond. Executive editor Michael Horn warned that online learning is “about to get a bad reputation at many campuses” as hastily built Zoom-based courses prove to be poor substitutes for the originals. As early as March 25, when only three states had announced that their schools would remain closed through the end of the school year, John Bailey of the American Enterprise Institute advised educators to prepare for a “series of rolling, targeted school closures over the next 18 months.” Robin Lake and John McLaughlin debated whether Congress should waive protections in federal law for students with disabilities as school districts shift to serving students at a distance. And Paul von Hippel of the University of Texas at Austin assessed how the coronavirus crisis is likely to affect student learning; his conclusion, in a word: “unequally.” Readers can expect similar content on a nearly daily basis in the months ahead.

The articles in this issue of the print journal, in contrast, were all conceived, reported, and filed well before the extent of our current crisis became clear. One of them, however, has special relevance for our present moment. In “The Grade-Level Expectations Trap,” Joel Rose of New Classrooms vividly illustrates how gaps in students’ skills can interfere with their subsequent progress—especially in subjects like math that are “cumulative, with new learning resting atop earlier mastery.”

Such skill gaps are certain to result from the school closures, notwithstanding all the valiant attempts by schools, teachers, and parents at distance learning. When schools reopen physically in the fall—if in fact they do—it will be important to start with a clear-eyed picture of how big those gaps are.

The best tactics and strategies to help students catch up are already a topic of intense debate. The remedy may require, as executive editor Michael Petrilli has urged, that many more students are asked to repeat a grade. Others are proposing to extend the school year into the summer to make up for lost time.

Whatever approach is chosen, the work is urgent. The next pandemic will require scientists to devise cures, epidemiologists to track, project, and slow outbreaks, economists to weigh costs and benefits, ethicists and religious leaders to provide moral guidance, doctors and nurses to heal patients, and politicians to provide leadership. The students in today’s classrooms, both physical and virtual, open and closed, will fill those roles. In that longer-term sense, at least, the tasks of education and education reform, of teaching and learning, are every bit as life-and-death as anything happening in a hospital emergency room or intensive care unit.

— Martin West

Read more from Education Next on coronavirus and Covid-19.

Teenagers are not known for their coolheaded decision-making, yet they face hundreds of choices with significant long-term consequences. In school, they must decide which courses to take, how much effort to invest, and whether and where to enroll in college. Many understandably lack the information and capacity needed to navigate such complex options.

Enter the school counselor. High-school counselors can communicate the benefits of doing well in school, help with college applications, and recommend courses of study to prepare students for the careers of their choice. Belief in a counselor’s potential to boost college success has drawn national attention and inspired policy changes, such as former First Lady Michelle Obama’s “Reach Higher” initiative and the expansion of counselor hiring in Colorado and New York City.

Yet there has been no quantitative evidence to date showing how high-school counselors affect student outcomes—or even confirming that they matter at all. In this study, I aim to close that research gap. I focus on Massachusetts, where a quasi-random assignment process is used to assign many high-school students to counselors based on the student’s last name. I look at the outcomes of each counselor’s students to determine individual effectiveness, as well as at counselors’ experience, educational background, and other characteristics to identify the attributes of those who are most effective.

As with teachers, counselors vary significantly in their effects on student outcomes. For example, improving counselor effectiveness by one standard deviation, which is equivalent to having a counselor at the 84th percentile of effectiveness rather than at the 50th percentile, makes students 2.0 percentage points more likely to graduate high school and 1.7 percentage points more likely to enroll in a four-year college. Unlike teachers, however, counselors’ impacts are most strongly connected to their providing information and assistance to students, rather than building students’ cognitive skills. Their effects are most pronounced among low-achieving and low-income students; low achievers, for example, are 3.4 percentage points more likely to graduate if assigned to an effective counselor. I also find that students benefit from being matched to a counselor of the same race and having a counselor who attended a local college.

High-school counselors’ large caseloads of nearly 250 students, on average, are often a cause for concern. However, my analysis indicates that students would gain more from being assigned to a more effective counselor than to a counselor with a moderately smaller caseload. I estimate that hiring a new counselor in every Massachusetts high school would lead to smaller gains in educational attainment than increasing the average counselor’s effectiveness by one standard deviation.

In short, boosting counselor effectiveness can be an important school-improvement strategy. It may also be a more cost-effective means of improvement than boosting teacher effectiveness, because counselors serve more students, there are far fewer counselors than teachers, and many high-school counselors receive little, or no, training on college advising. Counselors are an often-overlooked engine of educational improvement, and policymakers would do well to devote more time and attention to improving access to effective counselors.

What Do High-School Counselors Do?

Counselors are a common feature at most U.S. high schools, though the nature of the job can vary considerably across schools. In general, high-school counselors spend most of their time on course scheduling, college and career advising, and general student support, according to a 2018 survey of college-admissions counselors nationwide. The responsibilities reported in the survey suggest that there are four main ways in which counselors are likely to influence student outcomes: helping students build cognitive and non-cognitive skills and providing students with information and direct assistance.

Counselors may influence students’ cognitive skills by placing them in, or removing them from, particular classes. Most counselors are responsible for course scheduling, so they may direct students toward or away from effective teachers and advanced classes, as well as help students gain access to specialized services for English-language learners or students with disabilities. They may also work on improving students’ non-cognitive skills, such as behavior and engagement with school, through mental-health counseling, disciplinary actions, and general support.

In addition, counselors may provide information that many students lack about postsecondary education and labor-market options. This could include the costs and benefits of different options, as well as the steps to apply and enroll in college. Counselors can also influence what students do after high school by assisting them directly, such as by obtaining SAT fee waivers, writing letters of recommendation, and helping students complete forms and sign up for services. They also may help students with college or job applications.

In Massachusetts, the focus of my study, there are no regulations dictating counselor caseloads or professional duties, though the state requires counselors to be licensed and hold a master’s degree. It also requires that all schools have a “school adjustment” counselor who primarily supports the mental health, social, and emotional needs of students, which may free up time for guidance counselors to focus more on academic support. Massachusetts provides a recommended counseling model, which consists of suggested guidelines on how to provide services, and has a formal evaluation process for counselors.

A Range of Influence

I use student-level data from the Massachusetts Department of Elementary and Secondary Education, including student demographics, courses, grades, attendance, discipline, and standardized-test scores, as well as human-resources data on counselor employment, education, and demographics. I link these data to National Student Clearinghouse records on postsecondary enrollment and persistence for students projected to graduate high school between 2008 and 2017.

About one-third of public high schools in the state match students to counselors based on the beginning letters of the student’s last name. This process approximates random assignment. I identify 143 schools that post information about their last-name assignment policies on their websites, which yields a sample of 723 counselors serving about 155,000 students. To estimate individual counselor effects, I focus on the 131 schools for which I can link counselors to students in at least two different cohorts with at least 20 students in each cohort, for a sample of 510 counselors serving 142,000 students.

This sample and assignment process enable me to identify the causal impact of individual counselors on student outcomes. In particular, I compare the outcomes of students who attend the same school but who are assigned to different counselors because of the student’s last name. In making these comparisons, I control for students’ 8th-grade test scores, demographic characteristics, and indicators of services received in 8th grade. Consistent with the assignment process being quasi-random, however, I find no evidence that students are sorted to counselors based on these characteristics.

I use this approach to estimate counselors’ effects on a range of student outcomes, which I organize into five domains (see Figure 1). Cognitive and non-cognitive skills correspond to two of the four channels by which counselors may influence student outcomes. College readiness and selectivity, in turn, may capture the other two channels—the provision of information and direct assistance in the college application process. The fifth domain captures counselors’ longer-term effects on educational attainment.

I estimate counselors’ effectiveness within each of these domains, as well as their overall effectiveness across all five. The latter estimate serves as my summary measure of a counselor’s effectiveness in supporting student success.

Counselor Effectiveness and Student Success

Having an effective counselor matters for a wide range of student outcomes. Let’s start with educational attainment, which many see as an important long-term measure of student success. Students assigned to counselors who are one standard deviation more effective than the median are 2 percentage points more likely to graduate high school, 1.7 percentage points more likely to attend a four-year college, and 1.4 percentage points more likely to persist in college into a second year (see Figure 2). The graduation rates of the colleges students choose to attend are also 1.3 percentage points higher, suggesting that they also may be more likely to earn a degree.

These impacts are generally larger for students who are not white, scored below average on the state test in 8th grade, or are from low-income families. For example, a minority student assigned to an effective counselor is 3.2 percentage points more likely to graduate high school and 2.2 percentage points more likely to attend college. Low-achieving students assigned to an effective counselor are 3.4 percentage points more likely to graduate and 2.5 percentage points more likely to attend college. These results indicate that counselors may be an important resource for closing racial and economic gaps in college completion.

How do effective counselors boost educational attainment? It does not appear to be a result of building students’ cognitive skills, as I find little evidence that counselors vary in their effects on 10th-grade test scores and course grades. Counselors do vary in their impact on student suspensions, but I find no significant variation in effects on attendance or unexcused absences—and counselors’ effectiveness in improving these proxies for non-cognitive skills is unrelated to their effectiveness in increasing attainment.

In contrast, counselors have large effects on my measures of college readiness and selectivity. They cause students to be more (or less) likely to take the SAT, to earn a higher SAT score, and take AP tests and also play a major role in whether students enroll in a college that is selective or has a high graduation rate. Moreover, their effectiveness within these two domains is strongly related to their impact on educational attainment.

These results imply that counselors influence educational attainment by doing more than just influencing students’ cognitive and non-cognitive skills. Their effects on educational attainment must run through other channels, such as by providing information or direct assistance to their students. For instance, counselors may have large effects on SAT taking because they provide information about when to take the test or obtain fee waivers for students. More broadly, these results indicate that educators of all kinds can have important impacts on students’ long-term outcomes by providing them information or helping them access opportunities.

Specialization

To better understand counselor effectiveness, I consider the complexity of their responsibilities. They are assigned a diverse set of students and are charged with achieving many results, ranging from setting course schedules to boosting high-school graduation and supporting college enrollment. They are also expected to influence many intermediate outcomes, and it may be difficult for them to attain all desired outcomes given their large caseloads and limited training. Do counselors opt to specialize and focus their energies on certain domains? Or are especially effective counselors better in all aspects of their work? To answer these questions, I estimate counselor effects on specific outcomes and look for commonalities.

In general, the counselors who are effective at improving high-school graduation are also effective at increasing college attendance. This may not be surprising since students must graduate high school in order to attend college. If, however, we expect that the marginal student induced by a counselor to graduate high school is unlikely to be a college attendee, it suggests that counselors who are particularly effective in boosting educational attainment are able to do so for different kinds of students.

In contrast, I find that counselors who improve indicators of non-cognitive skills, such as students’ attendance and behavior while in high school, tend to differ from those who improve educational attainment and, in particular, from those who boost attendance at highly selective colleges. The pattern confirms that even good counselors are typically not good at everything. Some counselors appear to specialize in increasing educational attainment, others at improving non-cognitive skills, and still others at increasing the selectivity of the college a student attends.

Attributes of Effective Counselors

What distinguishes those counselors who are most effective in supporting student success? Advocates seeking to expand access to high-school counseling tend to focus on caseloads, assuming that counselors are more effective when they work with fewer students. I find only a modest relationship, however, between caseload size and students’ educational attainment. Similarly, counselor experience is not related to student outcomes. Instead, students appear to benefit from being matched with a counselor of the same racial group and with a counselor who attended a local college or university.

Students assigned to a same-race counselor—defined here as a white counselor for white students and a non-white counselor for students who are not white—are about two percentage points more likely to graduate high school, attend college, and persist in college compared to their peers who are assigned to a counselor of a different race (see Figure 3). These effects are largest for non-white students, who are 3.8 percentage points more likely to graduate high school and to attend college if matched to a non-white counselor. There is no detectable benefit from matching students to counselors based on their gender.

Minority students may benefit from being matched to a minority counselor if these counselors have a better understanding of students’ experiences and needs. Or a race-matched counselor could have different expectations for students based on their race, a pattern documented among teachers (see “The Power of Teacher Expectations,” research, Winter 2018). Unlike most research on teachers, however, I find that white students also benefit from same-race matches, and white students typically have many potential role models in schools.

These patterns could also be explained by how much students trust their counselor. There is often considerable discretion on both the student and counselor side in how they interact with one another. Students may be more willing to reach out to counselors if they share a salient characteristic such as race.

Counselors’ knowledge of the local higher-education context may also matter for students’ success. Students assigned to counselors who received their bachelor’s degree in Massachusetts are 2.5 percentage points more likely to graduate high school than those assigned to a counselor who earned a degree outside of the state. They are also more likely to attend college and enroll in colleges with higher graduation rates. It may be that these counselors better understand the local college options, the needs of local students, or state graduation requirements than counselors educated elsewhere. However, having a counselor who completed his or her master’s degree in Massachusetts is not associated with higher student educational attainment.

I find no evidence that counselors who attended more selective undergraduate or master’s institutions are more effective than their peers, but there is some evidence that counselors guide students to attend colleges that are similar to those they attended. For example, students with a counselor who attended an elite college are about 2 percentage points more likely to attend an elite college. Counselors who attended a public college also shift attendance to public colleges, and those who attended large undergraduate institutions increase student attendance at large institutions. Thus, counselors may use their own college experiences to guide the recommendations they provide to students.

Most measures of counselor experience, including the number of years they have spent in the role, are not positively related to student outcomes. In addition, counselors who hold teaching licenses are less effective in terms of rates of high-school graduation than their peers without a license. It may be that the skills required to be an effective teacher and an effective counselor differ, or counseling may be a path selected by ineffective teachers when they leave the profession. Either way, school administrators should not consider teaching experience a plus when hiring counselors.

Quantity vs. Quality: The Role of Caseloads

Given the time-intensive nature of advising, one might expect caseload sizes to influence how well counselors serve students. If, however, counselors have found ways to serve many students efficiently, such as with group sessions or by using technology to provide individualized guidance at scale, caseloads may not have large impacts on student success.

Counselor caseloads are difficult to study because they present a chicken-and-the-egg conundrum. Schools in high-income areas with high-achieving students and ample resources typically have the smallest caseloads—indeed, I find that four-year college enrollment rates are highest at schools with smaller caseloads. But is that a function of those counselors’ effectiveness? In fact, when I adjust that figure to take baseline student achievement and demographics into account, the relationship becomes statistically insignificant.

To learn more, I perform several analyses to nail down the causal relationship between caseloads and educational attainment. For example, I study what happens to high-school graduation and college attendance rates within a school when caseloads increase or decrease due to changes in enrollment. I also examine how these outcomes change when a school hires an additional counselor or loses an existing one.

Taken as a whole, these approaches tell a consistent story: larger counselor caseloads are probably bad for educational attainment, but this relationship is quite small. The largest estimate I obtain indicates that increasing caseloads by 100 students per counselor is associated with a decrease of 1.1 percentage points in high-school graduation and a decrease of 1 percentage point in four-year college attendance. This is roughly half as much as increasing counselor effectiveness by one standard deviation. On average, hiring a new counselor in a Massachusetts high school would reduce full caseloads by 74 students.

In short, my results suggest that we should not expect large returns to hiring one additional counselor in each Massachusetts high school. It remains possible that very large swings in caseloads—larger than those I am able to study in Massachusetts—would lead to larger changes in student outcomes. Caseloads may also matter for other outcomes, such as mental health, that I cannot measure with my data. However, to improve educational attainment, increasing counselor quality is probably a more promising approach.

An Overlooked Opportunity

High-school counselors matter—but some matter more than others. The information and assistance an effective counselor provides can have considerable and long-lasting benefits for his or her students, boosting college outcomes years after they graduate high school. Schools and districts can help students do better not just by improving teacher performance, but by supporting more effective counseling, as well.

Moreover, improving access to effective counselors may be a simpler and more cost-effective way to increase educational attainment than improving access to effective teachers. There are far fewer counselors than teachers, so it is probably cheaper, and possibly easier, to deliver training to them. Counselors’ limited (and often nonexistent) training on college advising means that even basic training may have large effects on postsecondary outcomes. And because counselors already work in nearly every U.S. high school, improving their effectiveness may be a more attainable goal than increasing student access to highly personalized (and often expensive) interventions aimed at improving college access.

While advocacy groups often focus on reducing counselor caseloads as a means of boosting their impact, my study suggests that making counselors more effective may be a better goal. That assumes there is a straightforward way to improve effectiveness, which is admittedly a matter for future research. There is one simple and inexpensive way to put these findings to work right away, however: increase the diversity of the counselor workforce, especially in schools serving large numbers of minority students.

In sum, high-school counselors have significant potential to sway the choices and outcomes of the students they serve. Future efforts to improve student behavior, high-school completion, and college enrollment may benefit from leveraging the positions of school counselors and increasing their effectiveness. Efforts to improve school counseling or expand access to the type of guidance provided by the most effective counselors may also have important social and economic benefits. Counselors are and should be considered an important resource for addressing educational inequities and increasing educational attainment.

Christine Mulhern is a doctoral candidate in public policy at Harvard University. This research was supported, in part, by the Institute of Education Sciences, U.S. Department of Education. The opinions expressed are those of the author and do not represent the views of the Institute or the U.S. Department of Education.

Virtue Politics: Soulcraft and Statecraft in Renaissance Italy
by James Hankins
Harvard University Press, 2019,$45; 768 pages.

As reviewed by Ian Lindquist

Education of the young presents the most important public policy challenge that any polity faces. This is not to say it is the most pressing—that designation would belong to the task of national defense, without which education could not occur. But if a polity wishes to last more than one generation, it must give considerable thought and attention to the way it forms its young people—to the skills, the knowledge, and, most importantly, the kind of character that children will need later, when they grow up and inherit the task of governance.

Education in this sense is not limited to the classroom. It occurs throughout society and at every level of a child’s experience, encompassing manners and morals taught in families and neighborhoods and communicated both explicitly and implicitly; values learned through participation in groups and organizations, from churches to fraternities and sororities to sports leagues; as well as education in the narrow sense of formal instruction and “classroom learning.” Political stability requires attention to how a society shapes its young people. When corruption or degeneration reigns in political life, it’s a safe bet that the task of forming the young has been neglected for some time.

Harvard historian James Hankins tells the story of one such age in his wide-ranging and magisterial Virtue Politics: Soulcraft and Statecraft in Renaissance Italy. The Renaissance humanists set out first to rectify the political disorder surrounding them in Italy. Quickly, however, the humanists learned that political rejuvenation would require educational rejuvenation. Because political life in the 13th and 14th centuries suffered from tyranny and corruption, the humanists believed that rectifying political life required the teaching of classical virtue, which would be passed on to the young—and, crucially, to those who would hold positions of political power—through a new educational program dubbed the studia humanitatis, or humane studies.

The scope of this task was wildly ambitious. The humanists sought to overturn the education offered by the scholastics of the High Middle Ages, which, in the humanists’ view, focused inordinately on technical legal education and a host of specialties that crowded out the traditional liberal arts. In politics and education, the humanists sought to do away with the scholastic focus on law, insisting that legitimacy of rule derived from “true nobility”—that is, virtuous character—rather than laws. The letter of the law is only as good as the character of those who enforce it and so the humanists placed education on a foundation of character formation.

To cast off the prevailing manner of education, the humanists looked to the age of ancient Rome and Greece. Not only was it the time when Italy’s greatness reached its height, but it was also an era when men practiced classical virtue. Access to classical virtue had been lost in the period between Rome’s decline and fall and the 14th century. The humanists set out to make Italy great again by emphasizing the practice of classical virtue and especially the reform of the character of those who governed. This shift required not simply a new manner of  education—classroom learning—but also a new institutio or, as Hankins puts it, “a fundamental reconstruction of the forms of culture used to educate and form its citizens.”

Hankins names this ambitious humanist project “virtue politics,” because it heavily emphasized political reform through what we would today call character education. Virtue politics, he writes, “focuses on improving the character and wisdom of the ruling class with a view to bringing about a happy and flourishing commonwealth.” This may seem like a reasonable enough goal, but in fact it constituted a radical approach to political reform: “The humanists saw politics, fundamentally, as soulcraft. Their overriding goal was to uproot tyranny from the soul of the ruler, whether the ruler was one, few, or many, and to inspire citizens to serve the republic.” Tyranny would not end unless the souls of rulers were rightly ordered.

The ambition of Virtue Politics doesn’t quite match that of the Renaissance humanists’ efforts to reform education and politics in Europe, but it doesn’t lag far behind. Hankins tries to interpret the entirety of the Renaissance. He does so by bringing an encyclopedic knowledge of Renaissance humanism to the table, and he includes both discussions of comparative literature and chapters on individual Renaissance writers, many of whom are not studied today except by specialists. The Renaissance and its treasures have much to teach.

Most important, they can teach us a lot about education. Renaissance education reform is in many respects the ancestor of a current education-reform movement in American that has been growing since the early 1980s. Like the Renaissance reformers, classical educators today introduce their students to classical virtue and classical authors. While there are internecine arguments about the ultimate grounds for and goal of classical education, practitioners by and large agree that liberal education—that is, an education based on broad reading and intellectual study that instills moral and intellectual virtue—is the right education in that it allows human beings the best opportunity to flourish. Today’s classical educators do not view their approach as an exercise in antiquarianism but rather as necessary for a life fully lived.

Similarly, Hankins does not view the Renaissance humanists as archaic museum pieces but as thinkers and political actors from whom today’s political and education reformers can learn. Three things stand out that bear consideration.

First, legal and policy fixes alone cannot achieve healthy political life. A polity’s health depends on its ability to cultivate virtue in the young, which is the true task of education. Education involves the formation of character—the ability to live a good life—and character is cultivated through the study of moral philosophy and history—“precept and example.”

Second, the formation of character includes and thrives on examples of virtue from the past. Especially in politically corrupt times, it is important to turn the gaze of the young toward examples of virtue. To those frustrated by the examples set by politicians today, the humanists would recommend putting a copy of Plutarch’s Lives in front of students and asking them to debate and discuss who is the most virtuous Greek or Roman. The past offers a treasury that helps to spur the young to excellent character. An education that neglects to bring students into communication with these riches is shrinking, rather than expanding, their worlds.

Third, the goal of liberal education is to allow human beings to become as fully human as possible. For the humanists, liberal education was “a training in humanity in the full sense of that word, the most effective form of soulcraft.” A liberal education provides the path to virtue, the most excellent exercise of the abilities human beings possess. Indeed, “the humanities were a prophylactic against the beast in human nature and a reminder of the goodness of which it was capable.”

Hankins is not optimistic that contemporary American society and culture will experience a return to the ideals of Renaissance humanism, and its commitment to character education, any time soon. But as his history teaches, a commitment to the formation of character often finds itself the odd man out in any given age. Even for the humanists, character education amounted to a counterweight to the excesses prevalent in souls and states in their age. That counterweight was provided partly through historical study and the attempt to breathe new life into texts and ideals, not for the sake of mere antiquarian interest, but rather to solve the challenges of the day.

If a greater focus on character education based on classical virtue does come about in our own time, it’s reasonable to think that this shift will take its bearings and example from the past. Hankins’s history of the Renaissance humanists offers a useful starting place for discovering what that kind of cultural rejuvenation might look like.

Ian Lindquist is a fellow at the Ethics and Public Policy Center in Washington, D.C.

Red State Blues: How the Conservative Revolution Stalled in the States
by Matt Grossman
Cambridge University Press, 2019, $24.99; 204 pages.

As reviewed by Andy Smarick

In the Trump era, it’s clear that much of the American right has lost track of the governing principles that once animated it. The president’s and some conservatives’ embrace of tariffs, industrial planning, “I-alone-can-fix-it” centralization, and imprudent language show that foundational beliefs related to free markets, limited government, decentralized authority, and stolid temperament simply don’t shape the thinking of leaders on the right as they once did—or as I believe they should.

Trump didn’t cause this problem. He just revealed it. By the time of his campaign, the right had spent down the intellectual capital it had built up during the heyday of conservative thinking from the 1950s through the 1980s. In recent years, few fresh approaches for applying venerable conservative tenets to contemporary social-political life have emerged. Rather, conservatives have lazily applied old interventions to new situations, as though particular policies were synonymous with conservatism. Instead of reasoning from conservative principles, activists and policymakers on the right have behaved as though these policies served as proxies for conservative thought. No matter the issue being addressed, conservatives reflexively proposed cutting taxes, eliminating cabinet agencies, scaling back regulations, and so on.

This phenomenon is manifest in the right’s approach to K–12 policy. We wrung every drop out of school choice and accountability, applying them vigorously and often indiscriminately for a couple decades. We dedicated little thought to conservatism’s first principles, neglecting to explore how concepts like pluralism, localism, incrementalism, tradition, and virtue could energize a new agenda.

As a result, conservatism has come across as exhausted and out-of-touch. It has offered little that is new, and too few ideas that are responsive to contemporary society. Conservatism has also become easy to caricature as a set of stodgy proposals instead of a robust, principled, flexible approach to governing.

Matt Grossman, a political scientist at Michigan State University, puts all of these failings on display in his serious, solid new book Red State Blues: How the Conservative Revolution Stalled in the States. Grossman uses an array of analyses to make the case that the Republican Party’s dominance at the state level in recent years has fallen short in producing new policy and social outcomes.

Grossman begins by detailing the truly remarkable ascendance of the GOP since 1990: capturing over a thousand state legislative seats and two dozen governorships, taking full control of the political branches in as many states. Governing Republicans have grown more conservative while maintaining their control of elected branches. Nevertheless, the author concludes, Republicans haven’t succeeded in reducing the size and scope of government, reversing liberal policies, or enacting a broad conservative agenda. And their longer-range influence on social and economic life has been limited. Grossman does find, however, that Republican states have slowed liberal policy gains.

Part of the explanation for this lack of success is that, worldwide, governments grow over time—a phenomenon the author refers to as “Leviathan’s Resilience”—for a number of reasons: modernization creates new social problems that invite government action, wars ratchet up state power, bureaucracies have insatiable appetites, and economic growth produces new taxable products and services. Moreover, policy seems to move generally leftward over time, so conservatives are constantly swimming against the current. The GOP may have made small advances on restraining abortion, protecting gun rights, and enacting some tax- and tort-reform policies, but on the whole, liberal progress continues. In cases where the GOP moves from no control to full control of state governments, Republicans have been able to check liberal policymaking. But the basic characteristics of states, such as their partisan and ideological leanings, seem to have substantially more influence than the party in power at the state capital, further inhibiting the policy influence of the Republican Party’s electoral success.

As examples of conservatism’s underwhelming influence, Grossman cites the successful teacher protests of 2018 that led to significant increases in school spending. Similarly, he finds that, from 2012 to 2017, states collectively increased early-childhood funding by nearly 50 percent. Grossman also cites a package of proposals advocated by the right-leaning American Legislative Exchange Council. Though the ideas caught steam with conservative leaders, many of these policies—such as teacher-evaluation reform and charter schools—also commanded the support of left-leaning philanthropists and the Obama administration. So  while school-choice programs might have expanded during this era, education overall appears to track the larger trend Grossman identifies:
that the GOP hasn’t substantially driven policy rightward.

The book’s great strength lies in its creative efforts to prove this thesis. Grossman marshals substantial bodies of research and assembles new sources, including interviews with statehouse reporters. He presents graphs of state spending; analyses of state statutes in terms of their conservative or liberal orientation; evaluations of partial and total partisan control of elected branches; and reviews of the effects of extended partisan control and lasting changes in public opinion.

All of this proves convincing, if you accept the book’s understanding of what constitutes conservatism.

Conservatism, however, is not a collection of policies; it is an interrelated set of dispositions and beliefs. For instance, conservatism does not mandate across-the-board cuts in government; rather, it supports the view that centralized, government-controlled decision making overestimates the merits of technocracy, undermines community pluralism and individual agency, and thwarts civil-society activity. It does not dictate a slate of policies on abortion, marriage, prostitution, and drug use; rather, it embraces the view, arising from empirical experience, that stable communities require stable families. That perspective, in turn, prompts conservatives to attach moral opprobrium to certain behaviors and moral sanction to others.

For example, it is not necessarily the case, as the book implies, that smaller government is always consistent with conservatism. The concept of “subsidiarity”—found in Catholic social teaching and widely embraced on the right—seeks to distribute authority and assign responsibility across individuals, families, community-based groups, and different levels of government. This idea could support more state-government activity in certain domains if it meant reduced federal activity or if that state-level activity was designed to catalyze local-government or civil-society action. Likewise, conservatives who prioritize family formation and social stability regularly support the active use of state-level “police powers” to protect the safety, morals, and health of communities. Some conservatives today advocate for increased infrastructure spending and subsidies to support manufacturing as ways of strengthening struggling communities and families.

Reducing principles into policies can cloud our understanding by creating false positives and negatives—identifying some things as conservative that are not entirely so and failing to identify as conservative some that arguably are. Labeling charters and private-school choice as “conservative” masks the deeply conservative reasons to oppose them—for instance, that they undermine both a sense of community cohesion and the authority of local school boards, which are arguably the embodiment of longstanding, small-scale mediating institutions. Similarly, one could view a position of support for the teacher protests as the proper conservative response: conservatism generally respects the autonomy and wisdom of local practitioners, which can act as a bulwark against centralization and technocratic grand plans. Indeed, many conservative state legislators oppose the expansion of school choice and loyally support their local school districts and educators.

Likewise, numerous conservative actions at the state level might not be recognized as such. For example, subtle resistance to the No Child Left Behind Act and Race to the Top reflects skepticism of Washington’s ostensibly expert opinions. Opposing or proceeding with caution on such policies as Common Core, standardized assessments, test-based evaluations of teachers, and novel policies on discipline and online learning reflect conservatism’s bent for incrementalism and the tried and true.

The book makes several passing references to the ability of conservatives to slow progressive advancements, but these instances are generally framed as the right’s eking out a minor moral victory in a larger loss: that is, the best the right can hope for is to stem the tide of progressivism. Similarly, in Grossman’s interviews with journalists he finds that GOP leaders give the impression of doing less than Democratic leaders. Republican leaders don’t move quickly or introduce as many new proposals; they allow other states to go first, and they don’t seek big changes. Democrats advocate for more new programs and have bigger across-the-board agendas.

But this assessment is music to a true conservative’s ears. Conservatives don’t aim to mechanically defend the status quo. As Edmund Burke wrote, “A State without the means of some change is without the means of its conservation.” What conservatives generally oppose is swift, centralized, uniform, pseudo-scientific change—change conjured up in theory and not supported by custom, longstanding institutions, or natural rights. Conservatives champion change that is organic, deliberate, pressure-tested, and consistent with an understanding of human nature based on experience. As such, conservatives understand that traditions can evolve, that markets can disrupt, that technology will advance, and that sensibilities will change. For the conservative, the process of change is essential. That the right has been a stabilizing, moderating force in an era of profound social, cultural, and economic change is not a bug of conservatism to be regretted; it is a feature to be celebrated. In a beautiful paragraph in the book’s final pages, Grossman acknowledges that conservatism is built on an appreciation of tradition, existing arrangements, voluntary action, and decentralizing devices as well as skepticism about swift, certain change. I wish that understanding had been woven throughout the book’s analysis. In my view, conservatism has played a larger role in state-level governing than the book argues.

Conservatives  deserve a good bit of the blame for the reduction of principles into policy, especially when it comes to education. The conservative agenda has been so static and so stale for so long that a reasonable observer could conclude that a certain collection of proposals is synonymous with the philosophy.

Since I began cutting my teeth in right-of-center education circles more than 20 years ago, conservatives have been pushing the same set of policies: new schooling options for kids, stringent rating systems for schools and districts, stronger content standards and tests, accountability for educators and their professional training programs. Not that there’s anything wrong with these policies—just as there’s nothing wrong with DVDs, flip phones, and iPods.

Conservatives need to stop skating by on old ideas and old proposals. The path to rejuvenation starts with a return to conservative principles. We need to reacquaint ourselves with the fundamentals—tradition, localism, liberty, community, markets, civil society, prudence. Then we need to identify the biggest challenges of the day—major changes in social and economic life; too much alienation and polarization; too little opportunity, upward mobility, and civic virtue. Then we need to bring them together, showing how timeless principles can inform the response to contemporary matters.

Red State Blues is a valuable contribution to our  understanding of state-level politics and policy in recent decades. I would have preferred a greater emphasis on exploring the nature of conservatism and the variety of ways it can manifest itself in state action. Unfortunately, modern social science seems to abjure such discussion in favor of phenomena that can be named, quantified, measured, and analyzed empirically—hence the book’s focus on specific policies instead of the principles that underlie them.

I wish I had a similar explanation for why conservative governing officials and thought leaders have taken the same limited approach for the last generation.

Andy Smarick is the director of civil society, education, and work at the R Street Institute. Formerly, he served as deputy commissioner of education in New Jersey and as president of the Maryland state board of education.

High-school exit exams have fallen out of favor in recent years, after research showed that pinning graduation to passing a high-stakes test can push some students to drop out. The future can be grim without a diploma: dropouts have higher rates of unemployment, earn far less money on the job, have poorer physical and mental health, and are more likely to be incarcerated. Wary of such risks, 18 states have dropped exit exams from their diploma requirements in the past two decades. This year, graduation requirements in just 11 states include an exit exam.

Beyond high-school dropout and graduation rates, exit exams might have other effects on students and communities. Little is known, for example, about the specific effects of exit exams on crime. Conventional belief holds that more and better-quality education reduces crime. Could exit exams improve teaching and learning in high schools such that criminal activity drops?

I looked at the arrest rates of jurisdictions and compared them during periods before and after graduation requirements were made more demanding in two ways: adding high-school exit exams and increasing the amount of academic coursework. I find that requiring exit exams decreases arrests by approximately 7 percent, primarily from a decrease in property crimes. Increasing course requirements, however, has no significant effects. My analysis supports earlier research regarding the beneficial effects of exit-exam use and is also one of the few studies to show that specific education policies can have crime-reducing effects.

Many states have temporarily suspended standardized testing, including exit exams, as part of ongoing school closures due to the Covid-19 pandemic. As decision-makers consider reinstating tests, these findings may be of particular relevance. While exit exams have come under fire for pushing students at the margins out of high school, my analysis indicates that they have more broadly positive effects on communities than previously understood.

The Rise and Fall of Exit Exams

For decades, educators and policymakers have been trying to make high school more rigorous, including by adding required coursework and introducing exit exams. The moves came after dire warnings that American public schools were failing to prepare students for college or the workplace, because classes were out of date and standards didn’t line up with employers’ expectations.

In 1983, A Nation at Risk, a report commissioned by the U.S. Department of Education, warned that the education system in the United States was failing to prepare students for the future. The report provided several suggestions for improvements, including all students taking four years of English, three years each of mathematics, science, and social studies, and one half-year of computer science. In response, many states increased their course requirements, resulting in a string of reforms from 1983 to 1986. A second wave of curriculum changes occurred in the mid- to late-1990s, as the importance of “rigor” took root in the nation’s statehouses.

During those decades, states also introduced new exit exams, or standardized tests that students must pass to receive their diplomas, inspired in part by the European model (see “A Steeper, Better Road to Graduation,” feature, Winter 2001). Across states, the new tests varied in difficulty, the subjects covered, and number of attempts allowed. In most cases, they covered material that students should have mastered between the 8th and 10th grades. When the 2001 No Child Left Behind Act required states give high-school students at least one exam in math and reading, many chose to make this a graduation requirement, and the number of exit exams grew rapidly, reaching more than half of all U.S. states (see Figure 1).

In theory, the presence of an exit exam focuses instruction on what the state deems important and guarantees that each graduate leaves high school with at least that minimum knowledge. And some research has shown positive effects: exit exams improve wages, employment, and college attendance. However, research also has shown that exit exams may prevent marginal students from graduating. And in general, high-stakes tests can create incentives for schools to prioritize tested materials and subjects—otherwise known as “teaching to the test.”

The tests have always caused controversy, as districts face the prospect of withholding diplomas from students who completed all other graduation requirements. That controversy grew more intense in this decade amid a general backlash against standardized testing in American schools. Many states introduced alternative assessments and other reviews that allowed students who did not pass exit exams to receive their diplomas. Others moved to strip exams of their status as a graduation requirement altogether. The number of states that require exit exams has consistently declined in recent years.

While much has been written about the educational effects of changing high-school graduation requirements, the literature is sparse when it comes to its effects on crime. There is a body of research detailing the connection between the highest level of education completed and a person’s subsequent rate of criminality—for example, a 2009 report found that nearly one in 10 males under the age of 25 without a high-school diploma was incarcerated on any given day, compared to one in 33 of their more educated young male peers. In addition, adults without a high-school diploma are nearly twice as likely to be unemployed as adult Americans overall, which a large body of research has shown is associated with higher rates of crime. A 2013 study by Olesya Baker and Kevin Lang examined the impact of exit exams on incarceration based on Census data, finding positive but mostly insignificant effects.

I pursue a more detailed investigation using yearly arrest rates, which can reveal the immediate effects of an exit-exam requirement, rather than later-life incarceration rates. These are important differences, as effects could fade over time or differ based on age and not all arrests lead to incarcerations long enough to be observed in Census data. My findings invite us to ask new questions about the value of exit exams.

Data and Method

I look at annual arrest data from the Federal Bureau of Investigation’s Uniform Crime Reporting Program, which is the primary source for crime statistics in the United States. Program statistics are voluntarily reported by nearly 18,000 police departments nationwide, including cities, universities and colleges, counties, states, tribal lands, and federal law-enforcement agencies. In all, my sample includes 14,888 unique police agencies across 48 states (Alaska and Hawaii are excluded). I limit my analysis to the annual arrest counts for men and women age 15 to 24 from 1980 to 2010, which results in a total of 2,627,438 observations.

The data are reported as aggregate counts of arrests by age, gender, and offense at each agency, by year. I make two assumptions: that the state where an arrest occurred is the state where the person attended high school and that their graduation year would have been the year in which they turned 18. The data include the number of arrests but not the total number of crimes or offenses committed.

While the arrest data are given as the total number of arrests, a more informative measure is the arrest rate, which accounts for population size. Since agencies are not a commonly measured geographical area, the data contain an estimate of the total population within the jurisdiction of each police agency. Unfortunately, these population estimates are not age-specific, so I use federal data from the National Institutes of Health to estimate age- and gender-specific population estimates in each agency’s jurisdiction.

I consulted a variety of sources to determine state requirements. For example, the minimum number of classes a graduation cohort would need to pass to earn a diploma is based on reports by the Education Commission of the States and National Center for Education Statistics. Exit-exam information is based on infor­mation published by the Center for Education Policy, among other sources.

To estimate the effects of expanding coursework require­ments and adding exit exams, I compare the arrest rates of graduation cohorts who were educated in the same state at different times. For example, consider a state that introduced an exit exam for students expected to graduate in 2005. When examining the 2008 arrest data for jurisdictions in that state, I compare the arrest rates of young people age 21 and under, who were subject to the exit exam requirement, to those age 22 to 24, who were not. By focusing on the arrest rates of different graduation cohorts in the same year, I eliminate the influence of other factors that vary over time and are known to influence arrest rates, such as the number of police officers a jurisdiction employs.

When making these comparisons, I control for the economic conditions in the jurisdiction when each graduation cohort was in high school. I also control for several characteristics of each state’s school system when the cohorts were in high school, such as the average pupil-to-teacher ratio, teacher salary, and per-pupil expenditures. I assume that, after making these adjustments, the only major difference between students from different graduation cohorts is that one group faced tougher graduation requirements.

Effects of Exit Exams

When states expand the coursework required to earn a diploma, there is no significant effect on arrests. But when they mandate high-school exit exams, arrest rates fall.

Students who face an exit exam have 2.2 fewer arrests per 1,000 individuals than those without any exit exams—approximately a 7 percent reduction from the local average (see Figure 2). This effect may be initially surprising given the increased dropout rates associated with exit exams found in prior research. However, exit exams may have far-reaching effects beyond those for the marginal students who are induced to drop out. For example, exams may help refocus the curriculum, build noncognitive skills, and potentially affect the labor-market value of a diploma.

I find different effects on arrests based on the types of alleged offenses: there is an 8 percent reduction in arrests for property crimes and a 5 percent reduction in arrests for violent crime (see Figure 3). In looking at coursework requirements, I find no strong effects on any offense type.

These results are in line with what we might suppose are the theoretical effects of exit exams, as well as previous research. If much of the effect is driven through an increase in income due to higher wages or likelihood of employment, for example, then it is reasonable to believe that crimes with monetary gains such as burglary, larceny, robbery, and auto theft would be most affected. If there are noncognitive mechanisms at work, then one might also expect changes in non-monetary crimes like murder, rape, or assault—which I find in these results. While it may be surprising to see effects on violent crime, prior research by Lance Lochner and Enrico Moretti found statistically significant reductions in violent crime due to increased education.

Impacts by Race and Income

I then look at which groups and community types are most affected by the presence of a high-school exit exam. Prior research on graduation requirements and academic outcomes has found a negative impact for African American and low-income students. Studies by Thomas Dee and Brian Jacob and by Steven Hemelt and Dave Marcotte found black students are more likely to drop out after schools require exit exams, for example, and a study by John Papay, Richard Murnane, and John Willet found exit exams made it more likely that low-income students would not graduate. However, other curriculum reforms are associated with positive effects for these groups; for example, Joshua Goodman found that in states where more math classes were required, African American students earned higher wages in adulthood.

For income, I find that for both types of graduation requirements (that is, the presence of exit exams and required courses for graduation), counties with the lowest average incomes appear to see the largest crime reductions. In particular, results for exit exams are large and statistically significant for the poorest counties, while results for counties in all other income quartiles are not significantly different from zero or each other.

The patterns with respect to the racial composition are quite different: Exit exams exhibit significant and large crime-reducing effects for the two quartiles of counties with the highest percentage of white residents, while the effects for the bottom two quartiles are smaller and statistically insignificant.

I also find that both rural and urban counties exhibit large crime reductions due to exit exam requirements, with no statistically significant effects in counties that are within large metro areas but suburban in character.

These results provide some insight into how these effects might occur. If exit exams are helpful in that they cause more advantaged students to increase their skills and knowledge and therefore commit fewer offenses, while simultaneously causing more vulnerable students to drop out of school in greater numbers and commit more offenses, we would expect to see less encouraging findings among populations where the dropout effect is largest. Based on the prior literature, that would be among non-white, low-income, and urban students, which is largely what my analysis finds. One exception to this explanation is the finding that the largest reductions in crime occur in the poorest counties. However, a potential reason for this finding is that those counties have the most crime before the reforms take effect. Perhaps other counties do not have as much crime to begin with, so it is more difficult to see large arrest-reducing effects in those cases.

From a policy perspective, it’s important to note that these results show no significant increases in arrests for any of the subgroups examined and are largest in low-income counties, which are often those most in need of crime reduction. However, the effects differ in counties whose populations are predominately white or predominately minority; arrest rates fall more in largely white counties than in counties with larger proportions of minorities. This suggests that these policies may widen already large gaps in arrest rates between white and minority residents, which could increase inequality in other dimensions.

A Complex Calculation

Taxpayers and voters often tout slogans such as “build schools, not prisons” based on the common belief that education can deter crime. Indeed, my analysis shows that specific education policies, such as requiring high-school students to pass an exit exam to graduate, lead to lower rates of arrest.

The question is, how? I see a few possibilities. The increased accountability of an exit exam could motivate schools and students to increase learning, or the effort needed to pass the exam could also support students’ developing better noncognitive skills. It also could boost the perceived value of a high-school diploma. Or, students could choose to attend school more often, because they know they will need to prepare for and pass the exam in order to graduate.

Yet exit exams have been subject to widespread skepticism in the public discourse, including criticisms that they are punitive and exclude too many low-income students of color from earning their diplomas. The findings of this analysis provide evidence in favor of their use. The accountability and rigor that exit exams impose appear to have farther-reaching benefits throughout communities than previously assumed.

It is important to realize that these benefits are an average effect, however, and that exit exams may not be helpful to every student. Indeed, some students on the margin of graduating could be forced to drop out. In addition, I find evidence that exit exams may be more beneficial for white students than minority students, which may work to increase already large racial gaps in arrests. Though there is also evidence that the effects benefit lowest income counties the most, which could help the areas most in need of crime reduction.

While the explicit goal of educational reforms is not to reduce crime, it is certainly a positive result. Public education is intended to prepare students with skills and knowledge they need to participate fully in their adult lives, including working, voting, and maintaining healthy communities. Surely decreased rates of arrest among young people are an important component of that overall goal. Raising the rigor of a high-school education, so long as students who may be left behind are given the support they need to succeed, can promote public safety as well as purposeful adult lives.

Matthew F. Larsen is an assistant professor at Lafayette University.