In 2002, years before the current fervor over personalized learning, the state of Florida embraced a primitive form of the concept. Under the leadership of then governor Jeb Bush, the state decided that 3rd graders who did not demonstrate basic reading proficiency on state tests should be held back and receive intensive remediation. Before advancing to 4th grade, where they would increasingly be asked to “read to learn,” they would need to show that they had learned to read.

Florida’s test-based promotion policy is part of a broader effort to improve early reading instruction, which has included state-funded reading coaches, evidence-based professional development, and mandatory blocks of time dedicated to reading in all elementary schools. The policy is also less draconian than it may sound. Most English language learners and students with disabilities are exempt, and other low-performing students are permitted to demonstrate reading proficiency through a portfolio of work. As a result, fewer than half of students not meeting the promotion standard are actually retained.

Some 16 states have now enacted versions of Florida’s policy, yet controversy continues to swirl around it. Proponents of test-based promotion argue that the threat of retention provides a powerful incentive for educators to help students become strong readers by 3rd grade, and that students who fall short could stand to benefit from an additional year of schooling. Critics, meanwhile, warn that retained students may suffer, due to stigma, reduced expectations, and the challenges of adjusting to a new peer group.

At least in Florida, the proponents’ argument at first seemed to hold sway. As Marcus Winters and Jay Greene reported in our pages, the first students who were retained saw dramatic performance gains in both reading and math, as compared to low-performing students in prior cohorts who had moved to 4th grade with their age peers (see “Getting Ahead by Staying Behind,” research, Spring 2006). Florida students’ performance on state reading tests and the National Assessment of Educational Progress rose rapidly in the years following the policy’s introduction (see “Florida Defeats the Skeptics,” check the facts, Fall 2012).

But skeptics warned that the academic gains made by retained students would diminish over time and that they would ultimately be less likely to complete high school: nationwide, students who are unusually old for their grade are far more likely to drop out. Would retained students in Florida suffer that fate?

Now, some 13 years later, it is possible to examine that concern. In a new study in the Journal of Public Economics, two colleagues and I report on the long-term outcomes of the first six cohorts of 3rd graders retained under the policy, two of which we can track through high school.

We find that the gains made by retained students did diminish over time, as critics predicted, but these students still entered high school performing at higher levels in both reading and math than similar peers who were promoted on time. The retained students needed less remedial course work in high school and earned higher grades overall. But they did not complete more credits and were no more (or less) likely to graduate from high school or enroll in college.

In short, the results are mixed, casting doubt on the worst fears of retention’s critics but not fully vindicating the Florida policy—especially given the cost of providing an additional year of instruction to retained students. Our study will not end the debate over test-based promotion, but we do hope it will prompt policymakers to take the next step: identifying and implementing changes in high schools that will help translate students’ better preparation into better results.

Since 2002, the tools available to policymakers and educators seeking to tailor instruction to students’ needs, interests, and abilities have multiplied. Some efforts to personalize learning turn the Florida model on its head, enabling students to move ahead with their grade cohort while still working to master specific concepts and skills. Others, like the Summit Learning Program model, combine competency-based progression through core academic content with an emphasis on projects designed to foster student agency (see “Pacesetter in Personalized Learning,” features). One result, as Michael Horn notes in this issue, is widespread confusion about how to define personalized learning (see “Now Trending: Personalized Learning,” what next).

Will the current generation of personalized learning strategies, in all its variety, find clearer success than the first? If the Florida experience is any indication, it may be more than a decade before we have solid evidence—and even then there will probably be room to debate what it means.

— Martin R. West

Laptop computers have become commonplace in K–12 and college classrooms. With that, educators now face a critical decision. Should they embrace computers and put technology at the center of their instruction? Should they allow students to decide for themselves whether to use computers during class? Or should they ban screens altogether and embrace an unplugged approach?

The right way forward is unclear, especially at colleges that pride themselves on connectivity. The vast majority of students carry laptops or tablets from class to class to take notes, consult references, collaborate with professors and classmates—and to update social-media sites, order takeout, and watch YouTube videos during lectures. The personal computer is a powerful tool. It can efficiently store and enhance student work; it can also effectively transport a student’s attention away from that work.

Not surprisingly, some professors have banned computers from class. But research shows many remain conflicted about their value: in a 2014 survey by Richard Patterson  and Robert Patterson of 90 professors at a liberal-arts school, 57 percent agreed that laptops enhanced learning, but 42 percent thought laptops decreased participation. Two-thirds of professors in a slightly larger survey from the same school had laptop-optional policies, and one in five required them for class.

Although students overwhelmingly like to use their devices, a growing research base finds little evidence of positive effects and plenty of indications of potential harm. To determine the impact of laptop usage on student performance, we conducted a randomized controlled trial among undergraduate students at the United States Military Academy, widely known by the name of its location in West Point, New York. In the study, we designated who was allowed to use and who was prohibited from using laptops or tablets to take notes in class.

We find that allowing any computer usage in the classroom—even with strict limitations—reduces students’ average final-exam performance by roughly one-fifth of a standard deviation. This effect is as large as the average difference in exam scores for two students whose cumulative GPAs at the start of the semester differ by 0.17 grade points on a standard 0–4.0 scale. Importantly, these results are from a highly competitive institution where student grades directly influence employment opportunities at graduation—in other words, a school where the incentives to pay attention in class are especially high.

We believe our findings raise important questions for colleges and college students about the impact of using Internet-enabled devices during class and may have implications for K–12 educators as well.

An Experiment at West Point

The United States Military Academy is a four-year undergraduate institution with an enrollment of approximately 4,400 students. West Point’s student body is unique, due primarily to the institution’s mission of generating military officers and attendant admissions requirements, including a recommendation from a local congressional representative. Students receive the equivalent of a “full-ride” scholarship; however, upon graduation, they become commissioned officers in the U.S. Army and incur an eight-year service obligation with a five-year active-duty requirement. Comparing the student population at West Point with that at other four-year institutions reveals broad similarities, aside from a major difference in the proportion of female students. At West Point, only 17 percent of students are female compared to more than 50 percent of students at other four-year schools nationwide, on average (see Figure 1).

West Point provides an ideal environment for conducting a randomized controlled classroom experiment about Internet-connected computer usage for a number of reasons. First, as part of their “core” curriculum, students are required to take several classes in sequence, resulting in high enrollment numbers. We chose to focus our study on one of these classes: Principles of Economics. Some 450 sophomores enroll in the class each semester, but individual section (or classroom) sizes are low due to an institutional commitment that caps the faculty-to-student ratio at 1:18 per class. Class sizes in our study were typically around 15 students. West Point professors also do not have teaching assistants, so all grading and interaction is done between the student and the professor. Additionally, all students are required to attend class unless they have an excused absence, so we were not concerned that attendance is affected by class-level technology policies.

Second, despite the large enrollment and small class size, student assessment in Principles of Economics is highly standardized. All classes use the same syllabus and students complete the same homework and tests. This allows us to compare grades between classes.

Third, within a given time slot, students are randomly assigned to their particular class. West Point centrally generates student academic schedules, and students cannot request a specific professor. Most importantly, prior to the first day of class, students are unaware of the computer policy of a particular class, and there is virtually no switching after the first day.

Fourth, all students at West Point are on equal footing in terms of access to educational resources: all students must purchase the same laptop computers and iPad tablets, and all academic buildings have wireless Internet access. Students also complete an introductory computer science class their freshman year prior to taking Principles of Economics.

Further, West Point uses class rankings to assign each student to a military occupation and a specific military base following graduation. A student, therefore, is especially motivated to have a high GPA so that he or she can have a better chance of receiving a preferred occupation or location.

Finally, classes are well-structured: a student who falls asleep in class, arrives late, or is otherwise disruptive may be reported to the military officer who is in charge of—among other things—disciplining the student. Cell phones are not permitted in any class, making laptops and iPads the most common Internet-connecting devices available to students. In a setting where students were less motivated or there was less discipline, we might expect any distracting aspects of technology to be even more pronounced.

Sample and Design

We conducted our experiment during the spring semester of the 2014–15 academic year and the fall semester of the 2015–16 academic year. Each term, we randomly assigned participating sections of the Principles of Economics course into one of three groups. The first group was “technology-free,” with students barred from using laptops or tablets at their desks.

The second group was intended to replicate the typical collegiate classroom environment, with students using Internet-enabled technology at will during lecture and discussion. In those classrooms, students were permitted to use laptops and tablets in the class. Ideally, students would use them for note-taking or referencing material, such as the “e-text” version of the textbook, although professors had limited ability to monitor every student’s computer. Professors did have discretion to stop a student from using a computing device if the student was blatantly distracted from the class discussion.

The third group allowed technology, with restrictions. This “tablet-only” group was designed to replicate the intended use of Internet-enabled technology as a non-distracting resource during class. In those sections, laptops were not permitted, but students could use iPad tablet computers so long as they remained flat, with the screen facing up and parallel to the desk surface. This modified tablet usage enabled students to take notes on the tablet or access their e-text or other class materials while allowing professors to observe and address student use of distracting applications. We cannot, however, be sure that students only used their tablets for class-specific purposes. For example, it is possible that instructors did not observe their students using iPad applications such as iMessage or other communication tools or games. Roughly 80 percent of students in classrooms that permitted laptops and tablets without restriction used an Internet-connected device during class, but only 40 percent of students in “tablet-only” classrooms used a device.

We randomly assigned sections to one of the three groups in a way that ensured each professor taught at least one section in the technology-free group and at least one section in either of the other groups. We limited our sample to students who took the class as sophomores and excluded students enrolled in classrooms of professors who chose not to participate in the experiment. Our final sample consisted of 50 classrooms and 726 students over the two terms.

Our primary outcome is student performance on the mandatory, high-stakes final exam for the course. This exam consisted of multiple-choice, short-answer, and essay questions. Students had 210 minutes to complete it. All were required to use a computer to complete the exam, and the software program automatically graded the multiple-choice and short-answer questions. Then, with those results in hand, professors manually scored all essay responses. All but 15 students in our sample sat for the final exam, which was worth 25 percent of a student’s final grade. Professors warned their students at the beginning of the semester that a failing grade on the final exam could constitute grounds for failing the entire course, regardless of marks earned on other assignments.

We focus our analysis below on the automatically graded multiple-choice and short-answer questions, which accounted for roughly 85 percent of the full exam grade. We excluded essay scores from our analysis because we found that some professors tended to grade essay questions in a manner that ensured students on the margin received a passing score on the exam.

Results

Overall, students in our sample did relatively well on the final exam, but those who were prohibited from using Internet-connected devices during class did best. The average score, looking at students’ multiple-choice and short-answer scores, was roughly 71.7 percent, with a standard deviation of 9.2 percentage points. Students in classrooms without Internet-connected devices earned the highest average score of 72.9 percent. Students in classrooms where laptop and tablet usage was not restricted earned the lowest scores, on average, at 70.5 percent, a difference of 2.4 percentage points. Students in classrooms where only tablets were allowed under strict conditions did slightly better, with an average score of 71.4 percent, but they still had lower scores than students in the technology-free group.

Our best evidence of the effects of laptop policy comes from a separate analysis that compares the exam scores of students assigned to the unrestricted-use and tablet-only classrooms to those of students in classes where laptops were banned, while adjusting for the minor differences in the backgrounds of students across groups and including controls for the instructor, the class hour, and the semester. Instructor controls are important, as we want to eliminate any differences from instructors who are better or worse at delivering the material. Class-hour controls account for whether students perform differently at different hours of the day, such as before or after lunch. Semester controls ensure that differences are not driven by slight variations in the course between the two semesters.

Our analysis indicates that unrestricted laptop use reduced students’ exam scores by 0.18 standard deviations relative to students for whom laptops were prohibited (see Figure 2). Perhaps surprisingly, the effect in the tablet-only classrooms was similar, at 0.17 standard deviations. Although both of those negative effects are statistically significant, our study was not large enough to determine whether the true effect of modified tablet use was more or less negative than the effect of unrestricted laptop use.

To put these findings in perspective, we compare the effect of prohibiting computers to the association between GPA at baseline and final-exam success. Banning computers gives students a leg up, grade-wise: we find that a student in a classroom that prohibits computers is on equal footing with a peer who is in a class that allows computers and whose GPA is one-third of a standard deviation higher—nearly the difference between a B+ and an A- average, for example.

In addition to analyzing the sample as a whole, we also looked separately at subgroups of students defined based on gender, race, scores on college-entrance exams, and entering GPA (see Figure 3). In no group did students appear to significantly benefit from access to computers in the classroom. We did find some suggestive evidence that permitting computers is more detrimental to male students than to female students and to students with relatively high entrance-exam scores. Future research is needed to verify the robustness of these differences, as they are based on smaller numbers of students and may have occurred by chance.

Implications

To be sure, Internet-connected computers may enhance the learning environment in some cases, and a 2006 study by Miri Barak, Alberta Lipson, and Steven Lerman suggests that students enjoy having computers in the classroom. In a traditional classroom, where computers and tablets are used only to take notes, the benefits may include the ability to take notes faster and carry notes at all times. However, a 2014 study by Pam Mueller and Daniel Oppenheimer found that students taking notes on laptops perform worse on conceptual questions than students required to use pen and paper. One theory is that the ability to record content quickly led the students to engage in transcription rather than the identification of a lecture’s most important points.

Outside of the classroom, increased connectivity on college campuses provides opportunities for students and teachers to collaborate, empowers student research via university library–enabled online search engines, and allows students to use enhanced electronic textbooks, which include embedded videos and hyperlinks to pertinent articles on the Internet. Evidence is mounting, however, that potential distractions from websurfing, e-mail, and electronic chatting with friends can hinder student learning. In a 2010 study, James Kraushaar and David Novak found that students using laptops in class had non-course-related software open and active 42 percent of the time, and a 2008 study by Carrie Fried found that students report increased multitasking when laptops are in the classroom. Multiple laboratory-style studies demonstrate the negative effects of laptop multitasking on test performance, including a 2013 study by Faria Sana, Tina Weston, and Nicholas J. Cepeda that found that test-score performance suffered not only if a student used a laptop during class, but also if he or she merely sat near a computer user.

In K–12 schools, where students do not typically take lecture notes, a growing body of research has found no positive impact of expanded computer or Internet access. For example, a 2002 study by Joshua Angrist and Victor Lavy found that installing computers throughout elementary and middle schools in Israel had no effect on student achievement, even though their teachers used more computer-aided instruction. Another study, published in 2006 by Austan Goolsbee and Jonathan Guryan, found that the federal E-Rate program expanded California students’ Internet access by 66 percent over four years but did not have an impact on student achievement (see “World Wide Wonder?” research, Winter 2006). Other studies have found no link between enhanced student outcomes and expanded information-technology spending, universal-laptop programs, and providing students with home computers.

Our study builds on this prior research by using random-assignment methods and measuring the cumulative effects of Internet-enabled classroom technology over the course of a semester rather than measuring immediate or shorter-term effects. Further, as is the concern with most lab experiments, participants may perform tasks differently or behave abnormally when being forced to use computers in a lab setting. Our study was performed in a high-stakes environment where students have the choice to use technology.

Our findings are consistent with those of a recent study by Richard Patterson and Robert Patterson, which found that in-class computer usage reduces academic performance by between 0.14 and 0.37 points on a four-point grade scale among undergraduate students at a private liberal-arts college. These effects were concentrated among male and low-performing students and in quantitative courses. That study differs from ours because it compares students who use computers to students who do not use computers within the same classroom. In contrast, our study directly measures the effect of a common classroom policy decision (that is, to allow computers or not) by comparing classrooms that permit computers to classrooms that prohibit computers. Within our study, only about 60 percent of students assigned to classrooms that permitted some form of technology actually used a laptop or an iPad. Thus, one potential reason our estimates are smaller in magnitude is that the harmful effects of computers in the classroom could be more pronounced among students who use computers than among students who choose not to use computers. Alternatively, considering the small classroom size and strict environment at West Point, the negative effects of technology could be larger in more standard college settings.

As stated above, we do not claim that all computer use in the classroom is harmful. Exercises where computers or tablets are deliberately used may, in fact, improve student performance. Rather, our results relate to classes where using computers or tablets for note-taking is optional. Further, it was beyond the scope of our study to identify how computer and tablet access lowered test scores. Was it because students’ note-taking was worse? Were students distracted by e-mail, social media, or other websites? Did instructors teach differently when students were on their computers? As computers in the classroom become more prevalent, research focusing on these areas is clearly necessary.

In the meantime, as we head into a new school year, educators at all levels may want to think twice before allowing students to open their laptops.

Susan Payne Carter is an assistant professor of economics at the United States Military Academy. Major Kyle Greenberg is a research analyst at the Army’s Human Resources Command. Major Michael S. Walker is a research analyst at the Office of Cost Assessment and Program Evaluation within the Office of the Secretary of Defense. A more detailed account of this investigation can be found in the February 2017 issue of the Economics of Education Review. The views expressed herein are those of the authors and do not reflect the position of the United States Military Academy, the Department of the Army, or the Department of Defense.

In the summer of 2011, my wife Dana and I had a lot going on. For the previous 11 years, I had worked at our medical-staffing company, both as CEO and as a certified first assistant in cardiothoracic surgery. Hispanics for School Choice, the nonprofit I had formed to help Latino families in Milwaukee navigate their education options, was celebrating its second anniversary. And Dana, a pediatric nurse practitioner, had just decided to pursue a PhD at Marquette University. So when she suggested that I apply to become president of St. Anthony, our son’s school, I didn’t take her seriously. I was an entrepreneur, not an educator!

The priest at St. Anthony Parish had asked me to help him find a new president for the school. I had gotten to know him and St. Anthony School primarily through my efforts with Hispanics for School Choice. Since our nonprofit served the Latino community, it was natural for us to form an alliance with the largest Latino elementary school in Wisconsin. St. Anthony, with nearly 1,400 students at the time, was 99 percent Latino, and nearly all of its students qualified for free or reduced-price lunch. Only about a dozen students, including my child, attended the school without the support of a state-funded voucher.

We had been searching for a new president for several months when Dana repeated her earlier suggestion. “You really should be president,” she said. “You’re perfect for the job.” This time I took her seriously.

But how could I lead a school? I had no experience in education leadership. However, I did have a mentor who had always told me to listen to my wife, especially when it came to career moves, because “a wife like Dana will not lead you astray.” So I applied for the job, and the school hired me.

Founded in 1872, St. Anthony was a traditional Catholic school in most ways. Mass and Confession were offered every week, students wore uniforms, classrooms were orderly, the curriculum was classical, and the love for children was palpable. People at the school believed in educating the whole child: mind, body, and spirit. The previous president had successfully expanded the enrollment from 200 to 1,400 students in just over eight years, and the school was in the process of adding a high school. Keeping up with such rapid change was challenging for faculty and staff, but their students came from some of the most economically disadvantaged families in the state, and they did not see failure as an option.

To my surprise, my entrepreneurial spirit served me well as I took over leadership of the school. Funding our programs was always a challenge. As an entrepreneur, I was able to step outside of the traditional ways of thinking in education and consider a range of fresh options for financing new initiatives. St. Anthony was able to add a pediatric health clinic, the nation’s first at an independent Catholic school, by utilizing BadgerCare, Wisconsin’s Medicaid program. Similarly, we tapped into funds from the state’s childcare subsidy system to support a new early-childhood program at the school.

Because the per-pupil voucher amount our students received from the state had not increased in more than five years, we had no reserves to fall back on, and no endowment. So when I initially proposed the new programs, it raised eyebrows. It wasn’t easy to convince our stakeholders to leverage our century-old buildings to secure a $500,000 loan to start the early-childhood component. My experience launching and sustaining multiple small businesses enabled me to walk our stakeholders through the process and help them understand the risks and potential rewards of our endeavor. It also gave the lender confidence that we understood how to run a business.

The risks paid off. Over the next three years, St. Anthony grew to more than 2,000 students, and test scores improved. More than 90 percent of the students in our first two graduating classes went on to college.

For me, deciding to leave St. Anthony was hard, but the need to move on to other projects is in an entrepreneur’s blood. I’m thankful to our students, families, and amazing staff for helping me realize my vocation and making our school a model of success. And above all, I learned my most valuable lesson—to listen to my wife.

Zeus Rodriguez is president of Rodriguez Corp and Hispanics for School Choice, both based in Phoenix, Arizona.

Teachers matter—and some matter more than others. That recognition has driven a tidal wave of controversial policy reforms over the past decade, rooted in new evaluation systems that link teachers’ ratings and, in some cases, their pay and advancement to evidence of classroom practice and student learning. Two out of three U.S. states overhauled teacher evaluations between 2009 and 2015, supported by federal incentives such as Race to the Top and Teacher Incentive Fund grants, as well as No Child Left Behind Act waivers.

What is the impact, so far, of these reforms? One common narrative would indicate a flop. Most states adopting new evaluation systems saw little change in the share of teachers deemed less than effective, arguably limiting their potential to address underperformance. Meanwhile, a broader backlash against reform, fueled by concerns about over-reliance on standardized tests, the accuracy of new evaluations, and the efficacy of performance-based incentives, has led some states to reverse course. Congress ultimately chose to exclude any requirements about teacher evaluation policies from the Every Student Succeeds Act of 2015, dashing some reformers’ hopes for a federal mandate.

A closer look at one high-stakes evaluation system, however, shows the positive consequences such systems can have for students. Since 2012, we have been studying IMPACT, a seminal effort by the District of Columbia Public Schools (DCPS) to link teacher retention and pay to their performance. Under IMPACT, the district sets detailed standards for high-quality instruction, conducts multiple observations, assesses individual performance based on evidence of student progress, and retains and rewards teachers based on annual ratings. Looking across our analyses, we see that under IMPACT, DCPS has dramatically improved the quality of teaching in its schools—likely contributing to its status as the fastest-improving large urban school system in the United States as measured by the National Assessment of Educational Progress.

Such reforms are often considered politically impossible, and the effort in DCPS shows the potential fallout. The district’s controversial chancellor, Michelle Rhee, resigned a year after IMPACT launched, when the mayor who appointed her, Adrian Fenty, lost a reelection bid in a campaign focused on school reform.

But IMPACT outlasted them both, to the benefit of students. DCPS dismissed the majority of very low performing teachers and replaced them with teachers whose students did better, especially in math. Other low-performing teachers were 50 percent more likely to leave their jobs voluntarily, and those who opted to stay improved significantly, on average, the following year. High-performing teachers improved their performance as well, especially those within reach of the significant financial incentive created by the system. Certainly, improvement was not universal, and some very good teachers decided to leave the district. Nonetheless, our analysis finds that improved teaching was common and that student achievement increased as a result.

The DCPS story shows that it may be politically challenging to adopt high-stakes evaluation systems, but it is not impossible. And it shows that well-designed and carefully implemented teacher evaluations can serve as an important district improvement strategy—so long as states and districts are also willing to make tough, performance-based decisions about teacher retention, development, and pay.

A More-Rigorous Approach

Teacher evaluations came into focus in 2009 alongside a growing research consensus that teacher quality has dramatic, long-lasting impacts on student success. DCPS was in the vanguard of these efforts, and was one of the first school districts in the country to link individual performance evaluations to high-stakes decisions about pay and retention when it launched IMPACT in the 2009‒10 school year.

IMPACT articulates clear standards for effective instruction, provides instructional coaches to help teachers meet those standards, and evaluates teacher performance based in large part on direct, structured observations of teachers’ classroom practices in addition to evidence of student learning. IMPACT’s features are broadly consistent with emerging best-practice design principles informed by the Measures of Effective Teaching project, and are intended to drive improvements in teacher quality and student achievement (see “Capturing the Dimensions of Effective Teaching,” features, Fall 2012).

Under IMPACT, all teachers receive a single score ranging from 100 to 400 points at the end of each school year based on classroom observations, measures of student learning, and commitment to the school community. However, the components of the scores and the weights assigned to them vary, depending on what measures of student performance are available, and DCPS has adjusted their composition over time. Below, we describe the components of IMPACT as it existed in its first three years.

All teachers were evaluated by five structured classroom observations aligned to the district’s Teaching and Learning Framework, which defined domains of effective instruction, such as leading well-organized, objective-driven lessons; checking for student understanding; explaining content clearly; and maximizing instructional time. Of the five observations, three were conducted by an administrator and two by “master educators” who traveled between several schools; only the first observation was announced in advance.

The weight of these observation scores varied, depending on whether a teacher also had a value-added score based on her students’ performance on standardized tests. For those teachers—who led reading or math classrooms in grades 4‒8 and accounted for less than one in five DCPS teachers—observations were worth 35 percent and value-added was worth 50 percent. For the majority of DCPS teachers who did not have value-added scores, observations counted for 75 percent of the total IMPACT score. An additional 10 percent was based on progress toward goals for student learning, which teachers and administrators set each year.

A teacher’s contribution to a school’s community, as assessed by the principal, was worth 10 percent of the overall evaluation score, while the final 5 percent was based on a measure of the value-added to student achievement for the school as a whole. Teachers could also have points deducted for failing to meet expectations for attendance, punctuality, and adherence to other policies and procedures.

Telling Teachers Apart

Unlike typical teacher-evaluation systems, IMPACT creates substantial differentiation in ratings. In 2009‒10 and 2010‒11, 14 percent of teachers were rated “highly effective,” 69 percent of teachers were rated “effective,” 14 percent were judged “minimally effective,”  and another 2 percent were deemed “ineffective.”

The system also sets swift consequences for teachers with very low or very high scores. Teachers rated “ineffective” are dismissed with rare exception, a rule that caused more than 200 teachers to exit DCPS in IMPACT’s first year. Those rated “minimally effective” are given one year to raise their score above the “effective” threshold; if they do not, they are dismissed.

On the other side of the spectrum, teachers are eligible for a bonus of up to $25,000 in each year they are rated “highly effective;”  those with “highly effective” scores two or more years in a row can earn increases in their base pay of up to $27,000. Teachers working in high-poverty schools teaching high-need subjects are eligible for the largest pay increases.

These design features create sharp incentive contrasts for teachers with scores on either side of the “effective” threshold, because scoring above it removes the threat of dismissal. And they create an incentive for teachers with scores just below the “highly effective” threshold, because scoring above it makes them eligible for a significant increase in pay.

Our research on IMPACT sought to understand whether those incentives impact teachers’ performance and retention—and whether those impacts, if any, improve student outcomes. We based our analysis on administrative data on all DCPS teachers and their students over the first three years of IMPACT: 2009‒10, 2010‒11, and 2011‒12. We limited our sample to general-education teachers working in schools serving K‒12 students. We also drew on an additional year of data, from the 2012‒13 school year, in assessing IMPACT’s effects on student achievement in tested grades and subjects.

Teacher Retention and Performance

DCPS teachers were retained differently depending on their IMPACT ratings. By the program’s third year, 95 percent of teachers deemed “ineffective” in the system’s first two years had been dismissed (see Figure 1). Overall, 3.8 percent of all teachers in the district were let go as a result of being rated “ineffective” once or after earning two consecutive “minimally effective” ratings under IMPACT between 2009‒10 and 2011‒12.

Under IMPACT, lower-rated teachers were also far more likely to exit DCPS voluntarily. Among all teachers rated “minimally effective,” 27 percent voluntarily left the district, compared to 14 percent of teachers rated “effective” and 9 percent of teachers rated “highly effective.” “Minimally effective” teachers whose scores were closest to the “effective” threshold were less likely to leave than those with lower scores; about one in four teachers whose scores were within 25 points of the “effective” threshold chose to leave their jobs, compared to about one in three whose scores were more than 25 points below. In other words, teachers under threat of dismissal were more likely to voluntarily leave than teachers not subject to this threat, and those who scored furthest from the “effective” threshold were even more likely to go.

These patterns are consistent with a restructuring of the teacher workforce in response to the incentives embedded in IMPACT, but other explanations are possible. For example, some studies have found that less-effective early-career teachers are more likely to exit than more-effective novice teachers, even in the absence of high-stakes evaluations. In addition, IMPACT scores for teachers in their first two years of teaching average 17 points less than those with three or more years of experience. Such considerations raise doubts about how to interpret the relationship between IMPACT and retention. Are we observing the effects of incentives or merely behavior that would have occurred in the absence of IMPACT?

To isolate the causal impact of performance-based incentives on teacher quality, we compare outcomes among teachers whose initial IMPACT scores placed them near the thresholds between categories: “minimally effective/effective” and “effective/highly effective.” We assume that whether a teacher scores just above or just below a certain threshold is essentially random, which is supported both by our knowledge of how DCPS calculates IMPACT scores and additional analyses confirming that teachers on either side of each threshold are quite similar in terms of experience and other characteristics. We look at teacher retention during the second year of IMPACT, when the threat of dismissal for “minimally effective” teachers became newly credible and financial incentives for “highly effective” teachers could be made permanent. We also examine performance among returning teachers in the system’s third year. By comparing teacher attrition and performance on each side of the performance cutoffs, we can get a better sense of how the threat of dismissal or prospect of a raise affects teachers’ behavior.

We first use this method to measure the effects on teachers scoring directly above or below the “minimally effective/effective” threshold in 2010‒11. We find that teachers near the threshold who received their first “minimally effective” rating at this time were considerably more likely to exit DCPS voluntarily, with retention dropping by 10 percentage points in 2011‒12 (see Figure 2). In other words, IMPACT’s minimally effective rating increased the attrition of lower-performing teachers from 20 percent to 30 percent, an increment of 50 percent.

Meanwhile, the teachers in this “minimally effective” group that did return to DCPS the following school year improved their performance by roughly 11 points on the IMPACT scale. This sharp increase in teacher performance is about 0.27 of a standard deviation (see Figure 3) and suggests that previously low-performing teachers who opted to remain on the job undertook successful efforts to improve.

Notably, the effects of a minimally effective rating on retention and performance occurred at the end of IMPACT’s second year, when the political credibility of the reform had been affirmed by the appointment of Kaya Henderson as chancellor and by the first instance in which teachers (roughly 140) were fired for having two consecutive “minimally effective” ratings. In contrast, following its first year of implementation, IMPACT had no statistically detectable effect on teacher retention or performance. Given the contentiousness of these policies and the political volatility associated with Mayor Fenty’s loss and Chancellor Rhee’s resignation, teachers may have reasonably doubted the staying power of IMPACT’s performance-based dismissal threats, opting not to respond to its incentives. We are just speculating regarding the cause, but the important lesson is that policies intended to induce significant behavioral responses may need time to take hold.

In contrast, the financial incentives for consistently high-performing teachers (i.e., jumping ahead on the salary schedule) appear to have been immediately credible. Among highly rated teachers who scored very close to the eligibility threshold for a permanent pay increase, retention increased by roughly 3 percentage points, though this effect was not statistically significant. Their performance in 2011‒12 improved by a statistically significant 10.9 points, or 0.24 of a standard deviation.

These performance gains imply increases of approximately 5 percentile points in the distribution of teacher performance among lower-rated teachers, and 7 percentile points among highly rated teachers. Comparing these figures to the typical improvement of a novice teacher during her first three years in the classroom, we find that the gain of 12.6 points for lower-rated teachers is 52 percent of this three-year gain, and the gain of 10.9 points for highly rated teachers is 41 percent of the three-year gain.

Teacher Turnover and Student Achievement

IMPACT’s effects also depend on the direct impact of teacher turnover and the quality of newly hired teachers. Teacher turnover is often assumed to have a universally negative influence on school quality, and replacing teachers in schools with high rates of turnover can place strong demands on district recruitment efforts. Not all turnover is the same, however. Under IMPACT, a substantial fraction of teacher turnover consists of lower-performing teachers who were purposefully compelled or encouraged to leave, which potentially alters the distribution of teacher effectiveness among exiting teachers.

To determine the effect of teacher turnover on student achievement under IMPACT, we examine the year-to-year changes in school-grade combinations with and without teacher turnover. In other words, what was the change in test scores for 4th graders from year to year at a school that had teacher turnover in that grade compared to the change in test scores between 4th graders at a school that did not have teacher turnover in that grade?

We find that the overall effect of teacher turnover in DCPS at worst had no adverse effect on student achievement and, under reasonable assumptions, improved it. This average combines the negative, but statistically insignificant, effects of exits of high-performing teachers with the very large improvements in student achievement resulting from the departures of low-performing teachers. Figure 4 illustrates these patterns by comparing the average value-added scores of new DCPS teachers entering the district with those who had exited the previous year.

Looking directly at impacts on student achievement, and carefully controlling for a variety of potentially confounding factors, we find results quite consistent with these averages (see Figure 5). For example, turnover among low-performing teachers leads to improved student achievement in both math and reading. In math, the exit of low-performing teachers is estimated to improve student achievement by 0.21 of a standard deviation—between one-third and two-thirds of a year of learning, depending on grade level. In reading, student achievement is estimated to increase by 0.14 of a standard deviation.

We also compare differences in turnover and their impact on student achievement at high- and low-poverty schools. Overall, we find that high-poverty schools appear to improve as a result of teacher turnover, though as in all schools, not all turnover is the same.

In high-poverty schools, we estimate that the overall effect of all teacher turnover on student achievement is 0.08 of a standard deviation in math and 0.05 of a standard deviation in reading. In comparison, in low-poverty schools, the estimated effects of turnover are close to zero.

However, 40 percent of teacher turnover in high-poverty schools is among low-performing teachers—and our estimates indicate consistently large gains for students when these teachers exit these schools. In math, student achievement improves by 0.21 of a standard deviation, with teacher quality improving by 1.3 standard deviations. In reading, student achievement improves by 0.14 of a standard deviation, with an improvement of 1.0 standard deviations in teacher quality.

In sum, DCPS has been able to replace low-performing teachers in high-poverty schools with teachers who are substantially more effective. DCPS also appears to be quite capable of replacing exiting high-performing teachers in low-poverty schools with comparable teachers. With the exception of math achievement in one year (2011‒12), however, the effect of this turnover on student achievement is not statistically significant.

Implications

The high stakes associated with IMPACT have been a source of contention, both within the District of Columbia as well as in broader discussions of education policy. While the importance of highly effective teachers is uncontroversial, the manner in which schools and districts identify, reward, and retain these educators is.

Our studies provide clear evidence that IMPACT’s exceptionally high-powered, individually targeted incentives linked to performance influence retention and performance in desirable ways. This multiple-measures system boosts performance among teachers most immediately facing consequences for their ratings, and promotes higher rates of turnover among the lowest-performing teachers, with positive consequences for student achievement.

Importantly, more than 90 percent of the turnover of low-performing teachers occurs in high-poverty schools, which constitute 75 percent of all schools. The proportion of exiting teachers who are low performers is twice as high in these high-poverty schools as in low-poverty schools. In comparison with almost any other intervention, these are very large improvements for some of the school district’s neediest students.

We do not claim that IMPACT caused all of the teacher turnover we observe. Although IMPACT did result in some teachers being dismissed, some attrition from teaching in DCPS as the system was implemented was surely voluntary. Nor do we know whether our results showing that teacher turnover benefited students in tested grades and subjects generalize to turnover for other teachers.

Policymakers looking to build on the DCPS model should also understand the implementation challenges and the potential for errors. Any teacher-evaluation system will make some number of objectionable mistakes in assigning ratings and consequences, and minimizing those instances will require expensive, sophisticated tools. Policymakers must weigh these costs against the substantial educational and economic benefits such systems can create for successive cohorts of students, both through avoiding the career-long retention of the lowest-performing teachers and through broad increases in performance in the overall teaching workforce.

They should also expect continual change and improvement. DCPS has made modifications to IMPACT nearly every year, such as reducing the number of teacher observations, altering access to bonus and base-pay increases toward high-poverty schools, and reducing the weight of value-added performance measures. Most recently, the district announced it would rely on school principals for all observations and incorporate student-survey results in teachers’ evaluations. DCPS also introduced LEAP, an intensive professional development program to help teachers improve their skills.

The challenge of improving the composition of teachers in DCPS is increasing. As the least-effective teachers exit, there are fewer such teachers who will leave over time. Whether DCPS can reap further performance benefits from compositional change in its workforce as it increases performance standards remains to be seen.

Regardless, our results indicate that, under a robust system of performance evaluation, the turnover of teachers can generate meaningful gains in student outcomes, particularly for the most disadvantaged students. The eight-year history of IMPACT shows that such efforts may incur political consequences, but are not politically impossible.

Thomas Dee is a professor of education and director of the Center for Education Policy Analysis at Stanford University, and a research associate at the National Bureau of Economic Research. James Wyckoff is the Curry Memorial Professor of Education and Policy and director of EdPolicyWorks at the University of Virginia. This article is based on research published in the March 2017 issue of Educational Evaluation and Policy Analysis and in the March 2015 issue of the Journal of Policy Analysis and Management.

“It was one of the most powerful visits I’ve ever taken,” says Sheila Briggs, an assistant state superintendent with the Wisconsin Department of Public Instruction. She is describing a visit last fall to Lake Pontchartrain Elementary School, a low-income school in St. John the Baptist Parish, Louisiana, about 30 miles northwest of New Orleans. “The ability to hear what the state education agency was doing and then go into classrooms and see direct evidence was phenomenal,” Briggs gushes. “I’ve never seen anything like it anywhere else.”

Officials of state education agencies are not known for hyperbole. Maintaining data systems, drafting rules and regulations, and monitoring compliance are not the stuff of breathless raves—especially in Louisiana, whose education system ranks near the bottom nationwide on measures of student achievement and high-school graduation rates. Yet in the last year, education leaders from across the country have beaten a path here to see what they might learn from state education superintendent John White; his assistant superintendent of academics, Rebecca Kockler; and their colleagues. Together, this team has quietly engineered a system of curriculum-driven reforms that have prompted Louisiana’s public school teachers to change the quality of their instruction in measurable and observable ways. These advances are unmatched in other states that, like Louisiana, have adopted Common Core or similar standards.

The linchpin of the state’s work has been providing incentives for districts and schools statewide to adopt and implement a high-quality and coherent curriculum, particularly in English language arts (ELA) and mathematics, and to use that curriculum as the hook on which everything else hangs: assessment, professional development, and teacher training. Most notably, White and Kockler have pulled off these reforms in the face of strident political resistance to Common Core and without running afoul of districts and teachers in this staunch local-control state. The state has also posted tantalizing gains in student outcomes: Louisiana 4th graders showed the highest growth among all states on the 2015 National Assessment of Educational Progress (NAEP) reading test, and the second-highest in math (see Figure 1). However, in both Louisiana and the nation as a whole, 8th grade scores in reading and math declined slightly that year. It’s too early to call Louisiana the new Massachusetts—2017 NAEP scores could indicate whether the needle is continuing to move in the right direction—but other states are taking notice and may be following Louisiana’s lead.

“Large and Intriguing Differences” 

Adopting new standards accomplishes nothing unless it gets teachers to change classroom practice. In 2016, when researchers at the RAND Corporation set out to study Common Core implementation at the state level, they found something unexpected. Using data from the organization’s American Teacher Panel, a standing nationwide sample of about 2,700 teachers, the researchers noticed “large and intriguing differences” between Louisiana teachers and those in other states. Louisiana’s educators were far more likely to be using instructional materials aligned with Common Core standards. They also demonstrated a better understanding of the standards and taught their students in ways the standards were meant to encourage.

“We saw consistently higher results in Louisiana,” says Julia Kaufman, a RAND policy researcher. “There were occasional high points in other states, but we kept seeing this difference between Louisiana [teachers] and other teachers, which is why we decided to write the report. We just thought there was a story there.”

There is a story, and it’s about curriculum—perhaps the last, best, and almost entirely un-pulled education-reform lever. Despite persuasive evidence suggesting that a high-quality curriculum is a more cost-effective means of improving student outcomes than many more-popular ed-reform measures, such as merit pay for teachers or reducing class size, states have largely ignored curriculum reform.

“People underestimate the power of curriculum. It’s like buying a new water heater. It’s not like getting a new kitchen,” says Litsy Witkowski, chief of staff for academic content at the Louisiana Department of Education. “It’s just not sexy.”

“We believe that states have a role in . . . helping teachers deliver constantly improving instruction for students,” says her colleague Kockler. Louisiana’s articles of faith included “a commitment to coherence and quality” and a belief that if curriculum, professional development, and assessment were not tightly connected, “it would confuse and frustrate teachers,” she adds. Louisiana adopted Common Core in 2010. Two years later, when White arrived and hired Kockler, the state launched a quality review of the curriculum and the instructional materials being used by the districts. Only 1 out of 60 programs passed muster, prompting the state to develop its own ELA curriculum. As White recently told Education Week, commercial publishers had been “relentless about an unwillingness to change and a desire for maximizing profits on old materials that [were] not helping students.”

States often recommend or require the use of various textbooks or curricula. The Louisiana Department of Education (LDOE) was determined to do neither. “We’re a local-control state, so we cannot force anyone to do anything,” says Kockler. “Districts are going to do what they believe is best, and we want to help them be positioned to do so.” The key was offering incentives for districts to make good decisions, a process Kockler describes as “making the best choice the easy choice.” Louisiana began publishing free, annotated reviews of K–12 textbooks and curriculum programs in ELA and math, sorting the materials into three “tiers.” If a curriculum was judged to “exemplify quality,” it earned the Tier 1 designation; programs judged to be “approaching quality” were labeled Tier 2; and those seen as “not representing quality” went into Tier 3. Notably, the quality reviews were not conducted by bureaucrats in New Orleans and Baton Rouge, but by the state’s network of “teacher leaders,” two from each school, chosen by the LDOE. “We basically created the Yelp of curriculum,” quips Witkowski.

The tiering process was rigorous, and few programs earned the Tier 1 designation. Eureka Math, Zearn Math (grades 1–5), and a pair of products from the Math Learning Center made the cut for elementary-school math. In ELA, the Core Knowledge Language Arts program in grades K–3 and the Great Minds Wit & Wisdom curriculum in grades 3–5 and 6–8, among other programs, received the Tier 1 label. The list of Tier 2 curricula is longer, and Tier 3 is longer still. The results of Louisiana’s tiering process are broadly in line with reviews conducted by independent evaluators, such as the nonprofit edreports.org, whose analyses are evidence-based.

The LDOE emphasizes that each of the state’s 131 local school districts should determine if a given curriculum is “appropriate to meet the educational needs of their students.” However, the state sweetened the adoption pot by giving all Tier 1 vendors statewide contracts. Typically, this enabled districts to use these vendors at discounted prices and without having to undergo their own procurement process. “There are a lot of barriers in the system that make it hard for districts and teachers to get their hands on the best stuff,” Kockler observes. “We want the best to be the easiest to access.”

Persuading districts to adopt a top-notch curriculum was just the beginning. “If teachers do not deeply understand their standards—or the instructional practices that are aligned with them—their instruction may fall short of helping students meet those standards,” observes the RAND Corporation’s Kaufman, who, along with Lindsey Thompson and V. Darleen Opfer, found that Louisiana teachers demonstrated a stronger grasp of the Common Core standards and adopted more classroom practices that reflect them than did teachers elsewhere. The trio found that higher percentages of Louisiana ELA teachers were able to correctly identify practices and approaches aligned with Common Core than educators in other states that have adopted the Common Core or similar standards. For example, less than half (47 percent) of Louisiana teachers thought that “selecting texts for individual students based on their reading levels” was an instructional approach aligned with standards (it’s not) compared to 70 percent of teachers in other states. Most teachers in Louisiana perceive—correctly—that their standards instead encourage them to teach particular grade-level texts and organize reading skills instruction around those texts rather than teaching reading skills and allowing students to apply them to any text. This approach represents a watershed instructional change that is still unusual—except in Louisiana.

The teachers didn’t acquire their instructional savvy by accident. State officials, having identified a small number of Tier 1 curricula, applied a similar winnowing process to professional-development providers, recommending only vendors such as Achievement Network (ANet), American Reading Company, Generation Ready, LearnZillion, and others that could offer training geared specifically to Louisiana’s Tier 1 curricula instead of general pedagogical strategies or techniques. This too is an unusual practice. Most states provide professional development, but the RAND researchers did not find other clear examples of state departments of education working to make explicit connections between professional-development providers and specific curricula.

The combined power of persuasion, purse, and professional development has reaped results: nearly 90 percent of teachers in Louisiana who responded to the RAND survey report using Eureka Math, a Tier 1 program.

Significantly, all of this work was done with teachers, not to them. The LDOE created a network of teacher leaders who were handpicked for demonstrated teaching and leadership ability, drawn from every region of the state and different grade levels. While the state created the rubrics for the curriculum, it was the teachers who did the evaluations—a feature that draws praise from the state’s largest teachers union.

“We had lots of buy-in,” says Larry Carter, president of the Louisiana Federation of Teachers. “There’s some sense of stability to how education is being delivered to students.”

This teacher-friendly approach to reform has also altered the relationship between the Louisiana DOE and school districts. Previously, LDOE visits were “very compliance-driven,” says Dana Talley, a veteran teacher and state network leader in northeast Louisiana. “Now it feels like support, not compliance,” she adds.

Carter agrees with Talley’s take. Although the union doesn’t “see eye to eye with the superintendent on testing,” they generally agree on “issues that relate to curriculum and standards.”

Joanne Weiss, the former chief of staff at the U.S. Department of Education under Secretary Arne Duncan, has brought education officials from more than a dozen states to Louisiana in her capacity as a consultant to the Council of Chief State School Officers (CCSSO). Weiss gives White and Kockler high marks for their clear-eyed view of their state’s education system. “I think people have a tendency to develop strategies that are completely absent the context of the moment in time or the place. Rebecca and John are doing the opposite,” she observes. “They’re saying, ‘Our work needs to sit at the nexus of good content, good professional learning for teachers, and good measurement tools and assessments. We need to get those three things right.’”

Curriculum in Action

On a Monday morning in January, Yasmin Haley, a 6th grader at Lake Pontchartrain Elementary School, is working with two other girls on changing the point of view in a story from third-person-limited to first person. The girls are studying a picture of two characters named Gerry and Perry who, according to a brief narrative written by another group of students, have been fighting. “It has to be the same concept, but a different perspective,” Haley tells me, explaining the assignment. She turns to her two classmates: “It might say what Gerry’s thinking,” she says. “We have to use the voice of the person.” Across the room, their teacher, Michel Delatte, circulates among the students. At one point she calls out to the class, “Be mindful of your grammar!”

Louisiana created “Guidebooks 2.0,” a homegrown language-arts curriculum, in response to lackluster reviews its teachers gave to commercial programs. It features lengthy units on the novels Hatchet, The Witch of Blackbird Pond, and Out of the Dust; and a nonfiction unit on Apple founder Steve Jobs. It is the primary ELA curriculum resource for grades 6–8 at Lake Pontchartrain, a Title I school where 100 percent of the students are eligible for free or reduced-price lunch. But this lesson was created by Delatte herself. “I supplement the state guidebooks because I know point of view is on the test,” she tells me. “A lot of times they ask [students] to change the point of view of a story. So this is a great way for them to do that. They like working in groups, and it’s fun for them to get out of their seats, move around, see what others wrote, and improve their own writing.”

In the next classroom I visit, accompanied by principal Jason Beber, 4th graders are working on a reading comprehension exercise on geology, part of “The Changing Earth” unit of the Core Knowledge Language Arts program, which the district has adopted in grades K–5. Lake Pontchartrain Elementary was badly damaged by 2012’s Hurricane Isaac; now, most of the classes meet in temporary trailers while construction of a new building gets underway nearby. In the next trailer, another 4th-grade teacher, Jennifer Brock, leads an equivalent-fractions lesson from Eureka Math, which St. John uses end to end, from K to 12. “Teachers have a clear picture of what they’re supposed to be doing in the classroom,” says Beber, who is in his third year running the school, which has gone from a D to a C on state-issued school performance reports, despite the disruption wrought by the storm.

“Soon to be a B,” he cheerfully insists as we move on to the next classroom. In nearly every room hangs an identical poster, a knockoff of the iconic World War II–era poster aimed at reassuring an anxious British public: “Keep Calm and Score Basic or Above.”

The decision to go with Core Knowledge, the state’s Guidebooks, and Eureka Math were district-level calls, and Beber sees no reason to second-guess the decision. At the school level, curriculum is not negotiable, he says. “This is our religion. This is what we do. But we need great teaching to deliver it and to make it happen for everybody.” As we saw in Delatte’s room, teachers have the freedom to supplement the curriculum to meet the needs of their students or to respond to the demands of state tests.

Policy and Politics

“American policymakers seldom view curriculum as a serious lever for change,” observes Ashley Berner, deputy director of the Institute for Education Policy at Johns Hopkins School of Education. Requiring children to learn anything in particular, she notes, is considered “pedagogically suspect.” Pitched and passionate battles over course content have made curriculum a third rail in many states. But the failure of states to exert influence or offer expertise on curriculum leaves these decisions to districts, schools, and even individual teachers, which risks robbing students of coherence and consistency. Local control is a central feature of American public education, but Louisiana’s reforms offer a glimpse of how to thread the needle, honoring community control while encouraging high-quality curriculum statewide.

“You can’t just tinker around the edges with high-level things,” says Kim Benton, chief academic officer (CAO) in Mississippi’s Department of Education. “You’ve got to reach deeper into the classroom to really make sure instructional quality is improving.” Benton was one of more than a dozen state CAOs who visited Louisiana with Joanne Weiss last fall, a trip funded by the Council of Chief State School Officers. CCSSO is developing a network of states interested in supporting their schools in identifying and adopting high-quality curricula aligned to standards.

It’s no secret that curriculum choices can be a significant factor in raising academic success, as Massachusetts has demonstrated for nearly 25 years. The state’s landmark 1993 Education Reform Act introduced not only high academic standards, accountability, and enhanced school choice, but curriculum frameworks with a subject-by-subject outline of the material intended to form the basis of local curricula statewide. Massachusetts has led the nation in student achievement ever since. And a small but rigorous body of evidence confirms that curricular choices matter. A 2009 federal study of four elementary school math curricula, for example, found that students in schools assigned to use the most effective program performed 12 percentile ranks higher after one year than students in schools assigned to the least effective program.

However, districts may find it challenging to choose textbooks and instructional programs based on demonstrated efficacy, because the research base on specific programs remains thin. A March 2017 paper by David Steiner, director of the Johns Hopkins Institute for Education Policy, notes that while changing curriculum is mainly “cost-neutral” (teachers are going to teach something and there’s little price difference between good and bad textbooks), we still don’t know exactly what makes a curriculum effective. This gap exasperates Steiner, himself the former state education commissioner of New York, where he championed the creation of the EngageNY curriculum website with funding from the state’s $700 million Race to the Top grant.

“What we teach isn’t some sidebar issue in American education. It is American education,” Steiner observes. “The track record of top-performing countries, early evidence of positive effects from faithful implementation of high-quality curricula here in the United States, and the persistent evidence that our classrooms are under-challenging our students at every level compel us to put materials that we use to teach at the core of serious education reform.”

Louisiana offers a kind of proving ground for Steiner’s view. The open question is how much of Louisiana’s apparent gains can be attributed to standards, curriculum, and talent development driven by changes at the state level—and how many other states will follow the path White and Kockler have blazed. The states that do so, Weiss cautions, “need to understand their [own] particular strengths and deficits, and work from that reality. What Louisiana has done may not export exactly as is, but there are still a lot of lessons to learn.”

Weiss is working with the CCSSO on a two-year effort to encourage states to adopt a standards-aligned, high-quality curriculum, to support them in its implementation, and to ensure that teachers have access to relevant professional development. CCSSO executive director Chris Minnich admires Louisiana’s model but stresses that one key to its success was the state’s diplomatic approach. “The brilliance of what happened in Louisiana is they didn’t make a single choice for any school district in the state. They simply provided good information, training, and incentives.” While not every state will choose to follow Louisiana’s lead, “we want as many willing and able states to go there as possible,” he says, adding that not all states have the structures in place to fully support local school districts in making better curriculum choices.

And the operative word is “support.” “States thought they only had one tool in their toolbox, which was to mandate curriculum or leave it entirely up to local districts,” Minnich notes. “Bottom line is, they shouldn’t be making these decisions for their districts. There’s no buy-in that way.”

A small number of states will be selected to participate in the two-year CCSSO project, funded by the Charles and Lynn Schusterman Family Foundation and the Bill & Melinda Gates Foundation. CCSSO’s effort aims to “significantly increase” the percentage of school districts in targeted states in which curriculum and materials adoptions are of high quality and aligned to state standards; and to increase the percentage of professional-development and teacher-prep programs that include training on those curricula. In sum, it seems quite possible that the moment is at hand for curriculum and instructional material to be taken seriously as a reform lever.

Making It Stick

State education leadership can flip as often as baseball managers change their lineups. Or their socks. This instability only compounds the difficulty of making reforms stick. Now in his sixth year on the job, John White is one of the longest-serving state supes—a job where the average tenure is less than two and a half years. He has proven to be a skilled political infighter, surviving bruising battles with the state’s two teachers unions, Tea Party members, and Common Core opponents. He crossed swords with Bobby Jindal, the governor who supported his appointment, over Common Core. Once an avid supporter, Jindal turned against the standards prior to his failed bid for the White House. Jindal’s successor, John Bel Edwards, announced during his campaign for governor, “I have no intention of allowing John White, who isn’t qualified to be a middle school principal, to remain as superintendent.” White’s still there. Louisiana’s curriculum-based reforms may well outlast even him, though he has said he hopes to continue to serve “until BESE [Louisiana’s Board of Elementary and Secondary Education] tells me to stop.” Once districts have affordable, sustainable, and successful practices in place, they are “less apt to remove them and replace them than they were to add them in the first place,” he says. It’s true that a single school-board member with an agenda or strong political views can jeopardize an entire curriculum, but the teacher-leader network that White and Kockler have constructed in Louisiana seems likely to insulate students from the passions and politics of the moment. If teachers’ response to uncertainty is to close their doors and teach what they know, most of what they now know is Louisiana’s set of Tier 1 curricula, which has grown to encompass materials for early childhood, social studies, and science programs, along with math and ELA interim and benchmark assessment systems.

For now at least, Louisiana remains a laboratory from which other states can learn as they evaluate their own efforts to make more rigorous standards stick.

“There was definitely a higher level of instruction going on in the Louisiana classrooms we visited as compared with what many other states’ CAOs were seeing at home. That was very compelling to people,” notes Weiss. Some of them, she adds, “resolved to go home and make curriculum reform a priority in their own states, saying, ‘We need to do this. This is really important, and we need to do this.’”

Robert Pondiscio is a senior fellow at the Thomas B. Fordham Institute. 

Reynold Essor was sure of two things when he got his high-school diploma last spring: he wanted to get out of Brooklyn, and he wanted to go to college. Earning a degree, his counselors told him, “can help get more money in your pocket.”

So he headed to SUNY Adirondack, a public two-year community college in upstate New York that describes itself as “a leader in the region’s workforce development, preparing the next generation of leaders for a bright future.” The bucolic campus included a comfortable residence hall, leafy grounds, a restaurant run by students, and even a zipline—all featured prominently on its website and brochures. Essor could apply for federal student-loan programs to help pay his tuition, and a non-competitive, open admission policy took the pressure off of his 2.6 GPA to win acceptance.

Less prominent in the school’s pitch were the relatively long odds of earning a degree in three years: about one in four at SUNY Adirondack, in line with the 21 percent average at all 1,462 community colleges in the United States. The national six-year completion rate stands at 39 percent, on average. Part of the reason is that community colleges, with their commitment to open access, admit millions of students each year who are unprepared for college-level work, even though they have earned a high-school diploma.

For decades, despite those daunting statistics, the schools had a built-in base of students attracted to their open doors and relative affordability. But enrollment at public two-year college has dropped by about 10 percent nationwide in recent years. Decreases in state funding and widespread tuition increases have hit a stagnating middle class. And a new emphasis on outcomes-based funding is putting pressure on public colleges in some states to show results.

A hard look at outcomes raises questions about the mission and promise of community colleges, not to mention the meaning of a high-school degree. Would funding commmunity colleges based on outcomes lead them to serve students better? Or would they instead water down standards or ignore students most in need of a leg up? At what point does a commitment to open access become a license to enroll students unlikely to succeed? Is “college for everyone” really college at all?

The Remediation Trap

Community colleges enroll about a third of undergraduate students in the United States, and their goals, profiles, and levels of preparedness are dizzying in their diversity (see Figure 1). About one in four two-year schools is private, and about the same number now offer dorm-style campus housing. They enroll a disproportionate share of low-income, first-generation, and minority students, and have become a popular symbol of bootstrapping and upward mobility, inspiring leaders from former president Barack Obama to Tennessee governor Bill Haslam to propose making them tuition-free.

To be sure, many students do not enroll with an associate’s degree in mind: community colleges offer professional training and credentials, lower-cost academic basics for students who move on to four-year schools, and other opportunities for skill aquisition or intellectual exploration. But for millions, the schools are the primary point of entry to the higher-education system, especially in light of the shrinking non-competitive for-profit sector. And their mission, to extend an open invitation to any high-school graduate looking for an educational leg up, is complicated by the reality students face once they get to campus.

Non-selective admission policies may get students through the front door, but they don’t necessarily allow students to enter a college classroom right away. First, the schools administer “readiness” tests in math and English, which students must pass before they can enroll in credit-bearing classes. Students who cannot pass the tests are required to enroll in remedial courses, to master material that, at least according to the college, they should have learned in high school.

That’s what happened to Essor, who landed on the remedial track after falling short on the placement exam at SUNY Adirondack. He spent two semesters taking, and then retaking, three required remedial courses. He used financial aid, including a federal Pell Grant, to cover the costs, is now a year behind his planned timeline for graduation, and earned no credit for his effort.

It’s a common scenario. While estimates vary, a review of student transcripts from 2003 to 2009 found that 68 percent of students at two-year public colleges took at least one remedial class, with higher rates among low-income, black, and Hispanic students (see Figure 2). Students spend an estimated $7 billion annually on remedial college classes. Yet only half of enrolled students complete remedial courses, and about one in seven completes a credential within six years.

Essor believes the source of his trouble is his high-school education. “I passed without learning,” he said. And despite being counseled in high school to take college-placement tests, and even taking an exam-prep class before graduation, he said he didn’t understand his academic weaknesses or how to correct them until it was too late.

The Disconnected High-School Diploma

Overall, just 37 percent of high-school students met college-ready standards in math and reading on the 2015 National Assessment of Educational Progress. Meanwhile, the high-school graduation rate was 83 percent that year, with 69 percent of graduates heading straight to college.

This illustrates the “misalignment” between graduation standards at the K–12 level and the definition of “college readiness” at community colleges, both of which can further differ from state to state, said Nikki Edgecombe, senior research scientist at the Community College Research Center (CCRC) at Teachers College, Columbia University.

Arriving to college unprepared can spell financial trouble for community-college students, who tend to borrow less for their education but are more likely to default on their student debt. While the average student debt load is just $3,700, one in five community-college students defaults on that debt, twice the rate of students at four-year schools. About 36 percent of all federal Pell Grants go to students at public two-year schools, totaling $10.5 billion in 2012–13.

Richard Kahlenberg, senior fellow at the Century Foundation, is one of several education researchers and analysts suggesting that one solution lies in creating a tiered high-school diploma system, where one is “for everyone, connected to an exit exam,” and another is based on “demonstrating core competencies” needed for success in college (see “Rethinking the High School Diploma,” forum, Winter 2015). High schools would tell students that the second diploma could help them avoid losing time and money in college, because they could avoid expensive remedial courses. They would also need to build in support for students who aim for the college-readiness diploma but struggle.

“The political system won’t support high rates of failure,” said Kahlenberg, so this change would avoid massive decreases in high-school graduation rates, while giving students “incentive to work harder.”

Another strategy, the college- and career-ready Common Core State Standards, has run into political opposition and uneven implementation from state to state. This year, fewer than two dozen states are administering Common Core–aligned exams to high-school students, nine fewer than two years earlier.

Other efforts to make high-school graduation requirements more rigorous are running into trouble as well, now that they are about to make headlines for real students. In Ohio, for example, a projected one in three high-school juniors is expected to fall short of new graduation requirements that take effect in May 2018 and require that they earn passing scores on seven new end-of-course exams. State officials are debating ways to allow students to graduate anyway, such as considering attendance records or substituting community service and capstone project requirements for test results.

Meanwhile, a growing share of high-school graduates have sought college degrees in recent decades, with much of that growth at community colleges, where enrollment peaked at 7.9 million in 2010. This change occurred as the achievement gap in K–12 persisted, fueled by continued racial and economic segregation, with fewer resources going to schools with substantial minority populations, said Kahlenberg of the Century Foundation. Community-college students are far more likely than their counterparts at four-year schools to be Hispanic or black and from a low-income household.

“The trend of college-going continued to rise—especially of low-income, minority students—but on the whole, there haven’t been dramatic changes in the K–12 system to accommodate these changes,” said Edgecombe.

Non-Traditional Students, Unique Needs

Many students have another problem: fuzzy academic skills, thanks to years lapsed between high school and college. The average age for community college students is 29, and nearly half are age 25 or older. Most have been out of high school for some time and face myriad challenges when it comes to succeeding in college—including being savvy about the importance of college-placement tests.

Consider Hope Newman-Heck. Now 34, the mother of two dropped out of high school long ago. Several years later, she was living in Queens, New York, and working at a Hale and Hearty Soups restaurant. She decided to study for a GED-style diploma when she found herself unable to help her son with his 5th-grade English homework. With the help of a local nonprofit organization, she got her diploma in 2014.

To her surprise, she said, “People there said, ‘You don’t need to stop here … This is a doorway to other opportunities.’” That gave her the idea that earning a high-enough score on her high-school diploma equivalency exam was all she needed to get into college. The program staff “didn’t even mention remedial courses.”

She decided to enroll at SUNY Adirondack community college soon after earning her diploma because it fit the practicalities of a parent: she knew the area, and the school offered a good childcare program for her young daughter. When she talked with an adviser at the school, the conversation covered subjects like the required number of credits, financial aid, and “a test to see where you’re at,” she recalled. The school’s website counsels that the tests will place students in the classes where they will have “the greatest opportunity to succeed,” and that “the level of first-semester courses can greatly affect the amount of time required to graduate as well as the total cost of your education.”

But Newman-Heck did not grasp the sweeping consequences of the required ACCUPLACER test—that “you have to score a certain amount of points to get to take classes,” she said. “They also don’t tell you your Pell Grant money will go to pay for remedial classes that don’t count against your diploma.”

“That student’s experience has a ring of truth to it,” said John Jablonski, vice president of academic affairs at SUNY Adirondack. The school has “totally overhauled the system of academic advising” in the last year, he said, including training and paying faculty members who have aptitudes for advising so that they in turn can help students “map out how much time and money it will take to complete a degree.” SUNY Adirondack has also developed “YouTube-style” videos created by the math faculty to help students prepare for math-placement exams. The school will start to gather data tracking the impact of these changes this year, Jablonski said.

Newman-Heck’s experience is not uncommon. According to a 2010 study by WestEd, many community-college students don’t receive clear information about what a school will expect of them academically or fully understand the course-placement process, including the assessment exams that can sign them up for semesters of noncredit remedial work. And not surprisingly, they don’t tend to seek help or prepare effectively for it. Another study found 70 percent of students did not study for placement tests, even though many of the schools—including SUNY Adirondack—offer study aids, such as free review classes, online tutorials, and other materials.

The day of the test, Newman-Heck panicked.

“It’s like four years of high-school English in one test … and on math … I didn’t know anything about algebra,” she said. The school told her the scores meant she would have to take remedial classes. “This is horrible!” she recalled thinking. “Should I even continue?” She persisted, because “I wanted to show my son I could do it,” she said.

Now, nearly three years later, she has taken out $15,000 in loans and requested a leave of absence from her current job as a security guard at an amusement park in order to devote more time to school. If Newman-Heck succeeds, it will take her at least four years to obtain a two-year degree.

“Why would they set you up to fail?” she said. “The college tells them, ‘Come on and come to our college’—and then when they don’t pass the placement exam, they get stuck with the bill.”

Looking to High School for Reforms

How can more students complete their community-college studies and earn a credential? Should reform start with American high schools, community colleges, or both?

Newman-Heck landed on one strand of reform when reflecting on her experience to date: “The system” should inform students while they’re still in high school what’s at stake if they’re struggling in math and English, and community colleges should tell applicants “how serious” placement exams are and help them study beforehand, she said.

That is one strategy becoming more popular at community colleges nationwide, including SUNY Adirondack, which has, in the last two years, begun collaborating with local high schools to allow students to take practice math-placement tests and classes to brush up their skills before graduation. Some states, including South Dakota and Florida, have made college-placement tests available to high-school students. In Florida, low-performing students are required to take a college-placement exam in 11th grade and complete remedial work in high school if they don’t pass.

Several states focused on reform at the high-school level are looking to support student preparedness based on their ACT scores. In Tennessee, K–12 teachers and college faculty have worked together to create math and English coursework for high-school seniors who take the ACT test as juniors and score below a certain point. The idea is for these students to catch up in their last year of high school, and begin college with courses that count toward a degree. In three years, the number of Tennessee students needing to take remedial math in college has dropped by 15.6 percent, according to the state’s records. The English program is still in its pilot phase. Both programs have yet to reach all the state’s students.

Redesigning Remediation

There’s another question at hand: whether the placement are effective measures of readiness.

A 2012 study noted that while great amounts of research had been done to determine the validity of ACT and SAT tests, much less scrutiny had been imposed on placement exams. Most studies of the then-dominant Compass, an ACT product, and ACCUPLACER, a product of the College Board, “have been conducted by the test makers themselves,” wrote researcher Judith Scott-Clayton. She concluded that using a combination of indicators, including high-school transcripts, would be more effective in sorting out students, leading to higher percentages of students reaching and passing college-level course work faster. ACT discontinued the Compass test in December 2016.

Other research has raised questions about the value of remedial classes, regardless of placement-test results. A study of three community colleges at the City University of New York (CUNY) found that students who were slated to take remedial classes based on their test performance did better and were more likely to persist in school if they were enrolled in credit-bearing classes right away (see “Reforming Remediation,” research, Spring 2017). And another study showed that as many as 18 percent of students referred to remedial math courses based on test results and 29 percent of students referred to remedial English could likely succeed in college-level courses.

About one in four community colleges nationwide is experimenting with how they assess and deliver remedial course work, according to Davis Jenkins, a senior research scholar at Columbia’s CCRC.

North Carolina and California are among the states backing away from using a single test to assess whether a student needs to take remedial courses. North Carolina has used a “multiple measures” approach since 2013, in which community colleges use ACT or SAT scores to determine placement. Students with a high-school GPA of 2.6 or higher and at least four years of math, including one course above Algebra II, are automatically placed into credit-bearing classes. A recent study found that students evaluated in this way earned a “C” or better in so-called “gateway” math and English courses at higher rates than students who had been evaluated by a college-placement exam.

Before the changes, “students were told, ‘You gotta do this placement test,’” said Lisa Chapman, chief academic officer for North Carolina’s 58 community colleges. “It was one thing checked off a list. And the problem was, students would go on a Monday and expect to enroll on Tuesday. A lot of times … information wasn’t shared with them so that they understood the entrance exam and what it meant.”

Other fixes include combining remedial reading and writing into a single “accelerated” course, by shortening two semesters into one, for example. At Chabot College in California, students in such a course were 29 percentage points more likely to enroll in college-level classes than others who were not. In math, a yearlong class called “Statway” combines remedial and college-level content. Among students at 18 community colleges that introduced the class, 48 percent completed credit-bearing math classes in a year, compared to 6 percent of students enrolled in remedial math.

Then there is co-requisite course work, in which students enroll directly in college-level courses and separate, supporting classes at the same time, occasionally taught by the same teacher. Tennessee has gained a lot of attention for results obtained in several pilots using this method since 2014, followed by a statewide launch. In the program’s first year, 51 percent of students in co-requisite math classes got credit for college-level math in their first semesters, compared to the previous tally of 12 percent reaching the same goal within a year of taking remedial course work. States including West Virginia, Indiana, and Colorado have reported similar successes.

An “Open” Question

These changes, however, don’t necessarily capture the tradeoff in offering open-access community colleges.

Several states have looked to a different sort of policy fix in boosting student success: outcomes-based funding. In Tennessee, 85 percent of base funding depends on a series of outcomes, including degrees and credentials earned, transfers, and the number of students who reach “momentum points,” such as completing remedial education or getting a job. The state has experienced an increase in certificate and associate’s degree completion rates in its community colleges, but how much of that is due to outcomes-based funding is unclear; it had already implemented changes in its K–12, assessment, and remedial course work when outcomes-based funding was introduced.

Many advocates approach such rules with caution. Some worry that community colleges under pressure to produce degrees would lower their standards. Others fear that schools would avoid enrolling students who graduate high school with very low academic skills, such as the aspiring college student who scores at a 6th-grade reading level. But is college even the best choice for that student?

This question bangs against the open-door credo at the heart of the community-college mission, and many advocates balk at the notion that some students may never catch up. “I would never go to the idea that ‘college is not the right choice’ for certain students, based on 50 years of experience,” said Kay McClenney, founding director of the Center for Community College Student Engagement (CCCSE) at the University of Texas at Austin.

Evelyn Waiwaiole, CCCSE’s current executive director, allowed that “maybe community college isn’t right for some people”—but she was careful to add a caveat that more research must be done, given the success of recent reforms. “Those who place three levels down … we’re not talking about them,” she said. “We haven’t done research on the lowest levels.”

One educator who is open to the idea is Jane Arnold, a veteran remedial English professor at SUNY Adirondack who has taught both Essor and Newman-Heck and is preparing to retire. Remedial courses can be very “frustrating,” she said.

“It’s frustrating for students, who waste so much time and don’t get the results they want. It’s frustrating for administrators, who are under pressure to graduate students faster … and tend to water down the curriculum, since they see students as customers who are always right,” she said. “But I’m not selling credits. I want to help students learn. So it’s frustrating for faculty as well.”

Helping students or adults with weak academic skills should involve a detailed review of possible options for educational and career advancement—something broader, yet more specific than opting in to community college as a default. Perhaps a student’s next step could be earning a technical certificate or completing a community-based program that leads to a job, she said.

Figuring out what those other options should look like may be the key piece of the puzzle to help students like Essor and Newman-Heck so that enrolling in a community college is truly a choice rather than the only viable path to advancement.

“We could be telling students, ‘We’re sorry—the system screwed up really badly, and you got here, but you’re not ready for this yet … It’ll take x amount of years to get your associate’s degree,” she said. “‘Or, there are these other options.’ … It doesn’t have to be community college.”

Timothy Pratt is an independent reporter based in Atlanta.

Kassidy Fann clears off the table in the tidy kitchen of the wooden triple-decker north of Boston that she shares with her husband and a pair of inquisitive cats.

Fann takes a seat and opens her laptop. There’s no sound but the creaking of the pipes and the rain against the windows. Then a disembodied voice with a slight southern accent says hello.

“I will be your proctor today,” says the voice, which belongs to a woman named Leigh Ann Majerik, who’s in an office in Hoover, Alabama. Majerik will supervise as Fann takes a test—called, in this case, an assessment—to gauge her progress toward the master’s degree she’s pursuing in science education and physics from the online Western Governors University.

Fann holds up her driver’s license and looks into a fisheye webcam so the proctor can confirm her identity and see that there are no unauthorized parties in the room other than one of the cats. After a few more clicks, the test pops up on Fann’s laptop screen, along with a countdown clock in the upper-right-hand corner.

“Start when you’re ready,” says the proctor.

It’s not just being tested in her kitchen toward a graduate degree that makes Fann something of a pioneer. It’s also how she’s being tested, and for what: her grasp of everything she’s learned so far, and not just some of it.

Unlike conventional colleges and universities, Western Governors doesn’t require students to spend a set number of hours in a classroom, average out their performance on assignments and tests, then hand out letter grades and credits. Using a complex system of assessments developed over the two decades the university has been operating, WGU’s method is competency-based, requiring that students prove they’ve mastered all the skills and knowledge offered in a given subject area. Until they do, they don’t advance.

“You have to pass. You have to meet the criteria. There’s no [grading] curve,” says Fann, a high-school science teacher who earned her undergraduate degree from Barnard College. “It does actually test whether I know this stuff or not.”

Fann not only likes this approach, she also thinks it could be applied to her 9th-grade physics students.

Although the model was originally designed for older-than-traditional-age college and graduate students, Fann believes “there’s nothing stopping” the principles of competency-based education and assessment from working in primary and secondary schools.

“I don’t know that all of my students would have the self-starter capabilities to succeed at this,” she says. But competency-based education, by definition, would let them progress at their own pace. Those who were ahead could push forward by working online; those who were behind could get more help from the teacher. At the end of the process, they would have to prove, just as Fann does now, that they understand the subject—not just some of it, but all of it. And they would have to demonstrate that mastery not to their classroom instructors but to separate and impartial assessment faculty.

“Every day my students ask me, ‘Is this going to be on the test?’” Fann says, shaking her head. “I don’t want to tell them. Here [at WGU], you actually have to know all the material.”

Like Fann, a growing number of people are thinking about how competency-based education and assessment might work in high schools and even lower grades. The Every Student Succeeds Act of 2015 opened the way for primary and secondary schools to experiment with this and other new kinds of testing, and states that include New Hampshire, Kansas, Maine, Arizona, Colorado, and Vermont are already giving it a try.

New Hampshire is the furthest along, with its Performance Assessment of Competency Education initiative, or PACE. Rather than counting how much time students spend in seats, this system tests whether they’ve met “learning targets,” requiring them to pass incremental assessments—demonstrating the skills they’ve acquired—in order to keep moving forward. Students in middle-school English, for example, write research papers showing they can analyze and present information from different sources. Fourth graders in math design a new park, estimate its construction cost, and produce a presentation arguing in favor of building it.

Competency-based education and assessment is relatively new to K–12 education, and at this point, there are not sufficient data or large-enough sample sizes to discern how well it works there. But in higher education the model has been used for more than 20 years, affording an opportunity to examine its track record and potential.

Competency-Based

A bipartisan collaboration from an era when such cooperation still happened, Western Governors University was jointly proposed in 1995 by the Republican governor of Utah and the Democratic governor of Colorado. It launched two years later with $100,000 from each of its 19 founding states, then struck out as an independent, nonprofit operation.

In spite of early skepticism toward online education—which persists, to some degree—Western Governors grew to 38,000 students by 2012. It is now in its 20th-anniversary year, and enrollment exceeds 80,000 undergraduate and graduate students in schools of education, business, health professions, and information technology (see Figure 1). It ranks fourth in the nation in the number of bachelor’s and master’s degrees awarded to nonwhite students in nursing, and awards 11 percent of bachelor’s and master’s degrees for teachers of technology, math, engineering, and science. Graduates hold top jobs at the likes of Aetna, American Express, Coca-Cola, Delta and United airlines, JPMorgan Chase, Pfizer, and Toyota. This year, the university debuted new degree programs in the hot field of data analytics.

But what Western Governors is best known for is its embrace of competency-based education.

Each of its degree programs includes a list of topics to be mastered, called “domains.” For a bachelor’s degree in accounting, for example, required domains include organizational behavior, business law and ethics, quantitative analysis, information technology, marketing and communications, systems administration, and the liberal arts. Accounting students must demonstrate that they understand such things as the nature and purpose of information systems and that they have mastered skills that include information systems auditing. These competencies are determined not by the faculty alone but in consultation with employers.

The point of involving employers in the process, says Scott Pulsipher, president of Western Governors, is “to develop a credential that has value in the jobs that [students] are pursuing.”

Students first take no-stakes “pre-assessments” to track their progress whenever they want (including at the very start of a course). If they fall behind, a mentor intervenes to catch them up on the material they don’t know. Advocates liken these pre-assessments to the PSAT many high-school sophomores and juniors take to practice for the SAT and discover where they still need work.

A dramatic departure from the conventional “seat-time” standard, competency-based assessment allows students to proceed at their own pace, a feature that’s attractive to both those who want to accelerate their degree program and for older-than-traditional-age students juggling schoolwork with families and jobs. (In the K–12 realm, more-advanced students can speed through, while others can take extra time if needed.)

Pre-assessments help students know when they’re ready to move on to “objective” assessments that determine mastery of the required domains. These assessments combine easy-to-score multiple-choice exams, such as the one Fann took in her kitchen under the remote eye of a proctor, with real-world “performance” assessments, including presentations and case studies graded by people whose only role in the process is providing this assessment. At the time of her exam, Fann had also just wrapped up an essay on thermodynamics, enhancing it with illustrations she created on her iPad.

If they don’t pass an assessment on the first try, students are usually asked to retake the pre-assessment before seeking permission from their faculty mentors to try again. (There are different forms of each assessment, so the questions are always different.) If they don’t pass on the second attempt, the process starts again, but with an additional $60 fee per test. More than 90 percent of students pass on the first or second attempt, WGU says.

The results of these evaluations are easier for third parties to interpret than a letter grade from a conventional class, say advocates of competency-based assessment. For instance, if a student gets a B- at a traditional university, it can be tough for another teacher, a college admissions officer, or an employer to know what he or she didn’t understand, or how that B- compares to the same grade given at a different institution.

“The feedback is durable,” says Kim Kostka, a chemistry professor at the University of Wisconsin, Rock County, who also teaches students in that university’s competency-based UW Colleges Flexible Option, or UW Flex. “In a traditional curriculum, let’s say a student passes the class but they have failed Unit Two. That just gets averaged into their final grades. What can I say about that student’s understanding of Unit Two? They didn’t demonstrate it. In competency-based education, you can’t average out a poor performance.”

Measuring Effectiveness

Western Governors preaches what it practices. It’s home to the online quarterly Journal of Competency-Based Education, hosts seminars on the topic for educators and policymakers, and, using data collected on its more than 80,000 graduates (see Figure 2), teams up with researchers from Harvard, Stanford, Carnegie Mellon, the University of Chicago, and other institutions who study competency-based education, or CBE.

Measuring the real-world effectiveness of CBE and its assessments is as complicated and contentious as it is essential; after all, the success of the approach depends on the acceptance of CBE credentials by licensing agencies, graduate schools, and employers. Competency-based education needs to earn external validation if it is to endure, pronounced a landmark paper from the Center on Higher Education Reform at the conservative-leaning American Enterprise Institute.

For Western Governors, its officials say, this largely means checking in on graduates and what they got from their education, to provide proof that—as the paper described it—a CBE credential “stands for a level of rigor and preparation equivalent to a traditional postsecondary degree.” The paper called a competency-based model “workable only insofar as its measures of learning yield trustworthy data about students’ prospects for future success.”

To test how well its methods work, Western Governors resorts to an unusual form of evaluation: a Gallup poll. It surveyed its alumni last year—2,676 of them, randomly selected, who graduated between 2000 and 2016—to ask if what they learned prepared them for life. Thirty-one percent responded that it did, which might seem low, except that only a surprising 26 percent of graduates nationally said this, 24 percent from public universities and 20 percent from for-profit universities.

In a separate Harris poll of 1,207 WGU graduates and 1,403 graduates of other colleges nationwide, also conducted last year, 78 percent of Western Governors grads said what they learned was directly related to their work, compared to 68 percent of the other graduates. Ninety-nine percent of employers said WGU graduates met or exceeded expectations, and 100 percent reported that the grads were prepared for their jobs. A Gallup poll conducted for the Lumina Foundation, which promotes increased access to higher education, found that just 11 percent of business leaders said they were getting the skills they needed from the college graduates they hire overall.

Another way of measuring WGU’s effectiveness is through student pass rates on professional licensing exams. Western Governors says its students pass teacher certification tests administered by Pearson Education at a rate of 96.2 percent, and that they pass the Educational Testing Service’s Praxis teacher tests at a rate that’s 3.8 percentage points above the national average. (The average national pass rate on the Praxis is difficult to calculate, because ETS provides such information largely on an institution-by-institution basis, and passing scores vary by state; some education colleges do better than Western Governors, some not as well.) Nearly 90 percent of WGU nursing graduates passed licensing exams last year, which is slightly higher than the national rate of 87.8 percent.

The university uses this kind of data to occasionally tweak its approach. “It’s really a continuous-improvement culture,” says Jason Levin, WGU’s vice president of institutional research. For example, although its model relies heavily on student self-direction, the university has added mentors from whom students can seek help during their course of study, and who check in with students if they fall off track. “We want to be a self-paced institution and we don’t want to get in the way of students who come with competencies and learn quickly,” Levin says, “but the reality is we have a wide range of ability and experience in our students.”

Western Governors also toughened its pre-assessments and has added an online coaching report, giving its students real-time looks at how they’re faring. The latter feature resulted in a nearly 4 percent improvement in the number of students who pass the final assessments, which is gradually helping the university raise the proportion of students who graduate within six years above the current 41 percent. (The six-year graduation rate from colleges and universities nationwide is 52 percent, though Western Governors compares itself to open-admission institutions, for which the rate is 36 percent.)

But some of the most noteworthy innovations have been built into the process since the outset, such as the practice of designing competencies in collaboration with employers. Western Governors convenes “program councils” composed of industry experts who advise on each of its colleges, on general education, and on assessment. These councils meet at least twice a year at WGU’s Salt Lake City headquarters, and members are paid stipends and compensated for expenses.

“We had a lot of concerns around the curriculum and the matching of the curriculum itself to what employers were looking for in the graduates,” says WGU’s Pulsipher, who previously worked at Amazon and chaired a technology industry advisory group at the school of management at his alma mater, Brigham Young University. There, he says, “it was basically academics competing with employers saying, ‘I get what you want to teach, but we’re trying to tell you that what you’re teaching doesn’t map with what we need.’”

Surveys bear out that perspective. In the Gallup poll of business leaders done for Lumina, only a third agreed that “higher education institutions in this country are graduating students with the skills and competencies my business needs,” compared to 96 percent of chief academic officers who thought so.

Other programs using CBE also enlist the advice of industry leaders. Srirajasekhar Bobby Koritala, CEO of a Naperville, Illinois, software company, was one of the businesspeople invited to help plan out the competencies for UW Flex. “I frankly never had any college ask me that before,” he says. “I found it refreshing. In some ways, employers are somewhat disconnected from colleges.”

He’ll get no argument about that from the Reverend Dennis Holtschneider, president of DePaul University from 2004 through June of this year. DePaul’s School for New Learning uses competency-based education and a mix of online and in-person courses in its degree programs for older-than-traditional-age students. The university has contracts with local businesses and public agencies, from the Chicago Police Department to Fifth Third Bank, to enroll their workers, and can ask these employers “exactly what they need” from their employees, Holtschneider says.

Atypical Assessment

In another significant departure from traditional education practice, Western Governors and institutions like it deploy separate groups of faculty for different areas of responsibility: establishing competencies, teaching, and testing. That means the teaching faculty are on trial in assessments, too, a complete upending of a long-held classroom culture in which instructors evaluate the knowledge of their own students.

“You don’t want the fox guarding the hen house,” Pulsipher says. “The design of [the assessment component] was important, because part of the integrity of a competency-based model says there are standards for proficiency … If you take assessment out of the hands of the teacher, I would argue there’s a higher level of integrity there.”

Separating teaching from assessment raises the stakes for instructional faculty, Pulsipher says. “They want to know that their students are ready to take the assessment.” That’s because a faculty member who has a cohort of students not doing very well will draw scrutiny from managers to see what he or she is doing differently, he notes. (A years-long audit is underway by the U.S. Department of Education’s inspector general over whether the Western Governors teaching faculty spend enough time with their students, though the question is principally a technical one over whether WGU is a distance-education program or a correspondence school for purposes of eligibility for federal financial aid.)

Having some faculty teach and others test “helps our faculty think really clearly about outcomes of their work,” says Holtschneider. “It introduces accountability when someone else is going to evaluate your students and what they’ve learned.”

Cynthia Suopis likens the approach to serving as a doctoral-student adviser. “You’re advising that student through the dissertation, but then it has to get through the committee,” says Suopis, who serves on the faculty of the University Without Walls at the University of Massachusetts Amherst, which follows the same model. UWW alumni include basketball great Julius “Dr. J” Erving and Jeff Taylor, the founder of Monster.com.

That “committee,” at WGU and similar programs, is a completely separate team of assessors. This firewall between instruction and assessment theoretically eliminates the bias that can occur when classroom teachers grade their own students.

With its enrollment swelling, Western Governors now has 900 faculty evaluators, up from 250 as recently as 2010, says Debbie Fowler, the university’s associate provost. A quarter of them are full time and the rest are part time and paid by the hour. Most work remotely. All have graduate degrees in the subject matters they assess—either master’s or doctorates, depending on the level of the course. (Just under a third have doctoral degrees, Fowler says.) Together, they score 98,000 performance evaluations a month, with individual evaluators returning them in an average of just under 40 hours.

The evaluators are on constant trial, too. Using pass-rate data, WGU looks at evaluators who might be outliers among the rest, because “we don’t want the student’s chance for passing to be determined by which evaluator is picked for the submission,” Fowler says. While some evaluators may be legitimately more accepting of a test response that the designers didn’t foresee, “we instill a real culture of collaboration and calibration, and that can sometimes be uncomfortable, because it’s not every faculty member doing their own thing. We’re all in this together and it’s a team determination.” As for the students, there’s no gray area: they need to correctly master all of the competencies to advance.

WGU won’t disclose how much all of this costs, contending that the answer to that question is proprietary. As a nongovernmental nonprofit, it doesn’t have to report such level of detail. But publicly available documents from 2015 show that those 900 full- and part-time evaluators make up more than a quarter of the university’s 3,517 employees, and employee salaries and benefits accounted for more than half of the organization’s $355 million in annual expenses. Whatever the total price of the elaborate assessment process, the average cost to students of a WGU bachelor’s degree is a comparatively low $15,000. In the academic year that just ended, the average annual resident tuition at public four-year universities was $9,650; for nonresidents, $24,930; and at private, nonprofit universities, $33,480, according to the College Board.

Despite the scale of this assessment model—in fact, because of it—Western Governors captures efficiencies, Fowler asserts. For example, if classroom faculty had to conduct and grade assessments in addition to teaching, she says, they’d need to spend more time and be paid more. Separating these roles means that faculty become highly skilled at their specific jobs and “can perform more effectively and more efficiently,” according to Fowler.

But while this assessment model and the use of online education seem to add up to potential savings for students, developing a first-rate CBE program is not cheap, higher-education leaders say.

“Recognize that you do need to invest a great amount of money in doing this right,” says Cathy Sandeen, chancellor of the University of Wisconsin Colleges and University of Wisconsin-Extension, which includes UW Flex.

Sandeen estimates that the university system has spent close to $1 million on developing each of its direct-assessment competency-based programs. “It’s a lot of work to identify those competencies . . . and then how to design the assessments for [them]. It’s not something to go into just because it’s the next shiny object. You have to put some thought into it. Remember MOOCs?” she says, referring to the massive open online courses into which many universities hurdled at full speed, only to find that their moneymaking potential had been overstated.

“This is not easy. It is not cheap. There are high startup costs as well as high operating costs,” echoes Holtschneider.

One of those costs could be for specialized professional development. Few teachers have been trained to work in settings tailored to individualized learning, says Neal Kingston, a professor of educational psychology and director of the Achievement and Assessment Institute at the University of Kansas, who is in the midst of a three-year project studying these issues. “If what you’re used to doing is preparing one lesson for today’s class, but kids are all over the place, that doesn’t work,” he says.

Pulsipher says Western Governors looks at adapting its model for use in lower grades “on a pretty consistent basis.” Secondary-school districts and others, he says, have already asked the university for help adopting it. But so far, he’s said no.

“The short answer is, the timing is: not yet,” Pulsipher says. “We would only want to partner with people who truly have the commitment to stick with this over the long term.”

He uses Western Governors’ own beginnings as an example of this.

“For the first five to ten years, there were a lot of naysayers, a lot of skepticism, a lot of challenges to be overcome. You have to have a high, high, high level of commitment … enough confidence over the long term that you’re willing to be misunderstood in the short term.”

Jon Marcus is higher-education editor at the Hechinger Report, a foundation-supported nonprofit based at Columbia University, and North America correspondent for the (London) Times Higher Education magazine.

Brown v. Board of Education, the landmark 1954 decision barring school segregation, is a cornerstone of the American legal tradition. After more than a half century, however, its precise meaning remains contested. While conservatives view Brown as prohibiting the government from using racial classifications except in extraordinary circumstances, liberals believe the ruling leaves ample room for elected officials to take race into account when seeking to promote equal opportunity. Which interpretation prevails will continue to determine the extent to which public colleges can use race as a factor in admissions decisions, as well as the scope of school districts’ efforts to create more integrated schools and classrooms.

In this issue’s forum, legal scholars Shep Melnick and James Ryan examine this debate through the lens of the education rulings of the late Supreme Court justice Antonin Scalia. Melnick argues that Scalia’s conservative reading of Brown has solid roots in the text of the U.S. Constitution and usefully prohibits judges from imposing their policy preferences on the nation’s schools. Ryan, meanwhile, contends that Scalia’s rulings reveal the extent to which the court’s most famous “originalist” was willing to depart from his principles in order to strike down policies he found objectionable. At stake in their debate is nothing less than this question: Is our Constitution colorblind?

 

Equal Protection Bars Racial Favoritism
by R. Shep Melnick

 

 

 

Choosing Judicial Activism over Originalism
by James E. Ryan

 

 

 

In his 30 years on the Supreme Court, Justice Antonin Scalia wrote surprisingly few opinions in education cases, and even when he did, he seldom mentioned education. Instead, he focused on issues such as standing, techniques of statutory interpretation, the meaning of the First Amendment, and the importance of judicial restraint. Scalia believed his job in education cases was to read and apply the text of the law, and not allow his personal views on education to come in through the backdoor via free-ranging interpretations of vague statutory and constitutional provisions.

This set him apart from his more-liberal colleagues, who viewed Brown v. Board of Education (1954) not as a prohibition on the use of racial classifications in education, but rather as a mandate for judges to do whatever they could to promote “equal educational opportunity.” Judges who embrace this understanding of Brown and equal protection feel compelled to listen to the “experts” on educational inequality and to use their judicial authority to remedy injustices. Scalia, in contrast, favored a colorblind interpretation of the equal protection clause, that, in his words, “proscribes government discrimination on the basis of race, and state-provided education is no exception.”

Political Jurisprudence?

In his companion essay for this forum, James Ryan maintains that Scalia’s defense of judicial deference is fraudulent. There are many who share this view. Behind Scalia’s “originalism” and “textualism,” they claim, lies a conservative political point of view. In two key respects these critics are right. Scalia’s interpretive method is political in that it rests on an understanding of the proper operation of the political institutions of a liberal democracy. And it is conservative in the sense that he believed our public institutions (including our education system) are basically healthy, and should not be subjected to frequent rounds of reform by unelected judges and self-appointed experts. The key question here is the soundness of these political judgments.

It is fair to say that Scalia was relatively content with the way we have traditionally organized education in this country—or at least less critical of it than his more-liberal brethren. Until relatively recently, most education decisions and funding have been within the purview of local government. Local control of public schools combined with the availability of private schools promotes both choice and experimentation. The major flaw in this system—de jure racial segregation—has been ended. Critics rightfully note that this decentralization allows many forms of inequality to persist. But it is difficult to eliminate these inequalities without producing a stultifying uniformity and reducing voters’ control over education.

There are undoubtedly many ways our education system can be improved. Scalia saw such efforts not as the job of judges following the abstract theories of academic experts but of elected officials and the administrators appointed by them. Judges, he believed, should focus on establishing a few simple rules about what is legally permissible and what is forbidden. The rule of law, Scalia emphasized, is the law of rules. Judges should therefore look for rules that curtail the worst abuses rather than try to micromanage public schools.

Critics of Scalia’s originalism claim that this approach to constitutional interpretation exaggerates the extent to which we can understand the intentions of those who wrote the original Constitution or the Fourteenth Amendment. Scalia recognized that “it is often exceedingly difficult to plumb the original understanding of an ancient text” and to apply that understanding to contemporary issues. But for him that difficulty provided yet another argument for judicial restraint.

The primary purpose of originalism, Scalia argued, is to dissuade judges from reading their personal understandings of what is fair, good, and just into the vague phrases of the Constitution. When the Constitution is clear—for example, when it says that a state can deprive a person of “life” so long as it provides “due process,” or when it gives those accused of crimes the right to “confront” their accusers—judges need to follow those commands. Where the Constitution is ambiguous, “this Court has no business imposing upon all Americans the resolution favored by the elite class from which the Members of this institution are selected.”

Scalia did not maintain, though, that the court should simply overturn decisions that have become embedded in our law and practices, however mistaken those decisions may have been. He usually adhered to the doctrine of stare decisis—respect for precedent. His approach looked not just to “text,” but to “tradition” as well. Regarding race and the equal protection clause, Scalia’s combination of text and tradition culminates in a simple rule: no governmental use of racial classifications except in extraordinary circumstances.

Brown, Green, and Colorblindness

For an originalist, Brown presents a serious problem. On the one hand, it has become a fundamental element of our legal and political culture. The authority of its central argument, that “separate is inherently unequal,” is now firmly established. On the other hand, it is far from clear that the original supporters of the Fourteenth Amendment believed that it prohibited school segregation. To advocates of a “living Constitution,” this lack of clarity is liberating: it frees them to do anything they think appropriate for promoting equal educational opportunity. For Scalia, in contrast, the challenge was to provide a solid constitutional foundation for Brown without empowering judges to wield it as a mandate to remake our schools.

Scalia claimed that the Fourteenth Amendment does provide support for school desegregation. In Rutan v. Republican Party (1990), he wrote that “the Fourteenth Amendment’s requirement of ‘equal protection of the laws,’ combined with the Thirteenth Amendment’s abolition of the institution of black slavery, leaves no room for doubt that laws treating people differently because of their race are invalid.” Scalia focused on the general understanding of the terms “equal protection of the laws,” “due process of law,” and “privileges and immunities” in the late 1860s and thereafter. Although he could not prove there was broad support for prohibiting de jure segregation in 1868, he did show that there was a clear and vibrant tradition in case law that viewed the use of racial classifications by government as pernicious, particularly because such a practice is so susceptible to the tyranny of majority faction. This position was presented most forcefully in Justice John Marshall Harlan’s well-known dissent in Plessy v. Ferguson (1896). And Congress, through its voting patterns during the 1860s and 1870s, expressed a similar opposition to racial classifications, as the constitutional scholar Michael McConnell has demonstrated.

That “colorblind” interpretation of the Fourteenth Amendment was endorsed by the NAACP lawyers who brought the long string of cases culminating in Brown. “That the Constitution is color blind is our dedicated belief,” they wrote in their 1953 brief. And in his arguments before the court, chief counsel Thurgood Marshall maintained that the Fourteenth Amendment denies states the authority “to make any racial classification in any government field.”

This was also the understanding of the presidents who proposed the Civil Rights Act of 1964 and the members of Congress who voted for it. That seminal law explicitly states that “desegregation” means the assignment of students to schools “without regard to their race, color, religion, or national origin,” and shall not be interpreted to mean “the assignment of students to public schools in order to overcome racial imbalance.” As President John F. Kennedy put it a few months before his death, “I think it would be a mistake to begin to assign quotas on the basis of religion, or race, or color, or nationality. I think we’d get into a good deal of trouble.”

In an important sense, Ryan is right to claim that Scalia’s embrace of Harlan’s colorblind interpretation of the equal protection clause is “results-oriented.” Scalia was above all concerned with the political consequences of allowing public officials to use racial classifications. Indeed, it would be hard to avoid addressing a question of this magnitude without thinking about the long-term consequences of competing interpretations. Here, Scalia quotes from the constitutional scholar Alexander Bickel, who argued that a “racial quota derogates the human dignity and individuality of all to whom it is applied; it is invidious in principle as well as in practice.” A quota, Bickel charged, is a “divider of society” and a “creator of castes” that “can easily be turned against those it purports to help.” Given the dangers inherent in the use of racial classifications, Scalia maintained, we should take this tool out of the hands of public officials, even if they claim to use it for “benign” purposes.

In his opinion for the court in Parents Involved v. Seattle School District (2007), Chief Justice John Roberts illustrated Bickel’s point. Roberts noted that according to the rules the Seattle School Board had established to promote “diversity” in its schools, “a school that is 50 percent white and 50 percent Asian-American . . . would qualify as diverse,” but “a school that is 30 percent Asian-American, 25 percent African-American, 25 percent Latino, and 20 percent white . . . under Seattle’s definition would be racially concentrated.” Especially at exam schools, boosting admissions for some groups comes at the expense of other groups—usually Asian Americans, who have also faced harsh discrimination over the course of American history. In Kansas City in the late 1980s and early ’90s, African American parents were justifiably irate when the federal court’s integration plan denied their children access to the magnet schools of their choice because so many seats had been set aside for white children—who did not show up in sufficient numbers to fill them. This shuffling of students on the basis of race reinforces racial thinking without providing countervailing benefits. Assertions of “benign” intent hardly ensure that public policies will not have perverse consequences.

Until the early 1970s, no one other than segregationists challenged the colorblind interpretation of the equal protection clause and Brown. This changed in a flurry of Supreme Court decisions on school desegregation, most importantly, Green v. New Kent County (1968) and Swann v. Charlotte-Mecklenburg Board of Education (1971). Justice William Brennan’s opinion for a unanimous court in Green set the stage for large-scale busing. It required school districts that had previously maintained a “dual” school system to take all steps necessary to convert to a “unitary” school system, one in which no schools are “racially identifiable,” because the enrollment of each school reflects the racial balance of the school district as a whole. District court judges took this to mean that desegregation orders must be revised on a regular basis to ensure racial balance. This practice continued for decades.

In two 1992 cases, Freeman v. Pitts and U.S. v. Fordice, Scalia addressed the question of how long such efforts at racial balancing must last. His principal purpose was to distinguish the extraordinary measures necessary for dismantling Jim Crow in the 1960s and 1970s from the “ordinary principles of our law, of our democratic heritage, and of our educational tradition.” He maintained that “plaintiffs alleging equal protection violations must prove intent and causation and not merely the existence of racial disparity,” and that “public schooling, even in the South, should be controlled by locally elected authorities acting in conjunction with parents, and that it is desirable to permit pupils to attend schools nearest their homes.” For Scalia, the proper response to a mistaken or outmoded precedent was not necessarily to overturn it but to stop expanding it, narrow it whenever possible, and thus “revert to the ordinary principles of our law.”

Ryan and other defenders of “benign” racial sorting, in contrast, insist that the use of remedies originally available only to judges charged with dismantling an entrenched racial caste system in the South should also be available to public officials presiding over school systems that have not violated the Fourteenth Amendment. The fullest presentation of this point of view is Justice Stephen Breyer’s impassioned dissent in Parents Involved. It is notable that Breyer never quoted from Brown, but only referred to its “hope and promise.” The main support for his position came from academic studies and Chief Justice Warren Burger’s opinion in Swann. Ryan, too, places much weight on the Swann opinion. That is a strange choice, given that it is among the Supreme Court’s most poorly constructed and internally contradictory opinions. Swann was the product of a complicated effort to extract a unanimous ruling from a deeply divided court.

In the end, the argument of Breyer and Ryan boils down to the claim that by using potentially dangerous racial classifications we can produce racially integrated schools that improve the educational opportunity of minority students. How do we know this? The experts tell us so, or, as Justice Clarence Thomas pointed out, not all the experts, just those Breyer chose to cite. Breyer “unquestioningly” relied upon “certain social science research to support propositions that are hotly disputed among social scientists.”

Can Brown be reconciled with a full-throated, doctrinaire understanding of originalism? Probably not. For that reason, no one endorsing that form of originalism has sat on the Supreme Court since 1954, and none are likely to be appointed in the future. But Antonin Scalia considered himself a “faint-hearted originalist” who saw Brown as part of a long and noble tradition that had been explicitly endorsed by Congress and the president in 1964 and that had since become deeply embedded in our political culture. At its heart lies a simple rule—no use of racial classifications except to remedy specific constitutional violations—that does as much to constrain as to empower judges. This rule might not lead us to the best possible education outcomes, but it prevents the worst type of abuses. Having unwisely expanded exceptions to the colorblind rule, Scalia argued, the court should now return to the original understanding of Brown.

This essay is abridged from a chapter in the forthcoming volume Scalia’s Constitution: Essays on Law and Education, edited by Paul E. Peterson and Michael W. McConnell, Palgrave Macmillan, 2018.

This is part of a forum on Antonin Scalia’s record on race and education. For an alternate take, see “Choosing Judicial Activism over Originalism” by James E. Ryan.

Justice Antonin Scalia was a staunch proponent of “originalism” in constitutional jurisprudence, an approach to deciding cases based on constitutional text as it was originally understood by its authors. Although he would occasionally follow precedent instead of the original understanding of constitutional text, Scalia argued that, in general, originalism was the only principled way for judges to avoid enshrining their own policy preferences into the law. He chastised those who instead believed in a “living” Constitution, which Scalia argued was simply a rationalization for “results-oriented” judges to decide cases however they chose.

In some high-profile cases, Scalia followed originalism even when it led to results that he almost certainly did not favor as a matter of policy. In Texas v. Johnson (1989), for example, Scalia joined the majority in striking down laws prohibiting the desecration of the American flag, an act he despised but that he nevertheless concluded was protected by the First Amendment.

But his education cases—particularly those relating to the use of race in student-assignment and admissions policies in K–12 and higher education—paint a different picture. In these cases, Scalia was faithful neither to originalism nor to precedent. Indeed, these cases show a justice who seemed just as results-oriented as the judges and justices he scolded.

Three cases in particular illustrate this point. The first two are Gratz v. Bollinger (2003) and Grutter v. Bollinger (2003), companion cases out of Michigan, in which the court struck down the University of Michigan’s affirmative-action plan for undergraduate admissions but upheld the law school’s admissions plan. The undergraduate plan used a numerical formula for considering race in admissions decisions, while the law school policy considered race as an undefined factor among many criteria. Scalia wrote a separate opinion in both cases, agreeing with the decision to strike down the university’s undergraduate-admissions plan and disagreeing with the decision to uphold the law school’s plan. The third case is Parents Involved v. Seattle School District (2007), in which the court limited the ability of school districts to explicitly consider race when attempting to integrate schools. Scalia did not write separately but joined in full the plurality opinion authored by Chief Justice John Roberts.

“Faint-Hearted Originalist”

I will start with Gratz and Grutter. Scalia’s view of affirmative action, generally, was that it was impermissible except to remedy a specific, identifiable harm. As many commentators have convincingly explained, it is difficult to square this position with the original understanding of the Fourteenth Amendment’s equal protection clause (see Eric Schnapper’s “Affirmative Action and the Legislative History of the Fourteenth Amendment”; Michael Klarman’s “Brown, Originalism, and Constitutional Theory”; and Jed Rubenfeld’s “Affirmative Action”). The conventional view among historians appears to be that the equal protection clause, as originally understood, would not have prevented states from helping African Americans as opposed to hurting them. There are historians who would disagree, undoubtedly, which makes Scalia’s failure even to engage in the historical debate genuinely puzzling. Remarkably, in some of the most high-profile cases the court ever heard—namely, cases involving race-based affirmative action—Scalia never offered an originalist defense of his views.

Scalia instead made simple, bare assertions about the meaning of the Constitution. A good example comes in his last line in Grutter, where he asserts that “the Constitution proscribes government discrimination on the basis of race, and state-provided education is no exception.” But this statement is oversimplified and inaccurate, and it fails to acknowledge that what counts as unconstitutional “discrimination” is a complicated question. There is no provision in the Constitution that explicitly prohibits any and all consideration of race; there is simply the equal protection clause in section one of the Fourteenth Amendment, which does not say a word about race. That clause vaguely says that no state shall deny to any person “the equal protection of the laws.” This principle is not self-enforcing but requires choices about the kinds of “discrimination” that are allowed and the kinds that are not. For instance, people who violate speeding laws are fined or arrested, while law-abiding drivers are not. The two groups are treated differently, but no one would ever think this sort of “discrimination” is prohibited, because it is obviously justified.

One would expect an originalist like Scalia to abide by the choices contemplated and understood in 1868, when the Fourteenth Amendment was ratified. In other words, an originalist would proscribe the sort of discrimination that was originally understood to be prohibited by the equal protection clause and tolerate the rest. As mentioned above, most historians appear to believe that the original, common understanding of the clause is that it permitted various kinds of discrimination on the basis of race, including “discrimination” in favor of African Americans. One might therefore suppose that Scalia would have no problem with affirmative action, even if he personally considered it bad policy. This makes it even more surprising, and disappointing, that Scalia never attempted to defend his decision from an originalist perspective. And it raises the obvious question: why abandon originalism in this context?

Scalia once famously remarked that he was a “faint-hearted originalist,” meaning in part that he would sometimes forsake originalism in order to obey the command of stare decisis, that is, he would follow established precedent. In Grutter, however, Scalia ignored the original understanding of the equal protection clause not to follow precedent but to break from it. Rather than abide by the precedent of Regents v. Bakke (1978), which allowed for affirmative action within certain constraints, Scalia expressed categorical opposition to race-based affirmative action. In Grutter then, Scalia appears to have abandoned both originalism and precedent to arrive at his position.

That position, moreover, was entirely consistent with his stated “policy” views about affirmative action. Scalia was not a fan of race-based affirmative action, as he had spelled out publicly (and sharply) before becoming a justice. One can agree or disagree with that policy position, of course. But to ignore originalism and break from precedent to reach a result that is consistent with a personal policy preference is difficult to defend as legally principled. This is not to say, of course, that Scalia’s view of affirmative action policy was itself unprincipled; reasonable people can and do disagree on the legal, moral, and practical merits of affirmative action. But in cases like Texas v. Johnson, Scalia remained true to his legal principles and struck down an anti–flag-burning law that, as a matter of policy, he obviously favored. Why he seems to have abandoned those principles when it came to affirmative action remains a mystery.

Voluntary Integration

One sees a similar approach in Parents Involved, which presented the question of whether K–12 schools could take voluntary steps toward integration, that is, whether and when schools could consider race in student assignments. In Parents Involved, Scalia joined the plurality opinion of Roberts, who took the categorical view that race can never be taken into account, even when districts are trying to integrate schools rather than segregate them. (“The way to stop discriminating on the basis of race,” Chief Justice Roberts wrote, “is to stop discriminating on the basis of race.”) Roberts argued that this position was commanded by the Constitution and was consistent with Brown v. Board of Education (1954), which in Roberts’s view was not about school integration but about prohibiting any use of race in school assignments, regardless of the purpose.

Here again, one finds Scalia willing to abandon originalism not to follow precedent but to break from it (or at least to make new law). Critics of originalism such as Michael Klarman have pointed out that Brown is difficult to justify on originalist grounds, as there is little evidence that the equal protection clause was originally understood to outlaw school segregation. If Brown cannot be justified on originalist grounds, some scholars contend, then originalism should be rejected, because Brown is a seminal case whose outcome has overwhelming support in our society—both legally and morally. Over time, Scalia responded in different ways to this contention, sometimes suggesting that critics were right about Brown but wrong in concluding that it discredited originalism, at other times suggesting that Justice John Marshall Harlan’s dissent in Plessy v. Ferguson (1896)—arguing for a colorblind constitution—captured the correct original understanding. Neither of his responses, however, justifies his position on voluntary integration.

To begin, many legal historians and constitutional scholars seem to agree that the equal protection clause, as originally understood, did not prohibit segregation, because integration—including integrated schools—involved a “social” right, not a civil right, and therefore fell outside the ambit of that clause. Scalia, as mentioned above, sometimes seemed to accept this argument and agreed that the original understanding of the equal protection clause could not justify the outcome in Brown. But in his view, it did not follow that originalism should be rejected out of hand. Yet if the equal protection clause does not apply to school segregation, it obviously would not prohibit the voluntary integration of schools, either. Under this view, states would be free either to segregate or integrate.

On the other hand, if school segregation was indeed incompatible with the original understanding of the equal protection clause, there are only two possible rationales for this view. The first is that the equal protection clause was actually intended to prohibit the perpetuation of a caste system, and that school segregation was obviously attempting to perpetuate a racial caste system. Attempts to break down that system—whether through courts or legislatures—would then be consistent with the original understanding. School segregation would be prohibited, but school integration would be tolerated—indeed, encouraged.

The second possible rationale is the notion that the equal protection clause requires colorblindness and prohibits any and all uses of race. The argument for colorblindness in this context, however, is no different from the argument used against race-based affirmative action. As mentioned, most historians seem to believe that argument is false and, again, Scalia never tried to make the originalist case that race cannot be taken into account even when the government seeks to help, not hurt, African Americans. Harlan’s dissent in Plessy, despite his rhetoric about a colorblind Constitution, hardly settles the issue; indeed, Harlan himself indicated in another opinion and in his extrajudicial writings that he believed school segregation was constitutional.

No matter how you approach it, then, when it came to voluntary integration, Scalia abandoned what a commitment to originalism would appear to require. He did not do so, moreover, in order to follow the clear command of precedent. First, the idea that voluntary integration is inconsistent with Brown, which Roberts suggested in the plurality opinion Scalia joined, is implausible. Brown dismantled state-enforced segregation with the expectation that doing so would lead to integrated schools. The whole thrust of Brown was that segregation was actually harmful to students, not that the use of race itself was always and everywhere to be rejected. In addition, the idea that legislatures would take voluntary efforts to integrate schools would have seemed far-fetched at the time of Brown. The related idea that the justices who voted in Brown, or the lawyers who argued against segregation, would have had objections to voluntary efforts to integrate seems equally implausible.

Just how implausible is demonstrated in Swann v. Charlotte-Mecklenburg (1971), the second precedent rejected by Roberts and Scalia. In that case, the court approved the use of busing to desegregate schools under court order. Writing for a unanimous court, Chief Justice Warren Burger explained that the case involved the limits of judicial authority, and he sought to distinguish the scope of judicial authority from the authority of school officials. In a telling passage, he wrote:

School authorities are traditionally charged with broad power to formulate and implement educational policy, and might well conclude, for example, that, in order to prepare students to live in a pluralistic society, each school should have a prescribed ratio of Negro to white students reflecting the proportion for the district as a whole. To do this as an educational policy is within the broad discretionary powers of school authorities; absent a finding of a constitutional violation, however, that would not be within the authority of a federal court.

Roberts did not have a convincing explanation as to why the court would have made this plain statement in a unanimous opinion if it were not obvious to the court in 1971 that this was the proper understanding of Brown. And it was this statement, as much as anything, that was behind Justice John Paul Stevens’s observation in his dissenting opinion in Parents Involved that no member of the court he joined in 1975 would have agreed with Roberts—and Scalia—that explicitly race-based voluntary integration is categorically prohibited.

It remains a puzzle why Scalia was willing to abandon originalism in some cases but not in others, even when, as in the flag-burning case, the originalist approach led to outcomes that he almost certainly disfavored personally. It also necessarily raises the question of whether Scalia’s commitment to originalism was principled, strategic, or a bit of both. At the very least, these three politically charged education cases complicate the picture of a jurist best known for a methodology—originalism—that was strikingly absent in these contexts.

This essay is abridged from a chapter in the forthcoming volume Scalia’s Constitution: Essays on Law and Education, edited by Paul E. Peterson and Michael W. McConnell, Palgrave Macmillan, 2018.

This is part of a forum on Antonin Scalia’s record on race and education. For an alternate take, see “Equal Protection Bars Racial Favoritism” by R. Shep Melnick.