This article appeared in the Spring 2021 issue of Education Next. Suggested citation format:

Peterson, P.E. (2021). To Critics of The Beautiful Tree, a Pearl of a Reply: Was that test on which the experiment depends taken in English or Telugu? Education Next, 21(2), 78-79.

In this journal, as in others, scientific evidence is regularly invoked in defense of one classroom practice or another. And on occasion, scientific evidence features prominently in federal education policy. It had a star turn in the 2002 No Child Left Behind Act, which used the phrase “scientifically based research” more than 50 times, and an encore in the 2015 Every Student Succeeds Act, which requires that schools implement “evidence-based interventions” and set tiers of academic rigor to identify programs by their proven effectiveness.

Yet teachers, for the most part, ignore these studies. Why?

There’s research about that, too. First, teachers may view research as somewhat removed from the classroom, with further translation needed for the practice to be ready to implement in a live setting. Second, teachers may judge a practice to be classroom-ready in general but delay implementation because their particular students and setting seem significantly different from the research context. Third, teachers may resist trying something new for reasons unrelated to its effectiveness—because it seems excessively demanding, for example, or because it conflicts with deeply held values or beliefs about what works in the classroom. Finally, teachers may be unaware of the latest research because they only rarely read it.

No matter the reason, it seems many teachers don’t think education research is directly useful to them. We think these teachers have it right. And we think the problem lies with researchers, not teachers.

The first three obstacles listed above—two concerning applicability of research and one concerning perceived constraints research puts on practice—are products of the methods researchers use. Research seems irrelevant to practitioners because it does not pose questions that address their needs. Teachers feel constrained by research because they feel pressured to use research-approved methods, and research creates clear winners and losers among practices that may be appropriate in some contexts but not others.

The root of these issues lies in two standard features of most studies: how researchers choose control groups and researchers’ focus on finding statistically significant differences. The norm in education research is that, for a finding to be publishable, the outcomes of students receiving an intervention must be noticeably different from the outcomes of an otherwise similar “control” group that did not receive the intervention. To show that an intervention “works,” you must show that it makes a positive difference relative to the control. But are such comparisons realistic, reasonable, or even helpful for teachers?

No—but they could be. Here’s how.

Better Than Nothing Is Not Enough

Let’s consider the hypothetical case of CM1, a new method of classroom management meant to reduce the frequency of suspensions. Suppose we recruit eight schools to join an experiment to assess the effectiveness of CM1. We randomly assign teachers in half of the participating classrooms to implement it. We could then compare the rate of suspensions from students in those classrooms to the rate observed in the classrooms that are not implementing CM1. This type of comparison is called “business as usual,” because we compare CM1 to whatever the comparison classrooms are already doing. A similar choice would be to compare the rate of suspensions before CM1 is implemented to the rate after it’s implemented within the same schools. This “pre-post” design is comparable to the business-as-usual design, but each school serves as its own control.

If suspension rates are lower with CM1, we can conclude that it “worked.” But with a business-as-usual control group this conclusion is weak, essentially that “something is better than nothing.” Even that may be too optimistic. We might be observing a placebo effect—that is, students behaved differently only because they knew they were being observed, or because something in their classroom changed. Or maybe CM1 isn’t especially effective, just better than whatever the teachers were doing before, which might have been actively harmful.

We can draw a somewhat stronger conclusion if we use an “active control,” which means that control classrooms also adopt a new method of classroom management, but one that researchers don’t expect will affect suspension rates. Active-control designs make researchers more confident that, if a difference in suspension rates is observed, it’s really CM1 that’s responsible, because both CM1 classrooms and control classrooms are doing something new. This model means we need not worry about placebo effects or that CM1 merely prevented ineffective practices. However, even the best-case scenario produces a weak conclusion, because the control method was predicted not to work. It’s still “something is better than nothing.”

Still another type of comparison tests an intervention that’s known to be effective against a newer version of the same intervention. The goal, obviously, is to test whether the new version represents an improvement.

The three research designs we’ve considered answer questions that will often be of interest only to researchers, namely, whether CM1 “works” or, in the case of the old versus new version comparison, whether CM1 has been improved. When “works” is synonymous with “better than nothing,” the answer can be important for distinguishing among theories and hence is of interest to researchers. But is this question relevant to teachers? Practitioners are not interested in theories and so would not ask, “Is this program better than nothing?” They would ask something more like, “What’s the best way to reduce suspensions?”

The answer “CM1 is better than nothing” is useful to them if no other interventions have been tested. But in the real world, classroom teachers—not to mention school and system leaders—are choosing among several possible interventions or courses of action. What about other methods of classroom management intended to reduce suspensions? If, say, hypothetical classroom-management program competitors CM2 and CM3 have each been shown to be better than nothing, practitioners would prefer that researchers compare CM1 to CM2 and CM3 rather than compare it to doing nothing at all. Is one much better than the others? Or are all about equally effective, and it’s up to practitioners to pick whichever one they prefer?

Best Practices—But for Whom?

If we set a goal of finding the best way to reduce suspensions, and there are no successful interventions known, comparing CM1 to business as usual makes sense. However, if there are successful interventions known, researchers should compare CM1 to what is currently thought to be the most successful intervention. We might think of this as the strong definition of the term “best practices.” It indicates that there is one champion method, a single preeminent way of reducing suspensions, and the goal of research is to find it.

But that’s generally not how the world works and indeed, “What’s the best way to reduce suspensions?” is probably not exactly what an educator would ask. Rather, they would ask, “What’s the best way to reduce suspensions at my school, with the particular students, faculty, and administrators found here, and with our peculiar set of assets and liabilities, and without negatively impacting other important instructional goals?”

CM1 may be terrific when it comes to reducing student suspensions, but it may also be expensive, demanding of administrators’ time, or workable only with very experienced teachers or with homogenous student bodies. And maybe CM2 is also terrific, especially for inexperienced teachers, and CM3 is helpful when working with diverse students. Research certainly shows such variability across contexts for some interventions, and teachers know it. As we’ve noted, one reason teachers don’t tend to use research is because they assume that whatever positive impact researchers found would not necessarily be the same for their particular students in their particular school.

If a universal champion “best practice” really emerges, improbable as that seems, it would be useful to know, of course. But teachers would benefit most not by researchers’ identifying one program as the best, but by their identifying or broadening a range of effective interventions from which teachers can then choose. Research can support that goal, but it requires a change in what we take to be an interesting conclusion. Instead of deeming a study interesting if the intervention is better than the comparison group, teachers would be interested in knowing whether a new intervention is at least as good as the best intervention. That would allow them to choose among interventions, all of which are known to be effective, based on which one they believe best fits their unique needs.

Null (and Void) Hypothesis

But that’s not the goal of research studies. Researchers are looking for differences, not sameness, and the bigger the difference, the better. Teachers might be interested in knowing that CM1’s impact is no different than that of another proven classroom-management method, but researchers would not. Researchers call this a null effect, and they are taught that this conclusion is difficult to interpret. Traditionally, research journals have not even published null findings, based on the assumption that they are not of interest.

Consider this from a researcher’s point of view. Suppose a school leader implements CM1 because the leader thinks it reduces suspensions. There are 299 suspensions in the school that year, whereas in the previous year there had been 300. Did CM1 help? A researcher would say one can’t conclude that it did, because the number of suspensions will vary a bit from year to year just by chance. However, if the difference were much larger—say there were 100 fewer suspensions after CM1 were put in place—then the researcher would say that was too large to be a fluke. A “statistically significant difference” is one that would be very unlikely to have occurred by chance.

This logic undergirds nearly all behavioral research, and it leads to an obsession with difference. Saying “I compared X and Y, and I cannot conclude they are different” because the outcomes were similar may be uninteresting to researchers, but it is potentially very interesting to practitioners looking to address a particular challenge. They would be glad to know that a new intervention is at least as good as a proven one.

Null effects matter for another reason. Interventions often spring from laboratory findings. For example, researchers have found that memory is more enduring if study sessions are spread out over time rather than crammed into a short time period. We should not assume that observing that effect in the highly controlled environment of the laboratory means that we’re guaranteed to observe it in the less controlled environment of the classroom. If spacing out study sessions doesn’t work any better in schools than cram sessions, that’s a null effect, but it’s one that’s important to know.

Researchers are right that null effects are not straightforward to interpret. Maybe the intervention can work in schools, but the experimenters didn’t translate it to the classroom in the right way. Or they may have done the translation the right way, but the experiment the wrong way. Nevertheless, null effects are vital to tally and include in a broader evaluation of the potential of the intervention. Researchers can make null effects more readily interpretable through changes in research design, especially by increasing the number of people in the study.

Publication Bias

How do these phenomena play out in recently published research? To find out, we did some research of our own. We examined a sample of articles reporting intervention studies published from 2014 to 2018 in four journals: American Education Research Journal, Educational Researcher, Learning and Instruction, and Journal of Research in Science Teaching. Our analysis looked at the type of control group employed and whether the intervention was reported to be significantly different from the control group. We predicted that most published articles employ weak control groups—those allowing the conclusion “better than nothing”—because these offer the greatest chance of observing a significant difference between intervention and control.

Of 304 studies examined, 91 percent were of the “better than nothing” sort: 49 percent employed business-as-usual designs and 42 percent used as the control group an alternative intervention that researchers expected not to influence the outcome. Some 4.5 percent used a control that was a variant of the intervention with the goal of improving it. Another 4.5 percent used a control group that was either known to have a positive effect or was expected to have a beneficial effect based on existing theory.

Coders also noted whether the key comparison—intervention versus control—was reported as a statistically significant difference and whether a particular interaction was emphasized. For example, perhaps the intervention group performed no better than the control group in early grades, but there was a significant difference in later grades. Alternatively, the key conclusion of the report may have been that the intervention and control group did not differ.

We found that 91 percent of the studies reported that the intervention was significantly different than the control group. Of those that did not, another 4 percent reported a significant interaction—that is, the intervention worked for certain subjects or under certain circumstances. Just 5 percent of studies reported null effects. None of these studies demonstrated that a new intervention is equivalent to another intervention already established as effective.

A More Useful Research Standard

In theory, the goals of education research are to build knowledge and improve decision-making and outcomes for teachers and students. But in practice, education research is shaped by the common practices and priorities of researchers, not teachers or school and system leaders. Most intervention research employs a better-than-nothing control group, and an intervention is deemed worth applying (or, at least, worthy of continued research) only if it makes a measurable and statistically significant difference. The drawback to this pervasive research design is clear: there may well be “research-based” interventions in the marketplace, but educators have no basis on which to compare the alternatives. They have all been shown to be “better”—but better than what, exactly?

Imagine instead that the common research design started with whatever trusted intervention is considered the current “gold standard” for the desired outcome and used that as the control group. Imagine too that the criterion of the comparison would be that a new intervention should be at least as good as the gold standard. In time, a group of proven interventions would emerge, roughly equivalent in effectiveness and known to be superior to other interventions not up to the gold standard. As a result, educators would have a range of high-quality interventions to choose from and could select the one that best fits their school context, skills, and personal preference. In addition, choice itself can be an important component of educational effectiveness—interventions with teacher buy-in tend to be more successful, and research has shown that the pervasive adoption of a single intervention that does not suit the broader array of individual differences may lead to less learning.

We see other benefits to adopting this approach as well. We predict that refocusing research on equivalence as the dissemination criterion will spur innovation. “At least as good as” is actually “better than” if the new intervention has fewer side effects, is less expensive, is less time-consuming, or is easier to implement compared to its predecessor. For example, consider electronic textbooks, which are less expensive to disseminate and easier to update. The salient question for educators and policymakers isn’t whether they are better than other texts, but whether they are associated with learning outcomes equivalent to those of using traditional, more costly textbooks. The research field’s narrow focus on ensuring the intervention is statistically “better than” the control group means that the workaday demands of the intervention in terms of time, money, space, and personnel are not emphasized—in fact, are often not even considered. This disconnect invites skepticism on the part of the teachers charged with implementing supposedly classroom-ready practices.

What will it take to effect this change? We believe researchers are sensitive to the incentives their profession offers. Most education research is conducted in the academy, where the coins of the realm are grants and peer-reviewed publications. There are some encouraging signs that journal editors are taking a greater interest in null effects, such as a recent special issue of Education Researcher dedicated to such studies. But change will most likely come about and endure if the foundations and government agencies that fund research make clear that they will view this change in study designs favorably when reviewing proposals. This would encourage journal editors to publish studies with null effects and reject those that use business-as-usual control groups.

Researchers are, in our experience, frustrated and saddened that teachers do not make greater use of research findings in their practices. But nothing will change until the researchers recognize that their standard methodology is useful for answering research questions, but not for improving practice.

Daniel T. Willingham is a professor of psychology at the University of Virginia. David B. Daniel is a professor of psychology at James Madison University.

For more, please see “The Top 20 Education Next Articles of 2023.”

This article appeared in the Spring 2021 issue of Education Next. Suggested citation format:

Willingham, D.T., and Daniel, D.B. (2021). Making Education Research Relevant: How researchers can give teachers more choices. Education Next, 21(2), 28-33.

For more, please see “The Top 20 Education Next Articles of 2023.”

This article appeared in the Spring 2021 issue of Education Next. Suggested citation format:

Elder, T.E., Figlio, D.N., Imberman, S.A., and Persico, C.L. (2021). Segregation and Racial Gaps in Special Education: New evidence on the debate over disproportionality. Education Next, 21(2), 62-68,

Lee Cheng graduated from San Francisco’s Lowell High School in 1985. He counts himself one of the lucky ones. Lowell High, which was the only local public school specifically for high-performing students, had a strict racial-quota admissions policy when he applied. No racial or ethnic group could comprise more than 40 percent of the school’s student body. The rule was aimed at desegregating the district, but even as a teenager, Cheng found it unfair. It meant that Asian students had to score higher on the entrance exam than white students, who in turn had to score higher on the exam than Black and Hispanic applicants. Cheng’s friend—his orchestra partner—was not admitted, though if he had been of a different race he might have earned a seat. The boy’s parents were poor immigrants—his father a waiter and his mother a seamstress. “He would have gotten in, but for being Chinese,” Cheng said.

Years later, after Cheng graduated from Harvard, he and several of his former classmates from Lowell joined a federal class-action lawsuit against San Francisco over the policy, Ho v. San Francisco Unified School District. After five years of litigation, the district agreed to a settlement and abolished its race-based admissions. Instead of adopting a single standard, Lowell tried a variety of systems to achieve the racial balance it sought. In 2001, it adopted a three-band approach. Seventy percent of students were admitted on the basis of their test scores and grades. Another 15 percent were admitted on a combination of those factors plus some more holistic considerations such as community service and demonstrated ability to overcome hardship. The remainder of the students were selected from underrepresented schools.

In October 2020, that system ended—perhaps temporarily, but in all likelihood permanently. Because of the coronavirus pandemic, all of the students in San Francisco public schools received only pass-fail grades in the spring semester of the 2019–20 school year, and the exam for admissions was never administered. In the spring of 2021, applicants will be chosen for Lowell with the same formula used at other schools in the city—some combination of lottery, sibling preference, and geography. Covid-19 may have provided the impetus for the change, but the new policy also plays into a political phenomenon: in the past few years, pressure has been building for the city to reconfigure the school’s student body so that it more accurately mirrors the racial composition of the city.

Lowell is not alone among the specialty schools, known as exam schools or magnet schools, in deciding to change its admissions standards this year. According to a survey conducted by Chester Finn and Jessica Hockett for their 2012 book Exam Schools, there are about 165 stand-alone public high schools with selective admissions processes in the United States. Some of the biggest and most renowned of them are currently undergoing existential debates. Boston will not be administering the test it previously required for admission to its three academically competitive schools. In Fairfax County, Virginia, the district has announced plans to eliminate the test for Thomas Jefferson High School for Science and Technology and admit students from a lottery of applicants who have at least a 3.5 GPA and a year of algebra. And in New York City, parents anxiously awaited an announcement for almost a year about the test date for Stuyvesant, Brooklyn Tech, and Bronx Science, three schools whose entrance is determined entirely by test scores. Admission to other “screened schools” in the city that use the same exam but also judge students on other criteria was altered to eliminate preference for students who live near the schools, and there is still a possibility that the test for those schools will be abolished.

While the leaders of all four of these school systems assert that the pandemic-driven modifications are temporary, they face tremendous pressure by activists and political leaders to make long-term changes to their admissions policies because the student populations of the specialty schools do not reflect the racial composition of the areas where they’re located. In some cases, that pressure is likely to result in permanent changes that would fundamentally alter the character of these schools. Indeed, the Covid crisis seems to have provided an ideal opportunity for activists, school administrators, and politicians to effect the systemic transformation they have been pushing for years.

Lowell High School in San Francisco

The proposed change at Lowell, a school of about 2,700 students whose graduates include Supreme Court Justice Stephen Breyer, comedian Carol Channing, and novelist Jennifer Egan, was announced with little fanfare the Friday before Columbus Day weekend in 2020. The San Francisco Board of Education met the following Tuesday to hear public comment about the proposal. At the five-hour Zoom meeting, which was recorded, the board set aside one hour—30 minutes for those who supported the change and 30 for those who opposed it—with each speaker given 60 seconds before his or her microphone was cut off. The opposition included parents, students, and alumni. A number of speakers wondered why the decision seemed so rushed. Others asked why grades from previous years couldn’t be used as a factor in the application process.

The supporters of the change spoke not only about the small percentage of African American students at Lowell but also about their experiences of racism there. In February 2016, about two dozen of the school’s Black students walked out of school and marched to city hall to protest a Black History Month poster created by a non-Black student. The poster featured rappers and a picture of President Barack Obama wearing a diamond stud earring, as well as the hashtag #gang.

At the October 2020 school-board meeting, Shavonne Hines-Foster, one of two student representatives on the board, spoke immediately after the public comment period. A Black student at Lowell, she decried the “privilege” of those opposed to the change, wondering why these people had not spoken up about the racism at her school or a suicide there or reports of sexual assault. Some of the opponents, who apparently had not been fully muted, began to speak over her, questioning the relevance of her comments to the proposal.

At this point, the other board members grew visibly angry—one seemed to be in tears. They not only demanded that the moderator mute the offenders but also that he kick them out of the meeting entirely. The moderator asked, “Did we catch names?” and the board members began to call out names of people who had interrupted. One board member said, “Yeah, I’m listening to a lot of racist people.” No racial slurs or insults were audible on the recording, but another board member noted, “This was not a good day for San Francisco,” referring to the interruptions as “disgusting” and a source of “ugliness.” Another noted that “when we talk about meritocracy . . . these are racist systems,” and she decried the “Trumpian language” of the proposal’s opponents, noting that tests like this one and the SATs “began as ways to justify racism.”

One white parent of an 8th grader told me in a phone interview that his son had been planning to apply to Lowell this year but no longer sees the point, given the new admissions policy. Lowell is one of the only schools in the city that this father thinks could give his son a real education. The man declined to be named in this article because his son was applying to private schools and he worried that the boy’s chances of admission there would be diminished if he were among the parents labeled “racist” for supporting merit-based admissions. A self-described progressive, this father says that he has been “shocked” at just how disconnected the board is from the people they represent. “I can’t deny their reality, but it’s so far from the reality I’m living in, it’s laughable,” he said.

San Francisco already has one of the nation’s largest percentages (55 percent) of students attending private schools. New York, by contrast, has only 20 percent. The change in Lowell’s admissions policy is likely to make more middle-class families either flee to the suburbs or send their children to private schools, especially since the board’s desire to remake the admissions system shows no signs of abating. At the October 2020 school-board meeting, the vice president of the board ended the discussion by saying that “if there’s a way to extend this [new admissions system] past this year, to end this illegal process, I will support that.” All but one of the other members made it clear that they agreed.

Boston’s Exam Schools

Boston School Committee chair Michael Loconto resigned in October 2020 after he was heard on a hot microphone mocking the Asian names of some of the speakers during a meeting about a plan to transform exam-school admissions. But that was not enough to derail the proposal.

The city’s three exam schools—Boston Latin School, Boston Latin Academy, and the O’Bryant School for Mathematics and Science—had required applicants to submit test scores and grades. For admission in fall 2021, the board has adopted a policy whereby the top 20 percent of admitted students will be chosen solely on their grades, with students across the city competing for the spots. The remaining 80 percent will be selected on the basis of grades and by zip code, with students from the lowest-income zip codes given preference. In 2019, Superintendent Brenda Cassellius replaced the Independent School Entrance Examination, a test also used by private schools, with the MAP, or Measure of Academic Progress. The objection to the ISEE was that it included questions about material not covered in typical public-school curricula, and research indicates that the exam puts Black students at a disadvantage. But what began as a move to replace one admissions test with another became a plan to eliminate the entrance exam for the 2021–22 admissions cycle because of Covid-19.

Boston schools did not return to in-person learning in fall 2020, so the administration of the test presented a challenge, though other tests such as the SATs were offered in Boston Public School buildings. But then, as with Lowell, many parents asked why the school couldn’t simply use grades that were available. Why the new zip code approach?

Boston too, it seems, is using Covid as an opportunity to solve what its leadership sees as a race problem. The problem is not new. In 1974, as part of the city’s busing plan, Judge W. Arthur Garrity required exam schools to reserve 35 percent of their seats for underrepresented minorities. But a quarter century later, the First Circuit Court of Appeals struck down these explicit racial set-asides. The percentage of Black and Hispanic students at the city’s exam schools fell to about 16 percent from about 30 percent. And the number at Boston Latin School—the most prestigious of the three magnet schools—is closer to 8 percent.

Racial tensions have continued to present a problem as well. The Boston Public Schools’ Office of Equity found that between November 2014 and January 2016 there were seven race-related incidents at Boston Latin School, most of which were related to social media postings by students. The office said that the school’s internal policy was violated in one of those incidents. In 2016, the U.S. Attorney’s Office launched an investigation and found the school to be in violation of the 1964 Civil Rights Act for failing to properly investigate the allegation of a threat by a white male student to lynch a Black female student, among other incidents. The school’s headmaster and assistant headmaster resigned.

Under this year’s admissions plan for the magnet schools, each zip code would be assigned a certain number of seats based on the average household income in the zip code and the number of school-age children who live there. Supporters believe that what they see as the more-equitable distribution of seats across the city will help the underserved communities and also make Black and Hispanic students feel more comfortable at the exam schools. The Boston Globe noted in reference to Loconto’s ethnic slur at the school-board meeting: “It was an awkward moment for members. . . . Nearly seven hours into the Oct. 21 meeting, exhausted members seemed to struggle to comprehend how a political leader who was on the verge of shepherding through a historic change that would provide Black and Latino applicants with greater access to the exam schools could at the same time disrespect another community of color.”

Critics of the new program, such as Darragh Murphy, a longtime Dorchester resident who attended Boston Latin School in the 1980s, don’t have trouble comprehending this at all. Murphy, who is white, said she believes that the school board decided on the result they wanted “and then worked backward.” The point here, she and others hold, was to reduce the percentage of Asians at Boston Latin School. Chinatown, for instance, has a high average income, driven up by the presence of some luxury apartments, but the families applying to exam schools from this area are not wealthy. Chinatown is projected by the school district to lose exam school seats under the changes.

Fania Feghali and her husband are first-generation Lebanese immigrants who live in West Roxbury, a neighborhood of the city that is projected to lose exam-school seats under the new system. Feghali is an auditor, and her husband is a diesel mechanic. She studied the website Greatschools.com before choosing a place to live and says the most important thing to her was “preparedness for college.” The cheapest private high school near her charges $23,000 annually. The Feghalis currently have two children in a local Catholic school (which they struggle to afford at $7,000 annually) but will consider moving from the city if their 6th grader cannot get into Boston Latin School. “My responsibility as a parent is to offer my kids the best education I can,” Feghali told me.

Like many other parents, Feghali does not understand why the exam couldn’t have been offered despite Covid. Her own children have been going to school in person every day since September, and there were no positive cases at the school in the fall. Why was offering a socially distanced in-person test deemed impossible?

New York City

That is a question eating away at parents all over New York City. With the city’s schools having returned to at least some form of in-person learning for most of fall 2020, the postponement of the Specialized High School Admissions Test (the test required for admissions to three exam schools and various other screened schools) makes even less sense in the Big Apple than it does in San Francisco or Boston. Mayor Bill de Blasio and Schools Chancellor Richard Carranza have expressed their outrage over the small number of Black and Hispanic students admitted to Stuyvesant, Brooklyn Tech, and Bronx Science. In 2019, for instance, Stuyvesant offered admittance to 895 students. Only seven of them were Black, down from 10 the year before. When the topic of schools came up at a recent mayoral press conference, a reporter noted that de Blasio campaigned on “a tale of two cities” and asked the mayor which one Asians belonged to. De Blasio replied, “The specialized high schools still don’t make sense.”

It is worth asking, though, why the schools are not seen as equitable to begin with. Wai Wah Chin, who has had four children graduate from Stuyvesant and is the president of the Chinese American Citizens Alliance Greater New York, notes that “when you have kids chosen only by merit, that’s fair. If you start jiggering, all sorts of abuses happen.” She asserts that with a strictly exam-based system, no one knows “whether parents are rich or poor or whether a kid is tall or fat. The only question is: Can you do the work?”

While some outsiders may assume that the fact that the big three exam schools in New York are majority Asian means that they are catering to a wealthier population, this is far from true. More than half of the students at these schools qualify for free or reduced-price lunch. (While New York City offers free breakfast and lunch to all students regardless of income, officials use Medicaid data to determine which students would qualify by income so the city can receive reimbursement from the federal government.) Many of them are the children of recent immigrants. Unlike middle- and upper-class New Yorkers, they don’t have resources for high-cost tutoring. But, as Chin notes, you can get practice tests out of the local public library.

Questioning Admissions Tests

Some observers wonder whether it makes sense for public high schools to require admissions tests at all. Why shouldn’t schools funded by tax dollars be open to anyone residing in the jurisdiction? If charter schools can help even students from the most difficult circumstances succeed without screening applicants, why can’t other public schools? Though an exam-based process may seem evenhanded, that’s not so when schools fail entirely to prepare broad swaths of the student population in the years leading up to high school. The racial disparities in exam-school enrollment, these critics say, just make the inequities among these students all the more glaring.

For one thing, the older students get, the harder it is to make up for these educational deficits. Perhaps that’s why no one seems to object that public universities don’t accept every state resident who applies. And it’s why almost no elementary schools require admissions tests—though some of them offer gifted and talented programs that may have academic criteria for participation. What’s more, the most successful charter schools require a strong degree of parental support and commitment that mirrors that of the low-income parents who manage to get their children into the exam schools.

Prior to the Covid crisis, de Blasio’s hands were tied. In 1971, the state legislature in Albany mandated that admission to the big three and five smaller schools be based on an exam, thereby heading off an attempt by New York City leaders to shelve the test. And state leaders so far have shown little signs of movement. Last year, the mayor pushed an alternative plan that would offer admission to the top 7 percent of students at each middle school. An analysis by the state budget office concluded that under de Blasio’s plan, “About 19 percent of admissions offers would have gone to Black students, compared with the less than 4 percent who actually attended specialized high schools in 2017–2018. Hispanic students would have received about 27 percent of admission offers, compared with the 6 percent of Hispanic 9th graders who attended the schools last year. The number of Asian students receiving admissions offers would have fallen by about half, to just over 31 percent, while offers to white students would have remained relatively flat.”

The mayor’s proposal did not even get a vote. Alumni, led by businessman and philanthropist Ronald Lauder, a Bronx High School of Science graduate, launched a multimillion-dollar campaign opposing the change. Asian groups also pressured lawmakers, who probably didn’t want to risk harming the schools’ international reputation. The big three have produced 14 Nobel Prizewinners among them, more than most countries. On the other hand, the November 2020 election brought Democrats a supermajority in the Assembly and Senate, meaning that they could change the rules, even over the governor’s objections.

In December 2020, the mayor announced other sweeping changes to the system. The city will eliminate all screening for middle schools for at least one year. Formerly, almost half of the city’s middle schools used grades, attendance, and test scores as admissions criteria. At least for now, all middle-school admissions will be determined by lottery. In January, the mayor also announced that the city’s education department would no longer test children entering kindergarten for gifted programs.

The exam-school question became a campaign issue in a state assembly race in Brooklyn, where a Republican challenger accused a Democratic incumbent of wanting to dismantle the schools, a charge the Democrat denied. The Republican lost by the narrowest of margins in a district that has been held by Democrats for 80 years.

Asians are hardly the only supporters of the Specialized High School Admissions Test and the idea that admission to these schools should be based strictly on merit. Barbara Rivera, who came to this country 20 years ago from Hungary and met her husband here, says the real problem is the pipeline from lower-school grades. She remembers that, when her son was entering elementary school, a judge who lived on her street in Borough Park advised her to get her son into a gifted program. She didn’t think he was gifted and dismissed the idea of sending him to a school farther away from home. By the time he was in 3rd grade, though, it became increasingly obvious to her that her son’s school was not very serious. She remembers a teacher marveling that her son “passed a test.”

“Why,” she wondered, “was the bar so low? Because his last name is Rivera?”

Indeed, the key question in so many of these debates over exam schools is whether it makes sense to wait until high school to try to address inequities in public education. Why aren’t more elementary and middle schools successfully preparing students for these exams in the first place? Commenting on Boston’s new admissions policy suspending the entrance exam, Darragh Murphy wrote in the Dorchester Reporter in November 2020, “The real tragedy is that all this disruption imposed on our 6th and 8th grade families and schools by our mayor and city council is designed to hide the real crisis that we have allowed to grow more dire with every passing year—that the mostly Black and Latinx kids in our Boston public elementary schools are being failed by our leaders and administrators.”

Preparing such students to compete for a spot in a specialized high school is not impossible. In 2018, students of color at three middle schools run by the large Success Academy Charter Schools network in New York were three to four times as likely to gain admission to the exam schools as students in the same demographic citywide.

In other words, it is possible for young people in all racial and ethnic groups to achieve academic success and gain admission to competitive exam schools, but for that to happen at scale would require the kind of systemic change in public-school education that politicians and other leaders have not been willing to make, in New York and elsewhere.

The dispute seems to be reaching a boiling point in New York City. While wealthier New Yorkers have been able to flee the city during the pandemic, the middle and working classes of all races have been stuck and increasingly frustrated with remote learning, as well as the confused messages and lack of leadership from the mayor and chancellor. A public opinion poll in October 2020 showed that three times as many New Yorkers want the state to take over the city school system as want the mayor to remain in control. In early December, a coalition of groups who oppose the exam wrote to Governor Andrew M. Cuomo, asking him to lift the testing requirement this year through executive order, but the governor’s unwillingness to change things before does not bode well for the activists’ demands. Another group filed suit against the city, demanding that the mayor and chancellor set a date for the test. Eventually, the mayor announced a January 27 test date for the city’s 8th graders.

The Boston plan could sail through without any state involvement, just as San Francisco’s has. Boston may face more legal scrutiny, though, because its convoluted formula seems designed to achieve a particular racial result. (San Francisco’s, by contrast, simply makes Lowell’s admissions criteria the same as every other school in the city.) As Joshua Thompson of the Pacific Legal Foundation, a nonprofit that has litigated racial discrimination cases, notes, “There are legal methods to challenge policies that have a facially neutral purpose but have a differential effect.” While he says these cases are hard to challenge during Covid, going forward opponents could “try to get at the true purpose of the law. If racial balance is the goal, it should be enjoined.” Given how open these politicians and school leaders have been about wanting more racial balance—and even their blatant bias in some cases—that would seem easy to prove.

Thomas Jefferson High School for Science and Technology in Fairfax County, Virginia

Earlier this fall, the Pacific Legal Foundation sent a letter to the Fairfax County School Board warning them that their attempt at racial balancing is “patently unconstitutional.” Citing the U.S. Supreme Court’s 2013 decision in Fisher v. University of Texas at Austin, the letter notes that “when examining the outcomes of the merit lottery process, it is impossible to see the changes to the TJ admissions process as anything other than an attempt at racial balancing according to the demographics of Fairfax County.”

A few weeks later, the board altered the plan. It now would reserve 100 slots for the highest-evaluated students based on a holistic review of their applications, including recommendations or a problem-solving essay. The remaining 400 seats would be filled by “merit lottery,” with seats divided evenly among students with a minimum 3.5 GPA in the geographic areas Thomas Jefferson serves. But these changes did nothing to address the legal criticisms made by the Pacific Legal Foundation lawyers.

The battle over Thomas Jefferson—a regional magnet school that consistently ranks at the top of the U.S. News and Newsweek surveys of America’s best high schools and that regularly claims winners of nationwide science competitions—began almost a decade ago with a civil rights lawsuit filed by the NAACP in 2012. The filing asserted that Black and Latino students were being shut out of Thomas Jefferson because Fairfax County consistently fails to identify them for gifted programs during the lower grades. Despite renewed outreach efforts, the district has failed to get more Black and Hispanic students into Thomas Jefferson. Today, the student body is 71 percent Asian, 19.5 percent white, 2.6 percent Latino or Hispanic, 1.7 percent Black, and 4.7 percent students who identify as another ethnicity.

Last year, the school’s racial soul-searching took on a new tone and urgency. In June, the principal sent a note to parents explaining that “the recent events in our nation with Black citizens facing death and continued injustices remind us that we each have a responsibility to our community to speak up and take actions that counter racism and discrimination in our society.” Among the concrete steps she recommended was replacing the school’s mascot, an American Colonial figure, because of “colonialism’s role in our country’s history where certain classes exerted power over others as a means to economically exploit, oppress, and enslave them.” After noting that Thomas Jefferson’s racial makeup does not mirror the community’s, she asked, “Do the TJ admissions outcomes affirm that we believe TJ is accessible to all talented STEM-focused students regardless of race or personal circumstance?”

Many parents objected to the changes in the admissions process, and the debate has become bitter. Suparna Dutta, the parent of a sophomore at Thomas Jefferson, worries that plans for racial rebalancing run the risk of making this country more like her native India. The quota system instituted for education and employment there to make up for the caste system of old, she says, has “led to a lot of brain drain. People flee for better opportunities.” Now, she says, “I’m afraid my adopted country is going down this road.”

What is particularly curious about the proposal to eliminate Thomas Jefferson’s exam and simply admit students by lottery is that the school’s white population is projected to more than double under the new criteria, going from 18 percent of Thomas Jefferson students to as high as 48 percent—figures that were calculated by a parent group called the Coalition for TJ. Even if Fairfax County accomplished all of the outreach to Black and Hispanic families that the district promises—something it has failed to do in the past—the end result of the district’s attempt to achieve racial balance would be the presence of many more white children at Thomas Jefferson.

The potential increase in white students at Thomas Jefferson is just one of many unintended consequences that could arise from the admissions-policy changes in San Francisco, Virginia, New York, and Boston. It seems likely that these urban areas will lose a significant number of families to private schools or to the suburbs. Some of the policy changes may prove to be temporary, but with cities seeing higher rates of crime and homelessness and so many professional jobs going remote, many families may see the shifting admissions practices at these schools as the final straw.

Like the school boards in San Francisco and Boston, the Fairfax County board has become the subject of public ire this year and seems increasingly disconnected from the families it is supposed to serve. Schools there did not reopen in the fall of 2020. A recent study found that remote learning resulted in an increase of 83 percent in students who are failing two or more classes. The district came under heavy criticism in fall 2020 after paying the anti-racism activist and scholar Ibram X. Kendi $20,000 for a one-hour presentation to school leaders. The district sent out a memo discouraging parents from forming educational pods for their children because “they may widen the gap in educational access and equity for all students.” Meanwhile, the superintendent himself has enrolled his own child in private school. The district’s parents seem more interested in the failures going on in their children’s (virtual) classrooms than any of the racial drama on offer.

Perhaps realizing that the lawsuits over racial bias seem rarely to provide clear answers and also can take years to wind their way through the court system, the opponents of this change at Thomas Jefferson have tried a new tactic. In November 2020, they filed a lawsuit arguing that the proposed change in admissions requirements goes against the Virginia law establishing and regulating Governor’s Schools—of which Thomas Jefferson is one—meant to serve gifted students. That law specifically requires that Governor’s Schools include a “nationally norm-referenced aptitude test” as part of the process for identifying children with particular academic aptitudes. The plaintiffs argue that “under the No-Testing Decision, the Thomas Jefferson Governor’s School will no longer be a high school devoted to the education of gifted students.”

The Indian-born author Asra Nomani, whose son is a student at Thomas Jefferson, said that demanding that the school district serve gifted children—just as they serve other kinds of learners—will prove more effective than a racial-bias argument. “Kids that are advanced learners have faster processing skills. They’re the ones firing questions at teachers and yet do not have great social skills,” Nomani said. These may be generalizations, but there is certainly a case to be made that school districts serve students with such varying levels of skills and aptitudes that a one-size-fits-all approach is not going to meet anyone’s needs well.

One thing that’s clear is that schools for gifted students are oversubscribed. When Michael Bloomberg was mayor of New York, for instance, he expanded the number of screened schools—but even that did not meet the demand.

It is possible that some of these urban parents are not even looking for “gifted education,” but simply a decent education for their children. Many parents of academically motivated students are not worried about science competitions and just want their children to learn in the company of other strivers. Even though there is some evidence that the schools themselves do not make an enormous difference in the future prospects of their students—at least when it comes to college enrollment, college graduation, or college quality—parents may want their children to be in an environment with peers who have similar interests.

Though many politicians, school-board members, and activists may see the Covid crisis as an opportunity to change longstanding admissions policies they view as faulty, the pandemic also seems to be a kind of double-edged sword for education policy. The switch to remote learning and the failures of public education to deliver even basic instruction during this crisis have forced a laser focus on school and district leadership by parents nationwide. Middle-class parents who were previously able to find ways for their children to get a decent education have found themselves frustrated and angry at local politicians and school administrators for running roughshod over the interests of their children. And plans for radically altering some of the only schools that seem to be working are not going over well.

Naomi Schaefer Riley is a resident fellow at the American Enterprise Institute and a senior fellow at the Independent Women’s Forum.

This article appeared in the Spring 2021 issue of Education Next. Suggested citation format:

Schaefer-Riley, N. (2021). Exam-School Admissions Come Under Pressure amid Pandemic: Efforts to change selective admissions policies fuel parent activism. Education Next, 21(2), 50-60.

To his credit, President Joe Biden has made reopening schools for in-person instruction a visible priority for his new administration. On his first full day in office, Biden issued an executive order directing the U.S. Department of Education to establish national reopening guidelines, support contact tracing in schools, and gather data on the pandemic’s impact on students. The stimulus package he’s proposed to Congress includes $130 billion that schools could use to upgrade ventilation systems, ramp up testing, or hire additional staff. The goal, the administration’s plan says, is to “safely reopen a majority of K–8 schools in his administration’s first 100 days.”

That goal may be less ambitious than it sounds. Most obviously, it leaves out high schools. It also does not define what “reopen” means. Does it mean that some students—perhaps only very young students and those with disabilities—are able to attend in person? Does using a hybrid model count? Or does it mean that all students must have the option of attending school in person full time? Organizations such as Burbio that are tracking school schedules report that most public schools nationwide were offering at least some in-person instruction even prior to the November election, suggesting that Biden’s team will have no trouble arguing that they have met their stated goal.

Biden is hardly the first new president to set policy targets that defy falsification. And one could easily sympathize with a desire for vagueness in defining this goal, given the current challenges to getting more students inside of schools. Covid-19 case counts remain near all-time highs through much of the nation. New, more-contagious variants of the virus threaten to hasten its spread. Vaccine distribution is running behind schedule. It is understandable, if also tragic, that conversation in some quarters has turned to mitigation measures that may need to be in place in the 2021–22 school year.

And then there’s the politics. The teachers unions resisting reopening are key members of Biden’s electoral coalition. The same day the president issued his executive order, Dr. Jill Biden, herself a member of the National Education Association, hosted the organization’s president, Becky Pringle, and Randi Weingarten of the American Federation of Teachers at the White House. In a statement, Pringle praised the president’s reopening plan but cautioned that “right now, the reality in far too many schools and institutions of higher education is that effective distancing, mask-wearing, ventilation, COVID-19 testing, contact tracing, and other crucial mitigation strategies are not in place.” Days later, in the nation’s third-largest school district, 71 percent of voting members of the Chicago Teachers Union rejected the district’s plan to return to in-person instruction on February 1. It isn’t only teachers who are opposing a return to school; some parents, too, are reluctant to send their children back, given that no vaccine has yet been authorized for use on U.S. children 15 or younger.

Viewed in context, then, the fact that the president has chosen to prioritize school reopening at all is remarkable—and praiseworthy. The latest survey data gathered by Education Next, reported in this issue (see “Pandemic Parent Poll Finds Perverse Pattern”), provide a baseline for assessing the administration’s progress. They also reveal just how badly progress is needed. As of late November, 53 percent of American students were receiving fully remote instruction; just 28 percent were in the classroom full time. Black and Hispanic children were even less likely to be learning in person full time, at 18 percent and 22 percent, respectively. The parents of 60 percent of all students report that their son or daughter is learning less than the child would have if the pandemic had not occurred, and parents express even greater concern about how Covid mitigation measures have affected their children’s social relationships and physical fitness. Yet reports of learning loss and other adverse effects are far less frequent for students attending school in person.

What actions can the new administration take to change this picture? Clear federal guidelines on reopening would be a welcome start but are unlikely on their own to be effective in overcoming union resistance. That need not be the only tool at the administration’s disposal, however. The $130 billion for schools in the president’s proposed stimulus package would amount to some $2,300 per student. With approval from Congress, the administration could require school districts in communities where safety conditions are met to provide all students the option to attend school in person in order to receive those funds. And what about students in districts that cannot safely reopen this school year—or refuse to do so? Parents could be given their share of the funds directly, to use as they choose to help their children catch up.

Martin West

This article appeared in the Spring 2021 issue of Education Next. Suggested citation format:

West, M.R. (2021). Condition Covid Aid on Opening Schools: Or else let funding flow directly to parents. Education Next, 21(2), 5.

 

Assessment Data Can Help Us Build Back Better

by Jessica Baghian

 

 

 

Focus on Instruction and Intervention, Not Testing, in 2021

by Scott Marion and Lorrie Shepard

This article appeared in the Spring 2021 issue of Education Next. Suggested citation format:

Baghian, J., Marion, S., and Shepard, L. (2021). Statewide Assessments in 2021: An Essential Lense or a Fruitless Imposition? Education Next, 21(2), 70-77.

This is part of the forum, “Statewide Assessments in 2021.” For an alternate take, see “Focus on Instruction and Intervention, Not Testing, in 2021,” by Scott Marion and Lorrie Shepard.

This article appeared in the Spring 2021 issue of Education Next. Suggested citation format:

Baghian, J., Marion, S., and Shepard, L. (2021). Statewide Assessments in 2021: An Essential Lense or a Fruitless Imposition? Education Next, 21(2), 70-77.

This is part of the forum, “Statewide Assessments in 2021.” For an alternate take, see “Assessment Data Can Help Us Build Back Better,” by Jessica Baghian.

This article appeared in the Spring 2021 issue of Education Next. Suggested citation format:

Baghian, J., Marion, S., and Shepard, L. (2021). Statewide Assessments in 2021: An Essential Lense or a Fruitless Imposition? Education Next, 21(2), 70-77.

This article appeared in the Spring 2021 issue of Education Next. Suggested citation format:

McPherson, M. (2021). Examining the Human Costs of a Narrow Meritocracy: Economic critique displays energy but not much evidence. Education Next, 21(2), 87-89.

Failure to Disrupt: Why Technology Alone Can’t Transform Education
by Justin Reich
Harvard University Press, 2020, $27.95, 336 pages.

As reviewed by Michael B. Horn

In Failure to Disrupt, Justin Reich aims to reset the hype around education technology and replace it with a more realistic set of expectations for what such technology can and cannot accomplish. Arriving amid the coronavirus pandemic, the book has heightened relevance, although it does not specifically focus on the abrupt transition to distance learning occasioned by the crisis.

At its core, the book offers a helpful framework for evaluating the likely impact of new edtech products. That framework entails asking four questions about any new education technology: 1) How will existing stakeholders use the technology, and will it help them extend their current practices? 2) What kinds of learning can and cannot be assessed with it? 3) How will learners from different backgrounds and circumstances access it? 4) How could research and experimentation improve the product?

Reich, who is director of the Teaching Systems Lab at MIT, evaluates a range of recent education technologies through this framework. He then concludes that the answers to his questions suggest four corresponding limitations on technology’s potential to bolster learning at scale: 1) Most educators use technology in familiar rather than innovative ways, a practice that replicates current outcomes rather than transforming them. 2) Routine assessment, which is all that machine-based learning systems can perform, measures only learners’ knowledge and their ability to complete rule-based tasks. Technology can’t measure more-complex types of learning at scale. 3) Those with greater resources benefit more from new products than those without such resources, who in turn fall further behind in their learning—a dynamic Reich calls the “EdTech Matthew Effect.” 4) Data and experimentation hold great potential to improve new technologies, but concerns about privacy and “experimenting” on children often keep this from happening.

The most useful observation to come from these insights is that by analyzing how any new edtech product is likely to be used in teaching, we can use research on past products with a similar profile to predict how effective and transformative the new technology will be. Past will be prologue, in other words.

Massive open online courses, or MOOCs, are simply a vehicle for traditional teacher-directed “instructionism,” Reich argues. That means they were fated to have low completion rates—particularly for students in poverty-impacted communities.

Adaptive-learning products, through which individual students work at their own pace, are similar to tools that date back to the 1960s. Reich dives into the meta-analyses of early adaptive-learning tools, as well as research on newer ones, to argue that we shouldn’t expect too much from such products.

And peer-guided learning has consistently shown that it can ignite passion in some learners, but it is also likely to leave many individuals behind. Reich suggests that such approaches should be evaluated based on the depth of learning for the specific individuals who are able to explore their passions, not the numbers of students advancing toward mastery. “Having a learner leave the Scratch [a platform for learning computational thinking] community or pass through with only a light touch isn’t necessarily a loss or a concern,” Reich writes.

In this vein, Reich makes a commonsensical appeal for “methodological pluralism” in education, that is, for an approach that draws on both teacher-driven instructionism and student-driven constructivism. In learning theory, constructivism holds that learners “construct” their own knowledge, while instructionism emphasizes the teacher’s role and sees learning as a behavioral change. He writes, “We need our entire population to have fundamental skills in reading, writing, numeracy, civics, science literacy, and communication; in these domains, we need to take the entire distribution of learners and help them move toward mastery. We also need learning environments that let young people discover their interests and explore them deeply, much more deeply than might be allowed if the environment were equally concerned with bringing along the unenthusiastic with the enthusiastic.”

So far, so good. But Reich’s argument stops short here. On constructivism, he fails to ask: what if learners are failing not because they lack interest but because the environment is poorly constructed or alienating? What if, as he suggests for constructivist learning, we were to evaluate instructor-driven technologies like MOOCs based on the number of people who were turned on to a subject? Would that number be equal to, less than, or more than the number for constructivist learning methods? And if it were equal, what would that say? Similarly, the considerations he offers for designing education technology for equity sound nice and offer good vignettes, but they lack data to support them. Are we to use data when it suits us but ignore it when it’s inconvenient? Reich doesn’t explore these questions.

Reich does concede that applying his framework isn’t as straightforward as it might seem, as he shares his own missteps with creating MOOCs in the MIT Teaching Systems Lab, where his team’s “progress is mixed.” For example, in offering a professional development course for teachers, the team hoped to serve educators in less-affluent schools but found instead that the teachers who participated were “disproportionately likely to come from independent or suburban schools serving affluent students.”

Reich concludes, “For MOOCs to serve populations beyond the already educated, there will need to be substantial support for the social elements of learning—coaching, advising, peer support, and so forth. All of these efforts will require seeing MOOCs not as a technological solution to a complex social problem, but as one element of a comprehensive solution.”

Sensible as it is, this conclusion begins to show how Reich’s narrative overreaches. In Reich’s telling, Harvard Business School professor Clay Christensen, who died in January 2020—and by extension Christensen’s allies and colleagues like myself—are the central bogeymen. Reich depicts us as cheerleaders for a disruptive future who are mistaken in our predictions and fail to understand the underlying historical research on the technology-based pedagogies we advance.

Actually, many of us are more bearish on traditional MOOCs than Reich. We were skeptical of the first generation of MOOCs because they replicated the passive pedagogy of the college classroom without support—not too dissimilar from Reich’s argument. I have also argued that most MOOCs suffer from other limitations, such as the lack of sound instructional design. That constricts their ability even to form a component of a more holistic solution, let alone serve as a robust standalone offering. It also limits the ability of designers and researchers—even with data and controlled experiments—to improve MOOCs, given their flawed starting points. What’s more, MOOCs as launched originally failed the test of being a disruptive innovation because they lacked a coherent business model and a technological driver that allowed them to improve.

Ironically, as MOOC platforms like Coursera and edX have pivoted to manage some universities’ online programs—evidence, Reich asserts, that MOOCs are becoming mainstream rather than disruptive—they are more likely to contribute to the demise of mid-tier universities that depend on pricey master’s programs to subsidize their expensive-to-run undergraduate offerings. Although Christensen and I did not argue, as Reich mistakenly asserts, that online learning would be the main driver of colleges closing, this move by the MOOC providers likely makes them more threatening to many universities and their troubled business model, not less.

Reich also attacks a big prediction that Christenson, Curtis W. Johnson, and I made in our 2008 book Disrupting Class—that by 2019, 50 percent of all high school courses would be delivered online. But Reich calculates the number of students in full-time virtual schools, a data point that is irrelevant to our prediction, as we anticipated that over 90 percent of the online learning would occur in physical schools. Our prediction centered on the extent to which digital materials would replace analog, offline curricula. Based on information I gathered from digital curriculum providers, at least 13 million K–12 students, or about 25 percent of them, were learning through digital curriculum for at least a portion of their day a couple years before the pandemic hit. According to a representative Digital Promise survey from April 2019, 35 percent of the responding K–12 public school teachers report using edtech daily and another 23 percent use it most days—adding up to more than 50 percent of such educators. To be clear, that hardly confirms that our prediction was correct, but it signals that digital learning has grown rapidly. That growth has only accelerated amid the pandemic. And although we were wrong about how much the costs of online learning would drop—Reich notes that we predicted they would fall by two-thirds—today, a class on a platform such as Outschool often costs less than half of what a provider would have charged for a comparable course in 2009. We were correct in forecasting the downward price trend, but wrong in our estimate of its magnitude.

What’s more, Reich asserts that in Disrupting Class we were cheerleaders for certain technologies that we in fact did not champion. As he dissects the research behind computer-adaptive instruction, he claims that we said adaptive online learning would come to dominate K–12 schools. But the word “adaptive” never appears in our book. We wrote that, for education technology to customize to individual learners, a facilitated-network model—in which students and teachers would teach each other—would need to emerge. Such a model would more closely resemble the peer-guided learning models Reich analyzes, not adaptive courseware.

Salman Khan, the founder of the free online-education platform Khan Academy, is another of Reich’s bogeymen. Reich misrepresents Khan’s recommendations on how learners can most effectively use the platform. In Reich’s telling, Khan says in regard to math learning that “the proper first step toward deeper learning is learning mathematical procedures and facts that might eventually lead to doing interesting collaborative projects.” Yet that is the inverse of how students in Khan’s own physical school—the Khan Lab School—typically use his platform. In that setting, students often study math through interest-based projects in which they call upon procedures and facts as needed and in turn have added intrinsic motivation to learn them. Here again, there’s more agreement than disagreement.

Reich is correct that our larger hope in Disrupting Class has failed to come to pass—that is, the hope that the U.S. education system would undergo transformation into a student-centered system in which young people would have more opportunities to build on their passions and fulfill their potential. After advancing one explanation for this disappointment—that many of the new education technologies introduced were based on well-studied pedagogies that could have been predicted to have limited impact—Reich turns to another reason: that schools are complicated.

In citing Stanford pro-fessors Larry Cuban and David Tyack, he turns to the same source we did in explaining why it would be difficult to transform schools by working within existing classrooms. A central tenet of the theory of disruptive innovation is that technologies that succeed in transforming an industry get their start not by challenging the industry’s dominant players directly, but rather by serving segments of the market the dominant players have ignored—that is, by competing against nonconsumption. As we wrote in Disrupting Class, “When disruptive innovators target nonconsumption for their foothold applications, they have a good chance of succeeding. But if those applications are then ensconced within a value network—a chain from suppliers to customers whose definitions of quality and profitability were honed in the established way of doing things—the disruption won’t fly unless it conforms to the rest of the players’ needs and expectations. That typically limits the scope of the innovation. And it is expensive. It is for these reasons that disruptive growth is truly unleashed only when the new technology is taken to the market not only through a disruptive business model, but also by utilizing a disruptive value network—from suppliers through distributors—whose economics are consonant with the disruption.”

Much of the disruption we wrote about in Disrupting Class has occurred within a larger system of existing schools and districts with a multiplicity of stakeholder interests and regulatory rules and practices. Dis-rupting from within can result in a change of modalities without a change of the larger rules of the game—and therefore create the incentives for technologies to fall in line with existing practice rather than change it. In the case of schools, that means technologies serve to perpetuate the traditional model of students advancing in their education at a uniform pace according to a content guide, for example. The transformative effects we had hoped for have, alas,
been limited.

Unfortunately, Reich doesn’t probe more deeply here toward finding a path forward. He misreads the most recent research on Teach to One, a software-based, adaptive, personalized math program. Reich cites one randomized evaluation that found that the program fails to produce improved student results on state tests. But he fails to note that a second study showed that, in those places where school accountability systems measured student growth and therefore incentivized teachers to address the gaps in student knowledge rather than just teach grade-level material, Teach to One produced significant gains on benchmark assessments (See “The Grade-Level Expectations Trap,” features, Summer 2020). In other words, tests that dictate a school’s priorities matter and can help us understand what pedagogies and practices can or cannot be successfully adopted.

Reich doesn’t grapple with these more nuanced parts of the theory of disruptive innovation. His writing also reveals a mistaken assumption: that anyone touting the power of disruption must see technology as the most important ingredient. We devoted the entire third chapter of Disrupting Class to showing that the model in which a technology is used carries far more weight than the technology itself, which helped explain why computers had not had a substantial impact in schools. Understanding this point—that the model in which something operates is far more important than the technology itself—is central to the theory of disruptive innovation.

With this insight, one realizes that much of Reich’s ultimate argument isn’t necessarily in conflict with ours. The two are consistent with each other.

His argument that community matters more than technology is similar to, although less precise than, our claim that the model—comprising the school resources, processes, priorities, resulting culture, and incentives for revenue—matters more than the technology. When he argues that “creative educators [should] find more spaces where peer-guided large-scale learning can be woven into the periphery of schools—in electives, extracurriculars, and untested subjects—so that learners can have some practice in navigating these new networks,” he’s making the same argument we made in Disrupting Class. Innovators seeking to introduce new technologies should start in areas of nonconsumption; only then will they have the freedom to create the proper model for what they want to achieve. Many education innovators have ignored this advice, given their short time horizons for achieving financial sustainability—perhaps another reason why our larger hopes from Disrupting Class have not materialized.

If Reich’s hope for MOOCs—that they become embedded in novel educational models with wraparound supports to help learners succeed—came true, it would be a showcase for our work, with digital curriculum replacing print and a comprehensive model with the power to displace the status quo. This points to perhaps the biggest problem with Reich’s book, which is the title itself: “Failure to Disrupt: Why Technology Alone Can’t Transform Education.” It is common sense, but it is also a straw man.

Michael B. Horn is co-founder of the Clayton Christensen Institute for Disruptive Innovation and an executive editor of Education Next.

This article appeared in the Spring 2021 issue of Education Next. Suggested citation format:

Horn, M.B. (2021). Effort to Debunk Education Technology Falters by Overstating Its Own Case: A sensible, but hardly novel, appeal for “methodological pluralism.” Education Next, 21(2), 83-86.