Off the Mark: How Grades, Ratings, and Rankings Undermine Learning (but Don’t Have To)
by Jack Schneider and Ethan L. Hutt
Harvard University Press, 2023, $29.95; 296 pages

As reviewed by Adam Tyner

In the years since the Covid-19 outbreak, the grades and test scores that anchor our education system have been relentlessly disrupted. As the pandemic swept the globe, American schools canceled annual standardized testing, college admissions went “test-optional,” and students were offered “hold harmless” policies that prevented their grades from dropping, regardless of whether they completed assignments or even attended virtual classes. Most end-of-year testing returned to K–12 schools in 2021, but much of the “assessment holiday” has endured. Most colleges continue not to require SAT or ACT scores, states are eliminating high school graduation tests, and grading standards have slipped to their lowest levels on record. States and districts are fueling grade inflation through policies that, in the name of equity, prohibit penalties for late work, recalibrate grading scales in ways that make passing easier, require teachers to assign credit for assignments that aren’t turned in, and even eliminate grading penalties for cheating.

Into this accountability recession arrives a new book arguing that the idea of holding students accountable through measures such as grades and test scores is inherently misguided. Penned by Jack Schneider and Ethan Hutt, two education-school-based researchers, Off the Mark is an ambitious volume combining history, policy analysis, and prescriptive recommendations. The authors evaluate the key “assessment technologies” of modern education systems—course grades and external tests—arguing that their presence undermines the aims of education. Although many of the book’s recommendations are sensible, its grandest claims are unsupported by research or contradicted by it.

The role of grades and tests in our education system does need better grounding in theory. Many education writers and researchers assume that these measures serve a single purpose, such as predicting postsecondary success, or that they matter only to one set of stakeholders, such as parents. Schneider and Hutt explain that many measures emerged to serve one role but now have multiple functions and stakeholders. The authors offer a helpful mnemonic for sorting out the stakeholders, explaining that the assessment strategies convey both “short-haul” messages to parents and students and “long-haul” messages to institutions such as colleges.

Unfortunately, short-haul messages are often garbled by the time they reach parents. Learning Heroes, a nonprofit organization that works to equip parents to support student success, has found via surveys that the “good” grades most students receive have about nine of ten parents convinced their kids are performing at grade level, despite only about one in four of them actually doing so. The organization’s most recent report shows that, even in our era of devastating learning loss, about four in five parents say their child is taking home mostly As and Bs. This disconnect is dangerous, because, as Schneider and Hutt note, “Families want to know how their children are doing, so that they can encourage, coax, and intervene as necessary.”

The book frames the multiple uses of grades and test scores as a dilemma, noting that the measures were not designed to support some of their current uses. The authors’ concern about the long-haul messages is not that they fail to communicate useful information, however. Grade point average and test scores are some of the best predictors of college performance and labor-market success, and the authors acknowledge that the utility of basing college admissions decisions on grades is one of their upsides. Their critique is that any long-haul message raises the stakes for student performance, as the rating will follow the student far into the future. The authors join prior critics of teacher-assigned grading, including James Coleman and John H. Bishop, in noting how the classroom dynamics around grading help explain grade-grubbing, “nerd harassment,” and other toxic dynamics between students and teachers and between students and their peers.

Yet the authors’ assumption that students’ having greater stakes in their academic performance undermines their learning is at odds with the work of those earlier critics. Indeed, the authors make assertions about grades and test scores harming student motivation that are either unsubstantiated, mostly contradicted by research, or missing analysis of the social dynamics around grades and test scores that researchers have identified.

The authors’ antipathy toward the use of grades and test scores as motivators stems from their unarticulated theory of learning—a version of the pop romanticism that is often attributed to philosopher Jean-Jacque Rousseau but is better represented by self-help and education writing of the last few decades, such as Daniel Pink’s Drive and Alfie Kohn’s Punished by Rewards. The pop romantics contend that the use of incentives in education undermines students’ intrinsic desire to learn. In the 1970s, psychologists, including Richard Ryan and Edward Deci, whom Schneider and Hutt cite, found that, under certain conditions, incentives can backfire. The pop romanticists, though, have reduced these findings to a simplistic dichotomy: intrinsic motivation is good, and extrinsic motivation is bad. In fact, psychologists have demonstrated that educators can leverage both kinds of motivation. Many studies show benefits to students when they are held accountable for their academic performance, whether from strict-grading teachers, large cash incentives for academic success, or classroom reward systems. Off the Mark fails even to mention this body of research, let alone engage with it to synthesize a new approach to assessment.

Schneider and Hutt also object to grading interim assignments such as homework. “If students are going to receive cues about the kind of work that is important in school, those cues should point to substantive knowledge and skills,” they argue. Yet one could counter that accountability for short-term performance serves a valuable purpose. Grading such work motivates students and deters them from procrastinating. Without shorter-term goals, even motivated students may wait until the end of the semester to cram for the final exam. By ignoring the substantial body of scholarship connecting student academic motivation to accountability, Schneider and Hutt’s analysis is left undertheorized and incomplete.

The authors’ recommendations for change include both level-headed suggestions and ideas that are less compelling. They make three main proposals for reform: allow students to “overwrite” prior grades; base assessment on “a common set of performance-based tasks . . . aligned with a common set of competencies”; and deepen the information that transcripts convey by making them “double-clickable.” As an example of the latter, they recommend the work of the Mastery Transcript Consortium, which places students’ secondary school experiences into a format akin to “a high schooler’s LinkedIn.”

The idea of overwriting grades offers a distinction without much difference, because transcripts already reflect observable progress (or lack thereof) in each subject a student takes. If the student earns a C in Algebra I and an A in Algebra II, the progress is obvious; students are free to highlight it, and college admissions officers are free to take it into account. Making grades “overwritable” adds another mechanism for inflating grades while encouraging students to procrastinate. “I’ll figure out how to factor polynomials later,” an Algebra II student might well conclude.

Their second suggestion, basing grades on “performance-based tasks,” is akin to using portfolio assessments. This concept is controversial, but if it is part of “a system that incorporates both grades and portfolios”—and some external tests—it could encourage students to focus on developing skills that other assessments might miss while conveying more qualitative information to stakeholders. In other words, if digital portfolios complement the traditional assessment technologies rather than displace them, they could add real value. Schneider and Hutt point to Advanced Placement and International Baccalaureate as examples of programs that, at least for some subjects, successfully combine a variety of assessments, including examples of student work.

As for their third recommendation, consolidating the information in digital portfolios with student transcripts, the Mastery Transcript Consortium they suggest as a model has already pivoted to a format that better incorporates traditional transcript material, ensuring that GPA and assessment outcomes from AP and college admissions exams are available alongside the new, qualitative elements.

The key reason the analysis in Off the Mark falters at times is that even as the authors view students as rational and strategic, they oppose leveraging those qualities to incentivize greater learning. They offer no evidence to suggest that relying on intrinsic motivation alone can address students’ disinterest in academics and today’s skyrocketing absenteeism. In their recommendations chapter, they write that “addressing extrinsic motivation [by removing stakes attached to grades] at least opens the door for conversations about how to foster intrinsic motivation.” Ignoring the idea that education systems might need to engage both types of motivation, the dichotomy leads the authors to recommend “minimizing, to the extent possible, the use of carrots and sticks.”

Left off the menu are reforms to address the faults of current accountability measures, such as improving standardized tests so they rely less on multiple-choice questions or separating teaching and assessment so as to disrupt the morally hazardous dynamic between students and teachers. Both could help solve the problems Schneider and Hutt identify in their book. Unfortunately, the authors’ distorted view of human motivation too often leads their analysis astray.

Sometimes it seems as if we’ve tried everything in our efforts to reform public education, yet nothing has worked to boost student achievement at scale. And despite all of our reform attempts, we have ignored one of the most promising catalysts for student success.

What is this magical, elusive factor?

Student effort.

As education economists John H. Bishop and Ludger Woessmann have put it, “Student effort is probably the most important input in the education process.”

The principle is simple: when students work harder, they learn more. In the United States, though, we don’t expect most kids to work very hard, and they don’t. For all of the talk about “raising standards” and implementing “high stakes testing,” the United States is an outlier among developed nations when it comes to holding students themselves to account, and linking real-world consequences to academic achievement or the lack thereof.

In this article, we look at the evidence that external motivation can encourage middle-school and high-school students to work harder and learn more. We then identify a number of state and local policies that could put constructive pressure on students to exert effort in their academics. Such policies include instituting external, curriculum-based exams linked to real-world consequences for kids; maintaining high standards for earning good grades; and experimenting with well-designed cash-incentive programs. We conclude by considering how student accountability and student agency might combine for an even more effective approach in the future.

Students as Stakeholders

It might seem obvious that students have the biggest stake in their academic success. Education is correlated with future income and important measures of quality of life, and it is the students themselves who will eventually reap the benefits of their efforts in school—or the costs of their indifference. But the operative word here is eventually. To many adolescents, the adult future feels far away, uncertain, and generally unrelated to mastering algebra, understanding the stages of mitosis, or identifying dangling participles.

When even adults debate the payoffs of academic learning, it should be no surprise that many students do not see the “real world” relevance of their schoolwork. But even when they believe in the value of academics, teenagers may still prefer to spend their energy on the more-compelling activities competing for their attention—friends, sports, afterschool jobs, Snapchat, video games, not to mention less-wholesome pursuits. Delaying gratification is hard for most anyone, but researchers have shown that young people are especially present-focused, averse to planning for the longer term and struggling to overcome the impulse to procrastinate. The education system puts students in a position where, as Alexandra Usher and Nancy Kober of the Center on Education Policy expressed it, the “costs are up-front . . . while the benefits are delayed and sometimes difficult to grasp.”

The question is, what might be done to motivate adolescent students to work harder? The optimistic—one might say unrealistic—answer is to make schools so engaging, and the student-teacher relationship so supportive, that adolescents will be intrinsically motivated to work hard, despite the other demands on their time and attention, and despite the social costs they might pay.

Yet it’s hard for policymakers such as governors, legislators, and even school board members to move the needle on students’ intrinsic motivation. They can try to do so indirectly, via initiatives to recruit and retain talented teachers, to implement high-quality curricula, or to include measures of student engagement in school accountability systems. But those are all bank shots at best.

Another approach—one that we believe is more realistic—is to hold students themselves accountable for their performance by ensuring that their work is tied to real consequences. This approach is based in research and used throughout much of the world. By giving students a greater and more immediate stake in their schoolwork and their learning, such student-accountability policies could bridge the gap between effort and reward.

Accountability Boosts Effort

The case for holding students accountable for their schoolwork and their learning has been undercut by the prevalent belief that incentives and other “extrinsic” motivators actually decrease student effort by eroding students’ intrinsic desire to learn. Psychologists in the 1970s discovered how extrinsic motivators could sometimes undermine intrinsic drive, and this idea has been widely popularized, most famously by Alfie Kohn’s 1993 book Punished by Rewards. Kohn and other education writers demonstrated how incentives can backfire, and they bolstered their cases with memorable anecdotes of daffy incentive initiatives, such as a Denver Planned Parenthood program’s offer to pay teenage girls a dollar a day not to get pregnant.

Yet these writers overstated the case against external motivators. The psychology literature never supported their blanket claims that “incentive plans cannot work,” as Kohn put it in the Harvard Business Review, and the conditions under which external motivators backfire are, according to a 1996 meta-analysis on the topic, “limited and easily remedied.” The evidence that external accountability lowers student motivation is mixed. Researchers found that external exams in Germany caused students to work harder, increased their performance, and made students more likely to want a job involving math, but the researchers also found that exams negatively affected students’ enjoyment of math and feelings of competence. When Bishop examined the effects of high-school exit exams, one traditional form of external accountability, on intrinsic motivation by comparing whether students subjected to this approach engaged in less reading for pleasure or were more likely to associate learning with rote memorization, he found no evidence that accountability undermined natural curiosity and even found some evidence of the opposite. The logic of Bishop’s finding is that systems that incentivize students to master academic material may in fact increase intrinsic drive, an unsurprising result for those of us who see learning as empowering.

Another way accountability can boost intrinsic motivation is by supporting pro-academic norms. As James Coleman observed as early as 1959, students often gang up to pick on the “curve raiser”: when students are graded on a curve relative to one another, those who work hard and raise the class average make things difficult for other students, who must then work harder for their grades (see “The Adolescent Society,” features, Winter 2006). This situation has been explored more recently by other social scientists, who have found that it can lead to social norms under which “nerds” are harassed and studious students of color are accused by their peers of “acting white” (see “‘Acting White,’” features, Winter 2006).

Smart student-accountability systems can help solve this problem—by setting high academic standards and, most crucially, by using external assessments to evaluate student progress. This means that policymakers may positively influence intrinsic motivation by optimizing student incentives, resulting in more pro-academic social norms as well as increased student interest and competence. In more recent years, behavioral economists have used experimental methods to better understand the connections between external motivation and human behavior and avoid the pitfalls Kohn and others have flagged. We discuss this further below, but behavioral economics has provided new experimental evidence that policymakers should be sensitive to the timing of accountability, ensure that positive incentives are not too small, and target students at the right ages.

And regardless of the interaction with intrinsic drive, external motivators can have powerful positive effects on student learning in their own right.

External Exams

Important evidence for the effect of student accountability on effort and achievement comes from the literature on curriculum-based external assessments. Several studies from the late 1990s and early 2000s support a strategy of using such external exams, showing that countries, Canadian provinces, and American and German states using content-based external exams for student accountability outperformed comparison jurisdictions, most likely because increased student stakes led to greater student effort. Yet such external exams have many forms and have not been equally successful in all contexts.

Substantial evidence from around the world has linked high-school exit exams to increased learning, but in the United States, where political pressures to relax graduation requirements have always kept the passing bar low, the evidence for their benefit has been inconclusive. Studies have variously found small positive effects, small negative effects, or, often, no effects. American researchers have also focused on whether such exams might induce students to drop out, with several studies finding greater dropout rates following the adoption of the exams.

Yet such pass-or-fail exams are not the only way to use external assessments to promote student accountability. In a recent paper, Anne Hyslop makes a case against the use of exit exams but argues that external assessments can be used in other ways to promote student accountability. In the past 20 years, many states have begun to require external end-of-course exams (EOCs) covering core subjects such as algebra, biology, and American history, often with consequences attached to a student’s performance. Some states have made passing the exams a condition for graduation, essentially turning them into exit exams, but others have increased the stakes for students instead by printing the EOC scores on student transcripts or factoring the scores into course grades. As with external exams in many other countries, EOC results here are typically reported in terms of specific performance thresholds (such as advanced, proficient, needs improvement) rather than as simple pass-or-fail grades, enabling clearer signals of academic performance. This more-nuanced form of signaling also increases the stakes for students, since it gives college admissions officers and potential employers additional information with which to evaluate candidates—an especially important factor in an era of grade inflation. While such a system is not yet mature in the United States, EOCs could form a powerful mechanismfor student accountability if adopted on a broader scale.

The benefits of external assessments are clear for the students enrolling in Advanced Placement and other elite programs that are trusted by colleges in large part because they are externally validated. AP helps solve the “curve raiser” problem by setting an external standard that is not controlled by the teacher, and one that all students in a given class can potentially meet. AP exams are graded by faraway educators, and high scores can earn students valuable college credit. In a sense, this turns preparing for AP exams into a team sport, giving the nerds permission to study hard and crush the test. It also breaks down the pernicious “avoidance treaties” between teachers and students, which Arthur B. Powell of Rutgers University has warned about: that is, the tacit agreement in some high schools that teachers won’t expect much of students, and vice versa. Without bargaining among students or between the students and the teachers, no one has an incentive to lower standards.

Yet even with the expansion of the AP program in recent years, only about a third of American students take at least one exam, and less than a quarter pass at least one test with a score of three or higher. The promise of high-quality EOCs is to extend the benefits of external assessment, and its virtuous cycle, to many more teenagers.

And non-elite students may disproportionately benefit from smart student-accountability policies, such as EOCs combined with real stakes for the students. Since incentives and external motivators have the strongest impact on students with low initial intrinsic motivation, such programs will have an outsized impact on low-achieving students, whose intrinsic motivation is often lower.

Additionally, the power of strong signals of academic performance—enabled by meaningful grades and test scores—has greater importance for students trapped in low-performing schools. Without meaningful signals of achievement, these students can excel yet have difficulty distinguishing themselves from their peers. Research shows that minorities accrue greater premiums from educational credentials that signal high achievement than whites, which means that watering down these signals through grade inflation, abolishing external exams, and lowering standards depletes a key resource for students from disadvantaged backgrounds. These students often lack the family connections and other advantages their more-affluent peers depend on, making academic signals even more important.

Don’t Forget the Carrots

Requiring students to pass end-of-course exams is certainly an eat-your-broccoli approach to student accountability. Carrots are worth considering, too.

Take, for example, the College Readiness Program of the National Math and Science Initiative (NMSI). Offering substantial cash rewards to students and their teachers, the NMSI program has helped hundreds of thousands of students from low-income families succeed in Advanced Placement coursework. Cash incentives for students have a mixed record, with researchers generally finding greater effects when behaviors (such as reading books) rather than outcomes (such as end-of-year test scores) are incentivized. Yet robust evaluations of NMSI’s program, conducted by the economist Kirabo Jackson, show how incentivizing outcomes can powerfully affect both short- and long-term student outcomes, particularly when coupled with teacher support (see “Cash for Test Scores,” features, Fall 2008). In this case, teachers play an especially important role, because even if incentives increase student effort, their work will not bear fruit if the students don’t understand how to achieve the desired outcome.

Jackson’s evaluations of the NMSI program show that it increases college attendance by 4.2 percentage points while increasing college readiness as well as longer-term workforce outcomes. For some students, the effects are particularly strong: Hispanic students see an impressive 11 percent gain in earnings when exposed to the incentive program. Although pay-for-performance policies have often targeted teachers and administrators, NMSI’s program demonstrates that including the students themselves in such policies, if done right, can have game-changing effects.

Policymakers thinking of adopting cash incentive programs should take to heart the lessons of behavioral economics. One rule put forth by Bradley Allan and Roland Fryer in a 2011 white paper on education incentives is, “Don’t be cheap.” A distant incentive that amounts to pennies per hour for increased effort is more likely to make students indignant that their work is not being valued than to stimulate additional effort. Timing is also critical. While we want students to develop greater self-control and the ability to delay gratification, assisting them in the mastery of academic skills requires that we chop some tasks into smaller chunks and help students overcome procrastination by offering shorter-term rewards. To optimize these policies, education policymakers should continue to examine the latest from psychology and behavioral economics.

Lowered Expectations

While end-of-course exams and cash incentives carry great promise, other current “reforms” actually serve to discourage student effort. The most concerning trend is the push to reduce teachers’ authority to assign low grades for poor performance or late assignments. A number of districts nationwide have adopted “no zeroes” policies, banning grades lower than a 50 or 60 on any given assignment or exam, under the rationale that such low grades could make it mathematically impossible for students to recover. Several districts have also implemented “mandatory retake” policies, requiring that teachers allow students to retake exams or redo assignments if they receive a low grade the first time.

Perhaps the intentions behind these policies are pure, but they amount to the soft bigotry of low expectations when it comes to student effort and responsibility. Kids soon figure out that they can procrastinate on assignments or studying for exams without having to face the music, at least in the short term. Teachers lose a valuable tool for discouraging that kind of behavior and promoting effort and diligence. When schools expect less and less of students, we shouldn’t be surprised that students game the system.

Accountability and Agency

A focus on student effort and accountability may sound old-fashioned in an era when personalized, “competency-based education” is all the rage. But here’s the good news: the two go together like peanut butter and jelly.

Consider, for example, an experiment conducted by the behavioral economist Dan Ariely: in one of his courses, he set a different policy for turning in assignments in each of three class sections. One section of students could turn in their assignments at any point during the semester, including the last day; the second group had deadlines spaced across the term; and students in the third section had the option of pre-committing to deadlines of their own choosing—deadlines that, if missed, would result in consequences for the students. In that third section, where students could choose restrictions or absolute freedom, all students chose some restrictions, voluntarily setting up consequences for themselves that enabled the instructor to hold them accountable. In other words, almost all the students with a choice opted for accountability that had teeth. And they were smart to do so, because it was those in the section with maximum freedom and no accountability for deadlines who performed worst on the class assignments. Middle-school and high-school students may sometimes require a more paternalistic approach, but Ariely’s experiment shows that accountability does not necessarily have to be imposed from the top down.

A promise of introducing new technology into classrooms is that it will customize and personalize a student’s experience, often by increasing her choice. Student accountability enables a kind of “loose-tight” management of students, by which they are afforded greater flexibility over how to acquire a set of knowledge and skills (loose) and held strictly accountable for their outcomes (tight). Giving students greater agency over their learning and allowing them to move at their own pace may boost student interest and allow students to learn more quickly and efficiently. But we shouldn’t naively assume that most students will put in the effort to make these new systems work without caring adults guiding them and holding them accountable. It’s telling that the darling of personalized-learning aficionados, Summit Public Schools, makes extensive use of the Advanced Placement program in its high schools (see “Pacesetter in Personalized Learning,” features, Fall 2017). The high standards, external exams, and incentives baked into the AP program provide effective mechanisms for holding students accountable for working hard and making progress.

Unfortunately, too many policymakers are moving schools in the wrong direction by removing the few tools, such as meaningful grading standards and high-quality end-of-course exams, that might encourage more student effort.

Students benefit from accountability, and, given the right circumstances, they choose it. As reformers and entrepreneurs seek new applications of technology and innovative models of instruction to revolutionize education systems, schools must reassess their comparative advantages. In their roles as academic-community builders and the gatekeepers of credentials, school leaders should embrace the responsibility of holding students accountable.

Adam Tyner is associate director of research at the Thomas B. Fordham Institute. Michael J. Petrilli is president of the Thomas B. Fordham Institute and executive editor of Education Next.