Ever since my colleagues at the National Assessment Governing Board announced our plans to release the latest National Assessment of Educational Progress results on schedule this month, there’s been speculation about how those results could influence the upcoming midterm elections. The NAEP Long-term Trend results for 9-year-olds, released in September, confirmed that the math and reading skills of American students had dropped sharply over the course of the pandemic. The Main NAEP results provide results for students in both 4th and 8th grade. More importantly, they make it possible to see which states and large school districts fared best in the Covid era—and which lost the most ground. Could the results make a difference in competitive governors’ races in states like Kansas, Nevada, and Wisconsin? Could they become a partisan talking point in congressional races nationwide?
Well, those results are now out, and, at least at the state level, the patterns defy easy politicization. Scores fell nearly everywhere. Math scores were down from 2019 by a statistically significant amount in 41 states in 4th grade and 49 states in 8th grade. Reading scores were down in 30 states in 4th grade and 33 states in 8th grade. Scores also generally fell in those states where achievement levels were officially unchanged; the drops just were not large enough to reach statistical significance.
The amount of ground that students lost does vary across states, but that variation doesn’t fit simple narratives. For example, the figure below shows the relationship between the drop in eighth grade math test scores between 2019 and 2022 and one of the best available measures of the prevalence of remote instruction during the 2021-22 school year. I focus on 8th grade math because that was the grade level and subject area where scores fell by the largest amount, and because math achievement tends to be more influenced by what is happening within schools. (That said, the relationship is very similar if I instead use the average decline in scores across all four tests.) The measure of remote learning is based on the American Enterprise Institute’s Return to Learn Tracker, which provides weekly data on mode of instruction from August 2020 through June 2021 for virtually all U.S. school districts with three or more schools. Following Dan Goldhaber, Tom Kane, and their colleagues in a recent analysis of the effects of remote instruction, I aggregate the data to the state level to provide a rough measure of the average share of students who had access only to remote instruction over the course of the year. States that went for Donald Trump in the 2020 presidential election are represented by red markers, while states that went for Joe Biden are shown in blue. The dotted horizontal line shows the best linear relationship between remote instruction and test score changes.
That relationship is negative (and statistically significant), suggesting that students lost more ground on average where remote instruction was more prevalent—but the relationship’s strength is relatively weak. The correlation coefficient is -0.3, which means that this measure of remote instruction explains less than 10 percent of the overall variation in test score changes across the 50 states (plus DC). The clustering of red states at the left of the figure confirms that states that went for Trump were in fact more aggressive in reopening schools for in-person instruction, but that wasn’t enough on its own to generate large differences in aggregate state-level test-score changes. Red states lost 7.2 NAEP scale score points in 8th grade math, on average, while blue states lost 8.8 points.
Looking more closely at the results for three politically salient states, we see that the vocal efforts of Florida Governor Ron DeSantis and Texas Governor Greg Abbott to reopen schools early were not enough to keep students from losing roughly the national average decline of eight points in eighth grade math. Texas students did manage to avoid a significant decline in 4th and 8th grade reading, and Florida students held steady in 4th grade reading. We can expect to hear them and others trumpeting those results. But any efforts to claim credit for them will be complicated by the fact that California students lost only six points in 8th grade math—far less than would be expected given the slow return to in-person instruction seen under Governor Gavin Newsom—and also fared well on a relative basis in other grades and subject areas.
None of this is to claim any kind of success for our national experiment with remote instruction over the past two school years. The AEI data on remote instruction are far from perfect, not least because safety concerns led many parents to keep their children at home even when in-person options were available. NAEP data are not well suited to parsing out the relative effectiveness of different modes of learning. Other research using different data sources provides convincing evidence that students learned far less over the course of the pandemic when they weren’t in school buildings. But recall that essentially all schools nationwide were shuttered in Spring 2020—and that school closures were hardly the only way in which the pandemic could have slowed student progress. The end result is a pattern of results across states that can’t be reduced to a single factor.
And that is probably a good thing. NAEP data are always better at telling us what has happened with respect to our students’ achievement than why. The latest results confirm that scores are down virtually everywhere. Rather than using them to score political points, let’s focus on what it will take to help students make up lost ground.
Martin R. West is editor-in-chief of Education Next and a member of the National Assessment Governing Board. He is academic dean and Henry Lee Shattuck Professor at the Harvard Graduate School of Education and deputy director of the Program on Education Policy and Governance.
