Standards, Testing, and Accountability - Education Next https://www.educationnext.org/ednext-blog/standards-testing-and-accountability-blog/ A Journal of Opinion and Research About Education Policy Thu, 18 Apr 2024 02:06:26 +0000 en-US hourly 1 https://wordpress.org/?v=6.5.5 https://i0.wp.com/www.educationnext.org/wp-content/uploads/2019/12/e-logo.png?fit=32%2C32&ssl=1 Standards, Testing, and Accountability - Education Next https://www.educationnext.org/ednext-blog/standards-testing-and-accountability-blog/ 32 32 181792879 Doing Educational Equity Wrong https://www.educationnext.org/doing-educational-equity-wrong/ Thu, 18 Apr 2024 09:00:14 +0000 https://www.educationnext.org/?p=49718126 In the pursuit of something good, there are potential wrong turns

The post Doing Educational Equity Wrong appeared first on Education Next.

]]>

A paper stamped with the word "Excellent" and a grade of "C"

This is the final article in a series on doing educational equity right. See the introductory post, as well as ones on school finance, student discipline, advanced education,  school closures, homework, grading and effective teaching.

For the past several months, I’ve been pumping out posts about “doing educational equity right.” Given that Eight is Enough, it’s time to wrap things up. Let’s conclude with a twist and look at three ways that schools are doing educational equity wrong:

  1. By engaging in the soft bigotry of low expectations.
  2. By tying teachers’ hands without good reason.
  3. By acting like equity isn’t just an important thing, but the only thing.

 

Equity as an excuse for the soft bigotry of low expectations

A recurring theme of this series is how misguided it is for schools to lower expectations for students “because equity.” Of course, the schools and the elected officials, advocates, and journalists who embrace these practices don’t say they are expecting less from students, but that’s precisely what’s happening.

It’s most obvious in the world of advanced education, such as when districts refuse to let anyone take Algebra in middle school because not everyone is ready for Algebra in middle school. The backlash to this mindset is growing, thank goodness.

But other examples abound and unfortunately continue to be lauded in polite company. For instance, the notion that it’s inequitable and unfair to grade, or even assign, homework because some kids don’t have a quiet place to complete assignments away from school. Just examine this idea for a moment. Do we really believe that lots of American families are so dysfunctional that they can’t figure out a way to clear a space for their kids to do their math problems? Or that teenagers can’t find a place—a community library, the school library, even a McDonald’s—where they could get homework done? Why are we infantilizing kids and their parents like this?

Same goes with policies that allow students to turn in assignments late without penalty. Are we trying to teach kids to procrastinate? To teach them that real life doesn’t deal in accountability and consequences?

Or take school discipline. Plenty of well-meaning people who would never say “we can’t expect poor kids and kids of color to learn fractions—it’s just too hard” are more than happy to argue that we must accept all manner of student misbehavior because of poverty or systemic racism. Journalists might be the worst at this. Just last week, a major article from the Hechinger Report decried the use of suspensions and the like for “subjective infractions like defiance and disorderly conduct.” It’s one thing to be concerned about bias in meting out penalties for disruptive behavior. But as my colleague Daniel Buck wrote, in the real world of classrooms, this leads to paralysis from officials in the face of flagrant, over-the-top, disrespectful behavior by kids. And to misery for their teachers.

Permitting low-level defiance—defining deviancy down in this way—facilitates and fosters more severe misbehavior. If a student comes to learn that adults can be ignored and rules flaunted, behavior escalates. A balled piece of paper is thrown, a teacher asks the offender to move seats, but he refuses. The next day, he’s wandering around the classroom singing. The teacher asks him to sit, but he refuses. Eventually, he’s wandering the halls, telling teachers to “fuck off” if they ask him to return to class, so most don’t. Many other students have joined in the fun, and now there’s cacophony in the halls. Students in class question why they must listen to adults if they don’t want to when other kids get to flaunt the rules. Rowdy, unmonitored halls mean more chances for student conflict and fights.

Surely we can agree that all students, regardless of the challenges they face due to poverty or racism, should be expected to treat their teachers respectfully and comport themselves in a reasonable manner. Teachers in other nations would be aghast if told they had to accept this sort of treatment as part of the job. Indeed, I bet 99 percent of these kids’ parents would be alarmed, if not angry, to learn that their kids were being allowed to behave so atrociously in school.

“Defining deviancy down”—whether in academics, homework, grading, or behavior—will only let our students down. We should stop doing it.

 

Tying teachers’ hands

Another big mistake some equity advocates make is reducing teacher authority and autonomy for no good reason. To be sure, educators shouldn’t always have carte blanche to do whatever they like; bias is real and it’s one reason we’ve worked to get high, consistent academic standards in place and required teachers to follow them—ideally with the help of well-aligned, high-quality instructional materials. Again, to push back against the soft bigotry of low expectations.

But too often advocates force educators to teach with one or both hands tied behind their backs—refusing to let them use time-tested, effective practices because they conflict with recent preachings of the high church of educational equity.

For example, some districts don’t allow elementary teachers to group students by achievement levels when teaching reading or math, and many more have moved to “de-track” middle school and high school courses, getting rid of “on-level” courses and putting everyone into (wink-wink) “honors” ones. Now imagine you’re a seventh-grade teacher. If your class is typical, your students enter your classroom at achievement levels ranging from third through eleventh grades. So your helpful district-provided instructional coach suggests that you cope by “differentiating instruction.” You might as well ask them for some magic beans so you can grow a sky-high beanstalk while you’re at it. Certainly they’re guilty of magical thinking.

Most research finds that grouping students by achievement tends to help everyone learn more, especially if those groups are flexible and continuously re-mixed. But because progressive education dogma declares any form of grouping or “tracking” to be suspect, we make life dramatically harder for teachers and make learning dramatically less effective for kids.

There are plenty of other examples. Telling teachers they can’t send disruptive students to the office and making them engage in lengthy “restorative justice” circles instead. Mandating minimum grades of 50 percent even when kids don’t turn in research papers or show up for tests. Not letting teachers dock students for missed homework assignments or refusing to participate in class discussions.

Constrained teachers are disgruntled teachers—which is bad for everyone and bad for equity.

 

Is equity like winning—the only thing that counts?

Finally, some educators and advocates act as if equity were the one and only value in education worth pursuing. I think this comes from a good place; no doubt our system has a long and sordid history of mistreating poor kids and kids of color. A swing of the pendulum was long overdue, and erring in the direction of equity is no terrible crime. But policies and practices that ignore everything else—and everyone else—will prove harmful and unsustainable.

So what are the other values that matter—or should, in our universal public education system? I would put excellence at the top of the list. That means doing right by our high-achievers, who hold particularly great potential for solving our world’s problems and boosting our economy someday. But it also means striving for excellence in everything that schools do, from the basics of teaching and learning, to tutoring and counseling, to extra-curricular activities and more.

A commitment to excellence need not conflict with a drive for equity. Indeed, as I wrote last year, excellence is not the enemy of equity. It’s mediocrity that is the enemy of equity as well as excellence. So we must raise the alarm when “equitable practices” promote mediocrity instead.

Another important value is efficiency. Even America’s relatively well-funded public education system doesn’t have unlimited resources. Trade-offs are inescapable. But we’ll be more likely to land on effective approaches if we look for practices that promote equity and excellence and efficiency. When it comes to discipline and student behavior, for example, it’s not enough to come up with strategies that might be ideal for the disruptive kids. We also must protect the learning environment of their peers and consider the demands on teachers’ limited time.

So it is with the difficult issue of under-enrolled schools. Equity advocates may want districts to avoid closing schools with high proportions of poor kids and kids of color. But if those are the schools with dwindling student populations, such an outcome may be unavoidable—again, because excellence (getting kids into better schools) and efficiency (not wasting money on tiny campuses) matter, too.

Equity advocates shouldn’t be myopic. Balancing their impulse for fairness with concerns for excellence and efficiency will make it more, not less, likely that they will achieve their goals.

 

* * *

I don’t want to end on a sour note. These many months (and words) spent digging into educational equity make me optimistic that common ground can be found, even on contentious issues. If we assume positive intent, look for practical answers, and avoid getting hung up on culture-war fights over language, we can move beyond the squabbles and toward solutions. Let’s do educational equity right—and let’s do it now!

Michael J. Petrilli is president of the Thomas B. Fordham Institute, visiting fellow at Stanford University’s Hoover Institution, and an executive editor of Education Next.

This post originally appeared on the Fordham Institute’s Flypaper blog.

The post Doing Educational Equity Wrong appeared first on Education Next.

]]>
49718126
The Alexander Doctrine: Governors are Agents of Change https://www.educationnext.org/the-alexander-doctrine-governors-are-agents-of-change/ Mon, 25 Mar 2024 09:01:07 +0000 https://www.educationnext.org/?p=49718043 The former governor and senator reflects on the importance of engaging with state executives in education reform

The post The Alexander Doctrine: Governors are Agents of Change appeared first on Education Next.

]]>
Lamar Alexander, left, and Martin West sitting on stage
Senator Lamar Alexander and Education Next editor-in-chief Marty West discuss the origins of the National Assessment Governing Board at the board’s quarterly meeting in February.

In education you need to figure out how to engage governors. So said former Tennessee Governor, U.S. Secretary of Education, and U.S. Senator Lamar Alexander to the National Assessment Governing Board in February in advance of its quarterly meeting.

Senator Alexander sat down for a conversation with board member and Education Next editor Marty West in Nashville, Tennessee, to talk about the creation of the Governing Board 35 years ago and how he approached his work by prioritizing accountability, transparency, and communication.

The interview has been edited for length and clarity.

Marty West: Improving K–12 education was arguably the dominant theme of your career in public service. What drew you to work on education in the first place, and what kept you coming back?

Senator Lamar Alexander: Why did I get involved? The bottom line is two things. One is that my parents were teachers. My mother was a preschool teacher, my dad a principal. So, I valued education. But when I became governor in 1979, Tennessee was the third poorest state, third poorest in family incomes. It’s kind of hard to think of that today. I mean, today we pat ourselves on the back for being a leading state in autos, but we didn’t have a single auto plant in 1979.

So I was trying to think about how to change that. We’re going to have to create the environment to grow jobs, not just recruit jobs, and growing jobs meant workers with skills. So better schools meant better skills, which meant better jobs. That was sort of the mantra. It took me a little while to figure that out, but that’s where I put my attention. And once I got into it, I saw it created results.

And then, as I moved along, I ended up as president of the University of Tennessee, and I’d seen how much a really quality university can help a state rise. Then President [George H.W.] Bush interrupted that, and I got to be the education secretary, and that was not long after when the governors and President Bush got together and created a set of national education goals.

And then in the Senate, I ended up on the committee in charge of education, and by that time there was a real need to fix No Child Left Behind. Almost every school in the country was, by definition, failing. Newsweek said everybody wanted NCLB fixed, and people said, well, your chances of doing that are 10 to one against. But working with President Obama, the Republican House, and Senator Patty Murray from Washington State, we succeeded in what President Obama called a “Christmas miracle” and basically turned authority back over to states with the passing of the Every Student Succeeds Act.

West: You just gave us several of the highlights of your career, but you left out your involvement in the creation of this body. In 1986, you agreed to chair a working group commissioned by the Secretary of Education to take stock of the National Assessment of Educational Progress. That led to the Alexander-James report, which you all put out in January of 1987 with the title “The Nation’s Report Card.” I’m told by Checker Finn, who was on that working group and who was the first NAGB chair, that you were the one who coined that term. Do we really have you to thank for that?

Sen. Alexander: With a name like National Assessment for Educational Progress, people are going to be asleep before you finish saying the name! Assessment is a dull topic to begin with, and you make it even duller with that, so I’d said, “Let’s call it the Nation’s Report Card, and somebody might pay attention to what you’re doing.”

West: The report’s most important recommendation was to begin assessing achievement not just at the national level, but state by state and for the District of Columbia. How did that idea fit into your vision?

Sen. Alexander: At the beginning of my second term as governor, “A Nation at Risk” came out from the U.S. Education Department. Well, the nation might have been “at risk,” but there was no way to find out if our states were. So, I was trying to say if we’re going to try to improve schools, we need to be able to measure results at the state level. Governors were saying in the early 1980s, “Give us some way to tell if our students are really learning.” And so we had the Alexander-James Commission, and it recommended state testing, it recommended an independent board to oversee the program—NAGB—and it recommended the system of achievement levels—basic, proficient, and advanced—that you use today.

And thanks to President Reagan working with Ted Kennedy, it was passed in 1988. That’s almost a world record. For coming up with a Commission in 1986, having a result in early 1987, and a law in 1988! That’s where this Board came from and the genius behind it is—I didn’t want the federal government involved in it because they’ll screw it up—and so the report said everybody on the Board needs to represent states and localities and needs to be from both parties.

West: You noted that governors in the 1980s were hungry for state-level results. We now give states the opportunity to participate separately on some NAEP assessments like history and civics. And when we give them that opportunity and say we’ll pay for it, we get only about a dozen states that opt in. What I think that tells me is that we need a new group of governors who have the same desire to learn about what’s working and not working and be transparent about it. It’s the same type of attitude that you expressed seeing with the education governors in the South back in the 1980s.

Sen. Alexander: Well, among other things, it’s good politics. I mean, when I was running for the Senate, if I would go into a room and say, “It’s time we put the teaching of American history and civics back in its rightful place in our schools so our children can grow up learning what it means to be an American,” people would stand up. And if you get a politician and you give him a line that will cause people to stand up, he’s likely to favor it and maybe even fund it.

Governors don’t really listen to each other that well. But if somebody comes up with a really good idea, they’ll go right back home and do it because it’s good for their state, and it’ll help them get reelected.

If a governor hears another governor say, “I found out that our kids were not doing what they should be doing in American history and civics. And we’re doing something about it,” you’ll find a lot of other governors doing it and paying for it. They might do the same with science.

The post The Alexander Doctrine: Governors are Agents of Change appeared first on Education Next.

]]>
49718043
Doing Educational Equity Right: Grading https://www.educationnext.org/doing-educational-equity-right-grading/ Thu, 14 Mar 2024 08:50:53 +0000 https://www.educationnext.org/?p=49717983 Schools need to empower teachers to expect students to work hard for that A

The post Doing Educational Equity Right: Grading appeared first on Education Next.

]]>

A student is handed a paper by a teacher

This is the seventh in a series on doing educational equity right. See the introductory post, as well as ones on school finance, student discipline, advanced education,  school closures and homework.

Student grading is one of those issues that has an enormous impact on kids and schools, yet for years it remained almost completely under the radar. Barely discussed in teacher preparation programs, rarely a topic in professional development sessions, there must have been an assumption that there was nothing much to it or that everyone knew how to do it well.

Thanks to a few intrepid advocates, including Joe Feldman and Douglas Reeves, the issue is a sleeper no more. As these and other analysts and commentators correctly observe, how we grade student work is one of the most important ways that we communicate expectations to youngsters, as well as relay critical information to parents, institutions of higher education, and potential employers. Yet as these folks explain, traditional grading practices can be riddled with bias, including racial bias. Hence the drive for “equitable grading,” which seeks to squeeze this bias out of the system.

So far so good. Yet like so many things in today’s K–12 world, “equitable grading” has often been implemented piecemeal, bringing along with it all manner of unintended consequences, the most important of which is lowered standards.

A new white paper by my colleagues Meredith Coffey and Adam Tyner, part of our “Think Again” series, unpacks all this. Here’s a useful table from their paper that illustrates the scope of what we’re talking about:

Table 1: Equity-oriented grading policies

Table 1: Equity-oriented grading policies

So how can school districts “do educational equity right” when it comes to grading? To address concerns about bias while keeping standards high? Let’s return to this series’ three big rules:

  • When aiming for equity, we should level up instead of leveling down.
  • We should focus on closing gaps between affluent students and their disadvantaged peers, not between high-achieving students and their lower-achieving peers.
  • We should focus equity initiatives primarily on class, not race.

Grading practices that level up instead of leveling down

This is the heart of the matter. As we discussed in our homework post, student effort is key to student learning. Especially as kids get older, they cannot master challenging material by osmosis. They have to put in the work—do the practice math problems, write and re-write and re-write the research paper, study for their history and science tests. And regardless of what you might have read by Jean-Jacques Rousseau or Alfie Kohn, most adolescents are not going to choose to spend extra time and effort on schoolwork because of their intrinsic love of learning.

Tough grading practices, then, are essential to motivating students to work harder. Indeed, several research studies, including a Seth Gershenson paper published by Fordham, demonstrate that teachers who are tougher graders have a clear positive impact on learning, even after their students leave their classrooms. That goes for students from all racial subgroups and those in high-poverty schools. If we’re making it easier for students to get A’s and B’s, that’s a problem.

Yet there’s no doubt that we’re making it easier. The evidence on grade inflation continues to pile up. Like so many negative trends in U.S. schools, this phenomenon started before the pandemic, but the Covid era made it markedly worse. That’s partly because schools understandably relaxed standards while kids were Zoom-schooling, but it’s also the case that the push for “equity” in grading gained steam after the racial reckoning of 2020.

Now we’re in a situation wherein elite colleges can’t rely on student grades to send clear signals because almost everyone applying for admission gets straight A’s. On the other end of the academic spectrum, our high school graduation rate is higher than ever, in part because standards for graduation—which are of course tied to getting passing grades in required courses—are lower than ever (yay?).

What this all means is that we need to take a critical eye to any grading practices that have the effect of lowering standards—even if their advocates didn’t intend them to. In that bucket I’d include the “50 percent rule”—mandating that teachers give no lower than a 50 percent on any test or assignment, even those that kids don’t turn in. For obvious reasons (at least obvious to anyone who has met a teenager or ever been one), some kids will adapt to this system and put in less work. Likewise with bans on grading homework, even for completion. Like many of us, kids need someone to hold them accountable for doing what’s in their own long-term self-interest. And that means hitting the books, doing their homework, and getting feedback on whether they did it well. Same with banning penalties for late work, or even cheating.

That leaves plenty of other grading reforms that are still worth embracing—those that are focused on maintaining high standards while reducing bias—as we’ll discuss below.

Grading reforms and high versus low achievers

One of the major arguments for practices like the 50 percent rule is that giving a kid a zero on a big assignment will tank their grade and encourage them to give up. That’s certainly a possibility. Zero is indeed a long distance from 60 percent, the traditional minimum score for a passing grade, and it very well might be mathematically impossible for a student to make up enough ground to pass a course if they are staring at a goose egg.

But these across-the-board rules don’t just impact the lowest-achieving students—those kids at risk of not passing their courses. They impact everybody—including the strivers and grinders. And those high achievers will adapt their behavior to the new policies, too. If they build up a high-enough A early in the quarter, they know they can bomb some later quizzes or skip some assignments and still get what they need for their transcripts. So they put in less work. That’s not good! (And please keep in mind, plenty of these high achievers are students from disadvantaged backgrounds, so it’s not good for equity either.)

A better approach, then, is to return authority to teachers over their grading practices so they can offer grace to students on a case-by-case basis. If students bomb a big assignment or test early in the quarter, their teachers can make it clear to them that, if they get their act together and do better on future assessments, those zeroes will be adjusted upward. That’s a much better solution than schoolwide or districtwide rules that apply to everyone.

What about bias?

Now for some grading reforms that are well worth supporting: those that indicate higher expectations and work to reduce bias. The bias piece is important; whether it’s because of students’ socioeconomic backgrounds, or race, or the intersection of the two, it’s not hard to see how unintentional bias can creep into the grading process. On the one hand, a teacher might go a little easier on a kid knowing the hardships he or she is facing at home—the soft bigotry of low expectations. Alternatively, a teacher might anticipate lower-quality work from a student given their zip code or appearance—or just their performance on previous assignments.

Thus, various forms of blind grading are to be commended. They could involve electronic grading systems whereby teachers don’t see the students’ name until after they’ve completed their evaluations, or mechanisms whereby teachers grade papers and tests from other teachers’ students. Regardless, those doing the grading benefit from clear rubrics, ideally those that enforce a tough standard for what it takes to be considered A-level work. These sorts of approaches are, of course, at the heart of the Advanced Placement and International Baccalaureate programs and how they evaluate student exams.

Let’s be clear though: We shouldn’t expect blind grading or similar techniques to eliminate disparities in student grades by race or ethnicity. As long as we have achievement gaps—and sadly, they remain quite large—we are going to see GPA gaps, as well.

* * *

The “grading reformers” have performed an important service by surfacing this issue as a critical topic. It’s nuts that for decades we largely just expected teachers to figure out grading on their own and put so little effort into making sure that grade expectations were consistent across schools or districts—or even classrooms!

But some grading reforms are better than others. Educational leaders: Be discerning. Empower your teachers. By all means work to eliminate bias. But make sure that, whatever you do, you keep expectations high.

Michael J. Petrilli is president of the Thomas B. Fordham Institute, visiting fellow at Stanford University’s Hoover Institution, and an executive editor of Education Next.

This post originally appeared on the Fordham Institute’s Flypaper blog.

The post Doing Educational Equity Right: Grading appeared first on Education Next.

]]>
49717983
Revising Graduation Requirements Could Improve Academic Rigor in New York https://www.educationnext.org/revising-graduation-requirements-could-improve-academic-rigor-in-new-york/ Mon, 18 Dec 2023 15:55:58 +0000 https://www.educationnext.org/?p=49717628 The state is right to acknowledge that Regents exams are not the right benchmark for all students

The post Revising Graduation Requirements Could Improve Academic Rigor in New York appeared first on Education Next.

]]>

New York state has a long and proud history of administering rigorous subject-area examinations, allowing its most advanced students to demonstrate mastery of demanding academic content. For the last 20 years, the state has required all students to pass five of these exams—later reduced to four—to earn a high school diploma. The state’s Board of Regents is poised to eliminate that requirement. Critics argue that to do so would mean dumbing down standards.

That criticism is wrong. Doing away with the tests in their current configuration could serve to restore rigor and meritocracy to New York’s high schools, while also ensuring that a more diverse array of students, including those who are not college-bound, are positioned for success in early adulthood.

For over 100 years, the Regents exams were not administered to every student. They aimed to measure the achievement of the highest performers—those deemed capable of college preparatory work. In New York City, home to the nation’s largest public-school system, these exams, like the SAT, were part of the ladder of social mobility for poor, immigrant, and first-generation American students. A “Regents diploma” signified high achievement across a broad range of college prep coursework through four years of high school. Other students received a “local diploma,” signifying that they had attended high school for four years and passed the required number of courses.

The last 20 years, which the Board of Regents is now seeking to undo, have witnessed a failed attempt to raise achievement by setting a bar that most students had never been asked to meet—college preparedness—while simultaneously lowering standards for the academically gifted and wasting the time and talents of the workforce-bound.

Since 2004 all students who pass the five (later four) tests required of all students have been awarded a Regents diploma. A student who passes the full sequence of nine tests receives a Regents diploma with advanced designation. Only students with special needs can earn a local diploma.

This change has resulted in numerous instances of diminishing rigor to accommodate lower-achieving students, thereby defeating the tests’ purpose. Incidents have ranged from outright cheating on the part of school administrators to more benign attempts to “scrub” exams, looking for the few extra points in an essay response that would get a kid over the bar for graduation.

The state education department also seems to have dumbed down the exams’ content and has reduced the number and percentage of correct answers required to pass the test. The English Regents exam, for example, was once a six-hour test administered over two days; it’s been shortened by half. As early as 1996, when the state was phasing in the new graduation requirement, the state threw out the results of a Regents Mathematics exam when seventy students failed it, allowing them to graduate if they had earned a passing grade in their coursework.

These practices, prompted by the ill-advised decision to require all high school students to take these formerly rigorous exams, reached absurd levels during the Covid school closures. The spring 2020 exams were cancelled and the spring 2021 exams were curtailed. In 2022, students were allowed to appeal exam results if they scored at least 50, as opposed to the traditional passing score of 65.

The Regents are right to be moving to undo this requirement. Where they seem poised to err, however, is with their apparent intention to use a single diploma policy for all students. Instead of three diploma types—local, Regents, and Regents with Advanced Designation—they would allow a single Regents diploma to include a notation or seal indicating superior achievement.

The Regents should return to the original system. They should acknowledge that students enter and leave high school with different abilities, motivations, and goals—and ensure all high school graduates meet a high standard of achievement commensurate to their chosen path. Those who enter high school with acute academic prowess should be guided through the old-style Regents exam track on the path to a full Regents Academic diploma—with tests restored to their prior level of rigor. Students more inclined toward workforce preparation should be allowed to pursue a sequence of courses aligned to industry standards in their chosen field and have their performance measured by demonstration of their proficiency on those standards, not by Regents examinations. After all, the labor market doesn’t need only white-collar workers; it needs students who can excel and lead fulfilling lives in technical fields.

Some students flourish in a more progressive educational environment, which reduces the importance of formal examinations. The schools currently in the Performance Standards Consortium, working under a partial waiver of the current testing requirements, are allowed to replace two of the four required Regents exams with their own rigorous assessments and have a strong record of success with these students. The Regents are right to expand that project, with support for the proper training of teachers in the Consortium’s methods. Finally, the local diploma should be reinstated, indicating that a student attended high school (with their attendance really tracked and at a reasonable level) and passed their courses.

The era of pretending that all students want or need to attend college is over. New York’s Board of Regents should accept reality and acknowledge that success and achievement look different for different students. Only then will our schools offer ladders of social mobility to all students.

Ray Domanico is senior fellow and director of education policy at the Manhattan Institute.

The post Revising Graduation Requirements Could Improve Academic Rigor in New York appeared first on Education Next.

]]>
49717628
Generation Lost: The Pandemic’s Lifetime Tax https://www.educationnext.org/generation-lost-the-pandemics-lifetime-tax/ Fri, 06 Oct 2023 17:11:46 +0000 https://www.educationnext.org/?p=49717137 Students’ Covid learning losses portend a future workforce with fewer skills, lower earnings

The post Generation Lost: The Pandemic’s Lifetime Tax appeared first on Education Next.

]]>

Illustration

Reports of drops in student achievement due to the pandemic are now treated as old news. Amid abstract reporting of test results, a sense of inevitability and complacency has developed. After all, could the fact that students’ math scores fell by “nine points” truly be important?

The reality is that the cohort of students in school in March 2020 has been seriously harmed—implicitly facing a lifetime tax on earnings of 6 percent. And the harm is not going away.

A simple way to assess learning loss from the pandemic is to compare the performance of students tested in 2023 to students taking the same tests in 2020. The most recent data come from the National Assessment of Educational Progress for 13-year-olds. Often called the Nation’s Report Card, NAEP provides regular assessments of American students’ math and reading skills at different ages. Comparing 2023 results with those for students tested just prior to the pandemic reveals that losses averaged nine points in math and four points in reading. This drop erased all the gains in students’ math scores since 1990 and moved reading scores back to where they were in 1975! Low-achieving students lost more than high achievers, poor students lost more than nonpoor students, and both Black and Hispanic students lost more than white students.

But NAEP, like most tests, uses an arbitrary scale to report scores that makes the size of changes hard to interpret. The implications of lost learning are better seen by translating these sterile numbers into economic losses. Past research confirms that people who know more, as measured by their performance on tests like NAEP, earn more. The research considers how individuals’ earnings throughout their working lives differ according to the skills measured by scores on standardized math and reading tests. Importantly, the U.S. labor market rewards these cognitive skills more than almost all developed countries—which in turn implies that the U.S. punishes the lack of these skills more than almost all developed countries.

Historical earnings patterns make it is possible to estimate what the learning losses documented by NAEP will cost the average student in the Covid-cohort: 6 percent lower lifetime earnings than those not in this cohort. In other words, the pandemic learning losses for this cohort are equivalent on average to a 6 percent income tax surcharge throughout the students’ working lives. This rises to 8 percent for the average Black student, who suffered greater learning losses according to NAEP.

The economic costs do not end there. The economies of nations with more skilled populations grow faster in the long run, and the pandemic learning losses imply that the U.S. population will be less skilled in the future than it would have been. Using historical growth patterns, it is again possible to project the aggregate losses to the U.S. economy of having this lower-skilled cohort move through the labor force. The economic loss from the lower-skilled workforce amounts in present value terms to $28 trillion.

Costs in trillions of dollars are perhaps no easier to understand than drops in test scores. To put this figure in perspective, consider that the projected loss of $28 trillion amounts to more than one year’s Gross Domestic Product. Or that the aggregate losses due to unemployment, business closures, and related economic fallout from the pandemic totaled about $2 trillion. The losses from the “Great Recession” in 2008 totaled about $5 trillion. In short, the impact on the economy we should expect from pandemic-era learning loss dwarfs the impacts that have so captured public and policymakers’ attention in recent years.

We are struggling as a nation even to get our schools back to where they were in terms of supporting student learning, but these costs will be permanent if we just return schools to the status quo in March 2020. Our schools must improve if we are going to eliminate the burden of lost learning. Evidence from a variety of experiences in other nations shows that the losses students experienced will persist if schools simply return to business as usual. For example, several German states had short school years in the 1960s when policymakers sought to standardize school calendars nationwide. The earnings of students educated during that period stand out throughout their careers from those of students educated before and after the adjustment, and not in a good way. Other examples of extended school disruptions—for example, due to prolonged teacher strikes—show similarly persistent impacts.

What has been done so far to address learning loss? The federal government provided almost $190 billion in Covid relief aid to schools under three separate appropriations. Only a small portion, however, was required to be spent on ameliorating learning loss, and most schools have yet to spend much of these funds even though they disappear in a year.

States and districts have adopted a variety of strategies that most frequently include added instructional time or intensive tutoring. Unfortunately, the results of these efforts to date have not been good. Even if we optimistically project that the best available programs will be implemented with fidelity, the losses will not be erased. The scale of current recovery efforts is simply insufficient to overcome the deficits. Moreover, when recovery programs are voluntary, as is typically the case, higher-achieving students are more likely to participate, leading to a widening of achievement gaps.

At the same time, the pandemic strengthened a number of harmful policy trends that may cause school quality to decline. For one, it reinforced a general drift away from test-based accountability policies. Additionally, teacher unions saw the pandemic as an opportunity to push a variety of their preferred policies—including policies well beyond pay, benefits, or anything related to learning. For example, the Oakland Teachers Association, after agreeing to a substantial pay and benefits hike, nonetheless went on an eight-day strike in May 2023 over “common good” clauses, including reparations for Black students and “environmental justice.”

There is a clear roadmap to success, albeit one that leads to political tension. The one policy that is known from research to be effective is ensuring that all students have an effective teacher. Recruiting and retaining more effective teachers has, of course, been the goal of many policy initiatives, but a variant of this emphasis can be the solution to the learning loss problem: Simply provide incentives for the most effective current teachers to teach more students. The highly-effective teachers could teach larger classes or added sections of courses with both monetary incentives and additional support for this work. Unused federal funds could immediately support this tactic. Indeed, one could go further and use part of the funds to buy out the contracts of the least effective teachers. These steps could instantly improve the average effectiveness of instruction, both making up for pandemic-era learning losses and improving schools going forward.

Such policies have been shown to work in a few large districts, including Washington, D.C., and Dallas. Deploying them now at scale could save the Covid-cohort from a 6 percent lifetime tax. The alternative, saying change is “too hard,” amounts to accepting the lifetime injury to current students along with a $28 trillion national loss.

Eric Hanushek is a senior fellow at the Hoover Institution of Stanford University and a prior member of the National Assessment Governing Board.

The post Generation Lost: The Pandemic’s Lifetime Tax appeared first on Education Next.

]]>
49717137
Massachusetts Teachers Union Aims to Eliminate Standardized Test as High-School Graduation Requirement https://www.educationnext.org/massachusetts-teachers-union-aims-to-eliminate-standardized-test-as-high-school-graduation-requirement/ Thu, 24 Aug 2023 12:34:23 +0000 https://www.educationnext.org/?p=49716898 Ballot initiative would go to voters in 2024

The post Massachusetts Teachers Union Aims to Eliminate Standardized Test as High-School Graduation Requirement appeared first on Education Next.

]]>

Illustration of a hand dropping a scantron sheet into a ballot box

Massachusetts appears headed for another high-profile, and expensive, education-related ballot initiative battle—this time, over standardized testing.

The Massachusetts Teachers Association, a 115,000-member union that is a powerful political force in the state, is backing an initiative that would eliminate the use of the Massachusetts Comprehensive Assessment Test as a high-school graduation requirement. Education reform and business groups are already beginning to mobilize to preserve the testing requirement.

If voters approve it, the initiative would change state law. Currently, passing the standardized test in math, English language arts, and one science is required for graduation. Under the proposed change, school districts would instead sign off that students had “satisfactorily” completed coursework certified as “showing mastery of the skills, competencies, and knowledge contained in the state academic standards and curriculum frameworks.”

Union leaders spearheading the campaign blame the graduation test requirement for worsening racial and economic differences.

“The MCAS has not only failed to close learning gaps that have persisted along racial and economic lines, but the standardized tests have exacerbated the disparities among our student populations. We are one of the last states using this outdated method of assessing academic mastery,” the union’s president, Max Page, and its vice president, Deb McCarthy, said in an August 6 statement.

The union leaders emphasized that they aren’t asking voters to eliminate the MCAS entirely, just to stop using it as a graduation requirement. “Indeed, the MCAS will, following federal law, continue to be taken by students,” the statement said. “At present the MCAS graduation requirement is doing nothing more than proving the wealth and education levels of parents, while also harming competent students who, for a variety of reasons, struggle with standardized tests.”

Advocates of the test firmly reject the idea that it is to blame for worsening racial or economic disparities. Voices for Academic Equity, a coalition that includes the National Parents Union, the Massachusetts Charter Public School Association, the Boston Schools Fund, Massachusetts Business Alliance for Education, Educators for Excellence Massachusetts, and Education Reform Now Massachusetts, issued a report that called the test “a tool for equity.”

“The test exposes our profound societal inequities. But without it, we lose our ability to hold up those inequities and demand better opportunities for the students who are entitled to a high-quality education and have not historically received it,” the report says. “Given our nation’s history of systemic racism, MCAS serves as a mirror to see reality so we can make it better. We do not break a mirror because we don’t like its reflection.”

The Massachusetts policy director of Democrats for Education Reform, Erin Cooley, warned in a statement to Education Next that removing MCAS as a graduation requirement “would diminish the value of a Massachusetts diploma.”

“Students, parents and families across the state want high standards for their children and want schools to meet those standards,” Cooley said.

The executive director of the Massachusetts Business Alliance for Education, Ed Lambert, told WBUR that the MCAS test and the graduation requirement had helped propel Massachusetts to the top of national achievement rankings. “Undoing it could set back Gov. Maura Healey’s efforts to make the state more economically competitive,” WBUR paraphrased Lambert as cautioning.

A statement from the Massachusetts Business Alliance for Education said that “Eliminating the MCAS graduation requirement would leave us without a common standard of achievement that all students, across all communities and all walks of life, in every corner of Massachusetts are expected to meet.”

If the union succeeds in winning passage of the initiative, it would be the third time in a decade that it prevailed in a statewide vote.

In 2016, Bay State voters considered a ballot resolution that would have lifted the state’s limit on charter schools, which are public schools that operate outside of the traditional school-district bureaucracy. Political spending on the question was more than $41 million, and the state’s residents wound up voting 62 percent to 38 percent to keep the cap on charters in place. It was a stunning defeat for charter-school advocates who had sponsored the initiative.

In 2022, teachers unions poured $23 million into a misleading campaign to raise Massachusetts’ state income tax to 9 percent from 5 percent on income over $1 million a year. The proceeds of the tax increase were to be spent on education and transportation. The anti-MCAS campaign is off to a similarly factually challenged start; a Boston Globe staff editorial and a Globe column by Scot Lehigh faulted the union for overstating the number of seniors that the test requirement prevented from graduation by ignoring the fact that the vast majority of students who fail the MCAS also do not meet local graduation requirements. Yet the union’s direct mailings and other paid media reach far more eyeballs than do the Globe or the similarly skeptical Contrarian Boston substack, which are behind paywalls.

The state attorney general has until September 6 to rule on whether the initiative is okay to proceed. If it gets the legal clearance, supporters will then need to gather 75,000 signatures to place the proposition on the November 2024 election ballot.

The union also supports legislation, the Thrive Act, that would both eliminate the MCAS graduation requirement and the threat of state receivership for school districts. If the legislation passes, it could render the ballot resolution moot.

In addition to the union-backed initiative, there was a separate initiative originated by Shelley Scruggs, a parent from Lexington, Massachusetts. On August 16, Scruggs and the union announced they were joining forces behind the union-backed initiative.

Ira Stoll is editor of FutureOfCapitalism.com. He was managing editor of Education Next from 2019 to 2023.

The post Massachusetts Teachers Union Aims to Eliminate Standardized Test as High-School Graduation Requirement appeared first on Education Next.

]]>
49716898
Will Dismal New National Test Results in Civics and History Finally Spark Improvements? https://www.educationnext.org/will-dismal-new-national-test-results-civics-history-finally-spark-improvements-naep/ Wed, 03 May 2023 10:22:04 +0000 https://www.educationnext.org/?p=49716625 Weak standards, poorly prepared teachers, and meager instructional time contribute to bleak outcomes

The post Will Dismal New National Test Results in Civics and History Finally Spark Improvements? appeared first on Education Next.

]]>

Text illustration about NAEP civics test

Aaargh. Here we go again. The new National Assessment of Educational Progress civics and history results are as deplorable as they were predictable. Whether they’ll also serve as the action-forcer that we need is far from certain. Is this to be a “Sputnik moment” on the civics front or another yawner?

NAEP has been testing U.S. history since 1994, civics since 1998, and the results have always been bleak. At its peak in 2014, the “Nations Report Card” showed a meager 18 percent of eighth graders to be “proficient” in history while 22 percent reached that threshold in civics. Year after year, assessment after assessment, those two key subjects have reaped the lowest scores of anything tested by NAEP.

Declines in both set in after 2014—well before Covid hit—and it was inevitable that post-pandemic scores would be even worse. Now we’re essentially back to the starting line, i.e. around the same levels as when these subjects were first reported by NAEP. And (as we’ve recently seen in reading and math) the declines are worst among low-scoring students, which is to say those who possess the least knowledge and skills in history and civics have experienced the most severe losses.

What’s doubly troubling—but perhaps doubly attention-getting—is that this has happened just as many in the education and policy worlds are striving to launch a renaissance in civics, citizenship and the historical understandings that must undergird them.

Any number of organizations and projects are working at this. (I’ve been engaged with several, including the estimable Educating for American Democracy venture and its “roadmap for excellence in history and civics.”) They’re responding not just to test scores but also—even more so—to the troubled state of the polity, the erosion of good citizenship, the travails of civil society, and loss of faith in the fundamental institutions and processes of our constitutional democracy.

Educating schoolkids in civics and history is in no way the whole solution to these deep-seated problems but it has to be part of any solution—and evidence abounds that we’re doing a lousy job of it. The new NAEP results just underscore the blunt fact that the vast majority of U.S. 8th graders don’t know squat about U.S. history or civics.

But why? I’m seeing five big contributing factors:

First, most states have lousy standards for these core subjects, meaning that their expectations for what K-12 students should learn are low, vague or otherwise lacking. My Fordham colleagues demonstrated this in a voluminous 2021 report that found just five jurisdictions (four states plus DC) with “exemplary” standards in both subjects. It’s true that standards alone don’t teach anybody anything, but it’s also true that if you don’t have a clear destination for your journey you could wander forever and get nowhere.

Second, weak standards are part of a larger “infrastructure” problem in social studies education, admirably documented in a recent RAND study. Although focused on the elementary grades, the failings itemized in that analysis—which include incoherent curricula, lack of teacher support, meager instructional time, ill-prepared teachers, an absence of accountability—apply pretty much across the entire span of K-12 schooling.

Third, curricular materials in this field—with history and civics often submerged in a “social studies” muddle that may be as much about pop-sociology and psychology as essential information and analytic skills—are mostly mediocre, the good ones are little used, and some popular texts are pretty awful. Check out EdReports and the What Works Clearinghouse and you’ll find the curricular cupboards barren for history and civics, this despite the fact that excellent tools exist by which to evaluate such curricula. And the culture wars and political posturings that have recently engulfed curricular deliberations are nowhere livelier than in the realm of social studies, although I’ve also called attention to a latent consensus across much of the land regarding what schools should teach in this realm.

Fourth, many teachers don’t know much about these subjects themselves. Typical certification requirements for social studies teachers include a smattering of “content” courses in any of the half-dozen disciplines that fall under this heading, but persons obtaining such certificates are then deemed qualified to teach any of those subjects. Which is to say a history teacher may have studied very little history and a civics teacher (who may also be the gym teacher) could have majored in anthropology.

Fifth, little time is devoted to history and civics over thirteen years of schooling and few schools or students are held to account for how well these subjects are learned. Though we routinely term them part of the “core curriculum” along with ELA, math and science, we don’t give them nearly as much attention as the other three and we’re far less likely to insist on any evidence of learning beyond, say, a passing grade in high school history and civics. It’s no help that few colleges pay attention to whether their applicants know anything about history or civics and almost none requires its own students to study these subjects. (Stanford is requiring freshman year civics as of next year.)

Is there hope? The bleak NAEP results could serve as a firebell in the night, the alarm we need to catalyze purposeful action, overcome our divisions and quell, at least for a moment, the curricular culture wars.

It’s not beyond imagining. Legislative action can already be glimpsed in many places and innumerable groups are actively promoting civics and history reforms of one kind or another. Advice abounds as to how to strengthen these elements of K-12 schooling.

My own advice is implicit in the five causes of decline that are spelled out above, as each implies its own remedy: Solid standards, robust infrastructure, quality curricula, well-prepared teachers, time-on-task, results-driven accountability. It’s really not rocket science.

But one more thing more is also crucial: we must improve our diagnostics, starting with NAEP itself. Why do we have history and civics results for 8th grade but not for 4th or (especially) 12th? It’s the end of K-12 when we most need solid data on what students have and have not learned. And why do we have only national data, not the state-level results that might drive serious action at the level that matters most? NAEPsters will offer all manner of explanations, starting with budget, but the fact remains that—here as everywhere—the problems likeliest to go unsolved are those that are poorly diagnosed in the first place. What we got from NAEP this week is necessary but in no way sufficient for a thorough diagnosis, the kind that points toward better targeted treatments.

That all this matters to the nation’s future is self-evident. That we will go beyond garment-rending and teeth-gnashing is less so.

Chester E. Finn, Jr., is a distinguished senior fellow and president emeritus at the Thomas B. Fordham Institute. He is also a senior fellow at Stanford’s Hoover Institution and a former chair of the National Assessment Governing Board.

The post Will Dismal New National Test Results in Civics and History Finally Spark Improvements? appeared first on Education Next.

]]>
49716625
Louisiana and Montana Test Out New, Less Time-Consuming Tests https://www.educationnext.org/louisiana-and-montana-test-out-new-less-time-consuming-tests/ Mon, 01 May 2023 09:01:10 +0000 https://www.educationnext.org/?p=49716623 "Teachers use the diagnostic information to inform instructional decisions"

The post Louisiana and Montana Test Out New, Less Time-Consuming Tests appeared first on Education Next.

]]>

New Meridian is an assessment company that launched in 2016 with the goal of making tests more useful for educators and students. Today, it works with more than 2,500 districts in five states. Given the need for good measures of student progress and better instructional support, especially after devastating pandemic-era declines in learning, I thought it worth taking a closer look at their efforts. Today, I talk with Arthur VanderVeen, the founder and CEO of New Meridian. Before founding New Meridian, he served as the executive director of college readiness at the College Board and as the executive director of assessment (and then chief of innovation) for the New York City Department of Education. Here’s what Arthur had to say.

Photo of Arthur VanderVeen
Arthur VanderVeen

Hess: So, Arthur, what is New Meridian?

VanderVeen: New Meridian is a new kind of assessment-design company. We started in 2016 with a mission to develop the highest-quality assessments—focused on critical thinking, deep engagement with meaningful content, and effective expression. We design assessments for grades 3-8 and high school, covering science, math, and English/language arts/literacy (ELA). We now work with over 2,500 districts in five states, plus the Bureau of Indian Education and the Department of Defense Education Activity (DoDEA) administering to millions of students each year.

Hess: What brought you to this role?

VanderVeen: I started New Meridian in 2016 to offer technical and operational support to the then-Partnership for Assessment of Readiness for College and Careers (PARCC) states that were transitioning away from a strict consortium model to a more flexible collaboration. As a consortium, PARCC states had to agree on the same test design and use the same test-delivery vendor, making it difficult to be responsive to local needs; in the new model, New Meridian customized the test designs to individual states’ needs, drawing from a shared bank of high-quality test items to maintain economies of scale. We also made the items available to other states through a licensing model. My desire to support states with this innovative new approach grew from my days as director of assessments for the New York City Department of Education, where I was very familiar with the original conception of the PARCC assessments, and I didn’t want the states to lose the high-quality assessments as they faced political headwinds associated with the consortium. High-quality assessments have a significant impact on classroom instructional practice. If the state assessment measures the things that matter—critical thinking, deep engagement with meaningful texts, mathematical reasoning, and effective communication—teachers will focus on developing these critical skills in the classroom and more students will have access to a quality education. So I launched New Meridian to step in and help shepherd those states toward a more flexible operating model while maintaining the same commitment to high-quality assessment.

Hess: Let’s make this simple. What assessment problem are you all trying to solve?

VanderVeen: We are trying to reduce overall testing time while providing greater value to those who need it most: teachers and students. There’s no question that an effective teacher using a coherent and research-based curriculum is the greatest lever for accelerating student learning. We want to design assessments that reinforce that quality teaching, not disrupt it. That is why we are developing a new system of modular mini-assessments that can be flexibly aligned to a local curriculum to inform instruction while also providing a reliable, comparable measure of students’ mastery of grade-level standards. This approach will create a single system of assessments that gives teachers actionable instructional data, enables district administrators to monitor school performance and direct resources, and meets federal accountability requirements.

Hess: What’s distinctive about your approach?

VanderVeen: We’re taking a classroom-up approach to developing this system. You cannot squeeze instructionally valuable information out of an end-of-year summative assessment—it’s not designed for that. And current interim assessments are designed primarily to measure growth and predict performance on the end-of-year summative. That’s fine for the district administrator, but classroom teachers can’t use that data—it’s not aligned with how concepts are taught or detailed enough to inform the next steps. We’re using new test designs and psychometric models to glean more instructional value out of our short mini-assessments. Students have an opportunity to “level up” and continue to demonstrate their mastery throughout the year. Then, we pull all that data together into a comparable, reliable measure of grade-level mastery, without the redundancy or intrusion of a big end-of-year summative test. This approach will significantly reduce overall testing time and eliminate the lack of coherence between what our local assessments are telling us and what the state test is saying.

Hess: That’s intriguing, but can we get a little more concrete about these new test designs and psychometric models? Just how does this work?

VanderVeen: Our test designs focus on providing information that’s usable for instructional decisions. For every mini-assessment, we ask educators and learning experts, “What information about students’ learning progress on this set of concepts or skills would help you adjust your instruction?” We identify those “attributes” of learning development and write test questions that differentiate which ones students are mastering and which they are not. This may include relevant misconceptions that can block students’ learning progress. We then use sophisticated scoring models that combine information from multiple test questions and testlets to highlight which attributes need further instruction. For example, students typically learn proportional reasoning in middle school through multiple representations, including looking at patterns in data tables, determining the slope of graphs, writing equations, and interpreting verbal descriptions. Our testlets measure students’ learning progressions through these different dimensions of proportional reasoning, while allowing flexibility in how this foundational concept for algebra readiness is taught.

Hess: How do teachers get the classroom feedback? Can you talk a bit about the infrastructure at the local, classroom level?

VanderVeen: We are designing innovative new reports for teachers, students, and administrators that combine the instructionally focused information with ongoing, cumulative progress toward end-of-year standards mastery. Teachers use the diagnostic information to inform instructional decisions while they and their students monitor progress toward their end-of-year learning goals.

Hess: I know you all are currently piloting a few programs. Could you share a bit about those?

VanderVeen: We have partnered with two mission-driven, forward-thinking state education leaders—Superintendents Cade Brumley in Louisiana and Elsie Arntzen in Montana—who are challenging the status quo on behalf of their students. Both leaders are working to make assessments more accessible, more relevant, and more equitable by adopting a through-year model and aligning assessments more closely to the taught curriculum. This is our first pilot year, and it’s been really exciting. We convened teachers from both states together to write test items and we’ve been conducting empathy interviews, focus groups, and surveys to better understand what teachers, students, and families want in next-generation assessments. We’ve had strong philanthropic support to launch these pilots, and both states were also awarded Competitive Grants for State Assessment to fund a multiyear development program.

Hess: What kind of evidence is there regarding the efficacy of your assessments? What are you learning?

VanderVeen: We have a robust research program in place to validate both the instructional utility of our classroom reporting and the technical quality of the summative scores we will report for accountability purposes. It is critical that we do both well to achieve our goal of transforming state assessments. This year, we are piloting the test questions and blueprints and getting feedback on the design and usability of the system. We are analyzing the student test data to validate and refine our scoring models. For example, we are analyzing early student data to determine whether our scoring models can reliably differentiate the dimensions of proportional reasoning I mentioned earlier. As we get more data across larger populations of students, we will continue to refine our scoring models to support the instructional decisions teachers are making. This is a multiyear process, and we are excited to have state partners, technical advisers, researchers, and philanthropic support who are all committed to this journey. It’s critical because teachers and students need better classroom assessments that reinforce the curriculum and replace the end-of-year test, reducing overall testing time. This is our vision, and we are excited to be working with numerous partners who are also committed to this ambitious goal.

Frederick Hess is director of education policy studies at the American Enterprise Institute and an executive editor of Education Next.

This post originally appeared on Rick Hess Straight Up.

The post Louisiana and Montana Test Out New, Less Time-Consuming Tests appeared first on Education Next.

]]>
49716623
State and Local NAEP Declines Were More Universal Than Commonly Reported https://www.educationnext.org/state-and-local-naep-declines-were-more-universal-than-commonly-reported/ Fri, 16 Dec 2022 10:00:10 +0000 https://www.educationnext.org/?p=49716123 (with a caution on the hazards of misinterpreting “statistical significance”)

The post State and Local NAEP Declines Were More Universal Than Commonly Reported appeared first on Education Next.

]]>

Illustration that reads "The Nation's Report Card"

The 2022 National Assessment of Educational Progress (NAEP), also known as the Nation’s Report Card, has raised alarms across the country. Educators, policymakers, researchers, and parents are concerned about the results showing nationwide declines in reading and math scores at the 4th and 8th grade levels.

With such a troubling national outlook, it’s natural to look for bright spots—states or districts that bore up against the tidal wave of learning loss. In the context of large declines nationally, districts or states that held steady might be viewed as bright spots. But commentators get the story wrong if they interpret the absence of a statistically significant decline as an indicator of holding steady. As noted in our previous post, a nonsignificant difference does not mean there was no change—and some changes that are educationally meaningful may not achieve statistical significance.

Nonetheless, the limitations of statistical significance do not mean we can’t look for patterns and for bright spots. There is useful information in the state- and district-level NAEP results, if we use the right tools to extract it. Bayesian statistical analysis fits this bill. Unlike statistical-significance testing, Bayesian methods incorporate contextual data—in this case, the wealth of results the National Center for Education Statistics provides for each of four subject and grade-level combinations across all states and 26 urban school districts. The results improve our understanding of outcomes for any specific state or district (and each specific subject and grade level in that state or district).

Because Bayesian methods strengthen results and provide a more flexible framework for interpreting them, they provide more information than a judgment about whether a change was statistically significant. Bayesian analysis can tell us the likelihood that a change was educationally meaningful, as defined in terms of student learning. And it is far less likely to be misinterpreted than a judgment of statistical significance (or nonsignificance), because unlike statistical significance, a Bayesian result can be clearly stated in plain English, as we’ll show you in the paragraphs below.

The first step is to define what counts as educationally meaningful. The National Center for Education Statistics commissioner has suggested that changes of only 1 or 2 points on the NAEP scale are educationally meaningful. For the purposes of this analysis, we set a slightly higher bar and define educationally meaningful as a 3-point change, which corresponds to just over a quarter of a year of learning, according to the Stanford Education Data Archive.

We re-analyzed the 2019 and 2022 average NAEP scores for subjects and grade levels at the state and district levels using Bayesian methods. The results, available in full below, provide a clearer picture for individual districts and states, and stronger evidence about patterns across districts and states. The results illustrate the probability that each state and district experienced an increase or decline in its average score, and the probability that the increase or decline was educationally meaningful. (For those interested, the final tab in the figure describes more details about the methodology).

Let’s consider, for example, results for the District of Columbia Public Schools, as shown in this figure.

Figure: Difference between 2019 and 2022 NAEP scores for District of Columbia (DCPS)

In math, declines in both grades in DCPS were unequivocal: the National Center for Education Statistics judged them to be statistically significant, and our Bayesian analysis also finds that the declines were substantial—virtually certain to be educationally meaningful: the red part of the bar indicates the probability of a decline exceeding 3 points.

In reading, the change in NAEP scores in DCPS was not statistically significant, according to the original results from the National Center for Education Statistics. But this does not mean that reading scores in DCPS “held steady,” as some stories reported. Bayesian analysis indicates that, rather than holding steady, DCPS 4th and 8th graders most likely lost ground in reading, as indicated by the collective probabilities in the red bar (probability of an educationally meaningful decline) and the orange bar (probability of a smaller decline). In fact, we estimate a 74 percent chance that DCPS 4th graders experienced an educationally meaningful decline in reading of 3 or more points and a 25 percent chance of a decline smaller than 3 points. For DCPS 8th graders, we estimate a 56 percent chance that reading scores decreased by an educationally meaningful amount and a 43 percent chance of a smaller decline.

The high probabilities of educationally meaningful declines in reading are consistent with the declines on DCPS’s scores on Partnership for Assessment of Readiness for College and Careers assessments, which are used for accountability in the District of Columbia and benefit from a more robust sample size than the NAEP. (As noted in our previous post, when sample sizes are small, meaningfully large differences may not be statistically significant.) But the NAEP results provide considerably more information than a simple assessment of statistical significance implies—if they are interpreted in a Bayesian framework.

The richer information produced by a Bayesian interpretation comes into clearer focus when we summarize changes in NAEP scores across districts in the local NAEP tab.  For each grade and subject the probability of declines appears to the left of the centerline, while increases appear to the right of it.  The results show educationally meaningful changes (greater than 3 points) in red for declines and dark green for increases. Smaller changes (less than 3 points) appear in orange for declines and light green for increases.

A quick glance at these results confirms one pattern that emerged when scores were released: Declines were larger and more common in math than in reading.

But the Bayesian results also show that declines were more universal than much of the commentary has suggested. For example, National Center for Education Statistics reported that 4th-grade reading scores declined by a statistically significant margin in 9 of 26 districts and did not change significantly in the remaining 17 districts. This is technically correct but easily misunderstood. Indeed, NCES communications encouraged the misinterpretation that 4th-grade reading scores “held steady” in these 17 districts.

In fact, the Bayesian analysis shows that that 4th-grade reading scores likely declined in 25 of 26 districts. Furthermore, declines were likely educationally meaningful in 17 of those districts. This means that, in 4th-grade reading, there was a meaningful educational decline in roughly half of the districts where changes were not statistically significant. And nearly all of these districts likely had at least a modest decline. The bottom line: There aren’t many bright spots here.

The counts of statistically significant and educationally meaningful declines diverge less at the state level than at the district level, as can be seen in comparing the Bayesian results on our state NAEP tab with the National Center for Education Statistics reports of statistically significant declines. NAEP includes more students in state-level samples than district-level samples, making it more likely that an educationally meaningful decline will be statistically significant. But even so, the Bayesian results make clear that declines were much more common across the country than counts of statistically significant results would suggest.

More generally, a Bayesian interpretation provides more information about the true likelihood of changes and the size of those changes. As researchers, we owe it to journalists, policymakers, and the public to report findings in ways that reduce misunderstandings and make full, efficient use of the available information.

Changes in average NAEP scores from 2019 to 2022

Our re-analysis fit two Bayesian models – one for states, and another for districts – that borrow strength across subjects, grades, and jurisdictions. Conforming with best practices in the literature, we chose weakly informative prior distributions that assume that parameters governing variability should not be too large. We fit our models using Hamiltonian Monte Carlo as implemented in the Stan probabilistic programming language and assessed convergence and mixing using the Gelman-Rubin diagnostic and effective sample sizes.

Our models used imputed 2019 scores for Los Angeles, as Los Angeles excluded charter schools on a one-time basis in 2019 (which comprise nearly 20% of Los Angeles’ public schools).

Model specification

We write each of our Bayesian models as follows, where jurisdictions represent states or districts, respectively. Let j represent jurisdictions, s represent subject (Math or Reading), and g represent grade (fourth or eighth). Let t indicate academic year (2018/19 or 2021/22).

Then yjtsg gives the NAEP score for jurisdiction j in year t for subject s in grade g.

  • yjtsg = αjsg + δjsg I{t=2022} + ϵjtsg
  • αjsg = αj^0 + α_j^S Ss + αj^G Gg + αj^X Ss Gg
  • δjsg = δj^0 + δj^S Ss + δj^G Gg + δj^X Ss Gg
  • ϵjtsg ∼ N(0,σjtsg^2 )

where standard errors σjtsg are specified using values from the NAEP data. In this parametrization, we let

  • SReading = -0.5
  • SMath = 0.5
  • G4 = -0.5
  • G8 = 0.5

so that neither grade nor subject is considered a baseline value. (Note that under this parametrization, the α_j^0 and δ_j^0 terms do not refer to a specific grade or subject, so are not directly interpretable.)

The eight random effects (four αjsg’s and four δjsg’s, for each subject-grade combination) are assigned prior distribution MVN(θ0,Σ) with an LKJ prior on Σ. We transform the NAEP scores to z-scores prior to fitting the model and assign other parameters standard normal priors, reflecting a gentle assumption that these parameters are unlikely to be too large.

Model fitting and validation

We fit our model using Hamiltonian Monte Carlo as implemented in the Stan probablistic programming language (Stan Development Team, 2021), via its R interface, rstan. Specifically, we used the brms R package to implement our model using rstan.

We specified our brms model statement as follows, where y represented NAEP scores, y_se represented the corresponding standard errors, Y2022 is an indicator for the 2021/22 academic year, and grade_ctr and subj_ctr represent the Ss and Gg variables defined above.

y | se(y_se) ~ Y2022 * grade_ctr * subj_ctr + (1 + Y2022 * grade_ctr * subj_ctr | jurisdiction)

We assessed convergence and mixing using the Gelman-Rubin diagnostic and effective sample sizes.

  • For both our local and state models, Gelman-Rubin statistics were well within recommended ranges for all parameters (from 0.99 to 1.01 for both models).
  • Effective sample sizes for all parameters were sufficient, with minimums of 838 for the local model and 506 for the state model.

Imputed scores for Los Angeles

Prior to fitting our models, we imputed two values for each subject-grade combination for Los Angeles in 2019 – the NAEP score, and its standard error.

  • We imputed Los Angeles’ scores by calculating the percentile across districts that Los Angeles achieved in 2017 and assigning the corresponding 2019 percentile, separately by grade and subject.
  • We used the same approach for standard errors, calculating the percentile of standard errors across districts for Los Angeles in 2017, ensuring that both the score itself and the level of precision reflect realistic scenarios based on Los Angeles’ 2017 performance.

Lauren Forrow is a senior statistician at Mathematica, where Jennifer Starling is a statistician and Brian Gill is a senior fellow.

This post originally appeared at Mathematica.org.

The post State and Local NAEP Declines Were More Universal Than Commonly Reported appeared first on Education Next.

]]>
49716123
Education Advice for Incoming Governors https://www.educationnext.org/education-advice-for-incoming-governors-ask-questions/ Wed, 07 Dec 2022 10:00:12 +0000 https://www.educationnext.org/?p=49716052 Ask questions. Everyone is incentivized to tell you things are going swimmingly.

The post Education Advice for Incoming Governors appeared first on Education Next.

]]>

Illustration

Dear Governor,

Congratulations on your big win. Whether you breezed to re-election, stormed in on a mandate for change, or squeaked it out over a flawed opponent, the seat is yours. The question, now, is what are you going to do with it?

No doubt, you are aware of the significant decline in student outcomes over the past three years. In fact, whoever you are, we can say with certainty that students in your state moved backwards in reading and math; and that the lowest performing students regressed more than the top performing students. Before, it was hard to run a state education system; today, it is harder and the stakes are higher.

What should you do? First — make the clear moral commitment to catch kids up. Announce it, put numbers behind it, and commit to having it at the top of your agenda during your time in office. Pick the most important metrics for your state’s future, and fix them. For example, if 20% of students in your state are scoring at the bottom level of your state’s 3rd grade reading assessment — a measure of functional literacy — commit to cut the illiteracy rate in half. If 8th grade Algebra scores have fallen significantly since 2019, commit to ensuring that the scores return to the previous levels and grow from there.

Second, measure and share the results, rinse and repeat. Commit to the parents of your state that they will know how their children are performing — no spin, real facts in real time. Talk about what the numbers show wherever you go. Highlight the places that are making the most progress. Call out the places that are not catching kids up. You have the bully pulpit — use it to talk about student achievement, not to foment the culture wars.

Third, invest in strategies that are proven to work. Ensure that every school in your state has the resources and training to teach literacy via the science of reading. Use the levers of state funding to incentivize districts and schools to adopt core curriculum that meets a high bar of evidence using Ed Reports or other tools.

Fourth, intervene quickly and aggressively for students who are behind. Tutoring—the focus of my nonprofit—has a strong body of evidence, but is difficult to implement effectively at scale. Narrow the focus, pick a couple of key academic areas and flood the zone with tutoring support using proven strategies. Extending the school day or the school year can also work, with strong academic focus and planning. Those approaches are expensive, but you have resources to spend—spend them on more time and better tools for students in need.

Finally, manage this like your students’ futures depend on it, because they do. Demand to see the numbers— attendance, interim benchmark assessments, end of year tests, spending on effective interventions. Ask questions. I hate to say it, but everyone is incentivized to tell you things are going swimmingly. Trust but verify—the stakes are high.

There is a temptation when faced with a daunting challenge in education to either look for a silver bullet, or to simply put more money into the system as it exists today. Unfortunately, neither of these will solve the challenges in front of you.

However, with clear and measurable goals, a commitment to tracking and management, smart strategies and resource allocation, and a long-term determination to keep going until the job is done, you have a profound opportunity to make a difference in the lives of kids and families. And after all—isn’t that why you ran in the first place?

Kevin Huffman, a former education commissioner of Tennessee, is CEO of Accelerate, a new national nonprofit organization that works to advance educational and economic opportunity.

The post Education Advice for Incoming Governors appeared first on Education Next.

]]>
49716052