Mark Berends, Author at Education Next

How Building Knowledge Boosts Literacy and Learning

Mark Berends — Wed, 13 Mar 2024 09:00:45 +0000

Educators and researchers have been fighting the reading wars for the last century, with battles see-sawing literacy instruction in American schools from phonics to whole language and, most recently, back to phonics again. Policymakers have entered the fray, after more than a quarter-century of stagnant reading scores in the United States. Over the last decade, 32 states and the District of Columbia have adopted new “science of reading” laws that require schools to use curricula and instructional techniques that are deemed “evidence-based.”

Such reading programs include direct instruction in phonics and reading comprehension skills, such as finding the main idea of a paragraph, and efforts to accelerate learning tend to double down on more of the same skill-building practice. But research increasingly points to another critical aspect of literacy: the role of student knowledge. For example, prior research by two of us found that a young child’s knowledge of the social and physical world is a strong predictor of their academic success in elementary school. And advocates for knowledge-based education often cite the so-called “baseball study” where students reading a passage about baseball who knew about the sport were far better at understanding and summarizing the story than students who didn’t, regardless of their general reading skills.

Knowledge-building reading curricula are rooted in these insights, and use materials and activities based on a sequence of integrated science and social studies topics, texts, and vocabulary. Yet the potential value of this approach is often an afterthought in state and district efforts to strengthen reading instruction, and the benefits to students of combining evidence-based curriculum with systematic efforts to build student knowledge have yet to be rigorously documented.

We conduct the first-ever experimental study of this topic, based on randomized kindergarten-enrollment lotteries in nine Colorado charter schools that use an interdisciplinary knowledge-based curriculum called Core Knowledge. To assess the long-term impact of experiencing a knowledge-building curriulum on student learning, we compares performance on statewide tests in grades 3–6 between kindergarten lottery winners who attended a Core Knowledge charter school with lottery losers who could not enroll.

We find that winning an enrollment lottery and enrolling in a Core Knowledge charter school boosted long-term reading achievement in 3rd to 6th grade by 16 percentile points, as compared to comparable applicants who did not win their enrollment lottery. The size of this gain is approximately equivalent to the difference between the mediocre performance of U.S. 13-year-olds on the 2016 Progress in International Reading Literacy Study and that of top-scoring countries like Singapore and Finland. Our results are also notable in their contrast with other studies of reading interventions, which typically find small, short-term effects.

Students and teachers in many public elementary schools spend up to two hours each day on reading instruction. While the component skills of literacy are critical to student development and learning, our findings point to a missed opportunity to accelerate literacy by building knowledge at the same time. Skill building and knowledge accumulation are separate but complementary cognitive processes, and while the adage “skill begets skill” may be true, a fuller description of cognitive development could be “skill begets skill, knowledge begets knowledge, and skill combined with knowledge begets them both.”

Kindergarten Lotteries for “Core Knowledge” Charters

The Core Knowledge curriculum was created in the 1980s by E.D. Hirsch, Jr., a researcher and advocate of knowledge-building education. Its content and activities follow a planned sequence of the knowledge and skills students should accumulate and master in grades K–8 in all academic subjects and the arts. This “knowledge-based schooling” approach is rooted in the belief that a common base of shared knowledge is foundational for not just individual students’ reading comprehension abilities but also for our ability as a society to communicate and promote equal opportunity. An estimated 1,700 schools across the U.S. use the curriculum today, including more than 50 in Colorado.

To assess the impact of the Core Knowledge curriculum on student achievement, we look at nine oversubscribed Colorado charter schools that all use the curriculum, had been open for at least four years, and held random enrollment lotteries to register kindergarten students in either or both of the 2009–10 and 2010–11 school years. Our study includes 14 separate lotteries with 2,310 students, almost all of whom are from high- or middle-income families.

These families generally have a range of schooling options, including private schools, other charter schools, and public schools outside their district under Colorado’s open-enrollment law. About one in five students in our sample applied to multiple charter lotteries—usually two instead of one. Some 41 percent won at least one lottery, and 47 percent of winners enrolled in that school. In all, 475 lottery winners went on to attend a Core Knowledge charter, while 1,356 students did not win the lottery and attended school elsewhere. In analyzing the effects of attending a Core Knowledge charter, we take into account the fact that not all lottery winners actually enrolled.

Attrition and Family Choice

We base our analysis on the performance of lottery applicants on the Partnership for Assessment of Readiness for College and Careers (PARRC) reading and math tests in grades 3, 4, 5, and 6, as well as the 5th-grade science PARRC test. By looking at these scores, we can compare the performance of students who did and did not experience a knowledge-building curriculum over up to seven years of their schooling.

However, roughly 36 percent of students in our sample did not complete all scheduled PARCC tests through grade 6, and the attrition rate for students who did not win the enrollment lottery is 5 percentage points higher than for lottery winners. Detailed student data reveals three major factors at play. First, some students stop participating in Colorado’s PARCC testing because they move out of state, transfer to a different school, or are homeschooled. A second group of students don’t have test-score data because they are exempted as language learners or special-education students. Third, other students are off-track with their expected kindergarten cohort in later years because of delayed kindergarten entry (“redshirting”) or due to having skipped or repeated a grade.

To ensure that this attrition does not skew our results, we exclude from our analysis both the four lotteries with the highest rates of differential attrition between lottery winners and losers and the youngest applicants, who are more likely to be redshirted by their parents regardless of their lottery outcome. We also adjust our results for students’ gender, race or ethnicity, and eligibility for a free or reduced-price school lunch to ensure that any demographic differences between lottery winners and losers do not introduce bias.

Accelerated Achievement

We find positive long-term effects on reading performance for students who are randomly selected by a kindergarten enrollment lottery and attend a Core Knowledge charter school. Across grades 3–6, these students score 47 percent of a standard deviation higher in reading than comparable lottery applicants who did not have a chance to enroll. This is equivalent to a gain of 16 percentile points for a typical student (see Figure 1). Students who attend a Core Knowledge charter also make outsized gains in science of 30 percent of a standard deviation, which is equivalent to a gain of 10 percentile points. Effects in math are positive, at about 16 percent of a standard deviation, but fall short of statistical significance.

The effects are slightly larger for female students than males (see Figure 2). In reading, female Core Knowledge charter students score 50 percent of a standard deviation higher compared to 44 percent for males, for a gain 17 of percentile points compared to 15 percentile points for males. Females gain about 12 percentile points in science and 9 percentile points in math, while males gain 6 percentile points in science and experience no gains in math. We also look at effects by student grade level and find no upward or downward trend, suggesting the effects may have stabilized by 4th grade (see Figure 3).

While prior non-experimental research has documented stronger reading performance among students who already have knowledge about a topic, our analysis shows positive long-term impacts in reading from systematically building student knowledge over time. In our view, these results suggest that the “procedural skills” approach that has dominated reading comprehension instruction over the last 30 years in public schools is less effective than a “knowledge-based” approach that teaches skills and also is designed to build a body of knowledge as the main mechanism for increasing comprehension.

These findings also build on the body of evidence linking students’ levels of general knowledge to achievement in reading, science, and math. Research also shows that levels of general knowledge are strongly correlated with socio-economic status and parental levels of education. However, unlike these factors, knowledge is malleable through curricular choices. The intervention we study, where students experience seven years of a knowledge-building curriculum, appears to set off a long-term, compounding process whereby improved reading comprehension leads to increased knowledge, and increased knowledge leads to even better comprehension.

A Call to Build Knowledge About “Knowledge”

In addition to informing current-day decision-making, we believe these results should inspire a new research and policy agenda to measure and track students’ knowledge development and understand the mechanisms involved in knowledge-building curricula. The effects our study finds are similar in pattern and magnitude to earlier non-experimental evidence, which suggests that gains in students’ general knowledge could have a larger effect on future achievement than similar gains in more widely studied non-cognitive domains, such as executive function, visual-spatial and fine motor skills, and social and emotional development.

The potential benefits of knowledge-building curricula could be far-reaching. The compounding process our analysis reveals would occur not only in reading, but also across all subjects to the extent that they depend primarily on reading comprehension for learning. Moreover, these achievement gains across all subjects would likely extend into future years, as increased comprehension in one year leads to increased knowledge and comprehension in the next, and so on. We believe that these curricula could also increase students’ educational attainment and future labor market success.

However, elevating student knowledge to a more central place and higher priority in research and policy will require a significant conceptual shift—the term “building knowledge” does not readily trigger a conceptual map linking the intervention to higher achievement, unlike common interventions like reducing class size, extending the school day, and raising teacher pay.

Well-designed measures of student knowledge should be considered as an important addition to other national measures for students in elementary grades. To be sure, they will carry an additional challenge. Any definition and measures of “general knowledge” will need both scientific validity and political viability at a moment when attempts to ban library books and shape course content are on the rise. Attempting to define what all public-school students should know will undoubtably trigger debates and a variety of viewpoints. However, the evidence points to building knowledge as a critical foundation of student literacy with potentially lifelong effects. The benefits of skillful reading and broad knowledge should be a shared starting point, from which a stronger approach to reading instruction can grow.

David Grissmer is research professor in the School of Education & Human Development at the University of Virginia, where Richard Buddin is education consultant, Jamie DeCoster and Tanya Evans are research assistant professors, and Chris S. Hulleman is research professor. Thomas G. White is a former senior researcher at the School of Education & Human Development. Daniel T. Willingham is professor of psychology at the University of Virginia. Chelsea A.K. Duran is a postdoctoral fellow at the Curry School of Education at the University of Virginia. Mark Berends is professor at the University of Notre Dame. William M. Murrah is associate professor at Auburn University.

This article appeared in the Spring 2024 issue of Education Next. Suggested citation format:

Grissmer, D., Buddin, R., Berends, M., White, T.G., Willingham, D.T., DeCoster, J., Duran, C.A.K., Hulleman, C.S., Murrah, W.M., and Evans, T. (2024). How Building Knowledge Boosts Literacy and Learning: First causal study finds outsized impacts at “Core Knowledge” schools. Education Next, 24(2), 52-57.

The post How Building Knowledge Boosts Literacy and Learning appeared first on Education Next.

Lessons Learned from Indiana

Mark Berends — Tue, 13 Feb 2018 00:00:00 +0000

The Indiana Choice Scholarship Program, launched in 2011, offers a rich opportunity to study how a large-scale tuition-voucher program works and to analyze the results it has produced in its first few years. As we consider the merits of private-school choice and what it would take to make it succeed, this initiative deserves particular attention: it is the nation’s largest voucher program, accounting for nearly 20 percent of all voucher students nationwide, with 34,299 students receiving vouchers and 313 private schools participating during the 2016–17 academic year. And in terms of student eligibility, Indiana’s program is the broadest, with its vouchers aimed at both low- and moderate-income families and no cap on the number of students who can take part. The average scholarship amount, based on the public-school district in which students live, ranges from nearly $4,500 in kindergarten to $5,600 in high school.

Our four-year evaluation of the Indiana program is one of a few recent studies that finds statistically significant negative effects on student achievement of using a voucher to switch from a public to a private school in the first years after a choice program’s launch. But that is only part of the story. Our research also shows that voucher students begin to recoup their academic losses in their third and fourth years of attending a private school. Students transitioning to a private school may need time to acclimate to what are usually more-rigorous academic standards and higher expectations for homework and schoolwork. Our findings also speak only to the achievement gains of students using vouchers to switch to a private school in grades 5–8. Starting students in private schools in earlier grade levels, and thus giving them more time to adjust, might produce better outcomes.

Given that many state and federal policymakers support the expansion of private-school choice, Indiana’s experience can offer lessons for the design of future voucher programs.

Who Participates?

About 76 percent of Indiana’s private schools—and almost 100 percent of its Catholic schools—participate in the voucher program. Not long after the program began, we conducted a survey of all Indiana private schools to learn why they did or did not decide to participate. Those that chose to join most frequently specified their mission, service orientation, and school finances as reasons (see Figure 1). Schools that opted not to participate cited reasons such as the desire to maintain autonomy from government, their lack of required accreditation (and unwillingness to become accredited), and concern that enrolling voucher students would require the schools to lower their academic standards.

When examining student participation, we looked at those who joined the program early and those who are participating now, because the program has changed over time. Initially, students could receive a voucher only if they had attended a public school for at least one year, or if they had attended a private school with the help of the state’s less-generous tax credit for some parents paying private-school tuition. In 2013, the program dropped the public-school attendance requirement. Thus, in its first year, 90 percent of Indiana Scholarship students had previously attended a public school, but by 2016–17, over half—55 percent—had never attended one. In other words, the program started out serving students who wanted to leave public schools, but it now serves a majority of students who have attended private schools from day one.

As the program changed, so too did the demographics of participating students (see Figure 2). In the first year, 24 percent were African American, but this number declined to 12 percent in 2016–17. Conversely, the percentage of white students receiving vouchers increased from 46 percent in the first year to 60 percent in 2016–17. The shares of Hispanic students (20 percent) and multiracial students (6–7 percent) remained consistent over time. Statewide in 2016–17, the K–12 student population was 69 percent white, 12 percent African American, 11 percent Hispanic, and 5 percent multiracial.

In the program’s first year, 69 percent of its students lived in metropolitan areas, 16 percent in suburbs, 7 percent in rural areas, and 7 percent in small towns. By 2016–17, the proportion of students from metropolitan areas decreased to 61 percent, while the share of those from suburban areas increased to 23 percent. Statewide in 2016, about 31 percent of students in grades 3–8 lived in metropolitan areas, 27 percent in suburbs, 14 percent in small towns, and 28 percent in rural areas. (Corresponding figures for the full K–12 enrollment were not readily available.)

How the Program Works

When the voucher program first began, we conducted about 100 interviews with principals, teachers, parents, and students in 13 participating private schools. We sought to learn how schools were providing educational opportunities to students using vouchers; how schools adapted their curriculum and instruction; and how students were integrating into their new environments academically, behaviorally, and religiously.

Statewide, students receiving vouchers were low-achieving before entering private schools (on average, performing at the 42nd percentile compared to public- and private-school students statewide). Principals, teachers, and students we interviewed said that students who transferred into private schools using a voucher had not been required to do much homework in the public schools. These students moved to private schools whose students were performing, on average, at the 53rd to 57th percentile in mathematics and English language arts (ELA), respectively. Schools responded to their new students by providing more individualized instruction and in some cases, by adding an “ability” group. One teacher, for example, observed a much larger gap between her high-achieving and low-achieving students than in previous years. In response, she provided more differentiated instruction and worked with smaller groups when possible.

Voucher students themselves also had to adapt to the higher academic and behavioral expectations in their new private schools. A principal’s comment typifies what we heard in our interviews: while the school did not change its expectations, he said, teachers worked to help the new students adjust to the expectations and understand that such standards applied not just to them but to all the students.

Effects on Achievement

Because Indiana does not cap the number of vouchers awarded, it has no lottery process to determine who’s in and who’s out. Without the benefit of random assignment, we used a variety of statistical approaches to determine the program’s impact on student achievement immediately after its launch. We focused on students using a voucher to switch from a public to a private school in grades 5–8 during the program’s first four years (2011–12 through 2014–15). Because Indiana public and private schools use the same assessment in grades 3–8, we could identify public-school students who shared similar achievement trajectories and demographic characteristics with these voucher students at baseline (the year prior to a student switching from a public to a private school) and track both groups’ academic progress for up to four subsequent years.

Overall, we found an average loss in mathematics of 0.12 standard deviations (roughly 3–4 percentile points) from baseline for students who used a voucher to transfer from public to private schools (see Figure 3a). The largest losses occurred during years one and two. However, voucher students began to show signs of improvement by their fourth year in a private school, and in that year there was no statistically significant difference between them and their public-school peers in terms of total achievement gains from baseline. The negative math effects in the early years are similar to recent findings for students participating in new statewide voucher programs in Louisiana and Ohio, though smaller in magnitude. In ELA, we find no statistically significant average difference in the performance of voucher and public-school students across all four years (see Figure 3b).

Our estimates of the effects of voucher use after three and four years are based on a relatively small number of students: fewer than 200 in year four, as compared to roughly 3,000 in year one and 1,700 in year two. That’s mainly because state test scores are not available for voucher students who had reached high school by that time. Nearly 15 percent of voucher students also return to a public school within one or two years, so our longer-term estimates represent the most persistent students. Furthermore, our main results are averages across all participating private schools, and estimates of the effects of using a voucher to attend specific private schools vary widely (that is, voucher students excel in some private schools and perform poorly in others). Even so, our analysis provides the most complete picture to date of the early effects on student achievement of a voucher program operating at scale.

Implications for Program Design

Although further research is necessary to understand the variation in the Indiana program’s effects—including the organizational and instructional environments of voucher schools—our research to date offers some clear policy implications:

Allow enough time for preparation. Implementing voucher programs incrementally might be more prudent than scaling up quickly. The state legislature passed the Indiana Choice Scholarship Program in the spring of 2011, and students began using vouchers that fall. Our interviews reveal that, despite their strong support for the program, private schools felt rushed in their communication with families and enrollment of students. Giving schools, families, and students more time to prepare for change might ease adjustment all the way around.

Start smaller. Before expanding voucher programs statewide, it might be better to start with smaller programs using a lottery process. This way, the randomization can allow policymakers, educators, and researchers to assess more definitively the programs’ effects.

Measure broadly, and begin in early grades. States should rely on a wider set of measures to evaluate voucher programs and begin this appraisal at earlier grade levels. Certainly, test scores are important proxies for what students are learning, but currently there is no standardized assessment taken by both public- and private-school students in grades K–2 in Indiana. What’s more, researchers know that other outcomes (for instance, social-emotional skills, engagement, motivation, and progression through K–12 schooling) are critical when considering the impact of education reforms on students of all ages. Going forward, our study will examine the impacts of vouchers on high-school students’ achievement and attainment.

Start students early. As we mentioned at the outset, it may be that implementing voucher programs in early grade levels would better acclimate students to private schools. Voucher students in upper grade levels appear to need time to adjust to the more demanding homework and high expectations in their new schools. In this regard, it is worth noting that a majority of students currently using vouchers in Indiana have enrolled in private schools from the start.

Make it easy for good schools to participate. States implementing voucher programs should ensure the quality and number of private schools willing to take part. About three quarters of those in Indiana are participating. This contrasts with Louisiana, where the supply of private schools has posed a challenge, likely due to the regulatory burden they would face if they signed on.

Consider teacher training. Private-school educators may need additional training to prepare for serving voucher students. Our findings suggest that teachers would benefit from professional development in mathematics curriculum and instruction and in learning to lead more-diverse classrooms.

Although we have much to discover about the impacts of statewide voucher programs, we are beginning to understand what policymakers and educators should consider when implementing new ones. Until research can show the conditions under which voucher programs succeed, the policy debates will rage on. In the meantime, Indiana’s experience to date provides useful guidance for states intent on moving ahead now.

This is part of a forum on private school choice. For alternate takes, see “Programs Benefit Disadvantaged Students,” by Patrick J. Wolf, or “Still Waiting for Convincing Evidence,” Douglas N. Harris.

This article appeared in the Spring 2018 issue of Education Next. Suggested citation format:

Wolf, P.J., Harris, D.N., Berends, M., Waddington, R.J., and Austin, M. (2018). Taking Stock of Private-School Choice. Education Next, 18(2), 46-59.

The post Lessons Learned from Indiana appeared first on Education Next.

Taking Stock of Private-School Choice

Mark Berends — Tue, 13 Feb 2018 00:00:00 +0000

In the past few years, new statewide voucher programs in Indiana, Louisiana, and Ohio and the steady growth of a tax-credit funded scholarship program in Florida have offered a glimpse of what expansive private-school choice might look like. What have we learned about the students and schools who choose to participate in statewide private-school choice programs and the academic results for participants? How do these programs work in practice? And what does research tell us about how states should design and oversee voucher programs—if indeed they should do so at all?

In this forum, we hear from Patrick J. Wolf, education policy professor at the University of Arkansas, Douglas N. Harris, professor of economics at Tulane, and the trio of Mark Berends, professor of sociology at the University of Notre Dame, R. Joseph Waddington, assistant professor at the College of Education, University of Kentucky, and Megan Austin, researcher at the American Institutes for Research, Chicago.

Programs Benefit Disadvantaged Students
by Patrick J. Wolf

Still Waiting for Convincing Evidence
by Douglas N. Harris

Lessons Learned from Indiana
By Mark Berends, R. Joseph Waddington, and Megan Austin

This article appeared in the Spring 2018 issue of Education Next. Suggested citation format:

Wolf, P.J., Harris, D.N., Berends, M., Waddington, R.J., and Austin, M. (2018). Taking Stock of Private-School Choice. Education Next, 18(2), 46-59.

The post Taking Stock of Private-School Choice appeared first on Education Next.