Ludger Woessmann, Author at Education Next https://www.educationnext.org/author/lwoessmann/ A Journal of Opinion and Research About Education Policy Thu, 17 Aug 2023 19:46:40 +0000 en-US hourly 1 https://wordpress.org/?v=6.5.5 https://i0.wp.com/www.educationnext.org/wp-content/uploads/2019/12/e-logo.png?fit=32%2C32&ssl=1 Ludger Woessmann, Author at Education Next https://www.educationnext.org/author/lwoessmann/ 32 32 181792879 Costs of Past and Future Learning Losses https://www.educationnext.org/costs-of-past-future-learning-losses-covid-19/ Wed, 09 Sep 2020 23:00:48 +0000 https://www.educationnext.org/?p=49712548 For the United States, the already accrued learning losses are expected to amount to $14.2 trillion, and would grow if schools are unable to restart quickly.

The post Costs of Past and Future Learning Losses appeared first on Education Next.

]]>

The worldwide school closures in early 2020 led to losses in learning that will not easily be made up for even if schools quickly return to their prior performance levels. While the precise learning losses are not yet known, existing research suggests that the students in grades 1-12 affected by the closures might expect some 3 percent lower income over their entire lifetimes.

These learning losses will have lasting economic impacts both on the affected students and on each nation unless they are effectively remediated. For nations, the lower long-term growth related to such losses might yield an average of 1.5 percent lower annual GDP for the remainder of the century. For the United States, the already accrued learning losses are expected to amount to $14.2 trillion in current dollars (present value). These economic losses would grow if schools are unable to restart quickly.

The losses are not evenly spread. The economic losses will be more deeply felt by disadvantaged students. All indications are that students whose families are less able to support out-of-school learning will face larger learning losses than their more advantaged peers, which in turn will translate into deeper losses of lifetime earnings.

Just returning schools to where they were in 2019 will not avoid the huge economic losses. Only making them better can. While a variety of approaches might be attempted, existing research indicates that close attention to the modified re-opening of schools offers strategies that could ameliorate the losses. Specifically, with the expected increase in video-based instruction, matching the skills of the teaching force to the new range of tasks and activities could quickly move schools to heightened performance. Additionally, because the prior disruptions are likely to increase the variations in learning levels within individual classrooms, pivoting to more individualised instruction could leave all students better off as schools resume.

As schools move to re-establish their programs as the pandemic continues, it is natural to focus considerable attention on the mechanics and logistics of safe re-opening. But the long-term economic impacts also require serious attention, because the losses already suffered demand more than the best of currently considered re-opening approaches. And not addressing the learning issues head-on at the outset will make it even more difficult to address them later.

The details of this analysis can be found in our recently released OECD paper.

Eric A. Hanushek is the Paul and Jean Hanna Senior Fellow at the Hoover Institution of Stanford University. Ludger Woessmann is Professor of Economics at the University of Munich and Director of the ifo Center for the Economics of Education at the ifo Institute.

Read more from Education Next on coronavirus and Covid-19.

The post Costs of Past and Future Learning Losses appeared first on Education Next.

]]>
49712548
The Achievement Gap Fails to Close https://www.educationnext.org/achievement-gap-fails-close-half-century-testing-shows-persistent-divide/ Tue, 19 Mar 2019 00:00:00 +0000 http://www.educationnext.org/achievement-gap-fails-close-half-century-testing-shows-persistent-divide/ Half century of testing shows persistent divide between haves and have-nots

The post The Achievement Gap Fails to Close appeared first on Education Next.

]]>

Income inequality has soared in the United States over the past half century. Has educational inequality increased alongside, in lockstep?

Of course, say public intellectuals from across the political spectrum. As Richard Rothstein of the liberal Economic Policy Institute puts it: “Incomes have become more unequally distributed in the United States in the last generation, and this inequality contributes to the academic achievement gap.” Harvard political scientist Robert Putnam, citing research by Stanford sociologist Sean Reardon, says, “Rich Americans and poor Americans are living, learning, and raising children in increasingly separate and unequal worlds.” Another well-known political scientist, Charles Murray, argues that “the United States is stuck with a large and growing lower class that is able to care for itself only sporadically and inconsistently. . . . The new upper class has continued to prosper as the dollar value of the talents they bring to the economy has continued to grow.”

These analysts have good reason to express concern. National competitiveness is at stake, as education advocates have argued since the Soviet Sputnik launch inspired the National Defense Education Act of 1958. Economic productivity and growth are greater in countries where students perform better in math, reading, and science than in those that do not provide their youth the same opportunities to learn (see “Education and Economic Growth,” research, Spring 2008). And while some might see income inequality as the result of adult life choices about matters such as how hard to work or where to live, educational inequality seems unfair, because the economic status of a child is outside the child’s own control. It is an inequality of opportunity that runs counter to the American dream.

Despite the topic’s importance, surprisingly little scholarship has focused on long-term changes in the size of the achievement gap between students from higher and lower socioeconomic backgrounds. Our new research, presented here, attempts to fill this void, using data from four national assessments of student performance administered to representative samples of U.S. students over nearly five decades.

Contrary to recent perceptions, we find the opportunity gap—that is, the relationship between socioeconomic status and achievement—has not grown over the past 50 years. But neither has it closed. Instead, the gap between the haves and have-nots has persisted.

The stubborn endurance of achievement inequalities suggests the need to reconsider policies and practices aimed at shrinking the gap. Although policymakers have repeatedly tried to break the link between students’ learning and their socioeconomic background, these interventions thus far have been unable to dent the relationship between socioeconomic status and achievement. Perhaps it is time to consider alternatives.

Before drawing this conclusion, though, it is important to document the long-term trends in the connection between socioeconomic background and school achievement. Press coverage of the subject typically mentions only the most recent shifts in achievement levels and gaps. Our study broadens the perspective by making full use of nearly 50 years’ worth of historical data available from four intertemporally linked assessments of achievement in math, reading, and science administered to nationally representative samples of adolescent students born between 1954 and 2001. (By “intertemporally linked,” we mean that the test makers in each of these assessments design the tests to be comparable over time by doing things such as repeating some of the same questions across different waves.) These testing programs also collect information on students’ socioeconomic backgrounds, which we use to construct an index of socioeconomic status. We report changes in the gaps in performance between students from more- and less-advantaged backgrounds over the past half century.

We find that the socioeconomic achievement gap among the 1950s birth cohorts is very large—about 1.0 standard deviations between those in the top and bottom deciles of the socioeconomic distribution (the “90–10 gap”) and around 0.8 standard deviations between those in the top and bottom quartiles (the “75–25 gap”). These are very extensive disparities, as 1 standard deviation is approximately the difference in the average performance of students in 4th and 8th grades, or four years’ worth of learning. But though these inequalities are large, they have neither increased nor decreased significantly over the past 50 years.

It could be, however, that the picture is not as dismal as suggested. If overall changes in society, coupled with policy initiatives, have proportionately lifted all boats at the same rate, everybody might be better-off, even if gaps have not significantly changed. Using the same data as for the gap analysis, we find gains in average student performance of about 0.5 standard deviations for students at age 14, or roughly 0.1 standard deviations per decade. But, surprisingly, over the last quarter century, those gains disappear for students by age 17. In other words, there is no rising tide for students as they leave school for college and careers.

Prior Research

The effects of family background on student achievement are well-documented, but few studies track changes in the relationship between demographic characteristics and student performance over time. This scarcity of longitudinal analysis partly reflects measurement challenges.

Family background and achievement. There is little dispute that students’ performance in school is strongly affected by their family background. James Coleman and colleagues, in their seminal 1966 study, Equality of Educational Opportunity, found that parental education, income, and race are strongly associated with student achievement, while school resources such as per-pupil expenditures and class size are much less significant. Subsequent research has confirmed these early findings (see “How Family Background Influences Student Achievement,” features, Spring 2016).

A variety of mechanisms link socioeconomic status to achievement. For instance, children growing up in poorer households and communities are at greater risk of traumatic stress and other medical problems that can affect brain development. College-educated mothers speak more frequently to their infants, use a larger vocabulary with their toddlers, and are more likely to use parenting practices that respect the autonomy of a growing child. Higher-income families have access to more-enriching schooling environments, and they generally do not face the high rates of violent crime experienced by those in extremely impoverished communities. All these and other childhood or adolescent experiences contribute to profound socioeconomic disparities in academic achievement.

Trends in the socioeconomic achievement gap. Des- pite firm documentation of a strong connection between socioeconomic status and student achievement, only two studies provide information on trends in the opportunity gap over time. In an appendix table of a 1998 paper, Larry Hedges and Amy Nowell report the relationship between student performance and several background characteristics across six nationally representative surveys administered between 1965 and 1992. Among these variables, parental education has the strongest correlation with student achievement, and that connection endures over time. The correlation between achievement and family income in the six surveys is weaker and declines over time.

In a second investigation, published in 2011, Sean Reardon draws on data from 12 surveys that contain information on both student achievement and reports of parental income to estimate gaps in math and reading performance of students at the 90th and the 10th percentiles of the household income distribution. In contrast to Hedges and Nowell, he finds that the “income achievement gaps among children born in 2001 are roughly 75 percent larger than the estimated gaps among children born in the early 1940s.” For those born after 1974, children in families at the median income were falling farther behind those at the 90th percentile, leading Reardon to conclude that “The 90/50 gap appears to have grown faster than the 50/10 gap during the 1970s and 1980s.”

Reardon’s study and its conclusions have been widely cited by both academics and in the general media, and the idea that income-related achievement gaps have dramatically increased has become contemporary conventional wisdom. In a 2012 article, the New York Times asserts that “while the achievement gap between white and black students has narrowed significantly over the past few decades, the gap between rich and poor students has grown substantially during the same period.” Another Times piece quotes Reardon as saying, “The children of the rich increasingly do better in school, relative to the children of the poor. . . . This has always been true, but is much more true now than 40 years ago.”

Differences between the findings reported in the two studies may be owing to the focus of Hedges and Nowell on overall correlations between socioeconomic status and achievement, while Reardon discusses disparities between the extremes of the income distribution. They could also reflect the fact that Reardon’s analysis makes use of twice as many surveys as the earlier study, including data on more recent cohorts.

We, however, explore a third possibility—methodological limitations common to both studies. Both estimate trends from data collected by different surveys that are administered to students of varying ages and use disparate methods of estimating achievement levels and socioeconomic characteristics. As Federal Reserve economist Eric Nielsen points out, when “data sources have income and achievement measures that do not map easily across surveys, they add an additional layer of complexity and uncertainty to the analysis.” It is this uncertainty that we seek to mitigate by relying on surveys that allow for consistent, intertemporally linked measures of both student achievement and socioeconomic status.

Method

We draw data from four testing programs: two that are part of the National Assessment of Educational Progress (NAEP)—the Long-Term Trend and Main NAEP; the Trends in International Mathematics and Science Study (TIMSS); and the Program for International Student Assessment (PISA). (See sidebar for details.) We include all tests administered to students age 14 or thereabouts and at age 17. (For convenience, we identify all those tested at ages 13 to 15 as “14 years old.”) All told, we compile observations of achievement levels and gaps from 46 tests in math, 40 in reading, and 12 in science, or a total of 98 intertemporally linked tests over a 47-year period. Across this time span, achievement data are available for 2,737,583 students.

To measure these students’ socioeconomic status, we use indicators of parental education and home possessions as reported by students to construct an index similar to one designed by PISA. The choice of indicators is determined by the fact that all four assessments collect information on family background directly from students themselves. Young people are thought to be aware of their parents’ level of educational attainment but to have only an imperfect knowledge of their parents’ earned income. As a classic study investigating this question puts it, income is “a matter of speculation for many students and thus inaccurately reported.” For this reason, the surveys collect economic information by asking students about household items, such as the number of durable goods and educational items present in the home. Students are likely not to have seen their family’s tax return. These same students, though, are well aware of whether they sleep in their own bedroom or share one. They also know whether their home includes a dishwasher or a computer. Our analysis thus differs from Reardon’s study, which excludes assessments that do not ask students or their parents a direct question about household income.

We use our constructed index to estimate two disparities for each test: 1) the difference in achievement between the highest and lowest deciles of the socioeconomic distribution (the 90–10 gap) and 2) the difference between the highest and lowest quartiles (the 75–25 gap). We then fit simple quadratic trend lines through these points in order to document how, if at all, the magnitude of these disparities has changed over time.

A Persistent Achievement Gap between Haves and Have Nots (Figure 1)

Achievement Gaps

As can be seen in Figure 1, the disparities in achievement between students from the highest and lowest socioeconomic status groups are strikingly persistent throughout the time period. The socioeconomic achievement divide hardly wavers over this half century. In the 1954 birth cohort, the achievement gap between the average of those in the top and bottom deciles of the socioeconomic distribution stood at slightly less than 1.2 standard deviations. For those born in 2001, the gap is only slightly less—about 1.05 standard deviations. That is, the most-disadvantaged students have made the same gains in achievement over the decades as those realized by the most-advantaged students.

The disparity between students in the top and bottom quartiles of the socioeconomic distribution was about 0.9 standard deviations for the 1954 birth cohort. This 75–25 gap falls slightly during the next two decades, settling at barely below 0.8 for the cohort born in 2001.

Trends are similar for math and reading separately. The gap in math achievement, particularly for the 90–10 comparison, shows a little movement over the period—narrowing in the early years but returning to a position below the initial level in recent decades. The 75–25 math gap narrows slightly over time. In reading, the pattern appears essentially flat for the entire period.

To see whether an alternative measure of socioeconomic status yields similar results, we estimate the gap between students who are eligible for the federal school-lunch program and those who are not, as reported on the Main NAEP, the one assessment that contains this information. The federal program provides free lunch to extremely poor students from households below the poverty line, while a reduced-price lunch is available to moderately poor students with somewhat higher incomes (1.85 times the poverty line). The gap between the extremely poor students and other students in the 1982 birth cohort is a sizable 0.73 standard deviations (Figure 2). When the extremely poor are combined with the moderately poor, the gap for this cohort is nearly as large. Over the next 20 years, the gap between the extremely poor and students from families above the eligibility line narrows by just 0.02 standard deviations, while the gap between ineligible students and all those eligible for participation in the program widens by 0.01. In sum, this alternative measure of the achievement gap between students from higher and lower socioeconomic backgrounds also shows only minuscule change over the course of the past two decades.

Stubborn Race and Socioeconomic Gaps (Figure 2)

Figure 2 also shows the white-black achievement gap. While this is not accurately thought of as a socioeconomic gap because of the improvements in black incomes, it represents another potential dimension of continuing societal disparities. As Figure 2 shows, there is a sizable shrinking of the racial gap in the early period but little change across the last two decades.

Some have hypothesized that the lack of success in diminishing the size of the socioeconomic gap is due to changes in the racial and ethnic composition of the school population. It is true that the ethnic makeup of the school-age population has changed dramatically over the past half century, with the share that is white declining from about 75 percent to 55 percent. However, these changes do not seem to have materially affected trends in performance gaps. The 90–10 socioeconomic achievement gap among white students born in 1954 was one standard deviation. By the middle of the period, the divide had declined by about 0.2 standard deviations, but it then rose again by a commensurate amount. Trends for the 75–25 socioeconomic achievement gap among whites are much the same, confirming that changes in the ethnic composition of student cohorts do not account for the unwavering divide between the haves and have-nots.

In sum, our results confirm Reardon’s finding of large gaps in academic performance between students at the extremes of the socioeconomic distribution. The average 90–10 income achievement gap across the surveys suggested by the Reardon analysis is very similar to the 90–10 socioeconomic achievement gap we identify. We are, however, unable to replicate Reardon’s finding that achievement differentials have risen by as much as 75 percent over the past 50 years. His results may be a function of a reliance upon cross-sectional studies that use disparate methods for collecting both income and achievement information. Whatever the reason, the trends estimated in his analysis differ markedly from the gaps we observe by using a uniform measure of socioeconomic status and data from intertemporally linked surveys administered to students of the same age.

Steady Gains for Younger, but not Older, Adolescents (Figure 3)

Rising Tides?

We might feel differently about these persistent achievement gaps if we found that all achievement was rising and thus suggesting improved economic futures for all. To place the achievement gaps in context, we describe changes in the average level of achievement among students at age 14 and age 17 for students born between 1954 and 2001. Figure 3 shows a significant upward trend in the average achievement level for all adolescent students of approximately 0.3 standard deviations over the course of the past half century, or approximately 0.06 per decade. This trend differs by the age of the student, however. Students at age 14 show an overall increase of about 0.43 standard deviations, or approximately 0.08 per decade, but gains among students at age 17 amount to only about 0.10 standard deviations, or 0.02 per decade. Further, we see no improvement in the performance of older students after the 1970 birth cohort.

Trends in average levels of achievement do differ in magnitude by subject, but the overall patterns are quite similar. In math, the younger adolescents register average gains of 0.9 standard deviations, while the older ones show a shift upward of only 0.25. At both ages, the reading gains are less. The trend among younger adolescents amounts to just 0.20 standard deviations over the half century and, among older ones, the trend is flat, showing no upward trend at all.

The differences in trend lines for students at different ages presents a puzzle for which we have no easy answer. Even setting aside the oldest students in our data, we see that the average improvement in test performance among 13- and 14-year-olds who take the NAEP tests and the TIMSS is larger than that registered by 15-year-olds on the PISA tests. This may reflect differences in test design, or it may suggest that the fade-out in gains begins in the early years of high school. The lack of a positive trend among 17-year-olds for the past quarter century also suggests that high schools do not build upon gains achieved earlier, a signal, perhaps, that the high school has become a troubled institution. In any event, there is no sign of a rising tide that lifts all boats at age 17 when these students are going into further schooling or into the labor force.

Importantly, the age anomaly that we see in the trends in achievement levels is not found in the performance gaps. Constant social gaps are found across all age groups.

Discussion

The achievement gap between haves and have-nots in the U.S. remains as large as it was in 1966, when James Coleman wrote his landmark report and the nation launched a “war on poverty” that made compensatory education its centerpiece. That gap has not widened, as some have suggested. But neither has it closed.

The question remains: why has the gap remained constant? The tempting answer is that nothing significant enough has happened to alter its size. But this would ignore a wide variety of factors that have shifted over the years. It is more likely that some changes within families and within schools have worked to close the socioeconomic achievement gap while other changes have widened it, with these factors largely offsetting one another.

Families. In terms of family background, there is the widening differential in household income that motivated Reardon’s work. Socioeconomic differences in the age of the mother at the birth of the child have also increased in the past 50 years. The incidence of single-parent households has increased and is likewise concentrated at the lower end of the socioeconomic spectrum. Each would tend to exacerbate socioeconomic achievement gaps.

But these negative factors could be offset by other, countervailing demographic changes. Most importantly, differences among children in their parents’ level of educational attainment have narrowed as overall education levels have climbed. So have differences in the number of siblings in the household. Both factors are important determinants of student achievement. The balance among all these factors may well have left the family contribution to the achievement gap at much the same level today as it was for cohorts born in the 1950s.

Schools. Similarly, there may be opposing forces within the educational system that have offset one another. On the one side, over the past 50 years, the federal government has enacted compensatory education programs for school-age children and the Head Start program for students at ages three and four. Brown v. Board of Education and the Civil Rights Act of 1964 accelerated school desegregation, particularly in the South. The Individuals with Disabilities Education Act funded school services for students with disabilities, a group disproportionately composed of children from low-income families. States systematically changed their funding of local schools, often in response to court orders, leading to more equal funding between rich and poor school districts. Overall school funding increased dramatically on a per-student basis, quadrupling in real dollars between 1960 and 2015. And finally, states have introduced measures holding schools accountable for student performance, as required by the 2002 No Child Left Behind Act. Accountability mandates were disproportionately directed toward schools serving low-income students. Each is aimed at closing gaps.

On the other hand, the quality of the teaching force—a centrally important factor affecting student achievement—may well have declined over the course of the past several decades. Women have greater access to opportunities outside the field of teaching. Teachers’ performance on standardized tests has slipped, along with other indicators of selectivity. Teacher salaries have declined relative to those earned by other four-year college-degree holders and are currently low relative to comparable workers in other occupations (see “Do Smarter Teachers Make Smarter Students?features, Spring 2019).

These changes affecting the quality of the teaching force are likely to have had a disproportionately adverse effect on disadvantaged students. Collective-bargaining agreements and state laws have granted more-experienced teachers seniority rights, leaving disadvantaged students to be taught by less-effective novices.

In other words, a growing disparity in teacher quality across the social divide may have offset the impacts of policies designed to work in the opposite direction.

Conclusion

Two surprises emerge from this analysis of long-term trends in student-achievement levels and gaps across the socioeconomic distribution. First, gaps in achievement between the haves and have-nots are mostly unchanged over the past half century. Second, steady gains in student achievement at the 8th-grade level have not translated into gains at the end of high school.

Because cognitive skills as measured by standardized achievement tests are a strong predictor of future income and economic well-being, the unwavering achievement gap across the socioeconomic spectrum sends a discouraging signal about the possibilities of improved intergenerational social mobility. Perhaps more disturbing, programs to improve the education of disadvantaged students, while perhaps offsetting a decline in the quality of teachers serving such students, have done little to close achievement gaps. These steadfast disparities suggest the need to reconsider the current direction of national education policy.

Two areas for further exploration seem especially critical. First, researchers have uniformly found that teacher effectiveness is a predominant factor affecting school quality. While there has been ample commentary on teacher recruitment and compensation policies, few programs and policies at scale have directly focused on enhancing teacher quality, particularly for disadvantaged students. Second, the achievement gains realized by students at age 14 fade away by age 17, yet policymakers have left high schools—like the achievement gap itself—in many ways untouched.

Eric A. Hanushek is the Paul and Jean Hanna Senior Fellow at the Hoover Institution of Stanford University. Paul E. Peterson is senior editor of Education Next; professor of government and director of Harvard’s Program on Education Policy and Governance; and a senior fellow at the Hoover Institution. Laura M. Talpey is a research associate at the Stanford Institute for Economic Policy Research. Ludger Woessmann is professor of economics at the University of Munich.


Data Sources

We use surveys from four testing programs to investigate achievement gaps and levels over time. These surveys use consistent data-collection procedures to trace the achievement of representative samples of U.S. adolescents over time. They also collect information about the cultural and economic resources of the students’ families using student reports of their parents’ education and of a wide variety of durable material and educational possessions in the home. Each data set comprises student-level data that we aggregate by demographic group.

Long-Term Trend National Assessment of Educational Progress (LTT-NAEP)

The LTT-NAEP dates back to 1971 and assesses students age 9, 13, and 17. Data are available for math in select years from 1978–2008 and for reading from 1971–2008. We create a panel of math and reading scores for students age 13 and 17, beginning with the 1954 birth cohort, who turned 17 in 1971. LTT-NAEP is the only source of information for cohorts born between 1954 and 1976. In a typical year, approximately 17,000 students participate.

Main National Assessment of Educational Progress (Main NAEP)

The Main NAEP started in 1990 and assesses students in grades 4, 8, and 12 every two to four years. We create a panel of math and reading scores for 8th graders from 1990–2013. The Main NAEP is aligned to school curricula and designed to provide results for representative samples of students in the United States as a whole and for each participating state. For each test administration, the Main NAEP 8th-grade sample is over 150,000 observations.

Trends in International Mathematics and Science Study (TIMSS)

TIMSS, administered by the International Association for Evaluation of Educational Achievement (IEA), is the current version of an international survey that originated as an exploratory study of mathematics conducted across 12 countries in the 1960s. The tests are curriculum-based and developed by an IEA-directed international committee. Beginning with the 1981 birth cohort (tested in 1995), the TIMSS tests have been designed to generate scores that are comparable over time. We use the TIMSS 8th-grade math and science tests beginning with this cohort by compiling national data files from 2003, 2007, and 2011, and international data files from 1995, 1999, and 2015. The only difference between the national and international data is that the latter do not contain an indicator of race or ethnicity. For this reason, our estimates of the black-white achievement gap for TIMSS are only available for 2003, 2007, and 2011. The U.S. TIMSS 8th-grade sample includes roughly 10,000 students for each administration of the test.

Program for International Student Assessment (PISA)

PISA, administered by the Organization for Economic Co-operation and Development, began in 2000 and assesses students’ math, reading, and science literacy at age 15 every three years. Its assessments are designed to measure practical applications of knowledge. The United States has participated in every wave of the test, though results are not available for reading for the 1991 birth cohort. We use national PISA data, available every three years from 2000 to 2015. PISA does not collect information on race or ethnicity, so these tests are not used in our analysis of the black-white achievement gap. The U.S. PISA sample includes over 5,000 students for each administration of the test.

This article appeared in the Summer 2019 issue of Education Next. Suggested citation format:

Hanushek, E.A., Peterson, P.E., Talpey, L.M., and Woessmann, L. (2019). The Achievement Gap Fails to Close: Half century of testing shows persistent divide between haves and have-nots. Education Next, 19(3), 8-17.

The post The Achievement Gap Fails to Close appeared first on Education Next.

]]>
49709511
Apprenticeship Programs in a Changing Economic World https://www.educationnext.org/apprenticeship-programs-changing-economic-world/ Wed, 12 Jul 2017 00:00:00 +0000 http://www.educationnext.org/apprenticeship-programs-changing-economic-world/ In a knowledge-based economy, early employment gains with vocational training may lead to later problems when specific skills become obsolete and workers lack the ability to adjust to a changed economic environment.

The post Apprenticeship Programs in a Changing Economic World appeared first on Education Next.

]]>

The nagging problem of significant numbers of youth leaving school unprepared for career employment has revitalized interest in vocational education, particularly apprenticeships. Support for vocational education comes from people across the political spectrum, from both labor and business groups, and from the popular media. The clearest manifestation in policy is President Trump’s executive order that calls for immediate expansion of existing apprenticeship programs while simultaneously disparaging the effectiveness of current education and training programs.

Recent evidence, however, suggests caution. In a knowledge-based economy, early employment gains with vocational training may lead to later problems when specific skills become obsolete and workers lack the ability to adjust to a changed economic environment.

Adapting to changed conditions is exactly the theme of much recent discussion about formerly middle-class jobs disappearing to technological advance. This situation will only be alleviated by building a strong educational foundation that provides workers with the ability to adapt as demands change. We should not think that expanded apprenticeships will provide this foundation. More likely they will reproduce the current skill mismatch for future generations.

Vocational/Apprentice Programs

While some countries, particularly in Europe, stress vocational education that develops specific job-related skills, others, like the US, emphasize general education that provides students with broad knowledge and basic skills. In assessing these alternatives, virtually all discussion of vocational education emphasizes its potential advantages in easing entry into the labor market by youth (with surprisingly mixed evidence).

But there is also the other end of the market. If people receive skills that are finely tuned to current employment opportunities, they might not be particularly prepared to adjust to new technologies. Thus, with higher growth rates and faster technological and structural change, people with vocational training may be more likely to be out of the labor market later in the life cycle.

In recent research with Guido Schwerdt and Lei Zhang, we turn to international comparisons to test the proposition that the life-cycle impact of vocational education might be quite different from that observed at entry into the labor market. We rely first on data on labor-market experiences of people at different ages and in different countries collected in the mid-1990s as part of an OECD-sponsored venture. The International Adult Literacy Survey (IALS) provides survey data for 18 countries, with information on some 15,000 workers between the ages of 16 and 65. This permits us to compare the life-cycle employment patterns of people with different educational backgrounds.

Our general strategy is to compare, within each country, the relative employment rates of old and young people with general education to those for people with vocational education. A key element of this “difference-in-differences” approach is confirming that today’s old people provide a good proxy for what today’s youth will look like in a few decades. For this, the IALS data are particularly useful because we have individual assessments of mathematics and reading skills that are valued in the labor markets, along with a variety of family background factors that might influence educational paths of students. We also know how the overall reliance on general and vocational education in a country has changed over time. Among other things, the data allow us to match each individual with vocational education to an individual with general education who is observationally comparable in terms of tested skills, family background, age, and years of education.

Over Life Cycle, Initial Advantage Turns into Disadvantage

We find clear evidence that the initial labor-market advantage of vocational relative to general education decreases with age. There is a trade-off between short-term benefits and long-term costs of vocational education. The skills generated by vocational education appear to facilitate the transition into the labor market but later on become obsolete at a faster rate.

Education Type and Male Life-Cycle Employment

Source: Hanushek, Schwerdt, Woessmann, and Zhang (2017)

Across our sampled countries, employment rates are higher for youth with vocational education, but this turns around by the age of 50. The employment patterns are most pronounced in the “apprenticeship countries” with combined school and work-based education programs (Denmark, Germany, and Switzerland in our data) and least noticeable in the countries with no formal system of vocational education such as the United States. The figure displays the employment patterns for the three “apprenticeship countries,” and the lower employment at older ages is very apparent.

Is the advantage of the early employment sufficient to make up for the later period of less employment for those with vocational education? An overall answer comes from comparing the life-time earnings of those with vocational and general education. When we calculate present values of earnings (discounted at 3%) for a young person, we find that lifetime earnings in vocational education are larger in Switzerland, but lower in Denmark and Germany. Interestingly, Denmark and Germany had noticeably higher growth rates of their economies over the past decades compared to Switzerland. This comparison is consistent with the idea that those with general education are more adaptable to changed economic demands.

Supporting Evidence

This general finding is reinforced by several complementary investigations. Three additional sets of analyses strengthen the interpretation that the distinct age pattern reflects depreciated skills rather than other forces inducing retirement. First, using data from the German Microcensus, we show that the same pattern holds in much larger and more recent samples and in estimation within occupational groups that excludes occupations where brawn is important. This latter estimation indicates that the differential movement out of employment is not simply a matter of physical wear and tear of people in specific vocationally intensive occupations.

Second, using Austrian Social Security data, we show that after a plant closure, the relative employment rates of displaced blue-collar workers (with more vocational training) are above those of white-collar workers at younger ages, but below them at ages above 50. The exogenous nature of the employment shock removes concerns about unobserved retirement preferences that could threaten identification.

Third, the analysis of the IALS data for the 1990s is reproduced in Woessmann’s recent work with Franziska Hampf for the 2011-12 period using the parallel data of some 29,000 adults from the Programme for the International Assessment of Adult Competencies (PIAAC). The same employment pattern with vocational and apprenticeship training is found for the recent period – despite the fact that several countries have enacted labor-market and pension reforms that have made early retirement more difficult in the meantime.

The decrease in the relative labor-market advantage of vocational education with age is apparent not only in employment, but also in income. One reason underlying the estimated labor-market patterns in the apprenticeship countries seems to be adult training. With increasing age, individuals with general education are more likely to receive career-related training relative to those with vocational education.

In sum, in dynamic economies, policy needs to consider the full life cycle.

More on Adapting to Change

Our estimates of the impact of vocational education on age-employment profiles indicate that much of the policy discussion about education programs is too narrow. Vocational education has been promoted largely as a way of improving the transition from schooling to work, but it also appears to reduce the adaptability of workers to technological and structural change in the economy. As a result, the advantages of vocational training in smoothing entry into the labor market have to be set against disadvantages later in life, disadvantages that are likely to be more severe as we move more into being a knowledge economy.

The key to understanding these results and to thinking more broadly about education policy is to recognize the importance of adaptation to changing economic conditions. Strong general education promotes greater cognitive skills that pay off in dealing with new economic processes (as seen in our recent paper with Guido Schwerdt and Simon Wiederhold). Importantly, the return to skills in the U.S. (or conversely the penalty for lack of skills) is greater than found in almost all developed countries.

Recent U.S. proposals for expanding apprenticeships by the Trump administration portray this as a way for compensating for the failure of U.S. K-12 education, but this is likely to make the long-run skill problems worse. One of the most significant labor-market problems facing the U.S. today is the number of workers whose middle-class jobs are slipping away and who are not prepared to adjust to a rapidly moving labor market. We are struggling to find ways of dealing with that problem. The caution here is that we should not lock in this situation for the future by failing to provide basic skills to the next generation. We should not delude ourselves into thinking that we can easily bring in apprenticeships to substitute for failing K-12 schooling.

— Eric A. Hanushek and Ludger Woessmann

Eric A. Hanushek is a senior fellow, Hoover Institution, Stanford University. Ludger Woessmann is Director, ifo Center for the Economics of Education and Professor of Economics, University of Munich.

This post originally appeared on the Brown Center Chalkboard.

The post Apprenticeship Programs in a Changing Economic World appeared first on Education Next.

]]>
49706835
It Pays to Improve School Quality https://www.educationnext.org/pays-improve-school-quality-student-achievement-economic-gain/ Wed, 20 Apr 2016 00:00:00 +0000 http://www.educationnext.org/pays-improve-school-quality-student-achievement-economic-gain/ States that boost student achievement could reap large economic gains

The post It Pays to Improve School Quality appeared first on Education Next.

]]>

ednext_XVI_3_hanushek_map_open

Projections and additional analysis for each state are available here.


Last year, Congress passed the Every Student Succeeds Act, supplanting No Child Left Behind and placing responsibility for public school improvement squarely upon each of the 50 states. With the federal government’s role in school accountability sharply diminished, it now falls to state and local governments to take decisive action.

Large economic benefits should accrue to states that take advantage of this new flexibility. When students learn more in school, they remain in the educational system longer and become more-skilled and -effective participants in the state’s workforce. While some graduates will migrate to other parts of the country, a majority will join the labor market in their own states, thus contributing directly to their economic strength. Over the long run, each state stands to receive an extraordinary rate of return on successful efforts to improve school quality.

Even though most education policy debates have focused on school quality and student achievement, most research on the economic impact of schooling has focused narrowly on the number of years students remain in the educational system. This metric is not an adequate measure of student achievement and thus not a reliable indicator of economic impacts: it hardly matters how long one sits at a school desk if one learns little while occupying that seat. Recently, mounting evidence has suggested  that measures of individual cognitive skills that incorporate dimensions of test-score performance provide much better indicators of economic outcomes—while also aligning the research with the policy deliberations. The importance of including direct measures of achievement is especially apparent when looking at differences in economic growth across states.

In this essay, we document the long-term economic impact of a state’s student-achievement levels, which in turn permits us to calculate the economic returns from school improvement. First, we show that in the 40 years between 1970 and 2010, the spread among the states in their per-capita gross domestic product (GDP) widened considerably. Next, we show that the level of student achievement is a strong predictor of the state’s growth rate in GDP per capita over that time period, even after accounting for both the standard measure of school attainment and other economic factors. Finally, we project for each state the large positive impact that improvements in student achievement would have on a state’s GDP (See Figure 1).

Any state political leader of vision would do well to make school quality a high priority.

ednext_XVI_3_hanushek_fig01-small

The Wealth of States

States vary sharply in the size of their per-capita GDP, that is, the total value of goods and services produced within a given year divided by the number of residents. In 2010, the wealthiest state, Delaware, enjoyed a per-capita GDP that was twice that of the poorest state, Mississippi (see Figure 2). Geographical and historical factors account for some of this variation among the states, but the discrepancies have grown over time. Importantly, the rate at which state GDPs have increased differs widely: for example, the per-capita GDP of North Dakota, the most rapidly growing state between 1970 and 2010, increased annually at a rate of 3.0 percent, while Nevada’s rate of increase was just 1.2 percent, the least of any state during this time period.

The spread among the states has remained wide and grown in absolute terms over the past several decades. In 1970 the spread between the 10th and the 40th state in the distribution was about $5,000 per capita in 2005 dollars, but by 2010 it had increased to nearly $12,000 per capita.

ednext_XVI_3_hanushek_fig02-small

Knowledge Capital vs. School Attainment

Economists have long used the term human capital to refer to the skills individuals possess that have economic value and that pay off in the labor market. But their near-ubiquitous reliance on school attainment to measure individual skill differences has made years of schooling virtually synonymous with human capital. That measure of human capital, however, implicitly assumes that each additional year of schooling translates into a comparable increment in the stock of relevant skills, totally ignoring any variations in the quality of the student’s home, community, school, teachers, and other factors.

Here, we combine the quantity of schooling with a measure of cognitive skills in order to develop a more complete understanding of differences in individuals’ labor-market skills. In the aggregate, we call this broader measure the knowledge capital of states in order to distinguish it sharply from school attainment, or conventionally measured human capital. We rely upon math test scores from the National Assessment of Educational Progress (NAEP) and various international tests to provide data on the cognitive skills of each state’s adult workers.

Using these estimates, we then consider the impact of knowledge capital on the growth in a state’s GDP. In that way, we can estimate the impact of knowledge capital on a state’s wealth and can explain, at least in part, the divergent growth rates in GDP per capita among the states between 1970 and 2010.

ednext_XVI_3_hanushek_fig03-smallDeveloping state-by-state measures of knowledge capital requires some effort. If we are to obtain an unbiased estimate of the achievement levels of a state’s adult workers, we cannot simply calculate the test scores of students currently attending the state’s schools. Many workers have migrated from a different state, and still others have immigrated to the United States from abroad; both of these groups will tend to differ in their cognitive skills from those who remain in a state after finishing their education. The degree to which state workforces consist of migrants from other parts of the United States is illustrated in Figure 3a, and the impact of foreign immigration is shown in Figure 3b. In 2010, less than 60 percent of adults living in the median state were also born in that state. The range across states varies from less than 20 percent (Nevada) to almost 80 percent (Louisiana). The share of adults not born in the United States in 2010 ranges from just 1 percent (West Virginia) to almost 30 percent (California).

To obtain our estimate of the student achievement component of the knowledge capital of a state at any point in time, we use census data to trace workers back to the place in which they were born. With that information, we can obtain a good estimate of the achievement of migrants from various states, because, on average, 86 percent of children age 14 or younger attend school in their state of birth. To estimate the achievement of workers born in the United States, we use mathematics test scores on the NAEP for 8th graders by birth state between 1990 and 2011. For workers born and educated outside of the United States, we use mathematics scores from the Program for International Student Assessment (PISA) and the Trends in International Mathematics and Science Study (TIMSS) conducted between 1995 and 2011.

We know, however, that both state migration patterns and the skills of interstate migrants are likely to differ depending on people’s educational background, so we estimate NAEP scores separately for workers with different levels of educational attainment. For example, we assume that we can assign to a college-educated individual born in Massachusetts (but possibly living elsewhere) the average test score of students with college-educated parents in Massachusetts. The achievement levels of international immigrants educated abroad are assumed to be the same as those of students performing at the 90th percentile of the distribution in their home country. We make this assumption because studies have shown that a country’s emigrants to the United States tend to be among its most talented people. (In a separate analysis, we modify this assumption to account for the less-selective nature of Mexican immigration into the United States; these results differ little from the ones reported here.)

Knowledge Capital and Economic Growth

To estimate how knowledge capital relates to the growth in a state’s GDP, we correlate the rate of GDP growth from 1970 to 2010 with our measures of the average knowledge capital of the state’s workers (based on the state’s workforce in 1970, the beginning of our growth period). Simultaneously, we adjust for the influence of three other factors that are usually hypothesized to be related to growth rates: the initial (1970) values of the level of GDP per capita, of physical capital per worker, and of the average number of years of schooling.

ednext_XVI_3_hanushek_fig04-smallFigure 4 reveals a strong relationship between the achievement component of the knowledge capital of a state’s adult workers and economic growth in that state. The cluster of states in the lower left-hand corner of the graph—Alabama, Mississippi, Utah, Nevada—have suffered from both low math achievement levels on the part of their workforces and disappointing rates of economic growth. Those in the upper right-hand corner—North Dakota, South Dakota, Minnesota, Texas, Massachusetts, and Virginia—have enjoyed both significantly higher levels of math achievement and higher rates of economic growth.

The connection between the two variables—achievement levels and economic growth—is not perfect, of course. Given the levels of achievement of workers in Kentucky, Maine, Vermont, and Montana, these states should have enjoyed higher rates of economic growth. Conversely, the economies of Connecticut, Maryland, Virginia, and Louisiana have performed better than expected, given achievement levels. But, overall, our results suggest that achievement levels that are 1 standard deviation higher—for example, having the average worker in a state achieve at the 69th percentile rather than at the 31st percentile of the overall distribution of cognitive skills—yield an average annual growth rate that is 1.4 percentage points higher.

Some may question whether this correlation actually reflects a causal relationship. One could argue that students simply learn more when their state is performing well economically, perhaps because growth generates additional resources that can be spent on education or because students are more motivated to learn when prosperity is close at hand. We are not persuaded by these arguments, in part because of the very weak correlation between increased spending on schools and higher levels of student achievement. Furthermore, the cross-state results are virtually identical to previous results from international research, and extensive analysis of the cross-country evidence has shown that a causal interpretation of the relationships is credible.

To test the credibility of our results further, we also undertook a standard accounting exercise used by economists to determine how much of the total variation in economic performance among states at any point in time can be attributed to differences in a specific factor. In particular, we use existing research about how much a high level of achievement boosts the earnings of an individual worker, combined with our new measures of the average achievement levels of workers in each state, to gauge the contribution of differences in achievement to differences in income levels across states. And we perform a parallel analysis to shed light on the role played by differences in average years of schooling.

The results of this exercise again suggest the importance of knowledge capital for state economic prosperity. We find that differences in achievement and attainment account for 20 to 35 percent of the current variation in per-capita GDP among states, with average years of schooling and achievement levels making roughly even contributions. In a sense, this estimate is surprisingly large, because both labor and capital are free to move across states—and thus tend to equalize rewards to workers with different skills. But our results are quite consistent with those obtained from similar analyses of the role of student-achievement levels in explaining differences in economic performance across countries (see “Education and Economic Growth,” research, Spring 2008).

Gains from Educational Improvement

The fact that the achievement level of a state’s workers is a key driver of its economic performance suggests that the gains from improved school quality could be substantial. Just how large would they be? We consider a range of improvements in student achievement and estimate the economic impact for each of the 50 states and for the nation as a whole. The various scenarios include:

1) Bringing every state up to the best state in the United States (Minnesota)
2) Bringing every state up to the best state in its region
3) Bringing all students in each state up to the NAEP basic achievement level
4) Bringing each state up to the best state, but assuming others do not make any gains at all, thereby isolating  the direct impact of a state’s efforts.

The calculations of the economic impact are straightforward. First, we estimate the expected growth of a state’s economy if the current skill level of workers were to remain unchanged. Then we compare this growth path to the one that would be achieved with better schools (and subsequently improved skill levels). The gains in GDP are discounted (at 3 percent per year), so that near-term gains are given more weight than gains in the more distant future. The resulting present values of income gains can be compared directly to current state GDP levels.

ednext_XVI_3_hanushek_fig05-smallOur projections account for the fact that improvement in worker skills is not instantaneous. First, we assume that education reforms take 10 years to be fully effective, with student skills improving steadily over that time. Second, the labor force improves only as new, more-skilled students replace retiring less-skilled workers. We assume that 2.5 percent of the labor force retires each year and that these workers are replaced by better-educated ones, implying that the labor force does not fully reach its ultimate new skill level for 50 years 
(10 years of reform followed by 40 years of retirements).

Figure 5 displays the economic gains from each reform scenario for the United States as a whole over the expected lifetime of a person born today (80 years), expressed in trillions of 2015 dollars. If all states improved their schools to the point where average student achievement matched that of the top state, Minnesota, the overall gains would be $76 trillion, or more than four times the current GDP of the United States. An alternative way to view this is that the nation would, on average, see a 9 percent higher level of GDP across the next 80 years. Such an increase is easily large enough to allow even the most cash-strapped state to meet current demands for public services while maintaining a balanced budget. In 2095, the GDP would be more than 36 percent larger than would be seen without school-quality improvements.

The projected economic impact of the school-improvement reforms varies considerably across the country, according to differences in the current economic position and knowledge-capital stock of each state. For example, Figure 6 shows the gains in economic outcomes that result when all states are brought up to the skill level of top-performing Minnesota (Scenario 1). This improvement means the least in North Dakota and Massachusetts, whose students are currently very close to that level, and the most in Alabama and Mississippi, where achievement levels are lowest. If California’s students could perform at the same level as Minnesota’s, the benefits to the state would exceed $16 trillion, assuming other states reached the Minnesota level. Even in North Dakota and Massachusetts, the current value of gains over the next 80 years would amount to 70 percent of current state GDP.

We find somewhat smaller gains from having each state meet the achievement level of the best state in its region (Scenario 2). This growth is necessarily less than that of the first scenario, because the achievement levels of the regional leaders vary widely. Nonetheless, the aggregate gains from Scenario 2 still have a present value of more than $35 trillion, almost twice the nation’s current GDP.

ednext_XVI_3_hanushek_fig06-smallScenario 3 essentially projects the results of realizing the achievement goals of NCLB—getting all students to a basic level of academic proficiency—but by the year 2025. The gains from all states getting students to the NAEP basic achievement level are roughly twice current GDP, or about the same as for Scenario 2.

The results for Scenario 4 represent what happens if one state acts on its own to improve school quality while all other states do not. This is an important perspective to consider, since no state that commits to a path of reform can necessarily expect others to join in, even though that would be desirable. In any given state, some of the students who profit from the improved quality of its schools will move out of the state. While the better-educated out-migrants will boost the economy of their new states, their native states will experience a brain drain.

So, what if a single state improves but others do not? Will it still benefit? Figure 5 shows that the single-state improvement strategy (summed up across all states) yields a gain of $46 trillion. When we compare this present value to that of Scenario 1, where all states move to perform at the level of the best state, we see that joint action yields gains that are 65 percent larger than the gains that would accrue to each state acting on its own. That is, aggregate rewards are smaller if any state acts without comparable efforts by others; at the same time, even the gains of acting independently are substantial.

Summing Up

Clearly, the United States stands to reap enormous economic gains from improving its schools. The goals for boosting student achievement considered in the separate scenarios of this paper are within the feasible range for most states. The largest gains would come from a coordinated improvement in performance, since states are all linked by flows of people over time. But even if states act individually, they can promote a better economic future for their residents through education reform. The gains projected here not only make the residents of each state better-off but also show how states’ fiscal problems can be tackled when knowledge capital increases.

A key feature of this analysis is that we built in realistic patterns of movements of the labor force across U.S. states and of the dynamics of school improvement. Simply put, raising the achievement of today’s students has no immediate impact on a state’s economy, because these students are not yet in the labor force. But as the skills of today’s students improve, the skills of tomorrow’s workers advance as well. Realizing these gains does require a sustained commitment on the part of a state’s political leaders. But such commitment to better schools has already given rise to dramatic gains in the United States (for instance, in Massachusetts) and abroad (as in South Korea). If we are to achieve prolonged economic growth in our nation, we have little choice but to strengthen the skills of our people.

Eric A. Hanushek is senior fellow at the Hoover Institution at Stanford University and research associate at the National Bureau of Economic Research. Jens Ruhose is an economist at Leibniz University Hannover. Ludger Woessmann is professor of economics at the University of Munich and director of the Ifo Center for the Economics of Education.

This article appeared in the Summer 2016 issue of Education Next. Suggested citation format:

Hanushek, E.A., Ruhose, J., and Woessmann, L. (2016). It Pays to Improve School Quality: States that boost student achievement could reap large economic gains. Education Next, 16(3), 52-60.

The post It Pays to Improve School Quality appeared first on Education Next.

]]>
49704396
Universal Basic Skills and Sustainable Development Goals https://www.educationnext.org/universal-basic-skills-sustainable-development-goals/ Mon, 01 Jun 2015 00:00:00 +0000 http://www.educationnext.org/universal-basic-skills-sustainable-development-goals/ A new report examines the economic impact of meeting a goal of bringing all children up to a level of basic skills.

The post Universal Basic Skills and Sustainable Development Goals appeared first on Education Next.

]]>
Ministers and education officials from a wide range of countries and international agencies converged on Incheon in the Republic of Korea last month to discuss a new set of development goals at the World Education Forum. A draft document lays out a set of Sustainable Development Goals (SDGs), which will follow on from the Millennium Development Goals (MDGs) that included education goals to be accomplished by 2015.

It is difficult to fault the SDGs as noble ambitions – end poverty everywhere, combat climate change, and more. But it is also clear that, even though they provide a plethora of targets, it will not be easy to use them either as policy levers for change or as a means of charting progress. There are also historical reasons to believe that what is not measured will not get done.

The MDGs were clearer on measureable goals. In education they called for universal access to secondary schooling. And, they showed real progress was possible: primary school enrolment rates in South Asia rose from 78% in 1999 to 94% in 2012 while they moved from 59% to 79% in sub-Saharan Africa over the same period.

Unfortunately, the best available evidence shows that many of the students appeared not to learn anything. The evidence on international achievement tests showed dismal levels of knowledge for many of the countries that improved in school access – seat time is not the same as learning. This is a huge problem, because it is knowledge and skills that pay off economically.

In a new report issued by the Organisation for Economic Co-operation and Development (OECD), Universal Basic Skills: What Countries Stand to Gain, we show the economic impact of meeting a quality goal of bringing all children up to a level of basic skills. The economic impact is huge, even for developed countries. We estimate that introducing universal basic skills by 2030 could boost GDP for lower-middle income countries by 1,302%, and 162% for high-income OECD countries.

Back to basics

Basic skills can readily be measured for a large number of countries, ones that participate in the international testing of either the Programme for International Student Assessment (PISA) or the Trends in International Mathematics and Science Study (TIMSS). These tests allow country comparisons of mathematics and science skills. In our research, we considered basic skills to be Level 1 on PISA. This level of skills corresponds to what might today be called modern functional literacy, and it provides a measuring rod for judging the skills needed for economic participation.

Our estimation of the economic impact of bringing all children up to this level comes from seeing how educational improvements translate into more economic growth. In other analysis that we have done, we show that differences in growth rates across countries are very closely related to the aggregate achievement of societies – what we call the knowledge capital of nations.

In measuring the economic impact of achieving universal basic skills, we placed countries in four income categories: lower middle income, upper middle income, high income non-OECD, and high income OECD. Based on the assumption that each of the 76 nations reaches the goal of all youth attaining at least basic skills by 2030, we calculated the average present value increases in “discounted future GDP” compared to current GDP. Discounted future GDP means that in all our calculations, the estimates further in the future were weighted less than those close to the present. By making these estimates of “present value”, we were able to make direct comparisons to current GDP when we project the impact of educational improvement on growth.

Poorest countries have the most to gain

As the graph below – which gives a selection of the results from our research – shows, the largest gains typically come for the countries in the lowest income group. The differences between countries reflects the variety in both current enrollment rates and the current achievement levels between countries.

Click to enlarge
Click to enlarge

Ghana, for example, has the lowest enrollment rate in secondary schools (46%) and also the lowest achievement levels for those in school (291 PISA points). It is extraordinarily unlikely that Ghana could move quickly enough to meet the universal skills goal in 15 years; but if it did, it would see a gain that in present value terms was 38 times its current GDP. This is equivalent to an average annual increase in discounted future GDP of 83%. The goal is more realistic for a number of other middle income countries, where the results would still be stunning.

A development goal of universal basic skills would also have meaning for high income OECD countries. High income countries have generally been left out of previous development discussions. While most of these countries have achieved nearly universal access to secondary schools, all continue to have a portion of their population that fails to achieve basic skills and that represents a group not included in any growth.

On average, these countries would see average GDP rise 3.5% over the next 80 years, which is almost exactly the average percentage of GDP they devote to public primary and secondary school expenditure. The present value of gains for the high income OECD countries averages a nontrivial 1.6 times current GDP.

Our research also separates out what would happen to economic gains under three different scenarios: increasing the quality of schools for all current students so that they reach basic skills; expanding access to secondary schools to universal enrollment at current quality levels; and simultaneously increasing enrollment and ensuring basic skills for all.

Quality over quantity

It is not surprising that the gains from expanded access are slight for the high income OECD countries, given that their average enrollment rate for secondary schools is already 98%. But even in the lowest income countries we looked at, where the secondary enrollment rate averages just 75%, the gains from improving the current quality of schools for those currently in school are three times as large as those from expanding enrollment with the current quality. Guaranteeing access to higher quality schools is six times more valuable than just expanding access to current quality schools.

The inclusive growth made possible through universal achievement of basic skills has tremendous potential as a way to address issues of poverty and limited healthcare, and to foster the new technologies needed to improve the sustainability of growth.

The SDGs on education being developed in Incheon could be substantially accomplished by focusing first on universal basic skills. No other approach has been identified that offers similar possibilities of facilitating the inclusive growth needed to address the full range of development goals. To us, the primary development goal should be universal basic skills.

—Eric A. Hanushek and Ludger Woessmann

This post originally appeared in The Conversation

The post Universal Basic Skills and Sustainable Development Goals appeared first on Education Next.

]]>
49703128
An International Look at the Single-Parent Family https://www.educationnext.org/international-look-single-parent-family/ Tue, 27 Jan 2015 00:00:00 +0000 http://www.educationnext.org/international-look-single-parent-family/ Family structure matters more for U.S. students

The post An International Look at the Single-Parent Family appeared first on Education Next.

]]>

This article is part of a new Education Next series on the state of the American family. The full series will appear in our Spring 2015 issue to mark the 50th anniversary of the 1965 release of Daniel Patrick Moynihan’s report “The Negro Family: The Case for National Action” (generally referred to as the Moynihan Report).

An unabridged version of this article is available here.

ednext_XV_2_woessmann_img00

When Daniel Patrick Moynihan raised the issue of family structure half a century ago, his concern was the increase in black families headed by women. Since then, the share of children raised in single-parent families in the United States has grown across racial and ethnic groups and with it evidence regarding the impact of family structure on outcomes for children. Recent studies have documented a sizable achievement gap between children who live with a single parent and their peers growing up with two parents. These patterns are cause for concern, as educational achievement is a key driver of economic prosperity for both individuals and society as a whole.

But how does the U.S. situation compare to that of other countries around the world? This essay draws on data from the 2000 and 2012 Program for International Student Assessment studies to compare the prevalence of single-parent families and how family structure relates to children’s educational achievement across countries. The 2012 data confirm that the U.S. has nearly the highest incidence of single-parent families among developed countries. And the educational achievement gap between children raised in single-parent and two-parent families, although present in virtually all countries, is particularly pronounced in the U.S.

Since 2000, there have been substantial changes in achievement gaps by family structure in many countries, with the gap widening in some countries and narrowing in others. The U.S. stands out in this analysis as a country that has seen a substantial narrowing of the educational achievement gap between children from single-parent and two-parent families. These varying trends, and the pattern for the U.S. in particular, confirm that family structure is by no means destiny. Ample evidence indicates the potential for enhancing family environments, regardless of their makeup, to improve the quality of parenting, nurturing, and stimulation, and promote healthy child development.

Evidence on Family Structure

The effect of family structure on child outcomes is a much-studied subject, and many researchers, including Sara McLanahan and Gary Sandefur (Growing up with a Single Parent, 1994), have explored the potentially adverse effects of single parenting on children. Single parents tend to have fewer financial resources, for example, limiting their ability to invest in their children’s development. Single parents may also have less time to spend with their children, and partnership instability may subject these parents to psychological and emotional stresses that worsen the nurturing environment for children.

Documented disadvantages of growing up in single-parent families in the United States include lower educational attainment and greater psychological distress, as well as poor adult outcomes in areas such as employment, income, and marital status. Disadvantages for children from single-parent families have also been documented in other countries, including Canada, Germany, Sweden, and the United Kingdom. But cross-country evidence has been difficult to obtain, in part because of differing methods for measuring family structure and child outcomes. The PISA studies, which asked representative samples of 15-year-olds in each participating country the same questions about their living arrangements, provide a unique opportunity to address this challenge.

At the same time, it should be noted that the descriptive patterns documented here do not necessarily capture a causal effect of living in a single-parent family. Decisions to get divorced, end cohabitation, or bear a child outside a partnership are likely related to other factors important for child development, making it difficult to separate out the influence of family structure. For example, severe stress that leads to family breakup might well have continued without the breakup and have led to worse outcomes for a child had the family remained intact. If single-parent families differ from two-parent families in unmeasured ways, then those differences may be the underlying cause of any disparities in children’s outcomes. It is even conceivable that problems a child has in school may contribute to family breakup, rather than being a consequence of it.

In addition to comparing the raw gap in educational achievement between children from single- and two-parent families, I present results that adjust for other background differences, including the number of books at home, parental education, and immigrant and language background. This type of analysis can provide useful information about the reasons educational achievement varies with family structure. It is important to keep in mind, however, that even these adjusted associations between child outcomes and family structure may well have causes other than family structure itself.

The Data 

The Program for International Student Assessment (PISA) is an internationally standardized assessment given every three years since 2000 by the Organization for Economic Co-operation and Development (OECD). PISA tests the math, science, and reading achievement of representative samples of 15-year-old students in each participating country. This analysis is limited to the 28 countries that were OECD members and PISA participants in 2000.

In nearly all countries, students living in single-parent families have lower achievement on average than students living in two-parent families.
In nearly all countries, students living in single-parent families have lower achievement on average than students living in two-parent families.

PISA collects a rich array of background information in student questionnaires. Students report whether a mother (including stepmother or foster mother) usually lives at home with them, and similarly a father (including stepfather or foster father). By including students living with step- and foster parents, the group of students identified as living in two-parent families will include some students who have experienced a family separation. It is possible that, as a result, any differences between students from single- and from two-parent families will be understated in the analysis. Evidence from 2000, the one year for which it is possible to separate out students living with stepparents, suggests that this is indeed the case. In the international sample, the achievement difference would be 16 points rather than 14 points if stepparents were excluded from the two-parent families.

I limit the analysis to students who live with either one or two parents, excluding students living with neither parent and students for whom information on either the father or the mother is missing. On average across countries, 1.6 percent of students with available data from 2012 do not live with any parent (1.9 percent in the United States) and 7 percent of the total student population (11 percent in the United States) have missing data on whether a mother and/or father lives at home with them. My total 2012 sample contains more than 230,000 students or about 8,500 students per country on average. The U.S. sample consists of more than 4,300 students living in either single-parent (student lives with either mother or father only) or two-parent (student lives with both mother and father) families.

Single-Parent Families and Student Achievement

ednext_XV_2_woessmann_fig01-smallIn the United States, in 2012, 21 percent of 15-year-old students lived in single-parent families (see Figure 1). Together with Hungary (also 21 percent), this puts the United States at the top among the countries. On average across all 28 countries, the share of single-parent families is 14 percent. New Zealand also has a share higher than 20 percent, while the Czech Republic has 18 percent, and Poland, the United Kingdom, Finland, Mexico, Denmark, and France have shares between 15 and 17 percent. At the other end of the spectrum, Greece, Korea, Italy, and Sweden have shares between 8.8 and 9.6 percent; Spain, Iceland, Norway, Ireland, and the Netherlands each have shares between 10 and 11.3 percent.

The vast majority of single-parent families are families with a single mother. On average across countries, 86 percent of single-parent families are headed by single mothers. In the United States, the figure is 84 percent.

To compare student achievement across countries, I focus on test scores in math, which are most readily comparable across countries. (Results for science and reading achievement in 2012, documented in the unabridged version of this study, are quite similar.) In each subject, PISA measures achievement on a scale that has a student-level standard deviation of 100 test-score points across OECD countries. That is, any achievement differences can be interpreted as percentages of a standard deviation in test scores, with one standard deviation in test-score performance representing between three and four years of learning on average. To illustrate, the average difference in math achievement between the two grade levels in our sample with the largest shares of 15-year-olds (9th and 10th grade) is 28 test-score points, which is a little more than one-quarter of a standard deviation and roughly equivalent to one year of learning or one grade level.

In nearly all countries, students living in single-parent families have lower achievement on average than students living in two-parent families (see Figure 2a). In the United States, the average raw achievement difference in math between students living in two-parent families and students living in single-parent families is 27 points, or roughly one grade level. The United States is one of six countries with achievement differences larger than 25 points. Belgium has the largest disparity in math achievement by family structure, at 35 points, followed by the Netherlands (29), and Poland, Japan, and the United Kingdom (27 to 28). On average across the 28 countries, students living in single-parent families score 18 points lower than students living in two-parent families.

ednext_XV_2_woessmann_fig02-small

There are exceptions, however. Mexico shows no achievement difference by family structure, and the difference is statistically insignificant in Portugal as well. The achievement difference is below 10 points in Portugal (6), Italy (7), Austria (8), and Germany (9).

ednext_XV_2_woessmann_fig03-smallFigure 3 plots these achievement gaps by family structure against the countries’ shares of students living in single-parent families. There is a slight tendency for countries with higher shares of single-parent families to have larger achievement disparities, although the correlation is not statistically significant.

The United States stands out in this figure in terms of the prevalence of single-parent families and the associated achievement gap. Belgium and the Netherlands exhibit the highest achievement disparities, although single parenthood is not particularly prevalent in these countries. The southern European countries (Greece, Italy, Portugal, and Spain) stand out as places with relatively low achievement disparities and relatively low prevalence of single parenthood. The German-speaking countries (Austria, Germany, and Switzerland) show similarly low achievement disparities despite their higher prevalence of single parenthood. The Asian countries (Korea and Japan) have lower levels of single-parent families but higher achievement disparities. The Nordic countries (Denmark, Finland, Iceland, Norway, and Sweden) all have similarly middling levels of achievement disparities despite varying levels of single-parenthood incidence. Finally, the eastern European countries (Czech Republic, Hungary, and Poland) have quite different achievement disparities despite the consistently high incidence of single-parent families.

The four quadrants divide the countries according to the degree of impact the prevalence of single-parent families is likely to have over the long term. For countries in the top right cell that have high values on both variables—the United States being the leading example—single parenthood may constitute a major concern for the next generation. It is quite prevalent, and the associated achievement gap is quite large. In countries in the bottom right cell, such as Hungary and Mexico, single parenthood is also quite common, but the achievement disparity is less severe. While single parenthood is less prevalent in the countries in the top left cell, such as the Netherlands and Ireland, the achievement difference is large and may still constitute a serious problem for affected students. Finally, in the bottom left cell, for countries, including Italy and Spain, where single parenthood is less prevalent and achievement disparities relatively small, there is less cause for concern.

Adjusting for Background Differences

The achievement differences reported so far are raw differences, not adjusted for background differences between students from single- and two-parent families. These raw differences may capture effects of disadvantaged backgrounds, as distinct from any independent effects of single parenthood. To provide a sense of the extent to which this might be the case, we next control for differences in family background beyond family structure.

In particular, we hold constant the number of books in the student’s home (as a proxy for socioeconomic background), the highest education level of the parent(s), immigration status (native, first-, and second-generation immigrants), and whether the national language is spoken at home. All these measures are strongly associated with student achievement, and across countries, books in the home and parental education tend to be negatively associated with single parenthood. In the cross-sectional data, though, we cannot observe whether some of these measures are preexisting characteristics of the families, in which case they represent potential biases, or whether they are an outcome of single parenthood.

Controlling for background factors has a substantial impact on the estimated achievement disparity between students living in single- and two-parent families (see Figure 2c). In the United States, the achievement disparity declines by more than 60 percent, from 27 to 10 points. On average across all countries, the disparity is reduced by half, from 18 to 9 points. While the United States still features above-average achievement differences by family structure after the adjustment, in absolute terms it differs less markedly from the international average. The countries with the largest adjusted achievement gap by family background are Belgium (22), Poland (21), and the Netherlands (17). In 12 countries, the adjusted achievement gap is below 5 points, or less than half the adjusted achievement gap in the United States. In seven countries, after the adjustment, the achievement disparity by family structure is no longer statistically significant. In Korea and Portugal, the adjusted relationship even turns negative.

With the exception of Mexico and Switzerland, where controlling for background factors hardly affects the results, the adjusted gaps are smaller in all countries than in the initial analysis. In the majority of countries (19 out of 28), the reduction in the achievement disparity between students in single- and two-parent families from controlling for observed factors is in the range of 40 to 80 percent of the raw difference in achievement.

The background factors do not contribute equally to the reduction in the achievement gap, however. In fact, controlling only for the number of books at home reduces the achievement gap by family structure across all countries to 9 points. By contrast, immigration status and language spoken at home hardly contribute to the reduction. This pattern is quite similar in the United States. That is, in the international sample, roughly half of the achievement difference between students living in single- and two-parent families simply reflects differences in socioeconomic status as captured by the number of books in the home.

To a large extent, the achievement gap between students living in single-parent and two- parent families reflects differences in socioeconomic background, as measured by the number of books at home and parental education, rather than family structure alone.
To a large extent, the achievement gap between students living in single-parent and two- parent families reflects differences in socioeconomic background, as measured by the number of books at home and parental education, rather than family structure alone.

With the available data, it is impossible to determine whether the relative lack of books in single-parent homes mostly reflects a preexisting feature of the families or whether it is (at least partly) an outcome of the family structure. The number of books may to some extent reflect the number of people living in the home. Figure 2b presents achievement differences between students living in single- and two-parent families, controlling for parental education, immigration status, and language spoken at home, but not for books at home. At 19 points, this alternative adjusted achievement gap in the United States lies roughly midway between the raw difference (27) and the gap as adjusted for books at home as well as the other characteristics (10). On average across countries, the achievement gap in this model is 15 points. Thus, while controlling for books at home may well capture in part the effect of family structure, some of the overall achievement gap clearly reflects preexisting differences.

Of course, the background factors considered here by no means capture all relevant differences in family background, although they have been found to be particularly relevant for student achievement. The adjusted achievement gaps by family structure above may partly reflect additional differences in family background rather than family structure alone.

Changes Over Time 

Finally, I analyze trends in the patterns over time. To do so, I perform the same analyses as above with data from
the 2000 PISA study, when the first of these surveys was administered. (See unabridged version for details.) Over the period from 2000 to 2012, the share of 15-year-olds living in single-parent families increased from 18 to 21 percent in the United States, and from 12 to 14 percent on average in the international sample, although there are substantial differences across countries. The average achievement gap in the international sample also increased by 33 percent, from 13.6 to 18 points.

ednext_XV_2_woessmann_fig04-smallIn general, countries with larger increases in the incidence of single parenthood from 2000 to 2012 tended to have larger increases in the achievement gap by family structure as well. The U.S. is a clear outlier from this pattern, however. The raw difference in math achievement between students from single- and two-parent families in the U.S. was substantially higher in 2000 than in 2012, at 37 points compared to 27 points (see Figure 4). Thus, over the course of 12 years, the achievement gap in the U.S. declined by 29 percent. In 2000, only the Netherlands, with a gap of 43 points, had a larger achievement gap than the United States. Korea (26) and Belgium (21) follow at some distance. At the other end, seven countries had achievement gaps lower than 5 points in 2000 (Iceland, Switzerland, Greece, Italy, Czech Republic, Ireland, and Mexico).

Conclusions

Single parenthood is prevalent in virtually all OECD countries, but the share of single-parent families is particularly high in the United States. Students from single-parent families perform significantly lower in math than students from two-parent families in virtually all countries. To a large extent, however, this achievement gap reflects differences in socioeconomic background, as measured by the number of books at home and parental education, rather than family structure alone. The United States belongs to the group of countries with the largest achievement gaps by family structure, although the United States was more exceptional in this regard in 2000 than in 2012. While the achievement gap between students from single- and two-parent families increased in most other OECD countries over the period, it declined in the United States.

This variation in trends shows that achievement disparities by family structure are by no means destiny. Ample evidence reveals that it is possible to enhance family environments to improve the quality of parenting, nurturing, and stimulation, and thereby promote healthy child development. Future research should investigate to what extent factors such as differing welfare systems, child support facilities, divorce regulations, and other country characteristics may lie behind the differences in achievement gaps between students from single- and two-parent families across countries and over time.

Ludger Woessmann is professor of economics at the University of Munich and director of the Ifo Center for the Economics of Education.

This article appeared in the Spring 2015 issue of Education Next. Suggested citation format:

Woessmann, L. (2015). An International Look at the Single-Parent Family: Family structure matters more for U.S. students. Education Next, 15(2), 42-49.

The post An International Look at the Single-Parent Family appeared first on Education Next.

]]>
49702603
U.S. Students from Educated Families Lag in International Tests https://www.educationnext.org/us-students-educated-families-lag-international-tests/ Tue, 13 May 2014 00:00:00 +0000 http://www.educationnext.org/us-students-educated-families-lag-international-tests/ It’s not just about kids in poor neighborhoods

The post U.S. Students from Educated Families Lag in International Tests appeared first on Education Next.

]]>

“The big picture of U.S. performance on the 2012 Program for International Student Assessment (PISA) is straightforward and stark: It is a picture of educational stagnation…. Fifteen-year olds in the U.S. today are average in science and reading literacy, and below average in mathematics, compared to their counterparts in [other industrialized] countries.”

U.S. secretary of education Arne Duncan spoke these grim words on the bleak December day in late 2013 when the international tests in math, science, and literacy were released. No less disconcerting was the secretary’s warning that the nation’s educational problems are not limited to certain groups or specific places. The “educational challenge in America is not just about poor kids in poor neighborhoods,” he said. “It’s about many kids in many neighborhoods. The [test] results underscore that educational shortcomings in the United States are not just the problems of other people’s children.”

In making his comments, Secretary Duncan challenged those who cling to an old belief that the nation’s educational challenges are confined to its inner cities. Most affluent Americans remain optimistic about the schools in their local community. In 2011, Education Next asked a representative sample to evaluate both the nation’s schools and those in their own community. The affluent were especially dubious about the nation’s schools—only 15 percent conceded them an A or a B. Yet 54 percent gave their local schools one of the two top ratings.

Public opinion is split on how well the nation’s schools educate students of different abilities. In 2013 Education Next asked the public whether local schools did a good job of teaching talented students. Seventy-three percent said the local schools did “somewhat” or “extremely” well at the task, as compared to only 45 percent who thought that was true of their capacity to teach the less-talented.

To see whether this optimistic assessment of the nation’s ability to teach the more able student is correct, we draw upon the latest tests of student achievement and find that, as Secretary Duncan has said, the nation’s “educational shortcomings” are not just the problems of the other person’s child. We have given special attention to math performance because math appears to be the subject in which accomplishment is particularly significant for both an individual’s and a country’s economic well-being.

When viewed from a global perspective, U.S. schools seem to do as badly teaching those from better-educated families as they do teaching those from less well educated families. Overall, the U.S. proficiency rate in math (35 percent) places the country at the 27th rank among the 34 OECD countries that participated in the Program for International Student Assessment (PISA). That ranking is somewhat lower for students from advantaged backgrounds (28th) than for those from disadvantaged ones (20th).

There are examples of excellence. The six states with high proficiency rates (58 to 62 percent) among students from families with high levels of parental education rank among the OECD top 13 on this measure. But students from these states are a small portion of the U.S. student population, and other states rank much lower down the international list. In many places, students from highly educated families are performing well below the OECD average for similarly advantaged students.

There can be little doubt that education shortcomings in the United States spread well beyond the corridors of the inner city or the confines of low-income neighborhoods where many parents lack a high school diploma. While bright spots can be identified—particularly in some states along the country’s northern tier—the overall picture is distressing to those concerned about the potential evolution of economic well-being of the United States in the 21st century.

Conventional Wisdom

Not everyone agrees that the nation’s schools are in trouble. In their apology for the American school, David Berliner and Gene Glass seek to reassure Americans by trying to isolate the problem to minority groups or those of low income. “In the United States, if we looked only at the students who attend schools where child poverty rates are under 10 percent, we would rank as the number one country in the world,” they write.

These claims are highly misleading. The important question to ask is, Do students with similar family background do better in the United States than in other countries?

Defenders of the American school also like to compare the highest-performing states within the United States to all students in other countries. “Massachusetts…scored so high that only a few Asian countries beat it,” Berliner and Glass declare. “The states of Massachusetts, Minnesota, and Colorado…ranked among the top-performing nations in the world.” It is true that Massachusetts schools stand up to world competition, but it is important to keep in mind that the K–12 students living in Massachusetts are just 2 percent of the nation’s total. One cannot generalize to the country as a whole from this small state.

The Study

Our state-by-state data come from the 2011 tests administered to representative samples of U.S. students in 8th grade by the National Assessment of Educational Progress (NAEP). Our country-by-country data come from the PISA tests, which are administered by the Organization for Economic Co-operation and Development (OECD). In 2012, OECD administered the PISA tests to representative samples of students at the age of 15 in 68 jurisdictions, including all 34 OECD countries. Our analysis compares U.S. performance to those of students in the other OECD countries.

ednext_XIV_4_peterson_method-smallThe proficiency and advanced standards used in this study follow those developed by NAEP. To equate proficiency and advanced performance rates across states and countries, we execute a crosswalk between the NAEP and PISA tests by identifying levels of performance on PISA that yield equivalent proportions of U.S. students that meet the NAEP proficiency and advanced standards (see Methodology sidebar).

To assess overall performance, we identify the percentage of students in the high school class of 2015 who are performing at proficient and advanced levels of achievement in math. (While not reported here, we also looked at reading and science, and the results are broadly similar to those for math.) We focus on how each state within the United States ranks relative to all 33 other OECD countries.

To ascertain whether the challenges facing the United States are concentrated among the educationally disadvantaged, we identify for each state and country the proficiency rate of students from families with parents of high, moderate, and low levels of education. If the problems are concentrated in ways that some would have us believe, U.S. students from families with high parental education should compare favorably with similarly situated students abroad. Such a finding would support the oft-repeated claim that the achievement challenges are limited to those who come from disadvantaged families (measured here by low levels of parental education).

ednext_XIV_4_peterson_fig01-small

How Well Do U.S. Schools Educate Different Students?

According to NAEP, 35 percent of the members of the U.S. class of 2015 reach or exceed the proficiency level in math. Based on our calculations, this percentage places the United States at the 27th rank among the 34 OECD countries (see Figure 1). The percentage of students who are math proficient is nearly twice as large in Korea (65%), Japan (59%), and Switzerland (57%). Other countries with performances that clearly outrank the United States include Finland (52%), Canada (51%), Germany (50%), Australia (45%), France (42%), and the United Kingdom (41%).

To see whether the low U.S. ranking in math is due mainly to social class factors separate and apart from the schools, we next identify proficiency ratings for students from families with differing amounts of parental education.

Low parental education. Only 17 percent of these U.S. students are proficient in math (see Figure 2). This is half or less than the percentage of similarly situated students (those whose parents also have low levels of education) in Korea (46%), the Netherlands (37%), Germany (35%), and Japan (34%). Among OECD countries as a whole, the United States ranks 20th, placing it slightly ahead of Austria and France and just behind Denmark and the United Kingdom. In simplest terms, many other countries do a much better job of educating young people whose parents lack a high school diploma.

ednext_XIV_4_peterson_fig02-small

Moderate parental education. The relative standing of the United States is even worse among students from moderately well educated families. The math proficiency rate (26%) for this group is again around half the rate enjoyed by Switzerland (57%), Korea (56%), Germany (52%), and the Netherlands (50%). Other major countries that outperform the United States include Japan (48%), Canada (43%), Poland (43%), the United Kingdom (39%), and France (35%). When it comes to instructing the children of the moderately well educated, the United States comes in at the 30th rank among the 34 OECD countries, 10 ranks lower than was the case for students from families with low parental education.

High parental education. The percentage proficient of 15-year-olds from families with high parental education is conventionally thought to be the exception to this bleak picture. Indeed, the proficiency rate of 43% is higher than the rate for families with low (17%) or moderate (26%) levels of education. But the relative standing of the United States vis-à-vis other OECD countries remains near the very bottom (see Figure 3), at the 28th rank. When viewed from a global perspective, U.S. schools seem to do as badly teaching those from better-educated families as they do teaching those from the less well educated.

ednext_XIV_4_peterson_fig03-small

Countries with high proficiency rates among students from better-educated families include Korea (73%), Poland (71%), Japan (68%), Switzerland (65%), Germany (64%) and Canada (57%). Perhaps the only comfort the United States can take is that it is only 5 percentage points behind its mother country, the United Kingdom (48%).

Across the OECD, there is a strong relationship between the math performance of students from families with high and with low educational backgrounds. Mexico and Chile are particularly weak at educating those from better-educated families, however. Conversely, Poland and Slovakia are particularly weak at educating students from families with less education, given the performance of those from families with high education. The relative performance of the U.S. education system is pretty much the same across social groups. It is weak at the bottom, no less weak at the middle, and just as weak with respect to educating the most-advantaged. As Secretary Duncan said, it is not a problem of some other person’s child.

Ranking States

The overall math proficiency rate of 15-year-olds varies widely among the states—from a high of 51 percent in Massachusetts to a low of 19 percent in Mississippi. Striking differences remain when one divides students according to parental education. For students from families with low parental education levels, Texas (28%) and New Jersey (25%) have the highest proficiency rates, well ahead of Massachusetts and Minnesota (both at 18%), putting them in 7th and 8th place among U.S. states for this category of students. Maryland and Illinois are at about the national average, while New York, in 27th place, falls slightly below. California (9%), West Virginia (6%), and Utah (5%) rank at embarrassingly low levels. (See the interactive map for a picture of the overall pattern throughout the 50 states.)

Many people assume that students coming from families with high education levels are keeping up with their peers abroad. Indeed, in some parts of the United States that is in fact the case. More than 62 percent of students from Massachusetts families with high levels of parental education are proficient in math, placing that state just behind Germany (64%) and Switzerland (65%), two of the top-five OECD countries. Only a bit further back are Vermont, Minnesota, Colorado, New Jersey, and Montana, all of which have a proficiency rate of 58 percent or 59 percent for students from better-educated families. Internationally, that places these states in the same league as the Czech Republic (58%), Canada (57%), and Finland (56%), which are among the OECD top 13.

But those six states are the highest-performing states in the Union. Other states rank much lower down the international list. In many places, students from highly educated families are performing well below the OECD average for similarly advantaged students. For example, Wisconsin, if ranked as a country on this measure, would come in 21st, just below Ireland. California is large enough to be an OECD country in its own right. If it were, its 43 percent proficiency rating would place it 30th, just below Italy, and New York’s 40 percent rating entitles it to assume position number 31, just below Turkey. Florida’s 38 percent rating gives it the 32nd position, just below Sweden, which has registered an abysmal performance given its level of economic development. Ranked near the bottom, Alabama, West Virginia, and Louisiana do worse than all OECD countries with the exception of Chile and Mexico. (See the interactive map for an overall portrait of the pattern among the states.)

Similar to the international comparisons, states that rank well for math education among students with high parental education tend also to rank highly for students from less-advantaged backgrounds. But some high-performing states, such as Massachusetts, Vermont, and Colorado, do relatively better with students from families with higher educational backgrounds than they do with their less-advantaged peers.

Advanced Performance in Math

The U.S. economic strength has been built in large part through its record of invention and innovation, things that themselves depend upon the country’s historic strength in science, technical, engineering, and math fields (STEM). The pool of people prepared to go into these fields in the future is dependent on students who have developed advanced skills in math and science in school.

Eight percent of the U.S. class of 2015 proved its merit by scoring at the advanced level on the NAEP in math. That could be regarded as a triumph were it not for the fact that it leaves the United States 28th on the OECD list. Other countries do a much better job at bringing students up to the advanced level of performance. The eight world leaders are Korea (30%), Japan (23%), Switzerland (20%), Belgium (19%), the Netherlands (18%), Germany (17%), Poland (16%), and Canada (16%). Disturbingly, our neighbor to the north turns out twice as high a percentage of students at the advanced level in math as the United States.

The percentage scoring at the advanced level is only 2 percent for U.S. students from families with low levels of educational attainment and only 4 percent for students from moderately educated families. Those disgraceful numbers could be offset by unusually high performances among the better-educated, however. Does the United States achieve a breakthrough at least among this group? Some may wish to take pride in the fact that 12 percent of the students from better-educated families reach the advanced level in math. But such pride is misplaced, as the feat still leaves the United States in the 28th position out of the 34 OECD countries. Only Sweden, Spain, Turkey, Greece, Chile, and Mexico do worse.

Advanced Performance by State

The four states with 13 percent or more students performing at the advanced level in math are Massachusetts, New Jersey, Minnesota, and Vermont, with the Bay State taking honors with 15 percent of its students scoring at that level. All of these states rank alongside the top 13 OECD countries, and Massachusetts ranks 9th, just below Canada, though still well below Korea and Japan. With less than 7 percent of students performing at the advanced level, New York and California rank 31st, just ahead of Turkey and Greece. The two lowest-performing states, Alabama and Louisiana, however, do outrank the two lowest-performing OECD countries—Chile and Mexico.

The same states—Massachusetts, New Jersey, Minnesota, and Vermont—are top performers on this measure for students from families with high educational backgrounds; in all four plus Colorado, 18 percent or more of such students perform at the advanced level. That places them in the same league as Canada and France but well behind Korea, Poland, Japan, Switzerland, Belgium, and Germany. But only 15 percent perform at this level in Pennsylvania and 14 percent in Wisconsin, and less than 10 percent do so in New York, Michigan, and Florida. If states do well with students from better-educated family backgrounds, they tend to do well with those from less-educated ones. But there are clear exceptions to this pattern. West Virginia, Louisiana, and Mississippi score particularly badly on their capacity to teach students from more-educated backgrounds.

Conclusions

Lacking good information, it has been easy even for sophisticated Americans to be seduced by apologists who would have the public believe the problems are simply those of poor kids in central city schools. Our results point in quite the opposite direction. We find that the international rankings of the United States and the individual states are not much different for students from advantaged backgrounds than for those from disadvantaged ones. Although a higher proportion of U.S. students from better-educated families are proficient, that is equally true for similarly situated students in other countries. Compared to their counterparts abroad, however, U.S. students from advantaged homes lag severely behind.

As long as the focus remains on distinctions within the United States, then the comfortable can remain comforted by the distance between suburbia and the inner city. But once the focus shifts to countries abroad and fair, apples-to-apples comparisons are made, it becomes manifest that nearly all of our young people—from privileged and not-so-privileged backgrounds—are not faring well.

Some say that we must cure poverty before we can address the achievement problems in our schools. Others say that our schools are generally doing fine, except for the schools serving the poor. Bringing an international perspective correctly to bear on the issue dispels both thoughts.

The United States has two achievement gaps to be bridged—the one between the advantaged and the disadvantaged and the one between itself and its peers abroad. Neither goal need be sacrificed to attain the other.

Eric A. Hanushek is senior fellow at the Hoover Institution of Stanford University. Paul E. Peterson is professor of government and director of the Program on Education Policy and Governance at Harvard University. Ludger Woessmann is professor of economics at the University of Munich and director of the Ifo Center for the Economics of Education and Innovation. An unabridged version of this report is available at hks.harvard.edu/pepg/.

This article appeared in the Fall 2014 issue of Education Next. Suggested citation format:

Hanushek, E.A., Peterson, P.E., and Woessmann, L. (2014). U.S. Students from Educated Families Lag in International Tests: It’s not just about kids in poor neighborhoods. Education Next, 14(4), 8-18.

The post U.S. Students from Educated Families Lag in International Tests appeared first on Education Next.

]]>
49701597
Is the U.S. Catching Up? https://www.educationnext.org/is-the-us-catching-up/ Tue, 21 Aug 2012 00:00:00 +0000 http://www.educationnext.org/is-the-us-catching-up/ International and state trends in student achievement

The post Is the U.S. Catching Up? appeared first on Education Next.

]]>

Read the unabridged version of this report here.


“The United States’ failure to educate its students leaves them unprepared to compete and threatens the country’s ability to thrive in a global economy.” Such was the dire warning issued recently by an education task force sponsored by the Council on Foreign Relations. Chaired by former New York City schools chancellor Joel I. Klein and former U.S. secretary of state Condoleezza Rice, the task force said the country “will not be able to keep pace—much less lead—globally unless it moves to fix the problems it has allowed to fester for too long.” Along much the same lines, President Barack Obama, in his 2011 State of the Union address, declared, “We need to out-innovate, out-educate, and out-build the rest of the world.”

Although these proclamations are only the latest in a long series of exhortations to restore America’s school system to a leading position in the world, the U.S. position remains problematic. In a report issued in 2010, we found only 6 percent of U.S. students performing at the advanced level in mathematics, a percentage lower than those attained by 30 other countries. And the problem isn’t limited to top-performing students. In 2011, we showed that just 32 percent of 8th graders in the United States were proficient in mathematics, placing the U.S. 32nd when ranked among the participating international jurisdictions (see “Are U.S. Students Ready to Compete?features, Fall 2011).

Admittedly, American governments at every level have taken actions that would seem to be highly promising. Federal, state, and local governments spent 35 percent more per pupil—in real-dollar terms—in 2009 than they had in 1990. States began holding schools accountable for student performance in the 1990s, and the federal government developed its own nationwide school-accountability program in 2002.

And, in fact, U.S. students in elementary school do seem to be performing considerably better than they were a couple of decades ago. Most notably, the performance of 4th-grade students on math tests rose steeply between the mid-1990s and 2011. Perhaps, then, after a half century of concern and efforts, the United States may finally be taking the steps needed to catch up.

To find out whether the United States is narrowing the international education gap, we provide in this report estimates of learning gains over the period between 1995 and 2009 for 49 countries from most of the developed and some of the newly developing parts of the world. We also examine changes in student performance in 41 states within the United States, allowing us to compare these states with each other as well as with the 48 other countries.

Data and Analytic Approach

Data availability varies from one international jurisdiction to another, but for many countries enough information is available to provide estimates of change for the 14-year period between 1995 and 2009. For 41 U.S. states, one can estimate the improvement trend for a 19-year period—from 1992 to 2011. Those time frames are extensive enough to provide a reasonable estimate of the pace at which student test-score performance is improving in countries across the globe and within the United States. To facilitate a comparison between the United States as a whole and other nations, the aggregate U.S. trend is estimated for that 14-year period and each U.S. test is weighted to take into account the specific years that international tests were administered. (Because of the difference in length and because international tests are not administered in exactly the same years as the NAEP tests, the results for each state are not perfectly calibrated to the international tests, and each state appears to be doing slightly better internationally than would be the case if the calibration were exact. The differences are marginal, however, and the comparative ranking of states is not affected by this discrepancy.)

Our findings come from assessments of performance in math, science, and reading of representative samples in particular political jurisdictions of students who at the time of testing were in 4th or 8th grade or were roughly ages 9‒10 or 14‒15. The political jurisdictions may be nations or states. The data come from one series of U.S. tests and three series of tests administered by international organizations. Using the equating method described in the methodology sidebar, it is possible to link states’ performance on the U.S. tests to countries’ performance on the international tests, because representative samples of U.S. students have taken all four series of tests.

Comparisons across Countries

In absolute terms, the performance of U.S. students in 4th and 8th grade on the NAEP in math, reading, and science improved noticeably between 1995 and 2009. Using information from all administrations of NAEP tests to students in all three subjects over this time period, we observe that student achievement in the United States is estimated to have increased by 1.6 percent of a standard deviation per year, on average. Over the 14 years, these gains equate to 22 percent of a standard deviation. When interpreted in years of schooling, these gains are notable. On most measures of student performance, student growth is typically about 1 full standard deviation on standardized tests between 4th and 8th grade, or about 25 percent of a standard deviation from one grade to the next. Taking that as the benchmark, we can say that the rate of gain over the 14 years has been just short of the equivalent of one additional year’s worth of learning among students in their middle years of schooling.

Yet when compared to gains made by students in other countries, progress within the United States is middling, not stellar (see Figure 1). While 24 countries trail the U.S. rate of improvement, another 24 countries appear to be improving at a faster rate. Nor is U.S. progress sufficiently rapid to allow it to catch up with the leaders of the industrialized world.

Students in three countries—Latvia, Chile, and Brazil—improved at an annual rate of 4 percent of a standard deviation, and students in another eight countries—Portugal, Hong Kong, Germany, Poland, Liechtenstein, Slovenia, Colombia, and Lithuania—were making gains at twice the rate of students in the United States. By the previous rule of thumb, gains made by students in these 11 countries are estimated to be at least two years’ worth of learning. Another 13 countries also appeared to be doing better than the U.S., although the differences between the average improvements of their students and those of U.S. students are marginal.

Student performance in nine countries declined over the same 14-year time period. Test-score declines were registered in Sweden, Bulgaria, Thailand, the Slovak and Czech Republics, Romania, Norway, Ireland, and France. The remaining 15 countries were showing rates of improvement that were somewhat slower than those of the United States.

In sum, the gains posted by the United States in recent years are hardly remarkable by world standards. Although the U.S. is not among the 9 countries that were losing ground over this period of time, 11 other countries were moving forward at better than twice the pace of the United States, and all the other participating countries were changing at a rate similar enough to the United States to be within a range too close to be identified as clearly different.

Which States Are the Big Gainers?

Progress was far from uniform across the United States. Indeed, the variation across states was about as large as the variation among the countries of the world. Maryland won the gold medal by having the steepest overall growth trend. Coming close behind, Florida won the silver medal and Delaware the bronze. The other seven states that rank among the top-10 improvers, all of which outpaced the United States as a whole, are Massachusetts, Louisiana, South Carolina, New Jersey, Kentucky, Arkansas, and Virginia. See Figure 2 for an ordering of the 41 states by rate of improvement.

Iowa shows the slowest rate of improvement. The other four states whose gains were clearly less than those of the United States as a whole are Maine, Oklahoma, Wisconsin, and Nebraska. Note, however, that because of nonparticipation in the early NAEP assessments, we cannot estimate an improvement trend for the 1992‒2011 time period for nine states—Alaska, Illinois, Kansas, Montana, Nevada, Oregon, South Dakota, Vermont, and Washington.

Cumulative growth rates vary widely. Average student gains over the 19-year period in Maryland, Florida, Delaware, and Massachusetts, with annual growth rates of 3.1 to 3.3 percent of a standard deviation, were some 59 percent to 63 percent of a standard deviation over the time period, or better than two years of learning. Meanwhile, annual gains in the states with the weakest growth rates—Iowa, Maine, Oklahoma, and Wisconsin—varied between 0.7 percent and 1.0 percent of a standard deviation, which translate over the 19-year period into learning gains of one-half to three-quarters of a year. In other words, the states making the largest gains are improving at a rate two to three times the rate in states with the smallest gains.

Had all students throughout the United States made the same average gains as did those in the four leading states, the U.S. would have been making progress roughly comparable to the rate of improvement in Germany and the United Kingdom, bringing the United States reasonably close to the top-performing countries in the world.

Is the South Rising Again?

Some regional concentration is evident within the United States. Five of the top-10 states were in the South, while no southern states were among the 18 with the slowest growth. The strong showing of the South may be related to energetic political efforts to enhance school quality in that region. During the 1990s, governors of several southern states—Tennessee, North Carolina, Florida, Texas, and Arkansas—provided much of the national leadership for the school accountability effort, as there was a widespread sentiment in the wake of the civil rights movement that steps had to be taken to equalize educational opportunity across racial groups. The results of our study suggest those efforts were at least partially successful.

Meanwhile, students in Wisconsin, Michigan, Minnesota, and Indiana were among those making the fewest average gains between 1992 and 2011. Once again, the larger political climate may have affected the progress on the ground. Unlike in the South, the reform movement has made little headway within midwestern states, at least until very recently. Many of the midwestern states had proud education histories symbolized by internationally acclaimed land-grant universities, which have become the pride of East Lansing, Michigan; Madison, Wisconsin; St. Paul, Minnesota; and Lafayette, Indiana. Satisfaction with past accomplishments may have dampened interest in the school reform agenda sweeping through southern, border, and some western states.

Are Gains Simply Catch-ups?

According to a perspective we shall label “catch-up theory,” growth in student performance is easier for those political jurisdictions originally performing at a low level than for those originally performing at higher levels. Lower-performing systems may be able to copy existing approaches at lower cost than higher-performing systems can innovate. This would lead to a convergence in performance over time. An opposing perspective—which we shall label “building-on-strength theory”—posits that high-performing school systems find it relatively easy to build on their past achievements, while low-performing systems may struggle to acquire the human capital needed to improve. If that is generally the case, then the education gap among nations and among states should steadily widen over time.

Neither theory seems able to predict the international test-score changes that we have observed, as nations with rapid gains can be identified among countries that had high initial scores and countries that had low ones. Latvia, Chile, and Brazil, for example—were relatively low-ranking countries in 1995 that made rapid gains, a pattern that supports catch-up theory. But consistent with building-on-strength theory, a number of countries that have advanced relatively rapidly were already high-performing in 1995—Hong Kong and the United Kingdom, for example. Overall, there is no significant pattern between original performance and changes in performance across countries.

But if neither theory accounts for differences across countries, catch-up theory may help to explain variation among the U.S. states. The correlation between initial performance and rate of growth is a negative 0.58, which indicates that states with lower initial scores had larger gains. For example, students in Mississippi and Louisiana, originally among the lowest scoring, showed some of the most striking improvement.  Meanwhile, Iowa and Maine, two of the highest-performing entities in 1992, were among the laggards in subsequent years (see Figure 3). In other words, catch-up theory partially explains the pattern of change within the United States, probably because the barriers to the adoption of existing technologies are much lower within a single country than across national boundaries.

Catch-up theory nonetheless explains only about one-quarter of the total state variation in achievement growth. Notice in Figure 3 that some states are well below the line (e.g., Iowa and Maine) while others are well above  (e.g., Maryland and Massachusetts). Note also that Iowa, Maine, Wisconsin, and Nebraska rank well below that line. Closing the interstate gap does not happen automatically.

What about Spending Increases?

According to another popular theory, additional spending on education will yield gains in test scores. To see whether expenditure theory can account for the interstate variation, we plotted test-score gains against increments in spending between 1990 and 2009. As can be seen from the scattering of states into all parts of Figure 4, the data offer precious little support for the theory. Just about as many high-spending states showed relatively small gains as showed large ones. Maryland, Massachusetts, and New Jersey enjoyed substantial gains in student performance after committing substantial new fiscal resources. But other states with large spending increments—New York, Wyoming, and West Virginia, for example—had only marginal test-score gains to show for all that additional expenditure. And many states defied the theory by showing gains even when they did not commit much in the way of additional resources. It is true that on average, an additional $1000 in per-pupil spending is associated with an annual gain in achievement of one-tenth of 1 percent of a standard deviation. But that trivial amount is of no statistical or substantive significance. Overall, the 0.12 correlation between new expenditure and test-score gain is just barely positive.

Who Spends Incremental Funds Wisely?

Some states received more educational bang for their additional expenditure buck than others. To ascertain which states were receiving the most from their incremental dollars, we ranked states on a “points per added dollar” basis. Michigan, Indiana, Idaho, North Carolina, Colorado, and Florida made the most achievement gains for every incremental dollar spent over the past two decades. At the other end of the spectrum are the states that received little back in terms of improved test-score performance from increments in per-pupil expenditure—Maine, Wyoming, Iowa, New York, and Nebraska.

We do not know, however, which kinds of expenditures prove to be the most productive or whether there are other factors that could explain variation in productivity among the states.

Causes of Change

There is some hint that those parts of the United States that took school reform the most seriously—Florida and North Carolina, for example—have shown stronger rates of improvement, while states that have steadfastly resisted many school reforms (Iowa and Wisconsin, for instance), are among the nation’s test-score laggards. But the connection between reforms and gains adduced thus far is only anecdotal, not definitive. Although changes among states within the United States appear to be explained in part by catch-up theory, we cannot pinpoint the specific factors that underlie this. We are also unable to find significant evidence that increased school expenditure, by itself, makes much of a difference. Changes in test-score performance could be due to broader patterns of economic growth or varying rates of in-migration among states and countries. Of course, none of these propositions has been tested rigorously, so any conclusions regarding the sources of educational gains must remain speculative.

Have We Painted Too Rosy a Portrait?

Even the extent of the gains that have been made are uncertain. We have estimated gains of 1.6 percent of a standard deviation each year for the United States as a whole, or a total gain of 22 percent of a standard deviation over 14 years, a forward movement that has lifted performance by nearly a full year’s worth of learning over the entire time period. A similar rate of gain is estimated for students in the industrialized world as a whole (as measured by students residing in the 49 participating countries). Such a rate of improvement is plausible, given the increased wealth in the industrialized world and the higher percentages of educated parents than in prior generations.

However, it is possible to construct a gloomier picture of the rate of the actual progress that both the United States and the industrialized world as a whole have made. All estimations are normed against student performances on the National Assessment of Educational Progress in 4th and 8th grades in 2000.  Had we estimated gains from student performance in 8th grade only on the grounds that 4th-grade gains are meaningless unless they are observed for the same cohort four years later, our results would have shown annual gains in the United States of only 1 percent of a standard deviation. The relative ranking of the United States remains essentially unchanged, however, as the estimated growth rates for 8th graders in other countries is also lower than for estimates that include students in 4th grade (see the unabridged report, Appendix B, Figure B1).

A much reduced rate of progress for the United States emerges when we norm the trends on the PISA 2003 test rather than the 2000 NAEP test. In this case, we would have estimated annual growth rate for the United States of only one-half of 1 percent of a standard deviation. A lower annual growth rate for other countries would also have been estimated, and again the relative ranking of the United States would remain unchanged (see the unabridged report, Appendix B, Figure B2).

An even darker picture emerges if one turns to the results for U.S. students at age 17, for whom only minimal gains can be detected over the past two decades. We have not reported the results for 17-year-old students, because the test administered to them does not provide information on the performance of students within individual states, and no international comparisons are possible for this age group.

Students themselves and the United States as a whole benefit from improved performance in the early grades only if that translates into measurably higher skills at the end of school. The fact that none of the gains observed in earlier years translate into improved high-school performance leaves one to wonder whether high schools are effectively building on the gains achieved in earlier years. And while some scholars dismiss the results for 17-year-old students on the grounds that high-school students do not take the test seriously, others believe that the data indicate that the American high school has become a highly problematic educational institution. Amidst any uncertainties one fact remains clear, however: the measurable gains in achievement accomplished by more recent cohorts of students within the United States are being outstripped by gains made by students in about half of the other 48 participating countries.

Methodology

Our international results are based on 28 administrations of comparable math, science, and reading tests between 1995 and 2009 to juris­dictionally representative samples of students in 49 countries. Our state-by-state results come from 36 administrations of math, reading, and science tests between 1992 and 2011 to representative samples of students in 41 of the U.S. states. These tests are part of four ongoing series: 1) National Assessment of Educational Progress (NAEP), administered by the U. S. Department of Education; 2) Programme for International Student Assessment (PISA), administered by the Organisation for Economic Co-operation and Development (OECD); 3) Trends in International Mathematics and Science Study (TIMSS), adminis­tered by the International Associa­tion for the Evaluation of Educational Achievement (IEA); and 4) Progress in International Reading Literacy Study (PIRLS), also administered by IEA.

To equate the tests, we first express each testing cycle (of grade by subject) of the NAEP test in terms of standard deviations of the U.S. population on the 2000 wave. That is, we create a new scale benchmarked to U.S. performance in 2000, which is set to have a standard deviation of 100 and a mean of 500. All other NAEP results are a simple linear transformation of the NAEP scale on each testing cycle. Next, we express each international test on this trans­formed NAEP scale by performing a simple linear transformation of each international test based on the U.S. performance on the respective test. Specifically, we adjust both the mean and the standard deviation of each international test so that the U.S. performance on the tests is the same as the U.S. NAEP performance, as expressed on the transformed NAEP scale. This allows us to estimate trends on the international tests on a common scale, whose property is that in the year 2000 it has a mean of 500 and a standard deviation of 100 for the United States.

Expressed on this transformed scale, estimates of overall trends for each country are based on all avail­able data from all international tests administered between 1995 and 2009 for that country. Since a state or country may have specific strengths or weaknesses in certain subjects, at specific grade levels, or on particu­lar international testing series, our trend estimations use the following procedure to hold such differences constant. For each state and country, we regress the available test scores on a year variable, indicators for the international testing series (PISA, TIMSS, PIRLS), a grade indicator (4th vs. 8th grade), and subject indicators (mathematics, reading, science). This way, only the trends within each of these domains are used to estimate the overall time trend of the state or country, which is captured by the coef­ficient on the year variable.

A country’s performance on any given test cycle (for example, PIRLS 4th-grade reading, TIMSS 8th-grade math) is only considered if the country participated at least twice within that respective cycle. To be included in the analysis, the time span between a country’s first and last participation in any international test must be at least seven years. A country must have participated prior to 2003 and more recently than 2006. Finally, for a coun­try to be included there must be at least nine test observations available.

For the analysis of U.S. states, observations are available for only 41 states. The remaining states did not participate in NAEP tests until 2002. As mentioned, annual gains for states are calculated for a 19-year period (1992 to 2011), the longest interval that could be observed for the 41 states. International comparisons are for a 14-year period (1995 to 2009), the longest time span that could be observed with an adequate number of international tests. To facilitate a comparison between the United States as a whole and other nations, the aggregate U.S. trend is estimated from that same 14-year period and each U.S. test is weighted to take into account the specific years that international tests were administered. Because of the difference in length and because international tests are not administered in exactly the same years as the NAEP tests, the results for each state are not perfectly calibrated to the international tests, and each state appears to be doing slightly better internationally than would be the case if the calibration were exact. The differences are mar­ginal, however, and the comparative ranking of states is not affected by this discrepancy.

A more complete description of the methodology is available in the unabridged version of this report.

Politics and Results

The failure of the United States to close the international test-score gap, despite assiduous public assertions that every effort would be undertaken to produce that objective, raises questions about the nation’s overall reform strategy. Education goal setting in the United States has often been  utopian rather than realistic. In 1990, the president and the nation’s governors announced the goal that all American students should graduate from high school, but two decades later only 75 percent of 9th graders received their diploma within four years after entering high school. In 2002, Congress passed a law that declared that all students in all grades shall be proficient in math, reading, and science by 2014, but in 2012 most observers found that goal utterly beyond reach. Currently, the U.S. Department of Education has committed itself to ensuring that all students shall be college- or career-ready as they cross the stage on their high-school graduation day, another overly ambitious goal. Perhaps the least realistic goal was that of the governors in 1990 when they called for the U.S. to be first in the world in math and science by 2000. As this study shows, the United States is neither first nor catching up.

Consider a more realistic set of objectives for education policymakers, one that is based on experiences from within the United States itself. If all U.S. states could increase their performance at the same rate as the highest-growth states—Maryland, Florida, Delaware, and Massachusetts—the U.S. improvement rate would be lifted by 1.5 percentage points of a standard deviation annually above the current trend line. Since student performance can improve at that rate in some countries and in some states, then, in principle, such gains can be made more generally. Those gains might seem small but when viewed over two decades they accumulate to 30 percent of a standard deviation, enough to bring the United States within the range of, or to at least keep pace with, the world’s leaders.

Eric A. Hanushek is senior fellow at the Hoover Institution of Stanford University. Paul E. Peterson is director of the Harvard Program on Education Policy and Governance. Ludger Woessmann is head of the Department of Human Capital and Innovation at the Ifo Institute at the University of Munich. An unabridged version of this report is available at hks.harvard.edu/pepg/

This article appeared in the Fall 2012 issue of Education Next. Suggested citation format:

Hanushek, E.A., Peterson, P.E., and Woessmann, L. (2012). Is the U.S. Catching Up? International and state trends in student achievement. Education Next, 12(4), 24-33.

The post Is the U.S. Catching Up? appeared first on Education Next.

]]>
49699458
Are U.S. Students Ready to Compete? https://www.educationnext.org/are-u-s-students-ready-to-compete/ Wed, 17 Aug 2011 00:00:00 +0000 http://www.educationnext.org/are-u-s-students-ready-to-compete/ The latest on each state’s international standing

The post Are U.S. Students Ready to Compete? appeared first on Education Next.

]]>

An unabridged version of this paper is available here.

On August 17, 2011 Paul Peterson discussed the findings of this study in a free online webinar. An archived recording of this webinar can be found here.


At a time of persistent unemployment, especially among the less skilled, many wonder whether our schools are adequately preparing students for the 21st-century global economy. Despite high unemployment rates, firms are experiencing shortages of educated workers, outsourcing professional-level work to workers abroad, and competing for the limited number of employment visas set aside for highly skilled immigrants. As President Barack Obama said in his 2011 State of the Union address, “We know what it takes to compete for the jobs and industries of our time. We need to out-innovate, out-educate, and out-build the rest of the world.”

The challenge is particularly great in math, science, and engineering. According to Internet entrepreneur Vinton Cerf, “America simply is not producing enough of our own innovators, and the cause is twofold—a deteriorating K–12 education system and a national culture that does not emphasize the importance of education and the value of engineering and science.” To address the issue, the Science, Technology, Engineering, and Math (STEM) Education Coalition was formed in 2006 to “raise awareness in Congress, the Administration, and other organizations about the critical role that STEM education plays in enabling the U.S. to remain the economic and technological leader of the global marketplace.” Tales of shortages of educated talent appear regularly in the media. According to a CBS News report, 22 percent of American businesses say they are ready to hire if they can find people with the right skills. As one factory owner put it, “It’s hard to fill these jobs because they require people who are good at math, good with their hands, and willing to work on a factory floor.” According to a Bureau of Labor Statistics report, of the 30 occupations projected to grow the most rapidly over the next decade, nearly half are professional jobs that require at least a college degree. On the basis of these projections, McKinsey’s Global Institute estimates that over the next few years there will be a gap of nearly 2 million workers with the necessary analytical and technical skills.

In this paper we view the proficiency of U.S. students from a global perspective. Although we provide information on performances in both reading and mathematics, our emphasis is on student proficiency in mathematics, the subject many feel to be of greatest concern.

Student Proficiency on NAEP

At one time it was left to teachers and administrators to decide exactly what level of math proficiency should be expected of students. But, increasingly, states, and the federal government itself, have established proficiency levels that students are asked to reach. A national proficiency standard was set by the board that governs the National Assessment of Educational Progress (NAEP), which is administered by the U.S. Department of Education and generally known as the nation’s report card.

In 2007, just 32 percent of 8th graders in public and private schools in the United States performed at or above the NAEP proficiency standard in mathematics, and 31 percent performed at or above that level in reading. When more than two-thirds of students fail to reach a proficiency bar, it raises serious questions. Are U.S. schools failing to teach their students adequately? Or has NAEP set its proficiency bar at a level beyond the normal reach of a student in 8th grade?

One way of tackling such questions is to take an international perspective. Are other countries able to lift a higher percentage—or even a majority—of their students to or above the NAEP proficiency bar? Another approach is to look at differences among states. What percentage of students in each state is performing at a proficient level? How does each state compare to students in other countries?

In this article, we report results from our second study of student achievement in global perspective conducted for Harvard’s Program on Education Policy and Governance (PEPG). In our 2010 PEPG report, we compared the percentage of U.S. public and private school students in the high-school graduating Class of 2009 who were performing at the advanced level in mathematics with rates of similar performance among their peers around the world (see “Teaching Math to the Talented,” features, Winter 2011). The current study continues this work by reporting proficiency rates in both mathematics and reading for the most recent cohort for which data are available, the high-school graduating Class of 2011.

Comparing U.S. Students with Peers in Other Countries

If the NAEP exams are the nation’s report card, the world’s report card is assembled by the Organization for Economic Co-operation and Development (OECD), which administers the Program for International Student Assessment (PISA) to representative samples of 15-year-old students in 65 of the world’s school systems (which, to simplify the presentation, we shall refer to as countries; Hong Kong, Macao, and Shanghai are not independent nations but are nonetheless included in PISA reports). Since its launch in 2000, the PISA test has emerged as the yardstick by which countries measure changes in their performance over time and the level of their performance relative to that of other countries.

Since the United States participates in the PISA examinations, it is possible to make direct comparisons between the average performance of U.S. students and that of their peers elsewhere. But to compare the percentages of students deemed proficient in math or reading, one must ascertain the PISA equivalent of the NAEP standard of proficiency. To obtain that information, we perform a crosswalk between NAEP and PISA. The crosswalk is made possible by the fact that representative (but separate) samples of the high-school graduating Class of 2011 took the NAEP and PISA math and reading examinations. NAEP tests were taken in 2007 when the Class of 2011 was in 8th grade and PISA tested 15-year-olds in 2009, most of whom are members of the Class of 2011. Given that NAEP identified 32 percent of U.S. 8th-grade students as proficient in math, the PISA equivalent is estimated by calculating the minimum score reached by the top-performing 32 percent of U.S. students participating in the 2009 PISA test. (See methodological sidebar for further details.)


Methodological Approach
In the United States, in 2007, the share of 8th-grade students identified as proficient on the NAEP math examination was 32.192 percent. The minimum math score on the PISA examination obtained in 2009 by the highest-performing 32.192 percent of all U.S. students was estimated to be 530.7. To cover a broad content area while ensuring that testing time does not become excessive, the tests employ matrix sampling. No student takes the entire test, and scores are aggregated across students. Results are thus estimates of performance obtained by averaging five plausible values, as PISA and NAEP administrators recommend.

Comparable numbers for the other categories are as follows:

Reading proficiency: 31.223 percent of U.S. students are proficient on the NAEP, which corresponds to 550.4 on PISA.

Advanced math: 6.998 percent of U.S. students scored at the advanced level on the NAEP, which corresponds to 623.2 on PISA.

Advanced reading: 2.767 percent of U.S. students scored at the advanced level on the NAEP, which corresponds to 678.1 on PISA.

What It Means to Be Proficient

According to the National Center for Education Statistics (NCES), which administers NAEP, the determination of proficiency in any given subject at a particular grade level “was the result of a comprehensive national process [which took into account]…what hundreds of educators, curriculum experts, policymakers, and members of the general public thought the assessment should test. After the completion of the framework, the NAEP [subject] Committee worked with measurement specialists to create the assessment questions and scoring criteria.” In other words, NAEP’s concept of proficiency is not based on any objective criterion, but reflects a consensus on what should be known by students who have reached a certain educational stage. NAEP says that 8th graders, if proficient, “understand the connections between fractions, percents, decimals, and other mathematical topics such as algebra and functions.”

PISA does not set a proficiency standard. Instead, it sets different levels of performance, ranging from one (the lowest) to six (the highest). A student who is at the proficiency level in math set by NAEP performs moderately above proficiency  level three on the PISA. (See sidebar for a statement of the 8th-grade proficiency standard and sample questions from PISA and NAEP that proficient students are expected to pass.)

Crossing the Proficiency Bar

Given that definition of math proficiency, U.S. students in the Class of 2011, with a 32 percent proficiency rate, came in 32nd among the nations that participated in PISA. Performance levels among the countries ranked 23rd to 31st are not significantly different from that of the U.S. in a statistical sense, yet 22 countries do significantly outperform the United States in the share of students reaching the proficiency level in math. Six countries plus Shanghai and Hong Kong had majorities of students performing at least at the proficiency level, while the United States had less than one-third. For example, 58 percent of Korean students and 56 percent of Finnish students performed at or above a proficient level. Other countries in which a majority—or near majority—of students performed at or above the proficiency level included Switzerland, Japan, Canada, and the Netherlands. Many other nations also had math proficiency rates well above that of the United States, including Germany (45 percent), Australia (44 percent), and France (39 percent). Figure 1 presents a detailed listing of the scores of all participating countries as well as the performance of the individual states within the United States.

Shanghai topped the list with a 75 percent math proficiency rate, well over twice the 32 percent rate of the United States. However, Shanghai students are from a prosperous metropolitan area within China, so their performance is more appropriately compared to Massachusetts and Minnesota, which are similarly favored and are the top performers among the U.S. states. When this comparison is made, Shanghai still performs at a distinctly higher level. Only a little more than half (51 percent) of Massachusetts students are proficient in math, while Minnesota, the runner-up state, has a math proficiency rate of just 43 percent.

Only four additional states—Vermont, North Dakota, New Jersey, and Kansas—have a math proficiency rate above 40 percent. Some of the country’s largest and richest states score below the average for the United States as a whole, including New York (30 percent), Missouri (30 percent), Michigan (29 percent), Florida (27 percent), and California (24 percent).

Click image to enlarge

Proficiency in Reading

According to NAEP, students proficient in reading “should be able to make and support inferences about a text, connect parts of a text, and analyze text features.” According to PISA, students at level four, a level of performance set very close to the NAEP proficiency level, should be “capable of difficult reading tasks, such as locating embedded information, construing meaning from nuances of languages critically evaluating a text.” (See sidebar for more specific definitions and sample questions.)

ednext_20114_Peterson_side2-small

The U.S. proficiency rate in reading, at 31 percent, compares reasonably well to those of most European countries other than Finland. It takes 17th place among the nations of the world, and only the top 10 countries on PISA outperform the United States by a statistically significant amount. In Korea, 47 percent of the students are proficient in reading. Other countries that outrank the United States include Finland (46 percent), Singapore,  New Zealand, and Japan (42 percent), Canada (41 percent), Australia (38 percent), and Belgium (37 percent).

Within the United States, Massachusetts is again the leader, with 43 percent of 8th-grade students performing at the NAEP proficiency level in reading. Shanghai students perform at a higher level, however, with 56 percent of its young people proficient in reading. Within the United States, Vermont is a close second to its neighbor to the south, with 42 percent proficiency. New Jersey and Montana come next, both with 39 percent of the students identified as proficient in reading. The District of Columbia, the nation’s worst, are at the level achieved in Turkey and Bulgaria, while the one-eighth of our students living in California are similar to those in Slovakia and Spain. (See Figure 2 for the international ranking of all states.)

Click image to enlarge

Ethnic Groups

The percentage proficient in the United States varies considerably among students from different racial and ethnic backgrounds. While 42 percent of white students were identified as proficient in math, only 11 percent of African American students, 15 percent of Hispanic students, and 16 percent of Native Americans were so identified. Fifty percent of students with an ethnic background from Asia and the Pacific Islands, however, were proficient in math, placing them at a level comparable to students in Belgium, Canada, and Japan.

In reading, 40 percent of white students and 41 percent of those from Asia and the Pacific Islands were identified as proficient. Only 13 percent of African American students, 5 percent of Hispanic students, and 18 percent of Native American students were so identified.

Given the disparate performances among students from various cultural backgrounds, it may be worth inquiring as to whether differences between the United States and other countries are due to the presence of a substantial minority population within the United States. To examine that question, we compare U.S. white students to all students in other countries. We do this not because we think this is the right comparison, but simply to consider the oft-expressed claim that education problems in the United States are confined to certain segments within the minority community.

While the 42 percent math efficiency rate for U.S. white students is considerably higher than that of African American and Hispanic students, they are still surpassed by all students in 16 other countries. White students in the United States trail well behind all students in Korea, Japan, Finland, Germany, Belgium, and Canada.

White students in Massachusetts outperform their peers in other states; 58 percent are at or above the math proficiency level. Maryland, New Jersey, and Texas are the other states in which a majority of white students is proficient in math. Given recent school-related political conflicts in Wisconsin, it is of interest that only 42 percent of that state’s white students are proficient in math, a rate no better than the nation as a whole.  (Results for all states are presented in the unabridged version of the paper.)

In reading, the picture looks better. As we mentioned above, only 40 percent of white students are proficient, but that proficiency rate would place the United States at 9th in the world. Its proficiency rate does not differ significantly (in a statistical sense) from that for all students in Canada, Japan, and New Zealand, but white students trail in reading by a significant margin all students in Shanghai, Korea, Finland, Hong Kong, and Singapore. In no state is a majority of white students proficient, although Massachusetts comes close with a 49 percent rate. The four states with the next highest levels of reading proficiency among white students are New Jersey, Connecticut, Maryland, and Colorado.

Are the Proficiency Standards the Same for Math as for Reading?

Has NAEP set a lower proficiency standard in math than in reading? If so, is the math standard too low or the reading bar too high?

At first glance it would seem that the standard is set at pretty much the same level. After all, 32 percent of U.S. students are deemed proficient in math and 31 percent are deemed proficient in reading.

But that coincidence is quite misleading. When compared to peers abroad, the U.S. Class of 2011 performed respectably in reading, trailing only 10 other nations by a statistically significant amount. Admittedly, the U.S. trails Korea by 16 percentage points, but it’s only 10 percentage points behind Canada. Meanwhile, U.S. performance in math significantly trails that of 22 countries. Korean performance is 26 percentage points higher than that of the United States, while Canadian performance is 18 percentage points higher. Judged by international standards, U.S. 8th graders are clearly doing worse in math than in reading, despite the fact that NAEP reports similar percentages proficient in the two subjects.

A direct comparison of NAEP’s proficiency standard with PISA’s proficiency levels three and four also indicates that a lower NAEP bar has been set in math than in reading. To meet NAEP’s standards currently, one needs to perform near the fourth level on PISA’s reading exam, but only modestly above the third level on its math exam.

Clearly, the experts set an 8th-grade math proficiency standard at a level lower than the one set in reading. Perhaps this is an indication that American society as a whole, including the experts who design NAEP standards, set lower expectations for students in math than in reading. If so, it is a sign that low performance in mathematics within the United States may be deeply rooted in the nation’s culture. Those who are setting the common core standards under discussion might well take note of this.

Of course, it could be argued that the math proficiency standard is correct but the reading standard has been set too high. In no country in the world does a majority of the students reach the NAEP proficiency bar set in 8th-grade reading.

What Does It Mean?

Many have concluded that the productivity of the U.S. economy could be greatly enhanced if a higher percentage of U.S. students were proficient in mathematics. As Michael Brown, Nobel Prize winner in medicine, has declared, “If America is to maintain our high standard of living, we must continue to innovate…. Math and science are the engines of innovation. With these engines we can lead the world.”

But others have argued that the overall past success of the U.S. economy suggests that high-school math performance is not that critical for sustained growth in economic productivity. After all, U.S. students trailed their peers in the very first international survey undertaken nearly 50 years ago. That is the wrong message to take away however. Other factors contributed to the relatively high rate of growth in economic productivity during the last half of the 20th century, including the openness of the country’s markets, respect for property rights, low levels of political corruption, and limited intrusion of government into the operations of the marketplace. The United States, moreover, has always benefited from the in-migration of talent from abroad.

Furthermore, the United States has historically had far higher levels of educational attainment than other countries, with many more students graduating from high school, continuing on to college, and earning an advanced degree. It appears that in the past the country made up for low quality in elementary and high school by educating students for longer periods of time.

As we proceed into the 21st century, none of these factors remains as favorable to the United States. While other countries are lifting restrictions on market operations, the opposite has been occurring within the United States. The U.S. has also placed sharp limits on the numbers of talented workers that can be legally admitted into the country. Our higher education system, though still perceived to be the best in the world, is recruiting an ever-increasing proportion of its faculty and students from outside the country. Meanwhile, educational attainment rates among U.S. citizens now trail the industrial-world average.

Even if some of these trends can be reversed, that hardly gainsays the desirability of enhancing the mathematical skills of the U.S. student population, especially at a time when the nation’s growth in productivity is badly trailing growth rates in China, India, Brazil, and many smaller Asian countries. Eric Hanushek and Ludger Woessmann have shown elsewhere that student performance on international tests such as those we consider here is closely related to long-term economic growth (see “Education and Economic Growth,” research, Spring 2008). Assuming past economic patterns continue, the country could enjoy a remarkable increment in its annual GDP growth per capita by enhancing the math proficiency of U.S. students. Increasing the percentage of proficient students to the levels attained in Canada and Korea would increase the annual U.S. growth rate by 0.9 percentage points and 1.3 percentage points, respectively. Since current average annual growth rates hover between 2 and 3 percentage points, that increment would lift growth rates by between 30 and 50 percent.

When translated into dollar terms, these magnitudes become staggering. If one calculates these percentage increases as national income projections over an 80-year period (providing for a 20-year delay before any school reform is completed and the newly proficient students begin their working careers), a back-of-the-envelope calculation suggests gains of nothing less than $75 trillion over the period. That averages out to around a trillion dollars a year. Even if you tweak these numbers a bit in one direction or another to account for various uncertainties, you reach the same bottom line: Those who say that student math performance does not matter are clearly wrong.

Given the integration of the world economy, a global perspective is needed for assessing the performance of U.S. schools, districts, and states. High-school graduates in each and every state compete for jobs with graduates from all over the world. Charles Vest, president of the National Academy of Engineering and president emeritus at Massachusetts Institute of Technology, has warned, “America faces many challenges…but the enemy I fear most is complacency. We are about to be hit by the full force of global competition. If we continue to ignore the obvious task at hand while others beat us at our own game, our children and grandchildren will pay the price. We must now establish a sense of urgency.”

Paul E. Peterson is the director of Harvard’s Program on Education Policy and Governance and senior fellow at the Hoover Institution. Ludger Woessmann is professor of economics at the University of Munich. Eric A. Hanushek is senior fellow at the Hoover Institution of Stanford University. Carlos X. Lastra-Anadón is a research fellow at the Program on Education Policy and Governance at Harvard University. An unabridged version of this paper is available here.

This article appeared in the Fall 2011 issue of Education Next. Suggested citation format:

Peterson, P.E., Woessmann, L., Hanushek, E.A., and Lastra-Anadón, C.X. (2011). Are U.S. Students Ready to Compete? The latest on each state’s international standing. Education Next, 11(4), 51-59.

The post Are U.S. Students Ready to Compete? appeared first on Education Next.

]]>
49698711
Merit Pay International https://www.educationnext.org/merit-pay-international/ Tue, 08 Feb 2011 00:00:00 +0000 http://www.educationnext.org/merit-pay-international/ Countries with performance pay for teachers score higher on PISA tests

The post Merit Pay International appeared first on Education Next.

]]>

An unabridged version of this article is available here.


American 15-year-olds continue to perform no better than at the industrial-world average in reading and science, and below that in mathematics. According to the results of the 2009 Program for International Student Assessment (PISA) tests, released in December 2010 by the Organisation for Economic Co-operation and Development (OECD), the United States performed only at the international average in reading, and trailed 18 and 23 other countries in science and math, respectively. Students in China’s Shanghai province outscored everyone.

Many have identified variations in teacher quality as a key factor in international differences in student performance and have urged policies that will lift the quality of the U.S. teaching force. To that end, President Barack Obama has called for a national effort to improve the quality of classroom teaching and repeatedly indicated his support for policies that would provide financial rewards for outstanding teachers.

In a March 2009 speech to the Hispanic Chamber of Commerce, he explained,

Good teachers will be rewarded with more money for improved student achievement, and asked to accept more responsibilities for lifting up their schools. Teachers throughout a school will benefit from guidance and support to help them improve.

In the administration’s Race to the Top initiative, the U.S. Department of Education encouraged states to devise performance pay plans for teachers in the hope that such an intervention could have a significant impact on student performance.

But is there anything in the data the OECD has accumulated to give policymakers reason to believe that merit pay works? Do the countries that pay teachers based on their performance score higher on PISA tests? Based on my new analysis, the answer is yes. A little-used survey conducted by the OECD in 2005 makes it possible to identify the developed countries participating in PISA that appear to have some kind of performance pay plan. Linking that information to a country’s test performance, one finds that students in countries with performance pay perform at higher levels in math, science, and reading. Specifically, students in countries that permit teacher salaries to be adjusted for outstanding performance score approximately one-quarter of a standard deviation higher on the international math and reading tests, and about 15 percent higher on the science test, than students in countries without performance pay. These findings are obtained after adjustments for levels of economic development across countries, student background characteristics, and features of national school systems.

I draw these conclusions cautiously, as my study is based on information on students in just 27 countries, and the available information on the extent of performance pay in a country is far from perfect. Further, the analysis is based on what researchers refer to as observational rather than experimental data, making it more difficult to make confident statements regarding causality.

It is possible that what I have observed is the opposite of what it seems: countries with high student achievement may find it easier to persuade teachers to accept pay for performance, thereby making it appear that merit pay is lifting achievement. More generally, both performance pay and higher levels of achievement could be produced by some set of factors other than all of those taken into account in the analysis. For example, performance pay could be more widely used in places where, as in Asia, cultural expectations for student performance are high, making it appear that performance pay systems are effective, when in fact both performance pay plans and student achievement are the result of underlying cultural characteristics. But even if my findings are not indisputable, I did carry out a variety of checks to see if any observable factor, such as Asian-European differences, could account for the conclusion. Thus far, I have been unable to find any convincing evidence that the findings are incorrect. Given that, let us take a closer look at what can be learned about the impact of performance pay from PISA data.

Prior Research

Standard economic theory predicts that workers will exert more effort when monetary rewards are tied to the amount of the product they produce. Not only does performance pay stimulate individual effort on the job, it is theorized, but jobs where rewards are tied to effort attract energetic, risk-taking employees who are likely to be more productive. This latter consideration, says Stanford economist Edward Lazear, “is perhaps the most important” way in which a merit pay plan can influence worker performance. But if economists expect positive results from merit pay, many educators believe that teachers are motivated primarily by the substantive mission of the teaching profession and that they do not respond to—indeed, they may resent and resist—monetary incentives that tie salary levels to performance indicators.

To see whether the education sector is an exception to general economic theory, a number of performance pay experiments have been carried out, and in Israel and India such studies have shown positive impacts on student achievement. Experimental studies have tracked only the short-term impact of merit pay, however, and so have not identified any long-term effects that might come from changes in the kinds of people who choose to go into this line of work. Conceivably, a merit pay system could discourage entry into the profession of potentially excellent teachers reluctant to subject themselves to the requirements of a pay-for-performance scheme. Alternatively, if performance pay makes teaching more attractive to talented workers, short-term evaluations could understate its benefits.

One way to capture the long-term effects of teacher performance pay, including changes in the characteristics of those choosing to become a teacher, is to compare countries with performance pay systems to those without. This is now possible because the OECD in 2005 administered a separate survey to each of its member countries concerning the teacher compensation systems in place during the 2003–04 school year, when the 2003 PISA study was conducted. The 2003 PISA provides test score results in math, reading, and science for representative samples of 15-year-olds within each country, or nearly 200,000 students altogether. (The relative performance of countries on the PISA changed only slightly between the 2003 and 2009 tests. For example, in no subject did the scores for the United States differ significantly between 2003 and 2009.)

The PISA study is particularly useful, because it also includes information on a wide variety of family, school, and institutional factors that are likely determinants of student achievement. My analysis adjusts, at the level of the individual student, for such characteristics as the student’s gender and age, preprimary education, immigration status, household composition, parent occupation, and parent employment status. Nine measures of school resources and location are available, including class size, availability of materials, instruction time, teacher education, and size of community. Country-level variables included in the analysis were per capita GDP, teacher salary levels, average expenditure per student, external exit exams, school autonomy in budget and staffing decisions, the share of privately operated schools, and the portion of government funding for schools.

The PISA sampling procedure ensured that a representative sample of 15-year-old students was tested in each country. The student sample sizes in the OECD countries range from 3,350 students in 129 schools in Iceland to 29,983 students in 1,124 schools in Mexico. I therefore use weights when conducting my analysis so that each country contributes equally to the estimated effect of performance pay on student achievement.

Measuring Teacher Performance Pay

The measure of performance pay available from the OECD survey is less precise than one would prefer. It simply asks officials in participating countries whether the base salary for public-school teachers could be adjusted to reward teachers who had an “outstanding performance in teaching.” While the survey asked about many other forms of salary adjustments, the study protocol reports that this was the only one that “could be classified as a performance incentive.”Among the 27 OECD countries for which the necessary PISA data are also available, 12 countries reported having adjustments of teacher salaries based on outstanding performance in teaching. The form of the monetary incentive and the method for identifying outstanding performance varies across countries. For example, in Finland, according to the national labor agreement for teachers, local authorities and education providers have an opportunity to encourage individual teachers in their work by personal cash bonuses on the basis of professional proficiency and performance at work. Outstanding performance may also be measured based on the assessment of the head teacher (Portugal), assessments performed by education administrators (Turkey), or the measured learning achievements of students (Mexico). Unfortunately, the coding of the measure does not allow my analysis to consider variation in the scope, structure, and incentives of performance-related pay schemes.

As an example of the limitation of this measure, note that the United States is coded as a country where teacher salaries can be adjusted for outstanding performance in teaching on the grounds that salary adjustments are possible for achieving the National Board for Professional Teaching Standards certification or for increases in student achievement test scores. That policy, however, affects only a few teachers in selected parts of the country. Given such weaknesses in the survey measure, it is all the more remarkable that I was able to detect impacts on student achievement.

Main Results

As noted above, my main analysis indicates that student achievement is significantly higher in countries that make use of teacher performance pay than in countries that do not use it. On average, students in countries with performance-related pay score 24.8 percent of a standard deviation higher on the PISA math test; in reading the effect is 24.3 percent of a standard deviation; and in science it is 15.4 percent (see Figure 1). These effects are similar to the impact identified in the experimental study conducted in India and about twice as large as the one found in a similarly designed Israeli study.

Figure 2 depicts the math result graphically. The figure’s vertical axis displays the average math test scores of students in each country after adjusting for all of the control variables in the model, with the exception of the variable measuring the use of performance pay. The horizontal axis in turn shows the performance pay variable, also after adjusting for those same control variables. The solid line on the figure shows the estimated relationship between these two variables across the 27 countries included in the analysis. It shows a clear positive association between the variation in country-average test scores and the variation in teacher performance pay that cannot be attributed to the other factors included in the analysis.

A lingering concern, however, is that the analysis may be contaminated by the fact that the very cultures that introduce merit pay are those that set high expectations for student achievement. The countries represent widely different cultures, including Asian ones, where expectations for students are often much higher than in Europe and North America. The best way to account for cultural differences among the continents of the world is to control in the analysis for the average effect of living on a particular continent, a strategy known to statisticians as continental fixed effects. Figure 1 thus also shows results based on models that include a fixed effect for each of the four continents with OECD countries: Europe, North America, Oceania, and East Asia. In these models, the effects of pay for performance are shown to be even larger than the results based on comparisons across continents. In other words, the findings cannot be attributed to cultural differences among the major regions of the world, because they are even larger when one looks only at patterns within these regions.

As a further test, I estimated the impact of performance pay for only the 21 participating European countries. Once again, the results showed even larger positive effects than those obtained for the full sample.

Other Sensitivity Tests

When findings are based on small samples, it is important to ascertain whether a conclusion is sensitive to the particular analysis being conducted. Even after conducting a preferred analysis that maximizes use of the information available and best conforms to underlying economic theory, it is important to make sure that the pattern that one has identified is not a statistical accident that readily disappears if a slightly different analysis is conducted. For this reason, I performed a variety of sensitivity tests for math achievement because the reliability of the math test across countries and cultures is usually considered higher than it is for reading or science. Remarkably, the relationship between performance pay and math achievement remained essentially unchanged, regardless of the sensitivity test that I ran.

My first sensitivity check focused on cultural differences among countries that were not captured by the continental fixed effects analysis. In this sensitivity check, I excluded two countries, Mexico and Turkey, which have particularly low levels of GDP per capita. Since it is known that the level of GDP is strongly correlated with educational performance, it may be that the inclusion of these two countries is producing misleading results. But dropping these countries hardly affects results.

The second sensitivity test excluded the level of educational attainment of the teachers, on the grounds that teacher quality might itself be affected by a country’s performance pay policies and therefore should not be used as a control variable. Excluding this variable did not materially change the results from those reported in Figure 1. In a third series of sensitivity tests, I excluded from the analysis one country at a time to make sure that the situation in no one country was driving the overall pattern of results. I found no evidence that that was happening.

The incidence of performance pay is, to some extent, clustered in two regions: Scandinavia (Denmark, Finland, Norway, and Sweden) and Eastern Europe (Czech Republic and Hungary). In a fourth set of sensitivity tests, I separately excluded the countries from these two regions from the analysis to see whether results were highly dependent on one or the other cluster. The results remained unchanged, indicating that neither of these regional clusters is solely responsible for the main result.

A fifth set of sensitivity tests was possible because I have information on other policies that lead to differential pay among teachers. Salaries may vary depending on 1) the teaching conditions and responsibilities (such as taking on management responsibilities, teaching additional classes, and teaching in particular areas or subjects), 2) teacher qualifications and training, or 3) a teacher’s family status and/or age. Since it is possible that student achievement is higher whenever pay schedules are flexible, regardless of the connection to teacher classroom effectiveness, I estimated the impact of each of these three sets of factors on math achievement. None showed a significant impact on performance, and the effect of performance pay remained large and significant, even when these other possible salary adjustments were included in the analysis.

In sum, the main results shown in Figure 1 survive a wide variety of sensitivity tests. That the results are robust to multiple model specifications provides strong evidence that performance pay helps to explain the variation in student performance on the PISA tests.

Differential Effects

With one exception (immigrants benefited less than native-born students from a performance pay regime), I found only small differences in the impact of performance pay on the math achievement of subgroups in the population. Since important differential effects were identified for only one subgroup, one cannot infer that the impact of performance pay on student math learning is concentrated on any particular group of students.

I did, however, find a surprising difference in the way in which a teacher’s education background affects math learning, depending on the presence of a pay-for-performance system. In countries with performance pay, teachers who have an advanced degree in pedagogy do not outperform those without such a degree (the only measure of a teacher’s education available in the PISA data base). However, in countries without performance pay, students learn more in math if they have a pedagogically trained teacher. Perhaps an incentive system washes out any differences that may be caused by variations in teacher training.

Conclusions

The analysis presented above represents the first evidence that, all other observable things equal, students in countries with teacher performance pay plans perform at a higher level in math, reading, and science. The differences in performance are large, ranging from 15 percent (in science) to 25 percent (in math and reading) of a standard deviation. Since one-quarter of a standard deviation is roughly a year’s worth of learning, it might reasonably be concluded that by the age of 15, students taught under a policy regime that includes a performance pay plan will learn an additional year of math and reading and over half a year more in science. However, this conclusion depends on the many assumptions underlying an analysis based on observational data.

Although these are impressive results, before drawing strong policy conclusions it is important to confirm the results through experimental or quasi-experimental studies carried out in advanced industrialized countries. Nothing in the PISA data allows us to identify crucial aspects of performance pay schemes, such as the way in which teacher performance is measured, the size of the incremental earnings received by higher-performing teachers, or very much about the level of government at which or the manner in which decisions on merit pay are made. Studies of such matters are probably better performed within countries, taking advantage of variation in policies within those countries. The study design also does not allow one to tease out the relative importance of the incentive to existing teachers of a performance pay plan as compared to the changes that may take place in teacher recruitment when compensation depends in part on merit rather than just on a standardized pay schedule. Since much more work needs to be done on all of these questions, a wit might insist that performance pay apply to scholars as well.

Ludger Woessmann is professor of economics at the University of Munich and head of the department of Human Capital and Innovation at the Ifo Institute for Economic Research.

An unabridged version of this article is available here.

The post Merit Pay International appeared first on Education Next.

]]>
49698106