An unabridged version of this paper is available here.
On August 17, 2011 Paul Peterson discussed the findings of this study in a free online webinar. An archived recording of this webinar can be found here.
At a time of persistent unemployment, especially among the less skilled, many wonder whether our schools are adequately preparing students for the 21st-century global economy. Despite high unemployment rates, firms are experiencing shortages of educated workers, outsourcing professional-level work to workers abroad, and competing for the limited number of employment visas set aside for highly skilled immigrants. As President Barack Obama said in his 2011 State of the Union address, “We know what it takes to compete for the jobs and industries of our time. We need to out-innovate, out-educate, and out-build the rest of the world.”
The challenge is particularly great in math, science, and engineering. According to Internet entrepreneur Vinton Cerf, “America simply is not producing enough of our own innovators, and the cause is twofold—a deteriorating K–12 education system and a national culture that does not emphasize the importance of education and the value of engineering and science.” To address the issue, the Science, Technology, Engineering, and Math (STEM) Education Coalition was formed in 2006 to “raise awareness in Congress, the Administration, and other organizations about the critical role that STEM education plays in enabling the U.S. to remain the economic and technological leader of the global marketplace.” Tales of shortages of educated talent appear regularly in the media. According to a CBS News report, 22 percent of American businesses say they are ready to hire if they can find people with the right skills. As one factory owner put it, “It’s hard to fill these jobs because they require people who are good at math, good with their hands, and willing to work on a factory floor.” According to a Bureau of Labor Statistics report, of the 30 occupations projected to grow the most rapidly over the next decade, nearly half are professional jobs that require at least a college degree. On the basis of these projections, McKinsey’s Global Institute estimates that over the next few years there will be a gap of nearly 2 million workers with the necessary analytical and technical skills.
In this paper we view the proficiency of U.S. students from a global perspective. Although we provide information on performances in both reading and mathematics, our emphasis is on student proficiency in mathematics, the subject many feel to be of greatest concern.
Student Proficiency on NAEP
At one time it was left to teachers and administrators to decide exactly what level of math proficiency should be expected of students. But, increasingly, states, and the federal government itself, have established proficiency levels that students are asked to reach. A national proficiency standard was set by the board that governs the National Assessment of Educational Progress (NAEP), which is administered by the U.S. Department of Education and generally known as the nation’s report card.
In 2007, just 32 percent of 8th graders in public and private schools in the United States performed at or above the NAEP proficiency standard in mathematics, and 31 percent performed at or above that level in reading. When more than two-thirds of students fail to reach a proficiency bar, it raises serious questions. Are U.S. schools failing to teach their students adequately? Or has NAEP set its proficiency bar at a level beyond the normal reach of a student in 8th grade?
One way of tackling such questions is to take an international perspective. Are other countries able to lift a higher percentage—or even a majority—of their students to or above the NAEP proficiency bar? Another approach is to look at differences among states. What percentage of students in each state is performing at a proficient level? How does each state compare to students in other countries?
In this article, we report results from our second study of student achievement in global perspective conducted for Harvard’s Program on Education Policy and Governance (PEPG). In our 2010 PEPG report, we compared the percentage of U.S. public and private school students in the high-school graduating Class of 2009 who were performing at the advanced level in mathematics with rates of similar performance among their peers around the world (see “Teaching Math to the Talented,” features, Winter 2011). The current study continues this work by reporting proficiency rates in both mathematics and reading for the most recent cohort for which data are available, the high-school graduating Class of 2011.
Comparing U.S. Students with Peers in Other Countries
If the NAEP exams are the nation’s report card, the world’s report card is assembled by the Organization for Economic Co-operation and Development (OECD), which administers the Program for International Student Assessment (PISA) to representative samples of 15-year-old students in 65 of the world’s school systems (which, to simplify the presentation, we shall refer to as countries; Hong Kong, Macao, and Shanghai are not independent nations but are nonetheless included in PISA reports). Since its launch in 2000, the PISA test has emerged as the yardstick by which countries measure changes in their performance over time and the level of their performance relative to that of other countries.
Since the United States participates in the PISA examinations, it is possible to make direct comparisons between the average performance of U.S. students and that of their peers elsewhere. But to compare the percentages of students deemed proficient in math or reading, one must ascertain the PISA equivalent of the NAEP standard of proficiency. To obtain that information, we perform a crosswalk between NAEP and PISA. The crosswalk is made possible by the fact that representative (but separate) samples of the high-school graduating Class of 2011 took the NAEP and PISA math and reading examinations. NAEP tests were taken in 2007 when the Class of 2011 was in 8th grade and PISA tested 15-year-olds in 2009, most of whom are members of the Class of 2011. Given that NAEP identified 32 percent of U.S. 8th-grade students as proficient in math, the PISA equivalent is estimated by calculating the minimum score reached by the top-performing 32 percent of U.S. students participating in the 2009 PISA test. (See methodological sidebar for further details.)
Methodological Approach
In the United States, in 2007, the share of 8th-grade students identified as proficient on the NAEP math examination was 32.192 percent. The minimum math score on the PISA examination obtained in 2009 by the highest-performing 32.192 percent of all U.S. students was estimated to be 530.7. To cover a broad content area while ensuring that testing time does not become excessive, the tests employ matrix sampling. No student takes the entire test, and scores are aggregated across students. Results are thus estimates of performance obtained by averaging five plausible values, as PISA and NAEP administrators recommend.
Comparable numbers for the other categories are as follows:
Reading proficiency: 31.223 percent of U.S. students are proficient on the NAEP, which corresponds to 550.4 on PISA.
Advanced math: 6.998 percent of U.S. students scored at the advanced level on the NAEP, which corresponds to 623.2 on PISA.
Advanced reading: 2.767 percent of U.S. students scored at the advanced level on the NAEP, which corresponds to 678.1 on PISA.
What It Means to Be Proficient
According to the National Center for Education Statistics (NCES), which administers NAEP, the determination of proficiency in any given subject at a particular grade level “was the result of a comprehensive national process [which took into account]…what hundreds of educators, curriculum experts, policymakers, and members of the general public thought the assessment should test. After the completion of the framework, the NAEP [subject] Committee worked with measurement specialists to create the assessment questions and scoring criteria.” In other words, NAEP’s concept of proficiency is not based on any objective criterion, but reflects a consensus on what should be known by students who have reached a certain educational stage. NAEP says that 8th graders, if proficient, “understand the connections between fractions, percents, decimals, and other mathematical topics such as algebra and functions.”
PISA does not set a proficiency standard. Instead, it sets different levels of performance, ranging from one (the lowest) to six (the highest). A student who is at the proficiency level in math set by NAEP performs moderately above proficiency level three on the PISA. (See sidebar for a statement of the 8th-grade proficiency standard and sample questions from PISA and NAEP that proficient students are expected to pass.)
Crossing the Proficiency Bar
Given that definition of math proficiency, U.S. students in the Class of 2011, with a 32 percent proficiency rate, came in 32nd among the nations that participated in PISA. Performance levels among the countries ranked 23rd to 31st are not significantly different from that of the U.S. in a statistical sense, yet 22 countries do significantly outperform the United States in the share of students reaching the proficiency level in math. Six countries plus Shanghai and Hong Kong had majorities of students performing at least at the proficiency level, while the United States had less than one-third. For example, 58 percent of Korean students and 56 percent of Finnish students performed at or above a proficient level. Other countries in which a majority—or near majority—of students performed at or above the proficiency level included Switzerland, Japan, Canada, and the Netherlands. Many other nations also had math proficiency rates well above that of the United States, including Germany (45 percent), Australia (44 percent), and France (39 percent). Figure 1 presents a detailed listing of the scores of all participating countries as well as the performance of the individual states within the United States.
Shanghai topped the list with a 75 percent math proficiency rate, well over twice the 32 percent rate of the United States. However, Shanghai students are from a prosperous metropolitan area within China, so their performance is more appropriately compared to Massachusetts and Minnesota, which are similarly favored and are the top performers among the U.S. states. When this comparison is made, Shanghai still performs at a distinctly higher level. Only a little more than half (51 percent) of Massachusetts students are proficient in math, while Minnesota, the runner-up state, has a math proficiency rate of just 43 percent.
Only four additional states—Vermont, North Dakota, New Jersey, and Kansas—have a math proficiency rate above 40 percent. Some of the country’s largest and richest states score below the average for the United States as a whole, including New York (30 percent), Missouri (30 percent), Michigan (29 percent), Florida (27 percent), and California (24 percent).
Proficiency in Reading
According to NAEP, students proficient in reading “should be able to make and support inferences about a text, connect parts of a text, and analyze text features.” According to PISA, students at level four, a level of performance set very close to the NAEP proficiency level, should be “capable of difficult reading tasks, such as locating embedded information, construing meaning from nuances of languages critically evaluating a text.” (See sidebar for more specific definitions and sample questions.)
The U.S. proficiency rate in reading, at 31 percent, compares reasonably well to those of most European countries other than Finland. It takes 17th place among the nations of the world, and only the top 10 countries on PISA outperform the United States by a statistically significant amount. In Korea, 47 percent of the students are proficient in reading. Other countries that outrank the United States include Finland (46 percent), Singapore, New Zealand, and Japan (42 percent), Canada (41 percent), Australia (38 percent), and Belgium (37 percent).
Within the United States, Massachusetts is again the leader, with 43 percent of 8th-grade students performing at the NAEP proficiency level in reading. Shanghai students perform at a higher level, however, with 56 percent of its young people proficient in reading. Within the United States, Vermont is a close second to its neighbor to the south, with 42 percent proficiency. New Jersey and Montana come next, both with 39 percent of the students identified as proficient in reading. The District of Columbia, the nation’s worst, are at the level achieved in Turkey and Bulgaria, while the one-eighth of our students living in California are similar to those in Slovakia and Spain. (See Figure 2 for the international ranking of all states.)
Ethnic Groups
The percentage proficient in the United States varies considerably among students from different racial and ethnic backgrounds. While 42 percent of white students were identified as proficient in math, only 11 percent of African American students, 15 percent of Hispanic students, and 16 percent of Native Americans were so identified. Fifty percent of students with an ethnic background from Asia and the Pacific Islands, however, were proficient in math, placing them at a level comparable to students in Belgium, Canada, and Japan.
In reading, 40 percent of white students and 41 percent of those from Asia and the Pacific Islands were identified as proficient. Only 13 percent of African American students, 5 percent of Hispanic students, and 18 percent of Native American students were so identified.
Given the disparate performances among students from various cultural backgrounds, it may be worth inquiring as to whether differences between the United States and other countries are due to the presence of a substantial minority population within the United States. To examine that question, we compare U.S. white students to all students in other countries. We do this not because we think this is the right comparison, but simply to consider the oft-expressed claim that education problems in the United States are confined to certain segments within the minority community.
While the 42 percent math efficiency rate for U.S. white students is considerably higher than that of African American and Hispanic students, they are still surpassed by all students in 16 other countries. White students in the United States trail well behind all students in Korea, Japan, Finland, Germany, Belgium, and Canada.
White students in Massachusetts outperform their peers in other states; 58 percent are at or above the math proficiency level. Maryland, New Jersey, and Texas are the other states in which a majority of white students is proficient in math. Given recent school-related political conflicts in Wisconsin, it is of interest that only 42 percent of that state’s white students are proficient in math, a rate no better than the nation as a whole. (Results for all states are presented in the unabridged version of the paper.)
In reading, the picture looks better. As we mentioned above, only 40 percent of white students are proficient, but that proficiency rate would place the United States at 9th in the world. Its proficiency rate does not differ significantly (in a statistical sense) from that for all students in Canada, Japan, and New Zealand, but white students trail in reading by a significant margin all students in Shanghai, Korea, Finland, Hong Kong, and Singapore. In no state is a majority of white students proficient, although Massachusetts comes close with a 49 percent rate. The four states with the next highest levels of reading proficiency among white students are New Jersey, Connecticut, Maryland, and Colorado.
Are the Proficiency Standards the Same for Math as for Reading?
Has NAEP set a lower proficiency standard in math than in reading? If so, is the math standard too low or the reading bar too high?
At first glance it would seem that the standard is set at pretty much the same level. After all, 32 percent of U.S. students are deemed proficient in math and 31 percent are deemed proficient in reading.
But that coincidence is quite misleading. When compared to peers abroad, the U.S. Class of 2011 performed respectably in reading, trailing only 10 other nations by a statistically significant amount. Admittedly, the U.S. trails Korea by 16 percentage points, but it’s only 10 percentage points behind Canada. Meanwhile, U.S. performance in math significantly trails that of 22 countries. Korean performance is 26 percentage points higher than that of the United States, while Canadian performance is 18 percentage points higher. Judged by international standards, U.S. 8th graders are clearly doing worse in math than in reading, despite the fact that NAEP reports similar percentages proficient in the two subjects.
A direct comparison of NAEP’s proficiency standard with PISA’s proficiency levels three and four also indicates that a lower NAEP bar has been set in math than in reading. To meet NAEP’s standards currently, one needs to perform near the fourth level on PISA’s reading exam, but only modestly above the third level on its math exam.
Clearly, the experts set an 8th-grade math proficiency standard at a level lower than the one set in reading. Perhaps this is an indication that American society as a whole, including the experts who design NAEP standards, set lower expectations for students in math than in reading. If so, it is a sign that low performance in mathematics within the United States may be deeply rooted in the nation’s culture. Those who are setting the common core standards under discussion might well take note of this.
Of course, it could be argued that the math proficiency standard is correct but the reading standard has been set too high. In no country in the world does a majority of the students reach the NAEP proficiency bar set in 8th-grade reading.
What Does It Mean?
Many have concluded that the productivity of the U.S. economy could be greatly enhanced if a higher percentage of U.S. students were proficient in mathematics. As Michael Brown, Nobel Prize winner in medicine, has declared, “If America is to maintain our high standard of living, we must continue to innovate…. Math and science are the engines of innovation. With these engines we can lead the world.”
But others have argued that the overall past success of the U.S. economy suggests that high-school math performance is not that critical for sustained growth in economic productivity. After all, U.S. students trailed their peers in the very first international survey undertaken nearly 50 years ago. That is the wrong message to take away however. Other factors contributed to the relatively high rate of growth in economic productivity during the last half of the 20th century, including the openness of the country’s markets, respect for property rights, low levels of political corruption, and limited intrusion of government into the operations of the marketplace. The United States, moreover, has always benefited from the in-migration of talent from abroad.
Furthermore, the United States has historically had far higher levels of educational attainment than other countries, with many more students graduating from high school, continuing on to college, and earning an advanced degree. It appears that in the past the country made up for low quality in elementary and high school by educating students for longer periods of time.
As we proceed into the 21st century, none of these factors remains as favorable to the United States. While other countries are lifting restrictions on market operations, the opposite has been occurring within the United States. The U.S. has also placed sharp limits on the numbers of talented workers that can be legally admitted into the country. Our higher education system, though still perceived to be the best in the world, is recruiting an ever-increasing proportion of its faculty and students from outside the country. Meanwhile, educational attainment rates among U.S. citizens now trail the industrial-world average.
Even if some of these trends can be reversed, that hardly gainsays the desirability of enhancing the mathematical skills of the U.S. student population, especially at a time when the nation’s growth in productivity is badly trailing growth rates in China, India, Brazil, and many smaller Asian countries. Eric Hanushek and Ludger Woessmann have shown elsewhere that student performance on international tests such as those we consider here is closely related to long-term economic growth (see “Education and Economic Growth,” research, Spring 2008). Assuming past economic patterns continue, the country could enjoy a remarkable increment in its annual GDP growth per capita by enhancing the math proficiency of U.S. students. Increasing the percentage of proficient students to the levels attained in Canada and Korea would increase the annual U.S. growth rate by 0.9 percentage points and 1.3 percentage points, respectively. Since current average annual growth rates hover between 2 and 3 percentage points, that increment would lift growth rates by between 30 and 50 percent.
When translated into dollar terms, these magnitudes become staggering. If one calculates these percentage increases as national income projections over an 80-year period (providing for a 20-year delay before any school reform is completed and the newly proficient students begin their working careers), a back-of-the-envelope calculation suggests gains of nothing less than $75 trillion over the period. That averages out to around a trillion dollars a year. Even if you tweak these numbers a bit in one direction or another to account for various uncertainties, you reach the same bottom line: Those who say that student math performance does not matter are clearly wrong.
Given the integration of the world economy, a global perspective is needed for assessing the performance of U.S. schools, districts, and states. High-school graduates in each and every state compete for jobs with graduates from all over the world. Charles Vest, president of the National Academy of Engineering and president emeritus at Massachusetts Institute of Technology, has warned, “America faces many challenges…but the enemy I fear most is complacency. We are about to be hit by the full force of global competition. If we continue to ignore the obvious task at hand while others beat us at our own game, our children and grandchildren will pay the price. We must now establish a sense of urgency.”
Paul E. Peterson is the director of Harvard’s Program on Education Policy and Governance and senior fellow at the Hoover Institution. Ludger Woessmann is professor of economics at the University of Munich. Eric A. Hanushek is senior fellow at the Hoover Institution of Stanford University. Carlos X. Lastra-Anadón is a research fellow at the Program on Education Policy and Governance at Harvard University. An unabridged version of this paper is available here.
This article appeared in the Fall 2011 issue of Education Next. Suggested citation format:
Peterson, P.E., Woessmann, L., Hanushek, E.A., and Lastra-Anadón, C.X. (2011). Are U.S. Students Ready to Compete? The latest on each state’s international standing. Education Next, 11(4), 51-59.
