Undoubtedly, the United States has much to learn from education systems in other countries. Once the world’s education leader, the U.S. has seen the percentage of its high-school students who are proficient trail that of 31 other countries in math and 16 countries in reading, according to a recent study by Harvard’s Program on Education Policy and Governance (PEPG) (see “Are U.S. Students Ready to Compete?” features, Fall 2011). Whereas only 32 percent of U.S. 8th graders are proficient in math, 50 percent of Canadian students and nearly 60 percent of Korean and Finnish students perform at that level. It may be misleading to point out that 75 percent of Shanghai’s students are proficient, as that Chinese province is the nation’s most advanced, but in Massachusetts, the highest-achieving of the states, only 51 percent of the students are proficient in math.

Given these performance disparities, it is only natural to think that there is something to be learned from practices elsewhere. Yet it is not easy to figure out what institutions and practices will translate into a different cultural milieu or how to do it. In the larger world of governmental constitutions, efforts to insert U.S. arrangements into distant political cultures have failed more often than not. Much the same could happen in reverse if the United States attempted to fix its schools simply by copying something that seems to work elsewhere.

It is tempting to undertake an in-depth study of those places that are performing at the highest levels—China’s Shanghai province, Korea, Finland, Singapore, Japan, the Netherlands, and Canada, for example. But a proper comparison requires that one contrast what successful countries do with the mistakes made by the less successful ones. International comparisons should look at information from all countries and adjust for factors that affect student performance, even though such rigorous studies typically face their own challenges, including collecting the requisite data. Moreover, countries are different across so many dimensions (from the political system to the cultural prestige of the teaching profession) that it is typically difficult to attribute differences between countries to any specific factors.

For these reasons, learning from international experience can be a bit like reading tea leaves: People are tempted to see in the patterns whatever they think they should see. But for all the hazards associated with drawing on international experience, the greatest risk lies in ignoring such information altogether. Steadfastly insisting that the United States is unique and that nothing is to be learned from other lands might appeal to those on the campaign trail. But it is a perilous course of action for those who wish to understand—and improve—the state of American education. If nothing else, reflection on international experience encourages one to think more carefully about practices and proposals at home. It is not so much specific answers that come from conversing with educators from around the world, as it is gaining some intellectual humility. Such conversations provide opportunities to learn the multiple ways in which common questions are posed and answered, and to consider how policies that have proved successful elsewhere might be adapted to the unique context of U.S. education.

That, perhaps, is the signal contribution of the August 2011 conference on “Learning from the International Experience,” sponsored by Harvard’s Program on Education Policy and Governance. Many who attended said the conference had sparked conversations well beyond the usual boundaries on thinking about U.S. education policy, whether the issue was teacher reforms, school choice, the development of common standards and school accountability, or the promise of learning online.

Need to Take Action

The conference opened with an urgent call from U.S. Deputy Secretary of Education Anthony Miller that action be taken. He highlighted two aspects of Harvard’s PEPG study in his remarks. First, by showing the dismal performance of students from families in which a parent has a college education, “the findings…debunk the myth that the mediocre performance of U.S. students on international tests is due simply to the presence of large numbers of disadvantaged students.” Indeed, the study shows that the percent proficient among U.S. students whose parents are college-educated or who are white is significantly less than the percent proficient among all students in countries such as Korea, Singapore, and Finland. Second, by breaking out results for every state, it shows that “the U.S. education system is comprised of 50 state systems, and therefore we must look at our performance on a state-by-state basis.”

Hoover Institution scholar Eric Hanushek built on Miller’s remarks by reporting that, according to work he did with University of Munich economist Ludger Woessmann, the United States could boost its annual GDP growth rate by more than 1 percentage point annually by raising student math performance to levels currently attained in countries such as Canada and Korea. That kind of increase in economic productivity could, over the long run, boost the U.S. economy by trillions of dollars. According to Hanushek, “the impact of the current recession on the economy is dwarfed” by the magnitude of the loss in wealth that has at its root subpar U.S. education performance.

Hanushek was careful to state that the goal was not to strengthen U.S. performance at the expense of other nations: The creation of well-educated citizens does not constitute a “a zero-sum game that countries or states are playing against each other,” but one in which every country and state can become more productive, and create more wealth for one another by boosting and sharing their talents. The United States can welcome the higher Canadian, Finnish, Korean, and Chinese performances even as those accomplishments make a compelling case for “changing the direction the United States is going.”

Further developing the case for reform, University of Arkansas scholars Jay P. Greene and Josh P. McGee (see “When the Best Is Mediocre,” features, page 34) provided conference participants with a glimpse of their new report, which identifies the international standing of nearly every school district in the United States. “People tend to think their own districts are OK,” even when the United States as a whole appears to be doing badly, Greene said. “But they really are not.” Even in expensive suburbs, student performance does not look very good from an international perspective, they said. “There is no refuge for ‘elite’ families in this country.” Greene and McGee reported that in 17 states they were unable to find a single district that performed at levels comparable to those reached by students in the world’s leading countries.

Teachers and Teaching

Offering hope that urgent action can be taken, Mona Mourshed told the conference that she and her colleagues at McKinsey & Company have shown that “systems can achieve significant gains in as short a time as six years.” Mediocre systems can become much better, and “those that are good can become great.” In her view, there are “clusters of interventions” that are appropriate for each stage of system development, and for each one, the key driver of change is teachers. The most important factor for every system’s journey of transformation, she said, is to develop teachers’ capabilities to their full potential. And others agreed. As New Jersey’s chief education officer Christopher Cerf put it, “The single greatest in-school variable driving [learning] outcomes is the quality of the teacher.”

But how can we ensure high-quality instruction? According to Mourshed, much depends on the stage a school system has reached. If a system is mediocre and has only low-performing teachers, then it can make the most progress through strong administrative actions that identify clear expectations for teachers and are fairly prescriptive. This may involve scripted teaching materials, monitoring of the time teachers devote to each task, and regular visits by master teachers or school inspectors. But, as the performance of the system rises and the teaching force reaches a higher level of quality, it can move “from good to great” by giving those teachers both greater autonomy and support. Among other things, great school systems decentralize pedagogical methods to schools and teachers, and put in place incentives for frontline educators to share innovative practices across schools. “Teacher teams” collaborate to push the quality and customization of classroom materials even further, and the educators rotate throughout the system, spreading peer learning and enriching mentorship opportunities.

Fernando Reimers of Harvard’s Graduate School of Education said that most teachers are trained in academic programs that have low prestige and are far removed from the activities of the classroom. Students in these programs are asked to think about sociological, psychological, and policy issues rather than to discuss what it takes to teach a particular lesson effectively. In this regard, schools of education are unlike other professional schools. He gave the example of business schools, which are increasingly asked to link instruction directly to the work future managers will be expected to do. Reimers urged that education-training programs combine mastery of the subject matter, needed especially today in math and science, with the ability to adapt teaching to different learners, to use technology effectively, and to enable project-based learning and teamwork.

In making these points, Reimers built on the presentation on Finnish training programs given by Jari Lavonen of the University of Helsinki. Advanced training at an education school in Finland is “more popular than medical school,” Lavonen told conference participants. Those admitted are a select group, and acceptance virtually guarantees a well-compensated and prestigious career. Rigorous training programs expect future teachers to demonstrate content knowledge in both a major and a minor subject, research competence, and classroom effectiveness. He admitted that the pedagogical research component was often contested by students (“we are teachers, not researchers”), but, he says, alumni later tell him that it was one of the most valuable parts of their educational experience. In his view, it is this component that enables them to tackle complex classrooms situations effectively later on. But, Lavonen cautioned, the system works in Finland only because the political situation was stable enough that the country was able to take “consistent decisions over the course of 40 years.”

Gwang-Jo Kim, former education vice-minister in Korea and current head of UNESCO in Thailand, also stressed the quality of those entering the teaching profession. Koreans are known for their “high regard for teachers and for the teaching profession.” Primary-school teacher-training programs receive many more applicants than there are spaces. There are multiple routes to certification as a secondary-school teacher, but the chances of getting a job are as low as 5 percent, as positions are avidly sought. Similarly, in Singapore, applicants to teacher-training programs are carefully selected, with a large proportion coming from the top 30 percent of the college population.

Kim said the Korean and Singapore success stories could not be understood apart from deep-seated cultural factors. The demand for teacher excellence comes from parents, who want their children to do well on national examinations that determine future education and occupational opportunities. As a result, teachers are under a lot of pressure. With unionization of the teaching profession in Korea, Kim wonders whether the current model can be sustained.

Building on these insights, White House education adviser Roberto Rodríguez reported that the Obama administration is developing models of teacher mentorship and induction that will support new recruits into the profession and renew teacher-preparation programs. “We don’t have a system that recruits talent. There is not a high bar for ed schools,” Rodríguez said. In addition, he emphasized the current lack of high-quality professional development for teachers and adequate mentorship for new teachers. Rodríguez confirmed the administration’s intention to create differentiated tracks for master teachers, administrators, and specialist teachers, in which teacher compensation is tied to progress on those tracks. Currently, he stated, “we lose too many good teachers to administration.” Underlining a point made by Deputy Secretary Miller, Rodríguez reminded the conference that, to be effective, change must come not only from the federal government but from “high levels of energy at the state and local level.”

Agreeing that state action is vital, New Jersey’s Christopher Cerf told conference participants that successful education systems do the same thing high-quality businesses strive to do: recruit from the very best, maximize the productivity of employees, evaluate responsibly and helpfully, deploy its workforce where it can be most helpful, and have a clear talent-retention strategy. But in the United States, he said, “We do all of these things badly in education. We recruit from whatever the ed schools give us, there is no productivity angle and no pay for results. We have taken the view that doing teacher evaluations is so hard that we should do nothing at all, and our retention strategy amounts to saying to high-performing teachers, ‘please stay.’” To change that system and lift the quality of the teaching force to international levels won’t be easy, cautioned Gerard Robinson, Florida’s chief education officer. “It is all about brute political force; the rest is a rounding error.”

Jason Glass, director of the Department of Education for the state of Iowa, reminded the audience that “we cannot take the challenges one at a time if they refuse to stay in line.” Glass said his priority is to alter the “one-minute interviews” used to make decisions on teacher hiring in too many school districts. He also seeks to improve the mentorship that teachers receive in their first year of teaching, which he says is virtually nonexistent in parts of his state. He plans to introduce more sophisticated systems that will identify—and retrain or remove—the state’s least-capable teachers. In reforming Iowa’s public school system, he intends to get beyond the prevalent false dichotomies, such as “cash for test scores versus step-and-lane compensation” and “due process versus random firing.” Performance measures able to identify the least capable teachers can and should be found. He concluded with a hopeful warning: “Watch out for Iowa over the next few years.”

Choice and Autonomy

Hindering the conversation on school choice was the fact that the mechanisms for choice in the United States do not resemble the choice mechanisms elsewhere. In the United States, private schools receive little government aid (except for transportation, lunch programs, and, in a few places, school vouchers), whereas in most other countries governments fund private schools at levels close to those for state-run schools. Charter schools are privately operated schools that are funded by the government, but they may not teach religion, while government-funded private schools in most other countries may do so.

Avis Glaze, former superintendent of the Ontario education system, correctly observed that Canada does not have charter schools, but others mentioned that the large number of religious schools that are both government-funded and subject to state regulation give Canadians even more choice than exists in the United States.

The conversation was also shaped by the recent release of a study by the Program for International Student Assessment (PISA), the same agency that collected the international data on which the PEPG report was based. According to the PISA study, international experience suggests that nothing is to be gained from expanding the private sector in education. Students in private schools do no better than students in public schools, once differences in family background characteristics are taken into account.

That finding, said Martin West, assistant professor at Harvard’s Graduate School of Education, is both misleading and, paradoxically, exactly what one should expect. When undertaking an international analysis of school choice, he argued, one should not compare the effectiveness of the public and private sectors but should instead look at the extent to which competition between the two sectors affects the achievement of all the students in the country, regardless of whether they go to public or private school. In countries such as high-achieving Netherlands, a large percentage of students attend private schools, with government paying the tuition. In countries such as low-performing Spain, only a few students attend private school. Other countries fall in between these two extremes. Using a sophisticated statistical technique, West showed that all students in a country learned more when the private sector was larger. Specifically, the study by West and his colleague found that an increase in the share of private school enrollment of 10 percentage points was associated with better than a quarter of a year’s worth of learning in math, though somewhat less in reading. Moreover, this increase in performance takes place within school systems that spend 6 percent less overall.

A degree of choice can be introduced in the state sector if decisionmaking is shifted to the school level, as has been done in Ontario, Glaze said. The U.S. Department of Education should provide support and oversight to local decisions and push specific “nonnegotiable” programs, such as the literacy program Ontario implemented in the 2000s. Paul Pastorek, Louisiana’s former chief education officer, agreed that the Ontario experiment had been successful but said the United States needed a different approach. The story of school reform has too often been one of a strong district or state leader driving reform until the end of her tenure, with stagnation afterward. Only the powers of competition embedded within a system can lead to sustained improvements. “The problem is that we don’t know how to leverage competitive forces in the multibillion-dollar business that is education in this country,” said Pastorek. “Our education system is a communist system; we don’t have anything that relates what we pay for resources to the economic value they generate.”

The introduction of competition in New Orleans, where 85 percent of the schools are now charter schools, said Pastorek, provided a foundation for continued reform and improvement. But choice works only if choice systems are equitable, schools are held accountable by the state or school district, and parents are given readily understandable information about school quality. In the view of many, a great system would be one in which through the power of competitive forces, as Pastorek described, states create a system that “self-corrects, self-challenges, and self-innovates” to achieve better results for children.

State Standards and Accountability

Common standards and tests that evaluate performance against those standards are to be found in most of the countries that are performing better than the United States, whether they be in Europe or Asia. Shengchang Tang, principal of the Shanghai High School (the leading high school in China), said that the standards and examinations in the Shanghai province are a powerful tool that parents use to exert pressure on their children as well as on teachers and principals. (These particular standards and exams do not extend to the whole of China, which is deemed too large to have a single set of exams.) In his view, that pressure focuses attention in schools and fuels the drivers of the successful Shanghai education system, including higher investments, a high-caliber teaching force, and a strategy tailored to the specific situation faced by each school.

Tang questioned whether common standards would be effective in the very different U.S. context. Specifically, he was skeptical that such standards would catalyze more effective parent pressure on U.S. schools, given parents’ comparatively low expectations of their children and their schools. In contrast, in a recent poll in Shanghai, 85 percent of parents declared that they expected their children to be in the top 15 percent of their age cohort. Standardized exams, in Tang’s view, serve as a necessary tool to measure reality against these high expectations. For Angus MacBeath, former school commissioner in Edmonton, Alberta, Canada, however, setting high standards in his home province allowed him to “tell the ugly truth,” and it was the necessary first step toward Alberta’s journey of educational improvement. Common standards allow parents, educators, and policymakers to be clear about current achievement levels so they can act on that knowledge.

This is perhaps the reason the Obama administration has lent its support to the Common Core State Standards Initiative, which has been embraced as a reform solution by 44 states and the District of Columbia. Still, many wondered with James Stergios of the Pioneer Institute in Boston whether one can set standards capable of driving high performance nationwide in a country that has great regional disparities in student achievement and a decentralized governmental system (where schools are “radically local,” as one panelist put it). Declaring himself “a massive opponent of common standards,” Stergios argued that the excellence achieved by Massachusetts so far could not be sustained if nationwide standards were substituted for state ones.

Gerard Robinson began his comments by acknowledging that he was chief education officer in Virginia while that state was opposed to common standards and is now chief in Florida, which is committed to common standards. He offered two reasons for embracing common standards: 1) students must compete with those in other states and, indeed, with students all over the world, and 2) companies need common standards in order to compare job applicants. “The difficult part is not to have consensus on having common standards,” he observed, “but on how to work on the political process to achieve them.”

In the end, the standards issue seemed to turn on the questions raised by Shanghai’s Shengchang Tang. Could the United States create common standards that were high enough to spur high achievement? While “having high state standards makes a big difference to underprivileged people,” as Christopher Cerf put it, common standards might be set too low and so, contrary to what the PEPG report showed, may not serve to raise standards of achievement when U.S. students are compared to their peers in high-achieving countries. He reminded the group that the same political context exists today as existed when No Child Left Behind was crafted. As prescribed in that legislation, every child was supposed to be proficient, but to comply with federal expectations many states “dumbed down” their definition of student proficiency.

Digital Learning

In her opening remarks for the panel on digital learning, New Mexico’s chief education officer Hanna Skandera stressed that the new technologies provided new opportunities to address together all the reforms under discussion. Digital learning that exploits online courses and broadband capabilities can expand choice for students, ensure transparency and accountability for courses offered online, and create opportunities for many more students to come into contact with the very best teachers. Further, it can serve as a catalyst for higher standards and can do all this without driving up the cost of education.

Shantanu Prakash, of Educomp Solutions, informed the audience about the business he started and now heads in India. Educomp serves more than 12 million students in India alone and operates in a number of other developing countries where traditional schools have limited resources and set low standards for instruction. Educomp targets schools with products it says are not only inexpensive but user-friendly and easily combined with traditional classroom instruction. “The whiteboard can be used with millions of modules that are very good, that will support any teacher,” he noted. Prakesh expects the demand for his products to grow rapidly, as “the pressure of parents will make the introduction of digital materials into the learning of children in a meaningful way inevitable.” It is an obvious means for parents with high expectations all over India to ensure that their children receive high-quality instruction, in a context of scarce resources and low teaching standards.

Susan Patrick, president and CEO of the International Association for K–12 Online Learning (iNACOL), agreed: “Education is no longer a cottage and local industry,” but one in which true competition can thrive, improving standards and driving productivity gains. Digital learning can give students greater choice, even down to the specific instructor for a particular course. Digital learning is a growing reality in many other countries. Citing numerous references, Patrick told of its widespread adoption across the world. In Singapore, for example, all schools blend online learning with classroom instruction, and the country’s schools of education have made online instructional techniques an integral part of the curriculum. South Korea is once again a leader, and virtual education has become a rapidly growing industry, partly to reduce the cost to parents of the “cram schools” that families expect their adolescent children to attend.

Also participating in the conference was Julie Young, president and CEO of the Florida Virtual School (FLVS), the leading example of digital learning in the United States. Since its beginnings in 1996, FLVS has grown steadily and currently has nearly 200,000 course enrollments. The reasons for its success, according to Young, include student access to teachers seven days a week and beyond the regular school day, choice in assignments, and a constantly improving curriculum and instruction that is transparent to administrators, parents, and outsiders.

The main barrier to the spread of digital learning in the United States, iNACOL’s Patrick noted, are “policies that were created 30 to 40 years ago for a different world. Digital teachers cannot easily be qualified in multiple states, funding follows student and sometimes physical attendance, and there are no common standards across states that would reduce the costs of development.” Only when those policies are upgraded purposefully to accommodate and encourage a different kind of classroom environment will digital learning become an integral part of the American education system.

Tea Leaves or Tea?

So what did the conference brew? No one can make the case that the conference provided secret bullets for school reform in the United States, and most every conference participant would agree that the particulars of the United States make it difficult to introduce wholesale many of the practices that have been successful abroad. Popular culture shows little appreciation for the educated citizen; a decentralized government arrangement with multiple veto points precludes rapid innovation; and education politics is marked by antipathy between teachers unions and school reformers. But a nuanced assessment of the conversation allows for at least preliminary conclusions that go beyond a simple call for urgent action:

• Teacher selection, teacher training, teacher evaluation, and teacher retention in the United States can be done much better than it is being done today. While no country has exactly the right model for the United States, none of the successful systems leave good teaching simply to chance the way the United States does.

• School choice plays a bigger—and perhaps more successful—role in the world’s educational experience than is usually recognized. It should not be seen as a threat but rather as an incentive for improvement for the public education system.

• Standards and testing systems that hold students accountable for their performance are part and parcel of most, if not all, of the world’s top education systems. If the United States has a heterogeneity that precludes the adoption of a uniform examination system as those found in Korea, Singapore, and in many parts of Canada, that provides no reason not to set clearer, and higher, expectations for students than is commonly the case.

• Digital learning has yet to prove itself fully and to develop into an integrated paradigm-shifting approach, but early stories of success are promising, provided digital learning respects the principles of transparency, accountability, and choice for students.

More than reaching any specific conclusion, the conference was most successful in inspiring participants with a renewed understanding of and dedication to their common commitment to a better system of education. The commitment is now informed by the experience of other countries with similar challenges that have managed, through sustained and consistent policies (as the Finnish representative, Jari Lavonen, insisted) to find solutions.

Carlos X. Lastra-Anadón is a research fellow at the Program on Education Policy and Governance. Paul E. Peterson is director of the Program on Education Policy and Governance at Harvard University and senior fellow at the Hoover Institution.

An unabridged version of this paper is available here.

On August 17, 2011 Paul Peterson discussed the findings of this study in a free online webinar. An archived recording of this webinar can be found here.

---

At a time of persistent unemployment, especially among the less skilled, many wonder whether our schools are adequately preparing students for the 21st-century global economy. Despite high unemployment rates, firms are experiencing shortages of educated workers, outsourcing professional-level work to workers abroad, and competing for the limited number of employment visas set aside for highly skilled immigrants. As President Barack Obama said in his 2011 State of the Union address, “We know what it takes to compete for the jobs and industries of our time. We need to out-innovate, out-educate, and out-build the rest of the world.”

The challenge is particularly great in math, science, and engineering. According to Internet entrepreneur Vinton Cerf, “America simply is not producing enough of our own innovators, and the cause is twofold—a deteriorating K–12 education system and a national culture that does not emphasize the importance of education and the value of engineering and science.” To address the issue, the Science, Technology, Engineering, and Math (STEM) Education Coalition was formed in 2006 to “raise awareness in Congress, the Administration, and other organizations about the critical role that STEM education plays in enabling the U.S. to remain the economic and technological leader of the global marketplace.” Tales of shortages of educated talent appear regularly in the media. According to a CBS News report, 22 percent of American businesses say they are ready to hire if they can find people with the right skills. As one factory owner put it, “It’s hard to fill these jobs because they require people who are good at math, good with their hands, and willing to work on a factory floor.” According to a Bureau of Labor Statistics report, of the 30 occupations projected to grow the most rapidly over the next decade, nearly half are professional jobs that require at least a college degree. On the basis of these projections, McKinsey’s Global Institute estimates that over the next few years there will be a gap of nearly 2 million workers with the necessary analytical and technical skills.

In this paper we view the proficiency of U.S. students from a global perspective. Although we provide information on performances in both reading and mathematics, our emphasis is on student proficiency in mathematics, the subject many feel to be of greatest concern.

Student Proficiency on NAEP

At one time it was left to teachers and administrators to decide exactly what level of math proficiency should be expected of students. But, increasingly, states, and the federal government itself, have established proficiency levels that students are asked to reach. A national proficiency standard was set by the board that governs the National Assessment of Educational Progress (NAEP), which is administered by the U.S. Department of Education and generally known as the nation’s report card.

In 2007, just 32 percent of 8th graders in public and private schools in the United States performed at or above the NAEP proficiency standard in mathematics, and 31 percent performed at or above that level in reading. When more than two-thirds of students fail to reach a proficiency bar, it raises serious questions. Are U.S. schools failing to teach their students adequately? Or has NAEP set its proficiency bar at a level beyond the normal reach of a student in 8th grade?

One way of tackling such questions is to take an international perspective. Are other countries able to lift a higher percentage—or even a majority—of their students to or above the NAEP proficiency bar? Another approach is to look at differences among states. What percentage of students in each state is performing at a proficient level? How does each state compare to students in other countries?

In this article, we report results from our second study of student achievement in global perspective conducted for Harvard’s Program on Education Policy and Governance (PEPG). In our 2010 PEPG report, we compared the percentage of U.S. public and private school students in the high-school graduating Class of 2009 who were performing at the advanced level in mathematics with rates of similar performance among their peers around the world (see “Teaching Math to the Talented,” features, Winter 2011). The current study continues this work by reporting proficiency rates in both mathematics and reading for the most recent cohort for which data are available, the high-school graduating Class of 2011.

Comparing U.S. Students with Peers in Other Countries

If the NAEP exams are the nation’s report card, the world’s report card is assembled by the Organization for Economic Co-operation and Development (OECD), which administers the Program for International Student Assessment (PISA) to representative samples of 15-year-old students in 65 of the world’s school systems (which, to simplify the presentation, we shall refer to as countries; Hong Kong, Macao, and Shanghai are not independent nations but are nonetheless included in PISA reports). Since its launch in 2000, the PISA test has emerged as the yardstick by which countries measure changes in their performance over time and the level of their performance relative to that of other countries.

Since the United States participates in the PISA examinations, it is possible to make direct comparisons between the average performance of U.S. students and that of their peers elsewhere. But to compare the percentages of students deemed proficient in math or reading, one must ascertain the PISA equivalent of the NAEP standard of proficiency. To obtain that information, we perform a crosswalk between NAEP and PISA. The crosswalk is made possible by the fact that representative (but separate) samples of the high-school graduating Class of 2011 took the NAEP and PISA math and reading examinations. NAEP tests were taken in 2007 when the Class of 2011 was in 8th grade and PISA tested 15-year-olds in 2009, most of whom are members of the Class of 2011. Given that NAEP identified 32 percent of U.S. 8th-grade students as proficient in math, the PISA equivalent is estimated by calculating the minimum score reached by the top-performing 32 percent of U.S. students participating in the 2009 PISA test. (See methodological sidebar for further details.)

---

Methodological Approach
In the United States, in 2007, the share of 8th-grade students identified as proficient on the NAEP math examination was 32.192 percent. The minimum math score on the PISA examination obtained in 2009 by the highest-performing 32.192 percent of all U.S. students was estimated to be 530.7. To cover a broad content area while ensuring that testing time does not become excessive, the tests employ matrix sampling. No student takes the entire test, and scores are aggregated across students. Results are thus estimates of performance obtained by averaging five plausible values, as PISA and NAEP administrators recommend.

Comparable numbers for the other categories are as follows:

Reading proficiency: 31.223 percent of U.S. students are proficient on the NAEP, which corresponds to 550.4 on PISA.

Advanced math: 6.998 percent of U.S. students scored at the advanced level on the NAEP, which corresponds to 623.2 on PISA.

Advanced reading: 2.767 percent of U.S. students scored at the advanced level on the NAEP, which corresponds to 678.1 on PISA.

What It Means to Be Proficient

According to the National Center for Education Statistics (NCES), which administers NAEP, the determination of proficiency in any given subject at a particular grade level “was the result of a comprehensive national process [which took into account]…what hundreds of educators, curriculum experts, policymakers, and members of the general public thought the assessment should test. After the completion of the framework, the NAEP [subject] Committee worked with measurement specialists to create the assessment questions and scoring criteria.” In other words, NAEP’s concept of proficiency is not based on any objective criterion, but reflects a consensus on what should be known by students who have reached a certain educational stage. NAEP says that 8th graders, if proficient, “understand the connections between fractions, percents, decimals, and other mathematical topics such as algebra and functions.”

PISA does not set a proficiency standard. Instead, it sets different levels of performance, ranging from one (the lowest) to six (the highest). A student who is at the proficiency level in math set by NAEP performs moderately above proficiency  level three on the PISA. (See sidebar for a statement of the 8th-grade proficiency standard and sample questions from PISA and NAEP that proficient students are expected to pass.)

Crossing the Proficiency Bar

Given that definition of math proficiency, U.S. students in the Class of 2011, with a 32 percent proficiency rate, came in 32nd among the nations that participated in PISA. Performance levels among the countries ranked 23rd to 31st are not significantly different from that of the U.S. in a statistical sense, yet 22 countries do significantly outperform the United States in the share of students reaching the proficiency level in math. Six countries plus Shanghai and Hong Kong had majorities of students performing at least at the proficiency level, while the United States had less than one-third. For example, 58 percent of Korean students and 56 percent of Finnish students performed at or above a proficient level. Other countries in which a majority—or near majority—of students performed at or above the proficiency level included Switzerland, Japan, Canada, and the Netherlands. Many other nations also had math proficiency rates well above that of the United States, including Germany (45 percent), Australia (44 percent), and France (39 percent). Figure 1 presents a detailed listing of the scores of all participating countries as well as the performance of the individual states within the United States.

Shanghai topped the list with a 75 percent math proficiency rate, well over twice the 32 percent rate of the United States. However, Shanghai students are from a prosperous metropolitan area within China, so their performance is more appropriately compared to Massachusetts and Minnesota, which are similarly favored and are the top performers among the U.S. states. When this comparison is made, Shanghai still performs at a distinctly higher level. Only a little more than half (51 percent) of Massachusetts students are proficient in math, while Minnesota, the runner-up state, has a math proficiency rate of just 43 percent.

Only four additional states—Vermont, North Dakota, New Jersey, and Kansas—have a math proficiency rate above 40 percent. Some of the country’s largest and richest states score below the average for the United States as a whole, including New York (30 percent), Missouri (30 percent), Michigan (29 percent), Florida (27 percent), and California (24 percent).

Proficiency in Reading

According to NAEP, students proficient in reading “should be able to make and support inferences about a text, connect parts of a text, and analyze text features.” According to PISA, students at level four, a level of performance set very close to the NAEP proficiency level, should be “capable of difficult reading tasks, such as locating embedded information, construing meaning from nuances of languages critically evaluating a text.” (See sidebar for more specific definitions and sample questions.)

The U.S. proficiency rate in reading, at 31 percent, compares reasonably well to those of most European countries other than Finland. It takes 17th place among the nations of the world, and only the top 10 countries on PISA outperform the United States by a statistically significant amount. In Korea, 47 percent of the students are proficient in reading. Other countries that outrank the United States include Finland (46 percent), Singapore,  New Zealand, and Japan (42 percent), Canada (41 percent), Australia (38 percent), and Belgium (37 percent).

Within the United States, Massachusetts is again the leader, with 43 percent of 8th-grade students performing at the NAEP proficiency level in reading. Shanghai students perform at a higher level, however, with 56 percent of its young people proficient in reading. Within the United States, Vermont is a close second to its neighbor to the south, with 42 percent proficiency. New Jersey and Montana come next, both with 39 percent of the students identified as proficient in reading. The District of Columbia, the nation’s worst, are at the level achieved in Turkey and Bulgaria, while the one-eighth of our students living in California are similar to those in Slovakia and Spain. (See Figure 2 for the international ranking of all states.)

Ethnic Groups

The percentage proficient in the United States varies considerably among students from different racial and ethnic backgrounds. While 42 percent of white students were identified as proficient in math, only 11 percent of African American students, 15 percent of Hispanic students, and 16 percent of Native Americans were so identified. Fifty percent of students with an ethnic background from Asia and the Pacific Islands, however, were proficient in math, placing them at a level comparable to students in Belgium, Canada, and Japan.

In reading, 40 percent of white students and 41 percent of those from Asia and the Pacific Islands were identified as proficient. Only 13 percent of African American students, 5 percent of Hispanic students, and 18 percent of Native American students were so identified.

Given the disparate performances among students from various cultural backgrounds, it may be worth inquiring as to whether differences between the United States and other countries are due to the presence of a substantial minority population within the United States. To examine that question, we compare U.S. white students to all students in other countries. We do this not because we think this is the right comparison, but simply to consider the oft-expressed claim that education problems in the United States are confined to certain segments within the minority community.

While the 42 percent math efficiency rate for U.S. white students is considerably higher than that of African American and Hispanic students, they are still surpassed by all students in 16 other countries. White students in the United States trail well behind all students in Korea, Japan, Finland, Germany, Belgium, and Canada.

White students in Massachusetts outperform their peers in other states; 58 percent are at or above the math proficiency level. Maryland, New Jersey, and Texas are the other states in which a majority of white students is proficient in math. Given recent school-related political conflicts in Wisconsin, it is of interest that only 42 percent of that state’s white students are proficient in math, a rate no better than the nation as a whole.  (Results for all states are presented in the unabridged version of the paper.)

In reading, the picture looks better. As we mentioned above, only 40 percent of white students are proficient, but that proficiency rate would place the United States at 9th in the world. Its proficiency rate does not differ significantly (in a statistical sense) from that for all students in Canada, Japan, and New Zealand, but white students trail in reading by a significant margin all students in Shanghai, Korea, Finland, Hong Kong, and Singapore. In no state is a majority of white students proficient, although Massachusetts comes close with a 49 percent rate. The four states with the next highest levels of reading proficiency among white students are New Jersey, Connecticut, Maryland, and Colorado.

Are the Proficiency Standards the Same for Math as for Reading?

Has NAEP set a lower proficiency standard in math than in reading? If so, is the math standard too low or the reading bar too high?

At first glance it would seem that the standard is set at pretty much the same level. After all, 32 percent of U.S. students are deemed proficient in math and 31 percent are deemed proficient in reading.

But that coincidence is quite misleading. When compared to peers abroad, the U.S. Class of 2011 performed respectably in reading, trailing only 10 other nations by a statistically significant amount. Admittedly, the U.S. trails Korea by 16 percentage points, but it’s only 10 percentage points behind Canada. Meanwhile, U.S. performance in math significantly trails that of 22 countries. Korean performance is 26 percentage points higher than that of the United States, while Canadian performance is 18 percentage points higher. Judged by international standards, U.S. 8th graders are clearly doing worse in math than in reading, despite the fact that NAEP reports similar percentages proficient in the two subjects.

A direct comparison of NAEP’s proficiency standard with PISA’s proficiency levels three and four also indicates that a lower NAEP bar has been set in math than in reading. To meet NAEP’s standards currently, one needs to perform near the fourth level on PISA’s reading exam, but only modestly above the third level on its math exam.

Clearly, the experts set an 8th-grade math proficiency standard at a level lower than the one set in reading. Perhaps this is an indication that American society as a whole, including the experts who design NAEP standards, set lower expectations for students in math than in reading. If so, it is a sign that low performance in mathematics within the United States may be deeply rooted in the nation’s culture. Those who are setting the common core standards under discussion might well take note of this.

Of course, it could be argued that the math proficiency standard is correct but the reading standard has been set too high. In no country in the world does a majority of the students reach the NAEP proficiency bar set in 8th-grade reading.

What Does It Mean?

Many have concluded that the productivity of the U.S. economy could be greatly enhanced if a higher percentage of U.S. students were proficient in mathematics. As Michael Brown, Nobel Prize winner in medicine, has declared, “If America is to maintain our high standard of living, we must continue to innovate…. Math and science are the engines of innovation. With these engines we can lead the world.”

But others have argued that the overall past success of the U.S. economy suggests that high-school math performance is not that critical for sustained growth in economic productivity. After all, U.S. students trailed their peers in the very first international survey undertaken nearly 50 years ago. That is the wrong message to take away however. Other factors contributed to the relatively high rate of growth in economic productivity during the last half of the 20th century, including the openness of the country’s markets, respect for property rights, low levels of political corruption, and limited intrusion of government into the operations of the marketplace. The United States, moreover, has always benefited from the in-migration of talent from abroad.

Furthermore, the United States has historically had far higher levels of educational attainment than other countries, with many more students graduating from high school, continuing on to college, and earning an advanced degree. It appears that in the past the country made up for low quality in elementary and high school by educating students for longer periods of time.

As we proceed into the 21st century, none of these factors remains as favorable to the United States. While other countries are lifting restrictions on market operations, the opposite has been occurring within the United States. The U.S. has also placed sharp limits on the numbers of talented workers that can be legally admitted into the country. Our higher education system, though still perceived to be the best in the world, is recruiting an ever-increasing proportion of its faculty and students from outside the country. Meanwhile, educational attainment rates among U.S. citizens now trail the industrial-world average.

Even if some of these trends can be reversed, that hardly gainsays the desirability of enhancing the mathematical skills of the U.S. student population, especially at a time when the nation’s growth in productivity is badly trailing growth rates in China, India, Brazil, and many smaller Asian countries. Eric Hanushek and Ludger Woessmann have shown elsewhere that student performance on international tests such as those we consider here is closely related to long-term economic growth (see “Education and Economic Growth,” research, Spring 2008). Assuming past economic patterns continue, the country could enjoy a remarkable increment in its annual GDP growth per capita by enhancing the math proficiency of U.S. students. Increasing the percentage of proficient students to the levels attained in Canada and Korea would increase the annual U.S. growth rate by 0.9 percentage points and 1.3 percentage points, respectively. Since current average annual growth rates hover between 2 and 3 percentage points, that increment would lift growth rates by between 30 and 50 percent.

When translated into dollar terms, these magnitudes become staggering. If one calculates these percentage increases as national income projections over an 80-year period (providing for a 20-year delay before any school reform is completed and the newly proficient students begin their working careers), a back-of-the-envelope calculation suggests gains of nothing less than $75 trillion over the period. That averages out to around a trillion dollars a year. Even if you tweak these numbers a bit in one direction or another to account for various uncertainties, you reach the same bottom line: Those who say that student math performance does not matter are clearly wrong.

Given the integration of the world economy, a global perspective is needed for assessing the performance of U.S. schools, districts, and states. High-school graduates in each and every state compete for jobs with graduates from all over the world. Charles Vest, president of the National Academy of Engineering and president emeritus at Massachusetts Institute of Technology, has warned, “America faces many challenges…but the enemy I fear most is complacency. We are about to be hit by the full force of global competition. If we continue to ignore the obvious task at hand while others beat us at our own game, our children and grandchildren will pay the price. We must now establish a sense of urgency.”

Paul E. Peterson is the director of Harvard’s Program on Education Policy and Governance and senior fellow at the Hoover Institution. Ludger Woessmann is professor of economics at the University of Munich. Eric A. Hanushek is senior fellow at the Hoover Institution of Stanford University. Carlos X. Lastra-Anadón is a research fellow at the Program on Education Policy and Governance at Harvard University. An unabridged version of this paper is available here.

The data used to determine the grades in Figure 1 are available here.

---

Much ado has been made about setting high standards over the past year. In his first major address on education policy, given just two months after he took the oath of office, President Barack Obama put the issue on the national agenda. They ought “to stop lowballing expectations for our kids,” he said, adding that “the solution to low test scores is not lowering standards—it’s tougher, clearer standards.” In March 2010, Secretary of Education Arne Duncan accused educators of having “lowered the bar” so they could meet the requirements set by the federal education law, No Child Left Behind (NCLB), which requires that all students be proficient in reading and math by the year 2014.

Current conversations about creating a common national standard largely focus on the substantive curriculum to be taught at various grade levels. Even more important, we submit, is each state’s expectations for student performance with respect to the curriculum, as expressed through its proficiency standard. Curricula can be perfectly designed, but if the proficiency bar is set very low, little is accomplished by setting the content standards in the first place.

To see whether states are setting proficiency bars in such a way that they are “lowballing expectations” and have “lowered the bar” for students in 4th- and 8th-grade reading and math, Education Next has used information from the recently released 2009 National Assessment of Educational Progress (NAEP) to evaluate empirically the proficiency standards each state has established. This report is the fourth in a series in which we periodically assess the rigor of these standards (see “Johnny Can Read…in Some States,” features, Summer 2005; “Keeping an Eye on State Standards,” features, Summer 2006; and “Few States Set World-Class Standards,” check the facts, Summer 2008).

The 2009 NAEP tests in reading and math were given to a representative sample of students in 4th- and 8th-grade in each state. NAEP, called “the nation’s report card,” is managed by the Department of Education’s National Center for Education Statistics and is currently the “gold standard” of assessments. Its proficiency standard is roughly equivalent to the international standard established by those industrialized nations that are members of the Organisation for Economic Co-operation and Development (OECD). If a state identifies no higher a percentage of students as being proficient on its own tests than NAEP does, then the state can be said to have set its standards at a world-class level. To ascertain objectively whether state standards are high or low, and whether they are rising or falling, we compare the percentage of students deemed proficient by each state with the percentage proficient as measured by NAEP. The state assessment data used in this report consist of those compiled in 2009 by the 50 states and the District of Columbia.

States have strong incentives not to set world-class standards. If they do, more of their schools will be identified as failing under NCLB rules, and states will then be required to take corrective actions to bring students’ performance up to the higher standard. As a result, the temptation for states to “lowball expectations” is substantial. Perhaps for this reason, a sharp disparity between NAEP standards and the standards in most states has been identified in all of our previous reports. In 2009, the situation improved in reading, but deteriorated further in math.

Every state, for both reading and math (with the exception of Massachusetts for math), deems more students “proficient” on its own assessments than NAEP does. The average difference is a startling 37 percentage points. In Figure 1, we provide a uniform ranking of the rigor of state standards using the same A to F scale used to grade students (see sidebar for the specifics on the methodology we used).

Racing to the Top?

Ironically, Tennessee received an F and had the lowest standards of all states, despite the fact that it is one of the two winners in the first phase of the bitterly contested Race to the Top (RttT) competition sponsored by the Obama administration’s Department of Education. Indeed, Tennessee has had the lowest standards of all states since 2003. Based on its own tests and standards, the state claimed in 2009 that over 90 percent of its 4th-grade students were proficient in math, whereas NAEP tests revealed that only 28 percent were performing at a proficient level. Results in 4th-grade reading and at the 8th-grade level are much the same. With such divergence, the concept of “standard” has lost all meaning. It’s as if a yardstick can be 36 inches long in most of the world, but 3 inches long in Tennessee.

Delaware, the other RttT First Phase winner, also had below-average standards, for which we awarded a grade of C- and ranked it 36th of the 50 states and the District of Columbia. Delaware claimed that 77 percent of its 4th-grade students were proficient in math, when NAEP shows that only 36 percent were. In 8th-grade reading, Delaware said 81 percent of its students were proficient, but NAEP put the figure at 31 percent.

From these findings one might conclude that the Obama administration is having a huge policy impact by getting states like Tennessee and Delaware to set standards they have been unwilling to establish in the past. But Tennessee earned almost full marks (98 percent) on the section of the competition (weighted a substantial 14 percent of all possible points) devoted to “adopting standards and assessments,” even though its standards have remained extremely low ever since the federal accountability law took hold. The proof will be in the pudding. If Tennessee and Delaware and other states now shift their standards dramatically upward, RttT will win over those who think it is performance, rather than promises, that should be rewarded.

Disparities in State Standards

Despite the incentive to lowball expectations, five states—Hawaii, Massachusetts, Missouri, New Mexico, and Washington—have set their standards at or close to the world-class level, earning them an A. Notice that we award grades purely for the expected standard for performance, not actual proficiency. New Mexico earned the same mark as Massachusetts, even though only about one-quarter of its students are proficient, while half of Massachusetts students score at that level. The two deserve equal grades, however, because both are rigorous in their expectations. Another eight states—Colorado, Maine, Minnesota, Montana, New Hampshire, New Jersey, Rhode Island, and Vermont—earned a B for their standards.

President Obama is undoubtedly correct, however, in suggesting that many states are “lowballing expectations.” Of the remaining 38 states, 27 earned a C, and 8—Arizona, Idaho, Illinois, Maryland, Michigan, New York, Texas, and Virginia—a D. Three states—Alabama, Nebraska, and Tennessee—had such low standards that we awarded them an F. All of the states that earned grades of F have been ranked D or below in all three of our previous reports. This suggests that once a standard, however low, has been set, it tends to persist—another reason to be concerned about promises from Delaware and Tennessee.

Changes in Standards

Secretary of Education Duncan is not altogether correct in suggesting that educators are lowering the bar, however. Figure 2 shows that in 2009 the differences between state and NAEP standards shrank by 0.08 standard deviations as compared to the average for the three prior surveys. This is a reversal of the trend of declining standards we observed between 2003 and 2007. Eight states improved the overall rigor of their assessments by a full letter grade or more since 2007: Georgia, Mississippi, Montana, New Jersey, New Mexico, Oklahoma, Utah, and West Virginia. By contrast, we gave just four states—Alaska, California, New York, and South Carolina—grades that were at least a full letter grade worse than they received in 2007.

The reversal in the overall trend is, however, driven wholly by an improvement in the rigor of reading assessments, which set expectations that are higher by 0.49 standard deviations in 4th grade and by 0.26 standard deviations in 8th grade. As a matter of fact, 17 states increased the rigor of their 4th-grade reading assessments by a whole letter grade since 2007, and 17 states did the same for 8th grade. But math standards have slipped by 0.12 standard deviations in 4th grade and by 0.31 in 8th grade. This means that at least some of the state-reported improvements in mathematics proficiency are misleading.

Converging on a De Facto National Standard?

Most changes to standards, as we noted, have been fairly small: only 12 states have made changes to their standards that alter their standing by a whole letter grade. But since our last report two states, Hawaii and South Carolina, have made major alterations to state assessments. The results of these moves have been at odds: while Hawaii’s increased alignment with NAEP raised its grade from a B+ in 2007 to an A, South Carolina dropped from an A to a C-.

States nonetheless seem to be continuing their trajectory of convergence toward standards of similar rigor in math (which, given the slipping standards noted above, constitutes a downward convergence), but are more divergent in reading since 2007, particularly in 4th grade. If the convergence of math standards were to continue, we could gradually attain something like a national standard. But it would take a great deal of national patience to achieve a national standard by convergence creep.

In this report, as in previous ones, we assess the rigor of standards that states set. This is an important task, as it reminds states that whether students have or have not learned cannot be a matter of how the test is designed and where the “proficiency line” is drawn. Rather, setting high standards for proficiency is the first step in the journey toward actually improving the learning of a high percentage of students. According to NAEP, less than one-third of students are proficient in reading and a similar proportion in math nationwide. For the sake of the children of this country, we should be doing much better than that.

Paul E. Peterson is professor of government at Harvard University, senior fellow at the Hoover Institution, and editor-in-chief at Education Next. Carlos Xabel Lastra-Anadón is a research fellow at the Program on Education Policy and Governance at Harvard University.