As dean of an education school, more and more I find myself asking, to my own surprise: Is it time to sever the link between the university programs that prepare teachers and the public bodies that certify them? For half a century, college- and university-based teacher-education programs have held a monopoly position in operating, in partnership with state governments, virtually the only route to the teaching profession in public schools. This has guaranteed us a fairly constant stream of students, revenues, and state and federal funding. These benefits have come with strings attached, of course: states have mandated much of the curriculum that prospective teachers must take.

The question is, has our monopoly position and the states’ intervention helped or undermined the quality of our programs? Has our monopoly made us unresponsive to the needs of our students and the schools that hire them? Have state mandates distracted us from designing a coherent, innovative, and high-quality curriculum? I think so. As the education system makes a transition to a focus on results-based accountability combined with more flexibility in school design, it’s time for those of us in the teacher-education profession to become more results-based and flexible too. We ought to concentrate on offering the best possible professional training programs while leaving it to others-state and local governments and individual schools-to decide who is qualified to be certified and hired. The breaking up of our monopoly would force us to convince students, their tuition-paying parents, and the school districts that do the hiring that our programs produce teaching candidates who are more qualified and skilled than candidates who obtained their training elsewhere or who come in with no training.

Policymakers and the public have realized that one of the keys, if not the key, to raising student achievement across the board is to improve the quality of the nation’s teachers. To my mind, nothing is more important to school quality than placing a passionate, dedicated, knowledgeable, and highly skilled teacher in every classroom. Doubts about the academic abilities of some of the nation’s teachers-as measured by various tests of student learning-have led to doubts about the university programs that prepare teachers. The conservative backlash against progressive education has also planted seeds of doubt about the content of the education provided in education schools. Moreover, the national teacher shortage has forced policymakers to create alternative pathways to teaching in order to encourage mid-career changers and talented college graduates who didn’t complete education programs to go into teaching. The success of programs like Teach for America and other alternative-certification programs has convinced many that a teacher needn’t attend an education school in order to be effective. Education schools need to recognize these doubts and answer them effectively.

This is not a call to move the professional preparation of teachers and administrators from the academy to the “real world” of practice in the schools. Such a move was a major part of the conservative agenda of British prime minister Margaret Thatcher, who sought to denigrate academic learning of any kind. A similar anti-intellectualism has come quickly to the United States. Classroom experience is a crucial part of any teacher’s training, but it is only a part. The apprenticeship approach works well in preparing people for a profession as it is currently practiced. It does not work very well in preparing people to be reflective or to innovate and be leaders of change. And if today’s schools need anything, it is some pretty basic reflection, rethinking, and reconsidering.

Nevertheless, we in teacher education do not have a stellar record of producing change agents, and we have failed at the task of policing ourselves. There are many wonderful teacher-preparation programs today, many more than there were only a few years ago. Leaders in teacher education have succeeded in raising the academic standards of their programs and promoting more interaction between education faculty and arts and sciences faculty. We have also answered the criticism that education programs were long on theory and short on practice by linking our programs much more effectively to schools through year-long internships and through the development of a clinical faculty of school-based teacher educators who are truly part of the teacher-preparation team. But the world of teacher education remains uneven. The teacher educator who has not been in a school in years; the one who advocates “multiple modes of instruction” but only lectures in class; the one who is just plain boring: these people are still all too real. Above all, the reality remains that we haven’t gained the respect of our colleagues in higher education, and many of our graduates themselves claim to have learned little of any value in education school. This alone should cause us to do some dramatic rethinking.

Teacher Preparation at Northeastern University At Northeastern, would-be teachers must complete a major in the College of Arts and Sciences in a field other than education as well as the following coursework. –Introduction to Education –Child Development, Learning, and Education – Learning and Teaching Process – Teaching Children’s Literature and the Arts – Integrating School Curriculum Through Social Studies – Teaching Math and Science to Children – Teaching the Language Arts – Introduction to Special Education – Health and Physical Education in Elementary School – Four cooperative education (practicum) experiences – One semester of student teaching Source: Northeastern University

A Brief History

The current university-state partnership in preparing and then certifying teachers is relatively new. For much of the nation’ s history, school districts often ran their own teacher-preparation programs. In many cases, elementary-school teachers were simply graduates of the local high school. Normal schools, which began in the 1830s, were in their early years much closer to being high schools than the equivalent of colleges. Most expanded their curriculum for future teachers to the equivalent of a baccalaureate degree only in the early decades of the 20th century. For all the debates over the virtues and vices of progressive education, before World War II the vast majority of Americans attended only elementary school, and they were taught by normal-school or high-school graduates who had often studied for one or two years at a school of pedagogy. Only in the mid-20th century did the notion that every teacher should have a college degree and some sort of professional preparation even begin to take hold. My home state of Massachusetts, long known as a leader in school reform, began requiring a bachelor’s degree for teachers only in 1954. At about the same time, state departments of education began the process of monitoring college-based teacher-preparation programs using standards that were usually the result of careful negotiations among the state, the representatives of the college, and-in some cases-representatives of teachers and school administrators.

There is no question that these mid-century reforms made a significant difference in the quality of teaching across the country. In many states the old normal schools were transformed into a state college system after World War II. Higher education expanded to offer amazing new opportunities to the waves of GIs returning from the war and many others who had previously been excluded. And a good number of these college graduates-especially the female graduates-became teachers. In the 1980s many states raised the standards once again. Especially after the 1986 reports of the Carnegie Forum on Education and the Economy and the Holmes Group of education deans, most states abolished the old education majors and required that future teachers combine a rich and full baccalaureate program with significant preparation in pedagogy, either as an undergraduate minor or through a fifth year or other kind of graduate program. The teachers of the past half-century have been the best prepared of any generation of teachers in history. They have had better undergraduate educations and deeper knowledge of their subject matter and the complex process of teaching and learning than any who preceded them.

Still, the nation is not satisfied with the quality of the nation’s teachers, and rightly so. Much of the blame has fallen on education schools. There are many other culprits, from the unequal funding of school districts, which leads to gross disparities in teacher pay, to the sheer lack of respect afforded to teaching as a profession. But we in teacher education are ultimately responsible for preparing prospective teachers to be great teachers. So as more and more states adopt various forms of high-stakes testing for both students and teachers, and as the reports of failure on a massive scale continue to proliferate, we in teacher education face harsh scrutiny. The logic is unavoidable: if the schools are failing, then the teachers are failing. If the teachers are failing, then the programs that prepare them must be failing too. Deeper analysis has raised many questions about the quality of some standardized tests. But high-stakes tests are here to stay. They speak for a public that has lost faith in the bright assurances of professional educators that teachers were well prepared and that students were learning.

At the same time, and arising from the same frustrations, many states have sought to impose even tighter regulations on the current state-approved college- and university-based teacher-preparation programs. In 1998 the U.S. Congress joined in this effort with its requirements for tough new reporting measures-the Title II Report Cards-that every teacher-preparation program in the country now files and that will be made public across the nation in the spring of 2002. Leaders within the teacher-education enterprise itself have also demanded much better preparation for future teachers. The 1996 report of the National Commission on Teaching & America’s Future made a series of dramatic recommendations, including a call for states and school districts to “get serious about standards, for both students and teachers” and for a thorough restructuring of teacher preparation, recruitment, and professional development. Meanwhile other reformers wish to dismantle the entire certification system and afford public school principals the same privilege as private school principals: the ability to choose any teacher, certified or not, based on a personal assessment of their skills and attitude.

A Modest Proposal

Overhauling the system of teacher education first requires that education schools recognize that for many in the larger public-the parents of today’s schoolchildren, the taxpayers who foot the bill, and the elected officials who represent them-the current system of teacher preparation is seen as part of the problem, not the solution. While we may complain that such judgments are unfair, we need to address the reality that, fairly or unfairly, teacher education has lost its legitimacy in many circles.

As someone who has devoted a large part of my professional career to teacher preparation, this situation worries me. I am not a fan of the old system, but I am deeply worried that “alternative routes” are going to create a generation of teachers who do not have the fundamental skills, knowledge, and inclinations needed to support student learning. I fear models that reinforce the status quo in our schools, because for me the status quo in American education is unacceptable. Too many kids are left out, marginalized, or taught that their failures are their own fault. Too many kids are excluded from the benefits of today’s economy and from participation in tomorrow’s democracy. What is to be done?

Perhaps the time has come for those of us in higher education to simply step out of the teacher-certification business. We should let school districts and, ideally, in many cases, individual schools make the certification and the hiring decisions. States, in turn, should step out of their role in regulating university-based teacher-preparation programs. Both the schools and the higher-education programs will thrive, I believe, in such a new atmosphere of freedom and accountability.

The role of university-based teacher preparation should be different-and if we do the preparation right, more significant. Let the schools and school districts certify and hire whom they will. Our role as teacher educators will be to provide programs that add such clear and obvious value that school districts prefer our candidates to others. If we are doing a good job, our graduates will obviously be the best teachers. If they are not, we will have only ourselves to blame.

If we give up the monopoly that has protected us in the past, a monopoly that is gone whether we like it or not, we should be given much more freedom to design the kinds of programs that we believe will truly prepare the best teachers. If the state no longer certifies teachers because they are our graduates, the state also must relinquish any right to regulate our curriculum. Let us clearly separate the two functions. We would not offer any courses “because the state mandated them.” We would be free to design the best curriculum we could imagine, to seek accreditation or not, as it served the education of our students and-more important-their students. Let our quality be judged by our graduates’ ability to get the best teaching jobs and by the learning of students in their classrooms.

I can already hear the first objection: “But what about nepotism? What about a politicized hiring process?” These are not idle worries. In the much romanticized “good old days” before teachers were certified, many a school board member hired teachers based on who could offer the best pay-off, in cash or in promised votes. Many a school administrator hired cousins and friends, sometimes their not-too-bright cousins and friends, who could not find other employment. The schools were the great employment agency of last resort, not only in corrupt big cities, but also much more in small towns, where fewer outside eyes were watching. But society has undergone a dramatic change. It is not that 21st-century America is magically free of corruption and cronyism; it is that now there are no places where outside eyes are not watching.

Ironically, it is the high-stakes testing movement that has made nepotism a losing proposition and the need for certification less pressing. In an accountability system where the very livelihood of educators depends on their schools’ performance, what school board member, superintendent, or principal would hire an underqualified teacher and allow that teacher to bring down the school’s or the district’s test scores, thereby incurring the wrath of parents, state legislators, and the media? In the past it was possible to have Lake Wobegon schools where school leaders assured parents and the community that wonderful learning was happening and all of the children were above average. Today the averages are out there for everyone to calculate. With high-stakes testing, the stakes have simply become too high for anyone to risk his or her career by failing to exercise informed judgment about the quality of the teachers being hired.

If we have high-stakes tests and all the public attention that goes with them, we should be able to trust school districts and schools to make wise decisions about whom to hire. Those of us in higher education should design admission standards, curricula, clinical experiences, and exit examinations that will allow us to be sure that our graduates really are the best women and men for the jobs. If we do that, and if we succeed, our programs will flourish and our graduates will be able to compete quite successfully for jobs in the most wonderful of careers-teaching our nation’s schoolchildren.

What will such programs look like? My guess is that the most successful of the unregulated programs will adopt many of the recommendations of the National Commission on Teaching & America’s Future. I cannot imagine quality teacher education that is not based on a rigorous academic curriculum alongside a series of increasingly complex and extended field experiences. These experiences allow students to mix the very best teaching with thoughtful and intellectually rigorous reflection on those experiences, along with a high-quality academic curriculum that is rich in content knowledge at both the undergraduate and the graduate levels.

Those of us who live our professional lives in college- and university-based teacher-preparation programs must face a radically changed and changing reality. We cannot return to the old system of states’ closely regulating our programs while guaranteeing ready access to the profession for our graduates by making us the only route to certification. It was not that good a system, and in any case it is gone. Our choice is to be defensive and to mourn for the past or to strike out in a new direction, trying something new and adventurous and engaging, for our students and ourselves. Who knows, it might even be more intellectually and professionally rewarding for us and-what really matters-it might begin to prepare a generation of teachers who can have the intellectual resources, the professional skills, and the moral courage to design schools in which no one is excluded, in which every child succeeds and thrives.

-James W. Fraser is dean of the school of education at Northeastern University in Boston, Massachusetts.

Picture Gerard, a 28-year-old business consultant who majored in economics at Williams College and graduated with a 3.7 GPA. Gerard has been working for a consulting firm in Stamford, Connecticut, but is looking for a new, more fulfilling position. He has demonstrated strong interpersonal skills and work habits. In addition, though he didn’t major in math, he aced several calculus courses in college. Yet if Gerard were to apply through normal channels to teach math at a junior high school in the Hartford public school system, his application wouldn’t even be considered. Why? Because he isn’t a certified teacher.

Why shouldn’t a principal or a faculty hiring committee in the Hartford schools even be allowed to look at Gerard’s application, to judge his qualifications against those of other candidates? The assumption undergirding the contemporary approach to teacher certification is that public school hiring personnel are either unable or unwilling to gauge the quality of applicants. Our response has been to embrace a bureaucratic solution that handcuffs the capable and incapable alike and supposedly keeps weak teachers out of the classroom. As a result, having discouraged or turned away Gerard and hundreds like him, many large school systems resort to last-minute fill-ins who teach on emergency certificates.

This is not to suggest, even for a moment, that candidates with “real world” experience or high GPAs are necessarily qualified or equipped to become teachers or that professional preparation for teachers is unimportant. It is only to say that some potential applicants might be more effective teachers than the alternatives that are currently available to public schools.

The central premise underlying teacher certification is that-no matter what their qualifications are-anyone who has not completed the specified training is unsuited to enter a classroom and must be prohibited from applying for a job. Presumably, the danger is that, in a moment of weakness, a school official otherwise will mistakenly hire such an applicant rather than an appropriately trained teacher. It is essential to remember what we often seem to forget, which is that allowing someone to apply for a job is not the same as guaranteeing him employment. Making applicants eligible for a position simply permits an employer to hire them in the event that they are deemed superior to the existing alternatives. The argument against certification is not that unconventional applicants will be good teachers; it is only that they might be. If one believes this, case-by-case judgments are clearly more appropriate than an inflexible bureaucratic rule.

Imagine if colleges and universities refused to hire anyone who lacked a Ph.D. They would lose the talents and insights of “lay practitioners” like poet Maya Angelou, journalist William Raspberry, or former public officials such as Alan Simpson, Julian Bond, and Al Gore. The artists and writers “in residence” at dozens of public universities would fail to meet the criteria implicit in the public school certification model. Do we really believe that these universities are ill-serving their students by hiring people whom the public schools would consider unqualified?

Competitive Certification

The theory behind certifying or licensing public school teachers is that this process elevates the profession by ensuring that aspiring teachers master a well-documented and broadly accepted body of knowledge and skills important to teaching. Supporters of teacher certification often make analogies to professions like law and medicine, where being an effective professional requires the acquisition of vast knowledge and skills. Licensure in these professions ensures at least minimal competency and boosts the public’s confidence in members of the profession.

The problem is that no comparable body of knowledge and skills exists in teaching. Debate rages over the merits of various pedagogical strategies, and even teacher educators and certification proponents have a hard time defining a clear set of concrete skills that makes for a good teacher. Yet most aspiring teachers are still forced to run a gauntlet of courses, requirements, and procedures created by accredited training programs that vary dramatically in quality.
This is not to deny that teacher education can provide valuable training. After all, one may think that journalism schools produce better journalists without requiring all journalists to complete a mandatory set of courses before seeking work in the profession. Instead, it is assumed that a candidate’s training is factored into the hiring process, along with considerations like aptitude, diligence, and energy.

Clearly some sort of screening process for aspiring teachers is essential; parents and the public rightly expect safeguards for those working with youngsters. What is needed is a competitive certification process that establishes key criteria for entry into the teaching profession; gives public schools greater freedom to hire and fire teachers; and treats teachers like professionals and their schools like professional institutions by allowing them to tailor professional development to meet the needs of teachers. Under such a model, aspiring teachers ought to be able to apply for a teaching job if they:

–Possess a B.A. or B.S. degree from a recognized college or university.

–Pass a test that demonstrates competency in knowledge or skills essential to what they seek to teach. The definition of “essential” knowledge or skills is obviously a loose one that can be interpreted in myriad ways and rightly should be different for those wishing to teach younger children or older students. The key point is to demand that teachers at least have an appropriate academic knowledge of the material they will be teaching.

–Pass a rigorous criminal background check. States conduct such checks now, but they tend to be compromised by the state’s need to engage simultaneously in related certification paperwork.

Beyond these minimal qualifications, the competitive approach presumes that preparation and training are not only desirable but also essential, as is true in other professions where subtle skills and interpersonal dynamics are essential to effective performance. The questions are where to obtain this training and who should pay for it. Contemporary teacher preparation imposes nearly all of the costs on candidates by forcing them into a system of training that removes key incentives for quality and relevance in teacher preparation. The competitive model instead treats teachers as autonomous professionals able to make informed decisions about developing their skills and expertise. In short, the competitive model would substitute meaningful professional development for what is essentially a guild system funded by levying a significant tuition-based tax on aspiring teachers before permitting them to enter the profession.

The Assumptions of Certification

Over the years, an array of studies has sought to determine whether certified teachers serve students more effectively than uncertified teachers. There are two problems with this line of work. First, the methodological wrangling has often obscured the larger questions and the central assumptions of the certification model. Second, the case for certification is thin whether or not certified teachers boost student achievement more than their uncertified peers. The issue is not whether teacher education improves the performance of graduates, but whether we ought to-as best we are able-bar from teaching those who have not completed an approved preparatory program. Certification systems deny school administrators the ability to take their context or the promise of a particular applicant into account when hiring. Even if certified teachers are generally more effective than uncertified teachers, such a policy only makes sense if we believe that uncertified applicants are uniformly incompetent to teach or that school administrators cannot be trusted to assess their competence.

The allure of certification rests on three implicit assumptions. They are the beliefs that: 1) the training one receives while getting certified is so useful that the uncertified will be relatively ill-prepared; 2) certification weeds out unsuitable candidates; and 3) certification makes teaching more “professional” and therefore a more attractive career. However, each of these presumptions is problematic in the case of teacher certification.

As a general principle, certification is most effective when the licensing body ensures that aspiring professionals have mastered essential skills or knowledge and denies a license to inadequate performers. Licensure is most essential when a professional’s tasks are critical and when clients may have trouble assessing a provider’s qualifications. For instance, licensure is considered particularly appropriate for engineers, doctors, and attorneys because those who design bridges, tend us when we are ill, or defend our rights all perform tasks essential to our well-being and are frequently charged with aiding us at our most vulnerable. Moreover, it can be difficult for members of the public to know whether a bridge is properly designed, whether a doctor is performing appropriately, or whether an attorney is knowledgeable in the law. Licensing is not an assurance that these professionals are talented practitioners, but it does ensure that they have demonstrated an established degree of professional knowledge.

Educators are also charged with a crucial task. However, the oversight challenge is very different in education, where we have not established a specific, measurable body of skills or knowledge that teachers must master. Educational “experts” themselves argue that teaching is so complex that it can be difficult to judge a good teacher outside of a specific classroom context. This makes it difficult, if not impossible, to determine abstractly which aspirants possess satisfactory “teaching skills.” Meanwhile, there is widespread agreement that colleagues, supervisors, and families have at least a proximate ability to gauge whether a teacher is effective. Given these circumstances, it is unclear how standardized licensing helps to safeguard teacher quality.

Such a conclusion does not require refuting the claims of teacher educators or the supporters of certification. It actually follows if one simply accepts their claims. Professional educators themselves have thus far been unable to explain in any concrete sense what makes a teacher competent or what teachers need to know and be able to do.

Consider the widely praised standards that the National Board for Professional Teaching Standards (NBPTS) has painstakingly constructed in 27 distinct fields, standards that certification’s proponents have hailed as a breakthrough in quality control. The area where the NBPTS ought to have the easiest time creating straightforward standards is high-school math and science teaching, where there is widespread consensus as to what teachers are supposed to do. Even in these areas, however, the NBPTS’s “exemplary” standards are so broad and vague as to make concrete judgments of competence nearly impossible. For instance, to receive National Board certification to teach high-school math, teachers are to demonstrate mastery of 11 standards, including: commitment to students and their learning, the art of teaching, reflection and growth, and reasoning and thinking mathematically. The board tries to clarify these standards by explaining, for instance, that “commitment” is interpreted as meaning that “accomplished mathematics teachers value and acknowledge the individuality and worth of each student, believe that all students can learn,” and so on. Mastering the “art of teaching” is taken to mean that teachers “stimulate and facilitate student learning by using a wide range of formats and procedures.” While these are certainly admirable sentiments, nowhere in the National Board’s rarified standards is it clear how we are to gauge just what constitutes “competence” in these tasks. The result, unsurprisingly, is that the board has been assailed for the capricious way in which the standards are being interpreted and applied. Despite the best of intentions in the drafting of the INTASC standards, which Mary Diez discusses, a cursory read makes clear that they are plagued by the same ambiguities evident in the NBPTS standards.

For another prominent example, consider education professors Gerald Grant and Christine E. Murray’s award-winning 1999 Harvard University Press book, Teaching in America. They identify five “essential [teaching] acts” that can be analyzed and taught: listening with care; motivating the student; modeling caring by hearing and responding to the pain of others, and by creating a sense of security in their classrooms; evaluating by clarifying, coaching, advising, and deciding on an appropriate challenge for this boy or that girl; and reflecting and renewing. How one is to teach these five “essential acts,” much less determine whether a teacher has satisfactorily mastered them, are questions that Grant and Murray never address.

If clear standards of professional competence do not exist, we typically (and appropriately) hesitate to prohibit some individuals from practicing a profession. This is not to say that we think incompetence is acceptable in such a profession-only that we recognize licensing as an ineffective and potentially pernicious way to control quality. While licensure could protect community members (including children) from exposure to “bad” entrepreneurs or journalists, we do not prohibit some people from seeking to start businesses or work for a newspaper. Instead, we trust that potential investors or employers are the best judges of who ought to be supported or hired. If an aspiring writer or entrepreneur is unsuccessful, we trust that they will eventually be persuaded to find a line of work for which they are better suited. This free-flowing process fosters diversity and ensures that unconventional workers are given a chance to succeed.

Even in professions with clear knowledge- or performance-based benchmarks for certification, as in law or medicine, licensure is useful primarily as a way of establishing minimal competence. A medical or a law license is not imagined to ensure competence in ambiguous, subtle skills like comforting a patient or swaying a jury-skills analogous to the interpersonal relations thought crucial to teaching. Basing certification on such traits is difficult, because we may disagree about what they entail or how they can be assessed devoid of context. The skills that teacher educators deem most important-listening, caring, motivating-are not susceptible to standardized quality control. Emphasis on these qualities is the norm in professions like marketing, journalism, consulting, or policymaking, where a subtle blend of people skills, knowledge, and relevant expertise is required. In professions like these, where there are a number of ways for practitioners to excel but where it is difficult to know in advance how any particular practitioner will perform, the most sensible way to find talent is to allow aspirants to seek work and to permit employers to screen on a variety of criteria-such as education, experience, and references.

A Dubious Screen

While certification can serve to screen out aspirants who fail to meet a minimal performance standard, our current system is not designed to do so. Generally speaking, schools of education are not selective, flunk out few if any students for inadequate performance, and see that many of their teacher education graduates receive teacher licenses. The licensing exams are simple, and standards for passage are generally so low that the Education Trust concluded they exclude only the “weakest of the weak” from classrooms.

More than 1,300 institutions provide the training required for licensure. While defenders of the current approach to certification often focus on the certification programs at elite institutions, the top 25 education schools train less than 5 percent of the roughly 200,000 new graduates that teacher programs produce each year. It is the regional colleges, such as Illinois State University, Cal State-Hayward, and Southwest Texas State University-not the Stanfords and the Ohio States-that train and license the vast majority of teachers. The value of certification turns not on the quality of elite programs but on that of regional colleges.

Teacher-preparation programs neither screen out nor weed out weak candidates. Even at elite schools, such as UCLA or the University of North Carolina, where admissions rates are about 5 percent for medical school and 25 percent for law school, the M.Ed. programs (which include those seeking postgraduate training for teacher certification) accept more than half of their applicants. Moreover, education-school officials often make it clear that they do not see their mission as weeding out students during their course of study. Notes one such official, “We’re here to develop teachers, not to screen people out. For the most part, everyone who enters the program is going to complete it, unless they decide that teaching’s not for them.”

The Costs of Certification

Especially for anyone who didn’t complete a teacher-training program as an undergraduate, the costs of certification can be significant. It is not unusual for postgraduate teacher training programs to require a full-time commitment of 16 or even 24 months or a part-time commitment that can stretch to three years or more. The cost of training and the loss of salary due to time spent out of the workforce can easily reduce a teacher’s real compensation during her first five years by 25 percent or more.

These barriers make other professions relatively more attractive, so that potentially talented teachers who are unsure about their interest are less likely to try teaching. Whereas candidates can readily try journalism or consulting or marketing for a year, they must make an extensive commitment before they can try public school teaching. The result is that many who might make fine teachers never enter the profession. There is disturbing evidence that certification may especially dissuade accomplished minority candidates-who have a number of attractive career options and who are often less well situated to absorb the costs of teacher preparation-from entering teaching.

This would pose no real problem if we were blessed with a surplus of good teachers. In such a case, we might scoff “good riddance” to those dissuaded from teaching. However, we have a desperate need for competent teachers. Moreover, rather than a lack of commitment to teaching, a reluctance to pursue certification may indicate that individuals have attractive alternatives. It is the most talented and hardest working individuals who have the most career options and who sacrifice the most by entering a profession where compensation is unlinked to performance and where opportunities for advancement are few. They may wish to teach but be unwilling to forgo work for a year, sit through poorly regarded courses, or jump through procedural hurdles. It is candidates with fewer attractive options who will find the tedious but intellectually undemanding requirements of certification less problematic. In fact, by suppressing the supply of teachers, certification provides teachers with enhanced job security. Coupled with a compensation scale that rewards seniority rather than performance, certification may well make the profession more attractive to graduates seeking a less demanding line of work. In this way, certification can actually harm the public’s perception of teaching as a profession-the very opposite of what certification proponents wish to do.

Creative Destruction

In a world without certification as we know it, districts and schools would have more flexibility to ensure that their new teachers are prepared, inducted, and supervised in a manner appropriate to the challenges at hand. Because aspiring teachers would no longer have to attend formal teacher-preparation programs in order to teach, they would be free to make professional decisions about training in the same manner as business school or journalism school students. Weaker teacher-preparation programs would likely fall by the wayside. The fact that schools of education could no longer rely on a captive body of aspiring teachers would expose them to the cleansing winds of competition. Schools would have to contribute value-by providing teacher training, services, or research that created demand and attracted support-or face significant cutbacks. Teacher-preparation programs would find it in their own self-interest to ensure that their graduates were knowledgeable and skilled, as this would help graduates to win desirable jobs amid increased competition, making preparatory institutions more attractive.

Under a competitive certification system, little is likely to change in many of our high-performing suburban districts, where officials are inundated with applicants and are unlikely to tamper with a formula that is “working.” In such districts, except in rare cases, we would expect that administrators would continue to cherry-pick from the nation’s top teacher-education graduates. It is in the less desirable and more troubled systems, the nation’s urban and rural school districts, that administrators currently have tremendous difficulty finding sufficient numbers of certified teachers. This is doubly true in the areas of math and science education. It is in these districts and subjects, where critics have fretted about the numbers of long-term substitutes and “burned out” veterans, where the wave of new teachers will most likely be recruited and welcomed. While many of the resultant applicants will no doubt be deemed unprepared or unsuited for the jobs they pursue, there are few urban or rural principals who would not welcome the chance to pick and choose from their ranks.

Critics may fear that the elimination of licensure requirements will mean the end of teacher preparation and professional development. Such concern is unfounded. First, allowing uncertified individuals to become teachers does not mean that they must be viewed as “completed” professionals. Such a mindset is one of the vestiges of our current system, which is erected on a premise that all teachers are certified and therefore competent. Here, a better model might be medicine or law, where entering professionals begin their career with a trial period (serving as a hospital resident or as a junior partner in a law firm, for instance) during which their full panoply of skills is developed and monitored. Beginning teachers might serve on a probationary basis, receiving substantial monitoring and counseling. However, legal and contractual language ought to make it much simpler to terminate ineffective teachers or to mandate that they engage in support activities designed to improve their performance.

Second, moving to competitive certification does not mean doing away with professional teacher-education programs. Many applicants attend journalism school or business school, even though such training is not officially required, because it may make graduates more effective and can help them find better employment more readily. Likewise, aspiring teachers would presumably continue to attend those teacher-training programs thought to enhance their employability. This change would introduce some much-needed market pressure in this area, as schools would be forced to compete for students based on the usefulness of their course offerings.

Giving districts more leeway to hire promising candidates does not mean they will always make good decisions. Some ineffective teachers will inevitably continue to be hired. However, if entry to the profession is eased, it is appropriate that exit be eased as well. If administrators are to have more leeway to make hiring decisions, they also must be given more leeway to fire-and they must be held accountable for both sets of decisions.

At the end of the day, the individuals best equipped to assess the qualifications of prospective teachers are the principals who will be responsible for them. These same principals ought to have the strongest incentive to see that teachers are effective. If we believe that the administrators charged with managing and supervising schools either are unequipped to evaluate prospective teachers or are unwilling to do so, teacher certification will not suffice to protect our children from such profound systemic dysfunction. If we trust administrators, then certification is unnecessary and entails significant costs. If we don’t trust administrators, let us address that issue directly and not rely on the hollow promise of flimsy parchment barriers. Regardless, it is past time to fully acknowledge the nuanced, multifaceted, and professional nature of teaching and to move beyond a system that restricts professional entry with procedural barriers characterized by ambiguity that are the result of an inability to clearly define the skills, knowledge, or training essential to good teaching.

-Frederick M. Hess is an assistant professor of education and government at the University of Virginia and directs the Virginia Center for Educational Policy Studies.

Inside the Washington, D.C., beltway, the reauthorization of the Elementary and Secondary Education Act (ESEA) is seen as either a sea change in federal education policy or a half-measure designed to demonstrate the political leadership’s willingness to “do something” on education. On one side are supporters of the legislation who point to its substantial tightening of school accountability; its granting of more flexibility to states and school districts in the use of federal funds; and its commitment to applying sanctions to and providing aid for failing schools. On the other side are those who argue that the bill doesn’t go far enough. Supporters of school choice believe that opportunities for real reform were lost when conservatives and New Democrats failed to persuade their colleagues and the president’s advisors to include vouchers as part of the reform package.

In other words, there is no genuine opposition in Washington to accountability rules that simply fail to understand the institutional realities of accountability in states, districts, and schools. And the law’s provisions are considerably at odds with the technical realities of test-based accountability. Never, I think, in the history of federal education policy has the disconnect between policy and practice been so evident, and possibly never so dangerous. What’s particularly strange and ironic is that conservative Republicans control the White House and the House of Representatives, and they sponsored the single largest–and the single most damaging–expansion of federal power over the nation’s education system in history.

The federal government is mandating a single test-based accountability system for all states–a system currently operating in fewer than half the states. The federal government is requiring annual testing at every grade level, and requiring states to disaggregate their test scores by racial and socioeconomic backgrounds–a system currently operating in only a handful of states and one that is fraught with technical difficulties. The federal government is mandating a single definition of adequate yearly progress, the amount by which schools must increase their test scores in order to avoid some sort of sanction–an issue that in the past has been decided jointly by states and the federal government. And the federal government has set a single target date by which all students must exceed a state-defined proficiency level–an issue that in the past has been left almost entirely to states and localities.

Thus the federal government is now accelerating the worst trend of the current accountability movement: that performance-based accountability has come to mean testing, and testing alone. It doesn’t have to. In fact, in the early stages of the current accountability movement, reformers had an expansive view of performance that included, in addition to tests, portfolios of students’ work, teachers’ evaluations of their students, student-initiated projects, and formal exhibitions of students’ work. The comparative appeal of standardized tests is easy to see: they are relatively inexpensive to administer; can be mandated relatively easily; can be rapidly implemented; and deliver clear, visible results. However, relying only on standardized tests simply dodges the complicated questions of what tests actually measure and of how schools and students react when tests are the sole yardstick of performance.

If this shift in federal policy were based on the accumulated wisdom gained from experiences with accountability in states, districts, and schools, or if it were based on clear design principles that had some basis in practice, it might be worth the risk. In fact, however, this shift is based on little more than policy talk among people who know hardly anything about the institutional realities of accountability and even less about the problems of improving instruction in schools.

The Implementation Problem

The idea of performance-based accountability was introduced in the mid-1980s by the National Governors Association, led by Bill Clinton, then the governor of Arkansas. It took the form of what was then called the “horse trade”–states would grant schools and districts more flexibility in making decisions about what and how to teach in return for more accountability for academic performance. This idea became the central theory of today’s accountability reforms. It was an appealing idea in principle: governors and state legislators could take credit for improving schools without committing themselves to serious increases in funding.

The movement got a major boost in 1994, when Title I–the flagship federal compensatory education program–was amended to require states to create performance-based accountability systems for schools. The vision behind the 1994 amendments was that Title I would complement and accelerate the trend that began at the state level; they required states to develop academic standards, assessments based on the standards, and progress goals for schools and school districts, all within ambitious timetables. The merger of state and federal accountability policies–or alignment, as it was called–was supposed to occur by the year 2000. By the end of the decade, it was difficult to find more than one or two states lacking some form of statewide testing program and public release of the results. However, in all but a few states the basic architecture of performance-based accountability systems remained relatively crude and underdeveloped. In those few states where the idea had been developed most extensively–Texas and Kentucky, for example–the systems worked well enough, according to the testimonials of their sponsors, to legitimate the idea that they were successful in general. Even in these states, however, there were legitimate criticisms of the accountability system’s actual effect on academic performance and drop-out rates.

Nevertheless, by the late 1990s, it was abundantly clear that the states had fallen well short of what the crafters of the 1994 Title I amendments had envisioned. It was also clear that the federal government possessed very little leverage with which to force them along. States varied vastly in their administrative capacities to implement performance-based accountability systems. Many did not have testing systems in place that met the federal requirements. Most did not have the capacity to administer and monitor testing programs of the scale required to meet the federal goals. More important, creating accountability systems at the state level is essentially a political act, not an administrative one, and the federal government’s harmless knuckle-rapping was hardly going to overcome the intransigence of a state legislature or governor. Indeed, the ability of the U.S. Department of Education to monitor and enforce compliance with the 1994 law was limited; budget cuts whittled away at the department’s Title I staff just as their responsibilities were increasing. As is always the case, the department’s senior political appointees were reluctant to make life too difficult for governors and chief state school officers, who are among their key political constituencies. The rub: by 2000, the target date for full compliance, fewer than half the states had met the requirements. In this environment, it came as no surprise to learn that by the year 2000 many schools with Title I-eligible students were simply unaware of the program’s major policy shift in 1994.

This situation should have signaled to the Bush administration and Congress that there were complex issues of institutional capacity at the state and local level that could not be brushed aside by simply tightening the existing law’s requirements. If more than half the states were unable or unwilling to comply with the requirements of the previous, less stringent, more forgiving law, why would one expect all the states to comply with a much more stringent and exacting law? Part of the problem is political. Even though virtually all the states have joined the accountability bandwagon, for many states it is largely a symbolic act. The basic designs of the systems are still primitive; the authority of state education officials to oversee school districts is still limited in many cases; and the political consequences of imposing large-scale, statewide testing in states with strong traditions of local control are risky. Consequently, support for accountability among state legislators and governors is often highly volatile.

Another part of the problem is administrative. Mounting a statewide testing system is a task beyond the capacity of most state departments of education. Those that have embarked on large-scale testing are stretched to their limits just managing test-development work or monitoring testing contractors. Many states are not doing this work particularly well; others still don’t know what the work entails. Still another part of the problem is technical. Standardized tests inevitably become highly politicized and, in the course of the debate, the limits of testing are subjected to public scrutiny. Many policymakers enter the accountability debate not knowing much about testing, and they often discover, much to their chagrin, that what they don’t know can hurt them. Many legislators, for example, are surprised to hear that off-the-shelf standardized tests may not validly measure the content specified in state-mandated standards and that norm-referenced tests (tests that deliberately create a normal distribution around a mean) may not be effective in measuring changes in performance.

The Capacity Gap

The working theory behind test-based accountability is seemingly–perhaps fatally–simple. Students take tests that measure their academic performance in various subject areas. The results trigger certain consequences for students and schools–rewards, in the case of high performance, and sanctions for poor performance. Having stakes attached to test scores is supposed to create incentives for students and teachers to work harder and for school and district administrators to do a better job of monitoring their performance. If students, teachers, or schools are chronically low performing, presumably something more must be done–students must be denied diplomas or held back a grade; teachers or principals must be sanctioned or dismissed; and failing schools must be fixed or simply closed. The threat of such measures is supposed to be enough to motivate students and schools to ever-higher levels of performance.

This may have the ring of truth, but it is in fact a naïve, highly schematic, and oversimplified view of what it takes to improve student learning. The work that my colleagues and I have done on accountability suggests that internal accountability precedes external accountability. That is, school personnel must share a coherent, explicit set of norms and expectations about what a good school looks like before they can use signals from the outside to improve student learning. Giving test results to an incoherent, atomized, badly run school doesn’t automatically make it a better school. The ability of a school to make improvements has to do with the beliefs, norms, expectations, and practices that people in the organization share, not with the kind of information they receive about their performance. Low-performing schools aren’t coherent enough to respond to external demands for accountability.

The work of turning a school around entails improving the knowledge and skills of teachers–changing their knowledge of content and how to teach it–and helping them to understand where their students are in their academic development. Low-performing schools, and the people who work in them, don’t know what to do. If they did, they would be doing it already. You can’t improve a school’s performance, or the performance of any teacher or student in it, without increasing the investment in teachers’ knowledge, pedagogical skills, and understanding of students. This work can be influenced by an external accountability system, but it cannot be done by that system. Test scores don’t tell us much of anything about these important domains; they provide a composite, undifferentiated signal about students’ responses to a problem.

Test-based accountability without substantial investments in capacity–internal accountability and instructional improvement in schools–is unlikely to elicit better performance from low-performing students and schools. Furthermore, the increased pressure of test-based accountability, without substantial investments in capacity, is likely to aggravate the existing inequalities between low-performing and high-performing schools and students. Most high-performing schools simply reflect the social capital of their students; they are primarily schools with students of high socioeconomic status. Most low-performing schools also reflect the composition of their student populations. Performance-based accountability systems reward schools that work against the association between performance and socioeconomic status. However, most high-performing schools elicit higher performance by relying on the social capital of their students and families rather than on the internal capacity of the schools themselves. Most low-performing schools cannot rely on the social capital of students and families and instead must rely on their organizational capacity. Hence, with little or no investment in capacity, low-performing schools get worse relative to high-performing schools.

Some changes in the new law provide relatively unrestricted money that states can use to enhance capacity in schools if they choose to. However, neither state nor federal policy is currently addressing the capacity issue with anything like the intensity applied to the test-based accountability issue. So an enormous distortion is occurring in the relationship between accountability and capacity, a distortion that is being amplified rather than dampened by federal policy.

Abusing Tests

During the cold war, just about anyone who raised questions about the distribution of wealth in America was branded a Communist, thus chilling debates over social justice. Debate in the realm of education reform is being similarly chilled. Critics who suggest that there might be problems with the ways in which tests are being used for accountability purposes have been essentially marginalized. They’re smeared with accusations of being against accountability of any kind and of being apologists for a broken system. The idea that the performance of students and schools can be accurately and reliably measured by test scores is an article of faith in test-based accountability systems. Consequently, tests are being misused and abused in ways that will eventually undermine the credibility of performance-based accountability systems.

Probably the most serious problem lies in the use of test scores to make decisions about students’ academic progress–decisions about whether they can advance to the next grade or graduate from high school. The American Psychological Association’s guidelines for test use, as well as the consensus of professional judgment in the field of educational testing and measurement, specifically prohibit basing any consequential judgment about an individual student on a single test score. The primary reason for this principle is technical, not ethical. Test scores have a significant margin of error associated with them. That margin of error increases as the number of cases decreases; individual scores are typically much less reliable than aggregates of many individual scores. The best that can be said about an individual test score is that it falls within a range that is described by its coefficient of reliability. Unless the range is extremely small, which it isn’t for any standardized test, the likelihood of error is high.

The solution to this problem is to use multiple measures of a student’s performance when making consequential decisions. But this solution is more expensive because it introduces a new level of technical complexity into the system. For instance, say that high-school graduation was based on a composite of grades, test scores, and portfolios of students’ work. Developing such a composite would not only be a challenging technical feat; it would also introduce a certain amount of human judgment into the system. Policymakers tend to distrust the professionals who make such judgments.

A similar problem arises at the school level. Under Title I, schools are expected to meet their annual yearly progress goals. This involves calculating a school’s annual gain in test scores from one year to the next. Title I also requires disaggregating these scores by students’ ethnic and economic backgrounds. But research has shown (see Thomas Kane, Douglas Staiger, and Jeffrey Geppert, “Randomly Accountable,” in this issue) that these measures are highly unreliable for schools the size of a typical elementary school, and they are particularly unreliable for even smaller groups of students. Schools are often misclassified as low or high performing purely because of random variation in their test scores, unrelated to any educational factor.

The standards and accountability movement is in danger of being transformed into the testing and accountability movement. States without the human and financial resources to select, administer, and monitor tests are now being forced to begin testing at all grade levels. This is the surest way to guarantee that the test will become the content. Instead of creating academic standards that drive the design of a standards-based assessment, low-capacity states will simply select a test based on its expense and ease of administration. Thus the criticism that many state assessments are invalid because they fail to test the curriculum that is being taught. The more the issue of validity is submerged in the political debate on accountability, the more likely that charges of “teaching to the test” will be essentially accurate. A test with no external anchor in standards or expectations about student learning becomes a curriculum in itself, which trivializes the whole idea of performance-based accountability.

Prognosis

The idea of performance-based accountability plays to the greatest weaknesses of the American education system. After World War II, most industrialized countries nationalized their education systems, but not America. Just the idea that students in Louisiana should be held to the same academic and performance standards as those in New York was enough to inspire heated political debate for decades. One consequence of leaving decisions about content and performance to states and localities for so long is that they never developed the institutional capacity to monitor the improvement of teaching and learning in schools, to support the development of new knowledge and skill in teachers and administrators, and to develop measures of performance that are useful to educators and the public.

The difficult, uneven, and protracted slog toward clearer expectations and supports for learning has barely begun in most states and localities. The history of federal involvement in that long endeavor is at best mixed and at worst a failure. The current law repeats all of the strategic errors of the previous ESEA reauthorization, only this time at a higher level of federal intervention. The prognosis is not good. The best we can hope for is that the capacity problems of states and localities will become more visible as a political issue at the state and federal levels, triggering responses that will help schools overcome the obstacles they face in improving the quality and intensity of teaching and learning. Likewise, it is to be hoped that the technical failures of testing will gain higher visibility and trigger a response that focuses more on the assessment of student learning and less on the administration of tests. The worst that can happen is that test-based accountability will widen the gap between schools serving the well-off and the poor, thereby confirming, at least in the public’s mind, that expecting high levels of learning from all children is unrealistic.

As with many policy innovations, performance-based accountability in education is mutating into a caricature of itself. Never has this been clearer than in the reauthorization of Title I. The idea of giving schools and school districts greater flexibility in return for greater accountability for student performance–the original principle behind the “horse trade” of the 1980s–makes a great deal of sense. What we have discovered, however, is that accountability for performance requires substantial investments in organizational capacity: state departments of education need the capacity to select, implement, and monitor sound measures of performance; schools need support in developing internal coherence and instructional capacity; schools and districts need help in creating reasonable, diverse ways of assessing student learning; and teachers need support in acquiring the knowledge and skill required to reach larger numbers of students with more demanding content. As performance-based accountability becomes test-based accountability, these critical issues recede, and a sensible policy becomes a nightmare.

Richard F. Elmore is a professor of educational leadership at the Harvard Graduate School of Education and a senior research fellow at the Consortium for Policy Research in Education.

Who should be recruited to fill the two to three million K-12 teaching positions projected to come open during the next decade? What kinds of knowledge and training should these new recruits have? These are the questions confronting policymakers as a generation of teachers retires at the same time that the so-called baby boom echo is making its way through the education system. Key to answering these questions is knowing how much influence teachers have over student achievement and what specific teacher attributes lead to higher student achievement. For instance, does holding a master’s degree make one a better teacher? Do the best teachers hail from elite universities? Did they earn high GPAs in college? Did they major in the subject they are teaching? How much does experience matter? Do traditional, university-based teacher-preparation programs produce the best teachers, or are alternatively certified teachers just as good?

These questions are particularly relevant given that researchers have raised concerns about the overall quality of today’s teaching workforce. As measured by standardized test scores (mainly the SAT and the ACT), students choosing to major in education tend to be drawn from the lower end of the ability distribution. In Who Will Teach?, Harvard University professor of education Richard Murnane and his colleagues write: “College graduates with high test scores are less likely to take [teaching] jobs, employed teachers with high test scores are less likely to stay, and former teachers with high test scores are less likely to return.” On average, according to the findings of University of Massachusetts economist Dale Ballou, the higher the quality of an individual’s undergraduate institution, the less likely a student is to choose a teaching career. Moreover, during the past 25 years the share of master’s and doctoral degrees in education granted by top-tier public and private research universities has declined dramatically. And of students who graduated from college in 1993 and 1994, data from the Baccalaureate and Beyond survey show that those who entered the public school teaching profession averaged a 923 on the SATs; the average SAT of those entering other professions was about 80 points higher. The results are even more dramatic when one compares the SATs of teachers with those of people entering technical professions, such as engineering.

The demographic shifts of the next decade provide the opportunity to thoroughly remake the teaching profession. This is a significant opportunity, given that the evidence suggests that teacher quality is the most important school factor in explaining differences in student performance. The difficulty from a policy perspective is that the relationship between readily quantifiable attributes–such as a teacher’s highest degree attained or level of experience–and student outcomes is tenuous at best. In other words, good teachers certainly make a difference, but it’s unclear what makes for a good teacher.

Overall Impact

Research dating back to the 1966 release of Equality of Educational Opportunity (the “Coleman Report”) shows that student performance is only weakly related to school quality. The report concluded that students’ socioeconomic background was a far more influential factor. However, among the various influences that schools and policymakers can control, teacher quality was found to account for a larger portion of the variation in student test scores than all other characteristics of a school, excluding the composition of the student body (so-called peer effects).

Much of the research published since the Coleman Report has confirmed the finding that high-quality teachers raise student performance–indeed, it appears that the most important thing a school can do is to provide its students with good teachers. The Coleman Report’s finding was based on the influence of a set of quantifiable teacher characteristics, such as years of experience, education levels, and performance on a vocabulary test. Since then, due in large part to the availability of new data sources that link and track teachers and students over a number of years, researchers have been able to estimate the overall contribution of teachers to student learning. This includes not only the effect of easily measurable attributes, such as experience and degrees obtained, but also the effect of harder to measure intangible attributes, such as a teacher’s enthusiasm and skill in conveying knowledge.

Tennessee is one of the few states with data systems in place that track teachers over time and link them to their students’ achievement scores. Researchers who have studied the Tennessee data, mainly former University of Tennessee professor William Sanders and his colleagues, found that the effectiveness of teachers has more of an influence on student achievement than any other schooling factor. They also found a wide range of effectiveness among teachers; there are some very good teachers, some very bad teachers, and a wide range of performance between them. Using the “Sanders methodology,” teachers are placed into effectiveness quintiles based on their students’ growth in achievement, or the “value added” by the teachers. Those teachers who fall into the first quintile, the least effective teachers, were found to elicit average student gains of roughly 14 percentile points a year. The most effective teachers elicited an average gain of 52 percentile points a year. The effects of teacher quality were also found to persist for years after a student had a particular teacher.

More recently, researchers have sought to isolate teachers’ contribution to student performance and assess how much of their overall contribution can be associated with measurable teacher characteristics, such as experience and degree level. Economists Eric Hanushek, John Kain, and Steven Rivkin estimated that, at a minimum, variations in teacher quality account for 7.5 percent of the total variation in student achievement–a much larger share than any other school characteristic.

This estimate is similar to what my colleagues and I found: that 8.5 percent of the variation in student achievement is due to teacher characteristics. We found that the vast majority (about 60 percent) of the differences in student test scores are explained by individual and family background characteristics. All the influences of a school, including school-, teacher-, and class-level variables, both measurable and immeasurable, were found to account for approximately 21 percent of the variation in student achievement. This 21 percent is composed mainly of characteristics that were not directly quantified in the analyses. Since we used statistical models that included many observable school-, teacher-, and class-level variables–such as school and class size, teachers’ levels of education and experience, and schools’ demographic makeup–it is clear that the things that make schools and teachers effective defy easy measurement.

Only about 3 percent of the contribution teachers made to student learning was associated with teacher experience, degree attained, and other readily observable characteristics. The remaining 97 percent of their contribution was associated with qualities or behaviors that could not be isolated and identified (see Figure 1).

Which Attributes?

What does the empirical evidence have to say about specific characteristics of teachers and their relationship to student achievement? First it is important to note that studies focus on different grade levels, subjects, and types of students taught, and in some cases the estimated effects of particular attributes are not consistent across the board. Furthermore, studies vary considerably in quality. Experiments, which provide the most credible results, are rare in education, and relatively few studies that address students’ outcomes observe the professional norm of having detailed controls for students’ background characteristics (including previous academic achievement). That said, here is a broad overview of the findings for various teacher attributes.

Teacher degree and experience levels: Teachers’ education (degree) and experience levels are probably the most widely studied teacher attributes, both because they are easy to measure and because they are, in the vast majority of school systems, the sole determinants of teachers’ salaries. However, there appears to be only weak evidence that these characteristics consistently and positively influence student learning.

In a 1986 literature review (and in follow-up reviews in 1989 and 1997) that has framed much of the debate, Hoover Institution senior fellow Eric Hanushek showed that only a small proportion of studies find these teacher characteristics to be statistically significant in the expected direction.

Not all studies reach the same conclusion. In a 1996 meta-analysis published in the Review of Educational Research, Rob Greenwald and his colleagues concluded that “school resources are systematically related to student achievement and that these relations are large enough to be educationally important” and “resource variables that attempt to describe the quality of teachers (teacher ability, teacher education, and teacher experience) show very strong relations with student achievement.”

It’s true in both sets of reviews that many more studies found statistically significant positive effects for teacher experience than found statistically significant negative effects, but the statistically significant positive findings were found in only about 30 percent of the studies. There are also statistical shortcomings in many of the studies cited by both Hanushek and Greenwald et al.

Part of the explanation for the mixed findings may be that experience and degree level matter only in certain circumstances. For example, there is little evidence that experience beyond the first couple of years in the classroom makes one a better teacher. And teacher experience implicitly captures the effects of the prevalent graduation requirements and labor market conditions at the time when teachers were hired. Furthermore, as I describe below, the effect of degrees appears to hinge on the subjects that are taught and whether the degrees are specific to those subjects.

Subject-matter knowledge: The evidence is somewhat mixed, but it suggests that teachers’ knowledge of their subject matter, as measured by degrees, courses, and certification in that area, is associated with high performance. Studies with more detailed measures of teachers’ education levels and coursework in subject areas found that, at least in math and science, academic preparation does positively influence student achievement. Having an advanced degree in subjects outside of math and science, however, does not appear to affect student achievement. Taking additional courses in one’s subject has some effect, but it depends on the level of the courses a teacher is teaching. Perhaps not surprisingly, in a 1994 study David Monk and Jessica King Rice found that the teaching of higher-level courses seems to require greater knowledge of subject matter than does the teaching of lower-level courses.

Teachers’ pedagogical knowledge: The value of teaching teachers how to teach, or pedagogy, is more hotly debated. Since there is little research directly assessing the influence of pedagogical training on student outcomes, this debate tends to focus on the impact of teachers’ performance on licensure exams and the merits of licensing teachers. A 1986 study found that the average National Teacher Exam scores of all the teachers in a school district have a strong positive correlation with the average performance of students on standardized tests. It is not clear, however, whether this reflects the influence of subject-matter knowledge or pedagogical training, since the National Teacher Exam tests both. A number of studies have found that fully certified teachers influence student achievement positively. For instance, a 1985 review by Carolyn Evertson, Willis Hawley, and Marilyn Zlotnik found that 11 of 13 studies judged regularly certified teachers to be more effective than those who held a provisional or emergency certificate. But only four of these studies were based on students’ outcomes and most of them were more than 25 years old, which means they predated the “value added” methodology of assessing educational effects that is now standard practice. More recently, Stanford professor of education Linda Darling-Hammond analyzed state-level data and concluded, “The most consistent highly significant predictor of student achievement in reading and mathematics in each year tested is the proportion of well-qualified teachers in a state: those with full certification and a major in the field they teach.” This study, however, potentially suffers from a statistical problem known as aggregation bias. There are many factors that influence achievement at the state level, many of which cannot be identified and controlled for, and it therefore may be inappropriate to conclude that the test-score results are linked to the proportion of teachers with specific characteristics in a state.

Dominic Brewer and I analyzed teacher certification and its relationship to student achievement at the level of individual teachers and found that teachers with standard certification outperform those who are not certified in their field, but they do not appear to be more effective than those who hold emergency credentials (teachers who have not satisfied all of the requirements necessary to obtain a standard certificate). One reason why findings are inconsistent is the lack of a standard system of licensure; each state has a unique set of requirements for individuals who wish to enter the teaching profession, and some states’ requirements may be more rigorous than others.

What is to be concluded about the effect of a teacher’s level of education and certification status? As Carolyn Evertson and her colleagues write, “Investigations of teacher education do not represent a strong body of research.” Most of the studies that find statistically significant relationships between teacher training and student achievement find that the effects of these characteristics are small and specific to certain contexts. If a large, consistent association between teacher training and student achievement existed, it probably would not be all that hard to detect.

Other teacher attributes: Recent studies suggest that measures of teachers’ academic skills, such as SAT or ACT scores, tests of verbal ability, or the selectivity of the colleges they attended, may predict their effectiveness more accurately than the characteristics discussed above. However, here too the evidence and the estimated relationships are relatively weak. Greenwald et al.’s 1996 meta-analysis found that teachers’ academic skills were shown to have a positive relationship to student achievement in 50 percent of the studies they analyzed, a much higher proportion than for teacher education or experience. Studies by Ronald Ferguson and Helen Ladd found positive relationships between aggregate teacher scores on the ACT, literacy examinations, or states’ licensure examinations and aggregate student performance on standardized tests. However, the fact that these studies were done at the aggregate (school or school district) level casts some doubt on them. It’s unclear whether higher-scoring teachers lead to higher-scoring students or whether affluent districts, which tend to have higher-achieving students, also tend to hire teachers with higher scores.

In a 1994 study, Ronald Ehrenberg and Dominic Brewer found that students score higher on standardized exams if their teachers attended more selective undergraduate institutions. In a separate study, Ehrenberg and Brewer reexamined the Coleman data and found a significant positive association between teachers’ verbal ability and student outcomes. As Hanushek wrote in 1989, “Perhaps the closest thing to a consistent conclusion across studies is the finding that teachers who perform well on verbal ability tests do better in the classroom.”

Little Guidance

Good teaching is clearly important to raising student achievement. In fact, most research suggests that the benefit of improving the quality of the nation’s teaching workforce is far greater than other policy interventions, such as lowering class size.

However, while we know that good teaching is important, it’s far less clear what makes for a good teacher. The measures of teacher quality that are used by most public school systems to screen candidates and determine compensation–certification, experience, and education level–have been well researched, but there is little definitive empirical evidence that these characteristics, defined in general terms, are associated with higher student achievement. Teachers’ educational levels appear to make a difference when the education is related to the subject taught, but advanced degrees do not appear to serve as a good measure of quality in general. There is also some evidence that experienced teachers are more effective with students, but the benefits of additional years of experience appear to level off early in a teacher’s career. Measures of teachers’ academic skills, such as their verbal ability, may more accurately predict their effectiveness, but there is far less evidence on this issue, and these findings are also not conclusive. There is little evidence on the issue of teacher certification as well, and the evidence that does exist is mixed.

What does the empirical evidence imply for policymaking? First, the importance of teacher quality cannot be overstated. Teachers can have a profound effect on students, and school systems make a significant long-term investment when they hire teachers. Unlike other education investments, such as class size, which may be easily altered from year to year, the tenure system implies that the employment of an individual teacher is near permanent. For these reasons, the selection of teachers is of paramount importance. I would argue that this function of school systems receives too little attention at the local level.

Second, the lack of clear evidence on the effectiveness of teacher certification suggests that policymakers should continue experiments, such as Teach for America, that allow individuals to enter the teaching profession through alternative routes. To the degree that superintendents, principals, and hiring committees accurately identify top-notch candidates through these alternative routes, such programs may serve as an important supply of future teachers. Alternative-route programs should be studied to determine the extent to which state-level restrictions on entry into the market are justified.

Finally, the compensation structure used by virtually all school districts is not well aligned to promote the acquisition of skills found to influence student outcomes. For instance, data from the National Center for Education Statistics show that salary schedules provide pay premiums of about 11 percent for master’s degrees and 17 percent for a doctorate. Generally, these premiums are received regardless of whether degrees are specific to the subjects in which teachers teach, despite the evidence that out-of-field degrees contribute little toward student achievement. The compensation structure also does not provide policymakers with tools to address areas of shortage, to reward job performance or the acquisition of skills deemed to be important, or to compensate for the difficulty of a teaching assignment. Do alternatives to the compensation structure currently in place in public schools work? Unfortunately, there is very little evidence to answer this question. However, given the concern about the quality of the existing teacher workforce, experimenting with alternative pay structures may be worthwhile.

-Dan Goldhaber is a senior research associate at the Urban Institute in Washington, D.C.

The accountability debate tends to devolve into a battle between the pro-testing and anti-testing crowds. But when it comes to the design of a school accountability system, the devil is truly in the details. A well-designed accountability plan may go a long way toward giving school personnel the kinds of signals they need to improve performance. However, a poorly designed scheme, which ignores the statistical properties of schools’ average test scores, may do more harm than good.

The recent debate over the reauthorization of the federal Elementary and Secondary Education Act (ESEA) is a case in point. From his first days in office, President Bush promised to make education reform a centerpiece of his administration, using the reauthorization of the ESEA as an opportunity to give the state-led accountability movement a dramatic shove forward. Within six months of his taking office, both houses of Congress had passed bills that imposed new federal standards for the states’ accountability efforts.

However, both bills were seriously flawed. They created standards that, over time, would have identified nearly every school in the nation as “low performing,” forcing them to spend precious resources developing unnecessary school-improvement plans. A tide of paperwork would have crowded out time for learning. This almost turned the most significant federal foray into education policy in decades into an embarrassment. Changes were made by a House-Senate conference committee, so the law, as enacted, remedied the most glaring problems, but created others. The saga illustrates the difficulties of designing an effective accountability system.

The House and Senate Bills

At the heart of both bills was a detailed formula for determining when a school is making “adequate yearly progress.” The consequences for schools that failed to meet their performance targets were progressively severe–after one year, districts would be required to offer public school choice to all the students in a school; after several years, districts would be required to replace school staff, convert the school into a public charter school, or hand the school over to a private contractor.

The problem is that such consequences place too much weight on single-year changes in test scores at the school level. Either bill would have required an increase in the proportion of students scoring above the proficient level in both math and reading, each and every year. However, test scores at the school level often fluctuate for reasons other than any underlying change in a school’s performance. Such volatility arises from two sources. The first is variation due to differences in the groups of students being tested each year. Even if the students are being drawn from the same families and the same neighborhoods, the average performance of a school can fluctuate from year to year depending on the attitudes and abilities of the students in each cohort. The average elementary school contains only 68 students per grade level. With a sample this small, having five particularly bright students (or a few students with undiagnosed learning disabilities) in any one year can lead to large fluctuations in a school’s test scores from one year to the next. The Department of Labor measures the monthly unemployment rate with a sample of nearly 60,000 households. Congress was proposing that the Department of Education measure the performance of the typical elementary- school grade with a sample nearly 1/1000 the size.

The second source of variation is one-time factors that lead to temporary fluctuations in test performance. Some of these factors are likely to be unrelated to the educational practices of a school. For instance, a dog barking on the day of the test, a severe flu season, or one particularly disruptive student in class could cause scores to fluctuate. There may be other sources of volatility that are more related to the educational mission of a school, such as the favorable chemistry between a teacher and a particular group of students or teacher turnover. Whatever the source of variation, single-year changes in test performance are very unreliable indicators of where a school is headed over the long term.

Consider the examples of North Carolina and Texas. Between 1994 and 1999, these states were the educational envy of the nation, raising proficiency rates in math and reading by 2 to 5 percentage points in the average year. However, the vast majority of schools in those states exhibited much less consistent progress: less than 2 percent of schools witnessed an increase in math and reading proficiency each and every year for those five years. Indeed, we estimate that between 98 and 100 percent of the elementary schools in North Carolina and Texas would have failed the House and Senate’s initial definitions of annual yearly progress at least once between 1994 and 1999.

Furthermore, both bills would have compounded the error by requiring annual increases in test scores for every racial subgroup in a school. The intent was admirable: to ensure that schools do not ignore minority children. But this provision was likely to have harmed its intended beneficiaries, by arbitrarily sanctioning schools that enroll students from several different racial or ethnic subgroups. Suppose that a school is solidly on the path to improvement, with a 70 percent chance of increasing the proficiency of any racial subgroup in a given year. A school with two racial subgroups in its student body would have a less than 50-50 chance of achieving an increase for both groups in a given year–because the year-to-year fluctuations are nearly independent for each racial group (therefore the probability is .70 times .70, or .49). The odds would be even longer for a school with three racial subgroups (.70 times .70 times .70, or .34). Since African-American and Latino students are more likely to attend schools with more than one racial group, they are more likely to see their education disrupted arbitrarily.

A number of states have established accountability programs that track the performance of racial and ethnic subgroups separately. For example, California requires schools to meet certain growth targets for all “numerically significant” subgroups in a school. In order to be numerically significant, a group must either represent at least 15 percent of the student body and have more than 30 students or have more than 100 students regardless of what percentage they are. There are eight different groups that can qualify as numerically significant, depending on the number of students in each group in a school: African-American, American Indian (or Alaska Native), Asian, Filipino, Hispanic, Pacific Islander, white non-Hispanic, and “socioeconomically disadvantaged” students.

We calculated the likelihood of a California school’s winning a Governor’s Performance Award by size and by the number of numerically significant subgroups. Among the smallest quintile of elementary schools, 47 percent of racially heterogeneous schools (those with four or more racial subgroups) won performance awards, versus 82 percent of similarly sized but racially homogeneous schools. This is particularly ironic given the fact that overall growth in performance was slightly higher for more-integrated schools between 1999 and 2000. Moreover, the reason for a school’s failure to win an award was often not that African-American and Latino students were lagging behind, but that white non-Hispanic students experienced slower growth in achievement: the average school with multiple racial subgroups witnessed larger gains for African-American and Latino students than for white students.

Separate achievement targets for racial and ethnic subgroups seem to be neither necessary nor especially effective in coaxing schools to focus on the performance of racial and ethnic minorities. In North Carolina, where there are no separate racial targets, African-American and Latino students experienced slightly higher improvements in proficiency than white non-Hispanic youth. Until this year, the rating system in Texas specified separate targets for racial subgroups that accounted for more than 10 percent of the student body (and more than 30 students). However, African-American and Latino students saw the same improvements in their test scores whether or not they attended schools with enough minority students to require a separate racial target.

Remaining Problems

The conference committee’s compromise bill remedied some problems but created new ones. Earlier versions of the legislation rated schools according to their year-to-year improvements in the share of their students who achieve a certain proficiency level. Now schools will simply need to have a certain minimum percentage of their students (and of racial, ethnic, and socioeconomic subgroups within each school) deemed “proficient” each year. The initial minimum proficiency rate will be the greater of the proficiency rate of the 20th-percentile school or the average statewide proficiency rate of the lowest-scoring subgroup. In many states, the effective minimum will be the proficiency rate of the 20th-percentile school. However, more than 20 percent of all schools are likely to fail, because the threshold will apply not only to the school as a whole but also to all the subgroups in a school. If any racial, ethnic, or socioeconomic subgroup within a school fails, the school fails. As a result, a disproportionate number of the schools that enroll disadvantaged minority subgroups are likely to fail. The minimum proficiency rate that schools are required to meet will be raised gradually to 100 percent over the next 12 years.

The main beneficiaries of the conference committee’s changes will be suburban schools whose initial rates of proficiency are above the minimum, for they will no longer be penalized for temporary downward fluctuations in scores. The primary losers will be schools with initially low levels of proficiency for any subgroup. They will now be required to achieve a 10-percentage-point increase in proficiency for those subgroups to avoid the sanctions in a given year.

One flaw in the new formula is that it provides a strong incentive for states to lower the score students must exceed on their state tests in order to achieve “proficiency.” The problem is that redefining proficiency simply because of the new federal requirements may create a credibility problem for the standards movement in a number of states.

Another problem not remedied in the final bill is that any federal definition of adequate yearly progress is likely to conflict with at least one of the state accountability plans that are already in place. There are three common variants in state accountability systems: some states, such as North Carolina, Arizona, and Tennessee, rate their schools with a measure of a school’s value-added, using the growth in performance for a given group of students since the end of the preceding school year; other states, such as Texas and Illinois, rate their schools on the percentage of students scoring above certain thresholds; still other states, such as California, rate their schools based on their change in test scores from one year to the next. (A fourth category of states rates schools based on some mixture of value-added, levels, or changes.) Thus states that have been rewarding schools based on value-added measures or on changes in scores may be required to sanction the very schools they have been rewarding.

The next battleground is likely to be the issue of how many students it takes to create a separate racial, ethnic, or socioeconomic subgroup for accountability purposes. The legislation only requires that there be a sufficient number of students to yield

statistically reliable information in order for the subgroup to count separately. The higher the threshold–say, requiring a subgroup to represent at least 15 percent of the student body, as opposed to 5 or 10 percent–the lower the failure rate will be for schools with small percentages of disadvantaged minority students.

Design Principles

State and federal officials ought to keep three basic principles in mind in designing test-based accountability systems:

•Multiple years of data are required to measure improvements in performance reliably.
Children arrive at school with widely varying levels of preparation. Even a mediocre school can expect high test scores if its students come from wealthy backgrounds. As a result, policymakers in many states have attempted to level the playing field by focusing on improvements in test scores. However, improvements are very difficult to discern with a couple of years’ worth of data, for two reasons. First, schools differ much less in the extent to which they improve test scores from year to year than they do in their beginning level of performance. Second, any measure of change in performance is likely to amplify the effect of sampling variation and other one-time factors that lead to fluctuations in performance. In other words, identifying improvements in performance as opposed to levels of performance in a single year is like looking for a smaller needle in a bigger haystack. If policymakers intend to measure and reward improvements in test performance at the school level, they will need to rely on multiple years of data.

Improvements can be measured in two basic ways: the improvement in performance for a given group of students from one year to the next (known as a value-added approach), or the improvement in performance across different groups of students (which we will refer to as cross-cohort changes). The improvement of scores for at least two contiguous grades (for example, grades 4 and 5) from one year to the next is a mixture of value-added changes (the 4th grade students who become 5th graders) and cross-cohort changes (the 4th grade students this year are a different group from the 4th grade students last year).

Kane and Staiger have analyzed the statistical properties of value-added and cross-cohort changes in test scores, using data from North Carolina (see Figure 1). (Full citations are available at www.educationnext.org.) We measured value-added with the average change in combined reading and math scores for a school’s students between the end of 3rd grade and the end of 4th grade; we measured cross-cohort changes with the change in 4th grade scores from one year to the next. Among median-size schools in North Carolina, roughly half of the variance between schools in value-added in 4th grade math and reading was due to sampling variation and other one-time factors. For the smallest quintile of schools, the percentage of variance due to non-persistent factors was even higher (58 percent), while for the largest quintile of schools the percentage was somewhat lower (29 percent). Cross-cohort changes in mean test scores from one year to the next were measured even more unreliably. More than three-quarters of the variance in the annual change in mean test scores among the smallest quintile of schools was due to one-time, non-persistent factors. This percentage was only slightly smaller (73 percent) for the largest quintile of schools. Such volatility can wreak havoc when rewards and punishments are doled out on the basis of changes in test scores; school personnel are at risk of being punished or rewarded for results that are beyond their control.

Therefore, when policymakers seek to reward schools for improvements in test scores, they should do so based on multiple years rather than a single year of data. Moreover, while a simple arithmetic average of improvements over multiple years would be an improvement, there are even more efficient ways to pool information over time. For instance, building on work by McClellan and Staiger (1999) in rating hospital performance, we have proposed a simple technique for pooling information over time, which improves on a simple arithmetic mean by taking into account the amount of “signal” and “noise” in a given measure of performance. For instance, for large schools, for which we would expect less noise in any given year’s measure, the proposed method would place more weight on more recent scores; for small schools, the method would place more equal weights on each of several years’ worth of scores.

•Incentives targeted at schools with test scores at either extreme–rewards for those with very high scores or sanctions for those with very low scores–affect primarily small schools and provide very weak incentives for large schools.

Each year since 1997, North Carolina has recognized the 25 elementary and middle schools in the state with the highest scores on the “growth composite,” a measure reflecting the average gain in performance among students enrolled at a school. Winning schools receive financial awards.

One indicator of the volatility of test scores is the rarity of repeat winners. Between 1997 and 2001, 101 awards were handed out for schools ranking in the top 25. (One year, two schools tied at the cut-off.) These 101 awards were won by 90 different schools, with only 9 schools winning twice and only 1 school winning three times. No school was in the top 25 in all four years.

Of the 840 elementary schools we analyzed, 59 were among the top 25 at some point between 1997 and 2000 (the top 25 each year included middle schools, which we are not analyzing here). Among all the schools, the average gain score was not strongly related to school size, but the variance between schools was much larger for small schools. The variance in mean gain scores among schools in the smallest size decile was nearly five times the variance among the largest decile of schools (.048 compared with .011). As a result, schools in the smallest decile were much more likely to be among the top 25 schools at some point over the period: Even though their mean gains were not statistically different, the smallest schools were 23 times more likely to win a top-25 award than the largest schools.

For the very same reason, small schools are also overrepresented among those with extremely low test scores. Beginning in 1997, the state assigned assistance teams to intervene in schools that performed poorly on state tests and failed to meet their growth targets from the previous year. All but one of the elementary schools assigned an assistance team were among the smallest 40 percent of schools. (The smallest decile of schools would have received an even larger share of the assistance teams, except for a rule requiring that the proportion of students scoring below grade level be statistically significantly less than 50 percent.)

This year, the state of California distributed $100 million to teachers in schools that started with test scores in the bottom half of schools in 1999 and achieved large gains in performance between 1999 and 2000. A thousand teachers in schools with the largest improvements received $25,000 bonuses on average. Small schools in California were considerably more likely to win one of these awards than were larger schools. Given the importance of sampling variation and the fact that the largest bonuses were reserved for teachers in schools with the most extreme increases in test scores, this is hardly a surprise.

A threshold at either extreme is likely to be irrelevant for large schools, since they are unlikely to experience such large swings in performance regardless of their efforts. If the marginal costs of improving are also higher at large schools, the problem of weak incentives for large schools would only be compounded. A remedy would be to establish different thresholds for schools of different sizes. For example, grouping schools according to size (as is done in high-school sports) and giving awards to the top 5 percent in each size class would tend to even out the incentives (and disparities) between large and small schools. An alternative solution would be to establish thresholds closer to the middle of the test-score distribution, where the disparity for large and small schools is less extreme.

Helen Ladd and Charles Clotfelter in 1996 and David Grissmer et al. in 2000 reported evidence suggesting that schools respond to incentives by raising student performance. However, the long-term effects of incentives may be quite different from their short-term effects. Even if teachers are not sufficiently aware of the statistical forces at work to recognize their rather limited influence on test scores in the short run, they may well become aware of this over time. If their best efforts are rewarded with failure one year and less work the following year is rewarded with success, they are likely to form negative opinions regarding the value of their efforts.

•When evaluating the impact of policies on changes in test scores over time, the natural fluctuations in test scores must be accounted for.
In 1997, North Carolina identified 15 elementary and middle schools with poor performance in both levels and gains and assigned “assistance teams” of three to five educators to work in these schools. The next year, all of the schools had improved enough to escape being designated “low performing.” The state Department of Public Instruction ascribed the improvements to the efforts of the assistance teams; the assistance teams were lauded in Education Week‘s annual summary of the progress of school reform efforts in the states as well. However, given the amount of sampling variation and other non-persistent fluctuations in test-score levels and gains, schools with particularly low test scores in one year would be expected to bounce back in subsequent years.

The schools that were assigned assistance teams seem to have had a particularly bad year the year they received the sanction. In the year before assignment, such schools had an average 4th grade combined reading and math test score that was .67 student-level standard deviations below the average school. This reveals that they were weak schools the year before being sanctioned. However, in the year of assignment, their average score was even lower, .79 student-level standard deviations below the average school. The year after assignment, their scores seemed to rebound to .52 student standard deviations below the mean. One is likely to greatly overestimate the impact of assistance teams by taking the change in performance in the year after assignment.

There are real differences in performance at the school level. And schools that are not improving should be identified for intervention. However, one year’s worth of test-score data is insufficient to discern such differences in a meaningful way. States should be allowed to experiment until the nation finds the ideal way to determine which schools are making adequate yearly progress. We understand the impulse to create a system that requires specific remedies sooner rather than later. However, impatience is an insufficient excuse for bad education policy.

-Thomas J. Kane is a professor of policy studies and economics at the School of Public Policy and Social Research at the University of California at Los Angeles. Douglas O. Staiger is an associate professor of economics at Dartmouth College. Jeffrey Geppert is a senior research analyst at the National Bureau of Economic Research in Palo Alto, California. A portion of this article is drawn from a chapter that will appear in the Brookings Papers on Education Policy 2002 (Brookings, 2002).

Since 1990 the New York-based Teach for America (TFA) program has placed more than 7,000 teachers in some of the nation’s most challenging school districts. The nonprofit organization recruits high-achieving seniors from top colleges and asks them to commit themselves to two years of teaching in inner-city or rural schools. TFA currently supplies teachers to 18 districts across the country, including Newark, New Jersey; rural districts in the Mississippi Delta; Phoenix, Arizona; and Houston, Texas.Most TFA recruits serve in schools that qualify for funding under Title I of the Elementary and Secondary Education Act due to their high concentrations of students living in poverty. These schools often find it difficult to fill teaching positions with strong candidates.

In number of teachers trained each year, TFA is one of the nation’s largest suppliers of teachers. According to the American Association of Colleges for Teacher Education, which represents university- or college-based teacher-preparation programs, only 10 percent of programs offering undergraduate-level training produce more teachers each year. TFA recruits clearly help to alleviate teacher shortages in the school districts in which TFA operates. But how do these recruits perform once they are hired? The evaluation results reported here provide the first evidence of TFA recruits’ actual performance in the classroom.

TFA’s underlying idea is that many smart, accomplished college graduates would make significant contributions to public education-if it weren’t for the fact that they must have majored in education or be willing to spend one to two more years (and significantly more in tuition) earning a master’s in education in order to get certified. Even if these bright, young, committed individuals don’t remain in teaching forever, TFA posits that they can make a difference in some of the nation’s most challenging schools. TFA gives what the program considers potentially excellent teachers-those with both solid knowledge of content and strong leadership skills-a low-cost opportunity to earn a teaching credential while discovering whether they have found a calling. The emphasis on knowledge and leadership is reflected in TFA’s recruiting: a typical TFA corps member earned a grade-point average of 3.4 out of 4.0, and 87 percent of recruits have leadership experience.

TFA recruits undergo five weeks of training during the summer. They complete intensive preservice coursework and spend the balance of the summer in classrooms as student teachers or team teachers. Once school starts in the fall, TFA recruits must participate in weekly TFA-sponsored professional development workshops-more than the typical new teacher. This is in addition to any coursework they must complete under their district or state’s alternative certification program. In Houston, the TFA site examined for this study, all uncertified teachers must enroll in the district’s alternative certification program at the beginning of their first year. It takes a year to earn a credential.

Advocates of alternative certification hail TFA for its creative response to teacher shortages and for its efforts to recruit star college graduates who otherwise might not have found their way to teaching. Many corps members leave teaching after fulfilling their two-year commitment, but they often remain in the education field-as school administrators, in government, or with nonprofits that are involved in education reform. In April 2001 Education Week reported, “It’s becoming clear that Teach For America is channeling a wealth of talent, energy, and creativity into educational leadership that might otherwise have wound up in such fields as medicine, law, or business.”

But TFA has its detractors as well. They point to the lack of pedagogical training and knowledge of child development theories among TFA teachers. They also tend to believe that TFA demeans teaching by treating it as a Peace Corps-style rescue mission rather than a true profession, with salaries appropriate to attracting solid candidates. “A frankly missionary program,” wrote Stanford professor Linda Darling-Hammond in an oft-cited 1994 Phi Delta Kappan article, “TFA has recruiters and advocates who have focused much of their attention on the advantaged college graduates for whom TFA serves as something useful to do on their way to their-real jobs’ in law, medicine, or business.”

Into the Breach

So far, these battle lines have been drawn with only rhetoric as a weapon, for evidence on the performance of TFA teachers in the classroom has been nonexistent. To help fill this vacuum, CREDO (formerly known as the Center for Research on Education Outcomes), an independent nonpartisan research group at the Hoover Institution of Stanford University, evaluated the performance of TFA teachers relative to both new teachers and all teachers in the Houston Independent School District. Houston is the 7th-largest school district in the United States, encompassing 186 elementary schools and 34 middle schools. Houston hires approximately 350 new elementary- and middle-school teachers a year; since TFA began supplying teachers to the district, its proportion of new teachers has ranged between 5 and 10 percent (see Figure 1). Latino (54 percent) and African-American (33 percent) students make up the majority of the district’s student population. The majority of students live in relatively poor households, with 75 percent receiving free or reduced-cost lunches.

To perform the analysis, the year-end learning gains of students with TFA teachers were measured against the gains of students with non-TFA teachers. Individual scores on the annual Texas Assessment of Academic Skills (TAAS) in mathematics and reading and English language arts were used as the measure of student performance. Texas has annually tested students in grades 3 through 8 since 1993. Having multiple years of test scores for the same student allowed us to estimate the annual progress of individual students by controlling for their previous year’s TAAS score.

Linked student- and teacher-level data were obtained through the Texas Schools Project at the University of Texas at Dallas. Two groups of students were studied: elementary-school students in grades 3 through 5 (where most students have a single teacher throughout the year) and middle schoolers in grades 6 through 8 (where students tend to have a different teacher in each subject). This article presents the findings for the elementary grades only. The analysis of middle-school teachers produced similar results, but faced methodological challenges arising from students’ having had multiple teachers. (For a description of the middle-school analysis, see the full report at: http://credo.stanford.edu/working_papers.htm.)

Our study was framed around two separate but related questions. First, we wanted to know whether TFA recruits were effective teachers. So we compared the performance of all TFA teachers with that of all non-TFA teachers in Houston, regardless of how experienced they were. The relevant comparison group was all Houston teachers in grades 3 through 5 from 1996 to 2000.

The second analysis was focused on the question that districts must ask when considering whether to work with TFA: How do TFA recruits measure up against other potential new hires? In other words, if a district had one remaining position to fill, would it be better off with a new TFA teacher or another candidate? For this analysis, we compared TFA teachers with one or two years of experience with other teachers in the district with the same amount of experience.

It is important to note that the group of new teachers with which TFA teachers were compared included teachers with other forms of alternative certification along with a number who were not fully certified. Such teachers need to be included because the purpose of this study is to compare TFA teachers with the pool of potential hires available to the district. As noted above, in both the comparisons of TFA teachers with all teachers and of TFA teachers with only new teachers, TFA teachers represented a small proportion of the total number of teachers. Achieving statistically significant results requires very large impacts under such circumstances, and so our findings, though stable, do not always reach a level that passes the conventional significance threshhold.

Our study examined the performance of TFA and non-TFA teachers from two perspectives. First, we made a straightforward comparison of the average test-score gains in classrooms run by TFA and non-TFA teachers, controlling for a variety of factors known to influence academic achievement, including students’ backgrounds, the students’ previous performance on the TAAS, characteristics of their schools, and characteristics of their classmates. In this way, we could isolate the typical effect of having a TFA teacher versus a non-TFA teacher. Second, we wanted to examine the distribution of performance among TFA and non-TFA teachers. For instance, how do high-performing TFA teachers compare with high-performing non-TFA teachers? How bad are the worst TFA teachers? The worst non-TFA teachers? Do the TFA teachers vary much in quality, or are they fairly similar? To address these questions, we estimated how much of the achievement gains in each classroom was attributable to that classroom’s teacher.

Our study analyzed and controlled for the following factors:

Teacher Characteristics

-TFA status
-Years of teaching experience

Student Characteristics

-The school attended
-School year
-Minority status
-Eligibility for free or reduced-cost lunch (a proxy for low socioeconomic status)
-Date of birth
-Assigned teacher
-English language proficiency
-Testing exemption status
-Test scores on the annual TAAS exam in each year in which the student was enrolled in a Texas school

School Characteristics

-Share of students who are African-American
-Share of students who are Latino
-Share of students receiving free or reduced-cost lunch

Classmate Characteristics

– Share of a class receiving free or reduced-cost lunch
– Share of a class that scored below the state mean on the previous year’s TAAS exam

Findings

Before the analysis was concluded, students’ raw scores on the TAAS exam were standardized to a mean of 0 and a standard deviation of 1. This procedure was performed separately for each year in order to account for variations in the TAAS test from year to year. The average score for a given year was transposed to 0, and all the other scores were distributed around that point in a standard normal distribution. Gains are therefore expressed in terms of standard deviations. A gain of 5 percent of a standard deviation in one year may be considered large, if it is thought that students will continue to experience similar annual gains through the elementary and secondary years. (A full standard deviation is approximately the difference between the test scores of black and white students nationally. More than half of the black-white test-score gap could be erased by steady gains of 5 percent a year among black students.)

Reading. In the elementary grades 3 through 5, students of new Teach for America teachers gained an average of 5.8 percent of a standard deviation more on the TAAS reading exam than did students with other new teachers, a difference that fell just short of statistical significance (see Figure 2). The students of the entire sample of Teach for America teachers made gains that were essentially similar to those made by students of all teachers in the Houston district.

There tended to be less variation in the performance of TFA teachers than there was among non-TFA teachers. More than 63 percent of TFA teachers generated achievement gains in reading that were higher than the median achievement gains for new non-TFA teachers (see Figure 3). More than 60 percent of TFA teachers did better than the median performance of all teachers. Moreover, TFA teachers tended to be more bunched up around the median than their non-TFA peers. In other words, there were fewer extremely low- and high-performing teachers among the Teach for America recruits; they were more consistent in eliciting achievement gains. The difference between the two distributions was found to be statistically significant in both cases.

Mathematics. Elementary students benefited from having a TFA teacher in math as well. The achievement gains of students with new TFA teachers were 12 percent of a standard deviation higher than those with other new teachers, a result that was statistically significant (see Figure 2). Students of all TFA teachers gained 2.9 percent of a standard deviation more in math than did students of all teachers in the Houston district, a difference that was not statistically significant.

Let’s look again at the distribution of performance among TFA and non-TFA teachers. Of new TFA teachers, 64 percent generated achievement gains among their students that were higher than the median achievement gains for new non-TFA teachers (see Figure 3). Fifty-eight percent of TFA teachers performed above the median for all teachers. The lowest-performing new teachers were much worse than the lowest-performing TFA teacher. The best-performing TFA teachers were better than virtually all the teachers in the district. In other words, if you were choosing between two math teachers, and the only thing you knew about them was that one was a TFA member and one was not, you would choose the TFA member. This would give you the best chance of selecting a good teacher.

Our results also suggest that years of teaching experience significantly affect student performance. The largest gains in teacher effectiveness occur early in a teaching career and diminish thereafter until flattening out at around 8 years for math and 11 years for reading. The effect of experience operates independently of teachers’ characteristics, so by isolating these effects, we provided a truer test of differences in teachers’ backgrounds.

More and Better

TFA appears to be a viable alternative source of high-quality teachers, compared with both the pool of new teachers available for hire in Houston and the district’s entire workforce of current teachers. All our results show the average TFA teacher improving her students’ performance by more than new teachers and at least as much as all teachers in Houston. The advantage of having a TFA teacher was largest in math, though TFA teachers also generated gains stronger than those of non-TFA teachers in reading. This dispels the notion that TFA is inferior to other sources of teachers. The findings in reading and the comparison between TFA teachers and all teachers in math were not statistically significant, owing largely to the small numbers of TFA teachers in proportion to the respective comparison groups. However, the fact that across many comparisons all the findings pointed in one direction is a strong indication that the general pattern is correct: that TFA teachers produce results as strong as or better than their peers.

Of course, as with any program, there were some TFA recruits who did not perform well in the classroom. This is likely to continue. However, the distribution of performance among teachers clearly shows that the lowest-performing teachers were consistently not TFA teachers and that TFA teachers make up a disproportionate number of high performers. Moreover, TFA teachers were more consistent in their performance than their non-TFA peers, which means that schools face lower risks in hiring them. The fact that many of the highest-performing teachers in the entire district were supplied by TFA is especially meaningful in light of the program’s small contribution to the size of the district’s workforce compared with other sources of new teachers.

Several caveats are in order. First, this study was based on data from a highly dynamic time in the Houston school district-and in public education more generally. Schools and teachers in Houston were subject to a wide variety of other programs, constraints, and opportunities that no analysis could possibly capture.

Furthermore, these findings cannot tell us what aspects of TFA and non-TFA teachers accounted for the differences in their students’ performance. Is the effectiveness of TFA teachers due to the type of people being recruited, the difference in their academic backgrounds, the support provided by TFA, participation in the district’s alternative-certification program, or a combination of factors? We encourage further study of these questions.

A third caveat relates to the criticisms levied against us by the National Commission on Teaching & America’s Future (NCTAF) after our report’s initial release. NCTAF, a strong promoter of traditional teacher-education programs and more stringent certification requirements, writes, “[CREDO] does not present data indicating how TFA teachers performed compared to teachers who came into the profession fully qualified and certified to teach. . . . To say that TFA teachers do just about as well as other new teachers [in Houston], an extraordinary number of whom are extraordinarily underqualified, is a weak endorsement at best.”

First, this study was not designed to test the benefits of the various teacher-preparation programs or of certification. We do not have the ability, for example, to isolate the effect of Houston’s alternative certification program, because an appropriate comparison group was not available. Instead, our evaluation was explicitly intended to assess Teach for America relative to all other sources of new teachers currently available to school districts like Houston. The Houston district draws from a pool of fully certified, traditionally prepared teachers and those who agree to work toward certification through the district’s alternative certification program. Our results show that TFA teachers perform modestly better than those from other sources of new teachers. Studies of the relative merits of various certification policies and programs are clearly important to the continuing policy debates concerning teacher preparation and teacher quality, but they ask questions that are fundamentally different from those of the study described here.

Second, we didn’t just compare TFA teachers with the Houston district’s other new hires, a fair share of whom are uncertified and didn’t attend a traditional education school (though not nearly as many as NCTAF claims). We also compared TFA teachers with all the teachers in the district, the vast majority of whom are certified. The TFA recruits performed just as well as-f not better than-this pool of teachers; in fact, at least in elementary-school math, the best TFA teachers performed better than the district’s best teachers after the data were adjusted for the effects of teaching experience.

Informing Policy

While these findings cannot speak to the effectiveness of various certification policies, they at least dispel the notion that only the traditional route to teaching can produce good teachers. What needs to be studied is the degree to which subject-matter skills and strong performance in college can substitute for pedagogical training. TFA corps members are an admittedly select group of college graduates, culled from the finest universities and often performing near the top of their class. These are not the types of students who ordinarily go into teaching. It’s possible that traditional certification programs and pedagogical training are less necessary for them than they are for the typical teacher.

The implications of the findings extend beyond teaching preparation. Earlier research has identified strong returns to experience in the early years of teachers’ careers. The year-to-year growth in teachers’ effectiveness, as measured by student achievement, is largest in the early years of their careers and then quickly tapers off. In the past, attention has been paid to the average student gains for each year of teaching. The findings in this study suggest that the benefits of teaching experience may not be uniform-that new teachers may vary considerably in their initial starting points and in how quickly they improve. Therefore, differences in the value teachers glean from additional professional development may also be in play. These questions cannot be answered without further study. The point is raised here, however, as a reminder that policy decisions concerning teacher training and certification cannot assume that a fixed set of requirements will make all teachers perform the same.

The present analysis of teacher effectiveness raises other questions beyond certification. One cherished tenet of the NCTAF position is that TFA is pernicious because the majority of TFA teachers leave after completing their two-year commitment. However, the mythical teacher-for-life doesn’t exist. Across the four cohorts of teachers included in this study who completed two years or more of teaching, more than a third of TFA teachers remained past their commitment. The NCTAF criticism ignores the fact that 45 to 50 percent of other new teachers leave after two years. The NCTAF assertion that no benefits can accrue to either students or the district from teachers who leave after a few years must apply to both groups or not at all. On the contrary, the evidence here points to the opportunity to identify early in teachers’ careers whether they are effective, thus allowing those who are not effective to be encouraged to consider alternative occupations.

Once other states begin to develop the rich data resources found in Texas, this evaluation should be expanded to include other TFA districts. Corroborating and expanding the results obtained here would contribute to both the TFA program itself and the larger policy world in which TFA continues to grow. TFA has been shown to be a viable source of new teachers for Houston, both in number and in quality. With continuing attention to retention during their two years of service and with many TFA teachers choosing to remain after their commitment has been fulfilled, TFA is likely to create an enduring positive presence in the Houston Independent School District and elsewhere.

Expanding the supply of high-quality teachers is of the utmost importance, but there is little consensus on how to do so. The findings reported here suggest that multiple paths into teaching are both possible and desirable. They suggest that the way to ensure high-quality teaching is to set performance goals and hold teachers accountable for meeting them. The ultimate measure of a teacher-preparation program, traditional or alternative, is its ability to produce good teachers, not its ability to meet an unrelated set of requirements that are not grounded in empirical proof of student performance.

-Margaret Raymond is the director of CREDO, a nonpartisan research group at the Hoover Institution, Stanford University. Stephen Fletcher is the assistant director of CREDO.