Cheating by educators comes in many forms, ranging from the subtle coaching of students to the overt manipulation of test results. For instance, a colleague of mine tells of a principal who would begin each morning’s announcements with a greeting to students, such as, “Good morning students, and salutations! Do you know what a salutation is? It means ‘greeting,’ like the greeting you see at the beginning of a letter.” Students learned the meaning of words like “salutation” from the principal’s daily announcements; they probably never learned that his choice of words like “salutation” was done with the vocabulary section of the state-mandated, norm-referenced test in hand. A recent U.S. News & World Report article described a case in Ohio, where one educator is accused of physically moving a student’s pencil-holding hand to the correct answer on a multiple-choice question. Another widely reported case involved a principal in Potomac, Maryland, who stepped down amidst charges that she had gone through students’ test booklets in the classroom and called them up to change or elaborate on their answers.

This isn’t just a recent phenomenon. More than a decade ago, at an evening reception following a conference for school district superintendents in one midwestern state, I happened upon a conversation among several superintendents who, with cocktails in hand, were chuckling and winking about how their quality-control procedures for student testing involved “pre-screening the kids’ answer sheets for stray marks.” What was so funny—I found out later from one of the superintendents—was that stray marks included things like wrong answers. Wink, wink. This practice apparently continues. In Texas, where the accountability system is particularly rigorous, 11 school districts were investigated for an unusually high number of erasures on students’ answer sheets in 1999.

Despite their headline-grabbing nature, such blatant cases of cheating are probably rare. There is usually more subtlety involved, such as when a teacher prods a student to review his or her answer: “Why don’t you take another look at what you wrote down for number 17?” Some examiners cheat by failing to monitor their students properly while proctoring an exam. Others cheat by omission, such as when a teacher reminds students who are likely to attain low scores that it would be okay for them to be absent on the day of the test. A more sophisticated version of cheating by omission occurred in the Austin, Texas, school district in 1999. School administrators entered incorrect student-identification numbers on the answer sheets of low-scoring students, which invalidated their scores and thus raised the schools’ average performance. In states that force schools to give absent students a score of zero for reporting purposes (which eliminates the incentive to encourage them to stay home), all students are encouraged to attend school on test day. But some students are afforded “testing disability accommodations” such as an individual aid, reader, extra time, or other assistance that isn’t usually a part of the student’s educational experience.

Perhaps the largest cheating scandal to date involved teachers and principals in the New York City school district. In December 1999, Edward Stancik, the district’s special commissioner of investigation, released an exhaustive study that found that cheating by 12 educators was “so egregious that their employment must be terminated and they should be barred from future work with the [Board of Education].” For instance, one teacher would have her students first write their answers on a piece of scrap paper. She would then correct their answers before they bubbled in their official answer sheets. The report named another 40 educators who were recommended for disciplinary action; 35 of them had engaged in actions that the investigators judged serious enough to warrant potential termination. The report concluded that the school district had known about the extensive cheating by educators “for years,” and that “educators were not held fully liable for their misconduct.” A follow-up report named another ten educators who had engaged in seriously inappropriate behaviors during testing in New York City, some of them so blatant—such as writing answers to test questions on the chalk board—that immediate termination of employment was recommended.

Some opponents of testing see cheating by educators as a reason to abandon high-stakes accountability systems. For instance, last year Alfie Kohn, a prominent critic of testing, told Congressional Quarterly, “The real cheating going on in education reform is by those who are cheating students out of an education by turning schools into giant test-prep centers.” These critics view cheating as the natural, and not so reprehensible, result of placing undue emphasis on a metric—standardized tests—that yields an incomplete picture of both student and teacher performance. Some even view cheating as a kind of civil disobedience. But, to collect useful information about educational progress, testing is an indispensable if imperfect tool. And if tests are to yield useful information, their validity must be ensured. The answer to cheating is not to abandon accountability. The answer is to limit if not eliminate the cheating. What follows is a survey of what we know about the extent of cheating and some proposals to guard against the subversion of accountability systems.

Testing Guidelines

Cheating can be defined as any action that violates the rules for administering a test. Commercial test publishers produce carefully scripted directions and clear guidelines for administering their tests. The guidelines lay out, in detail, all the actions that would compromise the test results. Similar instructions and rules accompany the customized tests that form the bedrock of many state accountability systems. Some states have strongly worded professional codes of ethics that explicitly define the responsibilities and boundaries associated with mandated testing. What constitutes cheating may even be codified in state law. For instance, the Ohio Revised Code proscribes “any practice that results solely in raising scores or performance levels on a specific assessment instrument without simultaneously increasing the student’s achievement level as measured by tasks and/or instruments designed to assess the same content domain.” The law provides for the termination of an offender’s employment, the suspension of an educator’s license for a violation, and the charging of an offender with a misdemeanor.

National organizations representing various professional associations have also developed standards for educators who administer standardized tests. For example, the American Federation of Teachers and the National Education Association helped to produce the Standards for Teacher Competence in Educational Assessment of Students, which require that teachers “should be skilled in recognizing unethical, illegal, or otherwise inappropriate assessment methods and uses of assessment information.” The most explicit statements regarding cheating can be found in the Standards for Educational and Psychological Testing, a product of collaboration among professional organizations in the fields of education and psychology. Among other guidelines, the standards explain that those involved in educational testing programs must “ensure that test preparation activities and materials provided to students will not adversely affect the validity of test score inferences” and “maintain the integrity of test results by eliminating practices designed to raise test scores without improving students’ real knowledge, skills, or abilities in the area tested.” In short, educators cannot plead ignorance; there has been no dissemination problem regarding what constitutes cheating. Professional codes of ethics that cover virtually every profession whose members work in school settings and state laws that govern those who have a license or credential in the field of education contain strict guidelines for administering tests. Anyone who is connected with testing in American education knows—or should know—how to conduct assessments that yield accurate and credible results.

The Consequences of Cheating

When cheating occurs, testing yields inaccurate information about individual students. The error is compounded when this information is then used for any educational purpose, and specific students wind up paying the price. One student may not receive the remedial instruction in reading that she needs. Another student may be incorrectly assigned to a special program for gifted and talented students that has a limited number of slots. Another may receive a scholarship that should have gone to one of his peers. And yet another may receive a diploma without having learned those minimum skills deemed necessary for success in college or on the job.

From the perspective of educational policy-making, the same invalidities that yield misleading test scores at the individual level also serve to muddle the interpretation of group test performance. Policy-makers and educational administrators have increasingly come to rely on group data to inform their decisions on staffing, curricula, professional development, and teacher-credentialing requirements and to measure the effectiveness of educational reforms. By distorting test results, cheating can lead to ill-advised initiatives, improperly focused resources, and inaccurate conclusions about the course of education reform. Given the confluence of achievement gains—in states ranging from Texas to North Carolina to New York—with the pervasive reports of cheating by educators, it is entirely reasonable to question how much of the former can be attributed to the latter.

Though not attempted in this article or elsewhere to my knowledge, the costs of cheating probably could be measured in dollars and cents. What cannot be measured are the effects of cheating at more fundamental levels. For example, when students learn that their teachers or principals cheat, what is the effect of this kind of role modeling? While fallen professional athletes might be able to say, “Don’t look at me as a role model, I am just an athlete doing a job,” educators cannot: a significant aspect of their job is the modeling of appropriate social and ethical behavior. Also, how might educator cheating affect students’ attitudes toward tests or their motivation to excel? How might it affect their attitudes toward education, their trust or cynicism with respect to other institutions, or their propensity to cheat in other contexts?

Research on Cheating

Shocking anecdotes don’t tell us much about how serious the cheating problem actually is. Just how prevalent is cheating by educators? Only a few studies have directly asked educators whether they have engaged in what have come to be referred to euphemistically as “inappropriate test administration practices.” The most common avenue of research is to poll educators regarding their general perceptions of cheating in their schools. One such study asked 3rd, 6th, 8th, and 10th grade teachers in North Carolina to report how frequently they had witnessed certain inappropriate practices. Of those polled, 35 percent said they had engaged personally in such practices or were aware of others’ unethical actions. The teachers reported that their colleagues engaged in a range of inappropriate practices two to ten times more frequently than they had. The practices included giving extra time on timed tests, changing students’ answers, suggesting answers to students, and directly teaching specific portions of a test. More flagrant examples included teachers’ giving their students dictionaries and thesauruses for use on a state-mandated writing test. One teacher revealed that she checked students’ answer sheets “to be sure that her students answered as they had been taught.” Other teachers reported using more subtle strategies, such as “a nod of approval, a smile, and calling attention to a given answer,” to enhance their students’ performance. In another study, of teachers who were drawn from two large school districts, 32 percent of the teachers surveyed reported allowing students to practice on old forms of standardized tests for two or more weeks.

A total of 40 schools were included in a study that was initiated in order to investigate suspected cheating in the Chicago public schools. Of the 40 schools, 17 served as “control” schools, which were compared with 23 “suspect” schools that had exhibited irregularities in the performances of their 7th and 8th grade students on the Iowa Tests of Basic Skills (ITBS). The irregularities consisted of unusual patterns of score increases in previous years, unnecessarily large orders of blank answer sheets for the test, and high percentages of erasures on students’ answer sheets. The researchers readministered the ITBS under more controlled conditions and found that, even after accounting for students’ reduced level of motivation on the retesting, the “suspect” schools clearly did worse than the “control” schools on the retest. The researchers concluded that they might have underestimated how much cheating was going on at some schools. A study of cheating in the Memphis school district revealed extensive cheating on the California Achievement Test, including a teacher who displayed correctly filled-in answer sheets on the walls of her classroom.

The Perceptions of Educators

The most troubling stream of research on cheating concerns the attitudes of educators toward cheating. They seem increasingly indifferent toward cheating, and there appears to be a growing sense that cheating is a justifiable response to externally mandated tests.

Several studies have attempted to investigate educators’ perceptions of cheating. In a 1992 study, 74 pre-service teachers were asked to judge the appropriateness of certain behaviors. Only 1 percent thought that either changing answers on a student’s answer sheet or giving hints or clues during testing was appropriate, and only 3 percent agreed with the idea that allowing more time than allotted for a test was acceptable. But 8 percent thought that practicing on actual test items was okay; 23 percent judged the rephrasing or rewording of questions acceptable; and 38 percent thought that practice on an alternative test form was appropriate.

The beliefs of pre-service teachers appear to translate into actual practices when they enter the classroom. In 1991, a large sample of 3rd, 5th, and 6th grade teachers in two school districts was asked to describe the extent to which they believed specific cheating behaviors were practiced by teachers in their schools. On the positive side, a majority of respondents (shown in Figure 1) said that, for all of the behaviors listed but one, they occurred rarely or never. Equally noticeable, however, is that in several cases, 15 percent or more of the respondents reported that a behavior occurred “frequently” or “often.” A full 23 percent of the teachers said that they thought teachers often gave hints to children who were having difficulty. Twenty percent said they thought teachers “frequently” or “often” gave students extra time to finish their tests. Likewise, 20 percent said they thought teachers “frequently” or “often” gave students practice on passages that were highly similar to those used on the test.

A 1991 survey examined perceptions about two specific kinds of “test preparation” practices: having students practice for a state-mandated, norm-referenced test using another form of the same test or having students practice on the actual test to be used. The survey polled six groups of educators, including teachers, principals, superintendents, and school board members in California. The results, shown in Figure 2, reveal fairly broad acceptance of these behaviors, even among board members. For instance, 36 percent of teachers in California thought it appropriate to practice with current test forms.

What Can Be Done?

What can be done to address the problem of cheating? At some point, we will need to reconceptualize testing entirely. We must find more effective ways to link, consistently and directly, successful test performance to student effort and effective instruction. If poor performance were accompanied by sufficient diagnostic information about a student’s weaknesses, then all concerned might view identification and remediation of those weaknesses as more beneficial then cheating. Such an initiative will require changing much of the status quo in curriculum, instruction, and assessment. But there are less far-reaching, more pragmatic actions that can be taken immediately. The following list provides a start.

Dissemination. It has been said that we more often stand in need of being reminded than we do of education. As mentioned earlier, every large-scale testing program provides a description of appropriate test-administration procedures; state regulations define the boundaries of legal conduct for test administrators; and education-related associations have produced guidelines for sound testing practice. Nonetheless, those caught cheating often protest that they did not know the behavior was wrong. If only as a reminder, every implementation of high-stakes tests should be accompanied by dissemination of clear guidelines regarding appropriate testing practices. Such reminders should be clearly worded, pilot-tested to refine the meaning that educators take from the guidelines, and distributed and signed by all who handle testing materials.

Procedures. Some minor procedural changes would hinder the ability of educators to cheat. For example, bar-coding or other methods of identifying testing materials plus a system of tracking testing materials would be easy to implement. Federal Express and United Parcel Service know the location of every package at any given time and can reconstruct precisely the hands that a package has passed through. The same tracking can be used for testing materials. Other simple steps would include the sealing of cartons and bundles of testing materials; delaying the delivery of testing materials to schools until just before test administration; and, once delivered, requiring that materials be maintained securely by a named person who is responsible for them.

“Truth in testing.” States with so-called truth-in-testing laws should reconsider their relative benefits. These laws often require that the content of state-mandated tests be disclosed following the administration of a test. They have the best of intentions, but the unforeseen consequence of such laws has been an increase in educators’ use of previous versions of tests for classroom practice, resulting in further narrowing of instruction. Moreover, the economic costs to “truth in testing” states have been staggering. Disclosing each year’s tests renders them useless, making it necessary to develop entirely new monitoring instruments one or more times each year.

Scaling Back. The expansion of testing and accountability systems has elicited two reactionary responses to the concurrent rise in cheating: 1) that large-scale testing for accountability be abandoned; or 2) that testing for accountability rely more heavily on constructed-response formats that, ostensibly, would be less prone to corruption. For instance, it is more difficult to forge or coach a student’s answer to an essay question or a science experiment than to alter a bubbled-in response or to provide the key to a multiple-choice item.

The difficulty with these reactions is that they fail to address the core issues. High-stakes pupil testing arose in the 1970s in reaction to the complaints of some business leaders—along the lines of, “We are getting high school graduates who have a diploma, but can’t read or write!” As UCLA Professor of Education James Popham observed at the time: “Minimum competency testing programs . . . have been installed in so many states as a way of halting what is perceived as a continuing devaluation of the high school diploma.” The public perception was that the gatekeepers were leaving the gates wide open. Perhaps a widespread misunderstanding of the relationship between self-esteem and achievement was to blame. Educators understandably wanted all students to have the personal esteem associated with high achievement. But awarding higher grades in order to boost self-esteem and stimulate further achievement too often had neither effect. The sense that grades weren’t accurate measures of achievement led to the imposition of externally developed and administered tests.

Thus the obvious error in calls to return to the past is that such a strategy only returns American education to the situation that caused accountability tests to be introduced in the first place. Moreover, though current tests are susceptible to cheating, the solution of returning to measures and procedures that are even more easily manipulated is unthinkable.

Nevertheless, we should consider limiting the amount of testing for accountability. Some tests serve solely an instructional function; they serve our need for solid diagnostic information on students in order to make informed decisions about their educational programs. When these tests are used also for accountability purposes, the expanded incentives to cheat can corrupt the information we have on student progress. Likewise, not all tests—especially those designed for purposes of decision-making—need have instructional value. If we clarify the purpose of each test, we can minimize the scope of mandated accountability tests, the time required for their administration, and the opportunities for cheating.

Consequences. In conjunction with limiting opportunities for cheating, we must revise the procedures that are used to unearth cheating and the penalties that are handed down. Many tests are currently administered behind closed classroom doors with little independent oversight; there are strong disincentives for educational personnel to report cheating; and, in most jurisdictions, the responsibility for investigating cheating rests with school personnel who have an inherent conflict of interest in ferreting out inappropriately high student achievement. Revised procedures should include: 1) random sampling and oversight of test sites; 2) increased protections for whistle-blowers; 3) stiffer penalties for cheaters, including permanent disqualification from teaching within a state and more coordinated sharing among the states of information regarding educators who have had their licenses revoked; and 4) assignment of investigative responsibilities to an independent authority.

A Qualified Boon

As we are learning, accountability is not an unqualified boon to American education. Nascent accountability systems have been difficult to implement and have had some undesirable consequences. For instance, in some situations the use of tests as the primary accountability mechanism has resulted in an extreme narrowing of what students are taught. When so much is at stake, educators tend to limit their instruction to the content that is covered on a mandated test. Moreover, some educators perceive the imposition of an externally mandated test as an inappropriate intrusion into an area of professional practice and discretion. They believe that their knowledge of a student’s true ability far exceeds whatever information can be gleaned from a single test; thus the idea that a single test should not be used against any student (for example, to deny grade-to-grade promotion or a high school diploma) is widespread among teachers. Here the same moral reasoning that many current teachers learned in their education courses during the 1970s may come into play: the ends do justify the means; cheating is a justified response to a system that punishes students and teachers on the basis of incomplete information.

But cheating itself leads to inaccurate information and misguided decisions about students. It signals that students have learned the skills we want them to, when in fact they haven’t. It threatens the values that we hope to impart to students via those we have charged with their education. It leads to mistaken conclusions about the efficacy and pace of needed educational reforms. Even if cheating is limited to a minority of educators, as it most likely is, its effects are devastating. It is no more justifiable than telling a sick patient that he is well and then sending him on his way.

–Gregory J. Cizek is an associate professor of educational measurement and evaluation at the University of North Carolina at Chapel Hill and the author of Cheating on Tests: How to Do It, Detect It, and Prevent It (1999).

The post Cheating to the Test appeared first on Education Next.

Parker’s motivating idea is its common, carefully focused and interconnected academic curriculum, which is shaped by “essential” questions in two major areas: Mathematics/Science/Technology, including Health, and Arts and Humanities, including Spanish. Each of our 340 students, ages 12 to whatever it takes to meet the graduation standard, has a Personal Learning Plan, effectively a contract among student, school, and family. Students are promoted through three Divisions, irrespective of their ages, by public presentation of portfolios and, for older students, Exhibitions. Our limited budget (per-pupil expenditure is up to 15 percent less than nearby district high schools) allows for a spare but spirited jazz band and interscholastic athletic, debate, chess, and mock trial teams. The best measure of Parker’s success? A long waiting list for the 2000–01 academic year.

Veteran educators Nancy and Ted Sizer and several colleagues accepted the challenge of Massachusetts’ charter school law—to build a new school upon a visionary foundation.

The school was born of three parents’ desire to have an alternative to their town’s school. They, in turn, were inspired by the ideas put forth in a book, Horace’s Compromise, written by one of their neighbors, Ted Sizer. “Horace” is Horace Smith, a fictional representation of a typical high school teacher who is caught in a job he cannot do at the standard he knows his students deserve. He serves too many of them to know each one well, to analyze their strengths and weaknesses, to meet with them in conferences, to be trusted enough to offer the constructive criticism that is necessary for their growth. Horace’s school, its routines mechanical, its purpose undefined, is an unconnected smorgasbord of programs, courses, activities, and more. In the end, it is the students who are expected to make sense of it all. And so Horace compromises to get along. Without substantial changes in the way his school is organized—indeed, in the way its best values are expressed—he will be stuck doing second-class work in a genial, well-intentioned, but basically directionless place.

Horace’s Compromise also inspired the creation of the Coalition of Essential Schools in 1984, first based at Brown University and now an independent non-profit organization situated in Oakland, California. The Coalition’s mission is to help create and support the growth of schools that salve Horace’s frustrations, by providing an education that is more integrated and focused than the usual. The experience of Essential schools in New York City, especially the celebrated Central Park East Secondary School, created by Deborah Meier and her colleagues, persuaded us that it was possible to create a high school that dared to develop the most essential intellectual habits in young people—in a way that was demonstrably practical and public (meaning open to all students) and had long-term positive effects on students.

Students celebrate Halloween at the Parker School.

The charter school provision of the Massachusetts Education Reform Act of 1993 gave these parents, the two of us, and several other colleagues our opportunity. With the Coalition’s principles of school practice in mind, we hastily began developing our school plan. Some decisions were easy: to provide a program from 7th grade through graduation; to move students through the program on an individual basis; to ask our teachers to be well educated, but to act more as generalists than specialists; to keep teachers’ student loads down, and to offer advisories instead of more formal and distant “guidance counseling”; to offer only one foreign language, but to expect all to learn it; to put our money into more adults, some of them young adults, rather than into high rents or new furniture.

Other decisions were harder. How theoretical did we want our curriculum to be? How practical? In a promotion system based on demonstrated competence, what would be our benchmarks? How would they be made clear to teachers and to students and their parents without becoming rigid and meaningless? How would we help our teachers to adapt to classrooms in which projects would often replace the usual chalk-and-talk? How could we be good neighbors to the schools in our area from which we might enroll students? How could we fashion a diverse student body, yet still draw students from towns that provide a higher per-pupil allocation?

What follows is the story of the ups and downs of going from the germ of an idea to a secondary school that serves students ages 12 and up—in short, the ways in which we avoided Horace’s compromises or coped with those compromises we couldn’t avoid.

Sweat Equity

The ideas behind our school and the motives for starting it were the relatively easy part.The start-up of any organization plunges its leaders into a welter of unanticipated or underestimated problems. From its very beginning, Parker confronted a number of challenges—everything from financial difficulties to engaging our teachers in a unique and demanding curriculum.

Like almost every charter school, the Parker School has had its share of financial concerns, especially during the start-up phase. For the most part, charter schools, once open, receive funds for ongoing operations, but the months before opening day are funded largely by the founders’ sweat. The development of an institution’s ideas takes time; not just anyone’s time, but the time of experienced and busy people. And so it was that parents and students were repairing, carrying, and washing furniture late at night, while teachers were doing every kind of job, from planning classes to washing windows to driving to the store to buy erasers. In each of our first meetings, whether we were hashing out our guiding principles or writing the charter application or recruiting our first potential students or deciding when—and how—to hire our first employees, we must have worked for love because we certainly were not working for money. When we trustees stopped by, we were likely to be put to work. Save for some federal money (which came late), the detailed planning of the Parker School and the training of its staff had to be done pro bono.

The course of study at the Parker School, where the focus is on developing students’ intellectual habits, runs narrower and deeper than the typical high school’s.

Massachusetts’ charter schools—places that were to break the mold and chart new directions, no less—apparently were to be designed in persuasive detail by spontaneous combustion, with hardly any support from the state. That some of us designing Parker had substantial experience made our task easier—though never easy—but we were the exception. Only the big for-profit companies have the financial horsepower to create fresh material. Schools that have sprung from the grassroots public interest of a community have had to make do with very little to nothing. It is, therefore, no surprise that too many charter schools are either weak or, in their unthinking copying of existing practice, still hampered by the compromises that bedeviled Horace. Powerful new, complex institutions require up-front research and development. That means money. Charters are getting little of it, and rarely at a scale that could make a substantial difference. Serious start-up schools need ready physical facilities, adequate R&D funds for the initial planning, and at least half of the faculty hired and involved in planning and training for at least nine months before the school opens.

Left Hand, Right Hand

Like most applicants for a charter, we were attracted by the idea that we could shape our own mission, present it to the state’s chartering authorities, and then be held to our own, but externally validated, standards. Thus Parker wrote an application for a charter that detailed our educational goals, our strategy for meeting those goals, and a means of carefully assessing, internally and externally, whether we had reached those goals. Parker chose to keep doggedly to the “essentials”—designing a curriculum that covered a small number of topics in extreme depth, rather than lightly touching on a broad range of material.

Parker’s planners worked “backwards.” We visualized the kinds of graduates we would be proud to send into the world.We started with the habits, skills, and content we wanted our students to learn, and then we planned the curriculum and teaching styles to reach those aims. We found that being reasonably clear about what a student should know and be able to do as a matter of habit led to an ever narrower and deeper program.

For example, if the school expected every graduate to display clear and even marginally graceful expository prose, then there had to a great deal of writing, not just in Humanities but in the other domains as well. That writing had to be assessed—which meant teacher time, often preceded by teacher training. We had to ask ourselves, What will be our standard for Division III expository writing? How will we determine whether students have met it, and what will it take to get most, if not every last one, of our kids to that level? And what standard should apply to children who are identified as having “special needs”?

Such “backwards planning”is commonsensical yet rare in public education. Instead there are lists of topics to cover and skills to achieve, and these are rolled out in central headquarters with little regard to pedagogical and scholarly realities. The giant textbooks so familiar in American high schools are testament enough to that; in most, there is something included for every political pressure group. The result is “scholarship” by hop, skip, and jump, one of Horace’s most painful compromises.

At times, these conflicting approaches made life at Parker rather difficult. For instance, the state Board of Education approved all the aspects of our charter proposal. But in a different policy context it asserted that the state’s charter schools had to follow the unusually comprehensive state curriculum frameworks (thus, in our view at least, guaranteeing a curriculum that will be geared toward superficial mastery by most students). Our students also had to pass standardized tests derived from those frameworks. In many ways, these conflicting signals reflected little more than the left policy hand not knowing what the right policy hand was doing, but it placed and still places Parker in a compromised position. Nevertheless, and counterintuitively, our students score competitively on state tests.

The strains of the state’s dual agenda are especially evident during testing season. Since we group our students (on the basis of their exhibited performance) into three Divisions, each with an approximately two-year curriculum designed by our teachers, the state’s pattern of testing 8th and 10th graders (a designation we don’t make, on philosophical grounds) is especially hard on our students. Our emphasis in Arts and Humanities shifts from world topics one year to American topics the next, which means that those who take the tests may well have studied the material only in the previous year. Furthermore, when “10th graders” are pulled out of class to take extensive tests, new curricula must be devised for the “9th graders” who are left behind. Too much energy must go toward mitigating the disruption, even more than in other schools. Everyone is stretched, with little actual learning to show for it.

Teachers at Parker benefit from low student loads that allow them to engage students on an individual level.

In the spring of 2000, we had yet another dilemma. For several weeks, the entire curriculum in Division II Arts and Humanities was devoted to preparing for a trip to Washington, D.C., during which every student participated in a visit to the office of a member of Congress to advocate for a bill that was before Congress. The students learned about the substantive reasons for and the politics behind the legislation they were researching, the ways in which bills become laws, the opportunities citizens have to take part in the process, and the most persuasive ways of presenting their cases. It was a superb civics lesson, and the vast majority of the students showed, in a variety of observable ways, that they had learned powerfully from it. However, while their enthusiasm about their experiences augurs well for the fate of the republic, what they learned is not likely to be an item on this year’s test. Effective charter school policy requires consistency from the state, both in protecting what the approved charter asserted and in assessing a charter school’s success in ways that are consistent with its charter.

Building a Staff

The challenge of any “new” school, charter or otherwise, and especially a school that wishes to adopt new ways of teaching, is to build a competent, experienced staff with the expertise and energy to work in a start-up. Teacher education has largely been a matter of the “great old dogs” teaching eager new dogs the best of their tricks. The problem is that some of the old tricks are not very effective in schools that rest their programs, as does Parker, on one or another branch of serious contemporary research. Teaching toward intellectual academic habits such as, in Howard Gardner’s phrase, “thinking like a scientist” is more than mere “coverage” of “material” to be displayed only at the immediate completion of a “course.” Truly demanding intellectual training is not cookbook, easily routinized stuff. It is challenging work, particularly if high school students are used to getting by within the old, familiar system. The clash of minds, the give-and-take that leads a young person to a responsible and usable grasp of a consequential idea is sophisticated business.

In such a school, teachers must have time away from children each day to focus on their teaching, on their students’ work, and on learning from their colleagues. Every teacher at Parker has one to two hours daily away from class. Within a tight budget, this is accomplished by offering only a narrow core program. Parker’s designers regarded this as a worthy trade-off.

Another personnel trade-off we tried to make didn’t work as well. In order to achieve low teaching loads, we tried to conserve money by keeping overhead low. Parker started with a “leadership team” consisting of only two “lead” teachers, a business manager, and a part-time “coordinator.” This helped to achieve low student loads—65 or fewer per teacher, including advisories of 12 students per teacher—but, by January of our first year, the trustees and the faculty knew this leadership arrangement would not work. The workload was more than an overworked quartet could handle.

And so, still believing that less is more when it comes to “administration” (which we too easily confused with “leadership”), we hired a proven teacher/principal. He carried both tasks for 18 months and then everyone, including him, cried Uncle. The work was too demanding. So much of the school was new, and creating (and then recreating) fresh practices took time, intense concentration, coordination, and a level of imagination that rarely emerges for overworked people at 2 A.M. Young teachers needed support. Parents and prospective parents needed endless explanations of our school’s design. The state expected the principal to serve not only as a typical Massachusetts school leader, but also as superintendent. Money, being in short supply from state sources, had to be raised. The list was endless.

Truly demanding intellectual training is not easily routinized stuff. It is challenging work, particularly if students are used to getting by in the old system.

At this point Parker’s trustees turned to two retired school veterans on the board (the authors of this essay) to act together, pro bono, as the principal. This bought Parker a year to think through its leadership requirements, to restructure the budget to allow for the appointment of experienced administrators, and to complete a search for a senior leader. But the general problem remains: There is a severe shortage of adequately prepared people rising to leadership, especially educational leadership of new ventures.

Culture Matters

We tried to create a distinct culture in our school, one that taught lessons in its own right. From the beginning, we were determined to include our students as full-fledged fellow pioneers. We valued them and admired their bravery and told them so often, especially as they progressed through our performance-based program. We tried to keep external motivators like grades and punishments at a minimum, in the hope that our students would develop internal motivators. Yet we knew that most of our students and many of their parents were coming to school with a different sort of expectation.

Parker, like most schools, has rules. It also aspires, however, to have a “culture” that will make these rules necessary only at extreme moments. No issue has been more difficult to master, especially since the school believes in giving students substantial say in shaping the community’s mores and life. On the surface, it is easier to “run” a “tight” school. However, any secondary school where the “rules” are merely delivered from on high without intense discussion by all parties of their meaning and application is a poor place for students to learn how to be members of a thoughtful community. Parker has made becoming such a community one of its goals, not only to make life in our building pleasant, but, equally important, to “teach” the restraints that make a community respectful and yet also free. This is easy to articulate, but exceedingly difficult to do, as it must be done one person at a time.

Creating a healthy school culture almost demands small scale. A community where people do not know one another is a place that must be policed rather than gathered. Developing the habits of getting along in constructive, principled ways is not simply a matter of building small schools, though. It must be worked at just as deliberately as a student works at his calculus. There are no short cuts to a lasting “moral education.”

Commencement

As we write this, we have just returned from attending Parker’s first commencement. If ever a charter school sought validation, this was it. Offered the chance to speak, a third of our graduates chose to do so. Most moving for those of us who have worked to bring this school into life and sturdy health was the message and the tone in their speeches. Their school was different, they said; different in ways that benefited them. The students have been part of an institution that, as they described it, was challenging and yet was always clearly on their side, was in fact designed around their growth. Student after student thanked the adults who had made it happen.

To have the chance to attack Horace’s compromises head-on is a worthy undertaking. In Massachusetts, to be able to run a public school that is open on a lottery basis to any child wherever that child may live—rich suburb or poor city—finally gives an honest definition to the word “public.” Though it will still take time, effort, and money to make this access across residential communities a practical reality, the state is headed in the right direction.

We still worry that too many policymakers and citizens will assume that “standards” by definition require “standardization”; that “reform” can be achieved by a combination of jawboning, regulation, and low-cost testing; that humiliation is a necessary part of competition; and that the universities will not wake up to their obligation to support reform vigorously. As pioneers, we all have summoned the extra energy required to climb mountains, but if long-term improvements are to be made, we need to have policies that will make life sustainable on the other side.

–Nancy Faust Sizer and Theodore R. Sizer are trustees of the Francis W. Parker Charter Essential School and served together as its acting coprincipals in 1998–99. The Sizers’ jointly written The Students Are Watching: Schools and the Moral Contract was published in 1999.

The post A School Built for Horace appeared first on Education Next.

Rewarding Expertise by Allan Odden.

Let the Market Decide by Dale Ballou and Michael Podgursky.

The post Defining Merit appeared first on Education Next.

Illustration by Stefano Vitale.

Such methods, although they have been in use for decades, have rarely worked well. The Celebration School was no exception. As the Times headline put it, there was “Trouble at the Happiest School on Earth.” The Times article began, “The start of the school year here is just a few days away, so it was no surprise that there was a line of parents at the Celebration School office the other day. But the reason for the line was: they were queuing up to withdraw their children.” Parents said they were dissatisfied with the lack of clear academic goals and measures of achievement, as well as with the lack of order and structure that accompanied the progressive methods.

The Celebration School’s failure was wholly predictable. In the 1980s, the distinguished sociologist James Coleman conducted carefully controlled, large-sample research that demonstrated the ineffectiveness of progressive methods in raising general academic achievement and in closing the achievement gap between advantaged and disadvantaged students. Coleman found that Catholic schools achieve more educational equity than public schools because they follow a rich and demanding curriculum; provide a structured, orderly environment; offer lots of explicit instruction, including drill and practice; and expect every child to reach minimal goals in each subject by the end of the year. All of this stands in stark contrast to the progressive ideals of unstructured, implicit teaching and “individually tailored” instruction that now predominate in public schools. As a result, disadvantaged children prosper academically in Catholic schools, and the schools narrow the gaps among races and social classes. When criticized for condemning public schools, Coleman pointed out that the very same democratic results were being achieved by the few public schools that were also defying progressivist doctrine. Along with large-scale international comparisons, Coleman’s work is the most reliable observational data that we have regarding the validity of progressive ideas, and it has never been refuted.

The evidence against progressive educational theories mounts still higher if you combine Coleman’s data with the research on so-called “effective schools.” Effective schools are characterized by explicit, agreed-upon academic goals for all children; a strong focus on academics; order and discipline in the classroom; maximum time on learning tasks; and frequent evaluations of student performance—all principles repudiated by the Disney school and also by many “new” education reforms. In fact, the progressive way of running a school is essentially the opposite of what the effective-schools research has taught us. A recent review of this research by the late, great scholar Jeanne Chall may be found in The Academic Achievement Challenge: What Really Works in the Classroom? (2000).

One would think that the failures of progressivism might induce more skepticism among both its adherents and the public. Yet the unempirical theories of progressive educators—generally dressed up with empirical claims—remain highly influential among teachers, administrators, and distinguished professors. Their unspoken assumptions work a hidden sway over the American public as well. For example, test-bashing wouldn’t be so popular if progressive theories about education didn’t resonate somehow with widespread American beliefs about children and learning. One can understand why progressives should want to bash tests, when their methods consistently fail to improve test scores. But why should others accept the disparagement of, say, reading tests, which are among the most valid and reliable of existing instruments?

In my mind, progressive educational ideas have proved so seductive because their appeal lies not in their practical effects but in their links to romanticism, the 19th-century philosophical movement, so influential in American culture, that elevated all that is natural and disparaged all that is artificial. The progressives applied this romantic principle to education by positing that education should be a natural process of growth that flows from the child’s natural instincts and interests. The word “nature” in the romantic tradition connotes the sense of a direct connection with the holy, lending the tenets of progressivism all the weight of religious conviction. We know in advance, in our bones, that what is natural must be better than what is artificial. This revelation is the absolute truth against which experience itself must be measured, and any failure of educational practice must be due to faulty implementation of progressive principles or faulty interpretation of educational results. Thus the results of mere reading tests must not be taken at face value, because such blunt instruments cannot hope to measure the true effects of education. The fundamental beliefs of progressivism are impervious to unfavorable data because its philosophical parent, romanticism, is a kind of secular theology that, like all religions, is inherently resistant to data. A religious believer scorns mere “evidences.”

The Chasm Between

There are many disputes within the education field, but none so vituperative as the reading and math wars—the battles over how best to teach children to read and to solve arithmetic problems. These aren’t just disputes over instructional techniques; they are expressions of two distinct and opposing understandings of children’s nature and how children learn. The two sides are best viewed as expressions of romantic versus classical orientations to education. For instance, the “whole language,” progressive approach to teaching children how to read is romantic in impulse. It equates the natural process of learning an oral first language with the very unnatural process of learning alphabetic writing. The emotive weight in progressivist ideas is on naturalness. The natural is spiritually nourishing; the artificial, deadening. In the 1920s, William Kilpatrick and other romantic progressivists were already advocating the “whole language” method for many of the same reasons advanced today.

The classical approach, by contrast, declines to assume that the natural method is always the best method. In teaching reading, the classicist is quite willing to accept linguistic scholarship that discloses that the alphabet is an artificial device for encoding the sounds of language. Learn the 40-odd sounds of the English language and their corresponding letter combinations, and you can sound out almost any word. Yet adherents of “whole language” regard phonics as an unnatural approach that, by divorcing sounds and letters from meaning and context, fails to give children a real appreciation for reading.

The progressivist believes that it is better to study math and science through real-world, hands-on, natural methods than through the deadening modes of conceptual and verbal learning, or the repetitive practicing of math algorithms, even if those “old fashioned” methods are successful. The classicist is willing to accept the verdict of scholars that the artificial symbols and algorithms of mathematics are the very sources of its power. Math is a powerful instrument precisely because it is unnatural. It enables the mind to manipulate symbols in ways that transcend the direct natural reckoning abilities of the mind. Natural, real-world intuitions are helpful in math, but there should be no facile opposition between terms like “understanding,” “hands-on,” and “real-world applications” and terms like “rote learning” and “drill and kill.” What is being killed in memorizing the multiplication table? The progressivist says: children’s joy in learning, their intrinsic interest, and their deep understanding.

The romantic poet William Wordsworth said, “We murder to dissect”; the progressivist says that phonemics and place value should not be dissected in isolation from their natural use, nor imposed before the child is naturally ready. Instead of explicit, analytical instruction, the romantic wants implicit, natural instruction through projects and discovery. This explains the romantic preference for “integrated learning” and “developmental appropriateness.” Education that places subject matter in its natural setting and presents it in a natural way is superior to the artificial analysis and abstractions of language. Hands-on learning is superior to verbal learning. Real-world applications of mathematics provide a truer understanding of math than empty mastery of formal relationships.

Natural Supernaturalism

The religious character of progressivism is rarely noted because it is not an overtly religious system of belief. Romanticism is a secularized expression of religious faith. In a justly famous essay, T. E. Hulme defined romanticism as “spilt religion.” Romanticism, he said, redirects religious emotions from a transcendent God to the natural divinity of this world. Transcendent feelings are transferred to everyday experience—like treacle spilt all over the table, as Hulme put it. M. H. Abrams offered a more sympathetic definition of this tendency to fuse the secular and religious by entitling his fine book on romanticism Natural Supernaturalism. The natural is supernatural. Logically speaking, it’s a contradiction, but it captures the romantic’s faith that a divine breath infuses natural human beings and the natural world.

In emotional terms, romanticism is an affirmation of this world—a refusal to deprecate this life in favor of pie in the sky. In theological terms, this sentiment is called “pantheism”—the faith that God inhabits all reality. Transcendent religions like Christianity, Islam, and Hinduism see this world as defective, and consider the romantic divinizing of nature to be a heresy. But for the romantic, the words “nature” and “natural” take the place of the word “God” and give nature the emotional ultimacy that attaches to divinity. As Wordsworth said,

One impulse from a vernal wood
May teach you more of man,
Of moral evil and of good,
Than all the sages can
—The Tables Turned (1798)

The romantic conceives of education as a process of natural growth. Botanical metaphors are so pervasive in American educational literature that we take them for granted. The teacher, like a gardener, should be a watchful guide on the side, not a sage on the stage. (The word “kindergarten”—literally “children-garden”—was invented by the romantics.) It was the romantics who began mistranslating the Latin word educare (ee-duh-kar’e), the Latin root word for education, as “to lead out” or “to unfold,” confusing it with educere (eh-diu’ke-re), which does mean “to lead out.” It was a convenient mistake that fit in nicely with the theme of natural development, since the word “development” itself means “unfolding.” But educare actually means “to bring up” and “instruct.” It implies deliberate training according to social and cultural norms, in contrast to words like “growth” and “development,” which imply that education is the unfolding of human nature, analogous to a seed growing into a plant.

The same religious sentiment that animates the romantics’ fondness for nature underlies their celebration of individuality and diversity. According to the romantics, the individual soul partakes of God’s nature. Praise for diversity as being superior to uniformity originates in the pantheist’s sense of the plenitude of God’s creation. “Nature’s holy plan,” as Wordsworth put it, unfolds itself with the greatest possible variety. To impose uniform standards on the individuality of children is to thwart their fulfillment and to pervert the design of Providence. Education should be child-centered; motivation to learn should be stimulated through the child’s inherent interest in a subject, not through artificial rewards and punishments.

Whether these educational tenets can withstand empirical examination is irrelevant. Their validation comes from knowing in advance, with certainty, that the natural is superior to the artificial.

A More Complicated Nature

Plato and Aristotle based their ideas about education, ethics, and politics on the concept of nature, just as the romantics did. A classicist knows that any attempt to thwart human nature is bound to fail. But the classicist does not assume that a providential design guarantees that relying on our individual natural impulses will always yield positive outcomes. On the contrary, Aristotle argued that human nature is a battleground of contradictory impulses and appetites. Selfishness is in conflict with altruism; the fulfillment of one appetite is in conflict with the fulfillment of others. Follow nature, yes, but which nature and to what degree?

Aristotle’s famous solution to this problem was to optimize human fulfillment by balancing the satisfactions of all the human appetites—from food and sex to the disinterested contemplation of truth—keeping society’s need for civility and security in mind as well. This optimizing of conflicting impulses required the principle of moderation, the golden mean, not because moderation was a good in itself, but because, in a secular view of conflicted human nature, this was the most likely route to social peace and individual happiness. The romantic poet William Blake countered, “The road of excess leads to the palace of wisdom.” But again, that would be true only if a providential nature guaranteed a happy outcome. Absent such faith in the hidden design of natural providence, the mode of human life most in accord with nature must be, according to Aristotle, a via media that is artificially constructed. By this classical logic, the optimally natural must be self-consciously artificial.

Renewed interest in evolutionary psychology has given the classic-romantic debate new currency. Darwinian moral philosophers such as George Williams reject the notion that evolution should be a direct guide to ethics or to education. On the contrary, evolutionary psychology reintroduces in its own way the classical idea that there are inherent conflicts in human nature—both selfishness and altruism, both a desire to possess one’s neighbor’s spouse and a desire to get along with one’s neighbor. The adjudication of these contradictory impulses requires an anti-natural construct like the Ten Commandments. Similarly, from the standpoint of evolution, most of the learning required by modern schooling is not natural at all. Industrial and postindustrial life, very recent phenomena in evolutionary terms, require kinds of learning that are constructed artificially and sometimes arduously on the natural of the mind—a point that has been made very effectively and in detail by David Geary, a research psychologist specializing in children’s learning of mathematics at the University of Missouri. Geary makes a useful distinction between primary and secondary learnings, with most school learnings, such as the base-ten system and the alphabetic principle, being the “unnatural,” secondary type.

The very idea that skills as artificial and difficult as reading, writing, and arithmetic can be made natural for everyone is an illusion that has flourished in the peaceful, prosperous United States. The old codger Max Rafferty, an outspoken state superintendent of education in Califor-nia, once denounced the progressive school Summerhill, saying:

Rousseau spawned a frenetic theory of education which after two centuries of spasmodic laboring brought forth… Summerhill…. The child is a Noble Savage, needing only to be let alone in order to insure his intellectual salvation… Twaddle. Schooling is not a natural process at all. It’s highly artificial. No boy in his right mind ever wanted to study multiplication tables and historical dates when he could be out hunting rabbits or climbing trees. In the days when hunting and climbing contributed to the survival of Homo sapiens, there was some sense in letting the kids do what comes naturally, but when man’s future began to hang upon the systematic mastery of orderly subject matter, the primordial, happy-go-lucky, laissez faire kind of learning had to go.

The romantic versus classic debate extends beyond the reading and math wars to the domain of moral education. The romantic tradition holds that morality (like everything else) comes naturally. The child, by being immersed in real-life situations and being exposed to good role models, comes to understand the need for sharing, kindness, honesty, diligence, loyalty, courage, and other virtues. Wordsworth’s account of his own education, which he called “Growth of a Poet’s Mind,” contained a section entitled, “Love of Nature Leading to Love of Mankind.”

The romantic wishes to encourage the basic goodness of the natural soul, unspoiled by habit, custom, and convention. The principal means for such encouragement is to develop the child’s creativity and imagination—two words that gained currency in the romantic movement. Before the romantics, using the term “creativity” for human productions was considered impious. But that ended when the human soul was conceived as inherently godly. Moral education and the development of creativity and imagination went hand in hand. In the 19th and early 20th centuries, textbooks like the McGuffey Readers strongly emphasized moral instruction and factual knowledge. With the rise of progressive ideas, however, the subject matter of language arts in the early grades began to focus on fairy tales and poetry. The imparting of explicit moral instruction gave way to the development of creativity and imagination. Imagination, the romantic poet and essayist Samuel Taylor Coleridge said, “brings the whole soul of man into activity.” When we exercise our imaginations, we connect with our divine nature, develop our moral sensibilities.

Romance or Justice?

One cannot hope to argue against a religious faith that is impervious to refutation. But there can be hope for change when that religious faith is secular and pertains to the world itself. When the early romantics lived long enough to experience the disappointments of life, they abandoned their romanticism. This happened to Blake, Wordsworth, and Coleridge. One of Wordsworth’s most moving works was the late poem, “Elegiac Stanzas,” which bade farewell to his faith in nature. Similar farewells to illusion were penned by the other romantics. There is a potential instability in natural supernaturalism. Romantic religion is vulnerable because it is a religion of this world. If one’s hopes and faith are pinned on the here and now, on the faith that reading, arithmetic, and morals will develop naturally out of human nature, then that faith may gradually decline when this world continually drips its disappointments.

So far, progressivism has proved somewhat invulnerable to its failures. But its walls are beginning to crumble, and none too soon. Only when widespread doubt is cast on public education’s endemic romanticism will we begin to see widespread improvements in achievement. Everyone grants that schooling must start from what is natural. But schooling cannot effectively stay mired there. With as much certainty as these things can be known, we know that analytical and explicit instruction works better than inductive, implicit instruction for most school learning. To be analytical and explicit in instruction is also to be artificial. Also, it is to be skeptical that children will naturally construct for themselves either knowledge or goodness.

The romantic thinks nature has a holy plan. The classicist, the modernist, and the pragmatist do not. And neither does the scientist. In the end, the most pressing questions in the education wars are not just empirical, scientific questions, but also ethical ones regarding the unfortunate social consequences of the progressive faith, especially the perpetuation of the test-score gaps among racial and economic groups. Are we to value the aesthetics of diversity and the theology of spilt religion above social justice? That is the unasked question that needs to be asked ever more insistently. Economic and political justice are strenuous goals. They cannot be achieved by doing what comes naturally.

–E.D. Hirsch is a professor of education and humanities at the University of Virginia and author of The Schools We Need and Why We Don’t Have Them. This article was adapted from a speech given at Harvard University in October 1999.

The post Romancing the Child appeared first on Education Next.

According to voucher advocates, parents are mainly concerned about school quality. In the absence of vouchers, only parents with enough money are able to seek out good schools by going private; but under a voucher system, they argue, with the cost of private education much reduced (or zero), many more parents would be able to—and would want to. This is particularly true for parents who are disadvantaged or trapped in failing public schools, for they have the most to gain from new opportunities. The upshot, advocates say, is that a voucher system would have very positive effects on society: it would get kids into better schools, give all schools incentives to perform, and promote social equity.

Photograph by Ralph Mercer.

Critics dispute all this. In their view, going private has little to do with school quality. The real motivations are largely social—and pernicious. Private schools have special appeal to people with money and education, who want their children separated from ordinary kids. They have special appeal to whites, who want to avoid blacks. And they have special appeal to the devoutly religious, who want schools of their own. So far, critics argue, the social downside has been limited because only 10 percent of American children go private. But if choice were expanded, pernicious motivations would be unleashed, and the education system would become more inequitable, more segregated, and more penetrated by religion.

These perspectives on parental choice couldn’t be more different or more fundamental to an assessment of vouchers. The question is: How do they seem to square with the facts? I have carried out research, based on a nationally representative survey of American parents, that provides some tentative answers. The details are reported in my book Schools, Vouchers, and the American Public (forthcoming in early 2001). Here I will briefly highlight some of the more basic themes and findings.

The Logic of Choice

Before turning to the data, let’s begin by thinking rather abstractly for the moment about the demand for private schooling. There is a logic to it, and this logic should tell us something about what to expect.

Under the current system, going private is costly, and parental choice is governed by a simple calculus. Parents will tend to go private if they can afford the tuition, and if the value they associate with going private—whether it derives from performance, religion, ideology, race, or other concerns—exceeds the costs. This calculus does not tell us what parents actually value. They may value performance, or they may not. They may value racial separation, or they may not. These are empirical issues that can be answered only by looking at the data. The simple fact that parents must weigh the benefits of going private against the financial costs, however, points to certain inferences that should guide our thinking.

Most obviously, private schools under the present system are more accessible to the financially well off, because they are better able to afford the tuition. The same applies for the well educated, both because education is highly correlated with income and because better educated parents tend to be more motivated by educational concerns. For these reasons, the current system should indeed promote a class bias in the types of parents going private, much as the critics of choice contend (and as supporters recognize). This bias would only be enhanced, of course, if some of these parents were elitist and sought separation for its own sake. But even if none were elitist, a class bias would be an inherent part of the system.

Now let’s consider what would happen if choice were vastly expanded, and parents were allowed—by means of vouchers, say—to send their children to private schools at no cost. The most immediate implication is that even the poorest parents—many of whom, by no coincidence, are trapped in our nation’s worst schools—would now find it within their means to pursue private schooling for their kids. Unless “other factors” (discrimination, for instance, or lack of information) prevented them from following through on their demand, then, many more low-income parents would probably go private than currently do, and the income biases associated with the current system would be reduced, perhaps drastically.

Biases due to education (controlling for income) might remain, because the better educated parents at all income levels would continue to be more motivated to seek out new opportunities. But in the aggregate, with income and education so highly correlated, and with information about private options increasingly available over time, many more people with low educational attainment would tend to find their way into the private sector—lowering the average level of parent education in that sector and thus leading to moderation on this score as well.

Here too we cannot say what parents might value about going private. But we can say that, whatever they value, the parents who currently go private are likely to be somewhat more “extreme” on these values than other parents are (because they care about them enough to pay for them)—and that, when the costs of choice are reduced, more people with these same values will be able to go private. It is reasonable to suggest, then, that the public parents who indicate an interest in going private should turn out to be similar on value grounds to the parents who already go private. They should just be more moderate.

In sum, if we think in analytical terms about the demand side of school choice, there are logical reasons for having certain expectations about what would happen if choice were expanded. Under the current system, in which choice is costly, private school choice can be expected to produce social biases that mirror some of the concerns of voucher critics. An expansion of choice, however, should not make these biases worse, as critics tend to argue. Rather, it is more reasonable to expect that just the opposite will happen, and that the biases of the existing system will actually be moderated, perhaps substantially.

The Data

The analysis is based on a 1995 survey designed to explore the voucher issue. The sample consists of 4,700 randomly selected adults from across the United States, including oversamples of 1,200 parents and 1,000 inner-city parents. With statistical weighting, the sample as a whole yields a representative cross-section of the population. The survey is supplemented by a separately collected data set on the demographics and academic performance of the school districts in which each respondent lives.

With these data, the desire of parents to go private (or not) can be explored by reference to a range of possible influences. Among the more notable are the following (the actual analysis includes many more than are listed here):

Background: income, education, religion, race, and political party.

Context: how advantaged the school district is (as measured by an index of test scores and socioeconomic characteristics) and whether the respondent chose where to live based on the schools.

Attitudes: support for diversity (racial integration), a perception of inequity (that the public schools provide a lower quality education for low-income and minority kids), support for voluntary prayer in the schools, support for greater parent influence, desire for smaller schools, belief in what I call the “public school ideology” (which measures a normative attachment to public schooling and its ideals), a belief in markets (that choice and competition are likely to make schools more effective), and a concern that moral values are poorly taught in the public schools.

Performance: an index of four items. One asks respondents whether the public schools are doing fine or are in need of major or minor change. Another asks them to rate the local public schools on a scale from 1 to 10. Another asks them whether their community is proud of its public schools. And still another asks them to compare the academic performance of the local public schools to private schools.

Who Goes Private under the Current System, and Why?

Now let’s take a first look at the evidence. Even voucher advocates would agree that, because private school choice is costly under the current system, parents who go private are likely to be more socially advantaged than parents who remain in the public schools. A simple descriptive comparison of parents in the two sectors documents as much. On average, private parents have higher incomes and more education than public parents, and they are more likely to be white. They also display the religious and partisan characteristics commonly associated with private schooling: they are more likely to be Catholic, born-again Christian, and Republican.

The existence of social biases, however, does not necessarily mean that the rest of the critics’ indictment, about pernicious motivations, is supported by the evidence. In fact, a multivariate (probit) analysis, incorporating all the background, context, attitude, and performance variables outlined earlier, suggests that it isn’t. Two findings stand out.

• Attitudes toward race (diversity) appear to have little to do with why parents go private: there is no indication that whites go private in order to flee blacks and other minorities.

• Of all the influences on parental choice, by far the most powerful is school performance: The less satisfied parents are with the performance of the public schools, the more likely they are to go private. The notion that private parents are really motivated by social concerns, and that performance matters little to them, misses the mark entirely.

This analysis also reveals that, aside from attitudes toward race, all the other attitudes in the model—inequity, public school ideology, prayer, moral values, school size, and markets (parent influence could not be included in this part of the analysis, for measurement reasons)—proved relevant to why these parents currently go private. The question is: do these same sorts of values and beliefs seem to be relevant for public school parents who, given the choice, would be interested in switching to private schools? We’ll soon find out.

Do Public Parents Want to Go Private?

Let’s focus our attention now on public school parents and see what we can learn about the reasons some of them are interested in going private.

The first step is to get a sense of how many public parents would like to go private if given a chance. What we are getting at here is a kind of hidden demand. For it is clear that parents who currently go private are doing so because they want to—but it is not clear that parents who currently send their kids to public school are doing what they want. Many might prefer to go private, but be unable to do so for cost reasons.

What is the extent of this hidden interest in private schools? The survey asks public parents the following question: “If you could afford it, would you be interested in sending your children to a private or parochial school?” The results are striking. They show that most public parents, 52 percent, would be interested in going private if money were not a problem, compared with 43 percent who say they would stay in the public sector. This is consistent with a 1999 survey by Public Agenda, which asked public parents a similar question and found that 57 percent were interested in going private.

The desire to go private is even stronger among low-income inner-city parents. In this group, 67 percent said they would be interested in leaving the public system. This is an early indication that, as advocates claim, choice has special appeal to the disadvantaged—and is not a policy whose support is grounded in elitism.

The notion that there is widespread interest among parents in going private is often disputed by critics. They cite figures that seem to show the opposite. A well-publicized report by the Carnegie Foundation for the Advancement of Teaching, for example, asked public parents the following question: “Is there some other school to which you would like to send your child? This could be public or private, inside or outside of your district, with your child’s grade level.” A full 70 percent said they would not switch, and just 19 percent indicated they would switch to a private school. Phi Delta Kappa (PDK) has included the same sort of question and gotten similar answers in its polls.

Properly interpreted, there is no conflict between these “opposing” sets of results. The Carnegie and PDK questions implicitly ask respondents if they know of some other school they regard as preferable to their current school. But most public parents have no incentive to be well informed about specific private schools (or even other public schools), so it is not surprising that they can’t point to specific schools where they’d like to send their kids. When asked whether they are interested in going private, however, most parents make it clear that they are. Both measures are useful and valid. They are just measuring different things.

Why Do Public Parents Want to Go Private?

Now let’s explore the desire to go private in greater depth. The question is: what would happen if choice were no longer costly, and all parents were given the option of sending their kids to private schools? Which parents would be most likely to go private, and why? Here are the highlights of a multivariate (probit) analysis that takes account of all the factors discussed earlier.

First, as the descriptive results for inner-city parents tend to suggest, an interest in private schools is especially high among low-income and minority (black and Hispanic) parents generally. When choice is freely available and income no longer a constraint, private schools have disproportionate appeal to those who are less well off, and whose need for new opportunities is clearly much greater. This association of choice with the disadvantaged is reinforced by the results for district context: it is the public parents in disadvantaged districts who are the most interested in going private.

Still, the more traditional influences associated with the current system continue to be relevant when choice is expanded. The public parents who want to go private tend to be better educated, Catholic, born-again, and Republican—suggesting that many of the same sorts of values and beliefs are at work.

The education result, it is important to note, reinforces a basic theme from the broader research literature: the parents who want to go private tend to be low-income and minority, but also (controlling for income and race) better educated. Choice advocates laud the equity-promoting effects of expanded choice and point to polls and programs showing that low-income families have a strong interest in going private. Yet opponents point out that, even within programs restricted to the poor, it is the better educated, more motivated poor who take greatest advantage of choice. And this, critics argue—not without reason—may give rise to certain inequities. These results provide support for both sets of claims.

Now let’s consider what values and beliefs seem to affect the desire to go private among public parents. Several findings deserve emphasis:

• Racial attitudes do not appear to play a significant role. This suggests that the critics’ claims about the pernicious effects of race, while perhaps justified decades ago, are probably wide of the mark today.

• Aside from race, all of the attitudes in the model—regarding inequity, public school ideology, prayer, moral values, parent influence, school size, and markets—appear to have an influence, and in the direction choice advocates would expect. Public parents appear to be influenced by the same basic values and beliefs that are important to current private parents. The consistent relevance of these concerns suggests that there is a common structure to the way parents think about their choice of schools.

• Of all the attitudes in the model, attitudes toward inequity stand out as the most salient. This dovetails nicely with our earlier results, which associate choice with the plight of the disadvantaged. Overall, as perceptions of inequity rise from low to high, the probability that a public parent will be interested in going private increases by 17 percent. If we break respondents down by income, moreover, we find that concerns about inequity are far more important to parents who are disadvantaged than they are to other parents. Among low-income parents, a growing sense of inequity makes them 26 percent more likely to be interested in private schools, as compared with shifts of 12 percent for middle-income parents and 11 percent for upper-income parents. The equity issue, then, seems to matter a great deal to disadvantaged parents, and they appear to connect it to private-school choice in a way that is entirely consistent with the argument voucher advocates have been making for the past decade: that choice is a way of promoting social equity.

• For parents as a whole, public school ideology is almost as important as equity concerns in shaping the desire to go private. Parents who score high on public school ideology are 13 percent less likely to be interested in going private than parents who score low. This underlines the pervasive role that normative commitments play in wedding parents to the public system and making them resistant to arguments for vouchers.

• Performance is by far the most powerful influence on the desire to go private. When satisfaction with public school performance drops from high to low, the probability that a public parent is interested in going private increases by 37 percent—which dwarfs the effects of all other variables. Performance looms as the number one consideration for parents. The familiar arguments to the contrary appear to be quite wrong.

Low-Income, Inner-City Parents

Now let’s take a closer look at inner-city parents. When we restrict the analysis to this population, one finding stands in sharp contrast to those we uncovered for parents generally. Among inner-city parents, it appears that race does matter, and in just the way critics have argued: White parents who are opposed to diversity are especially interested in going private. The obvious interpretation is that, because race is a more salient issue in the inner city than elsewhere, many whites see private schools as a way to avoid integration with minorities.

This is the first evidence (within our own study) that separatism and possibly even bigotry may be motivating some of the parental interest in private schools. And it makes sense that such effects would show up, if they show up anywhere, for low-income whites in the inner city—for these are the whites who are most directly affected by policies of diversity. Whites in the suburbs, and whites with money, are much more removed from the reality of integration.

There is a more benign interpretation, however. It may be that our diversity variable has little to say about racism per se, but is really measuring the extent to which respondents value diversity. People who score low are not necessarily racists. They just don’t value diversity as much as people who score high. Thus the model’s results may simply be telling us that whites who support diversity are especially inclined to stay in the public sector. Whites who lack that positive motivation are less wedded to the public schools and more open to private options, but this does not mean they are racists.

We should be careful, then, about jumping to conclusions. Nonetheless, it appears that race is relevant to the way inner-city whites approach going private, and this raises a red flag that choice advocates and program designers need to be concerned about. On this count, the critics may be right.

Aside from this finding about race, two other findings from this analysis of inner-city parents need to be underlined. The first is that their desire to go private is heavily influenced by their sense that the current system is inequitable. Indeed, race aside, inequity is far more powerful than any other attitude in the model—and indicates, once again, that disadvantaged parents make a strong connection between equity and school choice.

The second finding concerns performance. The critics of choice have long argued that low-income parents from the inner city are particularly unlikely to be guided by performance criteria. They construct an image of parents who need the help of school administrators and government agencies, and, if left to their own devices, would make ill-informed decisions that are not motivated by school quality. The empirical results, however, again show that performance is the most powerful factor in the entire analysis. When satisfaction with school performance drops from high to low, the probability that a low-income parent will express interest in going private increases by 30 percent, which is quite a large shift indeed. In this key respect, they look a lot like all the other parents. They are primarily interested in getting good schools for their children.

The New Public and Private Sectors

What would the public and private sectors look like under a system of expanded choice? Our analysis gives us a basis for drawing some provocative inferences.

Let’s begin by recognizing that, in reality, only some of the “swing” parents—the public parents interested in going private—would actually make the switch. The fact is, of course, that only some would actually want to: for they have merely indicated an interest in going private, not that they would actually do so, and there may be all sorts of reasons they would eventually stay put. Moreover, the private sector would have to expand tremendously to absorb them all, and it could not immediately do this. For the short run, demand would exceed supply, leaving many swing parents in the public sector.

In the analysis that follows, I assume that the most interested swing parents are the ones who make the switch. These I identify as the parents who, in the prior probit analysis, are estimated to be at the high end—the upper half—of the interest scale. Given this scenario, half of the swing parents switch from public to private, half remain in the public sector—and the public sector thereby retains about 75 percent of its families overall. Empirically, this seems very plausible.

Suppose now that the new private system consists of the existing private parents plus the swing parents most interested in going private, while the new public system consists of the swing parents who are less interested plus the remaining public parents. How do the new systems compare to the original ones?

Even when just half of the swing parents go private, an expansion of choice dramatically transforms the private sector along almost every social dimension. Compared to existing private parents, the new recruits are substantially lower in income and education (see Figure 1), more likely to come from disadvantaged districts, and more likely to be black or Hispanic. When these recruits become part of the new private sector, the usual social biases associated with private schooling are vastly reduced. Indeed, minorities now make up 33 percent of the transformed private sector (see Figure 2). As these changes almost dictate, there is also a shift in political partisanship: for with the socially disadvantaged comes a big influx of Democrats, and the original Republican bias is reduced considerably.

There is one social bias, however, that is reduced only slightly: religion. The parents making up the new private sector are almost as likely to be Catholic as current private parents are, and they are just as likely to be born-again. Even with the influx of disadvantaged parents, therefore, the private sector retains much of its religious character.

This same expansion of choice does not lead to major changes in the public sector. It remains pretty much as before, except that it is somewhat higher in income, contains proportionately fewer minorities, and retains more parents who are from better districts. Again, this happens because many of the disadvantaged choose to leave, and the parents who stay tend to be rather advantaged. But overall, the public sector is not affected by the expansion of choice to nearly the extent the private sector is—and the reason is pretty obvious. The current private sector is small and relatively select (due to the tuition requirement), and it is transformed by the influx of new recruits. The public sector is already large and heterogeneous, and it remains so even after many parents leave. It is affected at the margins.

Given the changes occurring in both sectors, how does an expansion of choice appear to affect the social gap between public and private? The transformation of the private sector does not succeed in eliminating all vestiges of the original gap. The new private sector is still somewhat higher in education and income than the public sector, and more Catholic and Republican. This is due to the new recruits’ (many of them disadvantaged) having been averaged in with current private parents (many of them fairly advantaged), and there are not enough of the former—under my assumption—to outweigh the latter.

Nonetheless, the changes are considerable. The effect of choice is to reduce the social differences between public and private—and thus to promote moderation—on every one of the dimensions we’ve considered. Moderation is weakest for religion, testifying to the tenacity of religion in the private-school equation. But the degree of moderation is quite pronounced for income and education. The income gap in particular is dramatically reduced, leaving only slight differences between public and private. On other social grounds, moreover, the expansion of choice goes further than this and actually reverses the traditional association of private schooling with social advantage. Most important, parents in the new private sector are actually more likely to be black or Hispanic than public sector parents are.

These results, of course, are not chiseled in stone. With somewhat different assumptions about how many parents switch from public to private and exactly who they are, the details of the analysis would be somewhat different. It is reasonable to suggest, however, that its basic thrust would remain essentially the same: that choice tends to break down existing biases of social advantage.

Conclusion

For the most part, the evidence from this study tends to support the claims of voucher advocates and to contradict those of critics. The appeal of private schools is especially strong among parents who are low in income, minority, and live in low-performing districts: precisely the parents who are the most disadvantaged under the current system. They would be the ones who disproportionately take advantage of an expansion of choice in education—and their shift from public to private should tend to produce a very substantial measure of social moderation, rather than the worsening of social biases that critics say would occur.

Critics also argue that performance has little to do with why parents find private schools attractive, that the real reasons are rooted in elitism, racial separation, and religion. It is true that religious and moral values play important roles in the current system and would continue to do so under a system of expanded choice. But the evidence suggests that school performance is the single most important factor in the choice to go private—and that elitism and racial separation have little to do with it.

Some of the concerns critics raise, however, do find limited support in the data. One is that racial motives may play a role among low-income whites in the inner city. The other is that, even among the disadvantaged, those with higher levels of education are more likely to go private (although I should emphasize that, because disadvantaged people are so interested in going private and because they are poorly educated as a group, the aggregate effect of choice is to lower the average education levels of private-sector parents, not to raise them). Both of these results point to possible problems that choice advocates and program designers need to be aware of in pursuing voucher plans that are truly equitable.

We also need to recognize that there is more to the critics’ argument than our data can address. The focus here has been on the demand for private schools, and thus on what parents want. But parents who want to go private might not be able to do so in their everyday lives for reasons that choice advocates too often dismiss. In the real world, even if everyone had the right to choose their schools, parents who are educated and financially well off are likely to be more motivated than other parents, to have better information about their alternatives, to have more resources at their disposal for getting their way, to have better social connections and more attractive opportunities, and to have children who are easier and less costly to teach. On the supply side, moreover, private schools may find these advantaged parents (and their children) desirable, and may discriminate against poor and minority families.

These are legitimate concerns. Critics are right to emphasize them, and choice advocates need to take them into account as they think about the proper design of choice systems. With the right designs, these problems may be mitigated and parental demand more freely pursued. In any event, this is an analysis of the demand side of the equation only, and what it has to tell us about the social effects of choice should be understood to hold as long as “other things are equal.” It tells just part of what must ultimately be a much larger story.

Nonetheless, the part that it tells is quite fundamental—and quite positive for the choice movement, which clearly has much to build on in attracting parents to its cause.

—Terry M. Moe is a professor of political science at Stanford University and a senior fellow at the Hoover Institution.

The post Hidden Demand appeared first on Education Next.

ILLUSTRATION BY DAN VASCONCELLOS.

What is strange about the debate over teacher policy, however, is that it has taken place largely within the confines of the existing public school system. Hardly anyone talks about how the growing movement toward parental choice and competition, in the form of vouchers and charter schools, will affect the teaching profession. Only the teachers unions seem to hold a clear, if unspoken, opinion on this question. Though they never say so, the unions obviously think that school choice would harm teachers. Both the National Education Association (NEA) and the AFT stand resolutely opposed to vouchers, and their attitudes toward charter schools have ranged from tepid support to outright hostility. While these unions don’t craft their agendas solely with the interests of teachers in mind, they seldom—if ever—oppose policies that would benefit their members.

Yet economic theory suggests that school choice would change the teaching profession in ways that would fulfill many of the reform movement’s goals. In short, theory predicts that schools that faced stronger competition would favor teachers who raised the schools’ ability to attract students. These schools presumably would strive to attract and retain teachers who were especially talented or hard-working or who possessed rare skills. In turn, you would expect their tolerance for less-effective teachers to wane. You would expect, in fact, that teaching would be transformed into a true profession, where workers are rewarded not only on the basis of seniority but also on the basis of their skills and performance.

Making a Profession a Profession

In the current system, teachers unions tend to compress wages within a school district so that teachers with the same seniority and the same degree are likely to receive similar (if not identical) salaries. Even nonunionized districts tend to adopt salary scales that resemble union scales. So teachers who excel at their jobs or teach hard-to-staff subjects, such as math and science, are paid the same as if they were mediocre or could be replaced easily.

This is one reason why teaching, as it stands, is not as attractive to candidates with high aptitude, a strong work ethic, or math and science skills. The evidence suggests that such people are less likely to start teaching than their lower-aptitude peers (even when those being compared are all certified to teach) and less likely to remain in teaching. This situation has worsened over time, as other professions such as management, law, and medicine have opened their doors to women. Women with high aptitude or math and science skills have chosen such professions over teaching, perhaps because these professions do reward workers based on their skills and performance. By making teaching a more market-oriented profession, school choice might also make it a more attractive career to people who would thrive in such an environment.

Why might teachers welcome such a shift? In a word, professionalism. Both the NEA and the AFT support so-called professionalization agendas that aim to make the route into teaching look more like the routes into the legal and medical professions. Much as lawyers and doctors ensure quality within their professions through bar examinations and board certification, the AFT and the NEA call for higher certification standards for new teachers (but rarely for current teachers), apprenticeships, peer review, and rewards for teachers who earn additional credentials, such as National Board certification. The movement also includes proposals to give teachers more control over the profession through independent, state-level standard-setting boards and to give the National Council for the Accreditation of Teacher Education, the body that accredits education schools, more control over the number and origin of undergraduate degrees granted in education each year.

In most professions, however, credentials such as board certification merely maintain minimum standards. The size of a professional’s client list or how much a professional earns usually depends on how the market values her services, since professionals typically compete in a market setting as employees of a competitive firm, in self-employment, or in partnership. The teachers unions, however, want to use credentials to restrict the supply of teachers while opposing competition, making their range of policies more characteristic of a craft union than of a profession. Professionalization in this sense is a misnomer for the current movement. Yet the name and rhetoric of the movement suggest that many teachers do want teaching to become more professional. Economic theory suggests that school choice would make it so, in the sense that it would raise demand for highly talented and skilled teachers who receive rewards closely linked to their performance.

Just a Theory?

Without evidence, of course, this is just a theory. And without full-fledged school choice, it is a tough theory to test, though not impossible.

Comparisons between public and private school teachers already give us some insight into how choice might affect the teaching profession. After all, the revenues of private schools depend on their ability to attract tuition-paying students. They thus have an added incentive to hire teachers who will help them do so. What do they look for? In a comprehensive comparison of private and public school teachers, economists Dale Ballou of the University of Massachusetts Amherst and Michael Podgursky of the University of Missouri–Columbia found that, in making hiring decisions, private schools value high aptitude more than public schools do. They also found that salaries in private schools correspond to aptitude and scarce skills (such as math and science skills) more than they do in public schools.

Private schools, however, are not a perfect guide to what teachers will experience under a system of school choice. Parents who pay private-school tuition must also continue to pay taxes that support public schools. Private schools thus face financial constraints that a true school of choice would not. As a result, they typically pay teacher salaries that are about 60 percent of local public school salaries.

To get a broader picture of how choice affects teachers, I used data both from traditional forms of school choice (choice among public schools through choice of residence and choice among private schools) and from charter schools. My analytical strategy relied on a simple economic argument: If schools that face stronger competition prefer certain characteristics in their teachers, then they should hire more teachers who possess those characteristics and pay them a higher wage. In economic terms, they would show greater demand for teachers with those characteristics; they would be willing to pay a higher price in order to buy more of them.

My findings suggest that school choice would change the teaching profession in significant ways. Schools that face tougher competition have greater demand for teachers who attended well-regarded colleges, majored in subject areas (especially in math and science), and who put in more effort and show more independence. These schools are more likely to hire such teachers and to pay them higher wages than they would earn in schools that face less competition. In general, they also have less demand for certification and master’s degrees. They pay teachers who hold such credentials less than similarly educated teachers earn in schools that are less choice-driven.

Making Choices

Before I discuss the results of this study, let me briefly explain the measures of choice I used and where some of the data came from.

Within the present K–12 education system, parents and students can make three types of choices. First, they can choose their public schools by choosing where to live (this is called Tiebout choice, after the economist Charles Tiebout, who called attention to its importance). Some metropolitan areas, however, contain more school districts than others, allowing for more choices among public schools. Compare the cities of Miami and Boston, for instance. In Miami, one school district encompasses the entire metropolitan area. Parents have almost no choice among public schools. Boston, by contrast, has 70 districts within a 30-minute commute of downtown and even more in the surrounding metropolitan area. In an area with so many choices, a town’s reputation and real-estate values depend heavily on the quality of its local school district, giving residents added incentive to ensure that their schools compare favorably with those in other towns.

Various factors besides the degree of choice might affect a district’s demand for certain qualities in its teachers. For instance, a school district in an area with high employment in the technology field might show greater demand for teachers who possess skills in math and science. But it also might be a school district that happens to face strong competition. Without controlling for such a coincidence, any comparison would seem to show that competition affected the demand for math and science skills in this district, when, in fact, another factor contributed to the district’s demand as well. Likewise, teachers in an area where the population is more educated tend to have attended more selective colleges. You must adjust the data to take account of this, or they may give you the false impression that competition raises the demand for high-aptitude teachers. Other factors, such as household income, ethnic makeup, and union membership among teachers, can also affect a district’s demand for certain qualities. After controlling for all these factors, I compared teachers in areas where parents have more choices among public schools with teachers in areas where they have fewer.

Second, parents can choose to send their kids to private school. But again, some metropolitan areas have more low-tuition private-school options than others, mainly due to historical variations in religious populations. Private schools in areas with large, long-standing religious populations tend to have higher endowments and more established donor bases. This allows the schools to charge a tuition that is lower than the actual cost of educating a child, thereby making it easier for parents to choose a private school. In some metropolitan areas, up to 15 percent of the elementary school population is enrolled in private schools where the tuition is about two-thirds of the per-pupil expenditure. (Religiously affiliated schools typically charge $1,600 in tuition while spending about $2,300 per student.) Other metropolitan areas have less than 1 percent of their elementary school populations enrolled in such schools. They may have seats available in schools where the tuition is higher and there are no tuition subsidies, but this hinders a parent’s ability to choose a private school. After again controlling for factors other than competition that might affect a private school’s demand for certain teachers, I compared teachers in areas where parents have more choices among private schools with teachers in areas where they have fewer.

Finally, and most recently, parents can choose to send their kids to a charter school. I compared teachers in charter schools with those in private and public schools in the same regions. Here you must control for the fact that the existence of a charter school may reflect factors in an area that also determine the area’s demand for certain teacher characteristics. I also tested for composition effects, meaning that a charter school might enter an area and favor the same teacher characteristics that the public schools want. After the charter entered the area, you might see teachers with that characteristic slowly draining away from the public schools. This is unlikely, given that a charter school has little incentive to move into an area where schools are already doing what it plans to do, but I tested for it anyway. For the comparison among charter, public, and private school teachers, I assumed that charter and private schools face more competition than public schools, since a greater share of charter and private schools get funding only if they attract students.

For both public and private schools, I used data from the Schools and Staffing Survey (SASS), a comprehensive survey of public and private school teachers and administrators. I examined variables such as salary, years of teaching experience, whether teachers planned to stay in teaching for the next few years, the number of hours per week (on top of required hours) that teachers spent on activities related to their students’ academic progress, and the number of hours that they spent on their students’ extracurricular activities. I also looked at where they earned their baccalaureate degrees, whether they were math or science majors, the number of undergraduate and graduate courses they took in math and science, and whether they earned their degrees in a field of the arts and sciences (rather than in education).

The SASS does not cover charter schools, so I had to conduct a survey of my own. I distributed surveys to one administrator and two randomly selected teachers in every charter school that was open as of October 1998. The survey questions were taken from the SASS. Data from the survey will be described in a forthcoming monograph.

Results

Remember that my test for whether school choice raises demand for certain teacher characteristics is two-fold: 1) whether a school that faces stronger competition hires teachers with more of a certain characteristic; and 2) whether that characteristic earns a premium in an environment of greater school choice. Remember also that I performed two different comparisons: 1) a within-sector comparison of teachers in both the public and private sectors; and 2) an across-sector comparison of teachers in public, private, and charter schools.

The first question I asked was whether competition raises demand for teachers who possess high aptitude. This is important to know because research has shown that teachers’ aptitude, as measured by scores on standardized tests, significantly affects student achievement. The SASS, unfortunately, does not have a measure of the aptitude of individual teachers, since only some states require teachers to take proficiency tests. Even if they did, teachers would not have taken comparable tests anyway. The SASS does, however, list which college teachers attended. This is a good measure of the quality of their education and, to some extent, of their aptitude. Colleges take account of students’ aptitude, high-school grades, extracurricular activities, and character in making acceptance decisions, so the selectivity of a teacher’s college summarizes a complex set of information on aptitude, work ethic, leadership, creativity, and so on.

The most obvious way to use a teacher’s college as a proxy for aptitude would be to assign that teacher the college’s average SAT score. But most teachers come from colleges that range from nonselective to somewhat selective, where SAT scores are poorly defined because so many students do not take admissions tests. Their average SAT scores would not be an accurate proxy for an individual teacher’s aptitude. So, instead of SAT scores, I used the rating each college received in the 1982 Barron’s Profiles of American Colleges. I chose the 1982 version because it represented a middle ground: Older teachers would have graduated before 1982, younger teachers after. In any case, the rankings are fairly stable from year to year, so the choice of year doesn’t make much difference. I assigned the teachers from each school a number that corresponds to that school’s Barron’s rating, from 9 for Barron’s top rating, “most competitive,” to 1 for its lowest, “nonselective,” which is reserved for schools that will accept nearly anyone with a high school diploma or GED.

My first finding was not so surprising. Several scholars have shown that teachers are drawn mainly from the bottom half of the college-going population’s aptitude distribution. Likewise, most of the teachers in the SASS sample came from schools rated between 4 (“competitive plus”) and 1 (nonselective).

My findings also suggest, however, that enhanced choice and competition could help to remedy this situation and thereby bolster the skills of the nation’s teaching corps. I found that teachers who work in areas with higher Tiebout and private school choice attended more selective colleges. Teachers in areas of maximal Tiebout choice attended colleges that were ranked 0.4 levels higher than in areas of minimal Tiebout choice. Areas with a high degree of private school choice have teachers who attended colleges that were ranked 0.1 levels higher than areas with minimal private school choice.

Because most teachers attended schools that rated in the 1–4 range, I was also interested in whether competition raised demand for teachers who went to schools rated 4 (competitive plus) or higher. It appears to. For instance, about 20 percent of public school teachers went to such schools, compared with 36 percent of charter school teachers and 36 percent of private school teachers (see Figure 1). In public schools, teachers from such colleges earn 3 percent more than their peers. In charter schools, they are paid 6 percent more, suggesting that charter schools have greater demand for teachers who have graduated from colleges that rate at least “competitive plus.” Their need to attract students in order to receive any funding seems to drive their hiring of higher-aptitude teachers.

Similarly, a teacher in an area with many public school choices (maximal Tiebout choice) is 8 percent more likely than a teacher in an area of minimal Tiebout choice to have attended a college that rates at least “competitive plus.” Teachers who attended such colleges also earn 8 percent more than their peers. A high degree of private school choice boosts the likelihood that a teacher attended a college rated at least “competitive plus” by 5 percent. The degree of private school choice does not have a meaningful effect on the wage for this characteristic.

Skills in Short Supply

My second question was motivated by the evidence that schools face a shortage of teachers who have math and science skills (only 7 percent of teachers in the United States were math or science majors). Indeed, schools need teachers who have baccalaureate degrees in any field of the arts and sciences (as opposed to education) because they have subject knowledge that is useful in grades 7 through 12.

Again, my findings suggest that enhanced competition and choice may help to ameliorate these shortages. Stronger competitive pressures appear to raise the demand for teachers who have math and science skills. In an area of maximal Tiebout choice, a teacher is 15 percent more likely to have majored in math or science than a teacher in an area of minimal Tiebout choice. A teacher in an area with a high degree of private school choice is 10 percent more likely to have majored in math or science than a teacher in an area with minimal private school choice. Teachers who majored in math or science earn 16 percent more if they work in an area with maximal Tiebout choice than they would if they worked in an area with minimal Tiebout choice. In private schools, math or science majors earn 14 percent more if they work in an area with a high degree of private-school choice.

In charter schools, 10 percent of teachers are math and science majors, versus 8 percent of public school teachers and 7 percent of private school teachers. In the public sector, math and science majors earn about 4 percent more; in charters, 8 percent.

The results also suggest that schools facing more competition are better at retaining such teachers. In general, math and science teachers are more likely to leave teaching.They have 1.2 fewer years of experience and are 6 percent less likely to say that they plan to keep teaching than other teachers. But in an area of maximal Tiebout choice and a high degree of private school choice, math and science teachers have only .4 fewer years of experience and are no less likely to say that they plan to continue teaching. This may be a result of such teachers’ being paid more in districts that face more choice-based incentives.

Schools that face stronger competition also appear to favor teachers who majored in subject areas. Of teachers in charter schools, 56 percent majored in a field of the arts and sciences (as opposed to education), compared to 37 percent of public school teachers and 42 percent of private school teachers (see Figure 1).

Effort and Independence

I also wanted to know whether school choice increased demand for teachers who put in more effort and show more independence. I used data that measure the number of hours teachers spend on instructional tasks (such as tutoring) and on noninstructional tasks (such as directing a school play) beyond the hours required in their contracts. The measure of independence is a teacher’s assessment of the control that she exercises over her teaching (particularly teaching methods and the organization of material).

In general, schools that face more Tiebout choice demand more effort and independence from their teachers, while the effect of private school choice is unclear. Teachers in an area with maximal Tiebout choice spend two extra hours per week on instructional work and one extra hour per week on noninstructional work than teachers in an area of minimal Tiebout choice. A high degree of Tiebout choice also raises the wage paid for every extra instructional hour per week by .7 percent and for every extra noninstructional hour per week by .3 percent.

Charter school teachers work 13 extra instructional hours per week, versus 9 extra hours for public school teachers and 9 extra hours for private school teachers. The difference between the charter and public school teachers is highly statistically significant. In public schools, teachers are not compensated for the extra instructional hours they work; in charter schools, they are paid 5 percent more for an extra 13 hours of work. In public schools, teachers claim their level of control over their teaching is 4.8 on a 6-point scale, while charter and private school teachers rate their level of control at 5.6 on a 6-point scale.

The professionalization movement among teachers includes a sharp focus on certification and on the importance of holding a master’s degree, so I was interested in how choice affects schools’ demand for such credentials. It is important to note that many teachers-union contracts specify salary increases for master’s degrees and for being certified. So examining the data shows whether choice motivates schools to make such salary increases larger than they would be in the absence of choice.

Overall, about 44 percent of public school teachers, 41 percent of charter school teachers, and 28 percent of private school teachers have master’s degrees (see Figure 2). The public sector pays teachers who hold master’s degrees about 25 percent more; in charter schools, they are paid about 20 percent more. Furthermore, nearly all public school teachers are certified, while only 87 percent of charter school and 65 percent of private school teachers are. In private schools, teachers who hold certification are paid 1.4 percent less than uncertified teachers; charter schools give a similarly negative premium, but this finding is not statistically significant. This suggests that private schools may slightly prefer uncertified teachers and that charter schools probably have less demand for them than public schools do.

Another way to look at this question is whether teachers actually teach in the areas for which they are certified. Most states put few restrictions on teachers—once certified—taking on assignments that are relatively remote from the fields in which they were certified. About 97 percent of public school teachers claim to be certified in their teaching area, while only 83 percent of charter school and 54 percent of private school teachers do (see Figure 2). Public school teachers who teach in their areas of certification earn a substantial wage premium, 9 percent, compared with a premium that is not meaningfully different from zero for charter teachers and a 2 percent premium for private school teachers. All in all, it appears that public schools have a greater demand for degrees and certification than private or charter schools.

Broadly speaking, my findings suggest that enhanced competition and choice raise the demand for high aptitude, skills in math and science, subject-area expertise, effort, and perhaps independence among teachers. Choice also seems to lower schools’ demand for certification and master’s degrees. These findings further suggest that school choice has the potential to create a professional environment for teachers in which more motivated and skilled teachers earn higher pay for such qualities. For new teachers, this could quickly change the profession in areas where a growing number of choice schools offer a large share of the new teaching positions. It would take longer for veteran teachers to feel the pinch. They would notice a change only when their schools began to feel competitive pressures and, as a result, began to demand teachers whose characteristics attract parents. Some teachers, obviously, would dislike such changes. Less able or less motivated incumbent teachers might find themselves earning smaller salary increases than some of their peers. Such teachers might be more likely to leave the teaching profession. That would be a welcome reversal of the current pattern, where the most able teachers are also the most likely to exit early.

–Caroline M. Hoxby is an associate professor of economics at Harvard University and a visiting fellow at the Hoover Institution, Stanford University.

The post Changing the Profession appeared first on Education Next.

by David W. Grissmer, Ann Flanagan, Jennifer Kawata, and Stephanie Williamson

RAND Corporation, 2000.

In the summer of 2000, perfectly timed to shape the election debate over education reform, came a new RAND study that claimed to contradict the conventional research wisdom on the connection between school expenditures and class size on the one hand and student achievement on the other. “Our results certainly challenge the traditional view of public education as ‘unreformable,’” the study’s director, David Grissmer, said in an accompanying press release. “But the achievement of disadvantaged students is still substantially affected by inadequate resources. Stronger federal compensatory programs are required to address this inequity.” While academic studies usually retire to footnote-land, a well-orchestrated PR blitz pushed the RAND report to the front pages. It even earned prominent campaign mentions: Both presidential candidates commandeered the study’s findings to their own ends—Al Gore to support his proposal to lower class sizes, George W. Bush to trumpet Texas’s accountability system.

A trusted name like RAND lent instant credibility to the study’s results—so much credibility that the major newspapers reported the findings without even a question mark. This, combined with the lack of statistical expertise among journalists and the crushing deadlines under which they work, allowed RAND to sculpt the dissemination of its results with a carefully worded press release that pumped its most provocative yet methodologically flawed conclusions. “The education reforms of the 1980s and 1990s seem to be working,” the release began. It went on to highlight the report’s finding that “[d]ifferences in state scores for students with similar families can be explained, in part, by per-pupil expenditures and how these funds are allocated.” In particular, RAND reported that, other things being equal, National Assessment of Educational Progress (NAEP) scores in math are higher in states that have:

• higher per-pupil expenditures
• lower pupil-teacher ratios in the early grades
• higher percentages of teachers reporting that they have adequate resources
• more children in public prekindergarten programs
• lower teacher turnover

These highlights were asserted without qualification or doubt, without any mention of weaknesses in the data or the analysis. RAND was notably less charitable with results that accorded with past research findings. The press release at least mentioned the study’s finding that “having a higher percentage of teachers with master’s degrees and extensive teaching experience appears to have comparatively little effect on student achievement across states. Higher salaries also showed little effect.” But here the authors were sure to qualify their findings, carefully emphasizing that “salary differences may have more important achievement effects within states than between states.” The authors quickly rushed past these less popular findings to boldly propose specific policy interventions: “To raise achievement scores, the most efficient and effective use of education dollars is to target states with higher proportions of minority and disadvantaged students with funding for lower pupil-teacher ratios, more widespread prekindergarten efforts, and more adequate teaching resources.” In short, any reader of the news release—or the articles it generated—might have reasonably concluded that RAND, the highly respected think tank, had overturned years of research (including this author’s).

What research does the RAND study purport to contradict? Between 1960 and 1995, per-pupil spending in the United States (in constant 1996–97 dollars) grew dramatically, from $2,122 to $6,434, a threefold increase. This trend cannot be explained by the country’s increased commitment to disabled students, which at most accounts for just 20 percent of the increase. At the same time that costs were rising, the student-to-teacher ratio fell by about a third, from 26:1 to 17:1. Nevertheless, despite our greatly enhanced commitments to public education—and despite the fact that children are growing up in better-educated and smaller families than ever before—student performance during this period, as measured by NAEP test scores for high school seniors in math and reading, moved hardly a hair’s breadth. Complementing these overall trends are more than 400 studies that have searched for a connection between spending and achievement in particular schools, districts, and, occasionally, states. In general, these studies have been unable to detect any consistent, positive relationship between increased resources and student learning.

This is not to say that schools don’t matter. The best of these studies, so-called value-added studies that concentrate on the determinants of growth in achievement across individual classrooms, find that differences in teacher quality have a profound impact. But they also find that teacher quality is not closely related to school resources. The only studies that consistently find positive effects of resources are those that rely on student performance and school data averaged across all students and schools in a state.These aggregate studies, of which the RAND study is one, rely on limited data and are prone to serious statistical shortcomings, so they have been heavily discounted in the past. Undaunted, RAND’s researchers argue that their results should lead to a reinterpretation of three decades’ research.

However much they might protest, RAND’s researchers for the most part have only confirmed what has been known all along. In fact, the RAND study is startling in its conformity to conventional wisdom. RAND’s best model for estimating the impact of spending increases on student performance yields an estimate that an additional $1,000 per student—a $50 billion annual increase nationally—would yield a rise in performance of about two percentile points (just 0.05 standard deviations), a trivial impact (see Figure 1). Moreover, the RAND study repeats the finding that teachers’ salaries, experience, and whether or not they hold a master’s degree bear little or no relationship to student performance.

High Costs, Low Returns (Figure 1) Though RAND said resources were inadequate, it actually found that huge increases in spending would raise test scores by only a trivial amount. National cost of obtaining a 2 percentile increase in NAEP performance using RAND cost estimates*

*The hardcopy of Education Matters erroneously refers to Normal Curve Equivalents (NCE) instead of percentiles in the text and presents cost estimates for changing NCE scores by two points in Figure 1. NCEs are a transformed version of percentile scores that follow a normal distribution with a mean of 50 and a standard deviation of 20.

What about the study’s most celebrated finding, on the impact of class size? The study found that class size, as measured by a state’s average pupil-to-teacher ratio, has a minuscule impact on the performance of the average student. At best, the RAND study is just another in a long list of reports that have demonstrated the minimal impact of school resources on the typical student’s performance. RAND attempts to distance itself from the conventional research wisdom by declaring that “money, if spent appropriately, is productive.” But who would be surprised by such a tautology? Only if we are told exactly which expenditures are productive can the study give much guidance. But the RAND study’s data are too weak and its methodology too flawed to support the specific policy recommendations its authors make.

A Sow’s Ear

Be skeptical when a research analyst tells you he has fashioned a silk purse out of the proverbial sow’s ear. Consider the limitations of the data with which RAND was working. The study’s sample consisted of 44 independent observations—the states that voluntarily participated in one or more of seven NAEP tests that were administered from 1990 to 1996. Moreover, the number of states participating in any one test varied from 35 to 44. Tests were given in 8th grade math in 1990, 1992, and 1996; in 4th grade math in 1992 and 1996; and in 4th grade reading in 1992 and 1994. Although RAND attempted a variety of analytical methods, its general approach was to estimate the impact of family background and measures of school resources on average student performance on as many of these tests as were administered in a given state.

The NAEP tests themselves have certain advantages.They have been carefully designed, the same test is given in all states, and they allow for comparisons from one time period to the next. Schools have few incentives to score high on the NAEP, leaving little chance that much “cheating” or “teaching to the test” goes on. It is troublesome that, when asked, a sizable number of schools exercise their right to refuse to participate in NAEP testing. Despite this drawback, though, the NAEP remains one of the best available measures of average student performance in most states.

But RAND’s analysis of the NAEP scores is another matter. First of all, 44 observations is a very small sample, so drawing any strong, statistically valid conclusions is at best difficult, at worst misleading. Moreover, data collected at the state level are marvelously imprecise. These aggregate data ignore the enormous differences within a state—implicitly assuming that the past three decades of legal challenges to the inequitable distribution of resources among well-to-do and poor school districts are groundless. When all these differences are averaged away so that it is impossible to identify their importance, how can we possibly have high quality data that trump all previous research on the subject?

While the measure of student performance with which RAND was working was adequate, not much else was. RAND attempted to control for the family background of the students taking the test, but the only information on family background available to RAND was census figures on the average statewide education and income of school-age families in 1990. RAND attempted to adjust these data to the actual years the students took the tests by assuming that these factors change precisely with changes in the racial composition of test-takers. But, of course, one cannot assume that the education and income of students of different racial groups change at the same rate in all 44 states. And RAND did not have any information from individual students; throughout its analysis it refers to average results across the state. So, from the very beginning, RAND was forced to work with an imprecise measure of the characteristics of students who actually took the tests.

Similarly, RAND used statewide averages as its measure of school resources, an extremely imprecise indicator of the actual resources being spent on particular students who attend specific schools. RAND also relied on statewide averages of teachers’ impressions of whether their school supplies were adequate, statewide averages of prekindergarten attendance, and statewide averages of class size. These averages obviously mask wide disparities within a state.

The RAND researchers insist that their study is superior because they factored in the average school resources for all the years that students were in school, a measure they find superior to studies that look only at current resources being spent on a student. This may be a worthy research innovation, though the average school resources available to a student from one year to the next do not change dramatically—unless the student moves, something that happens with surprising frequency. In 1995, 6 percent of the school-age population lived in a different state than they had in 1990; another 2.5 percent had been living outside the United States in 1990. These percentages vary widely among regions of the country. In the mountain states, as many as 15 percent of students had lived elsewhere in 1990. None of this movement was taken into account by the RAND study.

Where Are the Reforms?

These weaknesses in the data were exacerbated once RAND tried to glean specific policy recommendations from its findings. For instance, RAND says that we should reduce class size in states with higher shares of disadvantaged students. RAND, however, doesn’t ever look at whether or not disadvantaged students are in large classes, because they have averaged across all students in the state.

RAND also says that students perform better when teachers think that their supplies are adequate. This finding is plausible. If teachers have adequate materials, one would expect them to be more effective. But it suffers from the chicken-and-egg problem: We can’t be sure whether high-performing students make teachers feel better about their supplies, or whether the supplies themselves have a causal impact. And, of course, this subjective question means wildly different things to teachers in different schools and states.

Much the same can be said for the finding that low teacher mobility leads to higher student performance. Do high levels of teacher mobility lower student performance, or does low performance increase the chances that teachers will move on? One simply cannot tell from the kind of data with which RAND was working.

The authors admit that it would be preferable to have data from the schools that students actually attended. But they claim that using statewide data allows them to consider the fact that the states, not local school boards, are the ultimate political entities responsible for public education within their boundaries. Only by looking at states as a whole can one incorporate the panoply of state policies that may influence school achievement, RAND says. To be sure, statewide analyses can provide accurate estimates of the impact of school resources—but only if the analyst includes within the statistical model all the factors that affect student performance and, in the standard linear regression model generally favored by RAND, if these factors have a constant, additive effect on student achievement. In other words, if the same amount of class size reduction has similar effects on those originally in very large classes and those originally in quite small classes, and if all other factors in the model work in the same constant, additive manner, then relying on state-level data can provide unbiased statistical estimates. But RAND itself argues that the impacts of resources on student performance are anything but constant and additive. Witness its conclusions on class size, where it finds that class-size reduction has its greatest effect in states with high shares of disadvantaged children. Witness also its finding that it is particularly important to reduce class sizes in states that begin with high average pupil-teacher ratios.

Finally, while the motivation of the entire study was to investigate the role and effect of different state policies, the only policies RAND’s researchers actually built into their main statistical models were differences in per-pupil spending, student-teacher ratios, and other resource variables. Except in an ad hoc fashion, RAND overlooked state efforts to establish accountability in the form of standards and testing and the wide variance in teacher certification requirements. The researchers themselves claim that these policies are important—in fact, they even suggest that such policies explain why Texas students perform better than California’s—yet they didn’t include variations in these policies in the models they constructed, except by creating “fixed effects” models that have so few independent observations that their results can’t survive rigorous statistical tests. In other words, RAND’s analysis failed to include the precise variables that the study itself claims are key.

Overturned?

The RAND study’s authors want to convince people that they have identified the most effective interventions and that outcomes are improving as a result of past reforms. If true, the authors argue, then there is no need to consider more fundamental changes in the education system’s structure or incentives. In order to make this case, the authors must prove that most earlier studies of the impact of school resources on student achievement should be disregarded.

Most scholars believe that studies that look at the impact of resources available to individual schools and specific school districts should be given the heaviest weight because they are the most precise. These studies are also the least likely to find that per-pupil expenditures, teacher pay, or class sizes make a difference.

The studies most likely to find that school resources have a positive effect rely on statewide data, like RAND’s. In this sense, RAND simply repeats an already well-known finding: that if you rely on imprecise statewide data and if you ignore all other aspects of state educational policy, you will often find that average statewide school spending and class size have at least a minor effect on student performance. But as mentioned previously, these studies have a serious methodological limitation: They rely on average results obtained from large, heterogeneous units that differ from one another in many ways other than the amounts they spend on schools.

RAND claims that only by looking at statewide data can you include the impact of statewide policies. Yet statewide studies have not yet found a way of including information about these policies in their statistical analyses. As a result, it is difficult to place more weight on these findings than on those that look at individual schools and school districts.

Again, this is not to say that schools don’t matter. On the contrary, value-added studies find that teacher quality has a major impact on student performance. If we could find ways of keeping good teachers in the classroom—perhaps by giving these successful teachers the additional compensation it would take to encourage them to make teaching a lifelong career—then we could probably boost student performance significantly.

But the authors of the RAND study take exception to value-added research. They claim that value-added studies that measure gains from one point in time to the next fail to account for the fact that “two students can have pretest scores and similar schooling conditions during a grade and still emerge with different posttest scores influenced by different earlier schooling conditions.” Put simply, Suzie may learn more than Johnny in 3rd grade not because Suzie had the better teacher that year but because she may have had a better education the previous year, even though this was not reflected in her 2nd grade test score. Since value-added studies usually don’t incorporate a student’s entire educational history, their results, according to the RAND study, may be biased in some unknown direction.

RAND, however, doesn’t provide any persuasive evidence that this is the case either in its own study or from other studies. Of course, one cannot rule out the possibility that gains in a particular year may somehow be influenced by events in the past. But RAND’s critique of value-added studies comes back to haunt its own research. If its critique is valid, then RAND’s own results are just as flawed as the results of the studies RAND criticizes. If earlier school conditions are important and affect the impact of current resources on student achievement, then one cannot assume constant, additive effects across all students in the state—the RAND researchers’ own methodology. Instead it is necessary to know the specific paths of resources to the individual students in the state and to incorporate that information into the statistical analysis. In other words, the very arguments the authors use to make the case for the superiority of their estimates over the hundreds of previous estimates again undermine their own analysis.

The RAND researchers also try to bolster their methodology by referring to the Project STAR (Student-Teacher Achievement Ratio) experiment, which involved a substantial reduction in class size (from an average of 24 students to an average of 16 students) in Tennessee. The study has received a great deal of attention, in part because it is one of the few evaluations of school resources based on random assignment of students to test policy effects while controlling for other conditions, a method that is generally thought to be a high-quality research design. However, the findings from the study are often misunderstood and misinterpreted, and RAND’s scholars have only added to the confusion.

In essence, the Tennessee study shows that students in substantially smaller classes in their first year of schooling (whether kindergarten or 1st grade) perform better than those remaining in classes of larger size. No similar benefits were observed for students in older grades, however. Those in the smaller kindergarten classes maintained the same higher achievement level that they had realized in kindergarten.

The STAR study, while methodologically superior to the RAND study, has its own limitations. The principle of random assignment was potentially compromised in several ways, and no student test information was obtained before assigning students to “control” and “experimental” groups. As a result, it is unclear how much the study, as implemented, deviated from a random-assignment design. Since almost all the gains from small-class assignment were registered in the initial year, it is possible that even these small “gains” were apparent rather than real.

But even if the STAR study doesn’t suffer from these implementation flaws—without baseline data we’ll never know one way or another—the study is not open to the inferences made by the RAND researchers. First, RAND assumed that the STAR study demonstrates that class-size reduction is effective in multiple grades when in fact it demonstrates, at most, that a very large reduction in class size has positive effects only in the first year of schooling. After that, the initial effects only manage to survive—they do not continue to increase even when the student remains in much smaller classes. Yet RAND uses these results to justify its policy recommendation to lower class size throughout the elementary school years.

Second, the RAND authors try to validate their own problematic methodology by claiming that their estimates of the effects of class size reduction are essentially the same as those obtained from the STAR study. But assessing the validity of studies by their answers violates all scientific principles. Generally speaking, a study’s validity depends on the scientific merits of its methodology, not the results it obtains. And even if one were to accept RAND’s claim to validity by virtue of its match with the results of another study, this claim applies only to the class-size findings.

Conclusions

RAND’s claims to have overturned conventional research wisdom are highly problematic. The report draws sweeping conclusions from average statewide data for just 44 states. The analysis of these data is subject to significant analytical error. The authors leave out of the statistical equations factors that they themselves insist are of critical importance. Claiming that only state-level analysis can take state policies into account, the researchers then leave key state policies out of their most crucial equations.

Worse, the interpretation of the results far exceeds the normal bounds of inference, thereby suggesting that the authors had a policy commitment that shaped their handling of the material.

But let’s take the RAND study at its word. If we do, we would conclude that, in general, education expenditures have little effect on student performance, that increasing teacher pay yields no effect, that the effects of class-size reduction depend very much on the state in which it is implemented, that monies should be set aside so that teachers who say they need them have more materials. The study also asserts that the strong accountability systems in Texas and North Carolina led to particularly spectacular student achievement gains in the early to mid-1990s. This is not necessarily a bad policy agenda. But one can hardly cite the RAND study as scientific evidence that it is the correct one. The conclusions reached by the RAND authors are based more on their personal sense of plausibility than on results from high-quality data subject to properly specified statistical equations.

–Eric A. Hanushek is a senior fellow at the Hoover Institution at Stanford University and a research associate of the National Bureau of Economic Research.

The post Deconstructing RAND appeared first on Education Next.

What Do Test Scores in Texas Tell Us?

by Stephen P. Klein, Laura S. Hamilton, Daniel F. McCaffrey, and Brian M. Stecher

RAND Corporation, 2000.

Just two weeks before the presidential election, yet another team of RAND researchers released a short paper that seemingly contradicted Grissmer et al.’s celebration of Texas’s achievement gains on the NAEP. RAND II found only small NAEP achievement gains in Texas, similar to those nationwide and contrasting sharply with “soaring” scores on the Texas Assessment of Academic Skills (TAAS). These disparities, the authors suggested, point to potentially serious flaws in Texas’s state-run testing program.

The direct conflicts of RAND I and RAND II underscore the fact that RAND is a collection of franchisees. The parent company attempts to maintain some degree of quality control but ultimately is not able fully to adjudicate quality—particularly, one suspects, when the answers are fuzzy and when the sponsor pressures are high.

RAND II presents two separate analyses that, taken together, seem to undermine Texas students’ spectacular gains on the TAAS. First, Texas students showed substantially more improvement on the TAAS than they did on the NAEP during the 1990s. Second, in a sample of 20 schools that the authors had collected for other purposes, the expected negative relationship between a student’s TAAS score and his eligibility for the federal school lunch program, a common measure of disadvantage, didn’t arise on the TAAS. This latter finding led RAND II to conclude not just that the TAAS is a poor instrument but also that high-stakes testing leads to the artificial inflating of scores through “teaching to the test,” especially for disadvantaged students.

It should not be particularly surprising that student performance improved more dramatically on a test that was aligned with a particular state’s curriculum (the TAAS) than on a more generic test of subject matter (the NAEP). Thus, while the question of the TAAS test’s validity is an important one, the simple evidence presented in RAND II falls very short of yielding any solid answers.

Likewise, the fact that data on 20 schools show a peculiar relationship with any variable is unremarkable. After all, even if the authors attempted to draw a representative sample—which they did not—the idiosyncrasies of such a small sample would preclude any ability to generalize. Indeed, a simple plot or a formal statistical analysis of TAAS scores across all Texas schools reveals a clear, and expected, strong negative relationship between students’ scores and their eligibility for subsidized school lunches.

The point of clearest conflict with RAND I is the consideration of NAEP performance. RAND I—not as focused on the relationship between its statistics and presidential campaigns—considered all seven NAEP tests given between 1990 and 1996 and attempted to adjust for differences in the students’ backgrounds. The result was high marks for Texas’s performance improvements on the NAEP. RAND II, by contrast, ignored student background, placed more weight on a different subset of test results (including the 1998 results, which were not included in RAND I), used somewhat different approaches, and concluded that there was nothing special about performance in Texas.

What lessons might we take away from the RAND I vs. RAND II debate?

• Analyses of small amounts of imperfect data can yield widely different conclusions. Such analyses should be heavily discounted.

• Consideration of a study’s quality tends to get lost in the ensuing policy discussion. Neither RAND study holds up to a modicum of scrutiny.

• The desire for publicity apparently pushes some researchers to prepackage their own sound bites. The PR blitzes that accompanied both RAND I and RAND II undermined any public discussion of what turns out to be relatively impotent research designs.

• Journalists tend to judge a study’s quality—particularly a complicated statistical study—by its conclusions and by an undue emphasis on the study’s source rather than the strength of its analysis. RAND’s undeniable history of producing solid research doesn’t mean that every study under the RAND imprimatur deserves unquestioned repeating.

The result is a distorted and unhealthy policy discussion.

The post RAND versus RAND appeared first on Education Next.

Even more remarkable, both candidates offered evidence for their claims. They repeatedly cited scholarly studies that lent support to their policy proposals. Is it possible that we’re on the cusp of a new era of evidence-based education reform?

So we hope. In this spirit, we welcome you to the first issue of Education Matters: A Journal of Opinion and Research, available both in hard copy and on our website, www.educationnext.org. In the stormy seas of school reform, Education Matters will steer a steady course, presenting the facts as best they can be determined, giving voice (without fear or favor) to worthy research, sound ideas, and responsible arguments. Bold change is needed in American K–12 education, but Education Matters partakes of no program, campaign, or ideology. It goes where the evidence points.

Too often, in recent years, evidence and reform have been divorced from one another. Some assign responsibility for this disconnect to zealous reformers who are more interested in peddling their nostrums than in evaluating them, or to vested interests that aggressively protect the status quo instead of welcoming evidence about alternatives. Yet the scholarly community is no less culpable, due to its habit of reporting important education research in dense prose salted with tables, charts, and equations accessible only to other social scientists. The general reader—and the policy maker—is left confused as to what the research actually shows and whether it should be trusted.

Education Matters is committed equally to readability and scholarly integrity. The authors are committed to presenting their findings—as well as their conclusions and opinions—in crisp, readable language. More comprehensive versions of our authors’ research and essays, complete with notes and data, are available on-line in Education Next Unabridged Articles.

This journal has sections with distinctive missions. The Forum enables scholars and commentators to express differing views on major education issues and reform proposals—beginning, in this issue, with the pros and cons of for-profit schooling and merit pay for teachers in “Defining Merit.” The Features section provides notable authors with a place to reflect on important concerns. In this issue, we are pleased to have Nancy and Ted Sizer detailing the challenges of starting a charter school in “A School Built for Horace.” Then E. D. Hirsch Jr. shows how romanticism shapes our educational thinking—not always for the better. Next, Greg Cizek addresses “cheating to the test” and what can be done about it.

In Research, the journal shifts from interpretations and commentary to the presentation of new (peer-reviewed) studies. In this issue, Terry Moe in “Hidden Demand” uses survey data to estimate who would opt for private schooling if choice were publicly financed, and Caroline Hoxby in “Changing the Profession” shows what school choice could augur for teachers. Education Matters presents their key results in lively, readable prose; Education Next Unabridged Articles offers supporting documentation.

Check the Facts asks whether research that is already influencing policy actually withstands close scrutiny. In this issue, Eric Hanushek in “RAND versus RAND” reviews two RAND reports that were widely quoted during the recent presidential campaign. While partisans hailed these studies as supporting the claims of one or another candidate, Hanushek finds serious flaws in both reports.

As in this issue, our Book Review section will often supply more than one commentary on the same book. In this particular issue, Rogers Smith and Stephen Gilles review Diversity and Distrust: Civic Education in a Multicultural Democracy by Stephen Macedo in “Civics Lesson.” And on the journal’s last page, various authors speak in personal terms about how Education Matters to Me. Lisa Graham Keegan, Arizona’s innovative superintendent of schools, inaugurates this feature in “Graduation Wish.”

In future issues—expect to see them four times a year—we will publish readers’ letters. We encourage you to send us your thoughts. The journal will benefit from your feedback. We also invite you to submit manuscripts for consideration and to contact us with your ideas.

-THE EDITORS

The post Evidence Matters appeared first on Education Next.

Simon and Schuster, 2000, $30; 534 pages (including notes and index).

As reviewed by Gerald Grant

Left Back will be misread by some as simply another bashing of progressive education when, in fact, Diane Ravitch laments the failure to realize the ideals of the early progressives. In her acknowledgements, she virtually paraphrases the most famous progressive intellectual, John Dewey, in extolling the private, progressive school her own children attended in New York City. While the choice is often cast as being between desiccated traditional disciplines and mindless child-centered progressive schools, the school her sons attended was both “academically vigorous and pedagogically venturesome.” Her children were inspired by a legendary teacher of Shakespeare, history teachers who “knew how to bring the past to life, and many others who dreamed up projects that fired their students’ minds and imaginations.” Ravitch points to Dewey’s school in Chicago (which closed in 1904, when Dewey left the University of Chicago for Columbia), the Lincoln School at Teachers College in New York, and the Winnetka, Illinois, public schools of the 1920s as admirable models of educational reform. They were joyful and engaging places for children that avoided the dull recitations of many traditional schools. Ravitch sees Winnetka as one of a few public school systems that made intelligent adaptations of progressive methods—individualizing instruction, motivating children by tapping into their interests, developing cooperative group projects—in order to achieve the traditional aims of producing knowledgeable and skilled students. Intelligent and able teachers held leadership roles in these schools. They tried to figure out the best ways to teach children, and they wrote new textbooks. They conducted research to see whether their methods worked. Their school day was structured to allow them time to meet and discuss data with their colleagues. In short, the best progressive schools treated teachers as professionals in a way that has rarely been equaled since.

Progressive reforms failed because they were hijacked by experts and top-down reformers who believed that mass education could be had on the cheap. They thought they could employ low-paid and poorly trained teachers who would follow their manuals or stand aside while children’s interests were supposedly being served. After 40 years of watching the distortion of his ideals, Dewey despaired over schools where children spent the day making nut bread but could not read. He scolded followers who let pupils respond to things “according to their own desires” without the intelligent guidance of teachers. “Now such a method is really stupid,” said Dewey, in perhaps the clearest sentence he ever wrote.

First came the zealots of social efficiency such as W.W. Charters and John Franklin Bobbitt, who saw the progressive movement as a vehicle for the wholesale ditching of the traditional subject-matter curriculum. They would substitute vocational training for the jobs then in existence. Since only 6 percent of the boys would ever become professionals, and only 7 percent of the girls would become teachers (“the rest would work as ‘servants, cooks, waitresses, laundresses, saleswomen’ ”), the other 90-odd percent needed vocational training, Bobbitt argued. The curriculum would be derived from hundreds of job analyses, such as what a department store credit clerk actually did: “meets people who desire to open accounts, asks them for information, writes form letters or telephones for references.” Their work was buttressed by the development of mass IQ testing, which gave anxious school administrators a “scientific” means to assign children to the vocational track. Lewis Terman of Stanford believed that the IQ test measured innate and unchanging intelligence. He wrote in his landmark study, “The dull remain dull, the average remain average, and the superior remain superior.” The journalist and public intellectual Walter Lippmann attacked Terman in an article in the New Republic in 1922, arguing that he could not imagine “a more contemptible proceeding than to confront a child with a set of puzzles, and after an hour’s monkeying with them, proclaim to the child and his parents, that here is a C-individual.” But IQ testing became the norm: the short history of American education in the 20th century is that Terman won and Lippmann and Dewey lost. Even as late as 1959, Harvard president James B. Conant argued that only 15 percent of high school students had the mental ability to take rigorous mathematics, science, and foreign language courses. Conant repeatedly warned school counselors that they must be prepared to persuade “overambitious parents” that their children were not “academically talented.” Ravitch is aware that the rhetoric of latter-day progressives changed more than did actual practice in the schools, where many teachers paid little attention to their theories. But there is strong evidence for her central argument that the progressive reforms championed by Dewey not only failed but also became the shell for an inversion of his ideals. Under the name of democratic education for all, only a minority received the grounding in the liberal arts that all children need in order to use their minds well and to enjoy a full life, whatever their eventual occupation. Perhaps half of the children who were included in the massive expansion of American education in the last century—and especially the poor and the black—were tragically “left back” in dumbed-down curriculum tracks. Now many of them face high-stakes tests of subjects they were never properly taught.

What would American education look like if we had shunned IQ tests as a means of sorting children, used higher salaries to attract more able recruits to teaching, adapted the kind of engaging cooperative inquiry among both teachers and pupils that Dewey favored, and expected all children to do rigorous mathematics and science beginning in elementary school? Probably more like the Japanese education system.

–Gerald Grant is a professor of education and sociology at Syracuse University.

As reviewed by Jeffrey Mirel

On a recent trip to Kalispell, Montana, a front-page story in the local newspaper caught my eye. Apparently the town’s high school faced serious problems, among the worst a dropout rate of about “10 percent a year.” When interviewed, the superintendent of schools focused on the “largely college-directed curriculum” as a main reason why young people were leaving school early. The solution, he declared, was to provide “equal opportunity” by making “an equal effort for the kids who are not going to college”—in other words, to create new, nonacademic programs for these students.

As Diane Ravitch reveals in Left Back, educational leaders have been making this claim for most of the 20th century, often using exactly the same words and phrases. Their reforms, in turn, have successfully steered most American students away from a liberal education, what Ravitch defines as “the systematic study of language and literature, science and mathematics, history, the arts and foreign languages.” Thus these students have been denied the knowledge and skills necessary for full participation in American life. That educational leaders have routinely justified these policies on the grounds that they were enhancing equal opportunity and democratizing our schools is one of the great ironies that Ravitch identifies and explores. At the heart of that irony, and the subject of Left Back, is progressive education, a philosophy of and approach to pedagogy that inspired a host of educational leaders throughout the 20th century.

Developed in reaction to the sterile formalism and rigid discipline that characterized American public education in the early 1900s, progressive education promised to transform American schools by making learning active and meaningful. Ravitch applauds many of the improvements that progressives introduced, such as attending “to the needs of individual children, emphasizing students’ motivation and understanding, and making the schools responsible for the health and general welfare of children.” But most of Left Back is a devastating and dead-on accurate description of how this promising philosophy degenerated into a series of interrelated propositions that, Ravitch argues, gradually undercut the intellectual and cultural foundations of American schooling. These propositions include the Rousseauian belief in the innately wise child, the conviction that children gain more from immediate experience than from learning about things that are distant from them in time and space, and that all topics and subjects in a curriculum must be justified by their immediate usefulness. These ideas combined to become what is certainly one of the strangest oxymorons in the history of American education, an educational philosophy that championed anti-intellectualism.

Ravitch finds that even the great philosopher John Dewey eventually succumbed to the lure of Rousseauian romanticism, in essence rejecting his own argument in The Child and the Curriculum that the best education occurs when the natural curiosity of the child interacts with the organized curriculum. Dewey occasionally chastised his colleagues for becoming too child-centered. But throughout his long career, Ravitch maintains, Dewey routinely disparaged traditional subject-centered education and often stoked the rhetorical pyres built for such defenders of liberal education as William Chandler Bagley and Robert Maynard Hutchins.

Ravitch tracks how the anti-intellectualism of progressive educational leaders translated into policies and programs that restricted students’ access to traditional academic subjects. After dispensing with Greek and Latin early last century, progressives campaigned relentlessly to discredit all the basic components of liberal education, condemning modern languages, history, geography, literature, higher mathematics, and laboratory sciences as elitist, inappropriate, and even damaging for all but a small number of college-bound students. They urged school districts to reform curricula by creating practical, relevant courses (such as shop, home economics, and driver’s education) based on students’ needs and interests. As Ravitch cogently argues, these progressive policies contributed more to educational inequality than the reverse. She demonstrates, for example, how progressive policies thwarted efforts by black parents to place their children in academically oriented programs and instead aided efforts (many of which were explicitly racist) to steer black students into vocational and general track courses that led directly to subservient social and economic roles.

Ravitch’s argument that progressive philosophy is still a powerful force in American schools—particularly in colleges of education—subtly but strongly pushes school reform in a new direction. It calls into question a central tenet of many school reform initiatives—that educational improvement will come through changes in school structure and governance. Reform efforts such as school choice, charter schools, reconstituting schools, and reducing class size all rest on the belief that changes in structure or governance will result in higher student achievement. But Ravitch’s book reveals that as necessary as these changes may be, revitalizing our schools ultimately depends more on restoring liberal education to its rightful place at the center of the American curriculum and breaking the grip of harmful progressive ideas (particularly the progressive antipathy to subject matter) on educational policy and practice.

–Jeffrey Mirel is a professor of education history at the University of Michigan, Ann Arbor.

The post Distorting Dewey appeared first on Education Next.