The post Just the Facts appeared first on Education Next.

The post Book Alert appeared first on Education Next.

The post A Kibbutz Education appeared first on Education Next.

The post Rod Paige on teachers who cheat; the benefits of inclusion appeared first on Education Next.

The post Teachers Unions appeared first on Education Next.

California Dreaming: Reforming Mathematics Education
by Suzanne M. Wilson
Yale University Press, 2003, $29.95; 320 pages.

Reviewed by Ralph A. Raimi

California’s “math wars,” the struggle over what is sometimes called the “new New Math,” illustrate all the ills and disagreements that have plagued American education for the past century. They have been but a chapter in the efforts by “progressive” educators to legislate equality of results in the schools via the dumbing down of the curriculum. In place of academic achievement, progressives offer self-esteem and racial harmony as the principal prizes, though there are others, especially for the bloated education establishment itself.

Unfortunately, Suzanne Wilson, a professor of teacher education at Michigan State University, addresses neither the politics nor the mathematics of the debate (if a “math war” can be called a debate) with enough insight to shed light on the broader trends in America’s schools. Nor will her book help to instruct the public or teachers of mathematics on the fundamental issues that were at stake in California. Her generous attempt at an even-handed, sympathetic account portrays the math wars as an unnecessary fight that can be ended by a return to civility, as if misunderstanding among decent and disinterested parties were the problem.

California has been unique among the states in having a strong legal structure allowing it to require essentially all its public schools to teach mathematics according to “Standards” periodically published by the State Board of Education. Similar constraints are gradually becoming law in other states, but during the period from 1980 to 2000 California was the only real example. This constraint is applied through state-supplied textbook money for grades K through 8. The details are intricate and the stakes are very high, both for the theorists, who want their view of education to prevail over the whole state, and for the publishers, who must satisfy the state’s requirements in order to win entrance to the enormous textbook market.

During the years following the ill-fated “New Math” initiatives of the 1960s, there was no visible national mathematics curriculum; but in 1980 the National Council of Teachers of Mathematics (NCTM) published the manifesto An Agenda for Action, followed by a 1989 document generally referred to as “The NCTM Standards.” Taken together, these two documents provided firm support for the progressive view of what to call mathematics and how to teach it.

The NCTM’s progressivism called for teachers not to teach explicitly, but to elicit students’ response to problems, generally “real life” problems. Teachers should be nurturing, not judgmental; no child should be elevated above the others; testing itself is suspect as punishing the losers. Even a too-detailed announcement of required academic content is seen by NCTM, as its own 1989 standards attested, as an implied attack on the deprived and downtrodden.

In 1985 and 1992, California adopted “Frameworks” that embodied the NCTM Standards, thereby establishing progressivism in the state’s textbooks and examinations. A few years’ experience with these programs generated a vigorous public reaction, mainly from middle-class parents anxious about the intellectual vacuity of the new programs and the implications for the future of their children. As a result, in 1997 a new State Board of Education called a halt to the movement and approved a set of more rigorous state standards, standards that could not be satisfied by the progressivist textbooks that had won the education establishment’s endorsement.

The education authorities were not unresponsive. The California superintendent of education and even the National Science Foundation official in charge of financing the new math initiatives of the 1990s launched a campaign mischaracterizing the new California standards as a return to rote memorization and other evils of “traditional” mathematics teaching, as if pedagogical style were the issue. In fact, pedagogy itself was not addressed at all in the new California standards. More to the point, though not mentioned explicitly, was that the new standards would make it impossible for those content-impaired, NCTM-approved math programs to be continued in California.

The State Board of Education’s most infuriating ploy in this game of hardball politics was surely the hurried and unexpected commissioning of a last-minute rewrite of an establishment-sponsored standards draft. Even though the revision was written by four distinguished Stanford mathematicians, the educators’ propaganda machine persuaded much of the mathematics education community that the new state standards were purveying mindless rituals as mathematics, to the destruction of students’ “higher order thinking skills.”

Those already familiar with the politics surrounding the controversial 1997 adoption of the new California mathematics standards can doubtless tease such details out of Wilson’s rather bland account, for she duly notes the various commissions, frameworks, surveys, and reports as they succeeded one other during the stormy period from 1980 to 2000. But it is all set in deliberately neutral terms, implying that this debate is always an honest one, which it is not.

Wilson’s account fails to describe the enormous budget of deception, charlatanry, careerism, ill will, and ignorance that underlies so much of the politics in question. Had she been able to find and deliver to her readers the full stories of Janet Nicholas, the key member of the 1997 Board of Education; Williamson Evers, an anti-progressivist political scientist at the Hoover Institution and one of the pioneering voices against the California dreams of the 1990s; and Luther Williams, the head of the National Science Foundation division that was responsible for the sillier math programs of the 1990s, she could have amplified her story immeasurably beyond her diary of their public pronouncements. The unpublicized infighting of such people has had as much practical consequence at the schoolroom level as all the theoretical apparatus of the schools of education.

All told, California Dreaming gave me the feeling of reading a history of the Protestant Reformation that did not mention the genuine death-dealing armies in the fields of central Europe, as if the battalions were engaged in a learned dispute concerning interpretation of the Gospels. For the reader who wishes to understand the fundamentals of today’s math wars and the baneful progressivist influence on American schools in general, a history such as Wilson’s, though an excellent straightforward chronology, is both too much and too little. As a healthful supplement I should like to recommend an earlier, polemical view of the major problem that confronts us in education: Albert Lynd’s Quackery in the Public Schools, a neglected 1953 book whose title is not yet out of date.

–Ralph A. Raimi is a professor emeritus of mathematics at the University of Rochester.

The post Uncivil War appeared first on Education Next.

With reports that some of the nation’s finest universities have been handing out A’s like lollipops at Halloween, the lowering of standards in higher education has become a hot topic. But grading standards in primary and secondary education have received remarkably less attention.

There are two major questions related to grading standards. First, to what degree do the grades distributed by schools and teachers correspond to their students’ performance on state and national exams? Second, and more important, how does “tough” or “easy” grading affect students’ learning?

The literature on these questions is extremely thin. In fact, to our knowledge, the analysis presented here represents the first study to examine the grading standards of individual teachers and how those standards affect students’ performance on independent exams. Our data set enabled us to examine the test-score gains of individual students from grade to grade across three school years. Thus we can see how individual students perform on nationally normed exams as they move from “tough” to “easy” grading teachers and vice versa. Our results suggest that elementary-school students learn more with “tough” teachers, with the effects varying depending on students’ initial performance levels and on the overall performance level of their classrooms.

Measuring Grading Standards

For this study, we analyzed confidential data provided by the school board of Alachua County, Florida, which includes the city of Gainesville. The data consist of observations on almost every 3rd, 4th, and 5th grader in the school system between the 1995-96 and 1998-99 school years, allowing us to follow two cohorts with three years of data each. Alachua County Public Schools is a relatively large district (by national standards), averaging about 1,800 test-taking students per grade, per year. The county is racially diverse, with a student population that is 60 percent white, 34 percent African-American, 3 percent Hispanic, and 2 percent Asian. Nearly half of all students are eligible for subsidized lunches, while 19 percent are identified as gifted, 8 percent as learning disabled, and less than 1 percent as English learners.

Alachua County provides a unique advantage for a study of this nature because it administers both the Iowa Test of Basic Skills (ITBS), a nationally normed exam, and the Florida Comprehensive Assessment Test (FCAT). The FCAT was designed to measure the degree to which students are meeting the Sunshine State Standards, which are also supposed to be the basis for students’ letter grades. Today, the state uses the FCAT both to rate public schools as part of the “A+” accountability plan and to determine whether students in certain grades are promoted to the next grade. During the period covered by our data, however, these tests were used for informative and diagnostic purposes only (though they were publicly reported each year at the school level).

Students receive scores on the FCAT ranging from 1 (lowest) to 5 (highest), with the thresholds for each performance level designed to correspond with the letter grades A through F. Thus results from the FCAT are ideal for developing a measure of how generous individual teachers’ grading policies are. We can then examine the relationship between teachers’ grading standards and their students’ performance gains on a different test, the ITBS.

Our primary measure of teachers’ grading standards is the average gap between the letter grades given by particular teachers and the FCAT scores attained by their students. Students take the FCAT math exam in 5th grade and the FCAT reading exam in 4th grade. Consequently, this measure of grading standards is calculated using the math grades and test scores of 5th-grade teachers, and the reading grades and test scores of 4th-grade teachers. Examining students’ performance on the ITBS in the 3rd, 4th, and 5th grades enables us to see how their gains in reading from 3rd to 4th grade, and in math from 4th to 5th grade, were affected by their teachers’ grading standards that academic year.

Grade Inflation

It turns out that the grades teachers assign are highly correlated with students’ ITBS and FCAT scores. But teachers also tend to grade far less stringently than the state standards indicate they should (see Figure 1). For instance, just 9 percent of students who were awarded A’s by their teachers attained a score of 5 on the FCAT. In fact, just 50 percent attained even a 4. Only 11 percent of students awarded B’s by their teachers attained level 4 or above, and a mere 39 percent attained level 3 or above. And of students awarded C’s, only 14 percent attained level 3 or above, and only 39 percent attained level 2 or above. Put differently, 86 percent of “C students” failed to achieve the minimum level of competency accepted (Level 3) on the Florida standards, along with 61 percent of “B students” and 17 percent of “A students.” Yet not all teachers are so lax: Among the top half of teachers as ranked by their grading standards, 65 percent of A students attained level 4 or above while just 5 percent attained level 2 or below.

In short, teachers vary considerably in their grading standards, even within a single school district. In fact, teachers’ grading standards often vary as much within a single school as within the school district as a whole. For instance, during the 1997-98 school year, the district-wide standard deviation in teacher-level grading standards was 0.68, while the mean within-school standard deviation in grading standards was 0.60. Tough-grading teachers, it appears, often teach alongside teachers with far lower standards. For research purposes, this is reassuring, since our empirical strategy relies mainly on within-school variation in teachers’ grading standards to isolate the effects of those standards.

A Stable Trait

Estimating the effect of individual teachers’ grading standards on their students’ achievement gains assumes that these standards remain relatively consistent over time, that they are not unduly influenced by the composition of their class, and that they are not a reflection of some other observable characteristic that might account for any effects we observe. Fortunately, our data provide evidence in support of each of these assumptions.

To see whether teachers’ grading standards remained stable over time, we divided the full sample of teachers into thirds according to their grading standards each year and examined how the position of individual teachers changed from year to year. For instance, we found that 75 percent of the teachers whose standards put them in the “easy” category (on a scale from “easy” to “moderate” to “tough”) in one year remained in that category the following year, while just 6 percent evolved from easy to tough graders in a single year. This trend was essentially the same across the three categories, with very little movement between groups.

Nor does it appear that teachers’ grading standards are influenced by the ability level of their students. To gauge this, we compared teachers who taught a higher ability class, as measured by their average 3rd-grade test scores, in 1998-99 than in the previous year, and vice versa. We found that even large changes in the ability level faced by teachers do not seem to affect their grading standards.

Turning finally to the relationship between other observable teacher characteristics and grading standards, we found that relatively tough graders are in fact slightly more experienced and slightly less likely to have attended a selective or highly selective undergraduate institution, though these differences are not statistically significant. However, tough graders are significantly more likely to hold Master’s degrees. In any case, our analysis below controls for each of these measures of teachers’ qualifications in order to rule out the possibility that teachers’ observed characteristics drive the estimated effects of grading standards on student outcomes.

Classroom Assignment

The method by which students are assigned to teachers could also cause problems for an analysis of the effect of grading standards on performance gains. The fact that students are not assigned to teachers randomly would be especially troublesome in a cross-sectional analysis, in which one compares one classroom with another in the same year. For instance, looking in cross-section across our own data set reveals that teachers with high standards also have students who are more likely to be white or gifted and less likely to be low-income or learning disabled. This is true even within a given school. Hence it is unclear whether the outcomes associated with high standards are actually due to the standards themselves.

But our analysis looks at year-to-year changes in the grading standards faced by a student, making this less of a concern. We found that students are nearly as likely to move to a teacher with different standards as to experience the same grading standards from year to year. For instance, just 57 percent of students with teachers whose grading standards are below the median within their own school continue to have below-median teachers the next year. Likewise, 54 percent of students with above-median teachers continue to have above-median teachers the next year. This indicates that year-to-year differences in grading standards within schools are close to random. Similar patterns are observed for most subgroups–black and white students are approximately equally likely to move between groups, as are students who are eligible and ineligible for a free lunch. It is the case that gifted students (around 12 percent of the sample), no matter where they start out, are considerably more likely to be placed with a high-standards teacher the next year than are nongifted students. Nevertheless, the vast majority of students are almost as likely to move between low-standards and high-standards teachers as to experience the same level of standards across years. And the results presented below do not change materially when gifted students are excluded from the analysis.

Empirical Results

For our primary analysis, we controlled for the average annual gain made by all students in the relevant school during the period of analysis; for such classroom characteristics as the share of white students, the share eligible for free lunches, and the students’ average math score in 3rd grade; and for the teacher’s years of experience, education level, and the selectivity of his or her undergraduate institution. In the end, we were interested in the effects on ITBS scores of changing a student from one level of grading standards to another.

Using this strategy, we found statistically significant improvements in test scores associated with higher standards. An increase in measured grading standards of 1 standard deviation is associated with about one-fifth of a year of schooling’s worth of gains in test scores–a large effect relative to many other interventions. (A year of test-score gain is measured as the average gain from one year to the next in Alachua County Public Schools. Because Alachua County’s gain scores tend to be larger than the national average, these are more conservative estimates of years of gain than are those based on national grade equivalents.)

While the average effects of grading standards are important, the theoretical literature on grading standards suggests that higher standards could produce both winners and losers. Students who achieve a given standard may be made better off because the standard becomes a more meaningful accomplishment. But those students who are not able to achieve the standard precisely because it is now more rigorous are made worse off.

To study this issue, we tested whether the effect of high grading standards varied for students with different initial test scores in 3rd grade. We found that the positive effects of higher grading standards were restricted to those students who were no more than 0.8 and 0.9 standard deviation below the average score in reading and math, respectively. For students performing below this threshold, grading standards had no detectable effect, either positive or negative, on performance. We also found that higher-achieving classes, as measured by their average 3rd-grade test score in the relevant subject, may fare somewhat better than lower-achieving classes under teachers with tough grading standards.

Equally interesting, however, are the distributional effects of tough grading standards on performance gains within a given classroom. Put differently, are the benefits of high standards uniform within a class, or do some children benefit more than others? We found that high-achieving students benefit most from tough grading standards when they are placed in classrooms where the overall level of achievement is relatively low (see Figure 3). The opposite is also true: tough grading standards elicit the most improvement from low-achieving students when they are in classrooms with relatively high overall achievement.

This result has intuitive appeal. Since the grades assigned vary much less across classrooms than does students’ performance on standardized tests, high-achieving students should be more likely to earn high grades in classrooms where the other students, on average, do not perform well on external assessments. Likewise, low-achieving students in classes with many strong students run a greater risk of receiving a low grade than they do when in low-achieving classes. So it seems only sensible that initially high-achieving students benefit most from tough teachers when they are among the strongest members of a class. Similarly, initially low-achieving students are challenged more to get a good grade with tough teachers, but particularly when they are among the weakest members of a class.

Parental Involvement

What might explain the positive effects of higher grading standards? One possibility, of course, is that high standards motivate students to work harder. A second possibility is that parents may devote more attention to their children’s schoolwork if their grades suggest that they are struggling, as they might with a tough-grading teacher.

To assess the latter possibility, in the spring of 2001 we conducted a survey of parents with students in both 4th and 5th grades in Alachua County. We asked the responsible parent to report on how much time he or she spends weekly helping each of the two children with their homework. This allowed us to control for factors, such as parental motivation, that might be common to both siblings in a household. We found that, holding constant the child’s grade level, 3rd-grade test scores, and the average 3rd-grade test score in the child’s class, parents spend more time helping the child with the tougher teacher with homework than they do helping the sibling with the easier teacher. These results do not appear to be due to tougher teachers’ assigning more homework; parental reports suggest that the typical tough teacher assigns just 10 percent more homework than the typical easy teacher.

Another intriguing finding from this survey is that parents do not perceive tougher teachers to be better teachers. We asked parents to grade their children’s teachers from A to F. While there is relatively little variation in these grades (in their own form of grade inflation, two-thirds of the parents gave their children’s teachers A’s), the results suggest that, if anything, parents view tough teachers less favorably than they view easier teachers. Parents were 50 percent more likely to assign a grade of B or below to a tough teacher than to a relatively easy teacher, after adjusting for the same controls as above. This result suggests that our measure of grading standards is not merely reflecting some other attribute of a teacher that is viewed as desirable to parents. It also bolsters our argument that it is high grading standards rather than some unobserved measure of teacher quality that is responsible for the positive effects on students’ performance gains.

Conclusion

Our results indicate that students benefit academically from higher grading standards. However, these results were not uniform: high-ability students appear to benefit more than low-ability students from high grading standards. Moreover, initially low-performing students appear to benefit more from high grading standards when they are placed in high-achieving classrooms. Likewise, high-performing students appear to react best to high grading standards when placed in low-achieving classrooms.

It is, however, premature to conclude that high grading standards are unambiguously desirable. We cannot yet speak to the consequences of teacher-level grading standards at the secondary level, where the same effects might not hold and where high grading standards might even lead more students to drop out of school altogether. In addition, our results do not tell us anything about how to raise the grading standards of teachers whose standards are currently low. Before we can recommend a general policy of higher standards, we must understand the distributional consequences at all levels and know how to implement a policy of high standards. As the yawning gaps between the grades teachers presently give out and their students’ scores on state tests suggest, it will be no easy task to change teachers’ grading habits.

David N. Figlio is a professor of economics at the University of Florida and a research associate of the National Bureau of Economic Research. Maurice E. Lucas is director of research and assessment for the school board of Alachua County, Florida.

The post The Gentleman’s A appeared first on Education Next.

Like automotive models, women’s hemlines, and children’s toys, pedagogical fads come and go, causing an immediate stir but rarely influencing teaching practice in any significant way. The notion that every innovation dreamed up by reformers inside and outside public schools makes its way into the nation’s classrooms is popular among those hunting for reasons to malign the schools. But it is crucial to distinguish between mere intellectual chatter and ideas that provoke substantive change.

Where on this spectrum does the idea of the “open classroom” lie? At first glance, it would seem to be just another fad. It burst onto the American education scene in the late 1960s, only to fade away by the late 1970s. Appearances, however, can be deceiving.

British Invasion

The open-classroom movement originated in British public elementary schools after World War II. The movement, known then as informal education, spread slowly to the United States. In 1967 a parliamentary commission headed by Lady Bridget Plowden published a report, Children and Their Primary Schools, that promoted open education in all British schools. American educators who visited British schools during the late 1960s had read the Plowden report and visited classrooms where informal education dominated teaching and learning. They viewed informal education–or, as they came to call it, open classrooms or open education–as an answer to both the American education system’s critics and the problems of U.S. society.

For more than a decade, U.S. schools had been subjected to withering attacks, blamed for everything from the launch of Sputnik to urban decay. They were faulted for not developing enough engineers and scientists; for being racially segregated and hostile to disadvantaged children; and for producing uncreative graduates who seldom questioned authority. Critics thought that the schools could be the vehicle for winning the Cold War, furthering the civil rights struggle, and roiling a 1950s culture of conformity that suffocated imagination.

Open classrooms’ focus on students’ “learning by doing” resonated with those who believed that America’s formal, teacher-led classrooms were crushing students’ creativity. In that sense the open-classroom movement mirrored the social, political, and cultural changes of the 1960s and early 1970s. The era saw the rise of a youth-oriented counterculture and various political and social movements–the civil-rights movement, antiwar protests, feminist and environmental activism–that questioned traditional seats of authority, including the way classrooms and schools were organized and students were taught.

In both Britain and the United States, open classrooms contained no whole-class lessons, no standardized tests, and no detailed curriculum. The best of the open classrooms had planned settings where children came in contact with things, books, and one another at “interest centers” and learned at their own pace with the help of the teacher. Teachers structured the classroom and activities for individual students and small work groups. They helped students negotiate each of the reading, math, science, art, and other interest centers on the principle that children learn best when they are interested and see the importance of what they are doing.

Consider the scene from a 3rd-grade open classroom in a New York City elementary school described by two proponents, Walter and Miriam Schneir, in a 1971 New York Times Magazine article:

What is most striking is that there are no desks for pupils or teachers. Instead, the room is arranged as a workshop.

Carelessly draped over the seat, arm, and back of a big old easy chair are three children, each reading to himself. Several other children nearby sprawl comfortably on a covered mattress on the floor, rehearsing a song they have written and copied into a song folio.

One grouping of tables is a science area with . . . magnets, mirrors, a prism, magnifying glasses, a microscope. . . . Several other tables placed together and surrounded by chairs hold a great variety of math materials such as “geo blocks,” combination locks, and Cuisenaire rods, rulers, and graph paper. . . . The teacher sits down at a small round table for a few minutes with two boys, and they work together on vocabulary with word cards. . . . Children move in and out of the classroom constantly.

Schools without Walls

As the idea of open education gained momentum, thousands of elementary-school classrooms became home-like settings where young children moved from one attractive learning center for math to another for art. Additional learning centers engaged them in science, reading, and writing lessons. Teams of teachers worked with multiage groups of students and created elementary schools where children were no longer assigned to grade levels. Some school districts started alternative open education programs at the high-school level and gave teachers discretion to create new academic courses where students directed their own learning, worked in the community, and pursued intellectual interests. At both the elementary and secondary levels, open education meant teachers were acting more as coaches in helping students than as bosses directing children in every activity.

Avid promoters of open education commissioned architects to build schools without walls. Teams of teachers worked collaboratively with one another, using movable dividers to reconfigure the open space for large- and small-group projects and individual study.

By the early 1970s, the phrase open classrooms dominated educators’ vocabularies. Even though parents and practitioners found it hard to pin down exactly what open education meant, many school boards adopted open-education programs, and open-space schools were built across the country. Few superintendents or principals could risk saying aloud that they had neither heard of the innovation nor found it desirable without risking sneers from peers or criticism from bosses.

So many schools were adopting the physical attributes of open classrooms that some advocates wondered whether the spirit of informal education was truly being followed. In his 1973 book The Open Classroom Reader, Charles Silberman warned enthusiastic teachers and parents:

By itself, dividing a classroom into interest areas does not constitute open education; creating large open spaces does not constitute open education; individualizing instruction does not constitute open education. . . . For the open classroom . . . is not a model or set of techniques, it is an approach to teaching and learning.

The artifacts of the open classroom–interest areas, concrete materials, wall displays–are not ends in themselves but rather means to other ends. . . . In addition, open classrooms are organized to encourage:

• Active learning rather than passive learning;

• Learning and expression in a variety of media, rather than just pencil and paper and the spoken word;
• Self-directed, student-initiated learning more than teacher-directed learning.

Backlash

Just a few years later, however, the ground shifted. In the mid-1970s, with the economy stagnating and the nation deeply divided over the Vietnam War, critics again trained their sights on the public schools. The national crisis gave rise to a perception, amplified by the media, that academic standards had slipped, that the desegregation movement had failed, and that urban schools were becoming violent places. This time the call was not for open education but for a return to the basics, again mirroring general social trends–namely, the conservative backlash against the cultural and political changes of the 1960s and early 1970s.

Traditional schools sprang up in suburbs and cities. Open-space schools rebuilt their walls. States tried to raise academic standards by developing minimum competency tests that high-school students had to pass in order to receive a diploma. Citations in the media and academic journals indicate that interest in open classrooms peaked somewhere around 1974. By the early 1980s, open classrooms had already become a footnote in doctoral dissertations.

But were open classrooms just another fad? Perhaps in the sense that, like hula hoops and pet rocks, they had soared onto the scene and then disappeared without a trace. Considering them merely a fad, however, would miss the deeper meaning of open classrooms as yet another skirmish in the ideological wars that have split educators and the public since the first tax-supported schools opened their doors in the early 1800s.

The School Wars

For at least two centuries, competing traditions of teaching reading, math, citizenship, and morality have fired policy debates and occasionally touched classroom practices. In teacher-centered instruction, knowledge is often (but not always) “presented” to a learner (via lectures, textbooks, and testing) who is–and the metaphors vary–a “blank slate” or a “vessel to fill.” In student-centered instruction, by contrast, knowledge is often (but not always) “discovered” by the learner (via individual and small-group work, projects blending different subjects and skills, and inquiry and questioning), who may be described as “rich clay in the hands of an artist” or “a flourishing garden in need of a masterful cultivator.” On the whole, different forms of teacher-centered instruction have dominated U.S. classrooms for the past century.

However, major challenges to teacher-centered instruction were mounted at the beginning of the 20th century by “pedagogical progressives,” to use Lawrence Cremin’s apt phrase; in the 1960s by enthusiasts for open education; and again in the late 1980s and early 1990s by neoprogressives committed to integrated curricula, performance-based assessments (rather than standardized tests), and smaller schools. Nevertheless, a wide gap remained between ideas and actual practice. Among educators, mainstream classroom practices remained teacher-centered, even if substantial numbers of teachers–trained by progressive faculty members–grasped pieces of the student-centered tradition and created hybrid practices.

The present moment in American education, with its emphasis on standards-based curricula and test-based accountability, surely favors the teacher-centered crowd. Nevertheless, many teachers, particularly in elementary schools, continue to prize active student involvement, cross-disciplinary projects completed by small groups, and similar activities. And even full-fledged open education is still thriving in schools across the country, from the Los Angeles Open Charter School to the Irwin Avenue Open Elementary in North Carolina’s Charlotte-Mecklenburg school district. Many teachers and principals still embrace the principles of open education, but keep their heads low to avoid incoming fire.

In high schools, most teachers continue to use teacher-centered practices, leavened slightly by informal practices that have crept into their repertoires. Yet activists in the small-schools movement carry the torch of open education and progressive practices from earlier generations.

Why this long-running ideological war over the best ways to teach reading, math, civic engagement, and character building? The short answer is that these enduring pedagogical quarrels are proxies for deeper political divisions between conservatives and liberals on issues ranging from environmental protection to foreign policy. There are, of course, liberals who believe in traditional education and conservatives who embrace progressive ideas, but the lines are fairly well drawn.

So while the open classroom has clearly disappeared from the vocabulary of educators, another variation of open education is likely to reappear in the years ahead. Deep-seated progressive and traditional beliefs about rearing children, classroom teaching and learning, and the values and knowledge that should be instilled in the next generation will continue to reappear because schools historically have been battlegrounds for solving national problems and working out differences in values.

Since children differ in their motivations, interests, and backgrounds, and learn at different speeds in different subjects, there will never be a victory for either traditional or progressive teaching and learning. The fact is that no single best way for teachers to teach and for children to learn can fit all situations. Both traditional and progressive ways of teaching and learning need to be part of a school’s approach to children. Smart teachers and principals have carefully constructed hybrid classrooms and schools that reflect the diversities of children. Alas, that lesson remains to be learned by the policymakers, educators, and parents of each generation.

-Larry Cuban is a professor emeritus of education at Stanford University.

The post The Open Classroom appeared first on Education Next.

“Principal Stratification Approach to Broken Randomized Experiments: A Case Study of School Choice Vouchers in New York City,” “Comment,” and “Rejoinder”
By John Barnard, Constantine E. Frangakis, Jennifer L. Hill, and Donald B. Rubin; “Comment”
by Alan Krueger and Pei Zhu

Journal of the American Statistical Association, June 2003.

 

Another Look at the New York City School Voucher Experiment

By Alan Krueger and Pei Zhu
Presented at the National Press Club, April 2003.

In The Education Gap: Vouchers and Urban Schools (Brookings, 2002), we and our colleagues reported that attending a private school had no discernible impact, positive or negative, on the test scores of non-African-American students participating in school voucher programs in Washington, D.C., New York City, and Dayton, Ohio. But after one, two, and three years in New York City, and after two years in Washington and Dayton, significantly positive impacts for African-Americans were observed.

Our results came from randomized field trials, which are generally thought to be the gold standard for research on human subjects. In such studies, subjects are randomly assigned to treatment and control groups by means of a lottery. In the best of worlds, researchers are able to collect information on the subjects’ characteristics before the lottery begins, enabling them to confirm that the lottery, in fact, worked as intended. If the treatment and control groups are similar at the beginning of the study, any differences between the two groups that emerge over time can be attributed to the programmatic intervention-in the case at hand, using a voucher to switch from a public to a private school. The results reported in this article are thus to be understood as the difference in test scores between those students who used vouchers to attend a private school and those of their public school peers who would have used a voucher had they been offered one.

Despite the strength of our evaluation’s design, the findings have not been without controversy. Specifically, two secondary analyses of the New York City data have recently been published, with widely diverging results. One study, conducted by a group of distinguished statisticians, John Barnard, Constantine Frangakis, Jennifer Hill, and Donald Rubin (hereinafter referred to as Barnard), has confirmed our first-year results but has been virtually ignored in the public media. The other, by Princeton economists Alan Krueger and Pei Zhu, has contradicted our results and twice received favorable coverage in the New York Times, where Krueger is an occasional columnist.

From the standpoint of pure innovation and analytical rigor, Barnard has produced the more impressive piece. As befitting an article published in the nation’s leading statistics journal, it introduces new statistical techniques to deal with problems that often emerge in randomized field trials: 1) missing data (for instance, not all students who initially joined the study participated in the follow-up testing sessions), and 2) noncompliance (some students, for example, refused the vouchers that were offered to them).

It remains to be seen whether the statisticians’ proposed innovation becomes more widely used. At its current stage of technical development, it permits the examination of effects only after one year. Also, in using the technique, Barnard opted to restrict their analysis to those families with only one child participating in the voucher program.

Despite differences in statistical approach and in the selection of students to be included in the analysis, Barnard’s findings are largely consistent with those we reported. While we estimated that, after one year, African-American students scored 7 percentile points higher on the math portion of the Iowa Test of Basic Skills than their peers in public schools, Barnard reports impacts of 6 percentile points for African-American students from low-performing public schools. (Almost all the African-American students came from schools with average test scores below the district mean; the few that did not had almost identical average impacts, but the number of available observations was too small to recover precise estimates.)

By contrast, Krueger and Zhu concluded, “The provision of vouchers in New York City probably had no more than a trivial effect on the average test performance of participating black students.” This conclusion rests primarily on three methodological decisions that distinguish their research from both our study and that of Barnard:

• We and Barnard let the mother’s ethnicity define the student’s ethnicity, while Krueger and Zhu defined a student as African-American if either parent was African-American.

• We and Barnard considered the results for only those students in grades 1-4, almost all of whom took achievement tests before the lottery. This provided us with what are known as “baseline test scores” that can be used to obtain more precise estimates of program effects. By contrast, Krueger and Zhu also included a large number of kindergartners for whom no baseline test scores were available.

• We and Barnard always adjusted the data to account for students’ baseline test scores in estimating our results. Krueger and Zhu, in their preferred results, as presented in their “Comment” on Barnard, exclude these baseline test scores.

All three of these alterations to the research strategy must be made in order to obtain results that differ substantially from those that we and Barnard obtained. Using any one or two of these different strategies does not generate appreciably different results.

How to Define African-American

Let’s consider Krueger and Zhu’s decision to classify students as African-American if either parent was African-American. Krueger and Zhu regard this decision as a key reason why they obtained results different from ours.

To understand the issue, bear in mind that because many of the students were very young, their ethnic backgrounds were ascertained from information provided in questionnaires filled out by the adults who accompanied them to the testing sessions. These adults were asked to report the ethnicity of the student’s mother and, separately, the student’s father. They could assign parents to one of nine categories, five of which are: Black/African-American (non-Hispanic); White (non-Hispanic); Puerto Rican; Dominican; and Other Hispanic. Classifying a child’s ethnicity is usually straightforward, because both parents are of the same background. In cases where parents were not of the same ethnicity, we classified the child by the mother’s ethnicity, simply because most children lived with their mothers, 74 percent of whom were single parents. Sixty-seven percent of the students lived with only their mother, compared with just 2 percent who lived with only their father. Mothers accompanied 84 percent of children to testing sessions; in 94 percent of the cases, the accompanying adult claimed to be a caretaker of the child.

Given the fact that these children tended to live with their mothers (and, often, not with their fathers), the decision to link the child’s ethnicity to the mother’s appears perfectly sensible. Alternatively, one might classify students as African-American only if both parents are African-American or if the child’s primary parental caretaker (usually the mother, but on a few occasions the father) is African-American.

Eschewing these alternatives, Krueger and Zhu used a unique classification scheme. They identify students of mixed heritage as African-American as long as either the mother or the father is African-American. If the mother was white but the father was African-American, the child was defined as “black, non-Hispanic.” Even if a child had a Hispanic mother and an African-American father, Krueger and Zhu still classified the child as “black, non-Hispanic.” Unless one departs from the standard practice of using mutually exclusive categories, students could not be classified as Hispanic or white if either parent was African-American. Krueger and Zhu defend this classification scheme on the grounds that it is “symmetrical.” But symmetry is hardly the word for a scheme that classifies Hispanics, whites, and African-Americans according to different principles.

Nevertheless, not much turns on how one defines a child’s ethnicity. Regardless of one’s definition, impacts after three years that range between 7 and 8 percentile points are observed for African-Americans in New York City (see Figure 1). If one classifies a student’s ethnicity by the mother’s (the approach we prefer), the effects are 8 percentile points; if one uses either the mother or the father (the approach favored by Krueger and Zhu) the effects are 7 percentile points, a result that is not significantly different from the one originally reported. By itself, altering the definition of a child’s ethnicity provides no basis whatsoever for concluding that effects disappear.

Students without Baseline Test Scores

Figure 1 presents results for students with baseline test-score information-the first bar reporting impacts for the definition of African-American originally used, the latter three bars for alternative definitions. The figure’s results are based on analyses that exclude from the study all kindergartners, none of whom were tested at baseline. Also excluded are the 10 percent of the students in grades 1-4 who were sick, who refused to take the test, or whose tests were lost in the administrative process.

Krueger and Zhu object to the exclusion of any students from the study, claiming that this constitutes the “most important” deficiency of our analysis, as well as that of Barnard. But even when all students are included in the analysis, African-American students who attended private schools scored significantly higher than their public school peers (see Figure 2).

Nonetheless, it is problematic to include students in a study if you don’t know what their achievement level was at the beginning. How well students perform on a test at, say, age seven, is tightly connected to how well they will do at age eight, nine, or ten. In fact, the correlations between baseline and follow-up test scores in New York consistently hover around 0.7. By comparison, the correlations between mother’s level of education and follow-up scores were only about 0.1.

Restricting the study to those students for whom baseline test scores are available affords a check on whether the lottery worked as intended and whether any problems arose downstream. For these students, all looks fine on both accounts.

When including all students, even those lacking baseline test scores, one can only hope that the two groups are similar with respect to this critical characteristic. Nonetheless, Krueger and Zhu defend their inclusion on the grounds that “because assignment to treatment status was random . . . a simple comparison of means between treatments and controls without conditioning on baseline scores provides an unbiased estimate of the average treatment effect.” This claim, says Barnard, “is simply false.”

If not quite false, the claim is at least dubious, because there were many ways for the treatment and control groups to become unbalanced. For example, about a third of the students did not remain in the study into the third year-a fairly standard rate of attrition from this kind of research protocol, but one that raises concerns that the treatment and control groups might have lost students with different baseline test scores. For this reason, we limited our analysis to those students for whom baseline scores were available, and hence for whom we were able to verify that the treatment and control groups did not become unbalanced.

But perhaps something else is to be gained from including all students, regardless of whether baseline information was available. Krueger and Zhu suggest that by adding these cases one can generalize findings to another grade level (kindergartners). Unfortunately, this is a hazardous generalization, given the fact that the results for kindergartners were significantly different from those for the older students. African-American students in grades 1-4 scored significantly higher if they attended private school, a result observed in all three years of the study. The results for kindergartners, meanwhile, were considerably more erratic; the effect of attending a private school for three years was a negative 13.9 percentile points. In the absence of baseline scores, we don’t know whether the findings for kindergartners are genuine or simply the result of errors in the administrative process.

Krueger and Zhu also note that their inclusion of all students in the sample generates more precise estimates. But gains in precision obtained by increasing the number of students observed will be offset by losses associated with failing to control for baseline test scores. One can assess the extent to which these competing forces balance each other by comparing the estimates’ standard errors: the smaller the errors, the more precise the estimate. As it turns out, the standard errors are larger, not smaller, when estimating statistical models that include all students but do not control for baseline test scores.

A compromise strategy, suggested by Krueger and Zhu, includes all students and adjusts for baseline test scores whenever possible. This analytic approach generates more precise estimates, the results from which are presented in Figure 2. But since these analyses also introduce risks of bias (principally by including the kindergartners for whom no baseline scores were available), the results in Figure 2 are inferior to the results provided in Figure 1. Nonetheless, they still reveal significantly positive effects of attending private schools on African-American test scores. In other words, even if one includes kindergartners in the study, as Krueger and Zhu recommend, the essentials of our original finding remain intact.

Krueger and Zhu have not accepted these findings, however. Instead, they have said that they cannot obtain equivalent results when they attempt to conduct an analysis identical to ours. But this claim is misleading. In fact, Krueger and Zhu’s results, available by correspondence, hardly differ from ours. As the first set of columns in Figure 3 shows, among students with baseline test scores, we both find that the estimated year three private school impact is 8.4 percentile points for all African-Americans (as defined by the mother’s ethnicity, our preferred definition). And, as shown in the second set of columns in Figure 3, we both find an impact of 7.6 percentile points for African-Americans when using Krueger and Zhu’s preferred definition of African-American (students whose mother or father is African-American). Moreover, when students without baseline scores are added to the analysis, they obtain results that are, once again, virtually indistinguishable from ours (see the last two sets of columns in Figure 3).

In other words, Krueger and Zhu also now report consistently positive results for African-Americans, regardless of how ethnicity is defined, even when kindergartners are included in the analysis-as long as baseline scores (and only baseline scores) are taken into account in the statistical estimation of programmatic effects.

To Ignore or Not to Ignore Baseline Test Scores

Neither changing the definition of African-American nor adding students for whom baseline test scores are missing appreciably changes the results we originally reported. To get different results, still a third methodological step is required. Krueger and Zhu argue that, to avoid a biased estimate, one must ignore baseline test scores, even for those students for whom these are available. But if including baseline scores introduced bias, the magnitude of the effect would change substantially. It does not. Adding baseline scores shifts estimated effects by less than half a percentile point.

Not only is no bias introduced, but including baseline test scores has the advantage of yielding more precise results, allowing researchers to reach firmer conclusions about the efficacy of a programmatic intervention. Estimated impacts from models that control for baseline scores are significant at the .05 level (using the two-tail test), while the less-precise results in models that do not control for baseline scores are significant at only the .10 level, using a one-tail test (a significance level below the threshold Krueger and Zhu find acceptable).

In sum, Krueger and Zhu take three methodological steps to generate results that are not statistically significant: 1) changing the definition of the group to be studied, 2) adding students without baseline test scores, and 3) ignoring the available information on baseline test scores, even though this yields less precise results.

By contrast, Barnard agreed with our decisions to: 1) use the mother’s ethnicity as the basis for defining the child’s, 2) focus on those students in the grades for which baseline scores were available for most students, and 3) control for baseline scores, whenever possible. Using pioneering statistical techniques, Barnard reports similar findings, while Krueger and Zhu venture far afield to uncover contrary ones.

The Value of Randomization

Given differences of opinion among researchers, it is easy to jump to the conclusion that randomized field trials are not the gold standard they are thought to be. If social scientists can reach opposite conclusions from the same data set, then research, even from randomized field trials, may do little to inform policy debates.

We take a different view. In New York, the results reported by all parties are consistently positive-only the magnitude of the effects and the level of statistical significance fluctuate. Furthermore, different statistical techniques generate roughly equivalent results. Moreover, the results that we report are consistent with past research on public and private schools. More than 25 years ago, James Coleman and his colleagues found that attending a private school was more beneficial for black students than for whites, as measured by test scores. More recently, Princeton economist Cecilia Rouse, after reviewing the research literature, concluded that “the overall impact of private schools is mixed, [but] it does appear that Catholic schools generate higher test scores for African-Americans.” Another literature review, conducted by economists Jeffrey Grogger and Derek Neal, found few clear-cut gains for white students, while “urban minorities in Catholic schools fare much better than similar students in public schools.”

Controversies surrounding randomized experiments can nonetheless be reduced by collecting baseline data on the outcome variable of greatest interest-in this case, students’ test-score performance. In the absence of this information, experiments devolve into endless arguments over whether random assignment actually occurred and whether the two groups being compared are genuinely equivalent. Consider, for example, the recent skepticism directed toward Tennessee’s Project STAR study, a randomized field trial on class size that failed to collect baseline test-score data.

Still, whether or not one restricts the analysis to those cases where baseline test scores were available, results are clear. In New York, private-school attendance positively affected the test scores of African-American students, but not those of any other ethnic group. For this reason, we think that the evidence from New York continues to support the conclusion-also reached in a wide variety of earlier studies-that disadvantaged African-American students living in urban environments benefit from private schooling.

Paul E. Peterson, the editor-in-chief of Education Next, and William G. Howell are professors at Harvard University. They are the principal authors of The Education Gap: Vouchers and Urban Schools (Brookings, 2002). To view the unabridged version of this article, log on to www.educationnext.org.

The post Voucher Research Controversy appeared first on Education Next.

The post Teaching Citizenship appeared first on Education Next.