Vol. 1, No. 2 - Education Next https://www.educationnext.org/journal/vol-01-no-02/ A Journal of Opinion and Research About Education Policy Wed, 24 Jan 2024 20:00:59 +0000 en-US hourly 1 https://wordpress.org/?v=6.5.5 https://i0.wp.com/www.educationnext.org/wp-content/uploads/2019/12/e-logo.png?fit=32%2C32&ssl=1 Vol. 1, No. 2 - Education Next https://www.educationnext.org/journal/vol-01-no-02/ 32 32 181792879 Young Einsteins https://www.educationnext.org/young-einsteins/ Thu, 07 Jun 2007 00:00:00 +0000 http://www.educationnext.org/young-einsteins/ Should Head Start emphasize academic skills?

The post Young Einsteins appeared first on Education Next.

]]>

The Bush Administration is moving to change the mission of Head Start, from one of providing social services and care to low-income preschoolers and their families to also emphasizing early literacy skills. Is preschool too early to learn academic skills? In the following essays, David Elkind and Grover Whitehurst weigh the evidence, the respond to one another.

Much Too Early by David Elkind.

Much Too Late by Grover J. Whitehurst.

The post Young Einsteins appeared first on Education Next.

]]>
49696281
Head Start https://www.educationnext.org/head-start/ Tue, 05 Sep 2006 00:00:00 +0000 http://www.educationnext.org/head-start/ The War on Poverty goes to school

The post Head Start appeared first on Education Next.

]]>

President Lyndon Johnson’s early career was spent working as a teacher in the hardscrabble of west Texas. That is where Johnson saw poverty up close and developed his faith in the power of education to eradicate it. As Johnson quipped to Yale University psychologist Edward Zigler (whose essay appears on page 12) in May 1965, “If it weren’t for education, I’d still be looking at the southern end of a northbound mule.” Johnson’s faith became policy with the creation of Head Start, birthed during the heady, idealistic days of the Great Society’s “War on Poverty.”

Head Start’s roots lay in the troubles of the Community Action Programs, or CAPs, an early War on Poverty venture whose motivating idea was to mobilize the poor on their own behalf. The frequent controversies surrounding the programs made local officials somewhat skittish about applying for CAP grants. Left with a budget surplus and the bureaucratic tradition of “use it or lose it,” Office of Economic Opportunity (OEO) director Sargent Shriver went fishing for a more politically salable anti-poverty investment. Hence Project Head Start.

Head Start reflected the belief that quality early-childhood education could inoculate disadvantaged children against the turbulence of their home and neighborhood life. It was to be a cost-effective endeavor; an early investment in nurturing at-risk children would avert later strains on social services and the justice system.

The same rationale, educational historian Maris Vinovskis has written, underlay the “infant schools” movement of the 1820s. The infant schools, a movement that quickly spread and then just as quickly disappeared before the Civil War, withered under the now-familiar criticism that academic training before the age of six or seven could inflict “serious and lasting injury” on “both the body and the mind,” as physician Amariah Brigham wrote in 1833.

Serious interest in early-childhood education wasn’t seen for another century. At Head Start’s inception, Zigler reports, only 32 states had kindergarten programs. Preschool programs for four-year-olds were almost “unheard of.” In the early 1960s, however, faith in the fortifying powers of preschool blossomed again on the strength of scholars Joseph McVicker Hunt and Benjamin Bloom’s finding that children’s IQs were not fixed at birth. Moreover, Bloom argued, the first five years of children’s lives were crucial to the development of their intellectual abilities. This was, Zigler writes, the “golden age” of cognitive psychology.

From the beginning, President Johnson and other advocates strongly promoted the IQ-raising potential of Head Start. Nonetheless, Head Start’s founders viewed it as much more than an academic intervention. Head Start was to provide a range of educational, medical, social, and psychological services to poor children and their families. Children can’t focus on learning, the thinking went, when they don’t have nutritious meals, healthy bodies, emotional stability, involved and knowledgeable parents, and social services designed to soften the impact of poverty.

Head Start’s links to the Community Action Programs made parental involvement a crucial aspect of the program. Not only would low-income parents learn child-rearing skills, such as how to prepare a nutritious meal. They would also serve as employees and volunteers in Head Start centers and have a strong voice in how the local programs were run. The War on Poverty’s links to the civil rights movement only enhanced their role. OEO staffers saw Head Start as a way of granting the power to minority parents that they lacked in segregated public schools.

This is crucial to understanding the resistance to the Bush Administration’s proposal to shift Head Start from the Department of Health and Human Services (DHHS) to the Department of Education. Similar resistance confronted President Jimmy Carter’s attempt to move Head Start to his proposed Department of Education. At the national level, Head Start parents, children’s advocates, and civil rights leaders feared that a move would undermine Head Start’s comprehensive approach that emphasized health care as much as education. At the local level, minority parents feared losing their voice in a white-dominated public school system. Congress ultimately nixed the idea of moving Head Start to the new Department of Education in 1978.

The overselling of Head Start’s ability to raise IQ, a highly stable measure of cognitive functioning, eventually caught up with the program. When early evaluations of the program found that children’s gains in IQ were small and faded out as they aged, the resulting uproar quelled President Richard Nixon’s attempt to expand the program. Funding for Head Start stalled throughout the 1970s and ’80s. The mixed results, however, should have come as no surprise. In the beginning, grants were handed out in a frenzy to just about anyone who set up shop in a church basement. Moreover, the intensely local nature of Head Start led to wide disparities in quality from program to program. In fact, it was hardly accurate even to call Head Start a program, or to say that Head Start had succeeded or failed. Some local grant recipients ran exceptional programs, others ran mediocre ones. Nearly all programs suffered from a shortage of trained early-childhood educators, and few had the funds to pay decent salaries anyway. Besides, the 14 members of the original planning committee hardly mentioned IQ. To them, whether children received their vaccinations, proper dental care, and a warm, encouraging oasis amid the chaos of urban life seemed just as important.

Still, Head Start survived the slash-and-burn Reagan years and became newly relevant with President George H. W. Bush’s national educational goal of having all children starting school “ready to learn” by the year 2000. In addition, the promising findings from model, much-more-expensive preschool programs such as the Perry Preschool Project in Ypsilanti, Michigan, have renewed hopes for what Head Start could be. Funding quickened in the 1990s, with federal spending rising from roughly $1.6 billion in 1990 to $5.3 billion last year. Like its young charges, Head Start has proved remarkably resilient.

Tyce Palmaffy is the articles editor of Education Matters.

This article was included in a forum on Head Start. For an alternate take, please see “More than the Three Rs: The Head Start approach to school readiness” by Edward Zigler and Sally J. Styfco .

The post Head Start appeared first on Education Next.

]]>
49696122
More than the Three Rs https://www.educationnext.org/more-than-the-three-rs/ Tue, 05 Sep 2006 00:00:00 +0000 http://www.educationnext.org/more-than-the-three-rs/ The Head Start approach to school readiness

The post More than the Three Rs appeared first on Education Next.

]]>
Head Start is, and has always been, a school-readiness program. In 1964, the project’s planning committee convened and was charged with designing an intervention to help young, low-income children begin school on an equal footing with their peers from wealthier families. There was little scientific evidence at the time to identify the needs of poor preschoolers or to suggest how to meet them. The planners therefore had to build a construct of school readiness relevant to the population Head Start would serve.

The members represented a variety of professional disciplines, and each contributed the latest knowledge in his or her field. Together they crafted the comprehensive services, whole-child approach that has come to define Head Start. Because children cannot devote their full energies to learning when they are not in good health, Head Start would ensure access to medical care. Hunger can also take a child’s attention away from schoolwork, so Head Start would provide nutritious meals and snacks and teach parents to do the same at home. Cognitive skills would be emphasized, of course, but children would also be taught social skills so they could learn to get along with others and follow social rules in the classroom. Special attention would be paid to their emotional health so they could gain the confidence and motivation to succeed in school. Because parents are the child’s first and most influential teachers, they would be invited to participate in all facets of the preschool and in adult education and training as well. Finally, because poverty carries many stresses that can interfere with healthy functioning, social-support services would be available to children and their families.

Nearly four decades later, these components of Head Start have come to define quality early care and education. The effectiveness of the model has been proved in a plethora of studies over the years showing that Head Start graduates are ready for school and in fact show good progress in literacy, math, and social skills in kindergarten. However, their academic gains during preschool are not as great as they should be, leading some experts and some policymakers to propose making Head Start more academic and less comprehensive. Admittedly, Head Start teachers are not all well qualified, due in part to low salaries and community staffing patterns. But recent revisions in the Program Performance Standards, which govern the quality of Head Start services, have begun to address weaknesses in teacher training as well as curricula.

Strengthening the preschool-education component in such ways is the appropriate response to calls to bolster the school readiness of children who attend Head Start. Focusing on this component to the exclusion of the others is not. Children who have uncorrected vision or hearing problems, who are ill or malnourished, who don’t sleep at night because of fear or hurt, or who have parents too preoccupied with their own problems to pay attention to them, will struggle with learning to read no matter how good the teacher.

Edward Zigler is a professor of psychology at Yale University and was one of Head Start’s founders. Sally J. Styfco is the associate director of the Head Start section at the Yale Center in Child Development and Social Policy.

This article was included in a forum on Head Start

The post More than the Three Rs appeared first on Education Next.

]]>
49696123
Raising Black Achievement https://www.educationnext.org/raising-black-achievement/ Tue, 05 Sep 2006 00:00:00 +0000 http://www.educationnext.org/raising-black-achievement/ Vouchers and the Test-Score Gap

The post Raising Black Achievement appeared first on Education Next.

]]>

Beginning with James Coleman’s research in the 1960s, comparisons of public and private schools have suffered under a powerful critique: that such comparisons can never fully account for differences in the types of students who attend public and private schools. For instance, are families that choose private schools more committed to education? The only way to neutralize these concerns is to randomly offer students a chance to go to private school and see what happens—a condition that the voucher programs of the 1990s have satisfied. Studies of these programs, however, have met with no less sound and fury. Herewith the findings from voucher programs in four cities, followed by economist Dan Goldhaber’s commentary.

Vouchers in New York, Dayton, and D.C.
by WILLIAM J. HOWELL, PATRICK J. WOLF, PAUL E. PETERSON, & DAVID E. CAMPBELL

Vouchers in Charlotte
by JAY P. GREENE

Significant, but Not Decisive
by DAN GOLDHABER

The post Raising Black Achievement appeared first on Education Next.

]]>
49696124
Early Warning System https://www.educationnext.org/early-warning-system/ Thu, 20 Jul 2006 00:00:00 +0000 http://www.educationnext.org/early-warning-system/ How to prevent reading disabilities

The post Early Warning System appeared first on Education Next.

]]>
Illustration by Melissa Szlakowsky

Since the 1976-77 school year, when Congress first required public schools to count the number of children with learning disabilities (LD), the share of school-age children labeled LD has risen from 1.8 percent to 5.2 percent. Learning disabilities now account for more than half of all students enrolled in special-education programs, up from 22 percent a quarter century ago. In the past decade alone, the number of students aged 6-21 identified as learning disabled under the Individuals with Disabilities Education Act (IDEA) has increased 38 percent. The largest jump, of 44 percent, has been seen among adolescents aged 12-17–a significant yet poorly understood increase. Private and postsecondary schools have experienced similar increases.

The steadily growing number of students identified as LD invites public skepticism. Most scientific experts, however, agree that 5 percent, and likely more, of our school population suffer severe difficulties with language and other skills. Even so, the disproportionate rise in the incidence of LD, especially among adolescents, does raise questions about the methods of identifying and treating learning disabilities.

What explains the rise of LD? Is it the result of positive developments, such as improved identification methods? Or is the definition of LD too general and ambiguous to catch younger students before they fail? Are some students identified as LD simply the victims of poor teaching? Put another way, does the education profession adequately prepare teachers to address differences among children?

We propose that the rise in the incidence of LD is largely the result of three factors. First, remediation is rarely effective after 2nd grade. Second, measurement practices today work against identifying LD children before 2nd grade. Third, federal policy and the sociology of public education itself allow ineffective policies to continue unchecked.

The best mainstream research–studies that reflect the consensus of experts in such fields as child development, education, and neuroscience–shows that most longstanding difficulties in defining and treating LD stem from inaccurate assumptions about their causes and characteristics. Moreover, the data now justify very early identification and prevention programs for children at risk for LD. This is nowhere more true than with reading disabilities, which are by far the most common and most troublesome of these disorders, constituting 80 percent of all students with LD. Fortunately, reading disability is also the best understood and most effectively corrected learning disability. If children receive effective instruction early and intensively, they can make large gains in general academic achievement.

Early intervention can greatly reduce the number of older children who are identified as LD. Without early identification, children typically require intensive, long-term special-education programs, which have meager results. Early intervention allows ineffective remedial programs to be replaced with effective prevention while providing older students who continue to need services with enhanced instruction so they can return to the educational mainstream.

The Rise of LD

The term learning disability traditionally refers to unexpected underachievement in adequate educational settings. Students with LD do not listen, speak, read, write, or compute as well as their “potential”–usually as measured by IQ—suggests they should. Historically, neurobiological factors–put crudely, glitches in the brain–were blamed.

Conditions resembling unexpected underachievement are found in medical and psychological literature beginning in the mid-19th century, where they are described variously as word blindness, dyslexia, dysgraphia, and dyscalculia. In 1962 Samuel Kirk, a psychologist at the University of Illinois, coined the term learning disabilities, which quickly entered professional and popular parlance. Like his more medically oriented predecessors, Kirk focused on unanticipated learning problems in a seemingly capable child. He defined LD as “a retardation, disorder, or delayed development in one or more of the processes of speech, language, reading, spelling, writing, or arithmetic resulting from a possible cerebral dysfunction and not from mental retardation, sensory deprivation, or cultural or instructional factors.” Kirk posited LD as an amalgam of disabilities, all grouped under a single label, just as the federal definition of LD has ever since. This was an intellectually bold move–but it was at best half right, and today we labor under its adverse consequences.

The term learning disability gained rapid acceptance among parents and professionals in the 1960s and ’70s. Rhetorically it exuded optimism. It did not stigmatize children; it imputed no shortage of intelligence, no emotional disturbance, no troubled home life, no bad schools. It almost presumed cultural and environmental advantage. Pragmatically it spoke to a real problem: before the 1960s, underachieving children couldn’t receive special education unless they were mentally deficient, emotionally disturbed, or physically handicapped. Professional and parental advocacy soon led to the 1969 Learning Disabilities Act, followed by the 1975 Education for All Handicapped Children Act (renamed the Individuals with Disabilities Education Act in 1990).

Most definitions of LD have at least four elements in common. The four elements posit LD as: 1) heterogeneous; 2) intrinsic or neurobiological; 3) marked by a significant discrepancy between learning potential (measured intelligence) and academic performance (measured skills in reading, writing, mathematics, and oral language); and 4) not caused by cultural, educational, environmental, or economic factors or by other disabilities (such as mental deficiency, visual or hearing impairments, or emotional disturbance).

The tenacity of these elements in definitions of LD has grounded our public policies and pedagogy in outdated science and flawed understandings of the nature of these disabilities. Their validity is rarely scrutinized, and the results of these occasional examinations are typically ignored. We suspect this impasse owes not to the evidence as much as to professional and political inertia.

New Definitions

In federal legislation, LD is not a single disability but a category of special education composed of disabilities in any one or more of seven skill domains: listening, speaking, basic reading (decoding and word recognition), reading comprehension, writing, arithmetic calculation, and mathematics reasoning. These disabilities can be accompanied by emotional, social, and behavioral disorders, including attention deficits, but they cannot, in the federal definition, be the primary cause of the learning disabilities.

If the current policy definition of LD creates a deliberately wide net, with seven possible disabilities, it also invites sloppy science. The definition tells us only what LD is not. This has provoked recent calls for changing the various domain-specific categories to reflect evidence-based research and to specify criteria for identifying the attributes of different learning disabilities. This recommendation is based on the observation that generic definitions, while useful for political, advocacy, and compliance purposes, get in the way of good practice in identifying and teaching students with LD and conducting research. To date this recommendation has made little headway in entrenched special-education communities; it failed to find its way into the 1997 reauthorization of IDEA. Until this bold but elementary initiative is undertaken, the assessment and instruction of children with LD will remain unreliable.

New evidence on neurobiological causes. The study of learning disabilities was founded on the assumption that neurobiological factors were the basis of the problems with learning. Neurobiological dysfunction was inferred from what was known about the linguistic, cognitive, academic, and behavioral characteristics of adults with brain injuries, as well as from evidence that reading problems run in families. Until recently, explanations of LD continued to favor neurobiological (or intrinsic) rather than environmental (or extrinsic) causes.

New, objective ways to assess putative brain dysfunction have led to extraordinary breakthroughs, especially in the area of reading. A sizable body of evidence indicates that poor readers exhibit disruption primarily, but not exclusively, in the neural circuitry of the brain’s left hemisphere, the part that serves language. Investigations using post-mortem brain specimens and magnetic resonance imaging (MRI) suggest subtle structural differences in several brain regions between reading-disabled (RD) and nonimpaired readers. More significant is the converging evidence from studies based on functional brain imaging that reveals the activity of the brain while someone reads. These studies indicate a pattern of brain organization
in poor readers that differs from that seen in nonimpaired readers, with less activity in regions of the left hemisphere that are associated with proficient reading.

Functional brain imaging, conducted while a child reads words, is also being used to show how intensive teaching can influence the brain’s reading circuitry (see Figure 1). Imaging occurs before and after intervention, during which children significantly improve in reading ability. It appears that after intervention, brain activation patterns shift to the normative profile seen in nonimpaired readers. This finding, if it can be replicated, appears to show that the neural systems supporting reading develop during interaction with the environment, including instruction.

Figure 1-Neural systems supporting reading may not be fixed, but develop if challenged. Shown is an image of a 10-year-old with severe reading disabilities before and after 60 hours of intensive instruction, during which the child rose into the average range in word-reading ability. The “before” image captures a brain exhibiting the standard activity pattern of children with reading disabilities. The “after” image shows increased activity in the left hemisphere, a pattern common to nonimpaired readers.


Genetic studies of the reading disabled reinforce this interaction perspective. Reading problems have long been known to recur across family generations; an offspring of a parent with RD is eight times more likely to have a reading disability. Studies of twins strongly support the genetic link to LD, pointing to regions on chromosomes 1, 2, 6, and 15. However, genetic factors explain only about half of the variation in reading development; environmental factors account for the other half.

This finding suggests that what is inherited is a susceptibility to RD that the environment can trigger. For example, parents who read poorly may be less likely to read to their children. In such cases, high-quality reading instruction in school is critical, as confirmed by recent studies of identical and fraternal twins. Scientists at the University of Colorado led by psychologist Richard K. Olson have explored the relationship between IQ levels and genetic and environmental influences on RD. They found that genetic influences tend to be more important for children with RD who exhibit relatively high IQ scores, while environmental influences tend to be more important for children with RD who possess relatively low IQ scores. The researchers argued strongly against excluding such relatively low IQ children from intervention simply because there wasn’t a great discrepancy between IQ and achievement. It is a grim fact that the traditional requirement of a discrepancy winds up excluding children who most need help.

Discrepancy between ability and achievement. Not surprisingly, the discrepancy criterion has proved contentious. This criterion was federally recommended in the wake of the 1975 Education for All Handicapped Children Act, which called for “objectively and accurately” distinguishing the child with learning disabilities from children with other academic deficiencies.

The use of a discrepancy between aptitude and achievement to “objectively determine” the presence of LD probably seemed reasonable at the time. Long before severe discrepancy became synonymous with LD, practitioners puzzled over the inability of some children of average and superior intelligence to master academic concepts. The discrepancy metric also reflected the common but incorrect view that IQ scores were robust predictors of an individual’s ability to learn. Despite the admonition of Edward L. Thorndike and others that IQ scores primarily estimate current cognitive functioning and not learning potential, in 1977 policymakers recommended using the scores inappropriately, as they have ever since.

These false assumptions about IQ and learning potential lead to unreliable formulas for discerning discrepancies themselves. As has been widely documented, the formulas are so fraught with psychometric, statistical, and conceptual problems that they are often useless. The discrepancy metric may even harm more children than it helps, as when the formulas used by different states deprive some students of special education following a family’s move from one state to another. Even worse, the entire measurement practice interferes with the early identification of LD. Children must fall below a predicted level of performance before becoming eligible for special education. Yet such underperformance cannot be measured reliably until a child reaches about nine years of age, or the 3rd grade. However, epidemiological data show that most children who read poorly at age nine or younger read poorly as adults. The IQ/achievement-discrepancy method creates a “wait-to-fail” model: we wait until they fail.

Another flaw in using the discrepancy concept is the inequitable education that results. No converging evidence shows an intrinsic “processing” difference among students who are weak in reading, whether they exhibit a discrepancy or not. Likewise, no evidence shows a difference in the effect of intensive instruction. The discrepancy requirement ultimately denies special services to garden-variety poor readers. This inequitable pattern recurs in reading disability:

Some children who achieve average scores on tests of word recognition read connected text with difficulty. Their slow reading impairs their comprehension. These readers clearly require special instruction, yet they do not receive it because reading fluency is rarely assessed. By contrast, if a slow-reading student has a high IQ, a weak word-reading score may secure him special-education services. Broadly speaking, the discrepancy formulation focuses attention on IQ and not on the reading process, irrationally favoring some individuals while allowing many deserving students to drift further and further from the education they need.

Excluding environment and other disabilities. As noted, LD by definition cannot stem from certain conditions that commonly impede learning: mental deficiency; emotional disturbance; visual or hearing impairments; inadequate teaching; and cultural, social, or economic disadvantages. This rubric makes for disastrous policy as well. First, it adds little conceptual clarity and actually constrains our understanding of learning disabilities. As the noted neuropsychiatrist Sir Michael Rutter of the University of London argues, this approach to definition suggests that if all the ordinary causes of the disorder can be excluded, the unknown–LD–can then be invoked. Second, some excluded conditions are known to impair cognitive and linguistic skills and actually lead to academic difficulties identified as LD.

An important example, too little scrutinized, is exclusion based on a student’s instructional history, no matter how incompetent. It was once thought that children who profit from instruction do not have a biologically based learning disability. However, as we have seen, brain-imaging studies of reading indicate otherwise: Instruction appears to establish the neural networks that support reading. No child is born a reader; all children in literate societies must be taught how to read.

No data exist to support the hypothesis that differences in the brain make some children respond less to intervention than other children do. The data, however, do indicate that the environment–including instruction, nutrition, prenatal and postnatal care, and parental drug abuse–influences neural development for better or worse. Children thus affected need the best instruction at the earliest possible time, but current federal definitions of LD preclude such a basic, sensible policy. Studies show that programs like Head Start, which are designed to prepare children for school, do a poor job of getting children ready to read. The largest federal study to date found that, on average, children left Head Start knowing only one letter of the alphabet, and that many teachers were discouraged from teaching the alphabet. Yet alphabetic knowledge in kindergarten is the single best predictor of reading success.

Preventive Education

Proficient young readers are phonemically aware, understand how print represents the sounds of speech, can apply phonemics and phonics rapidly and fluently, and connect their reading to their background knowledge. Poor readers enter kindergarten lacking phonological awareness and word-reading skills. Their text reading is laborious, impeding their comprehension and robbing them of the enjoyment of learning. Frustration and loss of interest in reading follow, as do mounting weakness in vocabulary and trouble in other academic subjects. And the cycle goes on.

Therefore, we must intervene as early as possible in a child’s school career to avoid the reading failure that will otherwise occur. Since the causes of most early reading difficulties are similar regardless of whether a poor reader meets the aptitude/achievement-discrepancy definition of LD, it makes no sense to wait for a discrepancy to reveal itself.

The tragic likelihood of failure, in the absence of very early intervention, is confirmed in numerous studies. As noted earlier, children identified as reading disabled after 2nd grade rarely catch up to their peers. Figure 2 shows the development of reading skills from the Connecticut Longitudinal Study directed by Yale University pediatrician Sally E. Shaywitz, which followed children from kindergarten through grade 12. Three groups were traced: 1) unimpaired children; 2) children defined in 3rd grade as RD due to an aptitude/achievement discrepancy; and 3) those so defined simply by low achievement. The pattern is growth from age 6 to age 12, then a near plateau from age 12 to 18. Children in the two RD groups show similar growth; neither group catches up to the skilled readers, even though half the children were pegged for special education. More than 70 percent of children identified as RD in 3rd grade remain RD in 12th grade. This longitudinal study shows that children fall behind in reading long before 3rd grade. Fortunately, other studies show that such children can be reliably identified well before 2nd grade. Researchers have found a strong relationship between phonological awareness and word recognition in kindergarten and 1st grade on the one hand and reading proficiency at the end of 4th and 5th grades on the other.

Early intervention is especially warranted given the documented failure of remediation through conventional special education. In a revealing analysis of a large data set, Hoover Institution economist Eric Hanushek and his colleagues found that placement in special education in grades 3-6 was associated with gains of 0.04 standard deviation in reading and 0.11 in math; such small gains indicate that children with LD clearly are not closing the gap. Remediation for older children fails for several reasons, but two stand out. First, quality: the instruction is frequently skimpy, highly general, and unsystematic. Second, timing: even excellent instruction may come too late, since many children lose their motivation to learn to read after a year or more of failure.

A group led by Florida State University professor of psychology Joseph K. Torgesen assigned severely disabled readers in grades 3-5 to one of two remediation programs, where they received 67.5 hours of individualized instruction over an eight-week period. Figure 3 describes the children’s growth in word reading and reading comprehension during the 16 months they were in special education before the research intervention (pretest to pretest). It also describes their growth during intervention (pretest to posttest) and in the two years following. The children showed little change in the 16 months of special education but made major improvements after a stronger intervention and maintained their gains for two years afterward.

Although these results show that older children can be substantially helped by intensive teaching, many of them remained very slow readers, with scores two years later on measures of reading fluency that were below the 5th percentile. In accounting for these mixed findings, Torgesen’s group pointed out that children develop fluency by encountering more and more words through frequent reading; children who avoid reading because it is difficult accumulate enormous “practice deficits” in elementary school. After several years, they’re unlikely to catch up in the amount they have read, which is critical for improving fluency. Still, 40 percent of these children returned to general education after the intensive sessions, well above the 5 percent of special-education students who typically return.

Early Intervention

The ineffectiveness of remediation has prompted studies of prevention and early intervention, which together might reduce the number of children who eventually qualify as reading disabled or who require literacy services through federally mandated Title I programs. Prevention in the area of reading shows special promise; it has been endorsed by both the National Research Council and the National Reading Panel. Kindergartners can be identified as “at risk” for reading problems by assessing their phonemic awareness. By 1st grade, the child’s ability to read words predicts later success in reading. The National Reading Panel has identified a large body of research recommending the explicit teaching of the alphabet and phonics as part of comprehensive reading instruction programs in kindergarten and 1st grade. Comprehension and writing instruction are also important in the early grades, and they become more important as children develop accurate and fluent word recognition skills. More broadly, studies of early intervention sponsored by four federal agencies have pointed unambiguously to the effectiveness of prevention, with tantalizing evidence that it may reduce the incidence of reading disability at later ages.

The research on prevention and early intervention has its critics. Some argue that early identification is fraught with errors; they fear incorrectly labeling a student as at risk for academic failure. However, the costs of delay are too great and, besides, a label itself isn’t necessary. Ultimately all kindergarten and elementary school teachers should be trained to teach well enough so that prevention effectively occurs in the course of normal schooling. (Of course, some children will always need more help than they can get in a typical classroom.)

Other critics point to greater gains in word recognition than in reading comprehension and to diminished gains in the later grades, as in the so-called 4th-grade slump. It is important to note, however, that this slump marks a reduction in the rate of growth, not a loss of skills. The key is to teach reading throughout elementary school, increasingly integrating word reading, fluency, and comprehension.

What, finally, of the weakest learners? Such children probably require different, highly intensive, and systematic interventions over a prolonged period, with the aim of speedy improvement so they can “read to learn” in content areas. Such ideals of instruction can be realized only when, thanks to early intervention, the numbers of children with reading difficulties decline to manageable levels.

Rethinking LD

Our review of the sad state of policy and practice concerning LD leads us to recommend radical changes in three areas:

1) the definition and identification of LD; 2) teacher preparation; and 3) prevention, early intervention, and remediation.

Improving the definition of LD. First, replace the muddled, exclusionary definition of LD with evidence-based inclusionary definitions. These must specify and distinguish disabilities in reading, mathematics, written expression, and oral language. Children who need help cannot be identified reliably without domain-specific definitions.

Second, stop using IQ/achievement discrepancy as a primary marker for LD. This may require devising alternatives if they can be found to be valid. In most cases, however, student underachievement, particularly in reading, can be identified by testing skills in the academic domain in question; by direct comparisons of the student’s age and grade with speaking, reading, writing, and math skills; or simply by uneven overall performance regardless of IQ level. IQ is an unnecessary and distracting component of the ways in which we think about learning disability.

Third, admit environmental influences into consideration. Government policies exclude inadequate instruction, cultural and social factors, and emotional disturbance, because there are other categories in special education or other services for these children, not because their academic difficulties are different or because these excluded children need different types of interventions. We know that precisely these factors can hamper neural and cognitive development and place children at risk for LD.

Fourth, single out students who respond poorly even to early intervention. The complex identification criteria and expensive due process required in special education should be reserved (albeit simplified) for such children; for most others, early interventions should be made available.

One caveat: No child should be identified as learning disabled simply because of a score, without input from teachers, parents, and others responsible for the child’s education.

Improve teacher education. First, acknowledge the limitations of current teacher preparation programs. The statement that many children identified as LD are actually “teaching disabled” is unfortunately all too often accurate. Almost all children can learn to read if taught appropriately, but many miss out on the help they need because teachers are not adequately prepared. Teaching children to read is central to a teacher’s calling, but a large body of evidence shows that teachers get weak training in individual learning differences and in teaching reading to students with diverse backgrounds and abilities. Nor are teachers trained–or later encouraged–to read and apply research.

Second, do not rest hope on reforming most schools of education. There are too many reasons to doubt that change will occur any time soon, and our academically weak children are too important to wait for reform. Instead, teachers must be provided, through any means possible, with the necessary academic content, pedagogical principles, and knowledge of learner characteristics.

Third, evaluate and, if they are found effective, encourage alternative teacher preparation programs that have emerged across the country. In Texas, for example, all teachers in kindergarten and 1st and 2nd grades receive additional training in reading instruction.

Expand prevention and early-intervention programs. First, take advantage of new technology. In Texas and Virginia, teachers administer tests that help identify the instructional needs of children at risk for reading disability in kindergarten and 1st and 2nd grades. Thanks to the federal Reading Excellence Act, some states are aiming new, scientifically based reading programs at disadvantaged at-risk children. This targeting may reduce failure substantially.

Second, make prevention the work of both special and regular education, and begin early. This will mean, among other things, revising current language in the IDEA that allows states to tap children for special education based on a “developmental delay” only between the ages of 6 and 9. In this and many other instances, complicated federal mandates for determining eligibility tend to suck up precious funds, where a few simple criteria would reap greater benefits for children.

Third, make special education accountable for the results of these interventions. Current federal policy produces a focus on eligibility and compliance that creates a labyrinth of paperwork for schools, teachers, and parents. Let’s focus instead on results, on whether the child learns to read or do math!

If these recommendations succeed, the meaning of LD could change. The label “LD” would be reserved for children whose reading or other academic problems proved severe and intractable. They would receive more comprehensive and intensive help faster. In turn, they would prompt researchers to more focused study, accelerating understanding of how the environment, the brain, and heredity interact to mold cognitive competencies. This is by no means an attempt to write off children who do not respond to aggressive instruction; it is an attempt to maximize their learning potential through scientifically sound practice. The linkage between science and policy on learning disabilities has been fragile for more than 30 years. Policy has driven the scientific agenda. This should be reversed. Learning disabilities are not that mysterious. The best consensus research, especially in the past 15 years, has profound implications for understanding learning disabilities and contradicts current policy. Clear, convergent findings are only the first step, however. A radical restructuring will succeed only if educators acknowledge the complexity of the task.

G. Reid Lyon is a research psychologist and chief of the Child Development and Behavior Branch of the National Institute of Child Health and Human Development at the National Institutes of Health. Jack M. Fletcher is a neuropsychologist and professor of pediatrics and the associate director of the Center for Academic and Reading Skills at the University of Texas-Houston Health Science Center. To view their essay in its entirety, log on to www.edmattersmore.org.

The post Early Warning System appeared first on Education Next.

]]>
49696032
In Praise of Mediocrity https://www.educationnext.org/in-praise-of-mediocrity/ Thu, 20 Jul 2006 00:00:00 +0000 http://www.educationnext.org/in-praise-of-mediocrity/ Tattered Blue Ribbons at the Department of Education

The post In Praise of Mediocrity appeared first on Education Next.

]]>
Illustration by Greg Clarke

Since 1982 the federal government has recognized more than 4,000 exemplary schools through its Blue Ribbon Schools Program. From the program’s inception, Department of Education officials declared that it would not merely single out the highest-achieving schools, but would also recognize other forms of excellence. Putting aside this expansive view of excellence, how do Blue-Ribbon winners perform on tests of reading and math achievement? Perhaps they are not the highest-scoring schools in each state, but surely they must be near the top or at least score above average.

In a study first reported in the Brookings Institution’s Brown Center Report on American Education in September 2000, we compared the test scores of Blue Ribbon schools with those of an average school in several states. To simplify matters, we confined our analysis to achievement in reading and mathematics. The results were eye opening. We found that approximately a quarter of Blue Ribbon winners are indeed high achieving, scoring among the top 10 percent of schools after adjusting the scores for students’ socioeconomic status. However, an equal percentage of Blue Ribbon schools scored below average after making the adjustment for students’ socioeconomic status. The remaining schools, about 50 percent of Blue Ribbon winners, scored above average, but not extraordinarily so-above their state’s mean, but well short of the 90th percentile.

Here we update our findings by including the year 2000 Blue Ribbon award winners and examining the academic growth of Blue Ribbon schools in California from 1998 to 2000. This analysis reaffirms the previous study’s key finding-that blue ribbons are awarded to many mediocre schools. Which begs a question: Can this be squared with the federal government’s acknowledged goal of holding schools accountable for student learning? Even if one accepts the notion that excellence comes in many forms, imagine two schools in the same neighborhood, one that raises its students to extraordinary heights in reading and mathematics, the other with a lackluster record of academic achievement. What kind of excellence is so important that it compels the federal government to give a Blue Ribbon award-the highest honor that the nation bestows on a school-to the mediocre school while overlooking the exemplary one?

Choosing Blue Ribbon Schools

The Blue Ribbon awards alternate from year to year, going to elementary schools one year, middle and high schools the next. Schools nominate themselves. They first submit an application packet to the education department of their respective states. States screen applicants and forward promising nominees to the U.S. Department of Education. Some states nominate candidates for the national award in conjunction with their own school-recognition program. The Department of Education then convenes a panel of experts, the National Review Panel, to evaluate the quality of the applications. About half of the applicants advance to the site-visit stage. For all but a few schools, receiving a site visit is tantamount to receiving an award. In 1999, 377 schools were nominated by states, 202 schools received site visits, and 198 won awards.

The application packet includes dozens of questions developed by approximately 200 researchers, practitioners, and staff members from state and federal educational agencies, professional associations, and large foundations. Schools are asked to report how often they engage in what the experts on this panel regard as “best practices.” School practices are organized into eight categories: student focus and support; school organ-ization and culture; challenging standards and curriculum; active teaching and learning; professional community; leadership and educational vitality; school, family, and community partnerships; and indicators of success. The last category is the only one that requests data on student achievement.

A Blue Ribbon award is a source of pride for both schools and communities. Educators put loads of effort into preparing the application. A recent Washington Post article described how one school shut down its library for two weeks so that two employees could work 12 hours a day finishing a 71-page application, full of attractive bells and whistles such as charts and graphs. The school principal and a committee of 41 parents and teachers were involved in putting together the submission.

Academic Achievement


We analyzed test data from several states, adjusting for the percentage of students in a school eligible for the federal free and reduced-price lunch program. This removes, albeit imperfectly, the effects of family background on student achievement, so that we can focus solely on what the schools add to student perform-ance. We then examined their income-adjusted scores to discover how many Blue Ribbon schools scored in the top 10 percent of each state’s schools, indicating extraordinarily high achievement, and in the bottom 50 percent of each state’s schools-in other words, below average for similar schools in the state.

In 1999, the most recent year in which elementary schools were selected, the achievement of Blue Ribbon schools was above average, but hardly outstanding. Nineteen of the 70 elementary-school winners scored in the top 10 percent of all schools in their states, but almost as many, 17, scored below average. In 2000, the last year in which middle and high schools were selected, the achievement of Blue Ribbon schools was worse. Only 14 of 69 award winners scored in the top 10 percent; 18 scored in the bottom 50 percent. As a group, the Blue Ribbon schools score above average in reading and math, but not extraordinarily so. About a quarter of the winners actually exhibit lower academic achievement than the average school with a similar demographic profile (see Figure 1).

This pattern directly reflects the priorities of the Blue Ribbon program’s application packet. Academic achievement is one factor in being selected, but not the only factor and certainly not a make-or-break issue in winning the award. Schools that have experience filling out grant applications or other applications similar to those required for the Blue Ribbon award apparently know how to spin mediocre test scores so that they don’t diminish a school’s chances for an award.

Supporting this hypothesis, the low-achieving winners, those that scored in the bottom 50 percent, tend to be located in wealthy neighborhoods. They have about half as many students on free and reduced-price lunches as the average school in their respective states. Take Michigan, for instance. The three elementary schools that fell in the bottom 50 percent are located in the same leafy suburb north of Detroit. Less than 10 percent of the students in these schools receive free or reduced-price lunches. About 75 percent of their students score at the proficient level on Michigan’s test, the MEAP—a respectable number, but hardly impressive given that dozens of Michigan schools with similar demographic characteristics routinely report more than 90 percent of their students achieving at the same level of proficiency. Moreover, there are schools in downtown Detroit with higher test scores than the three Blue Ribbon schools. The share of disadvantaged children at these inner-city schools is several times that of the Blue Ribbon award winners.

After the release of the Brown Center’s report in September 2000, Stephen O’Brien, the Blue Ribbon program’s director, criticized both our reliance on academic achievement as the sole barometer of a good school and our reliance on absolute scores rather than gains in achievement from year to year. He stressed that honored schools may score below average, but they may also have significantly improved their scores over previous years. It’s true that demonstrating academic growth is certainly worthy of national honor. Indeed, if the assumptions of the Blue Ribbon program are correct, using the “best practices” endorsed in the application packet should yield above-average gains. If the winning schools were attaining above-average gains, no one would contest their receiving awards.

To investigate this question, we had hoped to use the same seven states as the original analysis, but most of them had not released year 2000 test data in time for this article. Fortunately, California had released 2000 achievement data on its website. Moreover, the state has a large pool of Blue Ribbon winners and has administered the same standardized test (the SAT 9) over the past three years, allowing for a good estimate of academic growth. We used these data to examine the aca-demic growth of Blue Ribbon schools from 1998 to 2000.

Blue Ribbon schools did make gains during this period, but they barely kept pace with California’s gains overall. In fact, they made less progress than the state’s average school at the elementary- and middle-school levels. From 1998 to 2000, while elementary schools statewide gained 7.7 percentile points, the Blue Ribbons gained only 5.7 points. Middle schools in the state registered a gain of 4.6 points; Blue Ribbons, only 3.9 points. High-school Blue Ribbon winners matched the state’s average 2.5-point gain (see Figure 2).


The mediocre achievement of the Blue Ribbons is apparent in California’s monetary awards program. The state set goals, based on student achievement in 1999, for each school to attain in 2000. Schools meeting these targets received cash bonuses, some as large as $25,000 per teacher. Schools falling short received nothing, but they have another shot at hitting their targets in 2001. The year 2000 achievement data determined the first award winners.

The state’s Blue Ribbon schools failed to earn significantly more cash awards than other schools in the state. Fully 75 percent of the state’s elementary schools received cash awards; 76 percent of the 1999 Blue Ribbon schools did. Among the state’s middle schools, 56 percent reaped awards for meeting their achievement goals, compared with 62 percent of the last two cohorts of Blue Ribbon middle schools (1998 and 2000). At the high-school level, 38 percent of the state’s schools won awards, but only 36 percent of Blue Ribbon high schools did.

This analysis has a serious limitation: California is only one state. Nevertheless, the results are stunning. Schools can win a Blue Ribbon award without demonstrating high levels of reading and math achievement or attaining large gains in these subjects. Which leads to a startling fact: In 2000, nearly two-thirds of Blue Ribbon high schools in California did not receive their own state’s monetary award for high achievement.

Setting the Right Goals


Poor implementation is not the reason why the federal government’s premier school-recognition program fails consistently to select schools with extraordinary achievement; there is no evidence of administrative malfeasance or cronyism. The flaws are structural, built right into the Blue Ribbon program. And they are nonpartisan, existing through Democratic and Republican administrations.

The problems start with the self-selection of nominees. This places low-income schools at a disadvantage. They are less likely than schools in wealthy neighborhoods to have the time and expertise, or to receive the necessary help from volunteers, to complete the application process or to campaign for the award. Indeed, schools with large numbers of disadvantaged students may simply have better things to do than to apply for awards. The Blue Ribbon schools that we identified as low achieving (in the bottom 50 percent) are often quite privileged. Eliminating self-nomination would raise the program’s costs, requiring program officials to find deserving schools on their own, but it would make the program more equitable.

The problems continue with the program’s support for “best practices” that have no basis in research. In the field of education, expert panels have a bad habit of embracing fads. The application for Blue Ribbon schools recommends student-initiated learning, curriculum integration, extracurricular opportunities, elimination of tracking and ability grouping, teaching to different learning styles, and technology integration. There is no convincing body of research showing that these practices are valuable. Extracurricular activities are helpful to a point, but they can undermine achievement if they absorb too much of a student’s time. Studies are inconclusive as to the effect of tracking and ability grouping on achievement. Instruction emphasizing student-initiated learning has a long history of failure, appearing as far back as the 1920s and reemerging in the 1960s in discovery learning and open schools.

Buzzwords are ubiquitous in the Blue Ribbon schools’ mission statements. Schools say they empower parents and teachers or act as catalysts in getting students to aim for a vision. They also offer catchy slogans, proclaiming a “move into the future” and their desire to forge close bonds with the “community.” Schools proudly describe their “site-based management model” or commitment to “student-centered education” and “open-spaced instruction.” These words echo popular platitudes but have little or nothing to do with academic achievement.

Finally, the program’s fundamental flaw is its failure to place a premium on learning. Instead of making high academic achievement the centerpiece of the selection process, the application asks last for evidence of student learning. High achievement in reading and mathematics is surely the defining characteristic of an exemplary school. The Blue Ribbon Schools Program should put it first and make it the most important qualification. The application packet is not shy in imposing other nonnegotiable requirements. A school is ineligible to apply if it is under investigation for violating special-education statutes or has failed to resolve a civil rights complaint. The same exalted status should be accorded academic achievement.

American education has entered a new era of accountability. Increasingly, schools are judged by results. Many analysts now argue that state and federal regulators should leave decisions regarding methodology to the educators within the school walls. Within this reform environment, the Blue Ribbon Schools Program is a dinosaur. It judges schools by what they do instead of what they accomplish. It brings experts together to create a checklist of best practices, asks schools to report which of the practices they use, convenes another panel of experts to judge those responses, and then sends evaluators out to verify the schools’ claims.

The result is a lost opportunity to highlight the best schools in the nation and to learn from their examples. In most walks of life, blue ribbons are given to those who compete and perform at a high level, finishing at the top of their group. In education, that means schools that teach their students more than other schools. They and they alone should receive the federal government’s Blue Ribbon award.
—Tom Loveless directs the Brown Center on Education Policy at the Brookings Institution. Paul DiPerna is a researcher in the Brown Center.

The post In Praise of Mediocrity appeared first on Education Next.

]]>
49696037
Speaking in Many Tongues https://www.educationnext.org/speakinginmanytongues/ Thu, 20 Jul 2006 00:00:00 +0000 http://www.educationnext.org/speakinginmanytongues/ The common stereotypes of Christian schools mask their healthy diversity

The post Speaking in Many Tongues appeared first on Education Next.

]]>

Walk into one Christian school in North Carolina, as I recently did, and you’ll find a 2nd-grade classroom wall covered with creative writing about a Monet painting. Below the painting of overcast skies and small boats on shimmering waters, the children’s stories answer the questions, “Where does the light in the painting come from?” and, “How does this painting make you feel?” Progressive educators would be impressed with the classroom’s open, child- and group-centered learning.

Visit another Christian school, just 70 miles away, and you’ll find 1st-graders reciting their phonics lesson in unison. A bright young woman leads the class in a chorus of “Aaah, Aaah, Apple,” “Eh, Eh, Egg.” The class has a community spirit, much like singing a hymn at church. The drill includes the teacher’s asking factual questions, looking for a single right answer. Though the classroom atmosphere is hardly grim, some educators would consider the structure too rigid. They would disapprove of the teacher-centered instruction, of the seemingly heavy-handed curriculum that leaves little room for individual styles of learning and creativity.

Two Christian schools, two very different styles. Yet the stereotype of conservative Christian schools survives: that of a monolithic movement, of carbon-copy schools rigidly structured according to conservative Protestant interpretations of the Bible and views of morality. Researchers are beginning to question this simple picture, discerning a healthy diversity among schools that were once portrayed as interchangeable. Both the debate over school choice and the general controversy over whether religious education can serve public purposes might benefit from understanding just how much diversity there is within the Christian-schools movement.

Distinctions

The explosive growth of the conservative Christian school movement in the 1970s and 1980s was a response to the events and trends of the turbulent ’60s: the consolidation of a secular science curriculum after the Soviet Union raced ahead in space exploration; the rise of the counterculture and the rioting in urban areas; and the Supreme Court decisions that restricted prayer and Bible reading in public schools. In the eyes of many Christian schoolers, they have not chosen to withdraw from American society; the culture and society have moved away from them.
Critics of Christian schools often recount a different history, one that is rooted in the creation of southern segregationist academies to allow white students to escape court-ordered busing. No doubt racial integration in the public schools played a large role in spawning many Christian schools in the past, but recent studies show that Christian schools often generate greater interaction among students of different races than do public schools. Most Christian schools now see racial inclusion as a way to demonstrate the ability of religion to create ties across races. Most of the schools I visited made special efforts–including offering reduced tuition–to reach out to African-American churches to increase the racial diversity of their schools.

At a time when Catholic enrollments have been declining and overall private enrollments are barely holding their own (Figures 1 and 2), Christian schools are booming. They numbered roughly 2,500 in 1972; today, their 9,000 schools account for about 25 percent of all private schools in the United States. Forty years ago, mainline Methodist and Episcopal schools, and perhaps even Catholic schools, were numbered among the country’s “Christian schools.” Yet the dramatic growth in the 1970s of schools within “conservative” Protestant religious movements led to a narrower definition: a “Christian school” is one that’s affiliated with one of the conservative Protestant denominations, such as Southern Baptist and Lutheran Church-Missouri Synod, and, in general, with one of the dominant streams within conservative Protestantism–the evangelical, charismatic, fundamentalist, and Pentecostal religious movements.


While they have a family resemblance, these conservative Protestant religious movements are by no means identical. The fundamentalist movement’s Christian schools, for instance, emerged from its radical stance on separating believers from “the world.” Often found among Baptist denominations, fundamentalists seek to nurture children in the faith by uniting church, home, and school. By contrast, the evangelical movement, which arose in the 1940s to oppose the fundamentalists, tends to think of Christian schools less in terms of walling out the world’s influence and more as a means of integrating conservative religious traditions into the broader society and culture.

Pentecostals share the separatist stance of fundamentalists, but their style of worship–their emotional and experiential faith, with its emphasis on spiritual experiences such as speaking in tongues, healing, and prophecy–contrasts with the more-staid expressions of fundamentalism. Pentecostalism’s origins among the poor further enhance its adherents’ feeling of being outsiders vis-à-vis secular society. The charismatic movement, a 1960s offshoot of the evangelical movement, is similar to Pentecostalism in its style of worship, yet it eschews highly structured forms of education in favor of individual expression and creativity.

Among conservative Christians, the strongest expressed support for Christian schools is found within the Pentecostal and charismatic movements. These movements are in greater tension with the status quo at public schools and define a much stronger boundary between their faith and the secular world. By contrast, evangelicals tend to believe that Christians should keep their children in public schools as witnesses and as sources of influence on non-Christians, the school, and the nation. Evangelicals usually leave public schools reluctantly and see Christian schools as a stopgap measure in these “troubled” times.

Denominational Diversity

In a few cases, the diversity among Christian schools also emanates from their denominations. For example, the Lutheran Church-Missouri Synod (LCMS), a conservative branch of the Lutheran church, has a tradition of Christian schools that reaches back to the 19th century. For a single denomination, the LCMS boasts the largest number of private religious schools (more than 1,000) outside of the Catholic church. Schools affiliated with the LCMS have their origins in the German ethnic communities of the Midwest. Like Catholics, German Lutherans were not welcomed with open arms into the growing public schooling movement of the late 19th and early 20th centuries, so they turned to establishing their own schools. Later, the LCMS schools grew stronger and took on greater symbolic importance in the denomination’s struggles against “modernism” within the liberal and mainline wings of the Lutheran church. Their German ethnic heritage and the struggle over the definition of Lutheranism have led to LCMS schools that are tightly integrated with their local churches, permeated with traditional Lutheran doctrine, and staffed by Lutherans trained in LCMS colleges. The importance of this connection between school and church is evident in the tuition policies of the schools, which offer substantial discounts for members of the LCMS church. The result is that LCMS students are often the majority in the schools, especially at the elementary-school level. In addition, the schools take on the task of preparing LCMS children for membership in the church and the partaking of communion. These “confirmation classes” are directly integrated into the LCMS school curriculum.

Another important denominational source of Christian schools–especially considering the small size of the denomination–is the Christian Reformed denomination, which traces its theological heritage not to Martin Luther but to John Calvin. Based largely in Michigan and Iowa, Christian Reformed churches developed schools in keeping with their Dutch ethnic heritage and their religiously grounded belief that education is inherently value-laden and, therefore, Christians must attempt to integrate a Christian perspective on knowledge into every educational nook and cranny. One of the most influential national Christian schooling organizations, Christian Schools International, grew out of the Reformed denominations.

The differences among Christian schools within the various conservative Protestant religious movements and denominations are not always sharp. Over time, ethnic distinctiveness has subsided, and some conservative Protestant religious traditions in schooling have diffused throughout the movement–partly through the growth and consolidation of Christian-schooling organizations and of publishers of Christian school materials. The financial realities of Christian schools also tend to dull the sharp edges of religious doctrine. Almost all Christian schools are financially precarious. Annual tuition, usually somewhere between $1,500 and $4,500 per student, covers some 80 percent of the annual budget. Christian schools simply can’t afford to turn away too many tuition-paying students. This requires them to appeal to a broad, diverse group of parents, which influences their policies and practices, sometimes forcing them to relax religious prescriptions.

There are at least three types of Christian schools. The first consists of Christian schools affiliated with the evangelical movement. It encompasses some Southern Baptist schools, most Christian Reformed schools, and many independent Christian schools. This type tends to be more open to professional educational techniques and more focused on an academic mission. The Fundamentalists–the whipping boy of the popular press–focus their schools on developing personal character and discipline and teaching fundamentalist doctrines, such as the literal interpretation of the Bible and six-day creationism. The third group of Christian schools, affiliated with the Lutheran Church-Missouri Synod, has a more specific ethnic bias but seems to mirror the fundamentalist Baptist schools in its emphasis on creationism in science classes.

Beyond these three types of conservative Protestant schools, the boundaries are less clear. Schools affiliated with the charismatic movement may look exactly like evangelical schools. Little is known about African-American Christian schools, but they are likely to differ not only because of race but also because of religion.

The Bible, apparently, can justify several very different schooling strategies. For example, some conservative Christian schools require that parents share the religious orientation of the school, while others invoke the Bible to support open-admissions policies. Some Christian schools use biblical passages to justify a disciplined learning structure. Others embrace learning environments designed to foster creativity and community by drawing on doctrines that see individuals as reflecting God’s image through creativity, imagination, and social solidarity.

The same is true for issues of science versus faith. Some conservative Protestants use a literal interpretation of Genesis to justify teaching “creation science” as opposed to evolution. Others see the biblical creation story as perfectly consistent with the general outlines of scientific theories of evolution. Some schools are divided from within, as even the most devout conservative Protestants struggle to figure out exactly what the Bible has to say about whether the classroom environment should be more authoritarian or egalitarian and whether schools should focus on the advancement of all students or expend special resources on the academically gifted–or on those who lag behind.

Parents

Contrary to common assumptions, only a small minority of conservative Christians take the stance that Christian kids should be in Christian schools as a matter of principle. In fact, most evangelicals have long considered participation in public schools as a religious calling. When asked whether Christians should try to fix public schools or build strong Christian schools, 56 percent of Christian schoolers favored working with the public schools. It is common for parents to send one child to Christian school and others to public school, based on their assessment of each youngster’s personal and moral strength and ability to withstand peer pressure at public schools. The American emphasis on expressing individuality has as much of an effect on most Christian parents as it does on everyone else.

Indeed, many Christian parents do not develop a strong, ideological commitment to the Christian school. More likely, the school is viewed as the appropriate vehicle in light of a child’s needs, his personality, his experience at public schools, and available alternatives. In this way and others, Christian schools confront the same cultural trends as other American institutions–trends that value individual choice over institutional authority. The American penchant for expressive individualism means that most conservative religious parents often place a higher priority on the unique qualities of individual children and the prerogatives of parental choice than on church dogma. This makes it difficult for schools to develop a solid and committed clientele.

Christian-schoolers, not surprisingly, are distinctive in what they see as the most important goals of a good education. They are less likely than other churchgoing Protestants to think that learning job skills is the top educational priority; they are much less likely to see building children’s self-esteem as an important educational goal. Interestingly, even the average Christian schooler is not much less likely than other churchgoers to think that a top priority in education is teaching children about diverse races, religions, and cultures (67 percent versus 73 percent). While Christian schoolers are commonly thought to choose Christian schooling so that their children may learn discipline and respect for authority, they are no more likely than other churchgoing Protestants to see this as a top priority in education (84 percent of both groups). Christian schoolers are much more likely, however, to say that teaching a Christian perspective on knowledge is a top priority in a good education.

Textbook Faith

Before reading the profiles of several Christian schools in North Carolina and Indiana that are scattered throughout this essay, it is important to understand the differences among the various curricula they use. (Note that the names used in the profiles are fictitious, since Christian schools, like most public institutions, are sensitive about their public image.) In terms of coursework, the curricula at Christian schools generally follow the standard secular models of what education is about. Surveys show that they are no different from public schools in the number of math, English, science, social studies, and computer-science classes that are required for graduation. Christian schools have a slightly greater emphasis on foreign languages, which may reflect the ethnic origins of some of them. Yet the evidence does not show that Bible and theology requirements are crowding out the standard curriculum found in public schools.

What is noticeably Christian about Christian schools is the content of the curricular materials they use. A Beka and Bob Jones University Press are two of the heavyweights in Christian school curriculum, offering everything from English and science to a full-fledged Bible curriculum. Many fundamentalist schools build their curriculum entirely around the publications of these presses. A Beka, which sees itself as building “the content of every textbook on the foundation of God’s Word,” states that it does not “paraphrase progressive education textbooks and add biblical principles,” but rather does “primary research in every subject and looks at the subject from God’s point of view.”

Evangelical schools often disparage A Beka and Bob Jones University Press, preferring secular textbooks or texts available from Christian Schools International (CSI) and–especially for classical Christian schools–Veritas or Logos. The curriculum available from evangelical publishing houses is less likely to promote the view frequently found in fundamentalist texts that the United States is a Christian nation that needs to be restored to its former greatness, or is a “redeemer” nation unto the world.

In terms of evolution versus creationism, fundamentalist science materials “teach young students the facts of Creation as presented in the Bible” and “set forth the days of Creation and what was created on each day.” Evangelical science texts, by contrast, do not insist on a literal six-day creation. CSI claims its science materials will “empower students to discover the infinite complexity and amazing orderliness of God’s world. . . . The Bible and science are complementary, together helping us to understand God’s plan and purpose for creation.”

Governance

Organizational and socioeconomic characteristics often create differences among Christian schools. Christian schools run by a board of parents from several different churches tend to differ in important ways from schools subsidized and run by a particular church. A school that is run by and for a local church is more likely to mirror the religious contours of the denomination in its classrooms, Bible classes, and chapel. Its mission and organization are shaped by its primary constituency, which is usually the members of the church itself. The elders and pastors of that church tend to have considerable say over decisions in the school. The principal may also be an associate pastor at the church. By contrast, schools run by a board of Christian parents, independent of any local church, tend to incorporate more generic forms of the Christian faith, partly because of the religious diversity of the board and parents.

In a similar vein, middle-class and more-educated parents tend to shape Christian schools toward less tension with the outside world, greater emphasis on academic excellence, less rigid social control of students, greater room for individual creativity and expression, and less denominationally distinctive ways of integrating religion into school life.

The day-to-day governance of Christian schools is very different from that of public schools, in large part because control over such things as hiring practices and curricular policies is vested entirely at the school level. Indeed, Christian school board members often account for a good percentage of the donations that keep the school running, functioning as shareholders of a corporation that always runs in the red. At one school, for example, the board compared tuition income with actual expenses at the end of the year, and divvied up the deficit among themselves.

The financial difficulties of Christian schools not only force them to put significant energies into fundraising, but also make them more market-driven than is often acknowledged. Market forces soften the hard edges of religion as schools compete to attract students and financial support. Pressures for religious particularity–such as teaching students the Baptist doctrine that only adult believers should be baptized–are nearly always checked by the need to attract more students. In fact, one study has found that Christian schools are a moderating force on religious sectarianism, since the schools must attract students from other churches, and parents from different churches come into close contact through involvement in a common school.
To ensure that family and school pull in one direction, most Christian schools require that one or more parents of an enrolling child sign a “covenant,” or statement of faith, and perhaps also a lifestyle agreement. These statements of faith, however, vary from a minimal assent to faith in Jesus Christ to a nearly complete doctrinal statement of a Baptist denomination.

A significant minority of Christian schools sees their mission as including applicants without regard to religion, as long as parents are aware of the religious nature of the school and will not oppose it. This organizational difference often marks a deeper fault line within the Christian school movement: While the more fundamentalist schools generally require covenants, the evangelical schools are split on this issue. Some believe that maintaining a Christian identity is done at the level of teachers and administrators, and they see their mission as enveloping Christian and non-Christian students alike.

Private and Public Purposes

With so much diversity, what makes the Christian-schools movement a movement? What unites them? One constant mission across Christian schools is that family, church, and school should work together for the intellectual, moral, and spiritual direction of the child. Reflecting this mission, the primary educational goal at Christian schools, according to 68 percent of Christian-school principals in the United States, is the religious or spiritual development of students. Not all principals, however, agreed: 13 percent saw the primary goal as basic literacy skills; 12 percent mentioned academic excellence. Only 1 percent mentioned the goal of promoting good work habits and self-discipline, often thought to be the forte of Christian schools (see Figure 3).


The runner-up goal most often mentioned by principals is academic excellence: 37 percent of the Christian schools indicating religious development as their top education goal said academic excellence was their second most important one. Again, neither self-discipline nor specific moral values was a popular choice, but achieving basic literacy skills was fairly popular: 27 percent of these schools said this was their second most important goal.

Religious mission leads to another constant across Christian schools: Nearly all of them require a fairly strong statement of faith from job applicants. Applications for teachers often include a written or verbal personal testimony of their faith and agreement on what are considered the basics of the faith, such as belief in Jesus Christ through faith and an obligation to live a holy life in response to grace. Some of the more conservative Baptist-affiliated schools require employees to be in accord with more detailed Baptist doctrine, though there is plenty of slippage between stated policy and actual practice. The LCMS schools take a slightly different approach: they require almost all of their teachers and administrators to be trained at LCMS colleges.

Still, despite their distinct religious missions, the sharp boundary between public schools and private religious schools makes it difficult for us to see the ways in which today’s so-called private schools perform public functions. Even fundamentalist schools spend much more time teaching the basics of reading, writing, and arithmetic than teaching the Bible and Baptist doctrine. Most of the “integration” of religion and basic subject matter at Christian schools would not offend the average American of any faith.

Christian schools also serve a public function by providing an organizational space for alternative teaching techniques and forms of schooling. They keep alive or develop instructional approaches and educational philosophies that may not have an opportunity to thrive in public schools, which are often governed by outside professional dictates and state mandates. For example, the trend toward “whole language” techniques for teaching reading in public schools contrasts with the hard-core phonics approach in fundamentalist Christian schools. Children with different learning styles may find what they need in a Christian school. Likewise, Christian schools offer models that may prove useful to public schools by eschewing standardized testing; reviving Latin, logic, and rhetoric; emphasizing the place of music and foreign languages; refusing to track students by ability; or choosing an unhurried approach to learning. Coherent missions and moral socialization through networks that incorporate family, school, and other aspects of life (in this case, churches) are among the hallmarks of Christian schools and may provide direction to public educators who are worried about alienated adolescents and withdrawn families. Christian schools have long served as models for local control of decision-making and smaller school size.

For parents whose children are not succeeding in public schools, the alternatives nurtured in Christian schools may provide a viable cure. One family I talked with turned to a Christian school after one of their children seemed overcome by negative peer influence, discipline breakdowns, and drug problems in their public schools. The smaller school size, the strong and overlapping social networks, and the sacred canopy over moral norms and values made it more possible to instill self-discipline in their child.

The real diversity among Christian schools and the fact that they do serve public purposes do not put to rest all of the standard critiques of Christian schooling–concerns about racial segregation, authoritarianism, dogmatism, insularity, and social inequality. Yet these criticisms are not nearly as far-reaching as they once seemed. The influence of the market on financially precarious Christian schools has forced them to moderate the role of religion and to create a more democratic governing structure that is responsive to parents. Parents’ commitment to a Christian school is also looser than it might seem, since American individualism and parents’ concern for their children often trump institutional and community authority. The old stereotype of “fundamentalist” Christian schools as rigid and insular distorts their public image. The pressures of mainstream culture and market forces, as well as internal differences in religious style and governance, have created a wide diversity of Christian schools–a diversity that in many ways contributes to the public good.

—David Sikkink is an assistant professor of sociology at the University of Notre Dame.

The post Speaking in Many Tongues appeared first on Education Next.

]]>
49696042
Vouchers in New York, Dayton, and D.C. https://www.educationnext.org/vouchersinnewyorkdaytonanddc/ Thu, 20 Jul 2006 00:00:00 +0000 http://www.educationnext.org/vouchersinnewyorkdaytonanddc/ Vouchers and the Test-Score Gap

The post Vouchers in New York, Dayton, and D.C. appeared first on Education Next.

]]>
Just ten years ago, the only data available on the impact of school vouchers came from a poorly designed public-choice program conducted during the 1960s in Alum Rock, California. But the early and mid-1990s brought new privately and publicly funded voucher programs to cities such as Milwaukee; Dayton; Cleveland; Indianapolis; San Antonio; Washington, D.C.; and New York City. With them came a wealth of new research opportunities.

The foundation in New York City offered 1,300 scholarships, each worth up to $1,400 annually toward tuition at a private school.


The privately funded voucher programs in New York City, Dayton, and the District of Columbia are especially conducive to study. In each city, vouchers were awarded randomly, generating treatment and control groups that are statistically indistinguishable from one another. Before conducting the lotteries, our evaluation team collected data on student test scores and family background characteristics. One and two years later, we retested the students. Since the two groups of students-the lottery’s winners and losers-had similar average abilities and family backgrounds, any subsequent achievement differences observed between them can be attributed to the effects of the vouchers.

As a result, our evaluations of the New York, Dayton, and D.C. voucher programs have yielded the best available information on students’ test-score outcomes and parental assessments of public and private schools. Here we use the data from all three cities to analyze the one- and two-year effects on academic performance of switching from a public to a private school. We find that vouchers have a moderately large, positive effect on the achievement of African-American students, but no discernible effect on the performance of students of other ethnicities.

The Literature


Earlier comparisons of public and private schools generally have found that low-income and African-American students who attend private schools outperform their public-school peers. For instance, University of Wisconsin economist Derek Neal’s analysis of the National Longitudinal Survey of Youth found that, even after adjusting for family background characteristics, students from Catholic schools were 16 percentage points more likely to go to college than were public-school students. The gap between Catholic-school students and public- school students was largest among urban minority children. Other studies have reached similar findings. University of Wisconsin political scientist John Witte’s review of the literature on school effects led him to conclude that studies of private schools “indicate a substantial private-school advantage in terms of completing high school and enrolling in college, both very important events in predicting future income and well-being.”

All of these studies, however, have one important limitation. They can account for only observed family background characteristics, such as the mother’s educational level, a student’s ethnicity, or family income. There is no assurance that these studies have successfully controlled for an intangible factor: the willingness of parents to pay tuition to send their children to private school and all that this implies about the value they place on education. As a result, it remains unclear whether these studies have unearthed actual differences between public and private schools or simply differences in the kinds of students and families attending them.

The best way to compensate for this limitation is to assign students randomly to experimental and control groups whose only substantive difference is whether they are offered a voucher. Past evaluations of voucher programs have not been able to take full advantage of a random-assignment research design. Consequently, the findings from New York, Dayton, and D.C. provide a unique opportunity to examine the effects of school vouchers.

The Programs


In several key respects, the three voucher programs followed similar designs. All were privately funded; all were targeted at students from low-income families, most of whom lived in the inner city; all provided only partial vouchers, expecting the families to supplement them; and all of the students in the evaluations previously had been attending public schools. Brief descriptions of the three programs follow.

New York City. The School Choice Scholarships Foundation (SCSF) in New York City offered 1,300 scholarships worth up to $1,400 annually toward tuition at a private school for at least three years. To qualify for a scholarship, children had to be entering grades 1 through 4, live in New York City, attend a public school at the time of application, and come from families with incomes low enough to qualify for the U.S. government’s free or reduced-price school-lunch program. More than 20,000 students applied between February and late April 1997. By the end of the scholarship program’s second year, 64 percent of the lottery-winning students were attending a private school.

Dayton, Ohio. In the spring of 1998, Parents Advancing Choice in Education (PACE) offered low-income students in grades K-12 the opportunity to win a scholarship to attend private school. For the 1998-99 school year, PACE offered scholarships to 515 students who were in public schools and to 250 who were already enrolled in private schools in the Dayton metropolitan area. During the program’s first year, the PACE scholarships covered 50 percent of tuition at a private school, up to $1,200. Support was guaranteed for at least four years, with a possibility of continuing through high school, provided funds remained available. Of those students offered scholarships, 49 percent enrolled in a private school during the second year of the program.

Washington, D.C. Established in 1993, the Washington Scholarship Fund (WSF) is the oldest of the three programs. By the fall of 1997, the WSF was serving approximately 460 children at 72 private schools. On receiving a large infusion of new funds from two philanthropists, the WSF announced a major expansion in October 1997.

To qualify, applicants had to reside in Washington, D.C., and be entering grades K-8 in the fall of 1998. Families with incomes at or below the poverty line received vouchers that equaled 60 percent of tuition or $1,700, whichever was less. Families with incomes above the poverty line received smaller scholarships. Families with incomes higher than two-and-a-half times the poverty line were ineligible. The WSF claims that it will maintain tuition support for at least three years and, if funds remain available, until students complete high school. In April 1998, the WSF awarded more than 1,000 scholarships by lottery, with the majority going to students previously attending a public school. Of those students offered scholarships, 35 percent were still using them to attend a private school in the second year of the program.

Evaluation Procedures


The evaluation procedures used in all three studies conformed to those used in randomized field trials. Our evaluation team collected baseline test scores and family background information before the lottery, administered the lottery, and collected follow-up information one and two years later.

Students took the Iowa Test of Basic Skills (ITBS) in reading and mathematics. Students who were entering grades 1-4 in New York City and grades 2-8 in Dayton (and other parts of Montgomery County, Ohio) and Washington, D.C., were included in the evaluations. Parents responded to survey questions about their satisfaction with their children’s schools, their involvement in their children’s education, and their demographic characteristics. Students in grades 4 and higher completed similar surveys. In all three cities, the follow-up procedures replicated the pre-lottery procedures: students again took the ITBS in reading and math; parents and older students filled out surveys about their backgrounds and educational experiences.

More than 5,000 students participated in pre-lottery testing in New York City. Of the families that did not win the lottery, approximately 1,000 were selected at random to compose a control group of approximately 960 families. All of these students were attending public-schools at the time. In Dayton, 1,440 students were tested before the lottery; 803 of them were attending public schools at the time. In Washington, D.C., 2,023 students were tested before the lottery; 1,582 of them were attending a public school. In Dayton and in D.C., separate lotteries were held for students who were enrolled in public and private schools at the time of application. The fact that only public school children were eligible to apply for a scholarship in New York obviated the need to hold separate public and private lotteries there. In all three cities, only those students who were in public schools at the time of the lottery are included in this study.

In New York City, 42 percent of the students participating in the second year of the evaluation were African-Americans; in Dayton, 74 percent; and in D.C., 94 percent. Hispanic students accounted for 51 percent of the New York City group and 2 percent and 4 percent of the Dayton and D.C. groups, respectively. Whites accounted for 5 percent of New York City’s evaluation group, versus 24 percent in Dayton and 1 percent in D.C. The remaining students came from a variety of other ethnic backgrounds.

In New York City, 80 percent of the students included in the evaluation attended the first-year testing sessions; 66 percent attended the second-year sessions. In D.C. the response rate after one year was 63 percent; after two years, 50 percent. In Dayton, 57 percent of families attended follow-up sessions after one year, 49 percent after two years.

We are reasonably confident that these modest response rates do not undermine the integrity of our findings. First, with the exception of the second year in New York, response rates were similar for both the treatment and the control groups after one and two years in all three cities. Second, comparisons of baseline test scores and background characteristics revealed only minor differences between the composition of the test and control groups in all three cities. Finally, to account for the minor differences between respondents and nonrespondents that we did observe, the test scores of children who, based on their demographic characteristics, were more likely to attend follow-up sessions were weighted less heavily, while the test scores of children who were less likely to attend follow-up sessions, but nevertheless did, were weighted more heavily. Given the slight differences between respondents and nonrespondents, however, the weights had little effect on the results.

Vouchers have a moderately large, positive effect on the achievement of African-American students, but no discernible effect on the performance of students of other ethnicities.

The randomized lottery ensured that lottery winners as a group were not significantly different from the control group (those who did not win a scholarship). In all three cities, the demographic characteristics and pre-lottery test scores of scholarship winners and losers (the treatment and control groups, respectively) resembled one another. Only in Dayton were there minor differences in the pre-lottery test scores: those offered a voucher scored 6.5 percentile points lower in math and 3.1 points lower in reading than those not offered a scholarship, a statistically significant difference.
To measure the effect on children’s test scores of switching to a private school, we estimate a statistical model that takes into account whether a child attended a public or a private school, as well as baseline reading and math test scores. Baseline test scores were included to adjust for the minor baseline differences between the treatment and control groups on the achievement tests and to increase the precision of the estimated impact.

The lottery generated two groups: those who were offered a voucher and those who were not. We’re not interested, however, in the effect of being offered a voucher. Rather, we’re interested in the effect of using a voucher to attend a private school. A significant number of the students who were offered vouchers did not use them; similarly, a smaller proportion of those students not offered a voucher attended a private school anyway. Therefore, a simple comparison between public and private school students is inappropriate because certain students may be more likely to take advantage of a voucher. Their parents may place greater value on education and be more willing to supplement the voucher, or they may live in a neighborhood with a broader selection of private schools. If these children differ from students who won a voucher but failed to use it in ways that are related to student achievement, it could bias our findings. To solve this problem, we used as an instrumental variable whether or not a student was offered a voucher to predict the probability that she attended a private school; with these predicted values, we can provide an unbiased estimate of the actual impact of switching from a public- to a private-school. This two-stage regression technique was first used in medical research and is now commonplace in econometric studies.

Results


Our findings varied by ethnic group. In all three cities, there were no significant differences between the test-score performance of non-African-American students who switched from a public to a private school and the performance of students in the control group-after either one or two years. For African- American students, however, the receipt of a voucher made a substantial difference. In the three cities combined, African-American students who switched from public to private schools scored, after one year, 3.3 percentile points higher on the combined math and reading tests (expressed as National Percentile Ranking [NPR] points, which run from 0 to 100 with a national median of 50). After two years, African- American students who used a voucher to enroll in a private school scored 6.3 percentile points higher than African-American students who remained in public schools (the control group) (see Figure 1).


We emphasize the overall test scores, which represent the average of the math and reading components. When using one-hour testing sessions to gauge student performance, combined reading and math scores serve as a better indicator of student achievement than either test separately. Theoretically, the more test items used to evaluate performance, the more likely performance will be measured accurately.

Nevertheless, the differences after two years were approximately the same for both the reading and the math tests. On average in the three cities, African-American students who switched from public to private schools scored 6.3 percentile points higher than their peers in the control group on the reading portion of the test and 6.2 points higher on the math portion.

The largest test-score differences between African-American students in private schools and African-American students in public schools were observed in the D.C. program. Black students who attended D.C. private schools for two years scored 9.0 percentile points higher on the two tests combined than did students in the control group. The smallest differences after two years were observed in New York City, where the combined test scores of African-American students attending private schools were 4.3 percentile points higher than those of the control group. In Dayton the difference was 6.5 percentile points for African-American students.

The trend over time also varied from city to city. In New York City, at the end of the first year, African-American students in private and public schools displayed substantial differences in test scores, but these diminished slightly in the second year. After two years the difference in scores is 4.3 percentile points, which is slightly but not significantly (in statistical terms) less than the 5.8 percentile point difference observed after one year. It is reasonable to conclude that African-American students’ initial gains in the New York City school voucher program were preserved but did not increase between year one and year two.

In Dayton, there appears to be a steady upward trend in the combined test-score performance of African-Americans. African-American students who switched from public to private schools performed 3.3 percentile points higher on the combined test in year one and 6.5 percentile points higher in year two.

In some ways, the most striking results in terms of trends over time concern African-Americans in D.C. After one year, no significant differences were observed for African-American students as a group, but older and younger students experienced significant differences. While younger students may have benefited slightly from the voucher program after one year, the older students who switched to private schools scored significantly lower than their public- school peers after one year. By the end of the second year, however, these students seemed to have overcome the initial challenges of changing schools. Both younger and older African-American students who switched from public to private schools posted positive and significant gains. On the combined reading and math tests, younger students in private schools scored 9.3 percentile points higher than those who remained in public schools. Older African-American students in private schools scored 10.3 percentile points higher.

The lottery ensured that scholarship winners as a group were not significantly different from those who did not win a scholarship. In all three cities, the
demographic characteristics and pre-lottery test scores of scholarship winners and losers resembled one another.



Controlling for Demographics


Most research on the impact of private schools attempts to control for differences in family income and other background characteristics among students attending public and private schools. When a lottery is used to assign research subjects to experimental and control conditions, however, such statistical adjustments are generally unnecessary simply because the two groups being compared are virtually identical.

After two years, African-American students who used a voucher to enroll in a private school scored 6.3 percentile points higher than African-American students who remained in public schools.

Nonetheless, after the release of our study, some analysts objected to the apparent absence of controls for family background characteristics. Bruce Fuller and his colleagues at the University of California, Berkeley, for instance, argued, “The experimental group may have been biased as some of the most disadvantaged voucher winners did not switch to a private school, and therefore were excluded from the group (possibly boosting mean achievement levels artificially).” An interest group, People for the American Way, lodged a similar complaint: “The study’s key finding improperly compares two dramatically different groups and may well reflect private-school screening-out of the most at-risk students.”

In the three cities roughly half the students initially took the voucher that was offered to them (the takers), and about half did not (the decliners). Takers had higher family incomes in New York and D.C., but lower incomes in Dayton. The New York and D.C. findings are not surprising, given that the voucher awards did not cover all the costs of a private education. These additional costs were the reason most frequently given by families for not using the voucher. Presumably acceptance rates would rise if the monetary value of the vouchers were increased.

However, we did not drop the decliners from the analysis, as some of our critics have charged. All voucher applicants were invited to follow-up testing sessions, and each of the families who participated, including those who declined a scholarship, is included in the analysis. To estimate the impact of switching from a public to a private school, we did not simply compare those students who used a voucher to enroll in a private school with all those who did not. Such a comparison would have introduced bias and squandered all the advantages of a random-assignment evaluation. Instead, we used a familiar technique, often used in medical and econometric research, that preserves the essence of a random-assignment evaluation. The outcome of the lottery, a random event, was used to create what statisticians refer to as an instrumental variable, which obtains unbiased estimates of the effects of attending private school on students’ test scores. According to the statistical theory that underpins this technique, results from lotteries are powerful instrumental variables, because the lottery, being a random event, is not directly related to students’ test-score performance. In other words, the use of this statistical technique fully corrects for any differences that arise from the fact that not all of the families who were offered a voucher made use of one.

When similar results emerge from school voucher
programs in three very different cities, we can be fairly confident that the intervention is the main cause of differences in achievement.

To see whether the instrumental variable worked in practice as it should in theory, we conducted a second analysis in which we controlled not only for the students’ pre-lottery test scores but also for their mothers’ educational level, her employment status, family size, and whether the family received welfare. If the critics were correct, the introduction of these background characteristics into the analysis should have diminished the estimated effect of attending a private school, because only after these adjustments were made would the analysis have adjusted for the background differences between those who used the voucher and those who did not. But if the use of the lottery as an instrumental variable works in practice as it is expected to work in statistical theory, it would already have corrected for these differences. The results should remain essentially the same.

As statistical theory anticipates, the average difference in the combined reading and math test scores of African-Americans in all three cities remained exactly the same-6.3 NPR points-after the adjustments for family background characteristics were introduced. Minor differences in the two estimates were observed within each city. The impact of switching to a private school without controlling for family background in New York City was originally estimated to be 4.4 NPR points; after accounting for family background, the impact was estimated to be 4.2 NPR points. Introducing controls in Dayton decreased the estimated impact from 6.5 to 5.9 NPR points. In Washington, D.C., the estimated impact increased from 9.0 to 9.1 NPR points. In New York and Washington, the estimated impacts, after adding controls for family background, remain statistically significant. In Dayton, the impact just missed the standard threshold for statistical significance.

Discussion


It is possible that conditions specific to each city or minor fluctuations in testing conditions might skew results one way or another. But when similar results emerge from the evaluations of school voucher programs in three very different cities, we can be fairly confident that the intervention is the main cause of the differences in achievement.

In general, we found no evidence that vouchers significantly improved the test scores of ethnic groups other than African-Americans, most notably Latinos in New York and whites in Dayton. The impact of vouchers for African-Americans, however, was moderately large. After one year, black students who switched to private schools scored 0.17 standard deviations higher than the students in the control group. After two years, the difference grew to 0.33 standard deviations, roughly one-third of the test-score gap between blacks and whites nationwide. These effects are approximately the same as those observed in Tennessee when class sizes were reduced from 24 students to 16 students, a much more costly intervention.

Whether the gains from these small, private scholarship programs will translate to large-scale, publicly funded school-choice programs in urban areas is unknown. Only a small fraction of low-income public-school students in New York, Dayton, and D.C. were offered vouchers, and these students made up a small share of the cities’ private-school populations. A much larger program carried out for longer periods of time could yield quite different outcomes. But we’ll never know unless we try. The nation’s capital, the city where the largest effects were observed, would be a good place to begin.

-William G. Howell is an assistant professor of political science at the University of Wisconsin-Madison. Patrick J. Wolf is an assistant professor of public policy at Georgetown University. Paul E. Peterson directs the Program on Education Policy and Governance at Harvard University, where David E. Campbell is a research associate. To view their study in its entirety, log on to www.edmattersmore.org.

The post Vouchers in New York, Dayton, and D.C. appeared first on Education Next.

]]>
49696046
Vouchers in Charlotte https://www.educationnext.org/vouchersincharlotte/ Thu, 20 Jul 2006 00:00:00 +0000 http://www.educationnext.org/vouchersincharlotte/ Vouchers and the Test-Score Gap

The post Vouchers in Charlotte appeared first on Education Next.

]]>
During the 1999-2000 school year, the private Children’s Scholarship Fund (CSF) offered partial scholarships to low-income students in Charlotte, North Carolina. The partial scholarships defrayed up to $1,700 in tuition expenses at the private elementary or secondary school of a family’s choosing. Scholarships were awarded by lottery to families who went through an application process, because not enough funds were available to provide them to all the interested families.

The awarding of scholarships by lottery created a rare opportunity in educational research: a field experiment in which students were assigned randomly to both public and private schools, thus allowing me to test the effects of receiving a voucher and, more generally, to compare the performance of public and private schools. The study used both standardized test scores and surveys of parents and students to evaluate the effect of the scholarship program on both academic performance and student and parental satisfaction.

Data


Only students enrolled in grades 2 through 8 were tested for this study. These students fell into three categories: 1) those who won a scholarship and used it to enroll in a private school (“choice students”); 2) those who won a scholarship but either moved out of the area or elected to remain in a public school for a variety of reasons (“noncomplying students”); and 3) those who didn’t win a scholarship (“control students”).

Overall, 388 students used a scholarship to enroll in private school; 413 students won the lottery but did not enroll in private school; and 342 students didn’t win a scholarship. Near the end of the 1999-2000 academic year, all of these students and their parents were invited to attend testing sessions where parents completed surveys while students took the Iowa Test of Basic Skills (ITBS) survey version. Students in grades 4 through 8 also completed a survey. Of the 1,143 students who were sent invitations to attend a testing session, 452, or 40 percent, participated in the study (within the three groups, 53 percent of the choice students, 20 percent of the noncomplying students, and 49 percent of the control students participated in the study).

Most of the private schools at which students used the scholarships operate with less than half as much per-pupil spending as the public schools.


Even as favorable a research design as a lottery is susceptible to the problems of nonparticipation and noncompliance. If students with similar demographic characteristics drop out of the study in large enough numbers, they can upset the balance between the experimental and control groups, rendering them less comparable. But my data show that the incomes of the participating and nonparticipating families were roughly equal for both the lottery winners and losers, as well as for the choice, control, and noncomplying students. In other words, while those who participated in my study differed somewhat from those who did not, the differences don’t appear to have biased the comparability of the groups.

My primary interest lies in identifying the effect of using a scholarship to attend private school, not the effect of a student’s being offered a scholarship but not using it. I therefore want to compare the choice students, the students who used a scholarship to attend private school, with the control and noncomplying students, the two groups who entered the lottery but ultimately stayed in public schools. Overall, the choice students and the comparison groups were quite similar in their demographic characteristics, though clearly not identical. The test-score data were adjusted statistically to account for any observed differences between the two groups, such as level of family income-an important predictor of academic performance-that might have biased the results. In calculating the results, I controlled for a host of background characteristics, including the mother’s educational level, her race, the student’s family income, whether a student lived in a two-parent household, and the student’s sex.

Concern about the unobserved differences between families who send their children to public and private schools has always limited scholars’ ability to draw conclusions from evaluations of public- and private-school performance. Even after adjusting for observed demographic differences, researchers always wondered whether unobserved differences that were not being accounted for, such as parental motivation or the intellectual richness of home life, played a larger role than the schools themselves in causing differences in academic performance between public and private schools.

In this case, however, the application process and lottery have produced comparison groups that are already quite similar on observed as well as (in all likelihood) unobserved characteristics. All families were motivated enough to complete an application for a scholarship. Only low-income families were eligible for a scholarship. A lottery was used to select which students would be offered scholarships, creating, as the statistical analysis has confirmed, two groups that were nearly identical. While noncompliance and nonparticipation have created differences between the two groups, they are similar enough that adjusting for observed characteristics is likely to produce highly reliable results.

A lottery was used to select which students would be offered scholarships, creating, as the statistical analysis has confirmed, two groups that were nearly identical.

Results


After one year, the results show that students who used a scholarship to attend a private school scored 5.9 percentile points higher on the math section of the ITBS than comparable students who remained in public schools. Choice students scored 6.5 percentile points higher than their public school counterparts in reading after one year (see Figure 1).

Using a statistical technique known as instrumental analysis to adjust for the potential bias of noncompliance yields results that remain strong and positive. The results of this analysis show that, after only one year’s time, attending a private-school improved student performance on standardized tests in math and reading by between 5.4 and 7.7 percentile points.

On average, a scholarship raised students from the 30th percentile to the 37th percentile. This is a fairly large gain approximately 0.25 standard deviation in math and reading. To put this gain in perspective, the difference nationwide between minority and white students is approximately 1.0 standard deviation. The benefits of the Charlotte CSF program are roughly one-quarter as large at the end of only one year.

Another important measure of the scholarship program’s performance is parental opinion. While the desire to affirm their decision may distort parents’ judgment, their intimate knowledge of (and interest in) their children’s well-being puts them in a good position to assess the benefit of a particular program. Our survey asked parents to assign their child’s school a letter grade, A through F. Nearly twice as many choice parents gave their child’s school an A (53 percent) as did public-school parents (26 percent). Choice parents were also far more likely to report being “very satisfied” with virtually all aspects of their children’s school: its safety, teacher quality, class size, clarity of school goals, teaching moral values, academic quality, teachers’ respect for students, and so on (see Figure 2).

Surveys of students in grades 4 through 8 found much higher levels of satisfaction among choice students than among public-school students. Roughly 40 percent of choice students gave their private school an A, compared with 32 percent of public- school students. Responding to a question asking how they feel about going to school each day, 24 percent of the public-school students said they didn’t want to go, compared with 9 percent of private- school students. And 24 percent of nonscholarship students said they didn’t feel safe at school, compared with 9 percent of choice students (see Figure 2).

Parental reports confirmed their children’s perceptions about safety at school. More than a third of public-school parents reported problems with fighting in school (36 percent), compared with 16 percent of choice parents. One-quarter of public-school parents reported problems with racial conflict, compared with 12 percent of choice parents. Of public-school parents, 22 percent reported problems with guns or weapons at their children’s elementary schools, compared with 11 percent of choice parents. Keep in mind that none of these children is beyond 8th grade.

School Facilities and Services


Rising test scores and high levels of parental and student satisfaction indicate more-luxurious private schools with better resources, right? Far from it. Most of the private schools at which students used the CSF scholarships operate with less than half as much per-pupil spending as the public schools. Tuition at most of the private schools is less than $3,000. Additional fundraising brings no more than a few hundred dollars per student.

The private schools actually offer sparser facilities and fewer services than the public schools. For example, only 70 percent of choice parents reported their school’s having a library, compared with 90 percent of public-school parents. Only 63 percent of choice parents said their school had a gym, compared with 91 percent of public-school parents. In terms of services, only 18 percent of choice parents said that a program for students learning to speak English was made available to them, compared with 50 percent of public-school parents. Only 49 percent of choice parents reported a program for learning disabilities being available, compared with 71 percent of public-school parents. Only 51 percent of choice parents reported that their school had a program for gifted students, compared with 72 percent of public-school parents. Choice parents were also less likely to report the existence of a counselor, a nurse, a music program, an art program, or prepared lunches at their schools.

Some amenities were equally or more available at the private schools. Private and public schools were equally likely to have a computer lab. And the private schools were equally likely to offer individual tutors and more likely to offer after-school programs. When parents report that they are more satisfied with the private-school facilities, they clearly must be focusing on the features they deem most important. Private schools appear to have far fewer resources, but they concentrate those resources on providing the facilities and services that parents value most.

Making Money Matter


Why do parents like these poorly funded, amenity-starved schools so much? What might account for the improvement in their children’s academic performance? While this study was not designed to address these questions fully, I can speculate based on the data I collected. Some of the most important differences between the private and the public schools pertain to the quality and motivation of their teachers. Parents gave high marks to the quality of instruction at the private schools. So, too, did the students. Choice students are almost twice as likely to report that their teachers are “interested in students” as are public-school students. Choice students are also significantly more likely to report that their teachers listen to them, that teachers are fair, and that students get along with teachers.
Despite having less money for salaries and benefits, private schools appear to be better able to recruit quality teachers and to dismiss poor ones. They may attract higher quality teachers because they can offer positive working conditions, an organization with a clear sense of mission, and greater autonomy in the classroom. The ability of public schools to offer such perks is hampered by layers of bureaucratic regulations and management, an unfortunate by-product of political governance. Teacher union and school district rules also make the removal of poor teachers much more difficult in public than in private schools.

The private and public schools in the study also differ in the size of their student populations and average class size. The median choice student is enrolled in a school that has between 151 and 300 students. The median public-school student attends a school with between 451 and 600 students. The median choice student sits in a class with between 11 and 15 students, while the median public-school student is in a class with between 21 and 25 students. Educational researchers are increasingly recognizing that smaller districts, schools, and perhaps classes seem to improve student performance and satisfaction. Smallness may encourage the development of a sense of community and common purpose, possible keys to school success. And smallness obviates the need for rigid rules that restrict the autonomy of principals and teachers.

Some critics of school choice have suggested that small classes in private schools “explain” the achievement benefits of private-school scholarships and voucher programs. If public schools had the additional resources necessary to reduce class size, so the thinking goes, they too would raise student performance. This, of course, begs the question: Why are private schools, despite having far fewer resources, able to provide significantly smaller classes than public schools? And what assurance is there that additional funds for public schools will lead to reduced class sizes-and not to higher salaries or more non-teaching staff?

Nevertheless, class size, when factored into my analysis, was not significantly related to student achievement. In other words, class size did not “explain” the achievement benefits of receiving a scholarship to attend private school in Charlotte.

There was little evidence to suggest that private schools were creaming the best students or dumping the worst.


Creaming and Dumping


Another oft-heard excuse for the success of private schools is that they are able to select their students by accepting the finest students (or creaming) and dumping the undesirable ones.
In my sample, there was little evidence suggesting that private schools were creaming the best students or dumping the worst. First, hardly any private schools asked applicants to take an admissions test. Of those families who were unable to place their children in the schools of their choice, more than three-fifths cited as the main obstacle the financial constraint of having to bridge the gap between the $1,700 scholarship and the full tuition. Parental reports indicate that an admissions test prevented only two of the lottery winners from attending their chosen private school.

Second, there is no evidence that private schools expelled undesirable students or asked them not to return. Any choice parents whose children switched schools during the academic year were asked why they left their original private schools. None reported that they switched schools because their children were expelled. Of those parents who were in doubt as to whether they would return to the same school next year, none reported that their children were asked not to return. In short, there is virtually no evidence that the choice schools academically screened their students for admission or expelled-or “counseled out”-students they found undesirable.

Parents were also asked whether their children had any physical handicaps, learning disabilities, or difficulty learning to speak English. Very few reported physical handicaps-only 3 percent of choice parents and 2 percent of public-school parents. Similarly low percentages of choice and public-school parents reported that English was not their children’s native language. However, choice parents reported fewer children with learning disabilities (4 percent) than public-school parents (13 percent) reported. Given their lower levels of funding, fewer private schools offer special programs for learning disabilities. The disparity also may be explained by differing incentives to label children “learning disabled.” Public schools receive additional resources for students labeled learning disabled, and they may be able to exempt learning disabled students from accountability testing.

Despite having less money for salaries and benefits, private schools appear to be better able to recruit quality teachers and to dismiss poor ones.

The higher share of learning disabled children in public schools also may be evidence of mere parental choice. Parents of children with special needs are more likely to choose schools that have the funds to offer special programs. A fair test of whether private schools are avoiding learning disabled students would compare the rates of learning disabilities when private schools are given the same additional resources to serve those children that public schools receive. Otherwise, the evidence on learning disability rates is ambiguous.

In general, students who used their scholarships to enroll in a private school were more likely to be minority and considerably less advantaged than the typical Charlotte student. Three-quarters of the choice students were African-American, compared with a little more than a third of all students in the Charlotte-Mecklenburg school district. As of 1990, the average family income in Charlotte was nearly $34,000, almost $10,000 more than the average family income of choice students ten years later. In 1999, 32 percent of choice families reported receiving some kind of public assistance, such as food stamps or welfare; only 5 percent of Charlotte households were on public assistance in 1990. Even after their first year in private school, choice students were still scoring well below the national average on standardized tests (although they were scoring significantly better than they would have had they not received a scholarship). It is clear that private schools are accepting the challenge of educating these disadvantaged students, not creaming off the best or dumping the worst.

 

The Next Level


It is important to understand that the privately funded scholarship program in Charlotte departs from a model, publicly funded, school choice program in a number of ways. First, the $1,700 scholarship always required a significant supplement from the family to meet the tuition expense. A voucher that is set at the level of per-pupil spending in the local school district is likely to cover the full cost of tuition at most private schools. This may alter the results we would expect to see from gaining access to private schools. The infusion of cash that a publicly funded voucher would provide to private schools might make them even more effective. But easing the financial burdens that families face might attract more lower-income and less-motivated parents who might act as a drag on test-score improvements.

Second, privately funded scholarships place little or no regulation on the activities of private schools. By contrast, publicly funded vouchers are likely to carry a variety of curbs that might distract private schools from their core mission and tie up resources that would otherwise have gone to instruction. In the end, however, the regulations might also improve the program by ensuring equal access and by requiring that schools provide accurate information on student performance and faculty qualifications.

Third, there was enough spare capacity in existing private schools to accommodate the small number of CSF scholarship winners. A larger, publicly funded school choice program would require the creation of new private schools. These schools might be better or worse than existing private schools.

There is no way to address these issues fully without expanding the number and scale of publicly funded voucher programs. There is now a body of evidence on the effects of school choice: the positive findings from Charlotte comport with the positive results of privately funded programs in New York; Washington, D.C.; and Dayton, as well as pilot voucher programs in Milwaukee and Cleveland. It is not yet clear whether these results will translate to a broader scale. But the existing evidence is encouraging enough to justify trying.

-Jay P. Greene is a senior fellow at the Manhattan Institute for Policy Research. To view his study in its entirety, log on to www.edmattersmore.org.

The post Vouchers in Charlotte appeared first on Education Next.

]]>
49696050
Significant, but Not Decisive https://www.educationnext.org/significant-but-not-decisive/ Thu, 20 Jul 2006 00:00:00 +0000 http://www.educationnext.org/significant-but-not-decisive/ A critique

The post Significant, but Not Decisive appeared first on Education Next.

]]>
Photography by Ericka McConnell

Research has yielded relatively few definitive findings to guide policymakers on what educational interventions might enhance student achievement. The studies of privately financed voucher programs in Dayton, Ohio; New York City; Washington, D.C.; and Charlotte, North Carolina, however, suggest that private schooling leads to significant improvements in achievement for some students. This finding is potentially quite important, given the continuing debate over public funding of voucher programs.

The survey data the researchers gathered show that, in general, voucher recipients were more satisfied than their public-school counterparts with a variety of conditions (such as safety, teacher quality, and so forth) at their chosen schools. This finding is consistent with the literature on school satisfaction. It is not clear, however, what conclusions one should draw from parents’ perceptions. They may represent mere “buy-in effects”: the idea that simply being given a choice heightens student and parental satisfaction. Greater satisfaction with a school certainly should be regarded as a positive outcome; however, we cannot know whether these findings reflect genuine differences in school quality or simply the satisfaction of being granted options. Given this ambiguity, this review focuses on reported results on student achievement.

The effects of private schooling, as reflected in achievement data, varied across studies, grades, and racial and ethnic groups. Positive effects were consistently found for African-American students who attended private schools for at least two years. The average estimated effect for African-American students in the Dayton, New York City, and Washington programs of attending a private school rather than a public school was more than 6 percentile points in both mathematics and reading after two years. In the Charlotte program, where effects were not reported by race, the private-school effect was estimated to be an increase of approximately 6 percentile points after one year.

The very fact that parents use a voucher and supplement it with their own funds could reveal a home environment that is conducive to academic achievement. (Photography by Ericka McConnell)

These gains are quite large by social science standards. For instance, the largest effects were found in Washington, D.C., where African-American students in grades 2-5 gained 10 percentile points in mathematics and 8.6 percentile points in reading after two years of private schooling. This represents a gain of about 0.5 standard deviation relative to African-American students whose applications for vouchers were unsuccessful (the control group, who remained in public schools). By comparison, a 1999 study by Jeremy Finn and Charles Achilles estimated that the well-known STAR class-size reduction program in Tennessee increased minority students’ 2nd-grade reading achievement by 0.33 standard deviation. This was with reducing class sizes by about eight students (from a class of 22-26 students to one with 13-17 students), a relatively expensive intervention. Thus, the estimates from the voucher studies reviewed here suggest that even modestly sized vouchers, which may be considerably less expensive than the marginal cost of educating students in the public sector, could provide substantial benefits for some students. This interpretation of the findings, however, deserves several cautions.

Potential for Bias

In assessments of program effects in nonexperimental settings, one concern is the potential for omitting important individual characteristics from the analysis. Many individual characteristics that influence student achievement, such as a student’s motivation, are difficult to quantify. Private- and public-school students and parents may differ from one another in both observable and unobservable ways. For instance, the very fact that parents use a voucher (and supplement it in the case of the privately funded scholarships studied here) could reveal a home environment that is conducive to academic achievement. These types of effects are difficult to measure. If unobservable variables such as parents’ commitment to education come into play, their influence may be wrongly identified as the effects of better schooling.

The strength of the studies reviewed here is that they take advantage of the fact that a lottery was used to allocate spots in these oversubscribed voucher programs. This allowed the researchers to compare the outcomes of students who were randomly assigned to either public (the control group) or private (the treatment group) schools. This strategy has intuitive appeal, is easy to understand, and is used in other contexts, such as medical research. Ideally, random assignment should mitigate concerns about the effects of unobservable characteristics. In fact, it should eliminate the need to control for any background characteristics, observable or unobservable. Of course, ideal conditions rarely, if ever, exist. Small deviations from the ideal are unlikely to result in biased findings. Larger deviations, however, can bias the results of even a carefully designed social experiment, possibly resulting in misleading conclusions.

Several findings are somewhat troubling in that they raise concerns about the quality of the control and treatment groups. First, the magnitude of the gains due to private schooling is substantially larger than that typically found in nonexperimental studies of differences between public and private schools, even those studies focusing on minority students. Second, the finding of no statistically significant gains for non-African-American students is surprising. It is not at all clear what accounts for private schools’ disproportionately benefiting a particular group of students. Finally, in some cases the estimated effects exhibited rather large swings from year one to year two. In Washington, D.C., African- American students in private schools declined by 9 percentile points in reading relative to their public school peers after the first year. The next year saw them climb to 8 percentile points above their public school peers in reading. This represented a net positive swing of 17 percentile points from one year to the next. An additional year of private schooling, in other words, is estimated to produce a staggering gain of about 0.9 standard deviation.

What might account for these anomalies? Of course, the quality of the results depends crucially on the integrity of the experiments. Differences in responses between the treatment and control groups or nonrandom placement into either group can contaminate the results.

The authors’ demographic data suggest that lottery winners and losers had similar background characteristics (although there were some statistically significant differences in Charlotte). However, many students who “won” vouchers in the random lottery chose not to use them. In Washington, D.C., 47 percent of lottery winners did not use their vouchers; in Dayton, 46 percent; and in New York, 24 percent. Voucher users and decliners may have appeared to be similar along observable lines, but may have differed in their unobservable characteristics. In fact, it is quite plausible that those parents who declined a voucher did so at least in part because they perceived that sending their children to private schools would not necessarily have been beneficial based on the specific public and private options available in their neighborhoods.

The authors use a statistical technique (known as “instrumental variables”) that explicitly accounts for potential differences, observable or unobservable, between those who use and those who decline vouchers. As a result, the findings are not likely to suffer from this potential source of bias. However, this technique cannot account for the potential bias associated with “response attrition”-that is, the potential that there are systematic differences between those who participated in the testing and survey sessions and those who did not. Some cities, such as Charlotte, encountered a fairly low participation rate in the testing sessions. The overall response rate in Charlotte was 40 percent, and only 20 percent of lottery winners who declined their vouchers participated in the study.

Similar concerns apply to attrition out of the sample over time. If students who leave the treatment sample (lottery winners) differ in important ways from students who leave the control sample (lottery losers), the results may be biased. For instance, if the students who left the treatment group tended to be of lower ability than those who left the control sample, the experiment would overstate the effect of private schooling. Given the sizable year-to-year drops in the sample sizes in all four cities, nonrandom attrition may be an issue. The attrition rates were generally similar between the treatment and control groups, and the two groups remained similar along observable dimensions (and the researchers used weights to adjust for differences in observed characteristics). Nevertheless, the concern remains that attrition over time resulted in unobservable differences between the two groups.

Findings from the similarly structured but publicly financed voucher program in Milwaukee show the potential impact of contaminating the treatment and control groups. Researchers studying this program using methodologies similar to those used in the studies reviewed here estimated relatively large effects for students who attended private schools: 6 percentile points in reading and 7 percentile points in mathematics after three years. By contrast, Princeton economist Cecilia Rouse, using a methodology that accounts for the possibility of unobservable characteristics such as greater motivation among students and parents who remained in private schools, found no statistically significant, positive effects of private schools in reading (but did find similar private-school effects in math of 1 to 2 percentage points per year).

There is no direct evidence that the results reported by William Howell et al. and Jay Greene are biased. If data collection continues on these voucher programs, it will be possible to use statistical techniques to account for the potential of bias. Regardless, if these findings accurately assess the effects of private schooling, they certainly lend credence to arguments in support of vouchers. Given this possibility, it is important to consider whether these gains can be replicated on a larger scale.

Photography by Ericka McConnell

To the Next Level?

Findings from relatively small, experimental programs may not generalize to a larger scale for several reasons, a point the authors acknowledge. The evaluations did not adjust for the demographics or ability of the non-voucher students attending private schools. What is thought to be the effect of private schooling may actually be the influence on voucher users of having peers of higher ability, better behavior, and so forth. To the degree that peers influence educational outcomes, we might expect to find positive effects of private schooling even if the private schools themselves are no better than public schools. An expansion of voucher programs could diminish the positive effects of peers if the flow of students caused the demographics of private schools to look more and more like those of public schools.

Only those students and parents who expressed an interest in attending private schools had a chance to receive a voucher. The effects of private schooling on students who want to attend private schools do not necessarily predict the experiences of the general population. A similar point applies to the schools that accepted vouchers. The quality of both public and private schools varies considerably, but the authors don’t know whether they compared average public schools with average private schools. One might assume that students attending ineffective public schools are more likely to apply for a voucher. One might also argue that only the least effective private schools would have room for additional students. Without knowing what kinds of schools we’re comparing, it is not at all clear that we should expect similar results from an expanded voucher program.

Too few seats exist currently in private schools to accommodate a large migration of students into the private sector. An increase in the demand for private schooling associated with the offering of more vouchers might be expected to stimulate the expansion of existing private schools or the building of new schools, but of what quality? Economic theory suggests that the most efficient suppliers are likely to be in the marketplace already. Thus, the quality of new private schools is likely to be lower than that of existing ones. Some evidence suggests that competition from private schools makes public schools more efficient, but our knowledge of how public schools would respond to the competitive threat of losing students under a voucher system is limited. The evidence presented by Howell et al. and Greene is not sufficient to be considered decisive in the policy debate over vouchers. Nonetheless, it is encouraging with regard to students who use vouchers, particularly minority students, and much more may be learned as additional data become available.

Dan Goldhaber is a senior research associate at the Urban Institute.

The post Significant, but Not Decisive appeared first on Education Next.

]]>
49696056