Catholic Schools: Private and Social Effects
Kluwer Academic Publishers, 2000, $100; 160 pages
By William Sander

The Education Gap: Vouchers and Urban Schools
Brookings Institution, 2002, $28.95; 275 pages
By William G. Howell and Paul Peterson, with Patrick J. Wolf and David E. Campbell

As reviewed by R. Kenneth Godwin

The advantage of reading The Education Gap and Catholic Schools together is in being able to appreciate their use of diverging research strategies. At the heart of The Education Gap is a large-scale study of privately funded school-voucher systems in three cities, New York, Washington, D.C., and Dayton, Ohio. The authors, Paul Peterson [editor-in-chief of Education Next] and William Howell, mounted a randomized field trial like those used in medicine in order to test the effects on achievement of being given a voucher to attend private school. Catholic Schools, by contrast, uses a nonexperimental approach in studying the influence of Catholic schools. The strengths of the experimental approach compensate for the weaknesses of the nonexperimental approach and vice versa. When both types of studies yield similar conclusions, the results inspire greater confidence.
In The Education Gap, Howell and Peterson call randomized field trials the “gold standard” of social-science research. Randomized trials, built on the model of medical experiments, allow researchers to estimate the effects of a policy change by randomly sorting individuals into two comparable groups—in the case of school vouchers, a test group that receives vouchers and a control group that doesn’t. The random allocation ensures that there are no systematic differences—such as income, achievement levels, or parental involvement—between the two groups that might influence the results. Despite their advantages, randomized trials have weaknesses, the most basic being that they do not tell the researcher whether the results may be generalized to other situations, to nonvolunteers, or how the treatment variable will interact with other policy-relevant variables.

Randomized field trials are so expensive that they often must be limited to only a subset of the population of interest. For example, the study might include only a few locations and only a few grades. Even if the outcomes of the treatment and control groups differ significantly, we cannot be sure that the results will generalize to other locations and to other grades. If the experiment was limited to a single income group, we cannot know if the treatment would work equally well for other income groups. Despite these limitations, however, randomized field trials are an extremely useful type of evaluation, and they have an exceptional level of internal validity. We generally can be more confident in conclusions reached by randomized field trials than those produced by nonexperimental research as long as we do not generalize beyond the subgroups studied.

Most social-science research cannot use randomized field trials because of their expense and because it is rarely feasible to assign a sample population to treatment and control groups. Social-science researchers therefore typically observe relationships in nonexperimental settings and attempt to adjust statistically for all the relevant variables. For example, if we are interested in whether private school students and public school students have different outcomes, we try to obtain good measures for all other variables that influence the outcomes and then use regression analysis to discover whether private school students did better than the public school students. In Catholic Schools: Private and Social Effects, William Sander uses this method to study the effects of attending a Catholic school on various academic and nonacademic outcomes.

Nonexperimental research in education has two important methodological problems to overcome: omitted variables and selection bias. Selection bias is a special case of the omitted variable problem. Assume for a moment that two families live next door to each other. The parents have the same occupations, educations, and incomes, and the children have similar abilities. One family pays private school tuition, while the other enrolls its children in public schools. If we observe that the children in the private school learn more, can we conclude that private schools are better? No, because the families obviously have different values and goals. It is likely that the parents paying tuition have other characteristics that encourage their children to value education highly and to work hard in school. Statistical comparisons of public and private school outcomes must find a way to account for these unobserved, but very important, characteristics.

Randomized field trials essentially eliminate the problem of selection bias. At the same time, nonexperimental research can use large sample sizes to test the effects of Catholic schools in a wide variety of situations and to test for interactions between attending a private school and such variables as race, class, religion, and ability level. That these diverging research strategies have common findings lends them serious credibility. The most important common finding was that attending a private school significantly improves the education outcomes of African-American children in the inner cities but not of other students.

A second common finding is that instruction at a religious school has a positive effect on the religiosity of students. A major reason that parents choose to send their children to religious schools is to obtain religious instruction. The Education Gap finds that such instruction increases the religious observance of students. Catholic Schools shows that this increase lasts into adulthood and affects religious behaviors such as the frequency of prayer and the contributions one makes to the church.

Because all of the randomized experiments occurred in urban areas, a question that could not be answered by the Howell and Peterson study is, “Do private schools make a difference in rural areas?” Catholic Schools tells us that they do not. Similarly, because the experiments in The Education Gap included only low-income students, we do not know if children from higher income families who attend private schools learn more than higher income students who attend public schools. Again, Catholic Schools says no.

The Education Gap provides important information concerning the effects of vouchers on political tolerance. Opponents of expanded school choice fear that allowing families to choose private schools, particularly fundamentalist schools, will encourage intolerance among children. Although previous nonexperimental research has found that students in private schools showed greater tolerance and concern for the less fortunate than did public school students, these results may have been caused by selection bias. That the randomized field trials found similar relationships gives us much greater confidence that private schools do not promote intolerance.

Given the weight that the black-white test-score gap carries in the American psyche, the advantage that Catholic schools give black children has substantial policy implications. The most immediate of these is that providing vouchers to low-income children in the inner cities appears to be a cost-effective tool for improving their education outcomes. Such a policy would increase educational equity across racial and ethnic groups. A second implication is that voucher policies that include sectarian schools will not reduce political tolerance in the United States and may increase feelings of civic duty and political participation. This should alleviate the fears expressed by such prominent philosophers of education as Amy Gutmann of Princeton and Eamonn Callan of Stanford. It doesn’t appear that vouchers would reduce the quality of democracy in the United States.

R. Kenneth Godwin is a professor of political science at the University of North Carolina at Charlotte and coauthor, with Frank R. Kemerer, of School Choice Tradeoffs: Liberty, Equity, and Diversity (2002).

The Education Gap: Vouchers and Urban Schools

By William G. Howell and Paul Peterson, with Patrick J. Wolf and David E. Campbell

As reviewed by John E. Coons

The “gap” that serves as The Education Gap‘s leitmotif is a catchall for educational inequalities among demographically distinct groups—racial, ethnic, and economic. The scene is painfully familiar, with the “haves,” well-off whites, receiving resources and opportunities not available to the “have-nots.” With little distortion (and some convenience) I will refer to these two groups respectively as the Lucky and the Unlucky. The hypothetical gaps between them lurk in an array of particular inputs and outputs of schooling. On the input side are certain personal and family attributes plus all those features of schools—skilled teachers, computers, updated facilities—that money now buys directly and that those without money could, instead, buy through school vouchers. On the output side are test scores and the various behaviors and attitudes of parents and children who enjoy (or do not) a subsidized choice.

The measured effects of choice on the relative positions of the Lucky and the Unlucky can be surprising and even paradoxical. For example, though today there is no systematic gap between the Lucky and Unlucky in per-pupil expenditures, the existing systems of vouchers, in cities like Milwaukee and Cleveland, automatically create one by setting the value of the voucher lower than the cost of educating a child in the public schools. Now the paradox: at least for black students, the authors of The Education Gap find, this decline in financial support is associated with higher test scores; the notorious gap between white and black students is diminished. Nor does the decline in dollars injure the test scores of whites and Hispanics who also choose the private sector.
The book takes the measure of other alleged gaps, bringing me to my principal criticism. Statistics are always about something that is selected before the counting begins. The Education Gap sometimes gives the impression that it has identified everything still needing to be counted, and, indeed, it does suggest various sensible projects. But these appear to have been selected—and the project thereby limited—largely in response to criticisms from opponents of choice. When Professor X complains that choice for the poor will skim the “best and the brightest,” the authors answer that it did not do so in the programs studied. Fine, but might we also wonder whether, over time, students switching from public to private schools could have a “pioneer” effect, alerting the more apathetic families to their opportunity for responsibility and autonomy?

Or, when Professor Y worries that vouchers will destroy the unity of the curriculum and thereby diminish our “civic” focus, the authors rightly report the superior record of private schools on measures of tolerance. What they fail to question is their opponent’s premise of an existing civic curriculum. Others observe that contradictory moral and civic theories flourish among public schools, making coercive assignment of the poor an intellectual lottery. If they are right, any civic criticism of choice is at best incoherent. If The Education Gap‘s substance and tone throughout is relentlessly fair to its ideological opponents, at some points it seems modest to a fault.

This forensic constriction of the authors’ agenda is illustrated more broadly in its dogged focus on test scores and the empirics of school life. Their questionnaires for parents inquire mainly about their experiences in the chosen school yet only skirt the possibility of deep effects on the family itself: “How often did you . . . help this child with . . . homework; . . . talk . . . about experiences at school; attend school activities; work on school projects?” And “How many parent-teacher conferences . . . . How many hours . . . volunteered . . . . Are you a member of the PTA,” and are you “satisfied” with this school and why?

I would not discourage such inquiries, as long as they do not scant the larger hypothesis that “the problem” to which choice may be an answer lies in the unlucky family itself. The middle-class parent who enjoys choice can function as a full and responsible human being; the child senses this and remains consciously integrated in this little platoon of potent and important people. By contrast, conscriptive school assignment strips the unlucky family of its dignity and power, inviting the parent to accept the passive role of the permanent loser; the child grasps this impotence as the deadly threat of his or her own isolation and vulnerability. Before all else the extension of school choice is an instrument of family policy, as Lyndon Johnson and Patrick Moynihan understood and as free-market theorists tend to forget.

Nearly as fateful is the question of the child’s own autonomy. If freedom is a value of childhood, is it imperiled by our subsidizing parents? Some “liberals” say as much, but all we really know in empirical terms is that small children are inevitably dominated by particular adults. The middle class nevertheless has supposed that such inescapable adult sovereignty can liberate when exercised by the family; autonomy is the product of love harnessed to the parent’s practical capacity to act responsibly over time. I hear that teenagers in the suburbs influence the family’s choice of school. Would it be any different within the unlucky family? The Education Gap tells us only that the effect of parental choice on student self-confidence “appears to be moderately positive.” I should think so. And the unluckier the family, the greater its rejuvenation.

“Family effects” of this sort could explain the grand phenomenon that so puzzles the authors: “How greater choice translates into achievement gains for African-American students remains unknown.” But which among our unlucky families presently have the least coherence simply as families? We know that Hispanics and Asians retain a surprising integrity; for them the pretensions of conscriptive state schools may be simply a huge annoyance and little threat to family identity. But, as Moynihan warned us long ago, the “Negro Family” verges on prostration for want of serious mission. Before all other therapies, it cries out for a devolution of responsibility—for real tasks that are important, dignified, and civic. What could be better for demoralized parents and children than to acquire the last word on school assignment? Perhaps it is deep inside the family that we should look to explain these happy effects of experiments with school choice—higher test scores included.

Chesterton wrote of his friend and opponent Bernard Shaw that he “is like the Venus of Milo; all that there is of him is admirable.” One could say the same of The Education Gap with this twist: what is well presented of the “statue” of choice are her amiable extremities—arms from an otherwise missing figure. It is time to seek the body, the head—and the heart.

John E. Coons is a professor of law at the University of California-Berkeley and coauthor, with Patrick Brennan, of By Nature Equal: The Anatomy of a Western Insight (1999).