Supporters say "Competition will Improve the Public Schools"

Summary

There is no evidence to support for the assertion by New Hampshire voucher supporters that competition from voucher supported schools improves the public schools. 

They cite "the study done by Florida's own Department of Education" in support of their assertion.  It was not done by the Florida DOE.  And it shows only very modest gains in a specific Florida voucher program that, the author says, is not comparable to the proposed New Hampshire voucher program.  Our review of the study is here.  The respected National Education Policy Center review is here (They gave the Friedman Foundation, prolific producer of bogus studies, the "Bunkum Award" for a series of studies on competition and other voucher issues.).  They review another competition effect study here, also finding that it did not make its case.

That should put the issue to bed.  However, additional material is presented below as well.

----------------------------------
Detail

Diane Ravitch puts the issue to rest
But Diane Ravitch, the nation's leading historian of American public education and a sharp critic of our public schools, sums it up simply on P. 132 of her recent book (here is an excerpt from the book), Death and Life of the Great American School System:

"In sum, twenty years after the initiation of vouchers in l\lilwaukee
and a decade after the program's expansion to include religious
schools, there was no evidence of dramatic improvement for the
neediest students or the public schools they left behind."

Her book provides as much detail behind that summary as anyone could want.  Here's a sample (emphasis added):

P. 129

"By 2009. studies hy different authors came to similar conclusions
about vouchers, suggesting an emerging consensus. Cecelia
E. House of Princeton University and Lisa Barrow of the Federal
Reserve Bank of Chicago published a review of all the existing
studies of vouchers in Milwaukee. Cleveland. and the District of
Columbia. They found that there were "relatively small achievement
gains for students offered educational vouchers. most of which are
not statistically different from zero."
...............
"In 2009. the same research learn released another study that
found no major differences between students in voucher schools
and those in regular public schools. The research group included
the strongly pro-voucher Jay P. Greene of the University of Arkansas
and John Witte, who was considered a critic of vouchers. The
researchers found "no overall statistically significant differences
between MPCP [voucher] and MPS [Milwaukee Public Schools] student achievement growth in either math or reading one year after
they were carefully matched to each other." Perhaps there would be
different outcome's in the future. but this was not the panacea that
voucher supporters had promised and hoped for.
.....

The first evaluation [of the Washington, D.C. voucher program] in 2008
reported that in the first two years of the program (2004 and 2005).
there was no statistically significant difference in test scores of reading
and math between students who won the lottery and those who
entered the lottery but did not win. However. the third-year evaluation
of the voucher program (released in 20(9) found that there was
"a statistically significant positive impact on reading test scores. but
not math test scores." The reading scores represented a gain of more
than three months of learning"

Supporters of vouchers were ecstatic about the third·year evaluation
because at last they had hard evidence that vouchers would
benefit students. They glossed over the finding that these gains \vere
limited to certain groups of students. The students who experienced
gains in reading were those who entered the program from schools
that were not in need of improvement, those who entered the program
in the upper two-thirds of the test score distribution. and those
who entered in grades K-8. Females also seemed to benefit, though
that finding was not as robust as the others. The groups that did
not experience improvement in reading (or math) were boys, secondary
students. students from SINI [Schools in Need of Improvement] schools. and sludents in the lowest third of the test score distribution. The students who did not
see any gains were those in the highest-priority groups. the ones for
whom the program was designed: those with the lowest test scores
and those who had previously attended SINI schools."

What does this "competition" argument really mean?
Parents often make the case that that the public schools already are under heavy competitive pressure from private schools and from accountability testing.  Voucher advocates respond that it is the existence of the voucher/ETC program itself that induces the competitive pressure.  This is what Rep. Will Smith (R-New Castle) said in a meeting with New Castle parents on January 24, 2011.

Rep. Smith's point makes clear, then, that advocates on both sides of the issue see it the same way:  When the state establishes a voucher program, it is saying to the public schools, "We have lost faith in your ability to educate our children and will now consider private, religious and home schools to be educational alternatives on a par with public schools.  You will now have to compete for students and money."

Therefore, embedded in the competition argument is the admission that the voucher program signals a disinvestment in the public schools.  This understanding is further supported by what the advocates themselves say.  Here are examples

After all is said and done, however, there does not appear to be any useful evidence that voucher programs do lead to improvement in the public schools.  Collected below are articles on the competition-benefits-the-public-schools theme.

Duke University's Center for Child and Family Policy is national leader in this kind of research.  They looked at the question and concluded that the grading system used by both Florida and North Carolina, in which every school is given at grade from A to F every year, drives positive change in the low rated schools.  The grading system had the same impact in Florida, which has voucher/ETC programs, and North Carolina, which does not.

So it's the grading system not the vouchers that makes the difference.  Nonetheless, here is a detailed discussion of "the Florida study" that advocates always refer to.

The Competitive Effect of School Choice Policies on Performance in Traditional Public Schools, NEPC, 3/08

posted Feb 15, 2012, 1:12 PM by Bill Duncan   [ updated Feb 15, 2012, 1:14 PM ]

NEPC

Executive Summary

This policy brief reviews research on what impact the competition
introduced by vouchers and charter schools has upon the effectiveness and
efficiency of traditional public schools (TPSs). Only recently has such
research been possible in the U.S., as choice options became sufficiently
widespread to elicit competitive responses from TPSs. We summarize
conflicting theoretical predictions about how competition affects students
who do not actively choose, and we identify features of policy design,
implementation and local settings likely to influence the nature of
competition. We find that results from available empirical studies are
mixed and do not yet allow for firm conclusions about the effects of
competition on traditional schools and non-choosing students. The review
notes methodological challenges and possible lines of future research.

We recommend that policymakers exercise caution when assessing
predictions that school choice policies will benefit students who are not
active choosers, since the evidence in support of this claim is not yet
strong or conclusive.

Review of A Win-Win Solution: The Empirical Evidence on How Vouchers Affect Public Schools, Nat. Ed. Policy Center

posted Feb 10, 2012, 3:20 AM by Bill Duncan   [ updated Feb 10, 2012, 3:22 AM ]

Link

This and the other work from the National Education Policy Center seems to be very strong and balanced, with very informative reviews

In this case it reaches a clear conclusion that the author of the study reviewed did prove his point that competition works.



Review of A Win-Win Solution: The Empirical Evidence on How Vouchers Affect Public Schools, Nat. Ed. Policy Center


A Win-Win Solution: The Empirical Evidence on How Vouchers Affect Public SchoolsAdvocates of vouchers argue that nearby public schools will be forced to compete for students, leading to improvements for voucher users and non-users alike. Critics worry that the students who use vouchers to leave public schools will have parents with higher levels of education and be less expensive to educate, and that losing these students will cause those schools to enter spirals of decline. This new report purports to gather all available empirical evidence on the question of the competitive effects of vouchers, finding a strong consensus that vouchers help public schools. But the report, based on a review of 17 studies, selectively reads the evidence in some of those studies, the majority of which were produced by voucher advocacy organizations. Moreover, the report can’t decide whether or not to acknowledge the impact of factors other than vouchers on public schools. It attempts to show that public school gains were caused by the presence of vouchers alone, but then argues that the lack of overall gains for districts with vouchers should be ignored because too many other factors are at play. In truth, existing research provides little reliable information about the competitive effects of vouchers, and this report does little to help answer the question.

Diane Ravitch on Competition and Finland's Success

posted Feb 1, 2012, 2:02 AM by Bill Duncan   [ updated Feb 1, 2012, 2:04 AM ]

(from the speech attached)

We know—or we should know—that poor and minority children should not have to depend on the good will and beneficence of the private sector to get a good education. The free market works very well in producing goods and services, but it works through competition. In competition, the weakest fall behind. The market does not produce equity. In the free market, there are a few winners and a lot of losers. Some corporate reformers today advocate that schools should be run like a stock portfolio: Keep the winners and sell the losers. Close schools where the students have low scores and open new ones. But this doesn’t help the students who are struggling. No student learns better because his school was closed; closing schools does not reduce the achievement gap. Poor kids get bounced from school to school. No one wants the ones with low scores because they threaten the reputation and survival of the school.

.....

In September, I visited Finland and I want to share with you what this tiny nation has accomplished. It regularly scores at the top of international tests in reading, mathematics, and science. It has the least variance from school to school, meaning that almost every school is a good school. Students in Finland never take a standardized test until they complete high school. Teachers in Finland are required to have a master’s degree. 

Teaching is a highly respected profession. Parents trust teachers. Teachers have autonomy to exercise professionalism.

Every child has regular medical checkups and healthcare, at no cost. Schools have health clinics. Whereas more than 20% of our children live in poverty, less than 4% of Finnish children do. Higher education is tuition-free.

Finland has no charter schools, no vouchers, no merit pay, no standardized testing. Instead, every teacher is trained to take care of the needs of individual children. If children are having learning problems, there are specialists and social workers in every school to take care of them early and provide whatever assistance is needed. Nearly half of all Finnish students get extra attention and services in the early years of schooling. Finland has no tracking. All children get the education and support they need to succeed in school. Finland does not have a longer school day or a longer school year. Finnish schools emphasize creativity, ingenuity, problem-solving, the arts, projects, activities, physical education, and risk-taking.

By the way, Finnish teachers and principals belong to the same union. It doesn’t seem to be a problem.

Death and Life of the Great American School System, Diane Ravitch, November, 2011

posted Jan 31, 2012, 7:51 AM by Bill Duncan

Diane Ravitch puts the issue to rest
But Diane Ravitch, the nation's leading historian of American public education and a sharp critic of our public schools, sums it up simply on P. 132 of her recent book, Death and Life of the Great American School System (excerpt attached):

"In sum, twenty years after the initiation of vouchers in l\lilwaukee
and a decade after the program's expansion to include religious
schools, there was no evidence of dramatic improvement for the
neediest students or the public schools they left behind."

Her book provides as much detail behind that summary as anyone could want.  Here's a sample (emphasis added):

P. 129

"By 2009. studies hy different authors came to similar conclusions
about vouchers, suggesting an emerging consensus. Cecelia
E. House of Princeton University and Lisa Barrow of the Federal
Reserve Bank of Chicago published a review of all the existing
studies of vouchers in Milwaukee. Cleveland. and the District of
Columbia. They found that there were "relatively small achievement
gains for students offered educational vouchers. most of which are
not statistically different from zero."
...............
"In 2009. the same research learn released another study that
found no major differences between students in voucher schools
and those in regular public schools. The research group included
the strongly pro-voucher Jay P. Greene of the University of Arkansas
and John Witte, who was considered a critic of vouchers. The
researchers found "no overall statistically significant differences
between MPCP [voucher] and MPS [Milwaukee Public Schools] student achievement growth in either math or reading one year after
they were carefully matched to each other." Perhaps there would be
different outcome's in the future. but this was not the panacea that
voucher supporters had promised and hoped for.
.....

The first evaluation [of the Washington, D.C. voucher program] in 2008
reported that in the first two years of the program (2004 and 2005).
there was no statistically significant difference in test scores of reading
and math between students who won the lottery and those who
entered the lottery but did not win. However. the third-year evaluation
of the voucher program (released in 20(9) found that there was
"a statistically significant positive impact on reading test scores. but
not math test scores." The reading scores represented a gain of more
than three months of learning"

Supporters of vouchers were ecstatic about the third·year evaluation
because at last they had hard evidence that vouchers would
benefit students. They glossed over the finding that these gains \vere
limited to certain groups of students. The students who experienced
gains in reading were those who entered the program from schools
that were not in need of improvement, those who entered the program
in the upper two-thirds of the test score distribution. and those
who entered in grades K-8. Females also seemed to benefit, though
that finding was not as robust as the others. The groups that did
not experience improvement in reading (or math) were boys, secondary
students. students from SINI [Schools in Need of Improvement] schools. and sludents in the lowest third of the test score distribution. The students who did not
see any gains were those in the highest-priority groups. the ones for
whom the program was designed: those with the lowest test scores
and those who had previously attended SINI schools."

Shaking Up Public Schools With Competition, The School Administrator, Arsen and NI,8/11

posted Jan 29, 2012, 4:28 AM by Bill Duncan   [ updated Jan 29, 2012, 4:44 AM ]

Link

(This analysis is about competition from charter schools, not vouchers, but is offered here, because it avoids the advocate hype of most analyses.  It also makes the complexities of studying this issue clear, whether the comparison is charters or vouchers. 

Highlighted emphasis is added.  They summarize their conclusion, highlighted: "the initial results offer little evidence that charter schools significantly improve or worsen student outcomes in traditional public schools.")


The School AdministratorAugust 2011 Number 7, Vol. 68| The Charter Movement|16-19
Feature
Shaking Up Public Schools With CompetitionWhat does evidence say about the effects of charter schools on improving outcomes for nonchoosers and their schools?

by DAVID D. ARSEN AND YONGMEI NI

The proposition that competition will spur public schools to perform better is now familiar to most Americans and self-evident to many. It is a compelling idea.

In his acceptance speech before the Republican National Convention in August 2008, presidential nominee John McCain brought the delegates to their feet with this pronouncement:

“Education is the civil rights issue of this century. Equal access to public education has been gained. But what is the value of access to a failing school? We need to shake up failed school bureaucracies with competition. Empower parents with choice.”

It is but one measure of the charter movement’s success that both major parties have long supported an expansion of charter schools.

Detroit Public Schools offers one extreme case of a school district shaken up by charter competition. More than a third of Detroit students attend charter schools. Under Michigan’s state-controlled school finance system, the loss of resident students to charter schools and suburban districts through interdistrict choice has produced an annual revenue loss for Detroit Public Schools of roughly half a billion dollars.

While public schools in Detroit confront many profound problems besides school-choice competition, this revenue loss contributed to a crippling and sustained financial deficit. In response, the governor appointed an emergency financial manager to run the district in 2009.

The Detroit district has closed 149, or more than half, of its school buildings since 2001. The city is an extraordinarily turbulent education setting, with students and staff moving from building to building. In March, the school district’s emergency financial manager solicited bids from private management companies to operate a third of the district’s remaining schools as charter schools starting this fall. When the district failed to receive enough promising charter proposals, it abruptly changed course.
David Arsen and Yongmei Ni contributed research on the competitive effects of charter schools on traditional public schools in a 2010 Harvard Education Press title.

In June, Michigan Gov. Rick Snyder and Detroit’s new emergency schools manager, Roy Roberts, announced the formation of an administrative authority that would assume control of nearly a third of the district’s schools in fall 2012 and expand thereafter. Although the new authority was compared to Louisiana’s post-Katrina Recovery School District, operational details have yet to be defined and likely will differ substantially from those in New Orleans. But the new authority is certain to further shrink Detroit Public Schools.

Who Benefits?
While much attention has focused on student performance in charter schools, a distinct and important policy question relates to how increased competition from charter schools affects the effectiveness of traditional public schools. That is, how does charter competition impact the students who are not active choosers? Only recently has research on this question been possible in the United States, as choice options became sufficiently widespread to elicit competitive responses from traditional public schools.

Discussions of charter schools’ competitive effect have been dominated by two opposing arguments. Choice advocates maintain that if charter school policies offer parents expanded choices and tie funding to enrollment, then educators will have an incentive to compete and increase their efficiency by working harder and implementing education improvements.

Critics of school choice argue that a more competitive education system will not benefit all students but rather create winners and losers relative to the status quo, increasing academic, racial and ethnic stratification while further concentrating many of the most disadvantaged students in schools depleted of the personnel and resources needed for improvement.

Which view comes closer to reality? While the literature of rigorous quantitative studies is still taking shape, the initial results offer little evidence that charter schools significantly improve or worsen student outcomes in traditional public schools. The existing research indicates that charter schools’ competitive effects on student outcomes are quite mixed and generally small.

Nevertheless, there now are isolated local areas, such as Detroit, where charter school growth has contributed to substantial shrinkage of traditional public school districts and a transformation of who provides educational services.

Dictating Factors
The competitive effects of charter schools [DNHPE comment: or vouchers] depend on an array of circumstances, including features of policy design, the local context in which the choice policy is implemented and the responses of families and schools. 

Among the many relevant policy features is whether state funding is adequately adjusted for higher-cost students (special education or secondary versus elementary school). If not, charter schools have an incentive to disproportionately enroll the cheapest students to educate. Insofar as charter schools succeed in enrolling low-cost students and excluding high-cost students, they reduce their own average cost while increasing the average cost for traditional public schools that continue to educate high-cost students. Without additional revenues to match the higher costs, districts would be obliged to cut services.

Meanwhile, a given state’s charter school policy will create less-intense competitive pressure in local settings where population is rapidly growing rather than declining. In high-growth communities, public schools may even welcome the departure of students to charters because it alleviates enrollment pressures. As such, the traditional public schools experience no competitive pressures to change their practices.

Positive competitive effects also are more likely if families base their choices on schools’ academic quality, which researchers typically define as average student learning growth net of student background characteristics. Yet we know families choose schools for various reasons, including the student racial or socioeconomic composition. This could undercut incentives for public schools to respond in ways that will improve education quality. In addition, if active choosers are disproportionately high-achieving students, this will likely generate negative peer effects on the learning of students who remain behind.

Finally, the prospects for charter schools to spur improvement also clearly depend on the quality of school district leadership and the leaders’ relationships with district employees. If leadership is weak, not trusted and subject to frequent turnover, a school district will have limited capacity to respond to competitive pressures.

District Responses
Case study and interview research clearly point to the diversity of responses of traditional public schools to charter competition. In many cases, charter competition elicits little or no school district response at all.

Paul Teske and colleagues in an analysis of five urban districts, “Can Charter Schools Change Traditional Public Schools,” which appeared in Charters, Vouchers & Public Education (Brookings Institution), conclude that the link between charter competition and public school response is “tenuous at best.” In a review of studies covering several districts in the same volume, “Responding to Competition: School Leaders and School Culture,” Frederick Hess, Robert Maranto and Scott Milliman note a common finding that the majority of school systems display no response to competition.

When school districts do respond, one way they do so is by changing school leadership, opening magnet schools and creating new programs consistent with parents’ preferences. Some districts may start add-on programs, such as all-day kindergarten, after-school child care and extracurricular activities. Several researchers have noted an increase in districts’ marketing initiatives to compete for students. Some observers also noted efforts by the leaders of traditional public schools to vilify charter competitors or otherwise obstruct their opening and operations.

Thus far, however, little evidence points to competition stimulating significant change or innovation in instructional practice.

On balance, while some of the reported responses of traditional public schools to charters can be viewed as enhancing school quality, many plainly cannot. Taken as a whole, however, such accounts cannot gauge competition’s overall impact on school quality.

Quantitative Evidence
A number of recent, rigorous studies have examined how charter school competition influences student test scores in traditional public schools. Conceptually, the causal effect of charter competition is the difference between a traditional public school’s performance when facing competition and the same school’s performance when facing no charter competition. Because this scenario is impossible to observe in the real world (a school cannot both face and not face charter competition at the same time), researchers use statistical procedures to approximate this test.

One approach compares student performance in schools that are similar in many respects except that some face charter competition and others do not. Another approach compares student performance in public schools before and after they face charter competition.

With either approach, researchers must control for two other considerations.

First, charter school location is nonrandom. Charters are likely to locate where families are the least satisfied with local public schools. So if researchers observe that higher levels of charter competition correlate with lower outcomes in other public schools, it would be difficult to isolate the direction of causation. It is possible that low public school quality induced more choice options — or alternatively, that charter competition lowered public school performance.

Second, as noted, students who choose to go to charter schools may differ systematically from those who remain in their assigned public schools in terms of their past performance, ability or parental motivation. If, for example, charter schools tend to attract lower-performing students, the average performance of remaining students in public schools would automatically increase, even without any positive competitive effect.

A variety of statistical techniques permit researchers to control for the nonrandomness of charter school location and charter-induced changes in student composition in traditional public schools. The most desirable ones use longitudinal student-level data.

So far, rigorous charter school competitive effect studies have been conducted in several states, including Arizona, California, Florida, Michigan, North Carolina, Ohio and Texas. We reviewed these studies in “The Competitive Effects of Charter Schools on Public School Districts,” which appeared in a 2010 book The Charter School Experiment (Harvard Education Press).

The results from these studies are quite mixed. About half report negative or no effect of competition on traditional public schools’ student performance. Half report small positive results. No consistent patterns in the results emerge when categorizing the studies by state or by research methodology.

Sizing Up Debate
The best available current research is conspicuously at odds with strong claims regarding the beneficial competitive effects of charter schools. Neither does it sustain sweeping claims of harmful effects on public school students.

Charter school growth may bring about other meaningful changes. But if the existing research has one thing to say, it is that competition from charter schools has little impact on student performance in traditional public schools.

Charter school competition is a blunt policy tool for generating improvements in public schools. Although it represents a compelling idea, it is also a long shot. Participants on both the demand and the supply sides of the market for schooling have many feasible responses (and nonresponses) to charter school policies, and only a subset of them will benefit nonchoosers who remain in traditional public schools.

Although competition can create financial pressure on school districts, it does not by itself create the capacity to improve teaching and learning. While the design and implementation of charter school policies can be undeniably improved, policymakers should remain wary about suggestions that large, systemic improvements in school districts’ student learning will result from major increases in the number of charter schools.

Charter Density
Ever since states began passing charter school laws in the early 1990s, charter schools in most areas have enrolled a relatively small percentage of local students. Until recently, charters could be viewed as a small, heterogeneous system of alternative schools that offer families expanded choices, possibly incubate education innovations and hopefully spur improvements in the larger, dominant traditional public school system. In a growing number of urban areas, however, the system of charter schools no longer is small in relation to the local district schools. What’s changed in these districts with high levels of charter penetration?

Detroit offers an interesting case in point. While increased competition has not produced broad-based improvement in student outcomes in either charter or district schools in Detroit, it has introduced many new providers of public education services. This is perhaps the most significant change. The new providers operate largely outside the traditional governance structure of elected school boards and without unionized employees. Many view these changes in a positive light.

Charter school growth in Detroit and in other cities has expanded the schooling choices available to families, but it also has contributed to a turbulent education setting. No one should underestimate how messy and traumatic this transition has been for students, families and educators.

It is too early to know the longer-term consequences for students, yet one thing is certain. The traditional system of public schools in Detroit and other cities will be much more limited in relation to an expanding system of charter and contract schools that operate under govern­ance and management arrangements that traditionally have been much more typical of the private sector.

David Arsen is a professor of educational administration at Michigan State University in East Lansing. E-mail:arsen@msu.edu. Yongmei Ni is an assistant professor of educational leadership and policy at the University of Utah in Salt Lake City.

Duke University: Claims for School Voucher Success in Florida not Justified

posted Jan 28, 2012, 6:19 AM by Bill Duncan   [ updated Jan 29, 2012, 4:49 AM ]

Link

DNHPE Comment: Although advocates continue to rely on the competition argument, Duke's Center for Child and Family Policy discredited the assertion that competition from vouchers improves public schools 10 years ago.

In fact, what this policy brief shows is that, New Hampshire legislators could stimulate the greatest improvement in public schools for the least cost just by instituting an A-F grading system.  Schools the get low grades are very motivated to improve their grades.

(The No Child Left Behind rating of "in need of improvement" doesn't count.  It does't have any real meaning when it applies to 80% of the schools)



Claims for School Voucher Success in Florida not Justified

Helen F. Ladd, Ph.D., Elizabeth J. Glennie, Ph.D.

Recently, much public debate has focused on the use of accountability systems and on vouchers as means to improving education. Advocates of vouchers believe that the adoption of a school voucher system, in which families can use vouchers to send their children to either public or private schools, will spur schools to improve their performance in order to succeed in a competitive education market. In early 2001, the Manhattan Institute published a widely disseminated report that supported this view by claiming that Florida’s voucher system had generated significant improvements in the performance of the state’s lowest performing schools. However, according to Duke University researchers Dr. Helen F. Ladd and Dr. Elizabeth J. Glennie, the improvement of Florida’s low-performing schools probably had more to do with the implementation of that state’s accountability system than with the threat of students leaving the schools via the voucher system.

Ladd and Glennie used North Carolina to replicate the Manhattan Institute study.  North Carolina is similar to Florida in that it has a school-based accountability system that rates schools, but it doesn’t have a voucher program.  Ladd and Glennie found that, as was the case in Florida, North Carolina’s low-performing schools showed significant improvement after receiving their negative label. They concluded that the increased scrutiny and shame associated with being a low-performing school and the receipt of additional state assistance were likely the driving forces for school improvement in Florida, not its voucher program.

.........

Policy Implications
This and similar studies conducted elsewhere suggest that the results cited in the Florida study have little or nothing to do with vouchers.  If vouchers were the explanation for the gains in the F rated schools in Florida, it is unlikely that the Duke University study would have found comparable patterns of gains in the Low Performing schools of North Carolina.

Comment on the Calder Center Florida Study: Competitive Effects of Means Tested School Vouchers Tested

posted Jan 25, 2012, 7:30 AM by Bill Duncan   [ updated Feb 4, 2012, 5:54 AM ]

Summary

Link to the study (and attached )

Voucher supporters used to assert that students got a better education in private schools but the data showed that that did not happen.  So advocates have turned to this new argument - the benefits will be in the impact of competition on the public schools.  This is  ideologically attractive but it has not turned out to be true either.

In support of the competition argument, advocates cite "the study done by Florida's own Department of Education that proves that competition from an education tax credit program like the one proposed for New Hampshire improves the public schools."

There is no Florida Department of Education study of the competitive effects of Florida's Education Tax Credit program on Florida public schools.  The study advocates are referring to does record modest improvements in the public schools in the year between when vouchers were announced in Florida and when the program actually started.  But a reviewer calls those effects "tiny" and the author says that his results would not apply to New Hampshire.  Here is a useful Florida newspaper report on the study.

There is no study from any source that demonstrates that competition from voucher or education tax credit programs materially improves the public schools.

-----------------------------------------------------
Detail

The study itself (two versions are attached below) is a Calder Center "working paper" meaning that it has not been peer reviewed.  Our comments on it are below.

While the paper was written recently, it uses old data.  It seems to be a replication of a study done over 10 years ago by Jay Greene, a well known voucher advocate.  Greene's conclusion that public schools improved in response to voucher competition was discredited when Duke University replicated the Greene study in North Carolina.  Here is the Duke University policy brief based on that study.  

The authors caution that their study may have very little relevance to any other state.  Even a quick comparison makes clear that it is not relevant to New Hampshire.

For the period studied, the Florida program was different from the proposed New Hampshire program in many ways.  

Florida had large scholarships targeted to very poor children coming strictly from public schools rated "F" by the state of Florida for two years in a row.  The scholarship was automatically renewed for a long as the student was in school.  Average family incomes were very low.  Even now, family incomes average only $24,000.  Participating schools were academically accountable.  Florida's schools were ranked among the lowest in the nation.

New Hampshire's proposed program has small scholarship not targeted to low income families (that might change to a modest level of targeting but incomes will be higher than in Florida) coming from any private or public school.  New Hampshire has no school rating system.  The scholarship will not be automatically renewed.  Participating schools would not be academically accountable.  New Hampshire's schools are ranked among the highest in the nation.

Both programs are funded by tax credits but beyond that, they have little in common.  Florida is a mostly urban state 14 times the size of New Hampshire, with a large school system known for poor performance.  

The paper itself analyses the year between the announcement of the voucher program and when the first voucher students leave public schools to attend private schools. Most importantly, the study did not address the impact of other concurrent policies on those schools.  Duke University showed that receiving an "F" grade from the state's school rating system is really what led to improvement in the schools, whether or not a voucher system existed.

The authors did not interview the public school administrators and teachers involved.  The benefit of face-to-face interviews is illustrated  here, with the Indiana school principle who is losing kids to the Indiana voucher program. It is clear that he is on the defensive from the losses, not on the verge of academic gains in response to the new competitive pressures.

The author himself makes the case against trying to apply his conclusions elsewhere:

It is possible that the dynamics between competitive pressures
and public-school students’ test scores are systematically
different in Florida than they are in the rest of the nation. In
particular, more than 90 percent of Florida’s students live in
the state’s top 20 most populous metropolitan areas. In states
with a greater share of the population in rural areas, the effects
of a scholarship program may not exert the same degree of
competitive pressure on public schools. It may also be the
case that Florida’s diverse range of private school options and
accompanying greater competition among the private schools
limit the study’s generalizability.



---------------------
Notes:
  • The Florida program studied is entirely different from the proposed NH program
    • one of the largest in the US in terms of the number of low-income students served
    • 185% of federal poverty
    • student must have spent the entire previous year in a public school to qualify for a voucher
    • the voucher was up to $4,100, about 90% of the tuition and fees at a typical Florida religious private school
    • "The policy change was very large for the target population - the number of scholarships available was larger than half the size of the low income population using private schools on the eve of the policy's introduction.  For low-income Floridians, the voucher program represented a major demand shock for private schooling."
    • The Florida student will get follow-on voucher grants.  Most New Hampshire students will not.
    • Excludes home schooling
    • Requires 90% utilization to grow, rather than 80%
  • The schools lose about $6,600 per departing student and state government assumes that, because incomes are so low, 90% of the students would not have gone to private schools without the voucher.  Family incomes of voucher recipients average.less than $25,000
  • This is a "new" paper but is uses old data, replicating the Jay Greene study from 10 years ago.
  • The authors' thesis rests on a close proximity of the competing private schools to the public schools affected.  It's an odd thesis to being with.  In most cases, there would not be a comparable density of private schools in New Hampshire.  .
  • The authors attempt to control for so many complex interacting factors that it is hard to trust the results.
  • The paper does not establish cause and effect, just correlation of some very subtle and derivative variables.  That's why it gets caught flat footed on the A-F rating issue.
  • Florida's schools rank far lower than New Hampshire's.  Based on 4th grade math in 2011, for instance, New Hampshire was second in the country behind Massachusetts and Florida was 31st.
  • The authors do not account for the other concurrent policy interventions that could have contributed to the changes.  
    • For instance, (SP.8) the state of Florida's school accountability system gave schools grades.  Repeated "Fs" triggered voucher eligibility (and, probably, other policy interventions unmentioned in this paper).  Studies cited find that the F grade itself leads to "improved student outcomes."  This is said to have affected other studies by not this one (SP 9.)
  • They do not talk with administrators - just run the statistics from a distance.
  • They measure the effect in the year before the students left public school for private school, so the premise is that these competitive effects took place immediately upon announcement of the voucher program. (p.12 of pdf, p.4 of study)
  • Study page 6 lists all the studies that found that vouchers had little or no competitive impact.
  • Study page 7: why the Milwaukee study is not reliable.
  • In the Calder report, Figlio said his report was valid because he tested only the first year  after the program announcement, before any students had left.  In the follow-up report, he analyses all years.


“Direct certification list” means the certified list of children who qualify for the food assistance program, the Temporary Assistance to Needy Families Program, or the Food Distribution Program on Indian Reservations provided to the Department of Education by the Department of Children and Family Services.

(j) “Unweighted FTE funding amount” means the statewide average total funds per unweighted full-time equivalent funding amount that is incorporated by reference in the General Appropriations Act, or any subsequent special appropriations act, for the applicable state fiscal year.

1. A student is eligible for a Florida tax credit scholarship under this section if the student qualifies for free or reduced-price school lunches under the National School Lunch Act or is on the direct certification list and:
a. Was counted as a full-time equivalent student during the previous state fiscal year for purposes of state per-student funding;
b. Received a scholarship from an eligible nonprofit scholarship-funding organization or from the State of Florida during the previous school year;
c. Is eligible to enter kindergarten or first grade; or
d. Is currently placed, or during the previous state fiscal year was placed, in foster care as defined in s. 39.01.
2. A student may continue in the scholarship program as long as the student’s household income level does not exceed 230 percent of the federal poverty level.
3. A sibling of a student who is continuing in the scholarship program and who resides in the same household as the student shall also be eligible as a first-time tax credit scholarship recipient if the sibling meets one or more of the criteria specified in subparagraph 1. and as long as the student’s and sibling’s household income level does not exceed 230 percent of the federal poverty level.

.....
A scholarship organization must:
(d) Must provide scholarships, from eligible contributions, to eligible students for the cost of:
1. Tuition and fees for an eligible private school; or
2. Transportation to a Florida public school that is located outside the district in which the student resides or to a lab school as defined in s. 1002.32.
(e) Must give priority to eligible students who received a scholarship from an eligible nonprofit scholarship-funding organization or from the State of Florida during the previous school year.
(f) Must provide a scholarship to an eligible student on a first-come, first-served basis unless the student qualifies for priority pursuant to paragraph (e).
(g) May not restrict or reserve scholarships for use at a particular private school or provide scholarships to a child of an owner or operator.
(h) Must allow an eligible student to attend any eligible private school and must allow a parent to transfer a scholarship during a school year to any other eligible private school of the parent’s choice.

.........

(e) The parent shall ensure that the student participating in the scholarship program takes the norm-referenced assessment offered by the private school. The parent may also choose to have the student participate in the statewide assessments pursuant to s. 1008.22. If the parent requests that the student participating in the scholarship program take statewide assessments pursuant to s.1008.22, the parent is responsible for transporting the student to the assessment site designated by the school district.

..............

(8) PRIVATE SCHOOL ELIGIBILITY AND OBLIGATIONS.An eligible private school may be sectarian or nonsectarian and must:
.........

(c) Be academically accountable to the parent for meeting the educational needs of the student by:
1. At a minimum, annually providing to the parent a written explanation of the student’s progress.
2. Annually administering or making provision for students participating in the scholarship program in grades 3 through 10 to take one of the nationally norm-referenced tests identified by the Department of Education. Students with disabilities for whom standardized testing is not appropriate are exempt from this requirement. A participating private school must report a student’s scores to the parent and to the independent research organization selected by the Department of Education as described in paragraph (9)(j).
3. Cooperating with the scholarship student whose parent chooses to have the student participate in the statewide assessments pursuant to s. 1008.22.
(d) Employ or contract with teachers who have regular and direct contact with each student receiving a scholarship under this section at the school’s physical location.
...............

(9) DEPARTMENT OF EDUCATION OBLIGATIONS.The Department of Education shall:
.............

(i) Maintain a list of nationally norm-referenced tests identified for purposes of satisfying the testing requirement in subparagraph (8)(c)2. The tests must meet industry standards of quality in accordance with State Board of Education rule.
(j) Select an independent research organization, which may be a public or private entity or university, to which participating private schools must report the scores of participating students on the nationally norm-referenced tests administered by the private school in grades 3 through 10.
1. The independent research organization must annually report to the Department of Education on the year-to-year learning gains of participating students:
a. On a statewide basis. The report shall also include, to the extent possible, a comparison of these learning gains to the statewide learning gains of public school students with socioeconomic backgrounds similar to those of students participating in the scholarship program. To minimize costs and reduce time required for the independent research organization’s analysis and evaluation, the Department of Education shall conduct analyses of matched students from public school assessment data and calculate control group learning gains using an agreed-upon methodology outlined in the contract with the independent research organization; and
b. According to each participating private school in which there are at least 30 participating students who have scores for tests administered during or after the 2009-2010 school year for 2 consecutive years at that private school.
2. The sharing and reporting of student learning gain data under this paragraph must be in accordance with requirements of 20 U.S.C. s. 1232g, the Family Educational Rights and Privacy Act, and shall be for the sole purpose of creating the annual report required by subparagraph 1. All parties must preserve the confidentiality of such information as required by law. The annual report must not disaggregate data to a level that will identify individual participating schools, except as required under sub-subparagraph 1.b., or disclose the academic level of individual students.
....
(n)1. Conduct random site visits to private schools participating in the Florida Tax Credit Scholarship Program. The purpose of the site visits is solely to verify the information reported by the schools concerning the enrollment and attendance of students, the credentials of teachers, background screening of teachers, and teachers’ fingerprinting results. The Department of Education may not make more than seven random site visits each year and may not make more than one random site visit each year to the same private school.

......

(12) SCHOLARSHIP AMOUNT AND PAYMENT.
(a)1. Except as provided in subparagraph 2., the amount of a scholarship provided to any student for any single school year by an eligible nonprofit scholarship-funding organization from eligible contributions shall be for total costs authorized under paragraph (6)(d), not to exceed annual limits, which shall be determined as follows:
a. For a scholarship awarded to a student enrolled in an eligible private school:
(I) For the 2009-2010 state fiscal year, the limit shall be $3,950.
(II) For the 2010-2011 state fiscal year, the limit shall be 60 percent of the unweighted FTE funding amount for that year. [DNHPE NOTE: Florida's state aid per student is approximately $6,600 in 2012]
(III) For the 2011-2012 state fiscal year and thereafter, the limit shall be determined by multiplying the unweighted FTE funding amount in that state fiscal year by the percentage used to determine the limit in the prior state fiscal year. However, in each state fiscal year that the tax credit cap amount increases pursuant to subparagraph (5)(a)2., the prior year percentage shall be increased by 4 percentage points and the increased percentage shall be used to determine the limit for that state fiscal year. If the percentage so calculated reaches 80 percent in a state fiscal year, no further increase in the percentage is allowed and the limit shall be 80 percent of the unweighted FTE funding amount for that state fiscal year and thereafter.
b. For a scholarship awarded to a student enrolled in a Florida public school that is located outside the district in which the student resides or in a lab school as defined in s. 1002.32, the limit shall be $500.
2. The annual limit for a scholarship under sub-subparagraph 1.a. shall be reduced by:
a. Twenty-five percent if the student’s household income level is equal to or greater than 200 percent, but less than 215 percent, of the federal poverty level.
b. Fifty percent if the student’s household income level is equal to or greater than 215 percent, but equal to or less than 230 percent, of the federal poverty level.

1-7 of 7