Pau Hill and Marguerite Roza’s response to my recent report – with Kevin Welner – and series of blog posts seems to offer as its central argument that we’re simply a curmudgeons, offering lots of complaints about the rigor of their arguments and their suggestions for improving schooling productivity and efficiency, but providing no creative or immediately useful ideas or solutions for school districts or states in these tough economic times.
My first response would be that bad ideas are bad ideas, even in the absence of alternatives. The fact that budgets are tight and many schools are underperforming is not an argument for implementing unproven, ill-considered policy solutions.
That said, my second response is that Kevin Welner and I did in fact offer our own solutions, both in our policy brief and elsewhere.
On the first response, allow me to point out that many of Hill and Roza’s own suggestions for how states or school districts should deal with these tough economic times are not cost-cutting measures at all. They provide little or no potential short-run cost savings – where cost savings means spending reduction while not harming (or perhaps even improving) student outcomes. For example, among the “cost-cutting” solutions offered in Petrilli and Roza’s Stretching the School Dollar[i], brief, one of the resources recommended by the U.S. Department of Education, is designing and implementing new teacher evaluations. I may disagree with the shape that most of these new systems are taking, but I certainly agree that improving evaluations is a good idea.
It is not, however, a cost-cutting measure. In fact, it would require significant up-front investments, and whether or not these new systems will improve student outcomes (to say nothing of doing so cost-effectively) is a completely open question.
In addition, Kevin Welner and I address numerous concerns regarding other proposals in Roza’s work on pages 9 to 15 of our policy brief, including proposals (in) for states to simply cut off aid for limited English proficient children after two years, or to cap the distribution of special education aid. These are simply spending cuts, not cost savings and there is little or no evidence that such cuts would cause no harm, much less generate improvements.
Hill and Roza also suggest that their brief on curing “Baumol’s disease” provides useful insights into how private sector industries have found cures that might be translated to public education. Kevin Welner and I thoroughly rebut the basic premise of the Baumols’ disease claim and Hill and Roza’s proposed solutions on Page 10-11 of our report. In short, Kevin Welner and I explain about this Baumol’s hypothesis (& solutions):
In sum, the report begins with two highly contestable claims. It then draws an unsupported causal connection between the two claims. Further, it assumes that the problem is universal—that the system as a whole is diseased. In making this assumption, the authors ignore any possibility that lessons may be derived from within the public education system.
My second general response is that, in contrast with Roza and Hill’s characterization of my work, Kevin Welner and I offer up lists of issues that have been studied regarding productivity and efficiency in public schooling, as well as frameworks for studying those issues and for guiding decision making. That is, our policy brief on this topic is not merely a list of curmudgeonly complaints and critiques of the work of Roza and others, though we do have serious concerns (elaborated through curmudgeonly complaints) about the quality of much of that work and claims made. In fact, I personally have even more serious concerns about related work from Roza and colleagues that has come to light since the writing of that brief. Yes, Kevin and I can really crank out those curmudgeonly complaints. But we don’t stop there… this time.
Among other things, Kevin and I point to research that offers potentially mundane solutions – the non-reformy stuff – like organizing schools within districts to optimize distributions of enrollments (school size) to achieve economies of scale – and we cite a handful of examples provided in a paper we cite by Brian Jacob & Jonah Rockoff[ii], including (a) changes to school start times for older students, based on research on learning outcomes by time of day; (b) changes in school-grade configurations, based on an increased body of evidence relating grade configurations, location transitions and student outcomes; and (c) more effective management of teacher assignments. Perhaps more importantly, we address methods and resources for guiding decision making on the basis of cost-effectiveness, and point out how cost effectiveness of particular options may, in fact, vary across settings.
It also bears mentioning that we don’t necessarily reject all of the ideas that Hill and Roza present. Rather, we explain that some might be explored, and might even be evaluated in pilot settings before we starting pitching them as large scale reforms. For instance, regarding large scale changes to teacher compensation systems, we explain:
That is not to say that such ideas cannot or should not be piloted and tested. They are, to some extent, researchable questions, most appropriately studied through relative efficiency analyses across districts and schools that are applying varied approaches, including the proposed new approaches.
In a forthcoming piece we go further to explain that some of our concerns regarding Roza and Hill’s current proposals are that they encourage large scale policy experimentation on the most vulnerable children, and we find that unacceptable given the extent of unknowns involved. We explain (forthcoming):
Further, the political rhetoric around the immediacy of reform focuses on so-called failing schools, and failure is identified through performance metrics heavily influenced by student demographics. Simply put, we have ethical concerns with imposing unproven and sometimes unstudied policies on schools in low-income communities of color. And this is what we see happening.
There is time to figure some of this stuff out, and there are things we can look to do more immediately to achieve short run cost savings where necessary. As I often point out, when advocates use language like “And they need these ideas NOW,” it is most often a ploy to compel people into expediting ill-conceived policies with as much potential to do harm as to do good.
Many of the things that can and should be done in the short run, to reorganize local school district budgets, and to cut spending with minimal negative impact on outcomes, are really mundane things. They’re just not sexy enough to make for a good political platform or to generate public outcry. They don’t even lend themselves to good acronyms or catch phrases, like LIFO (last in, first out). They don’t have really cool, catchy names like Parent Trigger (note – I’m not blaming or even trying to connect Roza and Hill to Parent Trigger). And they can’t be likened to a disease to be cured.
They are instead ideas with a strong, empirically-based track record.
In the longer term, yes – there are policies that should be considered and tested, including those pertaining to teacher quality. But they are substantive education policies, not cost-cutting measures, and we should not be using a budget crisis to justify unwarranted haste and recklessness. Good policy making must rely on good policy analysis, and this relationship should not be severed simply because money is tight. If anything, it should be strengthened.
===============
Supplemental Note:
Hill and Roza offer as evidence that their proposed strategies have been evaluated in terms of productive efficiency and cost effectiveness, three links to related “studies”. One of those links points to a paper by Dan Goldhaber and Roddy Theobald, regarding potential costs/benefits of Seniority based versus “Quality-Based” layoffs. On the one hand, the paper does not yield any decisive guidance for short term budget planning and on the other hand, suffers the circular logic I’ve discussed on numerous occasions on my blog regarding measuring the effectiveness of the policy by the same measure used to implement the policy (e.g. did firing teachers with low value added scores leave us with more teachers with high value added scores). The central conclusion of the paper(s) is that “Finally, simulations suggest that a very different group of teachers would be targeted for layoffs under an effectiveness-based layoff scenario than under the seniority-driven system that exists today.” This is hardly surprising, and of limited usefulness for informing state or local leaders on how to handle personnel decisions in tough budgetary times, or the expected benefits or downsides of such policies, and fails to address such basic issues as the costs of putting into place a system that might be used for making such decisions. I provide a hypothetical discussion of this topic in an earlier blog post. The other study cited is Eric Hanushek’s paper suggesting that teachers whose effectiveness ratings are one standard deviation above the mean can yield $400,000 benefit in present value of student future earnings with a class size of 20. This study also provides no guidance for how district administrators might cut costs, or even hold the line, while attracting or retaining teachers whose value added scores are a standard deviation higher than their current average. Rather, this study speaks to the kind of large scale deselection which I’ve discussed numerous times in this blog. The third “study” is not a study at all, but rather an opinion brief by Roza with relatively meaningless national ball park estimates of job loss under alternative dismissal scenarios.
[i] Petrilli, M., & Roza, M. (2011). Stretching the School Dollar: A brief for State Policymakers. Thomas B. Fordham Institute. Retrieved November 6, 2011, from http://www.edexcellencemedia.net/publications/2011/20110106_STSD_PolicyBrief/20110106_STSD_PolicyBrief.pdf.
[ii] See Jacob, B., & Rockoff, J. (2011). Organizing Schools to Improve Student Achievement: Start Times, Grade Configurations and Teacher Assignments. The Hamilton Project. Retrieved November 6, 2011, from http://www.hamiltonproject.org/files/downloads_and_links/092011_organize_jacob_rockoff_paper.pdf.
School Finance 101 
Instead of jumping to non-funded mandates and changing curriculum guides to the new common core. Districts could instead use the large amounts of money spent on implementing those new untested curricula to maintain and improve their teaching staff in house using many of the free PD available online. (DoingWhatWorks for example) That means not buying all the stuff being touted as great. (most of which ends up in some supply closet unused by reluctant teachers who weren’t shown how to use it)
We have always had to back into a budget even when they told us zero based budgeting was the modality. It wouldn’t be that difficult a task. Just like when I had to work two jobs in the summers and one after school to keep within a budget at home. If you know what you’ve got and can’t go in the red. Then that is what you have to do.
Good districts hire good principals who hire good teachers. Clean house from the top down. The school buses and kids arrive everyday. You don’t see the Superintendent or Board greeting them in the morning or waving good bye in the afternoon. They only seem to get in the way of those actually doing the work, the principals and teachers.
It is a complex systems problem that has a simple mechanical solution. Work with what you have, never fail to sharpen the saw or paint the barn, otherwise it will break down eventually.