Some statistical context for Central Falls

Posted on February 25, 2010



Pundits have been tweeting and blogging the Central Falls Rhode Island High School story this week, with many cheering the bold “turnaround” strategy of firing all of the school’s teachers. Essentially, the district Superintendent has dismissed all teachers in the school with the option for them to re-apply. The reason for the dismissal is that the school has performed very poorly in recent years on state assessments and when teachers were asked to work extra time, their union resisted – or so the reports go.

Pundits seem to think this is a great idea: http://www.usatoday.com/news/education/2010-02-24-all-educators-fired_N.htm

But Joe Williams of Democrats for Education Reform, a political action group, says: “This is what real political cover can do for public education. You see very clear signals coming from Washington that the Obama administration is serious about turning around our worst schools.”

Central Falls has long been one of the worst-performing in Rhode Island. Just 7% of 11th-graders tested last fall were proficient in math and 55% were proficient in reading. In 2008, 52% of students graduated within four years.

And: http://www.southcoasttoday.com/apps/pbcs.dll/article?AID=/20100224/NEWS/100229948

U.S. Secretary of Education Arne Duncan applauded the plan, saying students only have one chance for an education.

“When schools continue to struggle we have a collective obligation to take action,” he said in a written statement.

The U.S. Department of Education does not play a role in deciding which model schools choose and did not know Wednesday whether Central Falls was the first to opt to get rid of its teachers, said Sandra Abrevaya, a department spokeswoman.

The decision won praise from Republican Gov. Don Carcieri, a former math teacher who supports Gist.

“We can no longer stand by as our schools underperform,” Carcieri said in a written statement. “While we have some excellent individual teachers, our students continue to be held back by a lack of a quality education and by union leadership that puts their self-interests above the interests of the students.”

So, according to the above cast of characters, the way to fix low performing schools is simply to shake them up, get rid of the status quo current crop of teachers and others will be anxiously waiting in line to fill their shoes and fix these schools with the same or fewer resources and the same kids who’ve been there for years. I often point out that getting rid of your current teachers can really only lead to improvement if you are able to replace them with better ones. Are there 74 better teachers waiting in line outside Central Falls HS?

That issue aside, let’s take a look at where Central Falls fits in among Rhode Island High Schools in terms of a) student characteristics, b) spending per pupil and c) outcomes (based on 2006 data – which hasn’t shifted much over time). First, let’s look at the relationship between school level free/reduced lunch rates and combined (summed) proficiency rates across RI HS:

Yes, Central Falls is in tough shape – very high poverty and relatively low performing. But, not really off the trendline (above it, if anything) for performance given its poverty level and better than other high schools of similar poverty.

Now lets look at Central Falls performance with respect to school site spending from Rhode Island’s IN$ITE database:

Central Falls spending is somewhat above average. But again, its student needs are far greater than average – in fact, they are on the outer edge of the entire distribution. So, it is unlikely that “somewhat above average” per pupil spending is going to fully compensate for their high needs.

Here is an oversimplified, but still ugly enough statistical model of the relative costs, given student populations, economies of scale and current outcomes, of Rhode Island High Schools in 2006 (I had run this previously with 2003 to 2005 with similar results):

Admittedly this is no way to do “real” rigorous research. This is a single year of data on 30 high schools and too many covariates for the data. But, these data do reconcile with the model of the previous 3 years. I should also note that such analysis – those that try to pin down the relative efficiency of school district performance and/or cost – are generally unstable/unreliable. All that fun stuff aside, we have this crude model that tells us that at constant outcomes, per pupil costs are higher in higher poverty high schools and high schools with more special education children. And the model can produce for us an indicator of the extent to which each high school spends more or less than expected for the outcomes it receives – that is, the relative efficiency of the school. For the other stat geeks who might be reading, in this particular case, the estimates from the Stochastic Frontier Model and from the OLS regression model were identical.

Here’s the “relative efficiency” of Rhode Island High Schools with respect to cost, from lower to higher poverty high schools.

As it turns out, the relative efficiency of Central Falls HS stacks up pretty well with other Rhode Island High Schools. That is, the actual spending per pupil in Central Falls is not far off from the predicted amount to achieve their current outcomes, with their current population.

Here’s the above data, rescaled (from the OLS version of the model), and with the relative efficiency measures sorted from lowest to highest per pupil spending. The “0” line is the line where the district spends what it is predicted to spend to achieve its current outcome levels given its current students. Central Falls actually spends less than it is predicted to spend given its students and current outcome levels.  Many other high schools spend far more than expected, given their students and current outcomes.

This seems like a fairer comparison than simply casting stones at Central Falls teachers for their miserable test scores.

This is not to excuse low performance or to simply set a lower bar for this school because it serves a very high need population. But it is to point out that given their resources and their kids, they are doing as well as can be expected and better than many other Rhode Island high schools.

This analysis is far from definitive, but is illustrative. If it turns out, through more rigorous multi-year analyses, that Central Falls is efficiently producing its current (miserably low) level of outcomes at its current (relatively inadequate) spending level (at least by comparison with all other Rhode Island High Schools), then one answer here might actually be that Central Falls needs more resources to achieve better outcomes. Why is there no talk of this possibility?

The figures above suggest that Central Falls is doing as well with what it has as any other Rhode Island High School, after accounting for student needs. Where is the outcry over the amount Westerly High is spending to achieve its current outcomes (far less efficient that even Central Falls, in this quick analysis)? Yes, Westerly gets better outcomes, but with a much less needy student population (under 20% free or reduced compared to over 80%) and nearly $1,000 per pupil more in spending.

=====

Here’s the position of Central Falls based on my previous 2004-05 analysis. The overall cost inefficiencies are higher because the high schools were included in a model with middle and elementary schools (but with a dummy variable identifying them as high schools). This model included math outcomes only (language arts outcomes were non-significant). Again, Central Falls is not the standout inefficient school (Higher up in the graph is less efficient).

About these ads
Posted in: Uncategorized