Some statistical context for Central Falls


Pundits have been tweeting and blogging the Central Falls Rhode Island High School story this week, with many cheering the bold “turnaround” strategy of firing all of the school’s teachers. Essentially, the district Superintendent has dismissed all teachers in the school with the option for them to re-apply. The reason for the dismissal is that the school has performed very poorly in recent years on state assessments and when teachers were asked to work extra time, their union resisted – or so the reports go.

Pundits seem to think this is a great idea: http://www.usatoday.com/news/education/2010-02-24-all-educators-fired_N.htm

But Joe Williams of Democrats for Education Reform, a political action group, says: “This is what real political cover can do for public education. You see very clear signals coming from Washington that the Obama administration is serious about turning around our worst schools.”

Central Falls has long been one of the worst-performing in Rhode Island. Just 7% of 11th-graders tested last fall were proficient in math and 55% were proficient in reading. In 2008, 52% of students graduated within four years.

And: http://www.southcoasttoday.com/apps/pbcs.dll/article?AID=/20100224/NEWS/100229948

U.S. Secretary of Education Arne Duncan applauded the plan, saying students only have one chance for an education.

“When schools continue to struggle we have a collective obligation to take action,” he said in a written statement.

The U.S. Department of Education does not play a role in deciding which model schools choose and did not know Wednesday whether Central Falls was the first to opt to get rid of its teachers, said Sandra Abrevaya, a department spokeswoman.

The decision won praise from Republican Gov. Don Carcieri, a former math teacher who supports Gist.

“We can no longer stand by as our schools underperform,” Carcieri said in a written statement. “While we have some excellent individual teachers, our students continue to be held back by a lack of a quality education and by union leadership that puts their self-interests above the interests of the students.”

So, according to the above cast of characters, the way to fix low performing schools is simply to shake them up, get rid of the status quo current crop of teachers and others will be anxiously waiting in line to fill their shoes and fix these schools with the same or fewer resources and the same kids who’ve been there for years. I often point out that getting rid of your current teachers can really only lead to improvement if you are able to replace them with better ones. Are there 74 better teachers waiting in line outside Central Falls HS?

That issue aside, let’s take a look at where Central Falls fits in among Rhode Island High Schools in terms of a) student characteristics, b) spending per pupil and c) outcomes (based on 2006 data – which hasn’t shifted much over time). First, let’s look at the relationship between school level free/reduced lunch rates and combined (summed) proficiency rates across RI HS:

Yes, Central Falls is in tough shape – very high poverty and relatively low performing. But, not really off the trendline (above it, if anything) for performance given its poverty level and better than other high schools of similar poverty.

Now lets look at Central Falls performance with respect to school site spending from Rhode Island’s IN$ITE database:

Central Falls spending is somewhat above average. But again, its student needs are far greater than average – in fact, they are on the outer edge of the entire distribution. So, it is unlikely that “somewhat above average” per pupil spending is going to fully compensate for their high needs.

Here is an oversimplified, but still ugly enough statistical model of the relative costs, given student populations, economies of scale and current outcomes, of Rhode Island High Schools in 2006 (I had run this previously with 2003 to 2005 with similar results):

Admittedly this is no way to do “real” rigorous research. This is a single year of data on 30 high schools and too many covariates for the data. But, these data do reconcile with the model of the previous 3 years. I should also note that such analysis – those that try to pin down the relative efficiency of school district performance and/or cost – are generally unstable/unreliable. All that fun stuff aside, we have this crude model that tells us that at constant outcomes, per pupil costs are higher in higher poverty high schools and high schools with more special education children. And the model can produce for us an indicator of the extent to which each high school spends more or less than expected for the outcomes it receives – that is, the relative efficiency of the school. For the other stat geeks who might be reading, in this particular case, the estimates from the Stochastic Frontier Model and from the OLS regression model were identical.

Here’s the “relative efficiency” of Rhode Island High Schools with respect to cost, from lower to higher poverty high schools.

As it turns out, the relative efficiency of Central Falls HS stacks up pretty well with other Rhode Island High Schools. That is, the actual spending per pupil in Central Falls is not far off from the predicted amount to achieve their current outcomes, with their current population.

Here’s the above data, rescaled (from the OLS version of the model), and with the relative efficiency measures sorted from lowest to highest per pupil spending. The “0” line is the line where the district spends what it is predicted to spend to achieve its current outcome levels given its current students. Central Falls actually spends less than it is predicted to spend given its students and current outcome levels.  Many other high schools spend far more than expected, given their students and current outcomes.

This seems like a fairer comparison than simply casting stones at Central Falls teachers for their miserable test scores.

This is not to excuse low performance or to simply set a lower bar for this school because it serves a very high need population. But it is to point out that given their resources and their kids, they are doing as well as can be expected and better than many other Rhode Island high schools.

This analysis is far from definitive, but is illustrative. If it turns out, through more rigorous multi-year analyses, that Central Falls is efficiently producing its current (miserably low) level of outcomes at its current (relatively inadequate) spending level (at least by comparison with all other Rhode Island High Schools), then one answer here might actually be that Central Falls needs more resources to achieve better outcomes. Why is there no talk of this possibility?

The figures above suggest that Central Falls is doing as well with what it has as any other Rhode Island High School, after accounting for student needs. Where is the outcry over the amount Westerly High is spending to achieve its current outcomes (far less efficient that even Central Falls, in this quick analysis)? Yes, Westerly gets better outcomes, but with a much less needy student population (under 20% free or reduced compared to over 80%) and nearly $1,000 per pupil more in spending.

=====

Here’s the position of Central Falls based on my previous 2004-05 analysis. The overall cost inefficiencies are higher because the high schools were included in a model with middle and elementary schools (but with a dummy variable identifying them as high schools). This model included math outcomes only (language arts outcomes were non-significant). Again, Central Falls is not the standout inefficient school (Higher up in the graph is less efficient).

Advertisements

6 Comments

  1. Wow! Thanks for your work on this. Seeing the data presented in different ways reinforces what many educators believe really happened at Central Falls. They are making an example of this school, and its the first salvo in the road to breaking teachers’ unions. It our own air traffic controller nightmare. And I love when you write “Are there 74 better teachers waiting in line outside Central Falls HS?” This is exactly what people don’t realize. NO. There is no line. I feel for the kids and the community.

  2. see: http://bottomupeducation.wordpress.com/2010/03/01/central-falls-statistics-expectations-and-support/#comment-6

    My response:

    Thanks for jumping in on this. Nice point on the drops. Had to work with the data readily at hand for the “quick and dirty.” I should also point out that I was going through my RI staffing database for further insights the other day and that database includes a student load measure. Teachers at Central Falls appear to be carrying a higher than average student load (compared to other RI high schools, mostly lower need)- an indication of the extent to which they are/may be under-resourced.

    If one considers the findings of Bill Duncombe and John Yinger regarding additional costs of achieving comparable outcomes under varied conditions (especially student poverty), clearly Central Falls is under-resourced.

    http://ideas.repec.org/p/max/cprwps/60.html

    I agree with your main point above (which was essentially my point), that we should try to make appropriate comparisons across schools and that if we were to find that “all else equal” the teachers at CF aren’t cutting it, then it becomes a different story – and the scrutiny on them would be warranted.

    I’m just having a little trouble with the current punditry on this topic. If we do buy their logic, then we should just go around firing all of the teachers in every under-resourced, low performing poor urban school in the nation (they’re not all under-resourced) and expect that there will be plenty of “better” teachers waiting in line to replace them – without changing resource levels – or any other factors. Seems to be a problematic approach at best.

  3. What you had here was a school administration that wanted teachers to put in some extra time to help students, a union that insisted on getting $90 an hour for that time, and an administration that responded with a hard bargaining tactic to get the union thinking more about kids than about their pocketbooks. Once the union started doing the right thing, the administration rehired all the teachers. So in the end, it looks pretty useful for an administration to have that bargaining tactic in its pocket, right?

    1. The problem with this logic is that the “negotiation tool” as you put it, would generally only be available in poor, minority schools and/or under-resourced schools that are more likely to be “failing.” Teachers may be equally bad or much worse and higher paid in more affluent districts, but their students – simply by virtue of their family background, may surpass the minimum standards. Perhaps they should do much better, but all they have to do is surpass the minimum standards to avoid running into this negotiation nuclear option.

      My graphs in this post show that Central Falls High performed about as expected, given its student mix and available resources. Should these teachers at Central Falls HS be put in a weaker negotiating position than other teachers across the state because of where they work and the resources available to them? If so, how will that affect the likelihood that new teachers want to teach in a place like Central Falls? Even from a local perspective, is it helpful as a negotiating strategy to position yourself as a hostile district? How will that help when trying to fill vacancies?

Comments are closed.