Replicating Robert Treat Academy

With little doubt, Robert Treat Academy in Newark is one of those charter schools that is doing well by common outcome measures and likely by even more important measures than state tests. What we know about are the tests. And even if one controls for a variety of factors about student populations, Treat’s test scores are pretty darn good.

Here’s a figure from a model I re-ran the other day (based on older work), using a variety of school, student population and community factors to control for expected differences in student outcomes. Schools above the line are those that outperformed expectations and those below the line fell below expectations. Charters are in red, and again, there are roughly equal numbers of traditional publics above and below the red line and charters above and below the red horizontal line. Treat is one of those above the line.

Treat Beat

So the argument goes, Treat is producing these test scores with much less money, and therefore we should be able to do the same, with similarly less money across poor urban settings by emulating the Treat model.

I addressed in a previous post how charter schools receive less through the state aid formula than traditional public districts. Again, this should shift somewhat over time, but charters will remain relatively disadvantaged. Using Robert Treat’s IRS 990 for 2007 expenditures (instead of their NJDOE reporting of their expenditure of public charter funding only), Treat shows expenditures per pupil in 2007 around $12,600. I’m still not sure I’ve captured the full expenditure here, because Treat’s IRS 990s show unusually low levels of private contribution for a successful charter school.

That aside, is the Treat miracle replicable across Newark? Or, is Treat different in substantive ways that can’t be spread throughout the system. Here are a few numbers that raise concern.

First, as I noted on a previous post, Robert Treat’s student body is only 3.8% special education in a district with an average of 18.1%.¬† This is from the special education classification data from NJDOE. In the enrollment files, Treat reports 0%. At 100% additional average expenditure per special education pupil, matching district demographics would raise Treat’s expected spending to $14,868 (1.18 x 12,600 in 2007).

Second, while Robert Treat does show about 62.4% students qualifying for free (130% poverty level) and reduced (185% poverty level) lunch, the free lunch share is about 42.9%. That is, Treat’s free or reduced share is boosted by the share of children who are more well off among the less well off. Note that the model I used above used Free & Reduced shares, not Free alone or the ratio between them.

By contrast, Newark Public Schools in total has 82% free or reduced and 71% free lunch alone.

Treat also reports less than 1% limited English proficient students while Newark City schools report 8.7%.

It’s one thing for me to try to control for these differences in estimating who does and does not “beat” odds, but yet another to take a model that has been successful under certain circumstances and apply it widely under very different circumstances, at the same cost.

It’s all well and good to cite other studies from other cities¬† and states that show that charter schools on average aren’t “cream-skimming,” (where most of those comparisons are based either on student’s initial performance or on free + reduced shares) but the reality in this case is that Treat Academy is producing its current level of outcomes at its current price tag with a substantively different student population – most notably the absence of children with disabilities. Again, they’re doing well, and even in models I’ve run controlling for some of these things, they still stand out and should be applauded for their efforts and results.

But, given the demography of the entire student population of Newark in particular, replicating this model may prove difficult. Adding more schools that serve fewer of the poorest children and few or no children with disabilities may be significantly problematic for those schools which then serve the larger shares of both.