A short while back I did a post called Searching for Superguy in Gotham. In that post, I tackled the assumption that Superguy was easily identifiable as a hero leader of charter schools – or at least that was one distorted portrayal of Superguy in Waiting for Superman. Now, I should point out here that I really don’t know of anyone actually out there running charter schools who wishes to portray him/herself in this way. So, to be absolutely clear, this post is in no way an attack on those who are out there just trying to do the best job they can for kids in need.
This post IS a criticism of the punditry around charter schools- the notion that charter schools are easy to pick out from the crowd of urban (or other) schools- because they are necessarily, plainly and obviously better. That classic argument that the upper half is better than average!
This was the basis of my Searching for Superguy in Gotham activity. In that activity, I estimated a relatively simple statistical model to determine which schools performed better than expected, given their students and location and which schools performed less well than expected, given their students and location. I had been planning all along to do something similar with New Jersey Charter Schools. Now is that time!!!!!
As I did with New York City charter schools, I have estimated a statistical model of the proficiency rates of each charter school and each other school in the same New Jersey city. In the model, I correct for a) free lunch rates, b) homelessness rates, c) student racial composition (Hispanic and black). AND, I compare each test – grade level and subject – to the same test across all schools. AND, I compare each school to other schools in the same city (by using a “city” dummy variable). I obtained all necessary variables from a) NJ school report cards (outcome measures) and b) NJ enrollment data file (free lunch, race, homelessness) and c) NCES Common Core of data for “city” location of school.
So now, the search for Jersey Superguy begins! Let’s start with 4th Grade Math performance in 2009. This scatterplot includes all schools with ASK4 Math scores in cities where charters existed in 2009. Schools above the red horizontal line are schools that “beat the odds.” That is, they are schools that had proficiency rates that were above the expected proficiency rates for that school, given its students, the test, and the location (city). Schools below the red line are schools that did not meet expectations. So, is superman (mythical super charter school leader) hiding in one of those dots way at the top of the scatter? Is he in a high-flying, high poverty school? Is he in a high-flying low poverty school? Certainly, he could not be down in the lower half of the graph.
NOTE: I’m in the process of fixing a data error that occurs on a few charter schools (affecting merging of data). These figures still include the merge error, but the overall distributions are not affected. Schools affected include: Environment Community School, Liberty Academy, Hope Academy, International CS of Trenton, Jersey City Community CS and Jersey City Golden Door. I HAVE NOW EXCLUDED MISMATCHED SCHOOLS.
The source of the error is the NJDOE enrollment file, which, for example identifies Environment Community School as both 80_6232_920 (county, district, school) and as 80_6235_900. The first of these codes is correct. The second is for Liberty Academy CS (according to the School Report Card and according to NCES data).
Now, let’s take a look at the 8th Grade Math outcomes. Here’s the statewide scatterplot:
As you can see, there are plenty of charters and traditional public schools above the line, and below the line. The point here is by no means to bash charters. Rather, this is about being realistic about charters and more importantly realistic about the difficulty of truly overcoming the odds. It’s not easy and any respectable charter school leader or teacher and any respectable traditional public school leader or teacher will likely confirm that. It’s not about superguy. It’s about hard work and sustained support – be it for charters or for traditional public schools.
As I noted in my previous searching for superguy post:
Yeah… I’d like to be a believer. I don’t mean to be that much of a curmudgeon. I’d like to sit and wait for Superguy – perhaps watch a movie while waiting (gee… what to watch?). But I think it would be a really long wait and we might be better off spending this time, effort and our resources investing in the improvement of the quality of the system as a whole. Yeah, we can still give Superguy a chance to show himself (or herself), but let’s not hold our breath, and let’s do our part on behalf of the masses (not just the few) in the meantime.
And here is a separate model in which I test whether Charter schools on average outperform traditional public schools in the same city. This model shows that they don’t, or at least that their 1 to 3 percentage point edge on proficiency is not statistically significant. But whether charters on average outperform – or don’t – traditional public schools is not the point. The point is that like traditional public schools – they vary – and it’s important for us to get a handle on how and why all schools vary in their successes and failures – charter or not.
Complete slide set here: New Charter Figures Nov 12
Here are some updated maps of the demographics and adjusted performance measures of charter and district schools in Newark.
First, % Free Lunch 2009-10:
Finally, here are the Beating the Odds figures converted into color coded circles – with large purple circles being high performers – better than expectations – medium size pale dots being relatively average performers – and large yellow dots performing below expectations: