There’s no mystery or proprietary secret among academics or statisticians and data geeks as to how to construct simple comparisons of school demographics using available data. It’s really not that hard. It doesn’t require bold assumptions, nor does it require complex statistical models. Sometimes, all that’s needed to shed light on a situation is a simple descriptive summary of the relevant data. Below is a “how to” (albeit sketchy) with links to data for doing your own exploring of charter and traditional public school demographics, by grade level and location.
Despite the value of a simple, direct and relevant comparison using accessible data providing for easy replication, many continue to obscure charter-non-charter comparisons with convoluted presentations of less pertinent information. Matt DiCarlo recently published a very useful post (at Shanker Blog) explaining the various convoluted descriptions from Caroline Hoxby’s research on charter schools that make it difficult to discern whether the charter schools in her comparisons really had comparable student populations to nearby, same grade level traditional public schools.
As I’ve discussed in the past, charter advocate researchers tend to avoid these basic comparisons, instead showing that students selected through the lottery were comparable to those not selected but who still entered the lottery (excluding all those who didn’t enter the lottery). While this information is relevant to the research question at hand (comparing effectiveness among lottery winners and losers), it skips over entirely another potentially relevant tidbit – whether, on average, the charter students are comparable to students in surrounding schools.
Alternatively, charter advocate researchers will compare charter characteristics to district wide averages, or whatever comparison sheds the most favorable light. For example, Matt DiCarlo explains of Caroline Hoxby’s NYC charter research that:
“The authors compare the racial composition of charter students to that of students throughout the whole city – not to that of students in the neighborhoods where the charters are located, which is the appropriate comparison (one that is made in neither the summary nor the body of the report). For example, NYC charter schools are largely concentrated in Harlem, central Brooklyn and the South Bronx, where regular public schools are predominantly non-white and non-Asian (just like the charters).”
The better approach is, of course, to compare against the, well, most comparable schools – or those serving similar grade levels in the same general proximity – or even to be able to identify each individual school (such that one can determine comparable grade levels) among districts in similar locations.
Here’s my general guide to making your own comparisons using a readily available data source.
Go to: www.nces.ed.gov/ccd
Use the Build a Table function: http://nces.ed.gov/ccd/bat/
- Select as many years of data you want/need (first screen toggle)
- Select the “school” as your unit of analysis for your data (first screen, drop down)
- Select “contact information” from the drop down menu on next screen
- Select location zip code
- Select location city
- Select “classification information” from the drop down menu
- Select the “charter” indicator
- Select the “magnet” indicator (in case you want to include/exclude these)
- Select “total enrollment” from the drop down menu
- Select total enrollment
- Select “students in special programs” from the drop down menu
- Select students qualifying for free lunch
- Select students qualifying for reduced price lunch
- Select “Grade Span Information” from the drop down menu
- Select “school level” identifier
- Select “High Grade” and “Low Grade” indicators if you want more flexibility in comparing “like” schools
- Pick the state or states you want (you can’t use this tool to pull all schools nationally because the data set will be too large for this tool. Complete data are downloadable at: http://nces.ed.gov/ccd/pubschuniv.asp )
Calculate Percent Free Lunch and Percent Free & Reduced Lunch (divide groups by total enrollment)!
Here are some examples…
First, here are a handful of New Jersey Charter Schools compared to other schools (comparable and not) in their same zip code.
In this first figure, from a Newark, NJ zip code, we can see quite plainly and obviously that the shares of children qualifying for free lunch in Robert Treat Academy are much lower than all other surrounding schools, including the high school in the zip code (Barringer), where high schools typically have lower rates of students qualifying (or filing relevant forms) for free lunch.
Here are a few more.
Other “high flying” charters in Newark including North Star Academy, Gray Charter School and Greater Newark Academy, in a zip code with fewer traditional public schools, tend to have poverty concentrations more similar to specialized/magnet schools than to neighborhood schools in Newark. Other charter schools like Maria Varisco Rogers and Adelaide Sanford have populations more comparable to traditional neighborhood schools. But, we don’t tend to hear as much about these schools – or their great academic successes.
Things aren’t too different over in Jersey City. In the area (zip code) around Learning Community Charter School, other charters and neighborhood schools have much higher rates of children qualifying for free lunch than LCCS. Only the special Explore 2000 school has a lower rate.
Ethical Community Charter also stands out like a sore thumb when compared to all other schools in the same zip code, including those serving upper grades which typically have lower rates.
But what about those NYC KIPP schools? How about some KIPP BY ZIP?
So much has been made of the successes of KIPP middle schools, coupled with much contentious debate over whether KIPP schools really serve representative populations and/or whether they are advantaged by selective attrition. I included some links to relevant studies on those points here. But even those studies, which make many relevant and interesting comparisons, don’t give the simple demographic comparison to other middle schools in the same neighborhood. So here it is: