See updated post on this topic: https://schoolfinance101.wordpress.com/2010/04/16/cartel-recap/
Bob Bowdon is once again proving his numerical wizardry with ads for his schlockumentary The Cartel (which I refuse to see because I will likely blow my top in the middle of it, correcting every damn wrong and misguided supposed fact spewed in the film’s narration). There’s a commercial out for the Cartel Movie which lists about 2 or 3 supposed “facts” about New Jersey schools and how they compare to schools in other states – on funding and on graduation rates. 2 or 3 numbers in the commercial, and you’d think Bowdon could get at least one right… or interpret at least one in a way that is not completely misguided schlock. We’re not talkin’ any high level of manipulation here… but rather… at the same childish, buffoon-ish level as previous Bowdon brilliance (the claim that higher state spending lowers SAT scores… Yeah… you go Bob! Awesome. Cool. Freakin’ amazing! https://schoolfinance101.wordpress.com/2009/05/30/idiot-of-week-award-the-cartel-check-this-out/ ) Okay, so what am I complaining about this time?
The claim in the commercial goes… New Jersey is the highest spending state in the nation when it comes to schools… and, even though the NJDOE claims that we are tops in graduation rates, if you count only those kids who actually pass the high school graduation exam we’re really 24th. Yep, we spend all that (first, by a long shot, apparently) and all we can get is 24th in graduation rate.
First of all, graduation rates aren’t particularly a great statistic for comparing across states because graduation is highly dependent not only on varied state standards but varied local rigor. That is partly the (missed) point of the commercial. But the commercial goes on to imply that NJ is necessarily softer on grads than other states. Thus, we must correct those distorted NJ grad rates by the numbers who actually pass the state test… and then and only then… compare against all those rigorous states that really whoop NJ %$$. Now, if we were going to “correct” New Jersey graduation rates to represent only those kids who can pass the NJ exam, the only legitimate way to compare against other states would be against the same standard – the ability of kids in other states to pass the NJ graduation test. Fun idea… but I don’t think other states are giving the NJ tests.
Hey… you know what… there is actually a report out there (by the gov’t agency charged with doing such analyses) that shows how states’ individual tests compare to specific cut points on the NAEP test (the one national standard assessment)… and thus to each other. But you’d have to do research… reading… actual numbers to find such a crazy thing. You wouldn’t want that kind of thing to taint the “facts” in a “documentary.” Here it is for future reference.
This particular report shows the NAEP score that would be associated with scoring proficient or higher on each state’s tests. So, for NJ, a proficient student on 8th grade reading would be expected to score 250 on NAEP 8th grade reading (the national test). The standard was higher (NAEP score associated with proficiency on state test) in about 13 states, but a heck of a lot lower in many states (lower in 19). NJ’s position was similar for 8th grade math. So, the state assessments in NJ fall near the upper portion of the pack among states, though certainly not near the top. Sadly there is no direct comparison of which I am immediately aware for the HS tests.
But, in NJ and many other states, these tests really don’t mean a whole lot to individual children, whether the test is rigorous or not. Yes… if we wish for anyone to take these tests seriously and if they are good and rigorous tests, we should expect kids to pass them to graduate. And we should do what we can to compensate for inequities in the preparation of high school students to succeed on the tests. But this has little or no bearing on state-to-state comparisons of graduation rates and is no excuse for Bowdon’s contorted reasoning (putting a generous spin on it).
As I have pointed out in previous posts (https://schoolfinance101.wordpress.com/2009/06/17/vacuous_bowdon/) , when it comes to NAEP assessments, NJ students do very well, in spite of, or perhaps in part because of our funding levels and distribution of that funding. NAEP is the tool we have for making state by state comparisons, of students on average and by subgroups. Also in my previous post (above), I show how New Jersey compares with countries based on an analysis linking NAEP and international assessment scores.
Here’s a tool for comparing state NAEP scores. Have fun! http://www.nces.ed.gov/nationsreportcard/statecomparisons/
Other groups produce indices such as “college readiness” indices, and NJ does very well on these (http://www.edequality.com/content/map/ ), though I’m not fully confident in the rigor of such reports and measures.
Oh… and on that funding thing, when comparing total state and local revenues per pupil by state, without any adjustment for regional cost variation or other factors, NJ actually falls behind NY and VT, and barely ahead of WY based on my own run of the district level Census Fiscal Survey for 2006-07 (most recent year: http://www.census.gov/govs/www/school07doc.html ). After cost adjustment NJ drops lower (cost adj. here: http://www.nces.ed.gov/edfin/adjustments.asp ). So, even using the “bad” version (unadjusted) of the numbers, NJ is not #1. NJ was #1 in Current Expenditures per Pupil (including expenditure of federal dollars) in 2005 with no adjustments for regional cost variation, but fell behind Vermont and Wyoming and in a dead heat with New York and Maine in that year when cost adjustment is applied (the NCES Comparable Wage Index).
The point is that NJ is not the undisputed, year-after-year #1 and does not stand out as this huge outlier on public school spending as Bowdon’s grossly distorted and completely a-contextual references would have one believe. NJ does indeed stand out as a state that has put substantial additional funds into large urban, high poverty districts as a function of years of litigation. But, most rigorous accounts (actual research, with real statistics) find positive effects of the infusion of resources on student outcomes (see: https://schoolfinance101.wordpress.com/2009/10/12/real-info-re-nj-and-abbott-districts/ as just one example)
Table 1 and Table 2 of this report on school finances provide some perspective on school funding from the most recent NCES summary of the most recent Census fiscal survey (http://nces.ed.gov/pubs2009/2009338.pdf).
Here are a few related resources which discuss dropout and graduation rates, based on reasonable (albeit still problematic) approaches for measuring such things and comparing across states.
NCES Freshman Graduation Rates
Report on Dropout and Graduation Rates
Yeah. I know I’m talking to the wall here. Don’t let data and reasonable analysis get in the way. That’s just too geeky. Wouldn’t want a good number or some relevant context to taint the message of a “documentary.”