New Report: Why most international comparisons of spending & outcomes are total BS!

Posted on June 8, 2016



New Report: http://www.shankerinstitute.org/resource/deconstructing-myth-american-public-schooling-inefficiency

Here’s the summary:

In this paper, we begin by classifying the arguments that assert American schools are relatively inefficient into two categories: the long-term trend argument and the international comparison argument. Our focus herein is on the latter of these two. We then describe two frameworks for approaching either of these arguments: cost efficiency and production efficiency. We explain that the typical spending/outcome model used to make the case that the United States is a relatively inefficient nation is wholly unsuitable for drawing these or any conclusions. Accounting for differences in student populations is helpful, but still inadequate for building a model that can be used to assess a country’s relative efficiency. Evaluating education inputs such as teacher wages and class sizes can further refine comparisons between nations; however, it is unlikely that even these refinements are enough to conduct analyses that can credibly back claims about the relative efficiency of America’s education system. That said, an appropriately limited analysis can still inform our understanding of how the U.S public education system compares with systems in other countries.

What does this all mean?

  • First and foremost, we can say with some confidence that existing expositions of U.S. inefficiency (based on Organization for Economic Cooperation and Development [OECD] national spending data and Program for International Student Assessment [PISA] scores) are so lacking in methodological rigor that they are of little if any value in public discourse or for informing national education policies.
  • Second, it is unlikely that we could ever obtain data of sufficient precision, accuracy and comparability to meet the demands of more legitimate efficiency modeling for cross-national, intercontinental analyses.

Any and all comparisons using OECD and related data should be conducted with consideration of the limitations discussed herein. But some insights might be drawn from our analyses:

  • Among other things, the OECD per-pupil spending measure, as incomparable as it is, shows that the U.S. may have higher per-pupil spending than many nations, but falls right in line with expectations for nations of similar gross domestic product (GDP) per capita.
  • The U.S. is both a high-spending and high-GDP country, but some of that high education spending may be a function of the scope of services and expenses included under the education umbrella in the U.S.
  • We also know that despite seemingly high spending levels in the United States, teachers’ wages lag with respect to other professions, and the wage lag is not a result of providing relatively smaller class sizes.
  • In fact, our primary class sizes (roughly equivalent to schooling provided from about age 5 through 11 or 12 years of age) are average and lower secondary (roughly equivalent to schooling provided from about 12 to 16 years of age)[1] class sizes large. Our wage lag is, to an extent, a function of high non-teaching wages (related to our high GDP per capita), necessarily making it more expensive to recruit and retain a high-quality teacher workforce.

To summarize: The U.S. is faced with a combination of seemingly high education expense, but noncompetitive compensation for its teachers, average to large class sizes, and a high rate of child poverty. Again, it’s hard to conceive how such a combination would render the U.S. comparable in raw test scores to low-poverty nations like Korea or Finland, or small, segregated, homogeneous enclaves like Singapore or Shanghai.[2]

Finally, it is equally important to understand the magnitude and heterogeneity of the U.S. education system in the context of OECD comparisons, which mainly involve more centralized and much smaller education systems. Lower-poverty, higher-spending states that have been included in international comparisons, like Connecticut and Massachusetts, do quite well, while lower-spending higher-poverty states like Florida do not. This unsurprising finding, however, also tells us little about relative efficiency, and provides little policy guidance for how we might make Florida more like Massachusetts, other than by waving a wand and making it richer, more educated and perhaps several degrees colder.

[1] See UNESCO (2012). Note that the 2011 ISCED classification scheme does not specify age ranges, focusing instead on the purposes of the levels of education (basic preparation, etc.). But that system is crosswalked to the 1997 scheme which does specific age ranges and where the previous (1997) ISCED level 1 and level 2 remain aligned with the present (2011).

[2] Shanghai in particular has several mitigating factors that make comparing its scores to other nations highly suspect; see: http://www.brookings.edu/research/papers/2013/10/09-pisa-china-problem-loveless

 

Posted in: Uncategorized