Reality Check: Innovation & Substitution in Charter & Private Schools?

Market Induced Innovations in Charter & Private Schools?

A frequent critique of traditional public schools is that as bureaucratic public institutions, operating under collective bargaining agreements with politically powerful labor unions, public schools are beholden to those unions and inefficiently allocate resources into increasing numbers of unionized staff and into salaries and benefits for those staff (typically favoring quantity over quality/wage). Lacking any market competition, there is no need to seek more efficient alternatives.

Charter schools by contrast are not beholden to all of the same bureaucratic structures and obligations, and are subject to market competition. As such, if there are innovations out there which really can change the way schooling gets done, we might find those innovations emerging in the charter sector.  To an even greater extent, private schooling is also not subject to the same bureaucratic controls and labor agreements of public systems and private schools which drive the majority of their revenue from tuition paying parent-consumers must be particularly responsive to local and regional competitive market pressures.

While modern charter schooling was conceived by some as a way to spur innovation – try new things – evaluate them – and inform the larger system, studies of the structure and practices of charter schooling find the sector as a whole not to be particularly “innovative.”[i] Analyses by charter advocates at the American Enterprise Institute find that the dominant form of specialized charter school is the “no excuses” model – a model which combines traditional curriculum and direct instruction with strict disciplinary policies and school uniforms, in some cases providing extended school days and years.[ii] Further, charter schools raising substantial additional revenue through private giving tend to use that funding to a) provide smaller classes, and b) pay teachers higher salaries for working longer days and years.[iii] For those spending less, total costs are held down, when necessary, through employing relatively inexperienced, low wage staff and maintaining high staff turnover rates.[iv] In other words, the most common innovations are not especially innovative or informative for systemic, structural reform.

Lessons from High Profile Charter Operators in NYC

Studies of charter school effectiveness show vast differences across charter school operators (management companies) and across cities and regions. Studies of charter school efficiency will be discussed in a later post. Studies of charter schools in New York City have often shown strong positive effects, as have studies of major, established charter school operators like the Knowledge is Power Program (KIPP).[v]

Here, I provide an illustrative walk through of the characteristics of students, resources and teachers for major operators of charter schools in New York City, based on data from my prior research.[vi] The data are from the years 2008 to 2010.  I begin with Figure 20 which compares the demographics of major charter school operators in New York City to those of district schools serving the same grade ranges in the same borough of the city.  For each of these major operators, including the oft cited KIPP schools, the share of low income (those qualified for free lunch, or <130% income threshold for poverty status), English language learners (ELL), and children with disabilities is lower than for district schools, in some cases quite substantially.  That is, on average, these schools are serving far less needy and thus less costly student populations than district schools.

Figure 20

Slide24

Figure 21 shows the relative spending of these schools when compared with district schools in the same borough of New York City, serving the same grade levels and similar students.  In the lighter gray bars, schools are also compared with schools of similar size. Charter schools tend to be smaller and operate at higher expense as a result of lacking economies of scale. Harlem Children’s Zone, KIPP and Uncommon Schools significantly outspend similar district schools. Success charter network does as well, when setting aside scale related spending differences, which might be considered inefficient. That is, in a population dense urban center, why operate schools which are so small that they incur additional expenses? That they lack economies of scale?

Figure 21

Slide26

Figure 22 shows that the additional spending relative to district schools, at least in Harlem Children’s Zone, KIPP and Uncommon Schools translates into consistently smaller class sizes than in comparable district schools.  Achievement First has leveraged their dollars (a smaller available margin) specifically toward 8th grade math class sizes. That is, these schools are using their additional spending to provide smaller classes, in some cases, within a context of much smaller schools.

Figure 22

Slide28

Figure 23 shows the relative wages of teachers in these schools, compared to teachers with similar degree levels and experience in district schools.  Teachers in the charter schools tend to have far fewer average years of experience and thus lower average salaries. This occurs both because of higher turnover rates among young teachers in the charter schools and because many of the schools themselves, at the time were only 5 to 10 years old.  The dark bars show the relative annual salary difference, and the lighter bars show the salary difference adjusted for contract months.  Teachers in Achievement First and Uncommon Schools are paid substantially higher annual salaries than similar teachers in district schools. But, a share of that difference is due to the longer years they are assigned to work. While the margins of difference are smaller for Harlem Children’s Zone and KIPP schools, the patterns are similar. There exists a substantial annual salary differential, but nearly all of that salary differential is attributed to additional contracted months.

Figure 23

Slide29

Collectively, these figures tell a story of high profile, well-funded charter operators in New York City leveraging their additional resources in three, logical and rather traditional ways: 1) hiring more staff per pupil in order to provide smaller class sizes, 2) paying their teachers more at any given level of experience and degree, and 3) specifically paying those teachers more to work more hours, days and months.  That is, more people for more time.

Studies of Charter School Resource Allocation

The New York City examples above are somewhat unique. Not every city and state has the same well-endowed high flying charter operators and local philanthropy to support them. Our study which first illustrated these differences in New York, found that Ohio charter schools were much less well-off and Texas charter operators a mixed bag.[vii] Further, the examples above are by no means meant to imply that charter schools allocate dollars to human resources in exactly the same way as district schools. Rather, the point is that charter schools that have the resources to do so – the flexibility to make trade-off decisions – often choose to spend on logical things – like competitive wages, more money for more time and smaller classes.

Studies of resource allocation in charter schools have indeed found some systematic differences between charter school and traditional district schools spending. But those differences are hardly what one would consider a result of technological innovation or substitution. In most cases, observed resource allocation differences result from structural constraints like operating at inefficient scale, with inadequate access to capital for acquisition and renovation of facilities, or simply being new and having larger numbers of new, young and lower paid employees.

Two studies of Michigan charter schools, which operate fiscally independently of local public districts, have found them to have particularly high administrative expenses and low direct instructional expenses. Arsen and Ni (2012) found that “Controlling for factors that could affect resource allocation patterns between school types, we find that charter schools on average spend $774 more per pupil per year on administration and $1141 less on instruction than traditional public schools.”[viii] (p. 1) Further, they found “charter schools managed by EMOs spend significantly more on administration than self-managed charters (about $312 per pupil). This higher spending occurs in administrative functions traditionally performed at both the district central office and school building levels.” (p. 13)

Izraeli and Murphy (2012) found that district schools in Michigan tended to spend more on instruction per student than did charter schools, and the gap grew by about 5 percent to nearly 35% percent over the period studied (1995-96 to 2005-06) (p. 265).[ix] Further they found the spending gap for instructional spending to be greater than that for general spending. The overall funding gap between district and charter schools was approximately $230. The spending gap for basic programs was $562 and for total instruction $910. The authors note “much like a profit-maximizing firm, charter schools generate a surplus of revenue over expenditure.” (Izraeli & Murphy, 2012, p. 265)

Bifulco and Reback (2014) explore the complex relationship between fiscally dependent charter schools and their host districts in upstate New York cities. Particularly relevant to our investigation is Bifulco and Reback’s finding that having fiscally dependent charter schools separately affiliated with outside management companies and governance structures can create excess, redundant costs (p. 86).[x]

Others have explored teacher compensation in relation to instructional expense in charter schools. In a recent comprehensive review of charter school research, Epple, Romano and Zimmer (2015) summarize that “On the whole, teachers in charter schools are less experienced, are less credentialed, are less white, and have fewer advanced degrees.[xi] They are paid less, their jobs are less secure, and they turnover with higher frequency.” (Epple, 2015) Similarly, in a report on spending behavior of Texas charter schools Taylor and colleagues (2011)[xii] explain that much of the difference between instructional and non-instructional expense across differing types of charter and district schools is tied to differences in teacher compensation. The authors explain that “open-enrollment charter schools paid lower salaries, on average, than did traditional public school districts. Average teacher pay was 12% lower for teachers in open-enrollment charter schools than for teachers in traditional public school districts of comparable size, and adjusted for differences in local wage levels, average teacher pay was 24% lower. Average teacher salaries were lower not only because open-enrollment charter schools hired less experienced teachers, on average, but also because open-enrollment charter schools paid a smaller premium for additional years of teacher experience.” (p. ix)

Lessons from Texas

Variations across Texas operators provide additional insights. Table 3 provides an example, using two sources of Texas school expenditure data, of the very different approaches of two nonprofit education management organizations—KIPP and Harmony.[xiii] Specifically, Table 3 contrasts the revenues and expenditures of charter schools relying primarily on revenue enhancement versus those relying on expenditure cutting to free up resources for “other” uses, presumably including innovation. KIPP’s primary approach to supporting its various additional costs is one of revenue enhancement,[xiv] whereas Harmony’s is one of salary expense reduction.

Publicly subsidized revenues in Texas charter schools and in urban districts are relatively consistent. Nonetheless, KIPP schools in Texas in 2013, on average, increased their operating expenses by nearly 20%. While major urban district schools, other charter schools, and Harmony schools spent around $8,500 per pupil, KIPP schools reported spending $10,280—a conservative figure that likely misses some additional spending by KIPPs regional and national organizations on Texas KIPP schools. Despite the higher revenue, Table 3 indicates that KIPP’s instructional spending was second lowest. Harmony’s was lowest: even though in comparison to urban district and other charter school its total operating expense was lower, Harmony still spent a smaller percentage on instruction. KIPP’s total payroll per pupil remained higher than that of Harmony.

Despite having the highest total operating expense, KIPP schools diverted the largest shares to both central and school leadership activities. And, while Harmony’s central administrative expenses were lower than “other” charter schools as a share of operating expense, its “plant services” expenses were noticeably higher, approaching 20% of operating expense.  Again, while these operating expenses are notably different both among charter operators and between charter operators and district schools, it is difficult to attribute these differences to disruptive innovation or technological substitution. In particular, the more researched of the two operators, KIPP, has maintained expenditure on personnel, but shifted emphasis toward administrative personnel (school leadership).

Table 2

Slide42

Private Schools

Private schools have the greatest flexibility to operate differently than traditional public district schools and arguably the greatest competitive market pressure to do so. There exists much less evidence on private school spending, resource allocation or teacher compensation.  One difficulty in making private to public school comparisons is that private schools, by nature of their reduced regulation, vary widely and vary regionally, and financial reporting of charter schools is sparse, inconsistent and decentralized.  Like charter schools, private schools vary by their operator type. While many private school studies view schools only in terms of Catholic Schools, other religious schools and “other” non-religious schools, these distinctions are not sufficiently fine grained. In a 2009 study, I collected IRS financial filings of 1,500 private schools nationally, combining those data with national surveys of private school teachers, school enrollments and pupil to teacher ratios (for nearly all private schools).[xv]

Table 3 summarizes the comparisons with public schools of major private schools by their primary affiliation. Catholic schools (parish elementary and middle schools and diocesan secondary schools) for example, had very low nominal tuition prices, but per pupil spending much higher than their tuition, at $10,135, comparable to nearby district schools (metro area). Catholic schools had lower teacher salaries for similar teachers, but comparable staffing ratios. At the bottom end were low tuition and relatively low spending schools belonging to dominant Christian schools associations. These lower spending schools managed to maintain reasonable staffing ratios, but with low salaries.

At the other end of the spectrum are those schools belonging to the major independent schools associations (National Association of Independent Schools and regional affiliates, National Independent Private Schools Association), which includes the most elite (and high priced) private independent day schools (boarding schools were excluded from the analysis). These were also the schools for which the most comprehensive fiscal data were available. Tuition rates for these schools were nearly 50% higher than nearby district per pupil spending and per pupil spending nearly double nearby district per pupil spending.  These institutions in particular, have the opportunity to leverage resources creatively.

 Table 3

Slide43

As Table 3 shows, more elite private independent schools tend to leverage their resource advantage primarily in dramatically reduced pupil to teacher ratios of <10:1 compared to nearly 17:1 for nearby districts.  While annual salaries for otherwise comparable teachers are somewhat lower for private independent school teachers, they are higher than for other private school teachers. Further, elite private independent schools tend to operate shorter school years (by 10 or more days) compared to district schools, and independent school teachers serve much lower total student loads (total students taught per day, week or year) than their public district counterparts. Again, the financial resources translate to human resources, with some insightful differences in the trade-offs between staffing quantities and salaries and intertwined factors like numbers of students and courses taught.

One notable disruptive innovation of elite private schooling which continues to drive the human resource intensive model at the secondary level was the introduction in 1930 of large oval wooden tables (akin to corporate boardroom tables) which typically seat a maximum of 14 (some up to 18) to encourage student and faculty interaction.  Harkness Tables and the accompanying philosophy were introduced as a gift by wealthy philanthropist Edward Harkness to the elite New Hampshire boarding school Phillips Exeter Academy, which continues to promote Harkness learning in its marketing.[xvi] Harkness tables persist to this day as central to the teaching and learning philosophy of many elite private independent schools. But they are also a testament to the preference for a human resource intensive model of schooling involving small interactive classes staffed with highly qualified academic instructors.  To the extent that these schools also have the flexibility to introduce what modern edupreneurs might consider disruptive technologies (tablets & hand held devices, cloud-based computing, interactive video, white-boards, etc.), these schools are more likely to take a both-and rather than either-or path. That is, i-pads around the Harkness Table rather than in place of it.

Take Home Points

The central take home point of this post is that on average, the financial resources available to schools, whether public district (previous posts), charter or private schools generally translate to human resources. Financial resources translate to the quantities of staff which can be hired or retained and they translate to the wages that can be paid. And both of these matter.

To summarize:

  1. Schooling is human resource intensive, whether traditional public, charter or private;
  2. When schools have more money, they invest it in more staff and better paid staff and when they don’t, they can’t;
  3. Public systems enjoy the scale efficiencies which mitigate administrative and other overhead expenses, including construction, maintenance and operations of capital;
  4. Compensation for staffing across sectors remains largely a function of years of experience, credentials earned and contractual hours worked.

The third and fourth points were less fully elaborated in this post, but are equally important.  As noted in this post, charter schools often divert resources from direct instruction and personnel NOT to achieve greater flexibility and adopt disruptive technologies, but rather because they lack economies of scale and they also often lack access to capital needed to access necessary land and buildings.[i] My work on private schools also revealed significantly higher administrative expenses than for public schools partly due to scale differences, but also due to the choice to pay top administrators much higher salaries than superintendents of much larger public districts.[ii] Finally, despite much talk of creative, alternative, performance based compensation for teachers, my related work on charter and private schools finds that the primary determinants of differences in pay across teachers are their years of experience, the degrees and credentials they hold, and the amount of time they are expected to work and students they are expected to serve.[iii]

This is not to suggest that this is the way it should be or will always be. Rather, this is how schools currently work in the United States (and around the world for that matter). This is how schools use money, whether large public bureaucracies, publicly subsidized private operators, or fully independent private operators. As of yet, researchers, policymakers, pundits, pontificators and even self-proclaimed “thought leaders” have yet to conjure some new secret sauce or technological innovation which can greatly improve equity, adequacy and efficiency.  Human resources matter and equitable and adequate financial resources are necessary for hiring and retaining the teachers and other school staff necessary to achieve equal educational opportunity across all children. Understanding these baseline conditions and relationships is prerequisite for moving forward.

NOTES

[i] Preston, C., Goldring, E., Berends, M., & Cannata, M. (2012). School innovation in district context: Comparing traditional public schools and charter schools. Economics of Education Review, 31(2), 318-330.

[ii] Michael Q. McShane and Jenn Hatfield (2015) Measuring Diversity in Charter School Offerings. Washington, DC: American Enterprise Institute. http://www.aei.org/wp-content/uploads/2015/07/Measuring-Diversity-in-Charter-School-Offerings.pdf

[iii] Baker, B. D., Libby, K., & Wiley, K. (2012). Spending by the Major Charter Management Organizations: Comparing Charter School and Local Public District Financial Resources in New York, Ohio, and Texas. National Education Policy Center.

[iv] Epple, D., Romano, R., & Zimmer, R. (2015). Charter schools: a survey of research on their characteristics and effectiveness (No. w21256). National Bureau of Economic Research.

Toma, E., & Zimmer, R. (2012). Two decades of charter schools: Expectations, reality, and the future. Economics of Education Review, 31(2), 209-212.

[v] Gleason, P. M., Tuttle, C. C., Gill, B., Nichols-Barrer, I., & Teh, B. R. (2014). Do KIPP schools boost student achievement?. Education Finance & Policy, 9(1), 36-58.

Angrist, J. D., Dynarski, S. M., Kane, T. J., Pathak, P. A., & Walters, C. R. (2012). Who benefits from KIPP?. Journal of policy Analysis and Management, 31(4), 837-860.

Dobbie, W., Fryer, R. G., & Fryer Jr, G. (2011). Are high-quality schools enough to increase achievement among the poor? Evidence from the Harlem Children’s Zone. American Economic Journal: Applied Economics, 3(3), 158-187.

Dobbie, W., & Fryer Jr, R. G. (2013). Getting beneath the veil of effective schools: Evidence from New York City. American Economic Journal: Applied Economics, 5(4), 28-60.

[vi] Baker, B. D., Libby, K., & Wiley, K. (2012). Spending by the Major Charter Management Organizations: Comparing Charter School and Local Public District Financial Resources in New York, Ohio, and Texas. National Education Policy Center.

Baker, B. D., Libby, K., & Wiley, K. (2015). Charter School Expansion and Within-District Equity: Confluence or Conflict?. Education Finance and Policy.

[vii] Baker, B. D., Libby, K., & Wiley, K. (2012). Spending by the Major Charter Management Organizations: Comparing Charter School and Local Public District Financial Resources in New York, Ohio, and Texas. National Education Policy Center.

[viii] Arsen, D. D., & Ni, Y. (2012). Is administration leaner in charter schools? Resource allocation in charter and traditional public schools. education policy analysis archives, 20, 31.

[ix] Izraeli, O., & Murphy, K. (2012). An Analysis of Michigan Charter Schools: Enrollment, Revenues, and Expenditures. Journal of Education Finance, 37(3), 234-266.

[x] Bifulco, R., & Reback, R. (2014). Fiscal Impacts of Charter Schools: Lessons from New York. Education Finance & Policy, 9(1), 86-107.

[xi] Epple, D., Romano, R., & Zimmer, R. (2015). Charter Schools: A Survey of Research on Their Characteristics and Effectiveness (No. w21256). National Bureau of Economic Research.

[xii] Taylor, L.L. Alford, B.L., Rollins, K.G., Brown, D.B., Stillisano. J.R., Waxman, H.C. (2011) Evaluation of Texas Charter Schools 2009-2010 (Revised Draft). Texas Education Research Center. Texas A&M University, College Station.

[xiii] Harmony Schools are actually subsumed under the Cosmos Foundation Inc. and Harmony and Cosmos under the umbrella of the national network of Gulen charter schools.  http://gulencharterschools.weebly.com/

[xiv] Gronberg, T. J., Jansen, D. W., & Taylor, L. L. (2012). The relative efficiency of charter schools: A cost frontier approach. Economics of Education Review, 31(2), 302-317.

[xv] Baker, B. D. (2009). Private Schooling in the US: Expenditures, Supply, and Policy Implications. Education Policy Research Unit.

[xvi] https://exeter.edu/exeter-difference/how-youll-learn

Additional Notes

[i] Baker, B., & Miron, G. (2015). The business of charter schooling: Understanding the policies that charter operators use for financial benefit. Boulder, CO: National Education Policy Center. Retrieved January, 27, 2016.

[ii] Baker, Bruce D. “Private Schooling in the US: Expenditures, Supply, and Policy Implications.” Education Policy Research Unit (2009).

[iii] http://www.bostonglobe.com/metro/2017/09/03/most-boston-charter-schools-reject-performance-based-pay-for-teachers/i64tLVRwL9WqsRxzJ6Z7XJ/story.html?platform=hootsuite

 

Reality Check: Edupreneurs, “Tech-based Solutions” & Misguided Innovation

Elementary and secondary schooling, regardless of sector – public, publicly authorized and financed charter, or private – remains a human resource intensive industry.  Quality schooling requires sufficient numbers of sufficiently trained teachers, support staff, administrators and others to get the job done.  There must be enough of them and they must be sufficiently compensated to attract quality candidates into the field.

The share of school spending allotted to salaries and benefits of employees has been remarkably stable over time, at about 81% (Figure 10). The share allocated to employee benefits has climbed (about 6%) and has been offset by a commensurate decline in the share allocated to salaries and wages.

Figure 10

Slide13.JPG

Reformers and edupreneurs[i] have spent decades diddling around the edges of education systems pursuing the ultimate technological innovations. In the broad sense, technological innovation for improving productivity and efficiency refers to any change in the way things are done – the organization and deployment of staff, delivery of programs, use of materials, supplies, etc. where those changes are brought about by introduction of some new technology – as simple as pens and pencils replacing slate tablets. The idea is that the introduction of new technology and resulting change in the way things get done, leads to improved efficiency – lower cost of getting the same outcomes, or same cost of getting even better outcomes!  An extension of this concept, “Disruptive Innovation” refers to major shifts in the way things are done. Critics of public education systems assert that disruptive innovation is needed to completely overhaul our stagnant, costly, inefficient and failing system.

Modern edupreneurs and disrupters seem to have taken a narrow view of technological substitution and innovation, equating technology almost exclusively with laptop and tablet computers – screen time – as potential replacements for teachers – whether in the form of online schooling in its entirety, or on a course by course basis (unbundled schooling).[ii] For example, the often touted Rocketship model (a chain of charter schools), makes extensive use of learning lab time in which groups of 50 to 70 (or more) students work on laptops while supervised by uncertified “instructional lab specialists.”[iii] Fully online charter schools have expanded in many states often operated as for-profit entities.[iv] The overarching theme is that there must be some way to reduce the dependence on human resources to provide equal or better schooling, because human resources are an ongoing, inefficient expense.

In 2011, on the invitation of New York State Commissioner of Education John King (later, replacement of Arne Duncan as U.S. Secretary of Education), Marguerite Roza, at the time a Senior Economic and Data Advisor to the Bill & Melinda Gates Foundation,[v] presented the Productivity Curve illustration (Figure 11) at a research symposium of the New York State Board of Regents.[vi] Roza used her graph to assert that, for example, for $20,000 per pupil, tech-based learning systems could provide nearly 4x the bang for the buck as the status quo, and double the bang for the buck as merely investing in improved teacher effectiveness.

The most significant shortcoming of this graph, however, was that it was entirely speculative[vii] (actually, totally made up! Fictional!) – a) not based on any actual empirical evidence that such affects could be or have anywhere been achieved, b) lacking any definition whatsoever as to what was meant by “tech-based learning systems” or “improve teacher effectiveness”, and c) lacking any information on the expenditures or costs which might be associated with either the status quo or the proposed innovations. That is, without any attention to the cost effectiveness frameworks I laid out in the previous chapter. The graph itself was then taken on the road by Commissioner King and used in his presentations to district superintendents throughout the state![viii]

Figure 11

Slide11.JPG

In a comprehensive review of literature on the state of charter schooling in the U.S., including online charter schooling, authors Epple, Romano and Zimmer paint a rather ugly picture of the outcomes achieved by fully online “cyber” charter schools, noting: “online ‘cyber’ schools appear to be a failed innovation, delivering markedly poorer achievement outcomes than TPSs.”[ix] (p. 55) Perhaps we just haven’t figured out yet how to most effectively and efficiently deploy mass substitution of human resources with computing technology. Or, perhaps, such mass substitution simply isn’t productive. Most disturbingly, expansion of these alternatives has not been accompanied by serious efficiency or cost effectiveness analysis. Indeed it’s possible that the outcome reductions measured are consistent with the associated expenditure reductions, thus a break-even on efficiency.

That is, doing less for less.  

Notes

[i] http://www.gettingsmart.com/2013/09/rise-edupreneur/

[ii] Baker, B. D., & Bathon, J. (2013, October). Financing Online Education and Virtual Schooling. National Education Policy Center.

[iii] http://www.npr.org/sections/ed/2016/06/24/477345746/high-test-scores-at-a-nationally-lauded-charter-network-but-at-what-cost

[iv] Weber, M., & Baker, B. (2017). Do For-Profit Managers Spend Less on Schools and Instruction? A national analysis of charter school staffing expenditures. Educational Policy, 0895904816681525.

[v] http://www18.georgetown.edu/data/people/mr1170/cv.doc

[vi] http://www.p12.nysed.gov/mgtserv/docs/SchoolFinanceForHighAchievement.pdf

[vii] Note that the lines in the graph are actually drawn in, and not graphed from data. Also, the lines imply that at $5,000 per pupil expenditure, regardless of approach – status quo, improved teacher effectiveness, or tech-based solutions, nothing can be accomplished, though it is unclear whether the Y-axis represents an absolute standard, or change, or relative to what?

[viii] http://www.p12.nysed.gov/docs/commissioner-nyscoss-presentation-092611.pdf

[ix] Epple, D., Romano, R., & Zimmer, R. (2015). Charter schools: A survey of research on their characteristics and effectiveness (No. w21256). National Bureau of Economic Research.

Reality Check: From Revenues to Real Resources

Basic Model: From Revenue Sources to Classroom Resources

The basic model linking available revenues, schooling resources and ultimately student outcomes remains relatively simple. Setting aside federal revenue for the moment, which is about 10% of education revenue on average, Figure 12 illustrates that the amount of revenue state and local education systems tend to have is a function of both a) fiscal capacity and b) fiscal effort.  On average, some states put forth more effort than others and some states have more capacity to raise revenue than others. These differences lead to vast differences in school funding across states, and subsequently to vast differences in classroom resources.  Federal aid is far from sufficient to close these gaps. Similarly, within some states, local districts have widely varied capacity to raise revenue for their schools, and some differences in revenues also result from differences in local effort. State effort to close these gaps varies widely.

Whether at the state level on average, or at the local level, the amount of revenue available dictates directly the amount that can be spent. Current operating expenditures are balanced primarily between the quantity of staff employed, often reflected in pupil to teacher ratios or class sizes, and the compensation of those employees including salaries and benefits. Therein lies the primary tradeoff to be made in school district planning and budgeting. Yes, one could trade off teachers for computers (or “tech-based solutions”), but most schools don’t and those that do have not produced impressive results. At any given budget constraint, staffing quantities can be increased at the expense of wages, or wages increased at the expense of quantities. The combination of the two must be sufficient to achieve desired outcomes.

Figure 12

Slide14

These connections matter from an equity perspective, from an adequacy perspective and from productivity and efficiency perspectives.

From an equity perspective, if revenue sources remain unequal, leading to unequal expenditures, those inequities will lead to unequal staffing ratios and/or wages, resulting in unequal school quality and eventually outcomes. The argument that schools and districts with less money simply need to use it better – figure out how to hire better teachers in each position, with lower total payroll, is an inequitable requirement. This argument is based on loosely on the popular book Moneyball, which touted the success of the Oakland Athletics, applying new metrics to recruit undervalued players. But this argument forgets the book’s subtitle “the art of winning an unfair game,” a reference to the difficulties of small market teams competing successfully in a sport which only loosely regulates payroll parity. Clearly, baseball would be a fairer game if all teams could fund equal total payroll. While it’s up to Major League Baseball to decide whether greater parity across teams – a fairer game – is preferable (and profitable), public education shouldn’t be an unfair game.

From an adequacy perspective, sufficient funding is a necessary condition for hiring sufficient quantities of qualified personnel, to achieve sufficient outcomes.  Finally, from a productivity and efficiency perspective, we do know that human resource intensive strategies tend to be productive, and, as of yet, we have little evidence of substantial efficiency gains from technological substitutions resulting in substantial reduction of human resources.  Resource constrained schools and districts which already have large class sizes and non-competitive wages are unable to trade off one for the other. Put bluntly, if you don’t have it, you can’t spend it.

Variations in Revenues and Classroom Resources

Across states, average state and local revenues are partly a function of differences in “effort” and partly a function of differences in “capacity.” [i] Figure 13 shows the relationship across states, in 2015 between our measure of “capacity” – gross domestic product (GDP state) – and total state and local revenues per pupil. Differences in state GDP explain about 42% of the differences in state and local revenue. Higher capacity states like Massachusetts, Connecticut, New York, Wyoming and Alaska tend to raise more revenue for schools. Vermont stands out as a state where capacity is below the median but revenue is very high. Mississippi stands out as a state with very low capacity and very low revenue.

Figure 13

Slide15

[1] per pupil revenue projected for district with >2,000 pupils, average population density, average labor costs and 20% children in poverty[ii]

Figure 14 shows the relationship between a common measure of “effort” and state and local revenues. The effort measure is the ratio of state and local revenues to gross state product. That is, what percent of economic capacity are states spending on schools? Differences in effort explain about 1/3 of differences in state and local revenue per pupil. Despite having low capacity, Vermont has very high effort which results in high revenue levels. Other higher capacity states like Connecticut, Massachusetts and New York are able to raise relatively high revenue levels with much lower effort. Mississippi by contrast leverages above average effort, but because of its very low capacity, simply can’t dig itself out. Other states like Arizona and North Carolina, which have much greater capacity than Mississippi, simply choose not to apply effort toward raising school revenues.

Figure 14

Slide16

[1] per pupil revenue projected for district with >2,000 pupils, average population density, average labor costs and 20% children in poverty

Figure 15 displays the rather obvious relationship that, the more state and local revenue per pupil raised in a state, the more that is spent per pupil.

Figure 15

Slide17

[1] per pupil revenue projected for district with >2,000 pupils, average population density, average labor costs and 20% children in poverty

[2] current spending per pupil projected for district with >2,000 pupils, average population density, average labor costs and 20% children in poverty

Figure 16 shows that greater per pupil spending generally leads to more teachers per 100 pupils. Very low spending states (also low effort) like Arizona and Nevada tend to have very low staffing ratios compared to other states.

Figure 16

Slide18

[2] current spending per pupil projected for district with >2,000 pupils, average population density, average labor costs and 20% children in poverty

[3] teachers per 100 pupils projected for district with >2,000 pupils, average population density, average labor costs and 20% children in poverty

Figure 17 shows that states spending more per pupil, on average, tend to have more competitive teacher wages. Here, the indicator of competitive wages is the ratio of salaries of elementary and secondary teachers to same age, similarly educated non-teachers in the same labor market within each state.  That is, the competitiveness of teacher wages is relative. As indicated by decades of teacher labor market research, the relative competitiveness of teacher wages influences the quality of candidates who enter teaching as a profession.[iii] Teacher compensation is especially poor in states like Arizona, which has among the lowest per pupil spending in the nation. By contrast, teacher compensation is highly competitive in Vermont and Wyoming, both higher spending states, but also states where non-teacher compensation is not particularly high, compared to New York, New Jersey and Connecticut.

Figure 17

Slide19

District Spending Determines School Resources (Illinois Examples)

Similar disparities play out at the local level, especially in those states which have done the least to resolve disparities between rich and poor districts.  Year after year, a report on which I am an author – Is School Funding Fair? – identifies Illinois and Pennsylvania as among the most disparate states in the nation. Specifically, in these two states, districts serving high poverty student populations continue to have far fewer resources per pupil than their more advantaged counterparts. Here, I use Illinois as an example of the resulting relationship between a) district level spending variation, b) school site spending variation, and c) on-the-ground resources including staffing assignments per pupil.  These connections seem obvious. If a district has sufficient revenues, those revenues can be used to support the schools in that district, leading to more and better paid teachers. If a district doesn’t have the resources, neither will the schools in that district.

I raise this issue because, over the past decade, federal policy (and the think tanks which inform federal policy) have placed disproportionate emphasis on the need to resolve inequities in spending across (particularly staffing expenditures) schools within districts, while ignoring inequities in spending between districts.  But, especially in highly inequitable states like Illinois, resolving between school inequities in poorly funded school districts is akin to re-arranging deck chairs on the Costa Concordia.[iv]

Figure 18 shows the relationship between district level operating expenditures per pupil (horizontal axis) and school site staffing expense per pupil on the vertical axis, for the Chicago metropolitan area in 2012. Only a few districts have large numbers of schools across which resources might vary. The City of Chicago schools appear at approximately the $15,000 per pupil point on the horizontal axis as a vertical pattern of circles. Circle size indicates school enrollment. While there appears to be substantial variation in resources across Chicago schools, much (about half) of that variation is a function of differences in student populations (special education in particular) and grade ranges served.

On average, Figure 18 shows that schools in districts that spend more per pupil overall, spend more per pupil at the school level on staffing. 44% of the variation in staffing expenditure across schools is driven by spending differences across districts. And about half of the remaining differences in staffing expenditure across schools are driven by the distribution of special education populations across schools and district targeting of staff to serve those children.[v]  That is, much of the within district variation that does exist is rational. Resources that flow to districts pay for staff that work in schools. It really is that simple. If the district has more revenue coming in, it can spend more, and that spending shows up in the form of more and better paid teachers in schools.

Figure 18

Slide21

Figure 19 shows how, in the Chicago metropolitan area, differences in district level operating expenditures lead to differences in competitive wages for teachers.  Here, competitiveness of wages are measured in terms of the salaries of teachers with specific numbers of years of experience and degree levels (contract hours and days per year, grade level taught, and job assignment), compared to their peers in other schools in the Chicago metro area. Salary competitiveness measured in this way reflects the recruitment and retention potential of districts, for those who have already chosen to teach. Salary differentials within teaching may influence teacher sorting across schools and districts.

Figure 19 shows that teachers in low spending districts tend to have salaries at about 75% of the average teacher with similar credentials in a similar assignment. Teachers in high spending districts tend to have salaries around 25% higher than the average teacher in a similar position with similar credentials.

Figure 19

Slide22

Finally, Table 1 summarizes features of Illinois school districts which I identified in a paper in 2012[vi] as falling into quadrants where the upper right (advantaged) quadrant were districts with high spending and high outcomes and the lower left (disadvantaged) were those with low spending and low outcomes. To identify advantaged and disadvantaged school districts, spending was adjusted for the costs of providing equal educational opportunity to all students by methods discussed in a forthcoming post. Nominal and adjusted spending levels are reported in Table 1.

In my analysis, there were 146 high spending high outcome elementary districts serving over 250 thousand children and 120 low spending low outcome districts serving about 150 thousand children. There were 41 high spending high outcome secondary districts serving up to 150 thousand children and 25 low spending low outcome secondary districts serving about 50 thousand children. For unified districts, there were 82 that were high spending and high outcome, serving 250 thousand children and 156 that are low spending with low outcomes, serving over 800 thousand children, with about half of those children attending Chicago Public Schools.

Even without any adjustment for costs or needs, the average per pupil operating expenditures are lower in low spending, low outcome districts. The percent of children who are low income is substantially higher in low spending, low outcome districts. Table 1 shows that the district operating expenditure advantages of the high resource – high outcome district schools translate directly to a) more staff per pupil and b) better paid staff per pupil. In 2009, high resource schools had 86 and 79 teacher assignments per 1000 pupils in elementary and secondary schools respectively, compared to 79 and 72 for low resource schools. Further, for teachers with similar training and experience in similar assignments, teachers in low resource schools were paid $7,000 to $7,500 less in salary. That is, schools in financially advantaged districts have both more staff per pupil and are able to pay them more, while serving far less needy student population.

Slide23

The take-home point of this post is that on average, the financial resources available to schools generally translate to human resources. Financial resources translate to the quantities of staff which can be hired or retained and they translate to the wages that can be paid. And both of these matter, as will be discussed more extensively in future posts.  To summarize:

  1. Schooling is human resource intensive, whether traditional public, charter or private;
  2. When schools have more money, they invest it in more staff and better paid staff and when they don’t, they can’t!

More later on whether charter schools and private schools have much to offer in the way of technological innovation, substitution and creative resource allocation.

NOTES

[i] Baker, B. D. (2016). School Finance & the Distribution of Equal Educational Opportunity in the Postrecession US. Journal of Social Issues, 72(4), 629-655.

[ii] Baker, B.D., Srikanth, A., Weber, M.A. (2016). Rutgers Graduate School of Education/Education Law Center: School Funding Fairness Data System. Retrieved from: http://www.schoolfundingfairness.org/data-download

[iii] Baker, B. D. (2016). Does money matter in education?. Albert Shanker Institute.

[iv] Baker, B. D. (2012). Rearranging deck chairs in Dallas: Contextual constraints and within-district resource allocation in urban Texas school districts. Journal of Education Finance, 37(3), 287-315.

[v] Baker, B.D., Srikanth, A., Weber, M. (2017) The Incompatibility of Federal Policy Preferences for Charter School Expansion and Within District Equity.

[vi] Baker, B. D. (2012, March). Unpacking the consequences of disparities in school district financial inputs: Evidence from staffing data in New York and Illinois. In annual meeting of the Association for Education Finance and Policy, Boston, MA.

Realty Check: Trends in School Finance

Over the next few months, I plan to share some rough drafts of forthcoming work – Here, I present a brief reality check on what is often referred to as the “long term trend” argument proving that money doesn’t matter for schools.  Here’s Bill Gates’ version of that argument:

Over the past four decades, the per-student cost of running our K-12 schools has more than doubled, while our student achievement has remained virtually flat. Meanwhile, other countries have raced ahead.[i]

Bill Gates, Microsoft co-founder

[i] http://www.washingtonpost.com/wp-dyn/content/article/2011/02/27/AR2011022702876.html

Reality Check

Here, I look at long term national trends in actual student outcomes and in school resources.  Recall that the need for disruptive reform and magic elixirs derives primarily from our massive increases in spending coupled with our virtually flat student outcomes over time.  But are those claims even remotely correct?

Richard Rothstein of the Economic Policy Institute immediately critiqued Bill Gates’ assertions of “virtually flat” student outcomes in a memo titled “Fact-Challenged Policy.” Rothstein (2011) shows that, in fact, “On these exams [National Assessment of Educational Progress], American students have improved substantially, in some cases phenomenally.” (p. 1)[i] Related work by Rothstein and colleague Martin Carnoy (2013) confirms that, when accounting for differences in student disadvantage, U.S students perform much better than what is suggested by commonly cited, unadjusted rankings that fail to account for changes in subgroup proportions when aggregating test results.[ii]

In 2010, Educational Testing Service (ETS) released “The Black-White Achievement Gap: When Progress Stopped,” a report by Paul Barton and Rich Coley (2010), in which the authors explored the Black-White achievement gap from the 1970s to recent years.[iii] The goal of that report was to explore trends in Black-White achievement gaps, and changing conditions which may explain those trends.  Barton and Coley explained that “From the early 1970s until the late 1980s, a very large narrowing of the gap occurred in both reading and mathematics, with the size of the reduction depending on the subject and age group examined.” (p. 7) Reductions to achievement gaps were particularly pronounced in reading among 13 and 17 year olds, while still significant in mathematics. However, “During the 1990s, the gap narrowing generally halted, and actually began to increase in some cases.” (p. 7). The authors note some additional gap narrowing from 1999 to 2004 and mixed findings from 2004 to 2008. Rothstein (2011) shows that, even during the period from 1990 to 2008, achievement gains for Black 4th and 8th grade students have been substantial in mathematics in particular, and have outpaced their White peers.[iv]

Figure 3 displays the long term trends for black and white children at age 13 on the NAEP Long Term Trend assessment.  Both black and white scores trend upward, and, as noted by Barton and Coley, Black scores increase significantly from the 1970s through 1990.

Figure 3

NAEP long term trends on reading and math for 13 year olds

Slide4

Figure 4 shows more recent trends for children by income status for Reading and Figure 5 shows the trends for math at 4th and 8th grade. For reading, fourth grade scores have continued to trend upward for all groups, but for eight graders, dip slightly in 2015. For math, the overall upward trend is also consistent across grades, but also with a dip in 2015.  It would be premature to assume any causation for the 2015 dip.

Figure 4

NAEP reading by income status

Slide5

Figure 5

NAEP math by income status

Slide6

Figure 6 compares nominal (not inflation adjusted) current spending per pupil to current spending per pupil adjusted for the costs of maintaining competitive wages over time. The previous figure – THE GRAPH – uses a consumer price index, which is less appropriate for evaluating the value of the education dollar over time.  By 2015, the average school district nationally was roughly at a 10 year break-even point on per pupil spending. That is, per pupil spending hasn’t risen for a decade and has barely risen for over two decades (2.5%).  So, no, school spending is not dramatically increasing over time and has declined in real terms from 2009 to 2015, the most recent national district level data.

Figure 6

Current Operating Expenditures per Pupil Adjusted for Labor Costs

Slide7

            Over the longer term, Figure 7 shows that direct government expenditure on elementary and secondary education as a share of gross domestic product has oscillated over decades but is presently about where it was both 15 years earlier (2000) and 40 years earlier (1975). That is, education spending is not outstripping our economic capacity to pay for it.

Figure 7

Direct Education Expense as a Share of Gross Domestic Product

Slide8.JPG

Some have asserted that along with ballooning spending over time has been a staffing surge. That is, schools have hired more and more people and have especially hired more and more individuals with less direct classroom contact time – administrative and support staff, including specialists.[v] There is some truth to the fact that teaching specialist and school level administrative positions have increased over time, in part because of additional legal protections for specialized programs and services for children with disabilities.[vi]  But, as Figure 8, which includes all staff classified as teachers (specialists among them), total numbers of teachers per each 100 pupils are at a break-even point from about 12 to 14 years prior (2001 to 2003). That is, since the early 2000s (when NAEP progress seemed to slow down), teaching staff have remained relatively stagnant. There was a modest bump in the mid-2000s, which subsided during the “new normal” period and has not since rebounded. Notably, a recent comprehensive meta-analysis of interventions which improve outcomes for low income students found specifically that the most effective interventions were those involving increased human resources.[vii]

Figure 8.

Teachers (all) per 100 pupils over time

Slide9

Finally, Figure 9 shows the ratio of teacher weekly wages to those of college educated non-teachers from 1979 to 2015, estimated by authors from the Economic Policy Institute. For the past 15 years, teacher wages have held constant at about 77% of non-teacher wages. Some assert that teacher’s healthy and growing pension benefits are a substantial offset to this gap, noting that teacher benefits as a share of teacher wages have increased from about 10% to about 20% of wages since the early 2000s, whereas private sector benefits have held constant at 10% wages. [viii] One problem is that the difference is expressed in percent terms, and the supposed 20% rate for teachers is over a wage that is 23% lower to begin with.  Of course, it’s a larger and growing percent of a lower and declining wage. Even taking that percent at face value as correct, would at best raise teacher wages to about 84% of non-teacher wages in 2015.[ix]

Figure 9

Ratio of teacher weekly wages to college educated non-teacher weekly wages

Slide10

To summarize:

  • NAEP scores have increased substantively over the long term, but have slowed in growth more recently, as has closure of racial achievement gaps;
  • School spending has been relatively stagnant for the past decade and so too have staffing quantities and the competitiveness of teacher wages;
  • Over the shorter term, since 2008, spending, staffing and wage competitiveness have all declined.

These long term trends are roughly the opposite of those used most often to proclaim the failures of American education.  One implication of these trends is that U.S. public schools have, in fact, become more efficient over time, not less.

[i] Rothstein, R. (2011). Fact-Challenged Policy. Policy Memorandum# 182. Economic Policy Institute.

[ii] Carnoy, M., & Rothstein, R. (2013). What do international tests really show about US student performance. Economic Policy Institute, 28.

[iii] Barton, P. E., & Coley, R. J. (2010). The Black-White Achievement Gap: When Progress Stopped. Policy Information Report. Educational Testing Service.

[iv] On these exams, American students have improved substantially, in some cases phenomenally. In general, the improvements have been greatest for African-American students, and among these, for the most disadvantaged. The improvements have been greatest for both Black and White 4th and 8th graders in mathematics. Improvements have been less great but still substantial for Black 4th and 8th graders in reading and for Black 12th graders in both mathematics and reading. Improvements have been modest for Whites in 12th grade mathematics and at all three grade levels in reading (Rothstein, 2011).

[v] Scafidi, B. (2012). The School Staffing Surge: Decades of Employment Growth in America’s Public Schools. Friedman Foundation for Educational Choice.

[vi] Roy, J. (2012). Review of The School Staffing Surge. National Education Policy Center, 3

[vii] As summarized by ANOVA blogger Frederik DeBoer here: https://fredrikdeboer.com/2017/05/16/study-of-the-week-what-actually-helps-poor-students-human-beings/ see also: Dietrichson, J., Bøg, M., Filges, T., & Klint Jørgensen, A. M. (2017). Academic interventions for elementary and middle school students with low socioeconomic status: A systematic review and meta-analysis. Review of Educational Research, 87(2), 243-282.

 

[viii] http://www.uaedreform.org/downloads/2017/04/employer-contributions-for-retirement-4-4-17.pdf

[ix] Teacher Weekly Wage2015                               = $1,092 x 1.20 = $1,311

College Graduate Weekly Wage2015                   = $1,416 x 1.10 = $1,557

Adjusted Share      =   84%

 

The Sweeny-Prieto School Aid Proposal: An Analysis

Source: The Sweeny-Prieto School Aid Proposal: An Analysis

Mark Weber

Doctoral Candidate

Rutgers, The State University of New Jersey

POLICY BRIEF: Weber_SweeneyPrieto_June26_2017

SUPPLEMENTAL FILE (Regression Output): SweeneyPrietoLog

Executive Summary

This brief presents an analysis of the school funding plan presented by New Jersey Senate President Steve Sweeney and Assembly Speaker Vincent Prieto, referred to here as “Sweeney-Prieto.” The proposal:

  • Will drive more aid to districts with higher proportions of Hispanic, free lunch-eligible, and LEP students.
  • Will drive less aid toward districts with students classified as having a special education need.
  • Will drive more aid on average to districts in the CD District Factor Group; however, there is great variety among these districts, with some losing significant amounts of aid.
  • Will give less aid to very small districts.
  • Will drive aid towards districts making greater local taxing effort, holding school cost and taxing capacity

While this last characteristic makes Sweeney-Prieto more “fair” overall, there are still individual districts that are receiving significantly less or more aid than would be predicted by measures of cost, capacity, and effort.

In addition, the aid allocated under Sweeney-Prieto is less than 2 percent of the aid proposed by the governor’s budget for FY18; the proposal, therefore, has little overall effect on the bringing New Jersey’s school budgets to adequacy as designated by the state’s own funding law.

Based on these conclusions, I offer the follow recommendations:

  • Policymakers should ensure that those districts receiving significantly less aid per pupil under Sweeny-Prieto – particularly those whose changes in aid are far under prediction – do not suffer undue harm from the proposal.
  • Lawmakers should carefully consider the unintended consequences of basing the reallocation of aid largely on factors such as the Growth Cap or Adjustment Aid, and adjust the allocation of aid accordingly.
  • All stakeholders should realize the scale of Sweeney-Prieto renders it largely ineffective in making up for the chronic underfunding of SFRA over the last eight years.

 

Public* Goods & the Money Belongs to the Child Fallacy (in tweets)

//platform.twitter.com/widgets.js

//platform.twitter.com/widgets.js

//platform.twitter.com/widgets.js

//platform.twitter.com/widgets.js

//platform.twitter.com/widgets.js

//platform.twitter.com/widgets.js

//platform.twitter.com/widgets.js

//platform.twitter.com/widgets.js

//platform.twitter.com/widgets.js

//platform.twitter.com/widgets.js

//platform.twitter.com/widgets.js

//platform.twitter.com/widgets.js

//platform.twitter.com/widgets.js

//platform.twitter.com/widgets.js

//platform.twitter.com/widgets.js

//platform.twitter.com/widgets.js

//platform.twitter.com/widgets.js

 

*Education, or public schooling (public school systems) in particular is not typically considered a “public good” as the provision of public schooling does not comply with the definition of a “pure public good” which can be equally accessed by all, without reduction in benefits to any.  The intent here (in the above tweet-storm) was to shed some light on the importance of understanding the role/position of these publicly financed education systems in society and that there’s more to these systems than the year to year provision of “schooling” to those who happen to be school aged in a specific community at a specific point in time.

Thoughts on Junk Indicators, School Rating Systems & Accountability

Over the past few years, co-authors Preston Green and Joseph Oluwole and I have written a few articles on the use, misuse and abuse of student growth metrics for evaluating teachers and for imposing employment consequences based on those metrics. Neither of our previous articles addressed the use of even more nonsensical status and status change measures for deciding which schools to close or reconstitute (with significant implications for employment consequences, racial disparities, and reshaping the racial makeup of the teacher workforce).

I have written a few posts on this blog in the past regarding school rating systems.

I’ve also tried to explain what might be appropriate and relevant uses of testing/assessment data for informing decision/policymaking:

It blows my mind, however, that states and local school districts continue to use the most absurdly inappropriate measures to determine which schools stay open, or close, and as a result which school employees are targeted for dismissal/replacement or at the very least disruption and displacement. Policymakers continue to use measures, indicators, matrices, and other total bu!!$#!+ distortions of measures they don’t comprehend, to disproportionately disrupt the schools and lives of low income and minority children, and the disproportionately minority teachers who serve those children. THIS HAS TO STOP!

Preston, Joseph and I previously explained problems of using value-added and growth percentile measures to evaluate and potentially dismiss teachers in cases where those measures led to racially disparate dismissals (under Title VII). We explain that courts would apply a three-part analysis as follows:

Courts apply a three-part, burden-shifting analysis for Title VII disparate impact claims.”-‘ First, the plaintiffs must establish a prima facie case, showing that a challenged practice has an adverse impact on a minority group.”‘* Once the plaintiffs have established a prima facie case, the burden shifts to the employer to show that the employment practice in question has a “manifest relationship to the employment”;”^ in other words, the employer has to show a “business justification.””^ If the employer satisfies this requirement, the burden then shifts to the plaintiffs to establish that less discriminatory alternatives exist.’ ‘^

In other words, showing the disparate effect is the first step. But that doesn’t alone mean that a policy or the measures it relies upon are wrong/unjustifiable/indefensible. That is, if the defendant (state, local district, “employer”) can validate that those measures/policy frameworks have a “manifest relationship” to the employment, then current policy may be acceptable.  Plaintiffs still, though, have opportunity to show that there are less discriminatory alternatives which are at least equally justifiable.

What follows are some preliminary thoughts in which I consider the “usual” measures in state and local school rating policies, and how one might evaluate these measures were they to come under fire in the context of Title VII litigation. It actually seems like a no-brainer, and waste of time to write about this, since few if any of the usual measures and indicators have, from a researchers perspective, “manifest relationship to the employment.” But, the researchers perspective arguably sets too high a bar.

Types of Measures & Indicators in School Accountability Systems: An overview

I begin with a clarification of the distinction between “measures” and “indicators” in the context of elementary and secondary education policy:

  • Measures: Measures are based on attributes of a system to which we apply some measurement instrument at a given point in time. Measures aren’t the attributes themselves, but the information we gather from application of our measurement tool to the targeted attributes. For example, we construct pencil and paper, or computerized tests to “measure” achievement or aptitude in areas such as mathematics or language arts, typically involving batches of 50 items/questions/problems covering the intended content or skills. The measures we take can be referred to as “realizations” which are generated by underlying processes (all the stuff going on in school, as well as in the daily lives of the children attending and teachers working in those schools, inclusive of weather conditions, heating, cooling and lighting, home environments, etc.).  Similarly, when we take a child’s temperature we are taking a measure on that child which may inform us whether the child is suffering some illness. But that measure tells us nothing specific of the underlying process – that is, what is causing the child to have (or not have) a fever. If we wrongly assume the measure is the underlying process, the remedy for a high temperature is simply to bathe the child in ice, an unlikely solution to whatever underlying process is actually causing the fever.
  • Indicators: Indicators are re-expressions of measures, often aggregating, simplifying or combining measures to make them understandable or more “useful” for interpreting, diagnosing or evaluating systems – that is, making inferences regarding what may be wrong (or right) regarding underlying system processes. Indicators are best used as “screening” tools, useful for informing how we might distribute follow-up diagnostic effort.  That is, the indicators can’t tell us what’s wrong, or if anything really is wrong with underlying processes, but may provide us with direction as to which processes require additional observation.

Measures are only useful to the extent that they measure what we intend them to measure and that we use those measures appropriately based on their design and intent.  Indicators are only useful to the extent that they appropriately re-express and or combine measures, and do not, for example, result in substantial information loss or distortion which may compromise their validity or reliability. One can too easily take an otherwise informative and useful measure, and make it meaningless through inappropriate simplification.

Expanding on the body temperature example, we might want to develop an indicator of the health of a group of 100 schoolchildren. Following typical school indicator construction, we might simplify temporal body temperature readings for a group of 100 children to a binary classification of over or under 98.6 degrees. Doing so, however, would convert otherwise potentially meaningful (continuously scaled) data into something significantly less meaningful (if not outright junk). First, applying this precise “cut-score” to the temperature ignores the margin of error in the measurement, establishing a seemingly substantive difference between a temperature of 98.6 and 98.7, where such a small difference in reading might result from imprecision of the measurement instrument itself, or our use of it. Second, applying this cut-score ignores that a temperature of 103 is substantively different from a temperature of 98.7 (more so than a difference between 98.6 & 98.7).  Given the imprecision of measurement (where temperature measurement is generally more precise than standardized testing), if large shares of the actual temperatures lie between 98.6 and 98.7 degrees, then large numbers will likely be misclassified. The over/under classification scheme has resulted in substantial information loss, limiting our ability to diagnose issues/problems with underlying processes. We’ve taken an otherwise useful indicator, and converted it into meaningless junk.

Notes on Validity and Reliability

As noted above, for a measure to be useful it must measure what we intend it to measure, and we must be using/interpreting that measure based on what it actually measures. That is, the measure should be valid, which takes the forms of “face validity” and “predictive validity” (there are many additional distinctions, but I will limit the discussion herein to these two).  A test of “algebraic reasoning” should measure a student’s capacity to apply algebraic reasoning to test items which accurately represent the content of “algebraic reasoning.” That is, content validity, which relates to face validity.

“Predictive validity” addresses whether the measure in question is “predictive” of a related, important outcome. This is particularly important in K-12 education systems where it is understood that successful test-taking is not the end-game for students. Rather, we hope these assessments will be predictive of some later life outcome, starting, for example, with higher levels of education attainment (high school graduation, college completion) and ultimately becoming a productive member of and/or contributor to society.

Measures commonly used for evaluating students, schools and education systems can actually have predictive validity without face validity. Typically, how well students perform on tests of language arts is a reasonable predictor (highly correlated with) of how well they also do on tests of mathematics. But that doesn’t mean we can or should use tests of language arts as measures of mathematics achievement. The measures tend to be highly correlated because they each largely reflect cumulative differences in student backgrounds.

The measures should also be reliable. That is, they should consistently measure the same thing – though they might consistently measure the wrong thing (reliably invalid). If measures are neither reliable nor valid, indicators constructed from the measures are unlikely to be reliable or valid. But, it’s also possible that measures are reliable and/or valid, but indicators constructed from those measures are neither.

Often, conversion of measures to indicators compromises either or both face or predictive validity. Sometimes it’s as simple as choosing a measure that measures one thing (validly) and expressing it as an indicator to measure something else, like taking a test score which measures a students’ algebraic reasoning ability at a point in time, and using it as an indicator of the quality of a school, or effectiveness of that child’s teacher.

Other times, steps applied to convert measures to indicators, such as taking continuous scaled test scores and lumping them into categories, can convert a measure which had some predictive validity into an indicator which has little or none. For example, while high school mathematics test scores are somewhat predictive of success in college credit bearing math courses, simply being over or under a “passing” cut-score may have little relation to later success in college math, in part, because students on either side of that “passing” threshold do not have meaningfully different mathematics knowledge or skill.[1]

Aggregating a simplified metric (to proportions of a population over or under a given threshold) may compound the information loss.  That is, by looking at the percent of children over or under and arbitrary and likely meaningless precise bright-line cut-score through an imprecise though potentially meaningful measure provides little useful information about either the individuals or the group of students (no less the institution they attend, or individuals employed by that institution).

Misattribution, Misapplication & Misinterpretation

Far too often, face validity is substantially compromised in indicator construction.  The problem is that the end users of the indicators often assume they mean something they simple don’t and never could.  A common form of this problem is misattribution – or asserting that measure or derived indicator provides insights into an underlying process – where in fact, the measures chosen, their re-expression and aggregation provide little or no insight into that process.  I, along with colleagues Preston Green and Joseph Oluwole explain the misapplication of student growth measures in the context of teacher evaluation. Student growth indicators (Student Growth Percentiles) are rescaled estimates of the relative (to peers) change in student performance (in reading and math) from one point in time to another. They do not, by their creators’ admission, attempt to isolate the contribution of the teacher or school to that growth. That is, they are not designed to attribute growth to teacher or school effectiveness and thus lack “face validity” for this purpose.[2] But many state teacher evaluations wrongly use these indicators for this purpose. In the same article, and second related article[3], professors Green, Oluwole and I explain how related approaches, like Value-added modeling at least attempt to isolate classroom or school correlates of growth, partially addressing face validity concerns, but still failing to achieve sufficient statistical validity or reliability.

Neither of our articles addresses the use of crude, school aggregate indicators, constructed with inappropriately reduced versions of assessment measures, to infer institutional (school) or individual (teacher) influence (effectiveness), leading to employment consequence.  That is because these indicators clearly lack the most basic face validity for making such inferences or attribution, leading to employment consequence.  As such, we felt it unnecessary to bother critiquing these indicators for this purpose. As noted above, proportions of children over/under arbitrary thresholds on assessments tell us little about the achievement of those children. These indicators are aggregations of meaningless distinctions made through otherwise potentially meaningful measures. These aggregations of meaningless distinctions surely provide no useful information about the institutions (and by extension the employees of those institutions) attended by the students on which the measures were originally taken.

In the best case, the un-corrupted measure – the appropriately scaled test score itself – may be an indicator reflecting ALL of the underlying processes that have brought the child to this point in time in their mathematics or language arts achievement, knowledge or skills. Those processes include all that went on from maternal health through early childhood interactions, community and household conditions, general health and well-being along the way (& related environmental hazards), and even when entering school, the greater share of hours per day spent outside of the schooling environment.  Point in time academic achievement measures pick up cumulative effects of all of these processes and conditions, which are vastly disparate across children and their neighborhoods, which is precisely why these measures continue to reveal vast disparities by race and income, and by extension, across schools and the often highly segregated neighborhoods they serve.[4]

The Usual Indicators, What they Mean & Don’t Mean

Here, I provide an overview of the types of indicators often used in state school report cards and large district school rating systems.  Simply because they are often used does not make them valid or reliable. Nor does it provide the excuse for using these indicators inappropriately – such as misattribution to underlying processes – when the inappropriateness of such application is well understood.  Table 1 provides a summary of common indicators.  The majority of indicators in Table 1 are constructed from measures derived from standardized achievement tests, usually in math and English language arts, but increasingly in science and other subject areas.  State and local school rating systems also tend to include indicators of graduation and attendance rates.

Of all of the indicators listed in Table 1, only one – Value-Added estimates – attempts attribution of “effect” to schools and teachers, though, with questionable and varied success. Most (except for the most through value-added models) are well understood to reflect both socioeconomic and racial bias, at the individual level and in group level aggregation. More detailed discussion of these indicators follows Table 1.

Table 1. Conceptual Overview of School Performance Indicators

Indicator Type Facial Notes Attribution / Effect
Academic assessment (e.g. reading/ math standardized test) Scale score or group mean scale score Student or group status/ performance level All such measures norm referenced to an extent, even if attached to supposed criteria (content frameworks) Makes NO attempt to isolate influence of schools or  teachers

[no manifest relationship]

Percent “proficient” or higher (or above/below any status cut-point) Status of a group of students relative to an arbitrary “cut-score” through distribution Ignores that those just above/below threshold not substantively different.

Substantially reduces information (precision)

Makes NO attempt to isolate influence of schools or  teachers

[no manifest relationship]

Cohort Trend/ Change Difference in status of groups sequentially passing through a system Typically measures whether subsequent group has higher share over/under threshold than previous.

Influenced by differences in group makeup,

and/or differences in test administration from one year to next.

Makes NO attempt to isolate influence of schools or  teachers

[no manifest relationship]

Growth Percentiles Change in student test score from time=t to time=t+1 Usually involves rescaling data based on student position in distribution of student scores.

Does not account for differences in student background, school or home context (resources)

Makes NO attempt to isolate influence of schools or  teachers

[no manifest relationship]

Value-Added Change in student test score from time=t to time=t+1 conditioned on student and school characteristics Uses regression models to attempt to compare growth of otherwise similar students in otherwise similar settings.

Ability isolate classroom/school “effects” highly dependent on comprehensiveness, precision & accuracy of covariates.

Attempts to isolate relationship between influence gains and classroom factors (teachers) and schools.

Suspect in terms of manifest relationship.[5]

Persistence & Completion Graduation Rates / On-Time Progress / Dropout Rates Student Status / Performance Level Tracks student pathways through grade levels, courses against expectations (on track) Makes NO attempt to isolate influence of schools (resources, etc.) or  teachers

[no manifest relationship]

Attendance Proportion of “enrolled” students “attending” per day, averaged over specified time period Status of a group of students relative to an arbitrary “cut-score” through distribution Typically does not discriminate between types/causes of absences.

Known to be disparate by race/SES, in relation to chronic health conditions.[6]

Makes NO attempt to isolate influence of schools (resources, etc.) or  teachers

[no manifest relationship]

Common indicators constructed with standardized assessment measures are summarized below:

  • “Proficiency” Shares: Shares of children scoring above/blow assigned cut-scores on standardized assessments. Few states or districts have conducted thorough (if any) statistical analysis of the predictive validity of the assigned cut-points or underlying assessments.[7] Raw scores underlying these indicators capture primarily cumulative differences in the starting points and backgrounds of students, individually and collectively in schools and classrooms. Proportions of children over/under thresholds depend on where those arbitrary thresholds are set.
    • Whether raw scores or proficiency shares, these indicators are well understood to substantially (if not primarily) reflect racial and socio-economic disparity across students and schools.
  • Change in “Proficiency” Shares (Cross-Cohort Percent Over/Under): For example, comparing the proficiency rate of this year’s 4th grade class to last year’s 4th grade class in the same school, perhaps calculating a trend over multiple cohorts/years. These indicators capture primarily a) changes in the starting points and backgrounds of incoming cohorts of students (demographic drift), and b) changes in the measures underlying the accountability system (test item familiarity, item difficulty, etc.), whether by design or not. In many cases, year over year changes in shares over/under proficiency cut-scores are little more than noise (assuming no substantive changes to tests themselves, or cohort demography) created by students shifting over/under arbitrary cut-scores with no substantive difference in their achievement level, knowledge or skill.
    • These indicators do not tend to reflect racial or socio-economic disparity (except for trends in those disparities) in large part because these indicators usually reflect nothing of substance or importance, and are often simply noise (junk).[8]
  • Student Achievement Growth (Student Growth Percentiles): Constructed by comparing test score growth of each student, with respect to others starting at similar points in the distribution, among their peers. These indicators include only prior scores and do not include other attributes of students, their peers or schooling context. These indicators capture differences in student measures from one point in time to another, and are most often (read always, in practice) scaled relative to other students, so as to indicate how a students’ “growth” compares with an average student’s growth. These indicators may also capture primarily the conditions under which the student is learning, both at home and in school.
    • These indicators tend to be racially and socio-economically disparate because they control only for differences in students’ initial scores, and not for students school and peer context, or students’ own socio-economic attributes.[9]
  • Value-Added Model Estimates: Based on statistical modeling of student test scores, given their prior scores, and various attributes of the students, their peers and schooling context (breadth of factors included varies widely, as does the precision with which these factors are measured). These measures are the “best attempt” (as I often say “least bad” alternative) to isolate the school and/or classroom related factors associated with differences in student measures from one point in time to another, but cannot, for example differentiate between school resources including instructional materials, building heating, cooling and lighting and/or the “effectiveness” of employees.[10]
    • These indicators may substantively reduce racial and economic disparity of the measures on which they are based, by using rich data to compare growth of similar students in similar contexts.

This is not to suggest that value-added measures in particular have no practical use. But rather that they should be used appropriately, given their limitations. As Preston Green, Joseph Oluwole and I explain:

Arguably, a more reasonable and efficient use of these quantifiable metrics in human resource management might be to use them as a knowingly noisy pre-screening tool to identify where problems might exist across hundreds of classrooms in a large district. Value-added estimates might serve as a first step toward planning which classrooms to observe more frequently. Under such a model, when observations are completed, one might decide that the initial signal provided by the value-added estimate was simply wrong. One might also find that it produced useful insights regarding a teacher’s (or group of teachers’) effectiveness at helping students develop certain tested skills.

School leaders or leadership teams should clearly have the authority to make the case that a teacher is ineffective and that the teacher even if tenured should be dismissed on that basis. It may also be the case that the evidence would actually include data on student outcomes – growth, etc. The key, in our view, is that the leaders making the decision – indicated by their presentation of the evidence – would show that they have reasonably used information to make an informed management decision. Their reasonable interpretation of relevant information would constitute due process, as would their attempts to guide the teacher’s improvement on measures over which the teacher actually had control. (p. 19)[11]

Other measures receiving less attention (Why “attendance” rates are sucky indicators too!)

School rating reports also often include indicators of student attendance and attainment and/or progress toward goals including graduation. It must be understood that even these indicators are subject to a variety of external influences, making “attribution” complicated. For example, one might wrongly assume that attendance rates reflect the efforts of schools to get kids to attend. If schools do a good job, kids attend and if they don’t, kids skip.  If this is the case, there should be little difference in attendance rates between “rich schools” and “poor schools” unless there is actually different effort on the part of school staff.   This assertion ignores the well understood reality that children from lower income and minority backgrounds are far more likely to suffer from chronic illness including asthma, obesity or both, which is a strong predictor of chronic absence (>10).[12] Additionally, neighborhood safety affects daily attendance.[13] These forces combine to result in significant racial and economic disparities in school attendance rates which are beyond the control of local school personnel.

Once again… for my thoughts on productive use of relevant measures in education, see:

Notes:

[1] See for example, http://usny.nysed.gov/scoring_changes/MemotoDavidSteinerJuly1.pdf and/or Papay, J. P., Murnane, R. J., & Willett, J. B. (2010). The consequences of high school exit examinations for low-performing urban students: Evidence from Massachusetts. Educational Evaluation and Policy Analysis, 32(1), 5-23.

[2] Baker, B. D., Oluwole, J., & Green, P. C. (2013). The legal consequences of mandating high stakes decisions based on low quality information: Teacher evaluation in the race-to-the-top era. Education Evaluation and Policy Analysis Archives, 21, 1-71.

[3] Green III, P. C., Baker, B. D., & Oluwole, J. (2012). Legal and Policy Implications of Value-Added Teacher Assessment Policies, The. BYU Educ. & LJ, 1.

[4] Duncan, G. J., & Murnane, R. J. (Eds.). (2011). Whither opportunity?: Rising inequality, schools, and children’s life chances. Russell Sage Foundation.

Coley, R. J., & Baker, B. (2013). Poverty and education: Finding the way forward. Educational Testing Service Center for Research on Human Capital and Education.

Reardon, S. F., & Robinson, J. P. (2008). Patterns and trends in racial/ethnic and socioeconomic academic achievement gaps. Handbook of research in education finance and policy, 497-516.

[5] Baker, B. D., Oluwole, J., & Green, P. C. (2013). The legal consequences of mandating high stakes decisions based on low quality information: Teacher evaluation in the race-to-the-top era. Education Evaluation and Policy Analysis Archives, 21, 1-71.

Green III, P. C., Baker, B. D., & Oluwole, J. (2012). Legal and Policy Implications of Value-Added Teacher Assessment Policies, The. BYU Educ. & LJ, 1.

[6] http://www.changelabsolutions.org/sites/default/files/School-Financing_StatePolicymakers_FINAL_09302014.pdf

[7] Some concordance analyses relating ISAT scores to ACT “college ready” benchmarks have been produced. See: http://www.k12accountability.org/resources/Early-Intervention/Early-intervention-targets.pdf & http://evanstonroundtable.com/ftp/P.Zavitkovsky.2010.ISAT.chart.pdf

[8] See, for example: http://www.shankerinstitute.org/blog/if-your-evidence-changes-proficiency-rates-you-probably-dont-have-much-evidence

[9] See also: Ehlert, M., Koedel, C., Parsons, E., & Podgursky, M. (2014). Choosing the right growth measure. Education Next, 14(2) & https://njedpolicy.wordpress.com/2014/06/02/research-note-on-teacher-effect-vs-other-stuff-in-new-jerseys-growth-percentiles/ & http://www.shankerinstitute.org/blog/does-it-matter-how-we-measure-schools-test-based-performance

[10] Baker, B. D., Oluwole, J., & Green, P. C. (2013). The legal consequences of mandating high stakes decisions based on low quality information: Teacher evaluation in the race-to-the-top era. Education Evaluation and Policy Analysis Archives, 21, 1-71.

Green III, P. C., Baker, B. D., & Oluwole, J. (2012). Legal and Policy Implications of Value-Added Teacher Assessment Policies, The. BYU Educ. & LJ, 1

[11] Baker, B. D., Oluwole, J., & Green, P. C. (2013). The legal consequences of mandating high stakes decisions based on low quality information: Teacher evaluation in the race-to-the-top era. Education Evaluation and Policy Analysis Archives, 21, 1-71. http://epaa.asu.edu/ojs/article/view/1298/1043

[12] http://www.changelabsolutions.org/sites/default/files/School-Financing_StatePolicymakers_FINAL_09302014.pdf

[13] Sharkey, P., Schwartz, A. E., Ellen, I. G., & Lacoe, J. (2014). High stakes in the classroom, high stakes on the street: The effects of community violence on students’ standardized test performance. Sociological Science, 1, 199-220.

The Charter School Company Store

About a year ago, I released this report: http://nepc.colorado.edu/publication/charter-revenue

In which Gary Miron and I discuss various methods by which charter school operators work largely within existing policy constraints, to achieve financial gain. While working on this report, I explored various other topics that did not make the final cut, in part because I was then, and continue to have difficulty gathering sufficient information. The other day, however this article was posted on LA Weekly about wage extraction by “Gulen” charter schools: http://www.laweekly.com/news/did-a-california-charter-school-group-fund-an-effort-to-overthrow-the-turkish-government-7666698

This reminded me that I still had the related content I’ve posted below sitting on my hard drive, and that I should at least get around to posting on the blog.

Accessing Money by Creating a “Company Store”: Taxing Salaries through Affiliated Enterprises

There are many ways to “tax” teacher wages, and recapture a share of wages through closely affiliated entities (see Figure 6). In recent years, for example, there has been significant reporting on charter schools using imported labor for classroom teaching.[i] This staffing model provides two opportunities. First, there exists an opportunity to engage generally in non-traditional compensation agreements with imported labor, which may include much lower and differently structured salaries and benefits than would be paid to traditional domestic, certified teaching applicants. Second, there exist additional opportunities to “tax” the wages of these employees for such services as processing their visas and/or making travel arrangements. These services may be provided by private entities closely affiliated with the schools. That is, the money flows from one hand to the other. In this case, that money is obtained by obligating employees to pay a tax from their wages.

Public districts have been involved in similar schemes. In 2012, Louisiana school districts were caught up in a scheme involving Filipino teachers forced into exploitive contracts through a Los Angeles based placement firm.[ii] No evidence was presented, however, of any kick-back relationship between the placement firm and the districts. The arrangement came to light because (lawyers on behalf of) teachers brought litigation against the placement firm.

It can be difficult if not impossible to obtain documents specifying financial parameters of agreements between foreign teachers and charter school operators, because charter school operators refuse to release these documents, claiming exemption from public disclosure laws. Blog sites including Charterschoolwatchdog.com has posted partial documents indicating that Turkish teachers in U.S. charter schools teach under a “Tuzuk” (roughly translated as regulations or bylaw) involving an agreement to receive only a nominal stipend for their work; and, they are required to remit cash portions of their salaries as well as retirement benefits and tax refunds (although it is difficult to discern exactly which entity receives remittances).[iii]

Figures 7 and 8 illustrate salary structures derived from available statewide personnel files in Houston, Texas and the state of New Jersey. Figure 7 shows that Harmony/Cosmos schools in Texas, which rely heavily on Turkish labor,[iv] show relatively flat distribution of salaries by experience levels. It may be that actual salaries allocated show this distribution or that schools have simply reported a single salary figure of $40,000 for all teachers regardless of position, education or experience level, and not representing actual compensation. Other charter operators in Houston show growth in salaries by experience parallel to that of Houston district schools. KIPP schools pay higher than district salaries.

Figure 7. EMO Operated Charter vs. Houston ISD: Full Time Teacher Salaries by Experience at Constant Degree Level in 2010

slide1

Figure 8 shows that New Jersey schools similarly reliant on Turkish labor (Paterson Science and Technology, Central Jersey College Prep, among others[v]) also report flat salaries with respect to experience. Again, by contrast, the KIPP school (TEAM Academy) a) pays higher than other charter schools and public districts and b) shows growth with respect to experience. So too does North Star Academy, and Uncommon School. Yet, the schools employing larger shares of Turkish labor show a) very low salaries, b) little or no growth with respect to experience and c) in some cases, no variation at all in salaries with respect to experience.

Data Source: salaries modeled using statewide “Fall Staffing Report” data. Salaries estimated as a function of degree level and years of in-state experience, excluding administrative job codes, and using data from 2010 to 2015. Predictions based on teachers holding a bachelors degree in 2015.

Figure 8. New Jersey Charter School Salary Structure (by Total Experience)

slide2

Other options exist for recapturing portions of teacher salaries. But as is true for the Turkish Tuzuk, documentation of these schemes may be difficult to obtain from privately-managed charter schools that often claim these agreements are exempt from public disclosure laws. One common model is the “company store,” where employees are required to purchase goods and/or services from the affiliated entities. This model can be used for visa processing fees for foreign labor, but might also be used for obtaining relevant credentials, professional development, or even housing.[vi]

For example, founders of New York and Newark, NJ area charter schools and management companies have established their own Graduate School of Education (Relay GSE), staffed primarily by themselves—current and former employees of the charter schools and management companies. Relay GSE was criticized in public hearings over its use of under-credentialed and inexperienced faculty to deliver its programs, but was eventually granted accreditation.[vii]

The Relay Board of Trustees includes founders of KIPP, NYC; Achievement First; and affiliates of Uncommon Schools.[viii] In New Jersey, Relay’s graduate programs are offered on-site within North Star Academy,[ix] a Newark charter school affiliated with the Uncommon Schools network (established by a founder of Relay GSE). The Dean of Relay, Newark, is a co-founder of North Star Academy.[x] Former teachers from the affiliated charter schools report being obligated as a condition of employment to obtain credentials (MA degrees and related certifications) from Relay GSE. That is: employees at the charter schools are having a portion of their salary taxed to pay tuition to a “graduate school” run by founders of their own charter schools, operated within their own charter school facility (lease agreement unknown), where courses are often taught by their own teaching peers having only slightly more advanced education and experience.[xi] We elaborate on this example in Appendix A.

Another way for affiliated charter schools to channel money to Relay is to set aside a portion of their budget to subsidize graduate education—but only at Relay GSE. That is, some EMOs (including Uncommon) have a practice of paying for graduate degrees obtained from Relay, but not from any other institution (unless the teacher can prove that Relay does not offer a degree in the same field). Teachers agreeing to pursue their degrees from Relay with school support must complete those degrees or, as noted earlier, are required to reimburse their EMO for any/all tuition reimbursement they received.

Compensation taxing models could conceivably be taken one step further, to access not only a portion of teacher salaries, but also their retirement benefits, by adopting the Enron version of the Company Store: having employee retirement benefits invested directly back into the charter school operator or through a closely affiliated financial manager. In the Enron case, approximately 60% of the company’s 401(k) was invested in Enron stock.[xii]

The information provided in the main text describes a “company store” model at Uncommon Schools/Relay Graduate School of Education. But it only describes the information that is publicly accessible. A great deal more about the model is still unclear. Here we provide a few insights, based on the limited disclosures from, e.g., former employees. We stress here that it is difficult to obtain clear official documentation of the practices involved, because current employees are sworn to secrecy, including prohibitions against providing contractual documents and employee handbook materials. Accordingly, this appendix summarizes merely what we were able to ascertain through published accounts and acquired documents. This is an obvious area for officials and investigative journalists to delve into, since a great deal is unknown, and the limited accounts, discussed here, have yet to be fully corroborated.

A former Uncommon schools teacher who identified herself only as “Emily” created a blog with a single post, which states: “upon my being hired and deciding to take the job in August 2011, I was required to enroll at Relay. It was one of the conditions of my employment—to begin the process of getting my masters in education at Relay, which has a partnership with Uncommon Schools.”[xiii] That is, as we note above, obtaining the graduate credential was a condition of continued employment.

Regarding financial arrangements, the former Uncommon Schools teacher notes that “Relay offers all of its students a very doable tuition package—you can pay in small increments that are realistic with most charter school salaries. And, to be a Relay student, you must be employed at a charter or public school. Not only that, Relay will greatly subsidize your tuition from various sources that fund it, such as money from AmeriCorps.” This squares with Relay’s own explanation of “Tuition & Aid” from their web site:

Figure A1

 slide3

 It also squares with language we acquired from an Uncommon Schools employee handbook:

Figure A2. Handbook Excerpt

slide4

These limited pieces of information, taken together, suggest that if total annual tuition for Relay graduate programs is $17,500, the typical Uncommon Schools new teacher can be expected to pay about half that for two years (to degree completion) and have the other half subsidized. If the teacher leaves her position at Uncommon Schools prior to completion or otherwise fails to complete the graduate degree, the teacher may be responsible for reimbursing the school. New teachers would seem to have little choice in the matter.

Additional evidence gathered from individual school websites and through deep web searches (leading to Uncommon Schools Family Handbooks) reveals school faculty bios that indicate large shares of teachers who have a) recently completed degrees at Relay, b) are currently pursuing degrees at Relay, and/or c) currently serve as adjunct instructors for Relay.

Again, these elements of the Company Store need further investigation. If they are true, it raises serious concerns. If it is not, then taxpayers and public officials should expect public clarification on that point.

As we discussed in the main text of this brief, a small handful of individuals were involved in the founding and original governance of each institution. Our analysis here is not comprehensive, because Uncommon Schools and Relay GSE span many state boundaries. Thus our examples here focus primarily on the New York/New Jersey area, with particular attention to Newark, NJ. Table A1 shows the board membership and highest paid employees for four interconnected organizations (and there may be others we have missed thus far).

The two left hand columns include Uncommon Schools (data from IRS 990, 2014, 2013 tax year) and North Star Academy’s Consolidated Annual Financial Report (CAFR) from the New Jersey Department of Education. We expect these organizations to overlap because one (North Star) is subsidiary to the other (Uncommon). The right three columns include Relay Graduate School of Education; a separate non-profit entity providing financial support to Relay, called Uncommon Knowledge and Achievement, Inc. (UK&A); and a third organization, Zearn. We also expect these three organizations to overlap because it would appear that UK&A’s primary function is to raise funds for providing scholarship/aid for students attending Relay GSE (providing $500,000 to GSE in tax year 2013 according to IRS filings, and another $100,000 to Zearn). We understand less about the role of Zearn, except that it appears to be a company created for the aggregation and distribution of electronic teaching materials/modules for mathematics.

More significant potential concerns arise when parties cross over between the left two and right three columns, especially in significant governance or leadership roles or where significant financial interests are involved. Specifically, it would be questionable for leadership of Uncommon Schools to be requiring their employees to purchase goods/services from Relay if in fact those leaders have financial interests in Relay. This is likely true even if these individuals absolve themselves of voting roles in setting those policies. IRS Filings (and charter school financial audits) indicate that Uncommon Schools is a “related party” to its various subsidiary schools (and even their real estate arms). IRS filings similarly acknowledge the relationship between Relay, UK&A and Zearn. But, for some reason neither IRS filings nor external audits of Uncommon Schools mention a formal relationship between Uncommon and Relay, despite Norman Atkins’ role as CEO of one and chair of the board of the other.

The connectedness between Relay and Uncommon does not end there. Jamey Verrilli is Dean of Relay’s Newark, NJ branch, which operates out of an address shared by a North Star (Uncommon) school (10 Washington Place, Newark). Verrilli also sits as a “community member” representative on the board for North Star. That is, he sits on the board of a school that obligates (as we understand it) its teachers to pay his other organization, housed in the same building, substantial annual tuition. In addition, Relay Dean Verrilli’s wife is employed by North Star (duly noted on state conflict of interest reports).[xiv]

Deeper exploration of the Uncommon and Relay websites reveals that Uncommon’s Newark Managing Director also plays a significant role in Relay GSE, leading Relay’s Leverage Leadership program. Other Uncommon employees, including those in leadership roles, also have significant presence on the Relay website (noted in green in Table A1), including the starring role of Verrilli’s wife in demonstration videos.[xv] In sum, formal leaders of Uncommon/North Star Academy are in positions to influence policy decisions that would direct school resources to Relay and would require employees to make payments to Relay, as well as senior employees and administrators at North Star in a position to exert significant coercion over incoming and other junior staff. Note that according to the handbook, these individuals wield significant influence over what types of professional development and graduate pursuits might be approved for reimbursement.

Table A1

  CMO and Subsidiary School

(Acknowledged as Related)

UK&A Formally Related to Relay GSE (Acknowledged)
  Uncommon Schools

(IRS 990)

North Star Academy (NJ CAFR) Relay GSE

(IRS 990)

Uncommon Knowledge & Achievement Inc. (IRS 990) Zearn LLC

(IRS 990)

Board Members
  Norman Atkins (Chair) Rick Rieder (Chair) Larry Robbins (Chair) Norman Atkins (CEO) Norman Atkins (Chair)
  David Cooper Michael Lytle David Levin (Secretary)[1] Timothy Saintsing (COO, Prior) Evan Rudall (CEO)
  Donald Katz Robert Howitt Arthur Levine (Trustee) Thackston Lundy (COO, Current) Shalinee Sharma (Secretary)
  Charles Harris Gia Rys Julie Mikuta (Trustee) Robert Karr (Treasurer)
  Pearl Kane Nicole Albano David Saltzman (Treasurer) David Levin
  Robert Karr Paul Bambrick-Santoyo Dacia Toll (Vice Chair)[2] David Saltzman
  Rondo Moses Trisha Scipio-Derrick Carlos Watson (Trustee) Dirk Ziff
  Robert Jain Ravi Bellur Dacia Toll
  Brooke Reid Scott Sleyster Larry Robbins
  Neal Moszkowski Nkyah Taylor
Employees & Officers
  Brett Peiser (CEO) Carolyn Hack (Treasurer) Norman Atkins (President)
  Carolyn Hack (CFO/COO) Michael Ambriz (COO) M Yvonne Chao (CFO)
  Paul Bambrick-Santoyo (Managing Director, Newark) Julie Jackson (Principal) Piper Evans (Director of Finance)
  Julie Kennedy (Managing Director) Mike Mann (Head of School) Timothy Saintsing (COO)
  Joshua Phillips (Managing Director) James Verrilli (Community Member) Mayme Hostetter (Dean, Relay NY)
  Julie Jackson (Managing Director) Michael Larosa (CIO)
  Dana Lehman (Managing Director) Brent Maddin (Provost)
  Barbara Martinez (Chief External Officer) James Verrilli (Dean, Relay Newark)
  Laura Lee McGovern (Chief of Staff) Robert Underwood (CTO)
  Laura Maestas (Chief Talent Officer)
  Tara Marlovits (COO Uncommon NYC)
  Michael Ambriz (COO North Star Academy)
Overlapping

Others with current/past significant presence on Relay web site

[see, for example: http://www.relay.edu/programs/leverage-leadership-institute-fellowship-0/overview]

[1] KIPP Co-founder

[2] Founder, Achievement First

Figure A3 attempts to sketch out some of these relationships, with particular attention to Newark, NJ. On the left are charter schools, where not only Uncommon Schools, but also KIPP and Achievement First are official partners of Relay GSE. Clearly, as indicated in the table above, there exists significant, substantive leadership and financial overlap between Uncommon and Relay. In addition to the founding/leadership overlap, in Newark the two share a facility (financial arrangement unknown) and seem also to share at least a few employed staff, including the Newark Managing Director (compensated at $223,227 by Uncommon alone in tax year 2013).

About 50% of Relay tuition is directly extracted from new faculty, assuming the above sources to be correct, with the other 50% coming through: a) schools like North Star and b) UK&A.

Figure A3

slide5

One might question how lucrative it can be to rely heavily on tuition extraction (and other subsidies) from only new, incoming teachers required to pursue two-year graduate programs. After all, how many new teachers might there be in any given charter school? Could there possibly be a constant or increasing flow? Figure A4 uses statewide annual staffing files from the New Jersey Department of Education to tally the total numbers of teachers and numbers of teachers with zero or one-year experience in North Star Academy from 2009 to 2015. In the most recent three years, North Star alone had 103, 134 and then 153 novice teachers. Assuming that Relay could gross $17,500 per year for each, the total would approach $6 million by 2015 (about half extracted from teacher wages and half from grants made by or through related entities). The high rates of novice teachers in North Star and other Uncommon (NY) schools are a function of both growth and very high rates of turnover.

Figure A4

slide6A Closing Thought

So, I know many will say, what’s the big deal? Relay affiliated charter schools are (at least perceived as) highly successful and their success reliant on strict adherence to their specific pedagogy. So why not grow their own/train their own teachers? As I see it, that’s a clearly separable issue from what I outline above. First of all, having effective staff in-service training, even where provided by affiliates is more acceptable in a number of ways (depending on how those affiliate contracts are handled & disclosed). It’s localized, specific training, not degree/credential granting having broader public policy implications. No institution – a single public or private school  – should be granted the authority to self-credential their own employees in this way, to hold supposedly broad, transferable expertise, which is in fact anything but.  Provide specific training for their current job and context? OK. Provide degrees and credentials w/broader state accreditation? That’s the public policy concern (which should have been shut down earlier).  The second (ethical/COI) concern (which is also a public policy concern) is that no school leaders should have authority to require or have such opportunity to coerce teachers under their supervision to buy back services provided by their employer or close affiliates of their employer (resulting in financial benefit).

I am quite sure that if I was to collaborate with my Dean at Rutgers Grad School of Education to start a charter charter school chain in and around New Brunswick, placing us on the leadership team and board (along with other faculty and leadership at Rutgers),

AND THEN, require each of our new teacher employees in those charter schools to obtain graduate degrees from Rutgers GSE as a condition of maintaining their employment (even if indirectly, that we needed to give them assignments which required they be certified in certain areas, in which we provide the cert programs),

AND ON TOP OF THAT, use teachers who had just completed those programs to deliver the courses (as underpaid graduate assistants) to the new teacher on site… in the charter schools…

it would raise at least a few eyebrows.

And it should!

Additional Documents

relaygradsched-certauth-signed-sec-7-27-16

And a musical tribute:

NOTES

[i]  Pilcher, J. (2014). Charter schools use Turkish ties, visas to get teachers. Cincinnati.com http://www. cincinnati.com/story/news/2014/10/05/charter-school-turns-turkish-teachers/16791669/

Beauchamp, S. (2014). 120 American Charter Schools and One Secretive Turkish Cleric. The Atlantic. http:// www.theatlantic.com/education/archive/2014/08/120-american-charter-schools-and-one-secretive-turkish- cleric/375923/

Saul, S. (2011). Charter Schools Tied to Turkey Grow in Texas. New York Times. http://www.nytimes. com/2011/06/07/education/07charter.html?_r=1

[ii] Samuels, D. (2012). Filipino teachers win $4.5 million jury verdict, claim they were forced into ‘exploitative contracts.’ The Times-Picayune. http://www.nola.com/news/baton-rouge/index.ssf/2012/12/filipino_ teachers_win_45_milli.html

[iii] See, for example, the blog: http://www.charterschoolwatchdog.com/tuzuk—a-contract-to-steal.html

[iv] Saul, S. (2011). Charter Schools Tied to Turkey Grow in Texas. New York Times. http://www.nytimes. com/2011/06/07/education/07charter.html?_r=0

[v] Schools selected based on information provided here: http://turkishinvitations.weebly.com/list-of-us-schools. html

[vi] See for example: http://www.teachers-village.com/index.html

[vii] Mooney, J. (2013). Alternative Grad School Raises Concerns About Who’s Teaching NJ’s Teachers. NJ Spotlight. http://www.njspotlight.com/stories/13/07/09/alternative-grad-school-raises-concerns-about-who- s-teaching-nj-s-teachers/

[viii] See for example: http://www.relay.edu/about/people

[ix] See for example: http://www.relay.edu/campuses/newark (it is unknown whether Relay pays rent for use of space at North Star Academy).

[x] See for example: http://www.relay.edu/campuses/newark

[xi] See, for example: http://emily-thetruth.blogspot.com/2012/03/truth-about-relay-graduate-school-of. html?spref=tw

[xii] Schutz, E.E., Francis, T. (2002). Enron Executives’ Benefits Kept on Growing as Retirement Plans of

Employees Were Cut. Wall Street Journal. http://www.wsj.com/articles/SB1011745748757428600

[xiii] Blog post can be found here: http://emily-thetruth.blogspot.com/2012/03/truth-about-relay-graduate-school- of.html

[xiv] State ethics filings here:

http://education.state.nj.us/ethics/discforms.php?c=13;d=7320;s=100330;dl=20140201

http://education.state.nj.us/ethics/discforms.php?c=13;d=7320;s=134341;dl=20150201

[xv] To understand the extent of connections between Relay/Uncommon, see here (https://www.youtube.com/ watch?v=9XnrT4MC7fw) where Beth Verrilli (employed by North Star), wife of Jamey Verrilli (Dean, Relay Newark) is used in a demonstration video, linked from Relay GSE leadership program, but to an Uncommon Schools web site. So, in the video, Mrs. Verrilli is acting as an employee of Uncommon Schools (North Star), while being used in a demonstration for leadership training at Relay, applying techniques promoted by Bambrick Santoyo – Managing Director of Uncommon Newark, but also lead faculty member for Relay’s Leverage Leadership program.

On the Relative Efficiency of New Jersey Public School Districts

New Jersey Education Policy Forum

PDF of Brief: Baker.Weber.NJEfficiency_8_2_16

Bruce D. Baker

Mark Weber

Contrary to current political rhetoric, New Jersey’s least efficient producers of student achievement gains are not the state’s large former Abbott districts – largely poor urban districts that benefited most in terms of state aid increases resulting from decades of litigation over school funding equity and adequacy. While some Abbott districts such as Asbury Park and Hoboken rate poorly on estimates of relative efficiency, other relatively inefficient local public school districts include some of the state’s most affluent suburban districts and small, segregated shore towns. And yet these districts will be, in effect, rewarded under Governor Chris Christie’s “Fairness Formula,”[1] even as equally inefficient but property-poor districts will lose state aid.

Findings herein are consistent with previous findings in cost-efficiency literature and analyses specific to New Jersey:

  • There exists some margin of additional inefficiency associated with Abbott status relative to…

View original post 211 more words

Picture Post Friday: Newark Charter School Data Update

I haven’t revisited the Newark charter school data in a while now. Here are my most recent cuts at the most recent available data:

First, DEMOGRAPHICS of District and Charter Schools

RED bars indicate CHARTER schools. Blue are other Newark Schools. North Star highlighted w/diagonal pattern as it has been the subject of recent twitter/blog fodder/debate.

[reason for focusing on “free lunch” explained here]

Slide1

Slide2Slide3

Slide4

Next, COHORT reduction rates:

NJ Cohort Analysis Stata Code

Given the reshuffling of students across district regular and special schools, including reorganization during the period, the following is a much clearer comparison than the ones previously posted (aggregating “district” schools):

NEWARK CHARTERS Update

How, you ask, can a senior cohort exceed in size the 7th graders who started five years prior? Presumably, some of the many, many black boys who started in North Star in particular (but also TEAM) do end up back in NPS district schools.

Slide9Slide10

& Finally, some modeled estimates of relative GROWTH and relative PARCC Scale Scores (relative to predicted values) – noting that I a unable to effectively account for high rates of attrition (as in the North Star case specifically).

Slide11

Slide12Slide13Slide14

Slide15

 

 

%d bloggers like this: