The “Ed Schools are the Problem” Fallacy


I had the displeasure of waking up to this drivel in my in-box this morning:

“Those who can, do. Those who can’t, teach. And those who can’t teach, teach teaching.”

http://www.nytimes.com/2013/10/21/opinion/keller-an-industry-of-mediocrity.html?_r=0

yeah… and those completely lacking in critical thinking, basic research and data interpretation skills write op-eds for the Times.

I don’t really teach teachers myself, so I guess I shouldn’t take offense. But I do mainly because the core argument advanced here is so ill-informed and poorly conceived.

Allow me to start by pointing out that I have actually written detailed, quantitative research in peer reviewed journals on the very topic of who’s teaching the teachers. In fact, the article we wrote was done partly in response to the Arthur Levine report cited in the Times op-ed piece. And it’s not as if the article title really conceals its contents:

  • Wolf‐Wendel, L., Baker, B. D., Twombly, S., Tollefson, N., & Mahlios, M. (2006). Who’s teaching the teachers? Evidence from the National Survey of Postsecondary Faculty and the Survey of Earned Doctorates. American Journal of Education, 112(2), 273-300.

My apologies for the fact that this article is fire-walled. I really don’t expect all of my blog readers to go through the trouble of paying for it or finding an academic library that carries it. But any responsible journalist, pundit or author proclaiming a strong policy position on this issue ought to at least do some reading on the topic first. The above article is certainly not uncritical of teacher preparation. [UPDATE: Full version here, courtesy of the kind folks at AJE Baker2006]

And the issues of complexity and variation in teacher preparation I explore in the above research article are not the only massive omission or conflation put forth in the New York Times piece, which operates on the overly crude assumption of a uniform system of content-free instruction across any and all ed schools.

Let’s tackle the bigger and much simpler issue here – the broad notion advanced in this op-ed that Ed Schools are the problem!  Ed Schools are the primary threat to the quality of our public schooling system as a whole and by extension Ed Schools are a threat to our national security. [yeah… he didn’t really say that… but somehow it often goes there] And further, that if we can just replace ed schools – with some other unknown thing – we’ll all be better off.

A kinder, gentler variant on this argument is that it’s just the bad ed schools that are a threat and that we can weed out those bad ed schools by looking at how the students of their graduates perform. I’ve addressed this issue in a few previous blog posts. First, I’ve addressed the question of whether “ed school” is really some static, monolithic entity. Second, I’ve addressed the feasibility of rating ed schools by twice removed outcome measures.

But there’s actually a simpler logical fallacy at play here which lies at the root of many reformy arguments regarding causes and consequences – failure to acknowledge that  the U.S. has a wide range of elementary and secondary of schools that are both high performing and low performing and that the defining features differentiating higher and lower performing schools are not found primarily in their teachers or the preparation programs they attended – or whether they attended any at all – but rather in the communities they serve, the resources available to them and the backgrounds, health and economic well-being of the children and families they serve.

This is not about the poverty as excuse argument. This is about the simple point that our highest performing public schools also employ teachers from traditional public college and university preparation programs and in many cases, teachers from the same – or substantively overlapping – college and university preparation programs as teachers in our lowest performing schools in the same region.

If that’s the case, then how is it possible that teacher preparation programs are the problem?

I know… the good reformer at this point is thinking – but there are no good U.S. public schools or districts. They all suck and that’s precisely why teacher preparation is the problem. Of course, if that was the case – that all K-12 public schools suck – it would hard to, by research design – with a dependent variable that doesn’t vary – attribute that sucky-ness to a single cause.  But the dependent variable does vary… even when we rely on reformy resources like the Global Report Card I wrote about here.

First, here’s a location where you, yourself can actually download the reformy report card, which in large part was designed to shake the confidence of America’s suburban parents by taking a few statistical leaps to show them their leafy suburban schools wouldn’t stack up so well if we transported them to Finland or Singapore.

http://globalreportcard.org/docs/Global-Report-Card-Data-11.14.12.xlsx

I’ll save that argument for another day, and just select two sets of districts from this report card, from Illinois and Kansas, because I have the data readily available. Let’s look at local public school districts that are

1) Better than the Average Fin and those that are…

2) Worse than 80% of beer-swillin’ Hockey Lovin’ Canadians.

That’s quite a contrast (even though both are high performing countries – on average – setting aside demographics, etc.).

Here are the lists:

Slide1

Slide2

So, we’ve got some school districts in each state that are better than the average Finnish school and some that get trampled by the those syrup swillin’ hosers from the Great White North.

The only plausible explanation is that the teachers in the Better than Finland category are either from completely non-traditional ed schools or not ed schools at all while  the teachers in the not-so-great schools all come from your typical state ed school.

Certainly, we know from large bodies of teacher labor market research that graduates of various preparation programs, colleges and universities and alternative route programs more broadly,  sort themselves on the labor market, with those who possess stronger academic credentials often sorting into the “more desirable” jobs.

But that’s somewhat of an aside here. For the basic reformy premise of massive uniform ed school failure to be true – we would have to see little or no commonality in the ed school preparation of teachers across these settings – across totally awesome U.S. schools and totally sucky ones.

So, here’s the recent distribution of graduates of Kansas teacher preparation programs in the Kansas City metropolitan area which includes the Blue Valley School district – better than the average  Fin and Kansas City Kansas, which, well, gets its butt kicked by Canada!

Slide3

Hmmm… you can’t possibly be telling me that both KCK and BVSD have teachers who graduated from the major state teacher preparation colleges can you? If that’s the case, then their relative international rankings might not be determined by teacher preparation?

[ignore the poverty shading in the background…’cuz payin attention to poverty… well… just isn’t cool with the reformy crowd!]

There are some notable features to this map. One is that BVSD and and Olathe to its west were still significantly growing districts during this period. So it makes sense that they hired a lot of new teachers during that time. It makes less sense that KCK, more stagnant (and declining) in population hired so many new teachers – but for the relatively high turnover rate more common in such high poverty settings! There are also some distributional differences in the dots – which universities produce more teachers for which districts (or provide more credentials). Pittsburg state (blue dots) more prevalent in KCK provides a local program that feeds to KCK. I’d be hard pressed, however, to lay blame on Pitt state for KCK’s Canadian butt-whoopin’ and I’d be equally hard pressed to credit K-state in producing more teachers for Blue Valley as the cause of Blue Valley’s competitive match up with Finland! The fact is that all of these Kansas districts draw heavily on teachers produced by the public teachers colleges of that state – and some do as well as Finland while others struggle.

As such, it’s pretty darn hard to lay blame on traditional teacher preparation in Kansas for these differences in outcomes.

Now, let’s take a look at a few high performing and lower performing districts in Illinois.

First, here are the top 15 undergraduate degree producers for Chicago and Aurora East and for Naperville and Lake Forest. Rather than from the degree producers perspective, these data simply include all instructional staff in these districts, downloadable here: http://www.isbe.state.il.us/research/xls/2012-tsr-public-dataset-instr.xlsx

The data include where teachers got their undergraduate and advanced degrees.

Slide5

Wow… there’s actually quite a bit of overlap in the institutions. Sure, there are differences. Where a state name is listed, the teacher received his/her undergraduate degree from an un-named institution in that state (such are the shortcomings of state administrative data). The City of Chicago does have larger shares from some Chicago based programs. But there’s also overlap and there’s significant overlap for the state’s major public teacher preparation institutions, like Illinois State University, Northern Illinois University and the University of Illinois main campus (Champaign). How can that be? How can there possibly be school districts that compete favorably with Finland while employing graduates of traditional teachers colleges?

While the percentages of teachers in these districts who attended any one preparation institution tend to be small, the shares who attended major public preparation institutions for their bachelors degrees appears marginally larger in the high performers (Over 10% for both IL State and Northern).

That’s impossible! But… But… But… graduates of those same colleges are teaching in districts that got whooped by the Canadians? So how can we possibly place blame for systemic failure of American schools on teacher prep programs? I’m struggling with the logic here.

One more look… here are the advanced degree granting institutions for teachers in higher versus lower performing Chicago area districts. Note that “NULL” refers to those not holding (or reporting) advanced degrees and that the share holding only a bachelors degree is higher in the lower performing districts (poorer, minority districts).

Slide6

Again, these degrees – which in include both initial and additional certifications – are dominated by traditional credential granting institutions with substantial overlap across teachers between higher and lower performing schools.

This is a separable but related issue to the evaluation of ed schools by student outcome measures. I’ll continue digging more into that issue in future posts.

It is certainly hard to make a compelling case that traditional teacher preparation institutions are the primary cause of our supposed lagging national education system when our highest performing schools – those that compete favorably with Finland – also employ in large number, graduates of those preparation programs and in many cases employ significant numbers of graduates of the same programs that provide teachers for our supposed failing schools.

Advertisements

22 Comments

  1. This is a parallel clarification that exists with the “bad union” mantra that defies logic (How are unions THE problem when unionized states have higher student outcomes that non-union states?). Excellent post,

  2. I became a teacher because I believed in rational decision making and that education could help people learn this. Your dismantling of the idiot position of theop-ed writer is a perfect example of how we should evaluate such ideas. But what do we do when people literally refuse to face facts? Logic is impotent without data and the reformy crown is not interested in facts.

    1. A couple of years ago I went to a charter school “community education” session run by one of the top (mostly) reformy education organizations in Seattle. The first slide the facilitator put up read “The Fact’s (sic) Don’t Matter” I kid you not. She repeated this over and over, “the facts don’t matter, what matters is your values.” Huh? She’s a high level employee there. What to say? Except that the very next year their pro-charter-school initiative passed with massive private corporate and foundation contributions.

  3. Teachers are an easy target because people harbor long resentment from detention they were give 20 years ago. Also, it is easier for politicians to makes teachers the target than admit 1. As American families many of us have not done our best for the past 20 years, 2. We have been, are, and will be a nation on immigrants, and 3. Many of the international comparisons show U.S. students as doing better than the spooky picture painted in our minds. Most teachers, especially in California, are well prepared. http://pointeviven.blogspot.com/2013/06/the-common-core-shift-misconceptions-of.html

  4. What to do about quantitative illiteracy in the journalism profession? The first step is to make journalism more selective. Prospective journalists should score in the top 25% in statistics and data analysis classes, so they don’t fall for bogus “studies” from think tanks with agendas such as, say, privatization of public education. Keller must have missed this: http://tinyurl.com/n2lwg7r . Current journalists should have to take a standardized test in basic social science research methods. Working as a New York Times columnist is a cozy, lucrative position. It’s about time some of them DID feel threatened.

    1. Yup. The quantitative literacy should be a major qualifier for journalist, but hey Id be happy if they simply actually had a degree in a relevant content area even if it was from one of those shitty teacher ed schools they think so lowly of. ON what basis does NYTimes decide someone is an expert, informed and careful enough to provide widespread position on important public matters? Its going downhill the whole newspapers with increasingly yellow journalism tendencies, coverage of topics that one usually only found in the likes of Daily News, and with these young writers that write cutsy stuff that is light on content and even lighter on purpose.

  5. I always admire the thoroughness of your analysis, but a some factors seem to be missing about teacher training and where teachers are employed. One would be the quality of the graduate, not based on the institution attended, but on the grades received during their training. Graduates from the same program may be very different in the knowledge, drive, and skill they bring to their job in a classroom. The other factor is whether some districts in your analysis can be more selective about who they hire; again not based on where applicants trained, but the quality of the individual applicant. Are better qualified teachers being hired and then being placed in schools with better reputations or which are located in more affluent communities? The large affluent county where I taught for several years receives thousands of applicants each year and can be very selective about who it hires. In turn, those new hires, when given a choice among the dozens of schools in the county, can to some degree, self-select into those schools with a better reputation. The less affluent large county, where I also taught for many years, had a much smaller pool of applicants from which to pick, some not fully certified who then struggled for several years to become certified.

    The NYT piece is ludicrous, but at the same time maybe the US should move toward a model of teacher training more like that of Finland. The perception of teachers as being something less than professional because of the way they are trained is tough to overcome without some significant restructuring of that training. But the nation has to be willing to pay for that. What student will be able or willing to pay $200K out of pocket to earn a Master’s Degree to enter a profession offering $35K or less starting salaries?

  6. Here’s the problem: articles like the one referred to here get major press attention. Critiques get none.In a recent article I wrote I looked at the number of citations of articles vs. retractions of those articles in the same journal. The retractions were almost unnoticed. Lies that play in to prejudices are toxic because once out there, they BECOME the facts.

    1. which is why they keep doing it. it’s all about messaging. facts and basic logic really don’t matter much in today’s media. just create a message of good and evil… repeat, repeat, repeat… and so it shall be.

  7. It is important for both quantitative and qualitative analysis to appreciate what the other can discern. As someone who is trained in both (and has studied education independently for many years), I have to say your analysis fails to capture the real pattern and problem in education. Deep qualitative analysis does (you might consider some browsing on http://www.invisibleserfscollar.com/) and the numbers unfortunately capture the effects of parenting and environment, not of schooling.

    I am in Canada, and while we share in the general dysfunction, your system has a few unfortunate characteristics that have driven the alternatives to dysfunctional ed schools into almost equally dysfunctional directions, such as the one you’ve documented in the post that follows this one. In fact, that post – where a school can increase its scores simply by ejecting weak students – makes my point and negates the conclusions you’ve drawn here.

    Edschools ARE the problem; they are a problem so pervasive and so toxic that they are taking down the entire academic enterprise; other faculties are falling ill one after the other. A recent Medschool website I visited promised “student-centred instruction.” Education is leading other sectors into qualitative decline too – journalism, you’ve astutely observed, is another casualty. And a failure of academic rigour leaves a vacuum of purpose in universities into which other interests are happily stepping.

    The problem arises in the quality of the dialogue when someone posits that because you have identified the problem, the solution is obvious, which is what the article you’ve excoriated does. The solution isn’t obvious at all, and the best place to start depends on just what, exactly, you want to accomplish. And since different people want to accomplish different things (this is where an understanding of “vested” interest is a good thing), there are in fact different answers for different people.

    Your posts are always instructive, but not always for the purpose you intend, and you limit their usefulness when you fail to discern the value in other perspectives. In this case, the original article should not be discarded in its entirety.

    1. I concur with much of what you’ve said above… but believe you’ve misinterpreted my ed school post. Of course the outcomes I’ve shown (from the global report card) are largely a function of children’s backgrounds and that there is little or nothing that can be said legitimately about ed schools from characterizing our k-12 schools as succeeding or failing relative to international peers. My intent is also not to suggest that all is well and good in ed school land, but rather, by reference to my research article, that all is varied and often not what we believe it to be.

      My main point is that the NYT piece posits a ridiculous argument that a) because the U.S. does poorly internationally, that b) our ed schools must suck. My point is simply to point out that some of our k-12 schools don’t do poorly internationally but also have teachers from ed schools. Obviously not positive proof of ed school greatness, but certainly pokes a hole in the original NYT premise.

      What I agree with mostly from your comments above is that identification of a supposedly simple problem can lead to simple conclusions. I would point out, however, that the problem is also not so simple… not as simple as saying we have uniformly intellectually vacuous teacher preparation… lack of rigor… lack of entry standards across the board. And thus the solution is also not so simple. In fact, the implication that we can simply increase entrance requirements and rigor or even professional credential requirements… and improve the teacher workforce is also rather simple minded. A substantial body of literature addresses the relationship between relative compensation of the profession and the quality of entrants to it (see list below). And others have shown that credentialing tests also may not have their intended consequences.

      The suggestion that the fix necessarily lies with crappy, non-rigorous ed schools overgeneralizes the problem (to any/all ed schools) and oversimplifies the solution (by ignoring the broader labor market dynamics both pertaining to k-12 education systems and higher education systems). And that is most certainly my bigger point here and one that will be explored further in future posts.

      ========

      • Murnane and Olson (1989) find that salaries affect the decision to enter teaching and the duration of the teaching career.[1]
      • Figlio (1997, 2002) and Ferguson (1991) find that higher salaries are associated with better qualified teachers[2]
      • Loeb and Page (1998, 2000) find that raising teacher wages by ten percent reduces high school dropout rates by between three and six percent and increases college enrollment rates by two percent.[3]
      • David Figlio and Kim Rueben (2001) note that, “Using data from the National Center for Education Statistics we find that tax limits systematically reduce the average quality of education majors, as well as new public school teachers in states that have passed these limits.”[4]
      • Ondrich, Pas and Yinger (2008) “find that teachers in districts with higher salaries relative to non-teaching salaries in the same county are less likely to leave teaching and that a teacher is less likely to change districts when he or she teaches in a district near the top of the teacher salary distribution in that county.” [5]

      [1] Richard J. Murnane and Randall Olsen (1989) The effects of salaries and opportunity costs on length of state in teaching. Evidence from Michigan. Review of Economics and Statistics 71 (2) 347-352
      [2] David N. Figlio (1997) Teacher Salaries and Teacher Quality. Economics Letters 55 267-271. David N. Figlio (2002) Can Public Schools Buy Better-Qualified Teachers?” Industrial and Labor Relations Review 55, 686-699. Ronald Ferguson (1991) Paying for Public Education: New Evidence on How and Why Money Matters. Harvard Journal on Legislation. 28 (2) 465-498.
      [3] Susanna Loeb and Marianne Page (2000) Examining the link between teacher wages and student outcomes: the importance of alternative labor market opportunities and non-pecuniary variation. Review of Economics and Statistics 82, 393-408. Susanna Loeb and Marianne Page (19980 Examining the link between wages and quality in the teacher workforce. Department of Economics, University of California, Davis.
      [4] Figlio, D.N., Rueben, K. (2001) Tax Limits and the Qualifications of New Teachers. Journal of Public Economics. April, 49-71
      [5] Ondrich, J., Pas, E., Yinger, J. (2008) The Determinants of Teacher Attrition in Upstate New York. Public Finance Review 36 (1) 112-144

      On raising the bar through testing teachers….

      http://www.sciencedirect.com/science/article/pii/S0272775707000842

      The education reform movement includes efforts to raise teacher quality through stricter certification and licensing provisions. Most US states now require public school teachers to pass a standardized test such as the Praxis. Although any barrier to entry is likely to raise wages in the affected occupation, the theoretical effects of such requirements on teacher quality are ambiguous. Teacher testing places a floor on whatever skills are measured by the required test, but testing is also costly for applicants. These costs shift teacher supply to the left and may be especially likely to deter high-quality applicants from teaching in public schools. Moreover, test requirements may disqualify some applicants that schools would otherwise want to hire. We use the Schools and Staffing Survey to estimate the effect of state teacher testing requirements on teacher wages and teacher quality as measured by educational background. The results suggest that state-mandated teacher testing is associated with increases in teacher wages, though we find no evidence of a corresponding increase in quality. Consistent with the fact that Hispanics have marked lower licensure scores than non-Hispanic Whites or Blacks, testing appears to reduce the fraction of new teachers who are Hispanic.

      1. Thanks for your productive reply. I confess to not immediately reading all the references, but having read much in the vein of teacher improvement agree that there is enormous scope for further and better analysis (and I appreciate the list). I wish that such analysis would more often precede the implementation of programs such as merit pay. Most analysis of teacher quality issues is as bad as teacher training itself. My own (partial) conclusion is that many successful teachers draw the qualities they need to be successful from how they were parented or schooled, not from how they were trained. I really don’t think that quality can be measured or, for the most part, induced with money. It could, however, be trained into prospective teachers, and it simply is not.

        I must say also that I am losing patience with international comparisons. Quite aside from my suspicion of sampling methods, nature of test, and conclusions drawn, I can’t stand it when we (Canada) score well on international tests but I know that our schools are still putting out kids who can’t read or do arithmetic to any appreciable extent. (The international test I’m most interested in seeing will never be done, but it would really be the most indicative of ed school training: comparative evaluation of the bottom decile or quartile). A nation’s school system has the mandate to meet that nation’s needs and that society’s expectations, and the feedback loops should make that happen without bogus international pageantry.

        Anyway, point taken that I may have misinterpreted your post as being more pro-status-quo than it was, and I look forward to further instalments.

      2. “I must say also that I am losing patience with international comparisons. Quite aside from my suspicion of sampling methods, nature of test, and conclusions drawn. . . ”

        Considering that the conclusions that are drawn are “vain and illusory”, in other words invalid, as shown by Noel Wilson in his As shown by Noel Wilson in “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700

        And his review of the testing bible “Standards for educational and psychological
        testing” from the.American Educational Research Association; American
        Psychological Association; National Council on Measurement in
        Education. (2002) “A Little Less than Valid: An Essay Review” found at:

        http://www.edrev.info/essays/v10n5index.html

        http://www.edrev.info/essays/v10n5.pdf

        My question to you, Karin, is “Have you read these works and if so what are your thoughts on them?”

      3. It’s bad enough that we have had so many non-educators in this country, over the past 30 years, who don’t know jack about K12 and Ed Schools, attacking our system of K12 education and teacher preparation and making policy decisions –so what they have decided for three decades is actually the status quo. We don’t need any know-it-all non-educators from other countries trashing us, too. If Karin wants to tell everyone that Ed Schools are the source of all evil, she can use her non-educator credentials to provide evidence of that for the colleges in her own country of Canada, instead of judging the US from afar.

  8. Thank you Bruce Baker for this excellent post. Readers may be interested to know that the ‘Industry of Mediocrity’ phrase was coined at the media release of the NCTQ “Teacher Prep Review” (which has been categorically discredited for failure to meet basic evaluation research standards, including documented bias in promoting alternative certification programs). This collection includes an array of articles/links debunking the NCTQ http://bit.ly/res_eval

    It should be of grave concern to all that EdTrust (Gates-funded/TFA-infused) has based its *recommendations* for FEDERAL OVERSIGHT of Teacher Education on the unscientific NCTQ ‘review’ (with vast amounts of missing data for nearly 90% of the nation’s ed schools.)

    Duncan seems bought in (or bought by) the NCTQ spin and has been actively promoting the Teacher Prep Review as well as the Keller piece. My guess is that with the massive financial interests of the corporate ed reformers in pushing their standardization, testing, and privatization agendas (as happened a few weeks ago with TFA candidates being deemed “highly qualified” for another two years http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/10/16/the-debt-deals-gift-to-teach-for-america-yes-tfa/ (despite widespread opposition from education organizations: http://hecse.net/policy_documents/CTQFY14SenateAppropsLetter_9_12_13.pdf), it will likely just be a matter of time before the EdTrust recommendations become federal policy.

    Here is the Inside Higher Ed piece that features the NCTQ-based recommendations (last paragraph): http://www.insidehighered.com/news/2013/10/25/report-discusses-best-practices-evaluating-teacher-preparation-programs. I just read a few excellent posts on the Education Trust agenda on this same SchoolFinance101 blog from 2011… perhaps the time is ripe for another! Thank you!

Comments are closed.