Arne-Ology & the Bad Incentives of Evaluating Teacher Prep with Student Outcome Data

As I understand it, USDOE is going to go ahead with the push to have teacher preparation programs rated in part based on the student growth outcomes of children taught by individuals receiving credentials from those programs. Now, the layers of problems associated with this method are many and I’ve addressed them previously here and in professional presentations.

  1. This post summarizes my earlier concerns about how the concept fails both statistically and practically.
  2. This post explains what happens at the ridiculous extremes of this approach (a warped, endogenous cycle of reformy awesomeness)
  3. These slides present a more research based, and somewhat less snarky critique

Now, back to the snark.

This post builds on my most recent post in which I challenged the naive assertion that current teacher ratings really tell us where the good teachers are. Specifically, I pointed out that in Massachusetts, if we accept the teacher ratings at face value, then we must accept that good teachers are a) less likely to teach in middle schools, b) less likely to teach in high poverty schools and c) more likely to teach in schools that have more girls than boys.


Extending these findings to the policy of rating teacher preparation programs by the ratings their teachers receive… working on the assumption that these ratings are quite strongly biased by school context, it would make sense for Massachusetts teacher preparation institutions to try to get their teachers placed in low poverty elementary schools that have fewer boys.

Given that New Jersey growth percentile data reveal even more egregious patterns of bias, I now offer insights for New Jersey colleges of education as to where they should try to place their graduates – that is, if they want to win at the median growth percentile game.


It’s pretty simple – New Jersey colleges of education would be wise to get their graduates placements in schools that are:

  • 20% of fewer free lunch (to achieve good math gains)
  • 5% or lower black (to achieve good math gains)
  • 11% or lower free lunch (to achieve good LAL gains)
  • 2% or lower black ( to achieve good LAL gains)

Now, the schools NJ colleges of ed should avoid (for placing their grads) are those that are:

  • over 50% free lunch
  • over 30% black

That is, if colleges of education want to play this absurd game of chasing invalid metrics.

Let’s take a look at some of the specific districts that might be of interest.

Here are the districts with the highest and lowest growth producing teachers (uh… assuming this measure has any attribution to teacher quality).


Now, my New Jersey readers can readily identify the differences between these groups, with a few exceptions. Ed schools in NJ would be wisest to maximize their placements in locations like Bernards Twp, Essex Fells, Princeton, Mendham and Ridgewood. After all, what young grads wouldn’t want to work in these districts? And of course, Ed schools would be advised to avoid placing any grads in districts like East Orange, Irvington or Newark.


Rather, I am pointing out that newly adopted USDOE regulations in fact endorse this model by requiring that this type of data actually be used to consequentially evaluate teacher preparation programs.

It’s simply wrong. It’s bad policy. And it must stop!

And yes… quite simply… this is WORSE THAN THE STATUS QUO!

For further discussion on this point, I refer you to this post!






12 thoughts on “Arne-Ology & the Bad Incentives of Evaluating Teacher Prep with Student Outcome Data

  1. Perfect. Thank you for the well-done, to the point piece, which I’ll definitely be using.

    It’s an unbelievable mess of an idea. As a teacher educator at a univ that works hard to prepare our students to work in urban, high-poverty districts, I feel personally attacked and worried, but not surprised.

  2. So U.S. Secretary of Education Arne “Donuts” Duncan has decided that America’s collegiate teacher training programs are doing a lousy job of preparing undergraduate students to become professional educators. In order to reduce what he imagines to be the inadequacies of these programs he has announced the creation of a “plan.”

    There is more than a little irony in the fact that Duncan demands that teacher training programs be held accountable for what he imagines to be their failure. After all, he takes absolutely no responsibility himself for the last dozen years of the uninterrupted failure of his “school reform” efforts as the head of the public schools in Chicago and as U.S. Secretary of Education.

    Duncan refuses to understand what his failures should have taught him. The quality of a finished product is limited by the quality of the raw material no matter how skilled the craftsman. The academic success of a public school is a dependent upon the quality of the students it enrolls. The vast majority of children who fail in school do not need better teachers or better schools. They need better childhoods. Teachers who fail to educate badly raised children are the same teachers who succeed in educating well raised children.

    If Duncan would just spend more time playing pickup basketball games, perhaps he would do less harm to America’s schools and to America’s children.

  3. I have been teaching in Newark for fifteen years. I am currently in a renew school. What would Duncan’s recommended course of action be?

Comments are closed.