Rating Ed Schools by Student Outcome Data?

Tweeters and education writers the other day were  all abuzz with talk by U.S. Secretary of Education Arne Duncan of the need to crack down on those god-awful schools of education that keep churning out teachers who don’t get sufficient value-added out of their students.

see: http://www.educatedreporter.com/2011/10/teacher-training-programs-missing-link.html?utm_source=twitterfeed&utm_medium=twitter

Once again, the conversations were laced with innuendo that it is our traditional public institutions of higher education that have simply failed us in teacher preparation. They accept weak students, give them all “As” they don’t deserve and send the out to be bad teachers. They, along with the lazy greedy teacher graduates they produce simply aren’t  cutting it, even after decades of granting undergraduate degrees and certifications to elementary and secondary teachers.

This is a long post, so I’ll break it into parts. First, let’s debunk a few myths – a) regarding who is cranking out degrees and credentials in the field of education and b) regarding whether education policy should ever be guided by the actions of Louisiana or Tennessee. Second, let’s take a look at teacher production and distribution across schools in a handful of Midwest & plains states.

Who’s crankin’ out the credentials?

Allow me to begin this post by reminding readers – and POLICYMAKERS – that many initial credentials for teachers these days aren’t granted at the undergraduate level – but rather as expedited graduate credentials. Further, the mix of institutions granting those degrees has changed substantially over the decades, and perhaps that’s the real problem?

Here’s the mix of masters degree production in 1990:

And again in 2009:

Yes, by 2009, thousands of teaching credentials and advanced degrees were being churned out each year by online mass production machines. Perhaps if we really feel that there has been a precipitous decline in teaching quality, these shifts may be telling us something! What has changed? Who is now cranking out the credentials/degrees?

Now, I’m no big fan of the types of accountability systems and self-regulation that have been in place for education schools (specifically credential granting programs) in recent years.I tend to feel that these systems largely reward those who do the best job filling out the paperwork and listing that they have covered specific content standards (a syllabus matching exercise), while many simply lack qualified faculty to deliver on such promises. For more insights, see:

• Wolf-Wendel, L, Baker, B.D., Twombly, S., Tollefson, N., & Mahlios, M. (2006)
  Who’s Teaching the Teachers? Evidence from the National Survey of Postsecondary
  Faculty and Survey of Earned Doctorates. American Journal of Education 112 (2) 273-
  300

A colleague of mine at the University of Kansas (we’ve now both moved on) used to joke that we should simply list on our accreditation forms the names of all of the already accredited institutions that are plainly and obviously worse than us (Kansas). That should be sufficient evidence, right?

But, simply because current systems of ed school accountability may not be cutting it does not mean that we should rush to adopt the toxic foolish policies being thrown out on the table in current policy conversations, including the recent punditry of Arne Duncan on the matter.

First, let’s dispose of the notion that Louisiana and Tennessee can ever be used as model states.

Specifically, we are being told that states must look to Louisiana and Tennessee as exemplars for reforming teacher preparation evaluation. Exemplars yes. Positive ones? Not so much. Allow me to point out that I don’t ever intend to consider Louisiana or Tennessee as a model for education policies until or unless either state actually digs their public education system out of the basement of American public schooling. These states are a disgrace at numerous levels, and not because they have high concentrations of low-income children. Rather, because both put little financial effort into their education systems and perform dismally. Both have large shares of children exported entirely. They are not models!  Here’s my stat sheet on the two:

Sure, not a single measure in the table above relates to the teacher evaluation proposals on the table. And true, these states have adopted novel (putting the best light on it) models for evaluating teacher preparation programs. But, when put into the context of these states, one will likely never know whether or if those models of teacher prep program evaluation are worth a damn. Further, when placed into a context of states with such a historic record of deprivation of their public education systems, one might even question the motives of the “crack down” on teacher education. Can a state really be serious about improving public education with the record presented above?

Suggesting that these states are now models because they have decided to rate teacher education programs on the basis of the test scores of students of teachers who graduated from each program does not, can not, make these states models.

Perils of evaluating teacher preparation programs by value-added scores of the students of teachers who graduated from them?

Here’s where it gets tricky and really messy and for at least three major reasons. The proposals on the table suggest that the quality of teacher preparation programs can somehow be measured indirectly by estimating the average effect on student outcomes of teachers who graduated from institution x versus institution y.  Further, somehow, evaluation of these teacher preparation programs can be controlled through state agencies, with specific emphasis on state accredited teacher producing institutions.

• Reason #1: Teachers accumulate many credentials from many different institutions over time. Attributing student gains of a teacher (or large number of teachers) to those institutions is a complex if not implausible task. Say, for example that a teacher in St. Louis got an undergraduate degree from Washington University in St. Louis, but not a teaching degree. The teacher got the position on emergency or temporary certification (perhaps through some type of “fellows” program) with little intent to make it a career – decided he/she loved teaching – and eventually got credentialed time through William Woods University (a regional mass producer of teacher and administrator credentials). Is the credential institution, or the undergraduate institution responsible for this teacher’s success or failure?

• Reason #2: If one looks at the data on the teacher workforce in any given state, one finds that teachers hold their various degrees from many, many institutions – institutions near and far. True, there are major producers and minor producers of teachers for any given labor market. But, in any given labor market or state, one is likely to find teachers with degrees from 10s to 100s of institutions. In some cases, there may be only a few teachers from a given institution (for example Michigan State graduates teaching in Wisconsin).  That makes it hard to generate estimates of effectiveness. Should states simply cut off these institutions? Send their graduates home? Never let them in? Further, while teachers do in many cases come from within-state public institutions, they also come from a scattering of institutions in border states, especially where metropolitan labor markets spread across borders.  Value-added estimates of teacher effectiveness will depend partly on state testing systems (ceiling effects, floor effects).  What is an institution to think/do when its graduates are rated highly in one state’s value-added model, but low in another? Does that mean they are good, for example at teaching Iowa kids but not Missouri ones? Iowa curriculum but not Missouri curriculum? Or simply whether the underlying scales of the state tests were biased in opposite directions? Can/should states start to erect walls prohibiting inter-state transfer of credentials? (after years of working toward the opposite!)
• Reason #3: It will be difficult if not entirely statistically infeasible to generate non-biased estimates of teacher program effectiveness since graduates are NOT RANDOMLY DISTRIBUTED ACROSS SETTINGS. I would have to assume that what most states would try to do is to estimate a value-added model which attempts to sort out the average difference in student gains of teachers from institution A and from institution B, and in the best case, that model would include a plethora of measures about teaching contexts and students. But these models can only do so much in that regard. While this use of the value-added method may actually work better than attempts to rate the quality of individual teachers, it is still susceptible to significant problems, mainly those associated with non-random distribution of graduates. Here are a few examples from the middle of the country:

The first focuses on recent graduates of in-state Kansas institutions and the characteristics of schools in which they worked during their first year out. The average rate of children qualified for subsidized lunch ranges from under 20% to nearly 50%. Further, this average actually varies to this extent largely because teachers are sorted into geographic pockets around the state which differ in many regards. The most legitimate statistical comparisons that can be made across teacher prep graduates from these institutions are the comparisons across those working in similar settings. In some cases, the overlap between working conditions of graduates of one institution and another is minimal. And Kansas is a relatively homogeneous state compared to many!

Here’s Missouri, with teachers having 5 or fewer years of experience, and the percent free or reduced price lunch in schools where the teachers currently work. I’ve limited this figure to those institutions producing only very large numbers of Missouri teachers, which is less than half of the entire list. Notably, many of these institutions are from border states, including University of Northern Iowa and Arkansas State University. These universities tend to produce teachers for the nearest bordering portions of Missouri.

Again, there are substantial differences in the average low-income population in schools of graduates from various universities. Not here that graduates of the state flagship university – University of Missouri at Columbia – tend to be in relatively low poverty schools. Assuming the state testing system does not suffer ceiling effects, this may advantage Mizzou grads. Kansas grads above have a similar advantage in their state context. Graduates of Arkansas State, and of Avila College near Kansas City may not be so lucky.

Just to beat this issue into the ground… here’s a Wisconsin analysis comparable to the Missouri analysis. Graduates of Milwaukee area teacher prep institutions including UW-Milwaukee, Marquette and Cardinal Stritch may have significant overlap in the types of populations served by their graduates. But most are in higher poverty settings than graduates of the various state regional colleges. Again, only the BIG producers are even included in this graph. And the differences are striking statewide. And graduates are substantially regionally clustered further complicating effectiveness comparisons across teacher producing institutions.

These are just illustrations of the differences in one single parameter across the schools/students of graduates of teacher preparation programs. The layers difference in working conditions go much deeper, and include, for example, substantial variations in average class sizes taught, as well as significant often unmeasured neighborhood level differences in diverse metropolitan areas. Teacher labor markets remain relatively local. Teachers remain most likely to teach in schools like the ones they attended, if not the exact ones. Teacher placement is non-random. And that non-randomness presents serious problems for evaluating the quality of teacher preparation programs on the basis of student outcomes.

Is it perhaps interesting as exploratory research to attempt to study the relative “efficacy” of teacher prep programs by these and other measures to see what, if anything, we can learn? Perhaps so.

Is it at all useful to enter so blindly into using these tools immediately in making high stakes accountability decisions about institutions of higher education? Heck no! And certainly not because policymakers in Louisiana or Tennessee said so!

The Curious Duplicity of NCTQ

NCTQ fashions itself as a leading think tank on promoting teacher quality in K-12 education. NCTQ adopts a relatively extreme position that teacher quality is the one and only thing that matters! Teacher quality is THE determining factor of school quality.

I also believe that teacher quality is very important. I also agree with NCTQ on the point that content knowledge, at the middle and secondary levels especially, is particularly important and that simply being listed as “qualified” to teach specific content is no guarantee.

As part of their effort to improve teacher quality, NCTQ has been going around doing “studies” and applying ratings to the quality of teacher preparation institutions. Now, I noted on my previous post that NCTQ and others may actually be missing the boat on who is actually preparing teachers. But lets set that aside for a moment. One would think that if NCTQ is so interested in teacher quality as the primary determinant of school quality and student success, and teacher expertise as an important part of that equation at higher grade levels, that any analysis of the quality of undergraduate or graduate programs to train teachers would have to place significant emphasis on faculty quality and expertise? right? It would make little sense to simply review which textbooks are used or what the course descriptions say, or what the curricular sequence happens to be? Right?

Out of a multitude of indicators on teacher preparation institutions, NCTQ includes only 1 – yes 1 – regarding faculty quality, which is described as follows:

In our evaluation of programs, we examined teaching responsibilities for all faculty members, as indicated by course assignments in course schedules, excluding all clinical coursework. We looked for two specific examples of inappropriate assignments: 1) an instructor teaching across the areas of foundations of education, methods and educational psychology; and/or 2) an instructor who teaches both reading and mathematics methods courses. Other inappropriate assignments may well be made but were not included in our review.

http://www.nctq.org/edschoolreports/illinois/standards/26Methodology.jsp

Yep, that’s it. All that they address is whether a faculty member appears to teach across two areas that no faculty member, in their view, could be sufficiently prepared to teach. The rest is based largely on textbooks chosen, syllabi and course descriptions, regardless of faculty expertise. Clearly this was a matter of data convenience. It’s hard to figure out whether individual faculty members truly possess expertise in their fields, short of evaluating their individual academic backgrounds, research and writing on the topic.

But it is absurd for an organization that believes teacher quality in K-12 education paramount, and content expertise critical, to ignore outright faculty expertise in their evaluations of teacher preparation institutions.

Here’s their FAQ on the long-term project of evaluating teacher preparation programs: http://www.nctq.org/p/response/evaluation_faq.jsp

Related reading (actual research):

Wolf-Wendel, L, Baker, B.D., Twombly, S., Tollefson, N., & Mahlios, M. (2006) Who’s Teaching the Teachers? Evidence from the National Survey of Postsecondary Faculty and Survey of Earned
Doctorates. American Journal of Education 112 (2) 273-300

Ed Schools

Ed schools seem to make an easy target in public policy debates over the quality of American public schooling and the American teacher workforce.

In many recent lopsided “ed school as the root of all evil” presentations, “Ed Schools,” are treated as some easily defined, static entity over time. In the book of reformyness (chapter 7, verse 2), “Ed Schools” necessarily consist of some static set of traditional higher education institutions – 4 year teachers colleges including regional state colleges and flagship universities – where a bunch of crusty old education professors spew meaningless theory at wide-eyed undergrads (who graduated at the bottom of their high school class) seeking that golden ticket to a job for life – with summers off.

In order to craft a clearly understandable (albeit entirely false) dichotomy of policy alternatives, pundits then present teachers who have obtained alternative certification as a group of individuals, nearly all of whom necessarily attended highly selective colleges and majored in something really, really rigorous and then received their certification through some more expeditious and clearly much more practical and useful fast-tracked option.

This was certainly the theme of a discussion (hashtag #edschools) at Thomas B. Fordham Institute actively tweeted the other day by Mike Petrilli and a few others.  What I found most interesting was that no-one really challenged the assumptions that “ed schools” are some easily definable group of traditional higher education institutions – that this has been unchanged over decades – and that teacher training is some consistent, exclusive domain of traditional public higher education institutions – specifically as an undergraduate degree granting enterprise? That there are and have always been, oh… about a thousand or so ed schools… that well… keep on doing the same damn thing over and over again (for the past 50 years, one participant tweeted) … and well… no one ever shuts down the bad Ed Schools… and that’s why we’re in such bad shape! It’s really that simple.

Because this characterization is simply assumed to be true, the obvious way to crack this broken and declining system is to expand alt. certification and allow more non-traditional, for profit and entrepreneurial organizations – especially non-university organizations to grant teaching credentials – heck – let’s let them actually grant degrees. Who needs brick-and-mortar colleges anyway? Given the assumed static nature of the declining and antiquated system of “Ed Schools” that has brought us to our knees, this is the only answer!!!!!

One of my favorite tweets from the event was from Mike Petrilli, relaying a comment by Kate Walsh:

Walsh: There are 1410 Ed schools in the country. NCTQ spent 5 years determining that number.

You know what Kate, by the time you were done figuring that out (however you did), the number had already changed. Also, FYI, there are actually some data sources out there that might have been helpful for tabulating the existing degree granting programs and the numbers of degrees conferred by those programs.

So, let’s take a look at some of the data on degrees conferred across all education fields in 1990, 2000 and 2010.

Let’s start with a quick look at the total degrees conferred in “education” as defined by degree classification codes (CIP Codes), across all institutions granting such degrees nationally. The interesting twist here is that bachelor’s degree production of education degrees has been relatively constant over time for about 20 years and perhaps longer. Doctoral degree production increased from 1990 to 2000, but stagnated after that. On the other hand, Master’s degree production has skyrocketed.

Now, one might try to argue that what that’s really about is all of those currently practicing teachers who are just accumulating those worthless master’s degrees to get that salary bump. I will write more on this topic at a later point, but that’s not likely the dominant scenario. Yes, many of the master’s degrees are obtained to broaden fields of certification in order to give current teachers more options – either assignment options in their current districts, or other job opportunities. AND, many of the masters degrees these days are initial credentials granted to individuals who did not receive their teaching credential as an undergraduate. Many initial teaching credentials are granted at the master’s, not bachelor’s level. A substantial amount of teacher training goes on at the master’s, not undergraduate level. No matter the case, the master’s degrees – of which there are so many – and so many more being granted than bachelors degrees – are the interesting story here.

Is it really that the same old traditional higher education institutions with crusty old, out of date professors, are now just spewing out masters degrees? Or is something else at work here?

Well, here are the top 25 MA producers in education back in 199o. Even at that time, the largest master’s degree granting institutions were not the top universities – or even the top teachers colleges. But, some of those schools were at least in the mix. Teachers College of Columbia University, Ohio State, Michigan State and Harvard all appear in the top 25 in 1990.

Here are the top 25 master’s producers in 2000. Here, the tide begins to shift a bit. Schools like NOVA Southeastern with their online programs, and National-Louis grow even bigger than they had been a decade earlier. Teachers College retains a top 25 spot, as does Ohio State, and University of Minnesota makes the list. Harvard is gone.

By 2009, “Ed Schools” are a substantially different mix. Not only that, but look at the volume of degree production. Back in 1990, Ed Schools at respectable major universities were putting out about 600 master’s degrees in education related fields per year. They held on to similar rates in 2000 and still in 2009. But by 2009, Walden University and U. of Phoenix were each cranking out 4,500+ master’s degrees per year. Grand Canyon U. comes in next in line. These are the entrepreneurial up-starts that are the product of minimized regulation of teaching credentials.

If there truly has been a decline in the quality of the teacher workforce, and if pundits truly believe that this supposed decline is related somehow to “Ed Schools,” then it might behoove those same pundits to explore the dramatic changes that have, in fact, already occurred in the “Ed School” marketplace.

If there has been a dramatic decline in teacher preparation, and in specialized training, it may be worth taking a look at those institutions that have emerged to dominate the production of education degrees and credentials in recent years. After all, Walden and Phoenix each produce 5 to 10 times the master’s degree credentials in education of major public universities. And, production of education master’s degrees is now nearly double the level of production of education bachelor’s degrees. And many of these entrepreneurial start-ups specifically frame their master’s programs as an option for individuals with a bachelor’s degree in “something else” to obtain a teaching credential.

Is even more deregulation and entrepreneurial teacher preparation what we really need? Can one really blame the traditional higher education institutions, whose share of production has declined steadily for decades, for declining teacher quality? Only if you ignore these trends, which I expect these pundits will continue to do.

 

The research question that wasn’t asked

Recent discussions of the Vanderbilt University study on the effect of merit pay in Nashville raised a common and important issue pertaining to education policy research – or any research for that matter – What about the question that wasn’t asked? Or how important really is the question that was asked?

In the case of the Vanderbilt merit pay study, the researchers essentially asked whether providing sizable financial bonuses to randomly selected teachers could motivate those teachers to try harder and ultimately produce better student outcomes than teachers randomly selected to be in the group that could not get bonuses. That is, does the merit pay serve to make one randomly selected group of teachers produce better student outcomes than a control group?

Pundits quickly leaped on the question NOT ASKED – which was whether or not changing teacher compensation structures more generally – making teaching a profession based on rewards for performance or a profession where one could increase income over time by being a high performer would ultimately change the quality of individuals who would enter the teaching workforce.

That is, the study asked whether financial incentives could change the behavior of those already in the system, but not whether the existence of performance incentives would change those who choose to be in the system.

Now, when “reformy” types pointed to this question NOT ASKED, they also seemed to uniformly imply that we know the answer to the question not asked – and that is – “of course this would encourage better teachers to enter the labor market.” You know what – the question wasn’t asked. It wasn’t tested and we certainly do not know this to be the answer. For now, the answer is “we don’t know,” and it is likely fair to say that the answer is “it depends, on a variety of factors including how compensation is altered, the risk/reward ratios, etc. etc. etc.”

This brings me to a comment made by Andrew Rotherham in his recent Time Magazine post:

For example, it’s clear from abundant research that paying teachers only on the basis of their degrees and years of experience is not in the best interest of students or teachers. As the National Council on Teacher Quality, a research and policy organization whose board of directors I chaired for several years, put it, “the evidence is conclusive that master’s degrees do not make teachers more effective.”

Now, I think even this statement is a bit, well, overstated. The “research” Rotherham seems to draw on here (and NCTQ dreadfully overstates) is research that asked the following questions:

1. Do teachers who hold general masters degrees, versus those who do not, scattered across a variety of settings, show differences in the average outcome gains of their students?
2. Do teachers at varied levels of experience, scattered across a variety of settings, show differences in the average outcome gains of their students?

The first of these questions was beaten into the ground over and over in the 1990s, often using data from the National Education Longitudinal Studies (NELS ’88) with many of the studies showing no relationship between holding a masters or not and student outcomes, and at least a few showing positive effects of holding a content area masters in math/science (I’m doing this largely from memory).

The second of these questions has been addressed in a number of recent analyses, as well as some older ones. More recent studies have generally evaluated the average student value added ratings of teachers by their experience levels. Many of these studies find that teachers in their first two to three years tended to show smaller student achievement gains than teachers in their 4^th, 5^th or 6^th years, but after that, things really kind of level off. Here’s an example of such analysis: http://www.urban.org/UploadedPDF/1001455-impact-teacher-experience.pdf

Interestingly, pundits pushing so hard for major changes to the risk/reward structure of teacher compensation who are so quick to point out the question not asked in the Vanderbilt merit pay study fail to recognize that similar labor market questions were never asked in these studies either.  Researchers asked whether teachers with certain attributes had better student outcomes than teachers with different attributes. As far as I recall, no one ever asked whether differential compensation on the basis of these attributes produced any desirable or undesirable labor market effects – changes to the applicant pool, etc.

Studies of the association between different levels of experience and the association between having a masters degree or not and student achievement gains have never attempted to ask about the potential labor market consequences of stopping providing additional compensation for teachers choosing to further their education – even if only for personal interest – or stopping providing any guarantee that a teacher’s compensation will grow at a predictable rate over time throughout the teacher’s career.

Many, like Rotherham but even more so, NCTQ, present this as a “research given.”  That clearly, it’s just dumb to pay teachers more who possess attributes we know are not associated with student achievement differences (across teachers). Is it possible, however, that changing these conditions could have significant labor market consequences? Perhaps good… but equally likely… unintended negative consequences.

Yes, teachers with any old masters degree or teachers with more than 10 years behind them might not, on average, be “measurably more productive.” But does the option to pay and recruit more experienced teachers or teachers with masters’ degrees enhance the likelihood that a district can attract teachers who are actually better teachers? I’m not so sure that the answer to this question unasked is so obvious that we need not ask it. So let’s stop pretending that it is.

NCTQ: We’re sure it will work! Even if research says it doesn’t!

Last spring, I had the pleasure of presenting on teacher labor market research in the same conference session in which a very interesting paper on mutual consent teacher contract changes was also presented (by Bethany Gross). This paper is a product of an organization I’ve poked fun at in the past (Center for Reinventing Public Education) but this one is good stuff, by credible authors.  The methods are relatively tight, but it is a bit tricky to figure out the implications of the findings – discussed blow.  This study fits into the broader topic and policy concern of “how do we get a better balance of teacher quality across poorer and less poor schools in the same district?”

Now, pundits (not these researchers) like those from Center for American Progress, Education Trust, some from CRPE and those from New Teacher Project and National Council on Teacher Quality (NCTQ) all seem to argue that the biggest teacher quality/sorting problems are those that occur across rich and poor kids within school districts – all because of teacher seniority preferences, tenure and contractual issues which all favor the interests of adults over the interests of children – like letting the senior teacher keep his/her job in the cushy school in the district (or letting the senior teacher bump the junior teacher from such a position).

As I’ve shown in many recent posts, in general, rich kids and poor kids, black kids and white kids don’t often attend the same school districts – but for a few very large urban ones and some other sprawling countywide systems. The BIG disparities in resources and teacher characteristics are across not within districts. They are disparities that result across different bargaining units – not within bargaining units. So it’s pretty hard to argue that most disparities in teacher quality across rich and poor kids from one location to another are caused by seniority focused, adult interest, contractual provisions.

But, setting that broader issue aside, what do we actually know about within district disparities in the distribution of teacher characteristics, and whether changing contracts to remove these “offensive” protections can actually help redistribute teacher quality? Well, here’s what Gross and colleagues found:

http://www.nctq.org/docs/Mutual_Concent_8049.pdf

We conduct an interrupted time-series analysis of data from 1998-2005 and find that the shift from a seniority-based hiring system to a “mutual consent” hiring system leads to an initial increase in both teacher turnover and share of inexperienced teachers, especially in the district’s most disadvantaged schools. For the most part, however, these initial shocks are corrected within four years leaving little change in the distribution of inexperienced teachers or levels of turnover across schools of different advantage.

So, initially the policy change actually made things worse and in the end, the policy change made things no different. There may actually be some reasonable explanations for these findings. Perhaps most problematic, teachers who are really beginning to hit their stride in years 5 to 10 or so, might take advantage of their newly discovered mobility to jump more quickly from positions in higher poverty, higher need schools into more desirable positions once reserved for the most senior teachers. This could create a substantial drain of quality – non-novice but not really old – teachers from high need schools.

Notice the URL for this study. It is posted on the NCTQ website. That doesn’t mean they ever read it though. SOMEHOW, THE NEW NCTQ REPORT WHICH ARGUES THAT THESE CHANGES ARE PART OF THE SOLUTION, DOESN’T PAY ANY ATTENTION TO THIS!

Here’s a link to that report: http://www.nctq.org/tr3/docs/nctq_site_based_hiring.pdf

Here’s the list of the 3 “major barriers” to improving the distribution of teacher quality, as identified by NCTQ:

• Centralized hiring. In most districts, the human resources office controls the hiring process, determining whom to recruit and hire and where to place teachers. Principals, at most, are given the opportunity to voice their preferences.

• Inadequate evaluations. Teachers in most districts are not regularly, or sufficiently, evaluated, meaning that evaluations can only play a minor role in personnel decisions, when they should be paramount. It is seniority, not performance, that decides the movement of teachers within the district.

• Contractual obligations. Most teacher contracts stipulate that, if a teacher loses her current assignment—because of a shift in the student population, for example—the district has to find her a new assignment, regardless of whether another school wants to accept the teacher. Compounding the problem is that most state laws limit the reasons districts can dismiss a teacher, and being without a classroom assignment is not one of them. Districts are left with little choice but to either assign teachers to positions or keep them on the payroll, sometimes for years, even if they aren’t teaching.

See that third one – Yep – it’s those  contractual provisions that keep these disparities in place. Remove then and all will be fixed!

Now briefly on their first point – that centralized hiring is the other really big problem. The answer – let school site principals make decisions and teachers decide which principal they really want to work for in a district. That couldn’t backfire?   Well, I used to believe the same – that this could be a reasonable idea. The problem with this idea is that principal quality is so disparately distributed. I have recently worked on several studies of principal labor markets, the distribution of principals by their academic preparation and other factors across schools within districts and the relationship between principal attributes and the teachers they hire. Given what we are learning from these studies, it is in fact very likely to backfire! The weakest principals tend to be in the highest need schools and weak principals tend to attract and potentially even retain weaker teachers.

This line in the NCTQ press release is particularly fun, because it’s based on nothing but “gut” and “emotional appeal” – which is always the best basis for experimenting with the lives of low income and minority children, right?

“Giving principals the authority to hire who works on their staff is critical,” says Kate Walsh, NCTQ’s president. “It is the only fair way to hold schools accountable for results. But if the principal doesn’t have enough control over the quality of her staff, the school—and, of course, the students—will suffer.”

You know what – not addressing the larger resource disparities across and within districts – which lead to the disparities in leadership quality across schools and districts – and then handing greater control over teacher hiring/firing to the least qualified principals in the highest need schools – yeah… that’s when children will suffer. Even if we start by getting good principals where they are needed most, we must provide them the resources to attract and retain the “better” teachers.

To summarize:

1. Decentralizing control of teacher hiring to principals, where principal quality distribution is disparate, to the disadvantage of high need schools, is likely to lead to worse, not better distribution of teacher quality;

2. Altering contractual provisions, such as moving from seniority based to mutual consent placement, appears to disadvantage higher need schools initially and in the end, leads to little or no substantive change in the distribution of teachers across schools.

Yet, let’s go with it. What the heck. Why not – it chips away at those facially offensive protections of stubborn old selfish, lazy unproductive teachers (yes, I do know a few, but that’s not the point). We know it should work, even if we have no evidence to that effect. When organizations like NCTQ present policy recommendations against their own evidence and built on such flimsy logic, why do we even listen?

Additional resources

The NCTQ report above concludes with a rant about ESEA Comparability Regulations. For my thoughts, see: https://schoolfinance101.wordpress.com/2010/05/31/research-schmresearch-caps-misguided-analysis-again/

Regarding financial resources, “autonomy” and the distribution of teachers, see: Baker, B.D. Re-arranging deck chairs in Dallas: Contextual constraints on within district resource allocation in large urban Texas school districts. DeckChairsinDallas.Baker

Regarding the distribution of school leaders, teacher hiring and all that stuff, see:

Fuller, E., Young, M.D., Baker, B.D. Career Paths and the Influence of School Principals on Teachers. (Available on request)

Baker, B.D., Fuller, E. The Declining Academic Quality of School Principals and Why it May Matter. Baker.Fuller.PrincipalQuality.Mo.Wi_Jan7

Punswick, E., Baker, B.D., Belt, C. Principal backgrounds and school leadership stability: Evidence from Missouri. Educational Administration Quarterly Punswick.Baker.Belt.MoPrins09

Principal moves/exits: http://eaq.sagepub.com/content/46/4/523.abstract

Principals and hiring: http://eaq.sagepub.com/content/41/3/449.abstract

(the references in the above articles may provide some additional useful guidance on the role and the current distribution of principals)

NCTQ Teacher Policy Ratings: Where’s the quality?

First, to the media – the National Council on Teacher Quality Ratings are NOT ratings of actual differences in Teacher Quality across states. They are ratings of supposed steps which can be taken in state policy in order to improve teacher quality. Here, the blame goes on the media spin, not on NCTQ.

NCTQ does make some reasonable attempts to explain the research basis for their policy elements.  However, NCTQ fails miserably at understanding the importance of context within which policies are applied. For example, under AREA 2, NCTQ cites the importance of increasing numbers of teachers from more competitive colleges, and cites expanding the teacher pool as a way to accomplish this, through policies such as alternative certification. My own work a few years back on charter school hiring in states with more and less relaxed teacher certification requirements provides some support for this notion. But, my research also shows that in some cases, expanding the pool weakens, on average, the academic credentials of teachers. Some states and some regions of the country simply don’t have more competitive colleges and universities.

As many of these rating/grading systems which strongly favor deregulatory policies (and the power of state data systems) do, the NCTQ policy ratings favor those states that in fact have the weakest overall public education systems including the academically weakest teachers – of all things. NCTQ only handed out Cs and Ds for grades (and a few Fs). A quick tally based on my prior analyses of Schools and Staffing Survey Data finds that 6 of the 8 states that got a C (the high grade) fall in the bottom half of states in the percentage of teachers who attended highly or most competitive colleges (a factor acknowledged by NCTQ as important, and as a factor that would supposedly improve as a function of expanding the teacher pool). Louisiana, Alabama and Arkansas are all in the bottom 10. Most of these states also fall in the bottom half of states, and 3 in the bottom 10 states for the change in percent of teachers (03-04 to 07-08) who attended highly or most competitive colleges. None of the states that received the high grade were even in the top 20 in change in % of teachers from highly or most competitive colleges.

You know – it’s possible that teacher salaries might also be a factor here (there’s some pretty good research on this-see link), and a limiting condition might actually be the available funding for schools which is sadly lacking in many of these states. So too might the supply of high quality public colleges and universities for preparing teachers. States like Louisiana have been taking the axe to their public higher education systems of late. Deregulatory strategies cannot trump these conditions, and in fact, may worsen teacher quality and ultimately school quality under these conditions.

Increased regulatory strategies like improved data for teacher evaluation systems (also advocated by NCTQ, and quite reasonably so) are simply window dressing for states that are choosing to avoid the more difficult and more expensive problems facing their public education systems.

On numerous occasions on this blog, I’ve discussed the systemic failures of the public education systems in states like Louisiana – their failure to serve even 80% of school-aged children – or their failure to provide reasonable overall funding or target any funding to higher need districts (across most of these states).

So, if the Teacher Quality Policy ratings have little to do with actual teacher academic preparation in a state, or overall quality of the state’s education system, then what do they tell us? Apparently not much!