It’s time to just say NO! More thoughts on the NY State Tchr Eval System


This post is a follow up on two recent previous posts in which I first criticized consultants to the State of New York for finding substantial patterns of bias in their estimates of principal (correction: School Aggregate) and teacher (correction: Classroom aggregate) median growth percentile scores but still declaring those scores to be fair and accurate, and next criticized the Chancellor of the Board of Regents for her editorial attempting to strong-arm NYC to move forward on an evaluation system adopting those flawed metrics – and declaring the metrics to be “objective” (implying both fair and accurate).

Let’s review. First, the AIR report on the median growth percentiles found, among other biases:

Despite the model conditioning on prior year test scores, schools and teachers with students who had higher prior year test scores, on average, had higher MGPs. Teachers of classes with higher percentages of economically disadvantaged students had lower MGPs. (p. 1)

In other words… if you are a teacher who so happens to have a group of students with higher initial scores, you are likely to get a higher rating, whether that difference is legitimately associated with your teaching effectiveness or not. And, if you are a teacher with more economically disadvantaged kids, you’re likely to get a lower rating. That is, the measures are biased – modestly – on these bases.

Despite these findings, the authors of the technical report chose to conclude:

The model selected to estimate growth scores for New York State provides a fair and accurate method for estimating individual teacher and principal effectiveness based on specific regulatory requirements for a “growth model” in the 2011-2012 school year. p. 40

I provide far more extensive discussion here!  But even a modest bias across the system as a whole can indicate the potential for substantial bias for underlying clusters of teachers serving very high poverty populations or very high or very low prior scoring students. In other words, THE MEASURE IS NOT ACCURATE – AND BY EXTENSION – IS NOT FAIR!!!!! Is this not obvious enough?

The authors of the technical report were wrong – technically wrong – and I would argue morally and ethically wrong in providing NYSED their endorsement of these measures!  You just don’t declare outright, when your own analyses show otherwise, that a measure [to be used for labeling people] is fair and accurate!  [setting aside the general mischaracterization that these are measures of “teacher and principal effectiveness”]

Within a few days after writing this post, I noticed that Chancellor Merryl Tisch of the NY State Board of Regents had posted an op-ed in the NY POST attempting to strong-arm an agreement on a new teacher evaluation system between NYC teachers and the city. In the op-ed, the Chancellor opined:

The student-growth scores provided by the state for teacher evaluations are adjusted for factors such as students who are English Language Learners, students with disabilities and students living in poverty. When used right, growth data from student assessments provide an objective measurement of student achievement and, by extension, teacher performance.

As I noted in my post the other day, one might quibble that Chancellor Tisch has merely stated that the measures are “adjusted for” certain factors and she has not claimed that those adjustments actually work to eliminate bias – which the technical report indicates THEY DO NOT. Further, she has merely declared that the measures are “objective” and not that they are accurate or precise. Personally, I don’t find this deceitful propaganda at all comforting! Objective or not – if the measures are biased, they are not accurate and if they are not accurate they, by extension are not fair.

Sadly, the story of misinformation and disinformation doesn’t stop here. It only gets worse! I received a copy of a letter yesterday from a NY school district that had its teacher evaluation plan approved by NYSED. Here is a portion of the approval letter:

NYSED Letter

Now, I assume this language to be boilerplate. Perhaps not. I’ve underling the good stuff. What we have here is NYSED threatening that they may enforce a corrective action plan on the district if the district uses any other measures of teacher or principal effectiveness that are not sufficiently correlated WITH THE STATE’S OWN BIASED MEASURES OF PRINCIPAL AND TEACHER EFFECTIVENESS!

This is the icing on the cake!  This is sick- warped- wrong!  Consultants to the state find that the measures are biased, and then declare they are “fair and accurate.” The Chancellor spews propaganda that reliance on these measures must proceed with all deliberate speed! (or ELSE!!!!!!!). Then the Chancellor’s enforcers warn individual district officials that they will be subjected to mind control – excuse me – departmental oversight – if they dare to present their own observational or other ratings of teachers or principals that don’t correlate sufficiently with the state imposed, biased measures.

I really don’t even know what to say anymore??????????

But I think it’s time to just say no!

 

 

Advertisements

11 Comments

  1. At this point, I am arguing that these “evaluation systems” are analogous to using a rectal thermometer to measure your car’s speed. You *WILL* get a number, but it’s meaning will be curious at best.

    1. Even worse… the validity of any/all other numbers shall be measured against the validity of the first number already proven to be invalid.

  2. Wow! So if I read this correctly, the testing component is, let’s say only 20% of your evaluations, the other 80% must correlate or in other words be somewhat the same as the testing component. So, in other words, the flawed testing component becomes 100% of the evaluation!

    It is bad enough that States are forcing Schools to accept these testing components into their evaluations BUT to change the Schools APPR with the States correlation threat, not only is immoral but must be illegal!

  3. 3012-c 9. a. The department shall annually monitor and analyze trends and
    patterns in teacher and principal evaluation results and data to
    identify school districts, boards of cooperative educational services
    and/or schools where evidence suggests that a more rigorous evaluation
    system is needed to improve educator effectiveness and student learning
    outcomes. The criteria for identifying school districts, boards of
    cooperative educational services and/or schools shall be prescribed in
    the regulations of the commissioner.

    My guess would be that the regulation of the commissioner is that the evaluation be TOTALLY based on State assessment tests. Forget about all the other measurements stated in 3012-c and any local input into the evaluation system! DO IT MY WAY OR THE STATE TAKES OVER!

    1. Indeed, that is in effect what is required, if all other measures must be sufficiently correlated with the state measure.

  4. All of this coupled with reduced funding is the means to guarantee schools fail and therefore become a private business.It amounts to the total destruction of public education for the sake of profit.

Comments are closed.