When I first criticized the use of SGPs for teacher evaluation in New Jersey, the creator of the Colorado Growth Model responded with the following statement:
Unfortunately Professor Baker conflates the data (i.e. the measure) with the use. A primary purpose in the development of the Colorado Growth Model (Student Growth Percentiles/SGPs) was to distinguish the measure from the use: To separate the description of student progress (the SGP) from the attribution of responsibility for that progress.
Let’s parse this statement one more time. The goal, of the SGP approach, as applied in the Colorado Growth Model and subsequently in other states is to:
…separate the description of student progress (the SGP) from the attribution of responsibility for that progress.
To evaluate the effectiveness of a teacher on influencing student progress, one must certainly be able to attribute responsibility for that progress to the teacher. If SGP’s aren’t designed to attribute that responsibility, then they aren’t designed for evaluating teacher effectiveness, and thus aren’t a valid factor for determining whether a teacher should have his/her tenure revoked on the basis of their ineffectiveness.
It’s just that simple!
Employment lawyers, save the quote and link above for cross examination of Dr. Betebenner when teachers start losing their tenure status and/or are dismissed primarily on the basis of his measures – which by his own recognition – are not designed to attribute responsibility for student growth to them (the teachers) or any other home, school or classroom factor that may be affecting that growth.
(Reiterating again that while value added models do attempt to isolate teacher effect, they just don’t do a very good job at it).
Bruce Baker is an Professor in the Graduate School of Education at Rutgers, The State University of New Jersey. From 1997 to 2008 he was a professor at the University of Kansas in Lawrence, KS. He is lead author with Preston Green (Penn State University) and Craig Richards (Teachers College, Columbia University) of Financing Education Systems, a graduate level textbook on school finance policy published by Merrill/Prentice-Hall. Professor Baker has written a multitude of peer reviewed research articles on state school finance policy, teacher labor markets, school leadership labor markets and higher education finance and policy. His recent work has focused on measuring cost variations associated with schooling contexts and student population characteristics, including ways to better design state school finance policies and local district allocation formulas (including Weighted Student Funding) for better meeting the needs of students.
Baker, along with Preston Green of Penn State University are co-authors of the chapter on Conceptions of Equity in the recently released Handbook of Research Education Finance and Policy, and co-authors of the chapter on the Politics of Education Finance in the Handbook of Education Politics and Policy and co-authors of the chapter on School Finance in the Handbook of Education Policy of the American Educational Research Association.
Professor Baker has also consulted for state legislatures, boards of education and other organizations on education policy and school finance issues and has testified in state school finance litigation in Kansas, Missouri and Arizona. He is a member of the Think Tank Review Panel, a group of academic researchers who conduct technical reviews of publicly released think tank reports on education policy issues.
View all posts by schoolfinance101
Published
8 thoughts on “Briefly Revisiting the Central Problem with SGPs (in the creator’s own words)”
How large are these schools that teachers can’t be evaluated by principals? I always thought that was the purpose of having a principal. My brother used to teach (like 35 years ago). He feels that the quality of the school is entirely dependent on the principal.
Policymakers don;t trust that principals can make good judgments about teachers. They infer that since 97% of teachers are rated satisfactory or greater on teacher observation and less than 97% of students are proficient, then principals don;t know what they are doing. There are so many problems with this line of thinking that entire blog posts have been written about it. But that is what it comes down to–policymakers don’t trust principals to get rid of bad teachers. Policymakers trust “science”–even when the science was designed to do something completely different than how they want to use it–over educators.
What about the infamous EVAAS? The most recent version of the Ehlert et all paper waffles on whether EVAAS uses appropriate covariates. Their PA description mentions no covariates–only a truckload of prior scores.
Indeed, an oversimplified, original Sanders’ style EVAAS is equally problematic.
@Ed: “Policymakers don;t trust that principals can make good judgments about teachers.” This is much too facile. It is not just that policymakers think this: both principals and teachers think this, too. There are some great anecdotes in the CCSR report about the re-vamped teacher appraisal systems, and how principals (who acknowledge this) find it difficult to give critical feedback to teachers. I’ve seen similar tensions across Texas, and I’ve seen this by comparing teacher appraisal scores between campus/non-campus staff in a large district. This isn’t at all surprising.
And this works the other way, too. Teachers are often, legitimately, concerned about the subjectivity of principal ratings (“principal has it out for me/doesn’t like me”: I’m sure you’ve heard these). Which then forces people to consider supplementary measures like student achievement (or a wholesale redesign of the appraisal system). But, then that’s flawed, since that doesn’t account for pre-existing differences in the students that are assigned to teachers. But, then that leads to more advanced statistical techniques to account for these differences. But, then these are flawed for too many reasons to go into, but Bruce and others have covered extensively. So, then, I’m not sure where this leads us, but it sounds as though you are endorsing investing this authority solely with the principal, which I think is not necessarily wise, nor palatable to a lot of teachers.
Actually, there is some good research out there that principals are pretty darn good at identifying effective and ineffective teachers. No, I don’t think principals should have complete authority to fire teachers, although they certainly should have a large say in it. As Bruce suggests, I’d use the best VAM out there as a screening tool to determine where the principal should invest her/his time. If a principal determined a teacher should be removed, independent observers (someone outside the school who possesses instructional and content knowledge in the subjects that the teacher is assigned to teach) could review that decisions before the removal occurs.
@Ed: Indeed, there is. But, the irony is that, as I’m sure you know, in a lot of that work, objective measures of student performance (e.g., VAM results), are often used to assess the validity of these principal ratings, or at least to anchor them to something, and there are far fewer blog posts written about the shortcomings of these evaluations than of VAM models.
School Finance 101 
How large are these schools that teachers can’t be evaluated by principals? I always thought that was the purpose of having a principal. My brother used to teach (like 35 years ago). He feels that the quality of the school is entirely dependent on the principal.
Policymakers don;t trust that principals can make good judgments about teachers. They infer that since 97% of teachers are rated satisfactory or greater on teacher observation and less than 97% of students are proficient, then principals don;t know what they are doing. There are so many problems with this line of thinking that entire blog posts have been written about it. But that is what it comes down to–policymakers don’t trust principals to get rid of bad teachers. Policymakers trust “science”–even when the science was designed to do something completely different than how they want to use it–over educators.
What about the infamous EVAAS? The most recent version of the Ehlert et all paper waffles on whether EVAAS uses appropriate covariates. Their PA description mentions no covariates–only a truckload of prior scores.
Indeed, an oversimplified, original Sanders’ style EVAAS is equally problematic.
@Ed: “Policymakers don;t trust that principals can make good judgments about teachers.” This is much too facile. It is not just that policymakers think this: both principals and teachers think this, too. There are some great anecdotes in the CCSR report about the re-vamped teacher appraisal systems, and how principals (who acknowledge this) find it difficult to give critical feedback to teachers. I’ve seen similar tensions across Texas, and I’ve seen this by comparing teacher appraisal scores between campus/non-campus staff in a large district. This isn’t at all surprising.
And this works the other way, too. Teachers are often, legitimately, concerned about the subjectivity of principal ratings (“principal has it out for me/doesn’t like me”: I’m sure you’ve heard these). Which then forces people to consider supplementary measures like student achievement (or a wholesale redesign of the appraisal system). But, then that’s flawed, since that doesn’t account for pre-existing differences in the students that are assigned to teachers. But, then that leads to more advanced statistical techniques to account for these differences. But, then these are flawed for too many reasons to go into, but Bruce and others have covered extensively. So, then, I’m not sure where this leads us, but it sounds as though you are endorsing investing this authority solely with the principal, which I think is not necessarily wise, nor palatable to a lot of teachers.
Actually, there is some good research out there that principals are pretty darn good at identifying effective and ineffective teachers. No, I don’t think principals should have complete authority to fire teachers, although they certainly should have a large say in it. As Bruce suggests, I’d use the best VAM out there as a screening tool to determine where the principal should invest her/his time. If a principal determined a teacher should be removed, independent observers (someone outside the school who possesses instructional and content knowledge in the subjects that the teacher is assigned to teach) could review that decisions before the removal occurs.
@Ed: Indeed, there is. But, the irony is that, as I’m sure you know, in a lot of that work, objective measures of student performance (e.g., VAM results), are often used to assess the validity of these principal ratings, or at least to anchor them to something, and there are far fewer blog posts written about the shortcomings of these evaluations than of VAM models.