Last week, Diane Ravitch called for a cite on a frequently used factoid from the Big Book of Eduformers: research shows that students with effective teachers make three times the progress of students with ineffective teachers.
More than a few commenters found the cite: Eric Hanushek, “The Trade-off between Child Quantity and Quality,” Journal of Political Economy 100 no. 1 (1992): 84-117 (at p. 107).
The research conclusions are based on the data from the Gary Income Maintenance Experiment, which took place between 1971 and 1975, and which involved 1920 exclusively low-income black children.
Just to show how rarely anyone has read the original article, check out these quotes: “There is… no evidence that changing the immediate circumstances of the family will have any effect on student performance. The work behavior of the mother has no influence on the educational performance of the children. Neither does the absence of a father.” (page 113) and “Of the determinants of teacher expenditures per pupil (ie, teacher experience and degree level and class size), only years of experience are significantly related to student performance. (page 109) (emphasis mine, in both cases).
I suspect the emergence of the actual article will result in Ravitch and others calling bullshit on this cite in the weeks and months to come, and the eduformers will be backing off, for reasons that eduformer Stuart Buck make clear:
“So it’s not the most recent or externally valid finding one could wish for, that’s certainly true.”
Then comes the amusing part of Buck’s post, and the reason for mine:
“But is it so implausible that some teachers could produce 1.5 years of learning while others produce half a year? The real questions would be how many teachers are in each category and how we can identify them accurately, without crediting or blaming them for outside-school factors.”
Hahahaha.
Melinda Gates: Well, we know from good research that the fundamental thing that makes a difference in the classroom is an effective teacher. An effective teacher in front of a student, that student will make three times the gains in a school year that another student will make.
Suppose Gates had said “Well, I believe that effective teachers are fundamental. Is it so implausible that some teachers can produce three times the gains”? Doesn’t have nearly the ring of authority, does it?
So we’ve seen the research. It’s old, it’s demographically limited and the support is pretty weak to boot.
But hey, it’s pretty plausible, right?
Factoids are so much fun. Until, you know, someone actually thinks about them and the romance goes poof.
educationrealist 
June 20th, 2012 at 9:57 pm
My guess would be that most results are distributed along some sort of bell curve, so if you define the Best Teachers in terms of results on the bell curve, you can always find (given a large enough sample) somebody who is three times better than the Worst Teacher.
June 22nd, 2012 at 4:34 am
I would second the fact that performance is going to be distributed on a bell curve and so there will always be teachers who statistically seem to be improving student performance by an amazing amount and others that are doing nothing at all. This is not implausible at all.
The logical flaw is in the assumption that the high performing teacher and the low performing teacher are unrelated phenomenon. A good teacher who inherits students from a bad teacher can improve student performance more than a good teacher who inherits students from another good teacher. Thus, getting rid of the bad teachers, as many reformers would have us do, is going to reduce the value-add of the good teachers. This is important inasmuch as the reformers generally assume that removing bad teachers will allow students to consistently progress at the rate that they currently do only under good teachers. In all likelihood, were we to remove the bad teachers, the good teachers would all of sudden look less good and progress, though likely somewhat accelerated, would not be nearly as strong as hoped.
June 22nd, 2012 at 6:49 am
Thus, getting rid of the bad teachers, as many reformers would have us do, is going to reduce the value-add of the good teachers
We should be so lucky if that happened, since it would prove that there actually were a noticeable amount of bad teachers. In fact, though, I suspect there aren’t that many. But yes, I agree with your reasoning.
As I recall, the NY Times data eval showed that only 3% or so of the teachers were consistently in the top or bottom 5%.
My point was somewhat milder–I just thought it was funny the reformers have been hammering on this factoid declaring that “research shows” it to be true, and then, when the research turns out to be an old single study, at least one of them is, like, well, but it seems extremely plausible, so let’s make it true anyway.
July 16th, 2012 at 6:40 pm
There may be a more recent paper by Hanushek that touches on this matter:
“The Economic Value of Higher Teacher Quality,” by Eric Hanushek. Urban Institute, National Center for Analysis of Longitudinal Data in Education Research, Working Paper 56, December 2010. Published version can be found at Economics of Education Review, volume 30, Issue 3, June 2011, pp. 466-479.
Hanushek is interviewed by Russ Roberts on EconTalk: see the following link.
http://www.econtalk.org/archives/2011/08/hanushek_on_tea.html#highlights
Love your blog. Keep it up.
July 20th, 2012 at 10:03 pm
I’m new to your blog and working my way through your old posts. Love your work. I remember seeing this post quoted somewhere, probably at Steve’s place, and I remember not quite getting your point.
Is the study no good? The conclusions you quote seem counterintuitive, or at least running against the conventional wisdom. But I don’t see anything manifestly wrong with its methodology. Yes, it is demographically limited, but poor and black are the reasons we’re having this discussion. The data may have been from the 1970s, but again I’m not sure why this makes it invalid. And what does “poorly supported” mean in this context?
It’s kind of funny to see Buck walk back Melinda’s claim to mere “plausibility”, but I’m pretty sure I’ve seen other sources writing about teacher quality say something like this. TFA, for instance, and Matthew Yglesias (although he might have relied on TFA data).
I apologize in advance if you have addressed these issues in older posts I haven’t read yet.