The current Slate Star Codex, my favorite blog, has a link to an account of an interesting piece of research on student evaluations of teachers. It judged the quality of teachers by how well their students did in later courses, compared the result to student evaluations of teaching quality, and found that the two anti-correlated. On average, good teachers get bad ratings, bad teachers get good ratings. For details, follow the link.
It's a single study, I have not read the original paper, and the result might be wrong. But it is interesting to think about reasons why it might well be right.
The most obvious one is that many students don't like to work hard. A professor who does not assign much homework or reading and grades easily might get better ratings, from many although not all students, than one with the opposite pattern. My daughter, as a student at Oberlin, was struck by the fact that most of the other students in a class were happy when, for some reason, it didn't meet. The same pattern—study seen as a cost, not a benefit—might well apply here.
There is a second and less obvious possible reason. Correct ideas are frequently hard. Easy ideas are frequently wrong. My standard example is from popular discussions of foreign trade issues. Most of them take for granted a view of the economics of trade, the view implicit in terms such as "unfavorable balance of trade," that economists refer to as the theory of absolute advantage. That particular view of the subject has been obsolete for about two hundred years. But while the theory of absolute advantage does not make sense if you think about it carefully, it is considerably easier to understand than the theory of comparative advantage, which does. That is why the former was worked out first and why it has had such a successful postmortem career.
A professor who insists on telling the truth, on explaining hard ideas correctly, may well come across as a worse teacher than one who fudges, offers a simplified and less correct version. Half the students of the former end up believing that they do not entirely understand the subject being taught—and they are right. Almost all the students of the latter end up sure they understand it—and wrong.
10 comments:
I've a single data point, but it is contrary to one of your theories.
I'm a coder, not a professional educator, but I did teach a programming class at the graduate level for several semesters (one session a semester).
The number of students in my sessions increased over time. Perhaps this was not due to my reputation, but some statements by students indicated that it probably was.
Instructor evaluations were public and I know that I was getting better numbers than other instructors in my department.
I also know that my class was much more demanding than other classes. (I felt that the real value of my class was that a professional software engineer would carefully read and comment on your code, so there was a lot of writing code.)
How's this for an alternative theory? Highly rated instructors "spoil" students for subsequent mediocre instructors with the result that students perform less well for them.
There's a huge difference between students taking a course because they want to master a skill set that can be objectively measured, and students taking a course not because they're intrinsically interested in the material, but only because they're interested in getting a good grade and getting a credential.
Professor Friedman,
I actually agree with your interpretation of the evidence. In my experience,it has often been the case that other students did not appreciate the most demanding Professors. But I also though of a different, although implausible, explanation. The students who do well in future courses might give low evaluations to their teachers just because they expected more from them. I know it is a little implausible, especially given that we are talking about averages (and the average student is not likely to be very demanding), this also fits my experience.
I agree with everything you wrote. The only problem I see is in evaluating teachers by how well their students do in later courses. Except for maybe a few very focused areas of study, I find it unconvincing that this would be a reliable way to gauge the quality of teachers.
It becomes especially nebulous outside engineering and hard science, with extreme examples to be found in liberal arts. Can you really judge the quality of a film theory 101 professor based on how his students do in 300- and 400-level film theory classes?
I think there's also the fact that some of the skills of a proficient teacher aren't obvious, or even visible to students. For example, if a professor designs a high quality course and manages her TAs well, the student might get a lot of value. The student however might only see the professor bumbling through lectures rather than the work that is done "behind the scenes".
I would theorise that most students don't see professors as teachers, or course facilitators. Rather I think they see them more as presenters of information, and so they judge them on how they present the information. Other work they do will likely not be seen as adding much value for a student.
(NB: I am making most of my judgement from my experience, which mostly consisted of large course sizes in Engineering)
When I was taking classes, I judged my teachers on a number of metrics, including how well they presented the information, what sources/textbooks/homework they chose (most of my teachers were making those choices themselves), whether they were helpful if consulted outside of class, etc. Not managing TAs, but most of my classes didn't have TAs. So... it could be that the average student takes fewer things into consideration than I did, but our teacher evaluations had lots of different categories of "was your teacher good at X" to mark, so... I don't think, at least at Chicago, it was that simple.
Note, this opinion was formed from small, mostly liberal arts classes at the University of Chicago, probably not an average sample.
Additional note: At said university, I relied heavily on student evaluations to select teachers, and had very good results. I was relying on specific comments as well as numbers, and they did say skilled students tended to be an exception to their rule... but I'm still a little surprised by a result that so thoroughly contradicts my own experience.
I have added the Slate Star Codex to my blog roll, thanks.
The evaluations I was most familiar with, back in my undergraduate days, had two parts: One rated how "good" the instructor was, and the other rated how "hard" the instructor was. I didn't run any correlation tests on the results, but there was a noticable amount of scatter: The "good" instructors included both hard and easy ones, and likewise for the "bad" instructors.
If the student evaluations used in the study allowed only a single ranking-of-merit, then I'd expect easy-but-poor instructors to get higher rankings and hard-but-good instructors to get lower ones, muddling or even reversing any good-instructors-produce-good-future-results effect.
My experience is that the classes that were both hard and good in terms of added value were quite popular. I remember one freshman year course - linear algebra. The exercise class to that subject had many lecturers and in the first semester (this was a two-semester lecture, Linear albegra I and II) we - as complete freshmen - were assigned to them more or less at random. In the second, we would choose and a lot of people (me included) chose the teacher who had the reputation of being the most demanding (we were given quite a lot of homework every week, whereas some other lecturers did not demand any). It was not just more difficult, it was also presented in a better way and that guy ended up teaching two groups instead of one like most other exercise class lecturers. He was also given good ratings (at least good comments, I never felt the number ratings tell much and never really read them or gave them). He was good in terms of results of students in the following classes as well - people who had him in the first semester told us others that "if you take the exercise class with him, it is hard work, but you then have to study half as much for the exam, because you learn so much during the semester". This for me was a great incentive to switch the lecturers as I tend to be very lazy and this helped me (because the teacher demanded it) to actually study during the semester and not just passively attend the classes and try to catch it up 2 weeks before the exam.
I have also observed that some students are happy that a class is off, which is weird not only because they should want to learn (albeit one can argue that if you have a mandatory class you would not take otherwise it might not be the case) but also at least at our faculty (or at least in the mathematics program) attendance at lectures was not required at all and attendance at exercise classes was usually something like "at least 50%" which is not too hard to do. So mostly you can just not come if you don't want to anyway. And there were lectures where the teacher would just basically copy the lecture notes on the blackboard which were really not worth going there. Then again, maybe some people were happy a class was off, because it meant possibly a bit less stuff to learn for the exam and so easier to pass (or pass with a better grade, but these people usually did not care about grades much).
I have to confess I had a lot of trouble coming to one particular exercise class though, but that was because it started at 7:30 or something (during winter, so it was still dark outside) :) No other class thankfully ever started before 9, otherwise I perhaps would not have graduated :)
> The most obvious one is that many students don't like to work hard.
Of course, this means we're defining "good" teachers to mean something unrelated to how satisfied students are with the teacher, which feels a little like defining "good" restaurants by their customers' obesity rating rather than their satisfaction. Perhaps customers of some restaurants don't mind that the food is unhealthy.
Post a Comment