A publication of The Professional & Organizational Development Network in Higher Education. The posting and use of this publication on your institution's WWW server is covered by an End User License Agreement (EULA). By the terms of this agreement, the essay must be located on a secure server that limits access to your institution's students, faculty, or staff. Use beyond your institutional boundaries, including the linking of this page to any site other than this one, requires the express permission of the POD Network. To view the terms of the End User License Agreement,click here.
First and Second steps: Recognizing and challenging myths.
The following myths about SETs appear most commonly in my experience.(Each myth is followed, parenthetically, with rebuttals from the research literature.)
1) SETs reflect little more than a teacher's personality and popularity. Some of us employ this belief to help salve the pain of evaluations. Statements often take this form: "If I were an entertainer, my SETs would improve dramatically." (In fact, the gist of research is that measures of personality and popularity correlate at low, usually insignificant, levels with SETs.)
2) SETs mirror course difficulty and expected grades. Here again, we can devalue SETs by assuming that they decrease as we make courses tougher. The common statement: "Highly-rated colleagues pander for good evaluations by giving easy assignments and generous grades." (Research makes a strong case to the contrary. Anticipated grading and SETs tend to be uncorrelated; heavy work loads correlate positively with SETs.)
3) SETs of tough teachers improve when students are resurveyed years later. All of us, as teachers or parents, like to believe that our "charges" will appreciate us more later, once they have seen the wisdom of our discipline. Some professors cite anecdotes to this effect to excuse their currently low SETs. (Research, sadly, shows that SETs remain remarkably stable over periods of many years. In other words, demanding and misunderstood teachers generally do not get higher ratings in retrospect.)
4) Teaching is idiosyncratic and cannot be measured meaningfully. This misbelief, that teaching defies analysis, is used to reject SETs because they supposedly miss the unique qualities of professors' styles. (In fact, research shows that effective teaching consists of rather ordinary and measurable factors like clear communication and rapport. Moreover, as we shall see anon, SETs lend themselves nicely to added measures that tap dimensions not covered in standardized forms.)
5) Students are quick to complain and criticize. If faced with low SETs we may suppose that students expect too much and disapprove too readily. (Studies of SETs, in contrast, suggest that students evaluate us generously; sometimes at rates of 80% for combined categories of good and excellent.)
6) SETs reflect little more than classroom performance. Some professors faced with disappointing SETs dismiss them because they cannot identify what they need to do differently. (When observations extend beyond the classroom, however, the problems may become apparent. A common example: acting in ways before and after class that students see as abrupt and impersonal.)
Third step: Remedying another myth, that SETs must be painful.
As we begin to recognize that SETs may be credible, we may worry even more about the pain of getting poor evaluations from instruments that we now know are valid. One way of involving faculty in SETs that will be carefully considered and acted upon is to make the instrument painless. The first sample SET at the end of this article offers just such a format. By asking student simply to indicate desired directions of change along continua with no good or bad endpoints, faculty can get painless feedback about ways in which they might consider change. At its best, the painless SET becomes the topic of discussion with classes (e.g., "why do you suppose that as many of the indications for change face in one direction as the other on this item?")
In my experience, once previously reluctant faculty try painless SETs, they are far more likely to volunteer for greater investment in conventional SETs.
Items on the painless SET can, of course, be changed to suit the tastes and needs of those who administer it.
Fourth step: Countering a final myth, that SETs must come at the end.
The obvious problem in not giving SETs earlier than at semester's end is that faculty are unlikely to make changes that could help improve ongoing classes. The second sample of an SET format at the end of this article illustrates a simple means of getting early, informal feedback from classes.
Early and informal evaluations like this one offer several advantages: a)They encourage faculty to rely on more than casual comments as the index of how they are doing. Instead, faculty can actively solicit anonymous opinions from all students - even those who ordinarily remain quiet during the semester. b) Early evaluations help get students involved. As the instructions attached to the early SET indicate, students can help collect, analyze, and even discuss the results. c) Discussions of the results of early SETs in class help educate students as evaluators. Faculty discussing early SETs can do more than indicate intended changes in teaching-related behaviors. They can also give students feedback on what kinds of evaluative comments are constructive and which are not. Experience with this strategy indicates that many students become more proficient as evaluators and more interested in the teaching process as a result of paying attention to specific categories of performance. d) Early SETs provide an opportunity to collect something usually left out of evaluations - compliments.
This general plan for getting faculty to abandon the temptation to see SETs as capricious indices of pandering and vengeful students revolves around action. It stimulates us and our colleagues to supplant our usual passiveness with proactiveness. In actual practice, I find that the general sequence of steps outlined here works best to change attitudes and behaviors (not necessarily in that order). In essence, these steps involve educating ourselves about what SETs really mean and how they can help.
One advantage of the sort of approach suggested here, according to my own research, is that it leads to three positive changes: 1) raised SETs, 2) alternative teaching behaviors, and 3) improved classroom comfort for both faculty and students.
Ellis, R.S. (1985). "Ratings of teachers by their students should be used wisely or not at all." Chronicle of Higher Education, Nov. 20, p. 88.
Feldman, K.A. (1986). "The perceived instructional effectiveness of college teachers as a function of their personality and attitudinal characteristics." Research in Higher Education., 24, 139-213.
Feldman, K.A. (1989). "Instructional effectiveness of college teachers as judged by ..." Research in Higher Education, 30, 137-194.
Kulik, J.A., and McKeachie, W.J. (1975). "The evaluation of teachers in higher education." In F.N. Kerlinger (Ed.), Review of Research in Higher Education, vol. 3, Itasca, IL: Peacock.
Menges, R.J. and Mathis, B.C. (1988). Key Resources on Teaching, Learning, Curriculum and Faculty Development. San Francisco: Jossey-Bass.
Weimer, M. (1990). Improving College Teaching. San Francisco: Jossey-Bass.
This publication is part of an 8-part series of essays originally published by The Professional & Organizational Development Network in Higher Education. For more information about the POD Network, browse tohttp://www.podnetwork.org.