Essays on Teaching Excellence

First and Second steps: Recognizing and challenging myths.
The following myths about SETs appear most commonly in my experience.(Each myth is followed, parenthetically, with rebuttals from the research literature.)

1) SETs reflect little more than a teacher's personality and popularity. Some of us employ this belief to help salve the pain of evaluations. Statements often take this form: "If I were an entertainer, my SETs would improve dramatically." (In fact, the gist of research is that measures of personality and popularity correlate at low, usually insignificant, levels with SETs.)

2) SETs mirror course difficulty and expected grades. Here again, we can devalue SETs by assuming that they decrease as we make courses tougher. The common statement: "Highly-rated colleagues pander for good evaluations by giving easy assignments and generous grades." (Research makes a strong case to the contrary. Anticipated grading and SETs tend to be uncorrelated; heavy work loads correlate positively with SETs.)

3) SETs of tough teachers improve when students are resurveyed years later. All of us, as teachers or parents, like to believe that our "charges" will appreciate us more later, once they have seen the wisdom of our discipline. Some professors cite anecdotes to this effect to excuse their currently low SETs. (Research, sadly, shows that SETs remain remarkably stable over periods of many years. In other words, demanding and misunderstood teachers generally do not get higher ratings in retrospect.)

4) Teaching is idiosyncratic and cannot be measured meaningfully. This misbelief, that teaching defies analysis, is used to reject SETs because they supposedly miss the unique qualities of professors' styles. (In fact, research shows that effective teaching consists of rather ordinary and measurable factors like clear communication and rapport. Moreover, as we shall see anon, SETs lend themselves nicely to added measures that tap dimensions not covered in standardized forms.)

5) Students are quick to complain and criticize. If faced with low SETs we may suppose that students expect too much and disapprove too readily. (Studies of SETs, in contrast, suggest that students evaluate us generously; sometimes at rates of 80% for combined categories of good and excellent.)

6) SETs reflect little more than classroom performance. Some professors faced with disappointing SETs dismiss them because they cannot identify what they need to do differently. (When observations extend beyond the classroom, however, the problems may become apparent. A common example: acting in ways before and after class that students see as abrupt and impersonal.)

Third step: Remedying another myth, that SETs must be painful.
As we begin to recognize that SETs may be credible, we may worry even more about the pain of getting poor evaluations from instruments that we now know are valid. One way of involving faculty in SETs that will be carefully considered and acted upon is to make the instrument painless. The first sample SET at the end of this article offers just such a format. By asking student simply to indicate desired directions of change along continua with no good or bad endpoints, faculty can get painless feedback about ways in which they might consider change. At its best, the painless SET becomes the topic of discussion with classes (e.g., "why do you suppose that as many of the indications for change face in one direction as the other on this item?")

In my experience, once previously reluctant faculty try painless SETs, they are far more likely to volunteer for greater investment in conventional SETs.

Items on the painless SET can, of course, be changed to suit the tastes and needs of those who administer it.

Fourth step: Countering a final myth, that SETs must come at the end.
The obvious problem in not giving SETs earlier than at semester's end is that faculty are unlikely to make changes that could help improve ongoing classes. The second sample of an SET format at the end of this article illustrates a simple means of getting early, informal feedback from classes.

Early and informal evaluations like this one offer several advantages: a)They encourage faculty to rely on more than casual comments as the index of how they are doing. Instead, faculty can actively solicit anonymous opinions from all students - even those who ordinarily remain quiet during the semester. b) Early evaluations help get students involved. As the instructions attached to the early SET indicate, students can help collect, analyze, and even discuss the results. c) Discussions of the results of early SETs in class help educate students as evaluators. Faculty discussing early SETs can do more than indicate intended changes in teaching-related behaviors. They can also give students feedback on what kinds of evaluative comments are constructive and which are not. Experience with this strategy indicates that many students become more proficient as evaluators and more interested in the teaching process as a result of paying attention to specific categories of performance. d) Early SETs provide an opportunity to collect something usually left out of evaluations - compliments.

This general plan for getting faculty to abandon the temptation to see SETs as capricious indices of pandering and vengeful students revolves around action. It stimulates us and our colleagues to supplant our usual passiveness with proactiveness. In actual practice, I find that the general sequence of steps outlined here works best to change attitudes and behaviors (not necessarily in that order). In essence, these steps involve educating ourselves about what SETs really mean and how they can help.

One advantage of the sort of approach suggested here, according to my own research, is that it leads to three positive changes: 1) raised SETs, 2) alternative teaching behaviors, and 3) improved classroom comfort for both faculty and students.

1. Administer the ISE at least once before formal evaluations; the earlier the administration of the ISE, the more instructors generally benefit. Try to use by midterm at the latest.
2. Allow 5 minutes at the end of a class to administer the ISE. Simply say that you're interested in learning what you're doing well and what you could do better while there is still time for change.
3. Ask for student volunteers to collect and compile evaluation sheets. In fact, students do see this request as an imposition. In fact, students provide more useful feedback if they know that you will not see their handwriting (thus the reliance on students to collect and summarize the evaluation sheets.)
4. Ask the student volunteers to summarize the results on a copy of the ISE. Numerical ratings can be summarized as a sampling of the most common types (e.g., "the instructor treats students with respect"). Have the summarizers omit uncommon remarks.
5. Xerox copies of the summary sheet and distribute them to all students at the beginning of the next class. Plan to spend 5 minutes reflecting on the results and probing students about what some evaluative comments mean (and how you can address them in terms of changes in style, content, etc.)
6. Use the occasion to educate students about ways to provide useful feedback to you; about your assessment of the class on dimensions like involvement, preparedness, etc.; and about your rationales for teaching the way you do (i.e., you may want to defend some of your practices).
7. Choose a sample of items from the formal evaluation to be used later in the semester (as in the example ISE provided here). These can give you a preliminary sense of how students will rate you (and a chance, in your discussions with them, to determine the basis for their numerical ratings on formal items.)

1. What the instructor does well (please be specific):
2. What the instructor could do better (please be specific):
3. Please rate the instructor on the following scale, 1-7 (7 = maximum/excellent)
        a. Objectives and procedures were made clear. ___
        b. Instructor is well-prepared and organized. ___
        c. The course stimulates my thinking. ___
        d. Presentations are clear. ___</ol?