skip page navigationOregon State University
OSU Home.|Calendar.|Find Someone.|Maps.|Site Index.

Faculty Senate

Faculty Senate » Student Evaluation of Teaching (SET), Guidelines for Classroom Use

Advancement of Teaching Committee

Student Evaluation of Teaching (SET), Guidelines for Classroom Use
Goal:

The Student Evaluation of Teaching (SET) process is designed to complement self-assessment and peer review (both within and external to the department) of teaching at OSU. SET questions consider overall teaching quality and basic teaching functions or behaviors. The goal is for instructors and supervisors to identify teaching excellence as well as areas that may need attention.

OSU SET policy:

"Anonymous evaluations by all students in the class are required each term for each class the faculty member is teaching. A copy of tabulated results must be provided to the faculty member; a duplicate copy shall be placed in the faculty member's personnel records file" (http://oregonstate.edu/facultystaff/handbook/facrec/evals.htm). Extension faculty are expected to choose three events per year to evaluate teaching. Faculty teaching Extended Campus courses will use an electronic version of the SET form for all classes.

The new SET form:

The primary purpose of the revised SET form is to provide student feedback that confirms quality teaching or identifies topics for possible improvement. The first two questions are worded broadly to compare faculty across an entire campus, and were validated by the Office of Educational Assessment at the University of Washington (http://www.washington.edu/oea/describe.htm). Questions 3-12 were selected from validated instruments used at other universities to represent standard teaching functions and behaviors. All questions on the revised form were validated (AOT report, 2002) at OSU for use in most teaching situations including Extension Events and Extended Campus courses.

Faculty may add questions to the back of the form to assess personal teaching practices, measures of learning, facilities, or accreditation requirements. When adding questions to the back, survey research literature recommends that questions be worded carefully to assess only one item or concept at a time. Care must be exercised to avoid using synonyms such as "examples and illustrations" since they could mean different things to respondents.

Photocopying:

Alignment during photocopying is critical for accurate and complete scanning. Questions may be typed onto a master or a template (available in Microsoft® Word) and then photocopied onto the scan forms placed in the copy tray. Rather than photocopy questions onto the back, some faculty use overhead equipment to project the questions onto a screen while students respond on the back of the scan sheet.

Confidentiality:

To assure confidentiality, responses to narrative questions will be separated from demographic data. Faculty may provide questions on a separate page or ask respondents to answer questions on a separate paper.

Narrative Questions:

Examples of narrative questions are listed in the appendix.

Interpretation of SET data

Scanning and automatic generation of summary reports occurs at The Milne Computer Center. Because responses represent ordered qualitative data, medians, frequencies, and percentages will be calculated based on a 1-6 scale rather than ordinal numeric scales. Cross-tab data explained below will be a new feature for faculty and supervisors to consider. Responses to narrative questions will be separated and held within departments for study by instructors after grades are submitted.

The report consists of two pages; the first summarizes overall results of two questions and cross-tab data based on demographic summaries. These two questions are intended to compare faculty across an entire campus, referred to as norm referencing. It compares an instructor's or one faculty member's performance against the general level of performance by others in the department, college, or university. Thus, administrative reporting of one individual's teaching performance using medians, frequencies, and percentages for questions 1 and 2 are valid for promotion and tenure (P&T), awards, or merit comparisons. As with any survey or data set, clarity about what is being assessed is essential. For example, instructors involved with team teaching or other situations may require special explanations to avoid ambiguity while reporting results.

Cross-tab data summarized on the first page are based on correlations in SET literature or are intended to provide additional information for an instructor to consider for improving teaching within the discipline. For example, SET literature often accepts the correlation that compares questions 1 and 2 with "enrollment reason" (items 32 and 34 on the OSU SET form). Cross-tab data are included for student class status (level) and gender, but correlations are often very weak or lacking (except responses from graduate students) and therefore intended to clarify results with respect to demographics or to identify unanticipated concerns. Cross-tab data for class size and student status (level) are available to compare one class with others offered in the department or college.

The second page summarizes questions designed to improve teaching quality within the discipline. These include questions 3-12 and the questions designed by the instructor and represent criterion referencing in the literature. These data are meaningful only to the pertinent individuals and must be interpreted within the discipline. Although summary reports will be sent to both the instructor and the supervisor, the purpose is for teachers to consider strong or weak responses as indicators of quality teaching or as prompts for teaching improvement respectfully. Utilizing these indicators and changes over time may help confirm improvement and quality teaching for P&T or award documents by faculty or peers within the discipline. However, reporting median scores for criterion referenced data without disciplinary interpretation similar to norm referencing is invalid.

Using SET data to complement teaching improvement:

The science of teaching evaluation clearly reminds us that teaching is a tremendously complex activity that requires a similarly robust assessment process. SET represents the experiences or perceptions of students only. It must be complemented by self-evaluation, internal and external peer assessment, and the sciences of teaching, learning, and evaluation.

As you review SET data, note the practices and skills that should be continued or enhanced along with others that need improvement. Discuss results with peers to consider reasons and alternative methods. Develop ways to test these ideas the next time you teach this or other courses where you could try the approach, technique, or method. Attend seminars, search the literature, or ask a peer from your department or profession how they might improve one or more aspects of the course. Consider how you will assess this aspect of your teaching, how it might affect learning by students, and how it contributes to the curriculum within the discipline. How will these innovations in your teaching be communicated to peers and how will they interpret the results? Will it be necessary to consider postgraduation or post-school year surveys or other assessment techniques within the discipline? How do you feel about your teaching? Perhaps self-assessment is the most important aspect of teaching and teaching improvement at OSU.

SET Data and Accreditation:

Accreditation requirements for universities such as OSU are intended to improve and validate teaching performance by faculty and instructors. The SET process is intended to improve teaching within the discipline. Validating norm referenced teaching competencies within colleges requires aggregate data from questions 1 and 2 reported to deans and the Assistant Provost for Academic Programs. All other SET data are designed to assess teaching performance by individual instructors with reporting being restricted for this purpose only.

Instructions for Administering the SET
Instructions to students when distributing SET forms:

Research shows that it does make a difference what is said when asking respondents to complete the SET questionnaires. To improve comparability for individuals between terms and the possibility of comparisons within disciplines, a standardized set of instructions should be written for all instructors and courses. Since the SET process is intended to assess teaching skills and functions, it is imperative that the importance of this process be emphasized by instructions to the students in each class.

  1. Teaching at OSU is an essential part of each instructor's responsibilities. Your responses to this questionnaire will help instructors identify quality teaching skills and methods or discover behaviors that need improvement.
  2. Please take the time to answer each question individually and add your comments on a separate sheet of paper.
  3. Instructors will consider your comments carefully. Also, supervisors for each faculty or instructor will use this information to encourage teaching excellence.
  4. Please hand your completed evaluations to _____________. This person has been instructed to seal the envelope and hand it to the departmental secretary to ensure privacy until grades are completed.

Drafted by the Advancement of Teaching Committee of the Faculty Senate, 2003.

  • Ray D. William, Chair
  • Paula McMillen,
  • Ken Krane
  • Margie Haak
  • Molly Engle

Revised: John Morelock, 16 April 2003

Appendix

Sample Questions To Be Photocopied To The Back Of The Scan Form

Assessing Instructor/Teaching

The Office of Educational Assessment at the University of Washington (UW) http://www.washington.edu/oea/iasforms.htm has multiple forms based on teaching approaches, instructor's skills and organization, and educational outcomes (Form X).

Kansas State University IDEA Center also displays standard assessment questions along with a dozen learning objectives with relationships to teaching methods http://www.idea.ksu.edu/products/Sturatings.html

Both Centers permit OSU faculty to select a modest number of questions from their surveys to copy on the back of the OSU form. These questions have been tested for reliability and validity.

Assessing Student/Learner Responsibilities (examples):

  • Your performance in this course was?
  • Your assessment of completing readings and homework was:
    • Inspired critical thinking?
    • Challenging?
    • Too much/not enough?
  • Your assessment of learning new information was:
  • Your attendance in class was:
  • Your prior interest in this course was:

Assessing Learning Resources/Environment

  • Quality of learning resources (books, media, visual aids, etc.)
  • Quality of learning environment (seating, ability to see instructor, lighting, ventilation, noise, etc.) or (specialized equipment such as drawing tables, etc.)

Sample Open-ended Written Questions:

  • The comment sheet from the UW might have questions relevant to your teaching (http://www.washington.edu/oea/iascmmt.htm).
  • Is there anything else you would like to tell us about your participation in this class?
  • Was this class intellectually stimulating?
    • Did it stretch your thinking?
      • Yes
      • No
      • Why or why not?
  • What aspects of this class contributed most to your learning?
  • What aspects of this class, if any, detracted from your learning?
  • What suggestions do you have for improving the class?
  • What aspects of presentation helped you most?
  • What helped you least?
  • What would improve the presentation?

Open-ended Questions for Teaching Assistants:

  • What qualities of your TA do you regard as good or outstanding? Please be specific.
  • Are there areas in which you feel your TA needs improvement? If yes, please be specific.
  • Do you have any other comments you feel would be helpful about this TA's performance?