The Student Evaluation of Teaching (SET) process is designed to complement self-assessment and peer review (both within and external to the department) of teaching at OSU. SET questions consider overall teaching quality and basic teaching functions or behaviors. The goal is for instructors and supervisors to identify teaching excellence as well as areas that may need attention.
"Anonymous evaluations by all students in the class are required each term for each class the faculty member is teaching. A copy of tabulated results must be provided to the faculty member; a duplicate copy shall be placed in the faculty member's personnel records file" (http://oregonstate.edu/facultystaff/handbook/facrec/evals.htm). Extension faculty are expected to choose three events per year to evaluate teaching. Faculty teaching Extended Campus courses will use an electronic version of the SET form for all classes.
The primary purpose of the revised SET form is to provide student feedback that confirms quality teaching or identifies topics for possible improvement. The first two questions are worded broadly to compare faculty across an entire campus, and were validated by the Office of Educational Assessment at the University of Washington (http://www.washington.edu/oea/describe.htm). Questions 3-12 were selected from validated instruments used at other universities to represent standard teaching functions and behaviors. All questions on the revised form were validated (AOT report, 2002) at OSU for use in most teaching situations including Extension Events and Extended Campus courses.
Faculty may add questions to the back of the form to assess personal teaching practices, measures of learning, facilities, or accreditation requirements. When adding questions to the back, survey research literature recommends that questions be worded carefully to assess only one item or concept at a time. Care must be exercised to avoid using synonyms such as "examples and illustrations" since they could mean different things to respondents.
Alignment during photocopying is critical for accurate and complete scanning. Questions may be typed onto a master or a template (available in Microsoft® Word) and then photocopied onto the scan forms placed in the copy tray. Rather than photocopy questions onto the back, some faculty use overhead equipment to project the questions onto a screen while students respond on the back of the scan sheet.
To assure confidentiality, responses to narrative questions will be separated from demographic data. Faculty may provide questions on a separate page or ask respondents to answer questions on a separate paper.
Examples of narrative questions are listed in the appendix.
Scanning and automatic generation of summary reports occurs at The Milne Computer Center. Because responses represent ordered qualitative data, medians, frequencies, and percentages will be calculated based on a 1-6 scale rather than ordinal numeric scales. Cross-tab data explained below will be a new feature for faculty and supervisors to consider. Responses to narrative questions will be separated and held within departments for study by instructors after grades are submitted.
The report consists of two pages; the first summarizes overall results of two questions and cross-tab data based on demographic summaries. These two questions are intended to compare faculty across an entire campus, referred to as norm referencing. It compares an instructor's or one faculty member's performance against the general level of performance by others in the department, college, or university. Thus, administrative reporting of one individual's teaching performance using medians, frequencies, and percentages for questions 1 and 2 are valid for promotion and tenure (P&T), awards, or merit comparisons. As with any survey or data set, clarity about what is being assessed is essential. For example, instructors involved with team teaching or other situations may require special explanations to avoid ambiguity while reporting results.
Cross-tab data summarized on the first page are based on correlations in SET literature or are intended to provide additional information for an instructor to consider for improving teaching within the discipline. For example, SET literature often accepts the correlation that compares questions 1 and 2 with "enrollment reason" (items 32 and 34 on the OSU SET form). Cross-tab data are included for student class status (level) and gender, but correlations are often very weak or lacking (except responses from graduate students) and therefore intended to clarify results with respect to demographics or to identify unanticipated concerns. Cross-tab data for class size and student status (level) are available to compare one class with others offered in the department or college.
The second page summarizes questions designed to improve teaching quality within the discipline. These include questions 3-12 and the questions designed by the instructor and represent criterion referencing in the literature. These data are meaningful only to the pertinent individuals and must be interpreted within the discipline. Although summary reports will be sent to both the instructor and the supervisor, the purpose is for teachers to consider strong or weak responses as indicators of quality teaching or as prompts for teaching improvement respectfully. Utilizing these indicators and changes over time may help confirm improvement and quality teaching for P&T or award documents by faculty or peers within the discipline. However, reporting median scores for criterion referenced data without disciplinary interpretation similar to norm referencing is invalid.
The science of teaching evaluation clearly reminds us that teaching is a tremendously complex activity that requires a similarly robust assessment process. SET represents the experiences or perceptions of students only. It must be complemented by self-evaluation, internal and external peer assessment, and the sciences of teaching, learning, and evaluation.
As you review SET data, note the practices and skills that should be continued or enhanced along with others that need improvement. Discuss results with peers to consider reasons and alternative methods. Develop ways to test these ideas the next time you teach this or other courses where you could try the approach, technique, or method. Attend seminars, search the literature, or ask a peer from your department or profession how they might improve one or more aspects of the course. Consider how you will assess this aspect of your teaching, how it might affect learning by students, and how it contributes to the curriculum within the discipline. How will these innovations in your teaching be communicated to peers and how will they interpret the results? Will it be necessary to consider postgraduation or post-school year surveys or other assessment techniques within the discipline? How do you feel about your teaching? Perhaps self-assessment is the most important aspect of teaching and teaching improvement at OSU.
Accreditation requirements for universities such as OSU are intended to improve and validate teaching performance by faculty and instructors. The SET process is intended to improve teaching within the discipline. Validating norm referenced teaching competencies within colleges requires aggregate data from questions 1 and 2 reported to deans and the Assistant Provost for Academic Programs. All other SET data are designed to assess teaching performance by individual instructors with reporting being restricted for this purpose only.
Research shows that it does make a difference what is said when asking respondents to complete the SET questionnaires. To improve comparability for individuals between terms and the possibility of comparisons within disciplines, a standardized set of instructions should be written for all instructors and courses. Since the SET process is intended to assess teaching skills and functions, it is imperative that the importance of this process be emphasized by instructions to the students in each class.
Drafted by the Advancement of Teaching Committee of the Faculty Senate, 2003.
Revised: John Morelock, 16 April 2003
Assessing Instructor/Teaching
The Office of Educational Assessment at the University of Washington (UW) http://www.washington.edu/oea/iasforms.htm has multiple forms based on teaching approaches, instructor's skills and organization, and educational outcomes (Form X).
Kansas State University IDEA Center also displays standard assessment questions along with a dozen learning objectives with relationships to teaching methods http://www.idea.ksu.edu/products/Sturatings.html
Both Centers permit OSU faculty to select a modest number of questions from their surveys to copy on the back of the OSU form. These questions have been tested for reliability and validity.
Assessing Student/Learner Responsibilities (examples):
Assessing Learning Resources/Environment
Sample Open-ended Written Questions:
Open-ended Questions for Teaching Assistants:
