Faculty Senate Back to Main Page
 
Advancement of Teaching Committee
 
User's Guide For the Assessment of Teaching Scan Forms

Introduction

As part of the effort to encourage participation in, and increase the efficiency of, the Oregon State University Student Evaluation of Teaching (SET) process, the OSU Information Systems Administrative Computing (ISAC) has created this document.

Some of the benefits of the SET process include: valued feedback from OSU students and Extension attendees, generating a source of additional points of measurement on which an instructor could base improvement of teaching methods, creating additional sources of information about faculty members for the departments and colleges of the university, and providing timely credible information for analysis of performance.

It is to the credible part of the SET process that this document speaks. Because the SET form is one of the last pieces of paper a student sees for each class, it is imperative that the students be made aware that OSU is concerned with their input. The SET process can only remain a critical and valued part of the accreditation program, however, if a high degree of accuracy is maintained.

Form Scanning, Data Assimilation, and Report Generation

 The data used to initiate the SET process for each class could be thought of as consisting of two parts: the first part (the "magenta form" or Instructor Header Sheet) identifies the class and instructor, and the second part consists of one, or more, completed student response forms (the "green form" or Student Assessment of Teaching response form). In Extension, attendees complete a purple Citizen Assessment of Teaching form submitted through their academic department. The appropriately matched sets of completed forms are transmitted to OSU Information Services Operations (ISOps) for processing.

Using a pair of NCS OpScan7** form scanners, the processing of the forms creates a machine-readable, fixed-format plain text file. The text file is then processed by an analysis program that transforms the data into useful information (percentages, frequencies, and, as appropriate, medians), which is presented in either a class-by-class report or a departmental or college level summary.

While the forms may be processed on a class-by-class basis, it is preferred that departments submit the forms in as few submissions as practical to aid in efficient use of operations resources. As forms are processed, the data are either stored in a newly created file (for initial submissions) or appended to an existing data file (corrected or revised forms or late submissions). ISOps will maintain the data files for the various departments. The data will be available for historical reference. The data will not be maintained on any database.

How to Fill In the Scan Form

  1. Use only a standard Number 2 pencil - do not use ink pens, ballpoint pens, felt tip pens, highlighters, etc. These do not register with the mechanical reader at OSU, resulting in uncounted responses.**
  2. Fill in each response bubble completely - a check mark, a line or two lines through the response bubble, a few squiggles, or other partial fillers will not register with the mechanical reader and the response will not be counted.**
  3. Erase thoroughly - a partially erased entry might be readable by the mechanical reader causing a multiple-entry rejection for that response and the response will not be counted.**
  4. Fill in one, and only one, response bubble per topic - multiple responses void the count for that topic.
  • **Note: Instructions 1, 2, and 3 are critical because of the age and optical sensitivity of the OSU scanners-the scanners will not read anything but standard Number 2 pencils.
Common errors when filling in the scan forms
(and their solution)

  1. Fields not completely filled out-e.g., filling in 203 when 203a is correct, using ENG when ENGR is correct. Use only valid, complete, and correct entries.
  2. More than one bubble (the circular, oval, or elliptical response element) filled in for the same column-these get read and interpreted as more than one response, which causes the mechanical reader to invalidate the response because of the "no multiple responses" rule.
  3. Erasures are not complete-reads an incomplete erasure and a valid response, causing the mechanical reader to invalidate the response because of the "no multiple responses" rule.
  4. Answers not completely filled in-a single or a couple of lines will not do; the bubble must be filled in completely to be read.
  5. Extra marks made on the form that can be read by the scanner-any response column with more than one machine-readable response is automatically read as a no-response column.
  6. Using a felt tip pin, a ball-point pin, a highlighter, a colored pencil, or almost anything other than the generic Number 2 pencil-the mechanical reader might not see the mark.
  7. Invalid, incorrect, or incomplete entries used-entries not recognized by the analysis program will be rejected and scan forms for that class will need to be resubmitted (This is new. The valid list of abbreviations will be distributed to the departments prior to the end of the 2003 Spring Quarter.).
  8. Some letter fields were entered right justified (bubbles filled in starting from the right, leaving blank bubbles on the left)-fill in all nonnumeric response columns starting from the left column.

The Effect of Incorrectly Filled In Scan Forms

  1. The OSU Student Assessment of Teaching Survey Instructor Header Sheet (IHS), "the magenta form" - all fields are valuable and critical parts of the AOT process.
    1. Instructor's Name - Required
      1. No analysis performed if the name is missing; returned to originator (with green forms) for correction and resubmission.
      2. Incorrect value (misspelled name is undetectable)-analysis will still be performed; department or college will have to determine correct instructor.
    2. Sex - Response is not required
    3. Dept Code - Required
      1. Missing value - No analysis performed if value is missing; returned to originator (with green forms) for correction and resubmission.
      2. Incorrect value - No analysis performed; returned to originator (with green forms) for correction and resubmission.
    4. Course Number - Required
      1. Missing value - No analysis performed if value is missing; returned to originator (with green forms) for correction and resubmission.
      2. Incorrect value - Analysis performed, possibility of null entry on report - value is dynamic on a term-by-term basis; too costly in terms of resources to provide list each term.
    5. Section Number - Required
      1. Missing value - No analysis performed if value is missing; returned to originator (with green forms) for correction and resubmission.
      2. Incorrect value - Analysis performed, possibility of null entry on report - value is dynamic on a term-by-term basis; too costly in terms of resources to provide list each term.
    6. Course Type - Required
      1. Missing value - No analysis performed if value is missing; returned to originator (with green forms) for correction and resubmission.
      2. Incorrect value - Analysis performed, possibility of null entry on report - value is dynamic on a term-by-term basis; too costly in terms of resources to provide list each term.

  2. The OSU Student Assessment of Teaching, "the green form" - there are no required fields on this form. Even though there are no required fields, all responses are important for the analysis of the assessment. Please ask the students to provide their sincere response to each topic.

Processing the Completed Scan Forms

Review for Correctness and Completeness - The Instructor Header Sheet (IHS, also called the magenta form) needs careful review. Errors or omissions on this form will result in either no analysis report being created and the IHS and its corresponding green forms returned to the originator (department or college) for corrective action and resubmission or (possibly worse) an analysis report will be created under an erroneous instructor name, section number, or course number.

Submit to Information Services Operations (ISOps) - Once the IHS scan forms have been reviewed, it and the associated student response forms (the green forms) are delivered to ISOps (Room 206, Milne Computing Center). An AOT work order (the yellow form) is filled out to initiate the form scan operations, data analysis, and report generation phase.

Reports and Returned Materials - Upon completion of the ISOps operations, the completed AOT reports and exception reports are sent to the originating party.

  • IHS forms with a missing instructor's name, missing or incorrect department code, missing course number, missing section value, or missing course type create exception reports.

  • The originator will need to decide if the classes for which the IHS caused an exception report to be created are to be resubmitted. All data entries on the IHS are to be corrected prior to resubmission.

Resubmitted Reports - Classes for which resubmissions are made will automatically create updated summary reports along with the individual class reports.

Final Reports - There will be an option available whereby the report requested may be labeled "Final Report," and it will include the analysis of all data currently on file for that class, department, or college.

Data Storage - The files created by the scan form processing will be kept in electronic format and maintained by IS Administrative Computing. The data as kept is not in a format that would read sensibly to a person. It is in a machine-readable format and available for analysis processing and report generation. It becomes useful information after analysis processing. The files will be kept so as to provide a historical source for departments and colleges.
 
| Agendas | Bylaws | Committees/Councils | Faculty Forum Papers | Handbook | Meetings/Locations | Membership | Minutes |