OREGON STATE UNIVERSITY

APAA - Assessment

Annual Program Assessment Report FAQs




What do you mean by "full-cycle assessment"?

Full-cycle assessment means the complete assessment of a single learning outcome. The process begins by identifying a student learning outcome, aligning that outcome with a meaningful assessment measure, collecting and analyzing data from that assessment, making meaning from the results, and using the results to make informed and specific programmatic decisions or changes. Documenting full-cycle assessment means reporting on the components of the assessment from outcome to implemented action.

Figure: Full Cycle Assessment Phases

 

*Please note* Over the 5 –year reporting period you must:

a) submit an example of fully cycle direct assessment for each student learning outcome during that time frame

b) you must submit at least one example of full cycle assessment each year, even if your program has fewer than 5 student learning outcomes.



What do you mean by "direct" assessment?

Direct assessment means that an outcome is assessed based on specific student work or performance. Examples of direct assessment measures include:

  •  Course and homework assignments
  •  Exams and quizzes
  •  Standardized tests
  •  Term papers and reports
  •  Observations of field work, internship, service learning, or lab experiences
  •  Research projects
  •  Class discussion participation
  •  Case study analysis
  •  Artistic performances and products
  •  Grades that are based on explicit criteria related to clear student learning outcomes
  •  Capstone projects
  •  Licensure, certification or subject area tests
  •  Rubrics used to assess competencies or knowledge

Each student learning outcome must have at least one direct measure associated with it.



Can we use survey results as an assessment method?

Surveys are indirect measures. You must include at least one direct measure per student learning outcome but in addition to that you may also include indirect measure assessment and results. Indirect measures are often times very useful for gaining perspective on very particular questions about the program or what the students perceive. Decisions based on feedback and reflection from participants can be very useful. Part of this report, however, is to show the evidence of student learning through their work. It may help to think of direct measures in terms of what the student will be able to show they did to demonstrate they learned what they needed to from this program.

Other indirect measures include:

  • Course evaluations (ESet)
  • Number of student hours spent doing a particular activity or practicing a skill
  • Grades not based on explicit criteria related to clear student learning outcomes
  • Focus group interview
  • Registration or course enrollment information
  • Surveys or focus groups of faculty members, or supervisors
  • Student perception surveys
  • Employer or alumni surveys
  • Job or graduate school placement rates



What does APAA look for in the report?

For this submission and the future assessment reports we are seeking strong examples of full cycle assessment which includes answers to the following questions:

  • What outcome has been assessed?
  • How was the outcome directly assessed?
  • What meaning did you make from the results?
  • And what was done or changed with the information gleaned from the results of the direct assessment?

You can think of the assessment report submission as an executive summary that contributes to the story we tell accreditors about how we are effectively using direct student learning to inform changes and improve our programs. The report is a mechanism for you to clearly communicate and track program assessment. The intention is that anyone in your discipline or in the assessment office in the future can pick up the reports and get an accurate sense of where assessment is currently and where it is headed for the program.




Who reads these reports and what is done with them?

Starting in the winter of 2015, each report is read and reviewed by at least one member of the APAA team and reviewed using a review template. That feedback is brought to one-on-one meetings between the APAA reviewer and the assessment contact(s) for the individual program. Any changes or notes are made to the review and then a copy is kept in APAA and sent to the unit for their records. Moving forward, we will provide annual review feedback to each program based on their report submission.

The information from the report is used by the APAA office to:

  • Collect examples of full cycle assessment in the colleges and academic programs at OSU
  • Provide evidence to our accrediting body, NWCCU, that we are engaged in institutional-wide full cycle direct assessment
  •  Identify trends and places where APAA can offer support and guidance to help improve the practice and use of local program assessment



We do way more assessment than fits in the report template boxes. What should we do?

If you have additional reports, summaries, or supporting documentation (such as the example rubric, grading tool, or detailed analysis results and reflection), feel free to simply indicate “see attachment, NAME OF DOCUMENT” and include both your appendices and the final report in your annual submission.



Do I need to report on every student in the program?

Some programs may be set up to easily report on the entire program’s population, in which case, it is fine to report on the results from all participants. However, a statistically representative sample assessing direct student work is sufficient for the report. For questions on or assistance with statistically representative samples, please contact the APAA office.



Do we have to report results for non-major students in our courses?

You should exclude non-major students from your analysis and results as the assessment is of the student learning for the degree graduates.



Do we have to report for all campus offerings?

You must report assessment results that are inclusive of all of your program majors, regardless of their primary campus.



Where and how do we submit our report?

You submit your annual assessment report (by April 15th every year) by uploading your completed Assessment Report Template to the folder in SharePoint. For assistance see: SharePoint Instructions 

 



What is the difference between the annual assessment report and 10 year program review?

The annual assessment report is an intentional and targeted look each year at student learning in a program. This information is included as one component of the 10 year academic program review.



We are externally accredited; do we need to submit an assessment report to APAA? 

Yes, you must submit a report annually. However, if your accreditation report includes assessment of student learning outcomes and direct measures to assess student learning, you may simply submit a copyof your accrediation report to APAA. To be able to submit your accreditation assessment materials in lieu of the APAA assessment report format, you must submit a copy of the assessment requirements from your accreditation body and an example report to APAA to review. Additionally, if your accrediting body only requires a report every 3/5/7 years, you must still submit a report annually to APAA reflecting the assessment work that has been done each year.