Appendix F ­ Non­instructional Program Assessment

Student learning outcomes are like learning objectives for an instructional course in their focus on the measurable results of student learning. One difference is that verbs emphasizing what students will be able to “do” or “know” after the learning process is complete replace the rather vague verbs “comprehend” and “learn.”
 
The overarching goals of a non-instructional program, beyond service to the student, are the student learning out-comes. An emphasis, therefore, is on results which are sometimes reflected in the term “accountability.”
 
The other change between learning objectives and student learning outcomes is that the new accreditation standards now require colleges to collect data on the success of students meeting those overarching goals. Colleges are then charged with analyzing the data and making changes that will result in more effective student learning.
 
Student learning outcomes are the measurable skills or accomplishments which embody the overarching goals of anon-instructional program or instructional course. they represent the most important learning that takes place through interaction with a program or participation in a course. It may be helpful to think of them this way: when students complete their interaction with your program, you want them to be in firm possession of certain abilities or knowledge, and you want them to retain those abilities or that knowledge. those are the student learning outcomes.
 
Because there are numerous ways in which a student may interact and gain knowledge from a non-instructional program, managers directing those programs may choose to have more than one student learning outcome for which they assess student learning.
 
In order to begin formulating your program’s student learning outcomes, ask yourself the following questions:
  • Describe the ideal student or client to utilize your services?
  • What are the attributes, skills, and values that are supported and nurtured by the student’s experience when in contact with your services?
  • What does this student know as a result of using your services?
  • What can this student do as a result of using your services?
  • What does this student care about as a result of using your services?
  • What are the services provided by your unit that contribute to the development of the ideal student?
    1. Develop criteria (3 domains – see the tables below)
    2. Brainstorm
    3. Prioritize
    4. Select
Focus on what the student will be able to know, do, and feel (3 domains).
Use verbs appropriate to learning based on Bloom’s Taxonomy.
  • C: Is it consistent and supportive of the Mission Statement?
  • s: Is it appropriate for the ability of the students?
  • : Can it be observed and tested?
  • : A s : Is it important to the non-instruction service unit?
  • C : Is it a current service?
  • C l: Is it clear, precise, and simple?
Make sure that the program SLO is something that is readily observable and measurable – in other words, build in assessment from the beginning. Don’t create a program SLO that you can’t envision a way to observe or evaluate orthat requires data that you won’t be able to access.
According to Bloom’s Taxonomy, students experience several levels of learning from the acquisition of facts to the ability to think critically and solve problems.
  1. C– recall or recognition of specific facts, procedural patterns, and concepts that serve in the development of de嘀ning knowledge classifiation
  2. 2.n – performance defining physical skills
  3. 3.A– defining behaviors that correspond to values, appreciation, and attitudes.
The Cognitive Domain (related to knowledge)
  C A A   E
Student remembers or recognizes information or specifics as communicated with little personal assimilation Student grasps the meaning behind the information and interprets, translates, or comprehends the information Student uses information to relate and apply it or to a new situa-tion with minimal instructor input Student discriminates Organizes, and scrutinizes assumptions in an attempt to identify evidence for a conclusion Student creatively applies knowledge and analysis to integrate concepts or construct an overall theory Student judges or evaluates information based upon standards and criteria, values, and opinions
Cite
Label
List
Enumerate
Identify
Imitate
Match
Name
Quote
Recall
Reproduce
State
Write
Convert
Define
Describe
Discuss
Estimate
Explain
Generalize
Identify
Illustrate
Locate
Paraphrase
Restate
Summarize
Apply
Chart
Compute
Demonstrate
Determine
Dramatize
Establish
Make
Manipulate
Prepare
Project
Solve
Use
Analyze
Compare
Contrast
Correlate
Diagram
Dissect
Differentiate
Distinguish
Infer
Investigate
Limit
Outline
Separate
Assemble Create
Construct
Design
Develop
Formulate
Generate
Hypothesize
Initiate
Invent
Modify
Reframe
Synthesize
Access
Appraise
Conclude
Critique
Decide
Defend
Diagnose
Evaluate
Judge
Justify
Rank
Recommend
Support
 
The Psychomotor Domain (related to skills)
      C A C
Student translates sensory input into physical tasks or activities Student is able to replicate a fundamental skill or task Student recognizes standards or criteria important to perform a skill or task correctly Student uses standards to evaluate own performance and make corrections Student applies skill to real life situation Student is able to instruct or train others to perform this skill in other situations
Hear
Identify
Observe
See
Smell
Taste
Touch
Watch
Attempt
Copy
Follow
Imitate
Mimic
Model
Reenact
Repeat
Reproduce
Show
Try
Check
Detect
Discriminate
Differentiate
Distinguish
Notice
Perceive
Recognize
Select
Adapt
Adjust
Alter
Change
Correct
Customize
Develop
Improve
Manipulate
Modify
Practice
Revise
Build
Compose
Construct
Create
Design
Originate
Produce
Demonstrate
Exhibit
Illustrate
Instruct
Teach
Train
 
The Affective Domain (related to attitudes, behaviors, and values)
        Ch
Student becomes aware of an attitude, behavior, or value Student exhibits a reaction or change as a result of exposure to an attitude, behavior, or value Student recognizes value and display this through involvement or commitment Student determines a new value or behavior as important or a priority Student integrates consistent behavior as a naturalized value in spite of discomfort or cost. The value is recognized as a part of the person's character.
Accept
Attend
Describe
Explain
Locate
Observe
Realize
Receive
Recognize
Behave
Comply
Cooperate
Discuss
Examine
Follow
Model
Present
Respond
Show
Studies
Accept
Adapt
Balance
Choose
Diffrentiate
Defend
Influence
Prefer
Recognize
Seek
Value
Adapt
Adjust
Alter
Change
Customize
Develop
Improve
Manipulate
Modify
Practice
Revise
Authenticate
Characterize
Defend
Display
Embody
Habituate
Internalize
Produce
Represent
Validate
Verify
t (T)
Department, non-instructional service
unit: ______________________________________________
Mission statement for the department, non-instructional service unit
 
 
 
Cognitive
(knowledge or concepts)
 
 
Psychomotor
(skills or performance abilities)
 
 
Affective
(Attitudes or values)
 
 
 
Setting goals for their departments or programs is not a new idea to managers and supervisors; it is an integral part of planning and directing the work flow of a program or department. Assessing a student’s knowledge about services received or processes learned, on the other hand, may be a new concept.
 
Thee Student Learning Outcomes Assessment mandate focuses all of us on the strong links between statements of goals (SLOs) and their assessment. Here is a concise definition of assessment that explains those connections:
Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance. When it is embedded effectively within larger institutional systems, assessment can help us focus our collective attention, examine our assumptions, and create a shared academic culture dedicated to assuring and improving the quality of higher education (Thomas A. Angelo, AAHE Bulletin,November 1995, p. 7).
The WASC accreditation standard for student learning outcomes does not micromanage the assessment process. Instead, it leaves to managers and supervisors the decisions that will determine how useful the assessment process will be in improving teaching and learning. In other words, managers and supervisors decide how they will assess the student learning outcomes.
Any tool that measures the degree to which students have met a learning outcome qualifies as assessment. Such tools include surveys and pre- and post-tests. Most outcomes can be measured in a variety of ways.
Some of the principles of assessment to keep in mind when developing an assessment plan include:
  • Assessment is an integral part of teaching and learning.
  • Assessment is based on measurable criteria.
  • Assessment is accomplished with a variety of methods.
  • Assessment involves processes as well as outcomes.
  • Assessment improves teaching and learning.
  • Assessment informs planning and decision-making.
Some questions to ask:
  • What assessment instruments and methods may be used in the department or service unit?
  • Will they provide useful information?
  • What purpose will the assessments serve?
  • How will the assessment results be used?
  • Will the data collected from the assessment inform the unit’s decision-making?
Criteria for selection of assessment methods and implementation:
  • The learning outcomes selected for assessment are important.
  • The assessment methods measure student achievement.
  • The assessment methods are varied.
  • The criteria for determining success is established
  • The time frame for assessing student learning is doable.
  • The time and person responsible for the administration of the assessment is clear
  • The time and person(s) responsible to collect and analyze the data is clear.
Here are some steps that will help you develop an assessment plan:
First, check your SLOs:
  • How many are there? If there are more than three, they likely aren’t true SLOs – they may be objectives that were just moved into the SLO area. You should revise them into SLOs before creating an assessment plan.
  • Are the SLOs overarching (“big picture” learning for the department or program) or are they smaller objectives (things leaned in just one interaction with the department, for instance)? If they are not overarching, you should revise the SLOs before creating an assessment plan.
  • Is the student learning described in the SLO observable and measurable? If not, you should revise the SLOs to make them observable and measureable before creating an assessment plan.
Next, decide on an appropriate assessment tool. Consider:
  • What is the SLO asking the students to do?
    • Identify a fact?
    • Perform a skill?
    • Analyze a complex phenomenon?
    • Solve a problem?
    • Explain a concept?
    • Apply skills or knowledge to real-world situations?
    • Evaluate options and select appropriate resource tools
  • What types of activities will allow students to demonstrate the SLO (see Appendix A for more information about choosing an assessment tool)?
    • Pre and post-tests
    • Skill demonstrations?
    • Surveys?
  • What criteria will you use to measure success or failure to meet the SLO?
    • Rubric?
    • Raw score
Then, decide how and when you will do the assessment:
  • How often will you assess?
    • Will it be on a semesterly cycle? An annual cycle? Other?
    • Are there similar services that could be grouped together?
    • Which semester will you begin assessing this service?
    • If you make changes, when will you reassess to see the effects?
  • Will you assess all students or will you use sampling?
    • If you are sampling, how many students will be involved?
    • How will you decide which students to involve?
  • What do you need to do to prepare?
    • Do you need to set up meetings with staff?
    • Do you need to create a test or rubric?
    • How will you distribute materials?
    • Do you need any additional resources or training?
Finally, think about how and when you will share the assessment results and use the results in decision-making about the service and/or program (“closing the loop”):
  • What needs to be done to gather and present the data?
    • Do you need data from Institutional Research?
    • What format will you use to share the data? PowerPoint? Handouts? Other?
  • When will be a meaningful time for your department to reflect on the results?
    • Department retreats?
    • Department meetings?
    • Other?
  • What changes might be made to the service or program based on the results?
    • Changes to the assessment tool or method?
    • Changes to the service, program or department?
    • Changes to the service delivery methods?
    • Changes to student resources or services?
  • How will these results inform other decisions for the department or program?
    • How do the results of this assessment fit into the larger picture of the program or department?
    • Is there a need for professional development on specific topics?
    • Should budgeting priorities change?
    • Should staffing or other resources be adjusted?
    • Other
Types of Data
Quantitative (numeric scores) and Qualitative (narratives, observations, interviews)
Types of Assessment
Types Examples
Performance Use of tools/technology            Tutorial
Interview                                  Peer Review
Role-Play                                Demonstration
Portfolio Student Portfolio
Self-Assessment
Journal
Production Esssay                                    Visual
Oral presentation                       Speech
Survey Focus group
Survey of student satisfaction
Survey of student services accessed
Questionnaire
Quiz Informal evaluation
Test Placement test                          Pre-test
Diagnostic
 
Assessment Student Learning Outcome (Template)
Learning Outcome



Assessment method



Describe the tool or strategy



When will you assess and how often?



Resources needed (staff, equipment, materials)



What is the criteria for success?