The second step in the Assessment Cycle is programming with purpose. Now that you have specified your student learning and development outcomes (SLOs), you need to 1) determine how to achieve those outcomes (i.e., develop a program theory) and 2) clearly articulate the connection between program components and SLOs (i.e., develop a logic model and/or program-to-outcome map). During this step of the process, you will need to consider questions such as:
- Which components of your program are hypothesized to lead to which SLOs?
- Why do you believe these program components should lead to the stated SLOs?
- What do relevant theories say about how to impact your SLOs?
- What evidence-based practices have been shown to impact your SLOs?
What is Program Theory?
Program theory or logic explains how and why a program is expected to work. It makes explicit the connection between the things we do in our programs and the outcomes we hope to achieve. Ideally, this logic should be evidence-based. That is, we should seek to consult theory and empirical research when developing a program theory.
Program theory is critically important because without evidence for why the things we do should work, we risk implementing programs that are inefficient at best and ineffective at worst. In fact, in the Professional Competencies put forth jointly by ACPA and NASPA, there are multiple references to program theory. Competent professionals are described as those who are "intentional in using theory in the design and implementation of learning opportunities" and possess the "ability to articulate, analyze and apply theory to improve practice." The Council for the Advancement of Standards in Higher Education (CAS) also recognizes the importance of program theory. According to their standards, programs should be 1) intentionally designed and 2) guided by theories and knowledge of learning and development.
For more information on program theory, check out this didactic articles by Pope, Finney & Bare and Finney, Wells & Henning or these videos. The first video is short and focuses only on program theory. The second video provides examples of program theory, situates program theory in the assessment cycle, shows how program theory supports the collection of implementation fidelity data, and discusses how CAS program review can incorporate both program theory and implementation fidelity.
How to Build a Theory-Based Program
A theory-based program is one that is intentionally designed to address a certain problem or achieve a certain goal. Theory-based programs can be contrasted with less intentional programs that are often conceived with little thought as to 1) what changes are desired in students' knowledge, attitudes, skills, and behaviors, and 2) how such changes can be reasonably facilitated (i.e., program theory).
Major student affairs organizations like ACPA and NASPA make it clear that intentional, theory-based programs are the expectation. Toward this end, we provide a simple framework to guide student affairs practitioners in the development such programs. For additional assistance building theory-based programs, student affairs professionals at JMU can contact the Center for Faculty Innovation.
Developing a Theory-Based Program in Five Steps
Program Theory in Ac
Step 1: Articulate the Problem
Building a theory-based program begins by clearly articulating the problem you are trying to address or the goal you are trying to achieve. These problems/goals should be meaningful to stakeholders.
Step 2: Identify Plausible Causes and Solutions
Once you have a clear problem or goal in mind, the next step is to consult relevant theory and empirical research to learn more about the underlying causes of the problem and theory-based solutions/strategies given those underlying causes.
Step 3: Select a Course of Action
There is likely more than one theory that has garnered empirical support, which is advantageous as it indicates more than one approach to designing a program to meet the intended SLO. Thus, the next step is to select a reasonable course of action given what you know about your specific institutional context (e.g., the unique needs of your students, resource constraints, institutional priorities, etc.).
Step 4: Develop an Intentional Program
Once you have decided on a course of action (i.e., what you're going to do), the next step is to figure out how to do it. In other words, while theory may provide an outline for your program, it is up to you to fill in the details. A good starting point is to look for intervention studies, or empirical research articles where practitioners describe a particular theory-based intervention and evaluate its effectiveness. It may also be helpful to consult the literature on how students learn (learning theory) and/or how attitudes and behaviors are changed (persuasion theory). Additionally, there is extensive literature on evidence-based pedagogical techniques that can be particularly useful for educational or co-curricular programs.
For additional guidance on finding evidence-informed programming, please visit Evidence-Informed Programming.
Step 5: Evaluate Program Effectiveness
Once you've built and implemented a theory-based program, the next step is to evaluate its effectiveness. If you find that your program is ineffective, it may be an indication that 1) the theory/research you pulled from in Step 2 is wrong or does not generalize to your population of students, 2) the course of action you selected in Step 3 is inappropriate given your specific context, or 3) the particular methods you employed in Step 4 were ineffective or insufficient. Once you've identified where the breakdown occurred, the final step is to make intentional, theory-based changes to the program based on this information.
Click here to see each of these steps applied to a hypothetical program targeting binge drinking behavior.
1American College Personnel Association & National Association of Student Personnel Administrators (2015). ACPA/NASPA professional competency areas for student affairs educators.Washington, DC: Authors.
Developing a Logic Model
A logic model is a depiction of the relationships between program components and outcomes. In other words, a logic model shows which program components will lead to which outcomes, and most importantly, how (i.e., through what causal mechanisms) this change will occur. In this way, logic models can be conceived of as illustrations of the program theory underlying a program. As such, the arrows in these models should be evidence-based.
A Realistic Example
The logic model below was created by Jeremy Penn in the Office of Teaching and Learning at North Dakota State University.
Click image to enlarge
Creating a Program-to-Outcome Map
The primary purpose of a program-to-outcome map is to ensure that each SLO is sufficiently addressed by your programming. In other words, does your program provide students with an opportunity to achieve all of the stated outcomes?
Additionally, program-to-outcome maps help with use of assessment results. If you find, for example, that students did not achieve a certain SLO, you can use a program-to-outcome map to figure out what components of the program need to be reevaluated.
Example 1: Simple Coverage Map
In this map, an "X" indicates which program components contribute to the achievement of which outcomes.
Check For Unnecessary Program Components
- Note that the first program component (video and small group discussion) does not map to any of the program's outcomes. This should prompt facilitators to reevaluate this activity. Is it necessary? Does it detract from other more important activities? Although there are some instances where it may be appropriate to include an activity that does not map to a specific outcome (e.g., some introduction or "ice-breaker" activities), there should be a compelling reason to do so.
Example 2: Degree of Coverage Map
This map is more detailed; it not only specifies which program components map to which outcomes, but the degree of coverage that each program component provides.
Check For Unaddressed Outcomes
- Note that Outcome 4 has minimal coverage. Only one component (i.e., the "Risk Reduction Plan" activity) is mapped to it, and the coverage of this activity is slight. This should prompt facilitators to question whether the program adequately equips students to achieve Outcome 4. Is additional programming necessary? Or should existing programming be strengthened?
Additional Resources
Video: Program Theory
Video: Mapping Objectives to Program Components
Video: Aligning Programming/Curriculum with Objectives
Slides: Inspiration to Action: Using Program Theory to Enhance Learning Outcomes Assessment
Website: Evidence-Based Programming
Examples: Creating Logic Models for Restorative Justice Programs and Advantages of Logic Models for Restorative Justice