A Power Couple: CARS and the A&M PhD Program

CARS is directly linked to two academic programs in Graduate Psychology: 1) the Assessment & Measurement PhD program (A&M), and 2) the Quantitative Concentration of the Psychological Sciences MA program. This relationship fosters the cutting edge assessment scholarship produced and practiced by CARS. 

How are CARS and the A&M PhD Program Linked?

• All of the 9 faculty members in CARS have dual appointments as professors in the A&M doctoral program; balancing various assessment and instructional loads.

o    Assessment activities include: working directly with clients on campus (i.e., academic degree programs, student affairs, or general education) to evaluate student learning outcomes and ultimately improve JMU programming.

o    Instructional activities include: teaching courses in the A&M PhD program and advising graduate students.

Both assessment and instructional activities involve conducting research on best practices and issues in higher education assessment and measurement. For some faculty, assessment is 60% of their time, for others it may be closer to 30%. These personalized workloads allow CARS and the PhD program to capitalize on individual expertise. Faculty with a wider range of assessment knowledge contribute more heavily to assessment projects and faculty with a wider range of instructional expertise are able to offer additional courses.

• Students in the A&M PhD program and the quantitative psychology MA program are hired as staff in CARS. Students apply the training they receive via coursework and advising to work with assessment clients on campus. This makes the additional courses offered in the A&M program even more important. Moreover, being advised by faculty with dual appointments means students not only learn about best assessment and measurement practices in courses, they have the opportunity to see and apply these practices alongside their mentors.

How Does this Link Benefit CARS?

• Staff with current, comprehensive training: The connection between CARS and the A&M program results in CARS staff with current training. Students work with assessment clients while learning about best assessment and measurement practices in courses. They have the opportunity to apply these practices directly to their assessment work. Additionally, students have access to comprehensive courses and training sessions (e.g., on policy, simulations, structural equation modeling, item response theory) that may not be available to many assessment practitioners. These experiences promote excellence in assessment practice at JMU. In fact, many faculty and students collaborate on scholarly work that demonstrates or influences this assessment excellent. Some examples include:

DeMars, C. E. (2013). A tutorial on interpreting bifactor model scores. International  Journal of Testing, 13, 354-378.

DeMars, C. E., Sundre, D. L, & Wise, S. L. (2002). Standard setting: A systematic approach to interpreting student learning. Journal of General Education, 51, 1-20.

Rodgers, M., Grays, M.P., Fulcher, K. H. & Jurich, D. P. (2012). Improving academic program assessment: A mixed methods study. Innovative Higher Education, 38(5).

Horst, S. J., Ghant, W. A., & Whetstone, D. H. (2015). Enhancing assessment through the use of mixed    methods. Assessment Update, 27, 4-14.

• Scholarship on cutting-edge issues: Not only do CARS faculty and students encounter assessment and measurement issues in courses, but also in assessment work. Thus, they have two motives for tackling these issues empirically. Academic, student affairs, and general education programs on campus provide CARS with practical assessment problems. Courses from the A&M program arm CARS staff with the best tools to research solutions to these problems. The result is a powerful system in which higher education assessment questions are answered by thorough empirical investigations. For example, several faculty and students have explored the impact of student motivation on student learning outcome assessment from both applied and technical perspectives:

Finney, S. J., Sundre, D. L., Swain, M. S., & Williams, L. M. (in press). The validity of value-added estimates from low-stakes testing contexts: The impact of change in test-taking motivation and test consequences. Educational Assessment.

Swerdzewski, P. J., Harmes, J. C., & Finney, S. J. (2011). Two approaches for identifying low-motivated students in a low-stakes assessment context. Applied Measurement in Education, 24, 162-188.

Thelk, A. D., Sundre, D. L., Horst, S. J., & Finney, S. J. (2009). Motivation matters: Using the Student Opinion Scale (SOS) to make valid inferences about student performance. Journal of General Education, 58, 129-166.

Wise, S. L., Pastor, D. A., & Kong, X. J. (2009). Understanding correlates of rapid-guessing behavior in low stakes testing: Implications for test development and measurement practice. Applied Measurement in Education, 22, 185-205.

Zilberberg, A., Finney, S. J., Marsh, K. R. & Anderson, R. A. (2014). The role of students’ attitudes and test-taking motivation on the validity of college institutional accountability tests: A path analytic model. International Journal of Testing, 14, 360-384.

• Scholarship directly effects practice: Once an empirical solution to a practical problem is found, CARS staff are able to apply the solution, and other new empirical practices, in their own assessment work. For example, Dr. Sara Finney and five graduate students from the A&M and Quantitative Psychology programs have empirically explored implementation fidelity, a critical component of the assessment cycle.

Fisher, R., Smith, K., Finney, S. & Pinder, K. (2014). The importance of implementation fidelity data for evaluating program effectiveness. About Campus, 19, 28-32.

Gerstner, J. J. & Finney, S. J. (2013). Measuring the implementation fidelity of student affairs programs: A critical component of the outcomes assessment cycle. Research & Practice in Assessment, 8, 15-28.

Swain, M. S., Finney, S. J., & Gerstner, J. J. (2013). A practical approach to assessing implementation fidelity. Assessment Update, 25(1), 5-7, 13.

Scholarship has further informed assessment practices at JMU, such as student learning outcome assessment in Orientation. The connection between the A&M PhD program and CARS allows CARS staff to close the loop in the cycle of research and practice.

How Does this Link Benefit A&M?

• Funding for students: Allowing A&M students to work in CARS provides funding for graduate students. This negates competition for assistantships, allowing CARS staff and A&M students to focus more fully on coursework and assessment practice. Additionally, CARS aids student travel to conferences, expanding the reach of empirical work conducted by its staff. For more information on funding in the A&M PhD program, see the following short video. 
• Applied experience for students: CARS also provides A&M students the opportunity to apply the content they cover in courses. Students not only use this content on course-specific projects, but also with assessment clients. They get to use advanced techniques in real world settings on projects that impact programming at JMU. For more information on the courses available to A&M students, check out this short video. For more information on the applied experiences A&M students can take advantage of as CARS staff and graduate assistants, check out this short video from one of our recent graduates, Matthew Swain.

Interested in the Experience?

Interested in graduate studies in assessment and measurement? Check out our A&M PhD program or the Quantitative concentration of our Psychological Sciences MA program. For a thorough overview of important aspects of the PhD program, see the videos listed on our program website.