CARS Newsletter Fall 2020

This semester surely has been one to be remembered. Though it had its challenges, we also accomplished many great things. We hope you'll enjoy this newsletter that recaps all of the highlights we have to share from the fall 2020 semester.

Assessment Day

With many uncertainties surrounding COVID-19, our team, with the support of General Education, University Programs, and JMU Administration, decided to hold an entirely virtual Assessment Day for the incoming freshman in August. Students received an email from our office before they arrived on campus alerting them to their assessment requirements, and they had a multi-week window in which they could complete the assessments. Students were able to complete their assessments before moving into their dorms, once they got onto campus, and one week into the start of the semester. With the changes to instruction modes during the early weeks of the semester, and the initiatives taken to reduce the number of people on campus, the assessment deadline was extended to give students more time to meet their requirements. Dena Pastor and Paula Love published an article in a recent special COVID edition of AALHE's Intersection that details the logistical changes needed to make such adaptations to Assessment Day. You can read more about it here: University-wide Assessment During COVID-19: An Opportunity for Innovation

Our Assessment Day/Data Management Team (from top left to bottom right): Beth Perkins, Dena Pastor, (last year's team photo), Kate Schaefer, Samantha Harmon, and Yelisey Shapovalov

The Assessment Day/Data Management team in CARS has been working hard to plan for the upcoming Spring Assessment Day, which will also be held virtually. Instead of giving students a multi-week window, students will still be expected to complete their assessments in a single-day on February 9th as typically happens. The main difference will be that rather than taking their assessments in-person in a classroom or lab, students will be able to complete their assessments from their personal computers either at home, in their dorm, or any place they can access the internet. Classes are cancelled up until 4:00 pm on the 9th like usual, giving students almost a full day without course obligations so their attention can be directed toward Assessment Day. Testing will open at 8:00 am on the 9th, and close at 8:00 am the following day. The Assessment and Testing Center will be open for walk-in testing during the day on the 9th for students who need a stable internet connection, or quiet place to work on their assessments.

In 2020, our friends at IUPUI launched a podcast and Keston and Caroline were selected to be guests during the first season. In Season 1, Episode 6, Keston and Caroline talk about what learning improvement is and why it matters, and how CARS is positioning itself to address trends in higher education assessment over the coming 3 to 5 years. Keston and Caroline also talk about the themes of their upcoming book titled "Improving Student Learning at Scale: A How-To Guide for Higher Education" that is slated to be released in 2021. The podcast is hosted by Stephen Hundley, who currently serves as the Senior Advisor to the Chancellor for Institutional Improvement at Indiana University-Purdue University Indianapolis. Stephen is also the chair of the IUPUI Assessment Institute.

You can listen to the podcast here, or on most Podcast platforms such as Apple iTunes and Spotify.

In October, CARS hosted a virtual CARS Talk called "Careers Outside the Box" featuring two of our alumni who have found their way into rather unconventional careers with their assessment and measurement. Heather Harris, Ph.D. ('18) is currently working (remotely) on the Data Sciences team at Solstice Studios in Hollywood, California and as the Interim Director of Psychometrics at CommonLit, an education technology nonprofit organization in Washington, D.C. Jason Kopp, Ph.D. ('14) is currently working at Amazon in Seattle, Washington as an Assessment Research Scientist. In this CARS talk, our current students had the opportunity to hear about how Heather and Jason have been able to utilize skills they learned in our program in their careers, and learn about the different types of jobs that are available outside of the traditional paths our past graduates have taken.


Conference season looked a little different this fall, but that didn't stop our students, faculty, and staff from presenting, networking, and hosting virtual booths at a number of different conferences this fall.


In October, our students and faculty presented at the Northeastern Educational Research Association (NERA); normally we would load up a coach bus and head up to New England for the week, but we still had engaging sessions via Zoom. JMU friends from across the country logged on to present, and gather (as seen here) for our traditional JMU photo! The 2020 conference co-chairs were JMU alums Madison Holzman, Andrew Jones, and Thai Ong, along with Jonathan Rubright from the National Board of Medical Examiners.

Though the awards session was not held in person, a virtual video was recorded where our own Sara Finney, and former M.A. student Briana Craig received awards. Sara was given the Leo D. Doherty Award for Outstanding Leadership and Service, and Bri received the award for Best Paper by a Graduate Student for the paper she presented in 2019 titled "Sonography certification examinations: The influence of practice tests and self-selection bias”. Congratulations to both Sara and Bri for these well deserved accolades! You can watch the video of the full awards session here.

Workshops and Virtual Exhibitor Booths

In recent years, CARS has hosted consulting booths in the exhibitor halls at both the IUPUI Assessment Institute and the annual meeting of SACS-COC. At these booths we have students and faculty available throughout the conference to meet with attendees and talk to them about what we offer here at JMU. These booths not only offer an opportunity for us to provide attendees with information about our Assessment 101 workshop they also give students a chance to practice their consultation and networking skills. Over the years we have found that being able to have one-on-one conversations with assessment colleagues from across the country at these booths have led to some great relationships and fruitful partnerships. In fall 2020, we held these booths at these conferences, but like everything else, in a virtual capacity.

While there was certainly a learning curve for our crew, we navigated "zoom rooms" and virtual expo booths fairly seamlessly. With each virtual experience, we gained insight and made improvements at the next opportunity. We're preparing for two conferences in 2021 that we know will also be held virtually, and we expect a few others may follow suit. Despite not being able to engage face to face, we still made promising connections - a few of which have already resulted in enrollments in our Summer 2021 assessment workshops.

Assessment Institute Workshops and Sessions

One of the benefits of holding virtual sessions is that they can be recorded and shared! Our faculty and students presented 14 different sessions at the conference. A few of those workshops and sessions are listed here, with the links to view the recorded session.

Adapting Assessment Approaches in the COVID-19 Era - Special session featuring Keston Fulcher and Brian Leventhal

More Than Results: An Advanced Workshop Integrating Assessment with Learning Improvement - Keston Fulcher and Caroline Prendergast

Plenary Session and Panel Discussion - Featuring Keston Fulcher

Planning Assessment Professional Development Opportunities: Applying an Expectancy-Value Theory Lens - S. Jeanne Horst and Caroline Prendergast

A More Efficient Path to Learning Improvement: The Utility of Evidence-Informed Programming and Practices - Sara Finney and Holly Buchanan

Planning for Learning Improvement: The Role of Hypothetical Examples - Yelisey Shapovalov, Daigo Murakoshi, and Caroline Prendergast

Incorporating Program Theory and Implementation Fidelity in CAS Self-Study - Sara Finney and Gavin Henning

Developing an Assessment and Improvement Skillset - Keston Fulcher, S. Jeanne Horst, and Chris Patterson

One Criterion at a Time: A Guide to Developing Analytic Rubrics - Beth Perkins and Chris Patterson

Perspectives on Learning Improvement - Monica Stitt-Bergh, Gianina Baker, Kathleen Gorski, Keston Fulcher, and Ray Van Dyke

Here are the workshops provided by our faculty and students at the annual meeting of SACS-COC. As these were pre-conference workshops that attendees had to pay to attend, recordings are not available.

Investigating the Reliability and Validity of our Evidence Assessment - Brian Leventhal and Kathryn Thompson

More than Results: reframing Assessment as Learning Improvement - Chris Coleman, Caroline Prendergast, and Keston Fulcher

Tuning to the Same Key: Practical Student Partnerships in Assessment - Nick Curtis and Robin Anderson

Evaluating the Assessment of Student Learning in Higher Education Institutions - Yu Bao and Paulius Satkus

In December, Chris Patterson interviewed fellow doctoral student Caroline Prendergast about her work on the Assessment Skills Framework. This interview was published on the Student Affairs Assessment Leaders blog. You can read more here: A Guide to Professional Development: The Assessment Skills Framework


Yelisey Shapovalov is a second-year student in our M.A. program pursuing a degree in our Quantitative Psychology concentration. Yelisey has also been a Graduate Assistant in CARS since he began the program in fall of 2019. During his time in CARS, Yelisey has held different assistantship roles, but he currently works on the Professional Development team, and helps out with Ethical Reasoning.

Prior to coming to JMU, Yelisey received his Bachelor's degree in Psychology with a Sociology minor from Eastern Mennonite University located here in Harrisonburg. Yelisey held an internship in CARS during his senior year at EMU, and enjoyed his time here which led him to apply to the M.A. program. Yelisey says that his favorite experiences so far have been learning how to use SPSS and SAS in PSYC 604, and being able to use them in his GA roles.

Yelisey's research interests are related to in Assessment and Learning Improvement as well as developing his understanding of measurement for psychological sciences and statistical modeling. I have been researching raters’ effects using FACETS, a rasch program. He sueccessfully proposed his thesis this past week, which focuses on deciding who to select for rating assignments. When asked about his thesis research, Yelisey said "Rater-meditated scores may be unduly influenced by rater tendencies or rater effects. These are systematic difference in how raters score performance assessments that are not due to examinee ability. I am looking at using Many-Facets Rasch Measurement to diagnostically identify which raters exhibit rater effects; subsequently creating a pool of raters that have not exhibited rater effects from which to select from for future rating sessions."

As we break for the holiday season, Yelisey will continue working on his thesis, and prepare to enter his final semester in the Master's program this spring.

Looking Forward...

We don't know what the winter, and spring semester holds as far as the pandemic is concerned but our faculty, staff, and students are prepared for whatever may come. Most of our classes are being held in an online capacity for the spring semester, and many faculty, staff, and students continue to work at a distance. While we certainly miss being together, gather for staff meetings and other fun and instructional events, we know that this is only temporary.

We hope everyone has a wonderful, safe, and healthy holiday break and finds the rest they seek to return energized for the new year. Happy Holidays from everyone in CARS.

Created By
Paula Melester


JMU Creative Media