CARS Newsletter September 2019

Welcome to a new academic year in the Center for Assessment & Research Studies!

We had a very busy summer and the new year is off to a similar start. In June & July, CARS welcomed five new and twelve returning Graduate Assistants from the Assessment & Measurement Ph.D. and Psychological Sciences M.A. programs; and our first Graduate Assistant from the College of Education. These students make the work that we do in CARS possible, and we are so happy to have them on board for what we are sure will be another fruitful year. We also welcomed a new faculty member who will teach in our Graduate Psychology programs, as well as serve in an Assessment Specialist role within CARS. You can read more about Yu Bao below. In addition to orienting new students and staff, CARS hosted a number of summer assessment workshops that were attended by faculty & staff seeking professional development in assessment and learning improvement. Our boot-camp style workshop, Assessment 101, was held in both June and July; and for the first time ever we invited guests from outside of JMU to participate. This pilot year was a huge success thanks to the hard work of our professional development and administrative teams, and of course, our capable graduate student facilitators! In July, invited faculty from across Academic Affairs gathered to rate the 2018-19 assessment reports (APTs); and in August, we welcomed the JMU Undergraduate Class of 2024 through our annual Freshman Assessment Day. We have already hit the ground running in September and are pleased to share some of these exciting stories in our September 2019 Newsletter.

CARS welcomes new faculty member Dr. Yu Bao!

Yu Bao joins us from the University of Georgia where she received her Ph.D. in Quantitative Methodology. Her research focuses on advancing and applying new quantitative methods in educational assessment to provide accurate and reliable feedback for student knowledge and skills. She is also interested in developing efficient assessments using adaptive algorithms to reduce students’ test fatigue and anxiety. She is thrilled to join CARS and looks forward to developing collaborative relationships with faculties and graduate students.

Fostering relationships and shared scholarship: Doctoral students travel to University of North Alabama...

Over the summer, CARS partnered with the folks from Ethical Reasoning in Action to co-host a week-long workshop called "Learning Improvement in Action: Integrating Ethical Reasoning & Program-Level Learning Improvement" to teams of faculty from a number of different institutions. These working groups traveled to Harrisonburg from all over the country (and even internationally!) to work on learning improvement projects in a focused setting. Assessment & Measurement Ph.D. students Andrea Pope and Caroline Prendergast helped facilitate this workshop alongside faculty from CARS, and the Ethical Reasoning in Action office.

One of the participants in this workshop was Bliss Adkison, Associate Director for Academic Affairs and SACS-COC Liaison at the University of North Alabama. Bliss was so engaged by the workshop and the interactions she had with our students, she invited Andrea & Caroline to travel to Florence, Alabama to deliver a keynote and workshop on learning improvement to faculty and staff at UNA. Andrea and Caroline delivered their keynote and workshop based on the foundational learning improvement model developed by Keston Fulcher & Co. and outlined in the NILOA Occasional Paper "A Simple Model for Learning Improvement: Weigh Pig, Feed Pig, Weigh Pig." This is just one example of how our professional development opportunities often spark engaging dialogue between JMU and our colleagues at institutions around the world. It's no surprise that Andrea & Caroline's presentations were well received by the folks at UNA, and we look forward to further opportunities to grow our relationship with Bliss.

Pictured here: On the campus of UNA, Andrea and Caroline participate in a campus tradition of dipping their pencils in the fountain to acquire good luck for the beginning of a new semester.

fall 2019 Assessment Day

Fall Assessment Day was held on Friday, August 23rd, 2019. The Assessment Day team of graduate assistants and administrative support staff, under the leadership of Dr. Dena Pastor, Assistant Director for Assessment Operations, worked diligently in the months leading up to "A-Day" to ensure it was as successful as always. Thanks to the efforts of many new and returning proctors, we were able to assess approximately 4,500 incoming freshman students. We had an attendance rate of 94%, which is comparable to most years. The Assessment Day team held three proctored makeup sessions throughout September to allow for testing of students who were unable to attend on August 23rd.

In the following months, our Assessment Day/Data Management team will work with the data collected and deliver reports. Additionally, under the leadership of Dena Pastor, CARS will be implementing new data management policies to further enhance our data security measures for all future and retroactive Assessment Day data. In a quickly-changing technological world, it is vital that our team stays on top of the most effective and responsible data management policies. Thanks to Dena for her leadership, and for keeping CARS data safe and secure!

The 2019-2020 Assessment Day Team (left to right: Beth Perkins, Sean York, Dena Pastor, Paula Love, Tom Waterbury)

Assessment & Measurement PhD students present on their summer internship experiences...

Internships are a great way for our students to take the skills and knowledge they have acquired in their courses and their assistantships and apply them in real industry settings. On September 20th, Tom Waterbury, Derek Sauder, and Nikole Gregg presented about their summer internship experiences at a CARS Talk held in Lakeview Hall.

From left to right: Tom Waterbury, Derek Sauder, and Nikole Gregg

Tom spent his summer in Philadelphia, PA working as a psychometric intern at The College Board. While working with their equating team, Tom helped College Board explore ways of equating test forms using international test-takers, rather than the traditional method of using only domestic test-takers. Tom enjoyed his internship and liked exploring the Philadelphia area. Many companies that hire Psychometricians are headquartered in Philadelphia, so this was a great opportunity for Tom not only to learn about potential jobs, but to see what it would be like to live in Philadelphia when he graduates in May!

Summer Internships, continued...

Derek also spent his summer in Philadelphia while working at the American Board of Surgery (ABS) for ten weeks. During his internship, Derek worked on a variety of projects, including examining the performance of the Rasch model with extremely small sample sizes. He also helped develop a workshop on optimization tools and applications, and wrote a paper detailing the use of optimization techniques to assign raters to rater teams to minimize differences in overall rater team severity. Derek felt that the internship "was a great learning experience and a first look into the inner workings of a testing organization." He also enjoyed working with JMU Assessment & Measurement alumni Jason Kopp and Andrew Jones who currently hold positions at ABS.

Nikole traveled all the way to New Hampshire to work at The National Center for the Improvement of Educational Assessment (NCIEA), which is a consulting organization for K-12 assessment and accountability systems. During her internship at The Center for Assessment, Nikole was able to complete two projects related to the implications of the Every Student Succeeds Act (ESSA). Nikole really enjoyed her internship experience and the opportunity to learn about K-12 assessment. She was also able to attend Technical Advisory Committee meetings, and presented at a conference targeted to state assessment and accountability leaders in K-12 education. Nikole is currently a second-year PhD student in the Assessment & Measurement program. Typically, summer-long internships are taken later in the course of studies, but because Nikole is also a graduate of our Quantitative Psychology M.A. program, she was prepared for the experience, and won the internship in a competitive application pool!

Congratulations to all of our students on successful summer internships and we thank them for sharing these experiences with the faculty & students in CARS!

Did you know CARS has a new Vision?

In the summer of 2018, CARS Faculty & Staff held a retreat to set a new strategic plan for the immediate future. Out of this plan, we identified a need for a new vision. We kicked off this new plan in 2018-19 and were able to reflect this summer on how successful our efforts have been so far. We were able to accomplish so much in just one academic year; and much of what we do in the coming year will be building upon that foundation. Keep your eyes out for exciting opportunities to interact with CARS through our Professional Development initiatives.

None of this would be possible without the leadership of Dr. Jeanne Horst, and her capable Professional Development Team: Caroline Prendergast, Beth Perkins, Chris Patterson, and Samantha Gonzalez. Special thanks to Paula Love for providing the logistical skills to make these ideas a reality, and as always, thanks to the support and leadership of Dr. Keston Fulcher.

Looking forward...

We have a lot of exciting events coming up in October! We'll be at the Assessment Institute at Indiana University-Purdue University Indianapolis (IUPUI) with a consulting booth October 12th - 15th; and the annual meeting of the Northeastern Educational Research Association (NERA) in Trumbull, Connecticut October 15th - 18th. If you find yourself at either of those conferences, be sure to stop in and say hello! We'll also be holding the following CARS Talks here on JMU's campus in Lakeview Hall. All are welcome.

Upcoming CARS Talks

Generalized Boosted Modeling in R; October 4, 2019 from 9:00 - 11:00 AM in Lakeview Hall 1165. Presented by: Briana Craig, Beth Perkins, Dr. S. Jeanne Horst, and Dr. Heather Harris (Inteleos)

NERA & IUPUI Student Presentations; October 11, 2019 from 10:00 AM - 12:00 PM in Lakeview 1165. Come see our students present the material they will share at these upcoming conferences. Presented by: Derek Sauder, Kathryn Thompson, Paulius Satkus, Nikole Gregg, Chris Patterson, Daigo Blanco, and Yelisey Shapovalov.


JMU Creative Services; Created with an image by lecroitg - "test testing bubble form"