Taking Advantage of Technology in Exam Development: How to Streamline SME Involvement & Stay Psychometrically-Sound

I.C.E. Exchange Program Description: The credentialing industry needs to evolve in its responsible incorporation of technological advancements into measurement processes. The use of integrated technology-enabled solutions can span exam design, delivery, and maintenance, and should focus on how to increase efficiency and consistency of exam development while not diminishing validity arguments. The incorporation of these methods streamlines, but not eliminate, the involvement of subject matter experts (SMEs) and exam sponsors. It will appropriately shift their contributions to pivotal decision points that ensure the fidelity and accuracy of the outcomes.

This workshop will highlight how innovative approaches relying upon heuristics, natural language processing, machine learning, and artificial intelligence can be embedded into psychometric workflows across the exam development lifecycle. The presenters will suggest various pathways through which these processes can augment not supplant SME efforts to reduce exam time-to-market and enhance alignment to current domain practices and test taker expectations. Discussions will be facilitated regarding the responsible usage of technological innovations across exam design, domain analysis, content development and review, assessment development and analysis, and standard setting.

This interactive workshop will draw attention to the spectrum of available technological options that can reduce reliance on human input from program design through exam maintenance but retain psychometric rigor throughout the entire exam development cycle,. The presenters will discuss how balance can be established between program practicalities and psychometric practices by leveraging modern advancements to accelerate the exam development phases resulting in more automatically-generated yet psychometrically-sound exams.

The intent is for attendees to understand how, when, and where to combine modern technology with traditional exam development approaches to reduce the level of effort to produce yet enrich the quality of the resulting credentialing exams. Overall, the goal is for attendees to understand how to do more, responsibly, with less by leveraging modern and novel exam development workflows.

Thank you PSI for Sponsoring our Credentialing Innovations Track.