The Power of AI in Enemy Item Detection: Transforming Form Review Processes in Assessment

I.C.E. Exchange Program Description: Form review is a crucial aspect of ensuring the integrity and effectiveness of assessments. However, identifying enemy items poses a significant challenge for subject-matter experts (SMEs). In this session, we delve into how a leading healthcare assessment organization collaborated with two vendor partners to integrate AI solutions into their enemy item detection process.

The session will explore the complexities faced by SMEs in reviewing large volumes of assessment forms while simultaneously identifying enemy items. We will detail the various AI approaches investigated during this collaborative effort and the methodology behind selecting the most effective solution. Specifically, we will discuss competing methods for evaluating item similarity: bag-of-words analyses and analyses based on Sentence BERT (SBERT) large language models.

Furthermore, the session will shed light on the practical implementation of AI in form review, including the options available to test development and psychometric staff for comparing items within forms. Attendees will gain insights into how feedback provided by the AI model informs decision-making processes and aids test developers in refining assembled forms.

Additionally, the presenters will share key metrics used to evaluate the success of the AI-enabled enemy item detection system, along with valuable lessons learned throughout the implementation process. By examining real-world applications and outcomes, this session aims to equip attendees with actionable strategies for leveraging AI to enhance form review processes in testing.

Join us as we explore the transformative potential of AI in mitigating the challenges associated with enemy item detection, ultimately contributing to the validity of assessments.

Thank you Excelsoft for Sponsoring our Test Development and Administration Track.