Playing the Long Game: Using Data Forensics to Help Secure Longitudinal Assessments

I.C.E. Exchange Program Description: Like many other medical specialty boards, the American Board of Pediatrics (ABP) has developed a longitudinal assessment that board-certified pediatricians can take to meet the examination requirement for maintaining certification. Previously, pediatricians had to pass a proctored examination at a test center once every 10 years.

MOCA-Peds, the ABP’s longitudinal assessment, is an unproctored and open-book exam. A relatively small number of test questions are delivered at 3-month intervals (i.e., “quarters”), and within each quarter, pediatricians have the flexibility to take questions when they want and on whatever device they choose. Questions may be answered in one sitting or across multiple sessions with the quarter. However, participants are still required to answer questions independently. Communication with other individuals is not permitted, as the assessment is intended to measure the knowledge of the individual, not the group.

The ABP has conducted extensive data analysis to evaluate test-taking behavior in MOCA-Peds. While most pediatricians behave professionally and adhere to the honor code and participation agreement, the data suggests some violations are occurring. This session will summarize the ABP’s efforts to prevent collusion, to detect it when it occurs, and to respond to suspected incidents. More specifically, presenters will share analysis results, provide examples of collusion behavior, and discuss decisions made by the organization to help ensure the validity of the assessment.