Abstract:
Introduction: Resident physicians are expected to acquire competence at central venous catheter (CVC) insertion to a mastery standard. Valid competence decisions about resident physicians’ CVC performance rely on reliable data and rigorous achievement standards. This study used data from 3 CVC simulation-based mastery learning studies involving internal medicine (IM) and emergency medicine (EM) residents to address 2 questions: What is the effectiveness of a CVC mastery learning education intervention? Are minimum passing standards (MPSs) set by faculty supported by item response theory (IRT) analyses?
Methods: Pretraining and posttraining skills checklist data were drawn from 3 simulation-based mastery learning research reports about CVC internal jugular (IJ) and subclavian (SC) insertion skill acquisition. Residents were required to meet or exceed a posttest skills MPS. Generalized linear mixed effect models compared checklist performance from pre to postintervention. Minimum passing standards were determined by Angoff and Hofstee standard setting methods. Item response theory models were used for cut-score evaluation.
Results: Internal medicine and EM residents improved significantly on every IJ and SC checklist item after mastery learning. Item response theory analyses support the IJ and SC MPSs.
Conclusions: Mastery learning is an effective education intervention to achieve clinical skill acquisition among IM and EM residents. Item response theory analyses reveal desirable measurement properties for the MPSs previously set by expert faculty panels. Item response theory analysis is useful for evaluating standards for mastery learning interventions. The CVC mastery learning curriculum, reliable outcome data, and high achievement standards together contribute to reaching valid decisions about the competence of resident physicians to perform the clinical procedure.
Reference:
McGaghie WC, Adams WH, Cohen ER, Wayne DB, Barsuk JH. Psychometric Validation of Central Venous Catheter Insertion Mastery Learning Checklist Data and Decisions. Simul Healthc. 2020 Nov 4. doi: 10.1097/SIH.0000000000000516. Epub ahead of print. PMID: 33156260.