Search

We compared several commonly used instruments to determine which should be preferentially used to assess competence in CVC insertion” Lord et al (2019).

Abstract

BACKGROUND: Determining procedural competence requires psychometrically sound assessment tools. A variety of instruments are available to determine procedural performance for central venous catheter (CVC) insertion, but it is not clear which ones should be used in the context of competency-based medical education.

OBJECTIVE: We compared several commonly used instruments to determine which should be preferentially used to assess competence in CVC insertion.

METHODS: Junior residents completing their first intensive care unit rotation between July 31, 2006, and March 9, 2007, were video-recorded performing CVC insertion on task trainer mannequins. Between June 1, 2016, and September 30, 2016, 3 experienced raters judged procedural competence on the historical video recordings of resident performance using 4 separate tools, including an itemized checklist, Objective Structured Assessment of Technical Skills (OSATS), a critical error assessment tool, and the Ottawa Surgical Competency Operating Room Evaluation (O-SCORE). Generalizability theory (G-theory) was used to compare the performance characteristics among the tools. A decision study predicted the optimal testing environment using the tools.

RESULTS: At the time of the original recording, 127 residents rotated through intensive care units at the University of Calgary, Alberta, Canada. Seventy-seven of them (61%) met inclusion criteria, and 55 of those residents (71%) agreed to participate. Results from the generalizability study (G-study) demonstrated that scores from O-SCORE and OSATS were the most dependable. Dependability could be maintained for O-SCORE and OSATS with 2 raters.

CONCLUSIONS: Our results suggest that global rating scales, such as the OSATS or the O-SCORE tools, should be preferentially utilized for assessment of competence in CVC insertion.

You may also be interested in…

[rp4wp]



Reference:

Lord, J.A., Zuege, D.J., Mackay, M.P., des Ordons, A.R. and Lockyer, J. (2019) Picking the Right Tool for the Job: A Reliability Study of 4 Assessment Tools for Central Venous Catheter Insertion. Journal of Graduate Medical Education. 11(4), p.422-429. doi: 10.4300/JGME-D-19-00107.1.