Evaluation of a method to assess digitally recorded surgical skills of novice veterinary students.

Abstract

OBJECTIVE: To evaluate a method to assess surgical skills of veterinary students that is based on digital recording of their performance during closure of a celiotomy in canine cadavers. SAMPLE POPULATION: Second year veterinary students without prior experience with live animal or simulated surgical procedure (n = 19) METHODS: Each student completed a 3-layer closure of a celiotomy on a canine cadaver. Each procedure was digitally recorded with a single small wide-angle camera mounted to the overhead surgical light. The performance was scored by 2 of 5 trained raters who were unaware of the identity of the students. Scores were based on an 8-item rubric that was created to evaluate surgical skills that are required to close a celiotomy. The reliability of scores was tested with Cronbach's α, intraclass correlation, and a generalizability study. RESULTS: The internal consistency of the grading rubric, as measured by α, was .76. Interrater reliability, as measured by intraclass correlation, was 0.64. The generalizability coefficient was 0.56. CONCLUSION: Reliability measures of 0.60 and above have been suggested as adequate to assess low-stakes skills. The task-specific grading rubric used in this study to evaluate veterinary surgical skills captured by a single wide-angle camera mounted to an overhead surgical light produced scores with acceptable internal consistency, substantial interrater reliability, and marginal generalizability. IMPACT: Evaluation of veterinary students' surgical skills by using digital recordings with a validated rubric improves flexibility when designing accurate assessments.