Inter‐observer variability of two grading systems for equine glandular gastric disease

Abstract

Abstract Background Equine glandular gastric disease (EGGD) is recognised as a separate entity to equine squamous gastric disease (ESGD) and it is recommended that lesions are graded differently. Currently, no validated scoring system exists for EGGD. Objectives To determine inter‐observer reliability of two previously described grading systems for EGGD and to assess if agreement improved with gastroscopy experience, specialist training or familiarity with the descriptive system. Study design Cross‐sectional survey. Methods A link to an electronic questionnaire containing 20 images of glandular lesions was circulated. Respondents were asked to score lesions using descriptive terminology and a 0‐2 verbal rating scale (VRS). Krippendorff's alpha reliability estimate was used to assess inter‐rater agreement. A mixed effects model was used to determine which descriptive categories were associated with lesions being described as severe and decision to treat. Results Eighty‐two veterinarians responded, 49 diplomates and 33 non‐diplomates. There was no agreement when all four descriptive variables were combined (α = 0.19). Agreement was fair to moderate for severity (α = 0.52), distribution (α = 0.44), appearance (α = 0.38) and shape (α = 0.32). Agreement for the VRS was similar to that for severity (α = 0.53). Agreement was better among diplomates across all categories. Lesion appearance and shape, but not distribution, were associated with both a decision to treat; and lesions being described as severe ( P =< .05). A VRS score 2/2 was associated with a lesion being described as severe (OR 75.2, 95% CI 51.12‐110.48, P =< .001). Main limitations Intra‐observer variability was not assessed. The number of images is relatively small, and the decision to treat is based on several factors in practice. Conclusions Overall, agreement for the descriptive system was poor. Better delineation of descriptive category boundaries and characteristics should be determined. Agreement was similar when comparing the severity category and the VRS. Extrapolation to a VRS based on lesion severity may therefore be possible.