Objective: The aim of this project is to explore the interrater reliability of the standardized motor exam of the Unified Huntington’s Disease Rating Scale (UHDRS) between experienced neurologists and a learner recently certified by the European Huntington’s Disease Network motor rater training.

Background: Evaluation of motor manifestations is an important component in the assessment for Huntington’s Disease. The standardized motor exam of the UHDRS has been assessed for interrater reliability between experienced clinicians with variable results [1,2]. However, it is unknown how the reliability of a new learner, recently trained to complete the motor exam component of UHDRS, compares with that of experienced raters.

Method: Experienced movement disorder neurologists conducted the UHDRS for the NIH-funded PREVENT-HD research (#NS103475, #NS105509). Per protocol, motor exams are recorded for group consensus if required. After successfully completing the training required, one fourth-year medical student interested in neurology used the recorded exams to complete the UHDRS on each participant. The student rater was kept blinded to the ratings by the experienced raters. Gwet’s reliability statistic was used to estimate the inter-rater reliability [3]. All analyses were completed using SAS version 9.4 (Cary, NC).

Results: At the baseline visit, N = 76 participants completed the UHDRS motor exam: 22 (29.0%) participants had no motor abnormalities, and more than half (n = 42 or 55.3%) had a diagnostic confidence level (DCL) of less than 50%. Few had a DCL between 50% – 89% (n = 10 or 13.2%) or between 90% – 98% (n = 2 or 2.6%). No participant had a DCL exceeding 98%. Overall, there was strong agreement between raters on the 32 motor assessment tasks (Gwet’s AC1 = 87.2%, 95% CI: 85.8% – 88.7%). Among these tasks, the raters were in perfect agreement for five (15.6%) items, strong agreement for 20 (62.5%) items (all Gwet’s AC1 > 0.80), moderate agreement for two (6.3%) items (both Gwet’s AC1 > 0.70), and nominal agreement for five (15.6%) items (all Gwet’s AC1 > 0.40). Results were similar for the N = 36 participants completing a follow-up motor examination.

Conclusion: These results suggest that learner ratings can show strong agreement with experienced raters following certification. Future studies can include additional learner raters to assess the reproducibility of the study.

References: [1] Winder, J. Y., Roos, R. A. C., Burgunder, J. M., Marinus, J., & Reilmann, R. (2018). Interrater Reliability of the Unified Huntington’s Disease Rating Scale-Total Motor Score Certification. Movement disorders clinical practice, 5(3), 290–295. https://doi.org/10.1002/mdc3.12618

[2] Hogarth, P., Kayson, E., Kieburtz, K., Marder, K., Oakes, D., Rosas, D., Shoulson, I., Wexler, N. S., Young, A. B., & Zhao, H. (2005). Interrater agreement in the assessment of motor manifestations of Huntington’s disease. Movement disorders : official journal of the Movement Disorder Society, 20(3), 293–297. https://doi.org/10.1002/mds.20332

[3] Gwet, K. (2008). Computing inter-rater reliability in the presence of high agreement. British Journal of Mathematical & Statistical Methodology, 61(1), 29-48. doi:10.1348/000711006×126600

« Back to 2024 International Congress

MDS Abstracts - https://www.mdsabstracts.org/abstract/investigating-interrater-reliability-of-the-unified-huntingtons-disease-rating-scale-motor-component-between-learner-and-experienced-rater/