Objective: Here we present a system which is capable of performing nearly-real time monitoring of UPDRS part-3 motor assessments using a combination of multidimensional statistics and computer vision.

Background: The development of neurotherapeutics is often reliant on subjective end-points. These are subject to issues such as inter-rater and intra-rater variability, rater bias, rater drift or simply poor quality assessment. Consequently quality control is a major concern for those performing clinical trials in this field, and has led to costly innovations such as central rating. Intelligent systems, so-called A.I., are of great interest in this regard because they offer the prospect of an automated “second opinion” in real time, which does not suffer from the problems associated with subjectivity.

Method: Firstly, a large data-set of over 15,000 UPDRS part-3 assessments was used to characterise a high dimensional feature space, induced by treating a part-3 assessment as a 33 element feature vector. Using a specific distance metric as a test statistic this allowed a p-value to be allocated to a part-3 assessment immediately and in real time. Secondly, computer vision was used to analyse videos and estimate the correct UPDRS score, allowing a second p-score to be allocated on the basis of this independent analysis within a few minutes or an assessment being completed. Combining these two filters results in a highly sensitive system, which could be used to vastly reduce the cost of quality control by focusing attention onto those assessments which appear to be highly improbable, under the assumption of high quality and consistent rating.

Results: The system was validated using a high quality video and rating data set, collected across multiple clinical sites and scored by trained assessors, which we were able to artificially “corrupt”. The system was shown to be both sensitive and specific for detecting “corrupted” assessments.

Conclusion:
 To our knowledge this system represents the first near-real time intelligent quality control system, which it is apparent could be extended to encompass many more subjective endpoints in this and other conditions.

« Back to MDS Virtual Congress 2020

MDS Abstracts - https://www.mdsabstracts.org/abstract/real-time-quality-control-for-subjective-endpoints-in-clinical-trials/