Objective: The goal of this work is to train a network that detects a gait cycle precisely to improve the accuracy of a video-based gait analysis.

Background: The video-based gait analysis (Gaitome) established previously in our study [1] provides the spatiotemporal gait parameters with single 2D RGB video. The algorithm detects the gait cycles based on contact information from the mobility of the estimated foot position traced in the 2D video. However, the current algorithm shows relatively low performance when pose estimation results are inaccurate. This calls for the reliable gait cycle detection algorithm (GCDN) based on more objective and robust dataset.

Method: For training GCDN, we used a portion of the dataset produced by Warmerdam et al. [2]. They collected motion data with a 3D motion capture system, and the video were taken from two different view cameras simultaneously. We used gait related 3D motion capture data (walk turn, free walking, and walking at various speeds) of 8 subjects (2 healthy adults and 6 patients with Parkinson disease). The total length of video data we used for training is 25.46 minutes (45834 frames). We trained a recurrent neural network (RNN) in a supervised manner. The contact state of toe and heel point of the feet was defined with 3D motion capture data (the ground truth value, or GT), we used motion capture data. The input of the network is the 13 keypoints of the lower body estimated using the pose estimator (OpenPose [3]) and the GT, and the output is contact states of the toe and heel of the feet.

Results: Figure 1 shows the gait cycle detection results obtained by GCDN. For evaluation, we measured the correspondence between the gait cycle result of the GCDN and the GT (motion capture data). The gait cycle result of GCDN was 58.8% consistent with the GT. Correspondence values of key points (left toe, left heel, right toe, and right heel) are 0.60±0.11, 0.60+0.11, 0.58+0.11, 0.57+0.11, respectively (Table 1). Even though we used relatively small amount of training data, we verified that GCDN has potential to detect a gait cycle of patients with movement disorders.

Conclusion: In this study, we propose the novel GCDN that was derived from the 3D motion capture system. The algorithm may be used to improve the performance of a gait analysis algorithm based on single 2D RGB video (Gaitome).

Figure 1. Gait cycle detection result

Table 1. Correspondence of gait cycle

References: [1] Shin JH, Yu R, Ong JN, et al. Quantitative Gait Analysis Using a Pose-Estimation Algorithm with a Single 2D-Video of Parkinson’s Disease Patients. 2021, JPD, :1-13.
[2] WARMERDAM, Elke, et al. Full-body mobility data to validate inertial measurement unit algorithms in healthy and neurological cohorts. Data, 2022, 7.10: 136.
[3] CAO, Zhe, et al. Realtime multi-person 2d pose estimation using part affinity fields. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. p. 7291-7299.

« Back to 2024 International Congress

MDS Abstracts - https://www.mdsabstracts.org/abstract/the-novel-gait-cycle-detection-network-gcdn-for-video-based-gait-analysis-based-on-single-2d-rgb-video/