Up to one's knees in data - Data-driven intent recognition using electromyography for the lower limb
Robert Schulte is a PhD student in the department Biomedical Signals and Systems. Supervisor is prof.dr. J.H. Buurke from the faculty of Electrical Engineering, Mathematics and Computer Science and co-supervisors are dr. E.C. Prinsen from the faculty of Engineering Technology and dr. M. Poel from the faculty of Electrical Engineering, Mathematics and Computer Science.
Data-driven intent recognition using electromyography (EMG) has the potential to make actuated prosthesis more intuitive. Although electromyography has its challenges, it also forms an opportunity to realize more intuitive control of prostheses through intent recognition. Compared with mechanical sensors it can provide more appropriate information on what a user wants to do, instead of what the user is doing. However, data-driven approaches require a lot of data. Therefore, large scale data collection needed to be facilitated. In chapter 2 we developed and validated a synchronization method for wearable motion capture and EMG measurement systems. Using continuous cross-correlation of acceleration signals we managed to obtain satisfactory synchronization results below the sampling frequency of the slowest modality. Using this new synchronization technique we collected a large database, MyPredict, one of the biggest of its kind, containing 55 able-bodied subjects measured in 85 measurement moments. These data sets contain kinematic and EMG data from subjects performing gait-related activities such as stair climbing and ramp walking, freely transitioning from one activity to the next. This database, containing three data sets MyPredict 1-3, is described in chapter 3.
Using our novel database, we investigated the use of genetic algorithms to construct optimized feature sets to be used in lower limb prosthetic control in chapter 4. Four different optimized feature sets were constructed based on the results of the genetic algorithm. These four optimized feature sets were compared against the best performing state-of-the-art feature set on two different data sets. All optimized feature sets outperformed the state-of-the-art feature set on the MyPredict 1 data set and most optimized feature sets outperformed state-of-the-art on the ENABL3S data set. These results showcase the possibilities of data-driven feature selection and the potential to optimize current control systems.
Myoelectric pattern recognition systems suffer from increase in error rate over time, which could lead to unwanted behavior. This so-called concept drift in myoelectric control systems could be caused by fatigue, sensor replacement and varying skin conditions. To circumvent concept drift, adaptation strategies could be used to retrain a pattern recognition system, which could lead to comparable error rates over multiple days. In chapter 5 we investigated whether concept drift is an issue in lower limb pattern recognition and we compared three adaptation approaches, entropy-based, based on backward prediction and a combination of the two to circumvent this concept drift using the MyPredict 3 data set. We concluded that an adaptation strategy is necessary as the baseline error rate significantly increased from day 1 to day 2. Although none of the adaptation strategies was as good as using ground truth labels for retraining, the entropy-based adaptation worked best overall. However, backward prediction might be beneficial to use in stair climbing scenarios.
Direct control using surface electromyography enables more intuitive control of a transfemoral prosthesis. However, EMG is a noisy signal which can vary over time, giving rise to the question what approach for knee torque estimation is most suitable for multi-day control. In chapter 6 we compared three different modelling frameworks to estimate knee torque in non-weight-bearing situations using MyPredict 3. The first model contained a convolutional neural network which mapped EMG to knee torque directly. The second used a neuromusculoskeletal model which used EMG, muscle tendon unit lengths and moment arms to compute knee torque. The third model (Hybrid) used a CNN to map EMG to specific muscle activation, which was used together with NMS components to compute knee torque. CNN had the best performance in general and on each day. The Hybrid model was able to outperform NMS. The NMS model showed no significant difference between measurement days. The CNN model and Hybrid models had significant performance differences between the first day and all other days. CNNs are suited for multi-day torque estimation in terms of error rate, outperforming the other two model types. NMS was the only model type which was robust over all days.
Finally, in chapter 7 the main findings of this thesis are discussed, as well as the limitations and future recommendations. Summarizing, many data-driven approaches are suitable to increase the performance of intent recognition systems compared to state-of-the-art. However, the amount of data and standardization of evaluation protocols remains a critical point of attention moving forward. Data-driven intent recognition has the potential to make prostheses more intuitive, thereby possibly leading to better prosthesis control.