Assessment of hand kinematics and interactions with the environment
Henk is PhD-student in the research group Biomedical Signals and Systems. His supervisor is Peter Veltink from the faculty Faculty of Electrical Engineering, Mathematics and Computer Science.
The hand is one of the most important instruments of our body. Its versatility enables the execution of a wide range of tasks that ask for a powerful, precise or gentle approach. Measuring hand and finger movements, and interaction forces, is therefore important for the assessment of tasks in daily life. However, measuring on-body kinematic and kinetic quantities is a delicate procedure due to the dexterity of the hand, and moreover, the little and complex shaped skin places for sensor attachment. This thesis proposes a new on-body assessment system that allows the measurement of movements and interaction forces of the hand, fingers and thumb.
The first objective, the development, evaluation and validation of an inertial and magnetic sensing system for the measurement of hand and finger kinematics is the topic of chapters 2 to 5. The second objective, assessment of the dynamic interaction between human hand and environment using combined force and movement sensing, is the topic of chapter 6.
Chapter 2 describes the hardware and algorithms for a sensing system which can be attached to the hand, fingers and the thumb. The hardware consists of multiple inertial and magnetic sensors to measure angular velocities, accelerations and the magnetic field. Each individual finger and the thumb is modelled as a kinematic chain where the bones correspond to the linkages and each joint is considered as an ideal ball-socket joint. Segmental lengths were determined by manual measurement, whereas the inertial sensors provided the input for a Kalman filter to estimate the 3D orientation of the corresponding segment. Hereafter, the orientation and tip position of each finger was estimated by applying forward kinematics. To our knowledge, it is the first system that uses inertial sensors for estimating finger kinematics. The estimation quality was expressed in terms of static and dynamic accuracy, dynamic range and repeatability. Differences with an active optical reference system were found to be a maximum of 13 mm for the finger tip distance difference during circular pointing movements. A standardized test protocol for instrumented gloves showed very good repeatability results compared to other datagloves, proven by the mean angle difference of < 2 degrees. Finally, a dynamic range was specified as a measure of how well the system is able to reconstruct joint angles when experiencing large angular velocities. The system showed accurate reconstruction up to 116 full index finger flex- extension movements per minute.
Chapter 3 reports an extensive comparison of our inertial sensing system against a passive optoelectronic marker system. It aims on typical hand-function tasks, including tapping, (fast) finger flexion, hand opening/closing, ab- adduction and circular pointing, which are used to quantify various motor symptoms for clinical diagnosis. Three subjects were included and instrumented with both systems. Differences in position, Range of Motion (RoM) and 3D joint angles were noted of which the largest were found in fast and circular pointing tasks (between 3.3 deg and 8.4 deg). The differences between both measurement systems were attributed to three sources: optical marker movements, inertial sensor range and the anatomical calibration. First, despite ad- equate fastenings, relative marker displacements up to 8.4 mm were found during fast movements of rigid segments, indicating a limitation of the opto-electronic system. This relative displacement can result in segment orientation errors of 10 deg for typical adult finger dimensions. Secondly, a consistency investigation of the inertial sensor system revealed that the angular velocities estimated by the sensor fusion algorithm, taking the biomechanical model into account, were different compared to the angular velocities measured by the rate gyroscopes. Largest difference were found in fast tasks and pointing tasks which could be explained by either skin artefacts or sensor drift effects. Latter is possible when the filter cannot rely on the accelerometer inclination updates because the inertial accelerations, especially at the very distal ends, are too large or when rotations take place about a joint axis directed parallel to the global vertical. Thirdly, the anatomical calibration is of utmost importance for both the assessment of 3D joint angles as well as for a proper determination of the forward kinematics. Unfortunately, the anatomical calibration of both systems was not based on the same measurement set due to marker visibility issues during the inertial sensor hardware calibration procedure. Although the same helical axis definition had been used, the performance of both procedures could have large effects on the calibration quality. Chapter 3 concludes that the inertial measurement hardware can be used in a clinical setting but requires awareness of its limitations.
Chapter 4 describes a new method to ease the typical anatomical segment and sensor calibration procedures by estimating these parameters implicitly along with the estimation of the state variables. An optimization approach was presented by a set of stochastic equations for the description of inertial sensor readings, as well as, the kinematic relations applicable for the hand and fingers. Next, a general objective function was formulated and subsequently used to solve for different calibration parameters. These parameters include the sensor biases, the pose of sensor modules with respect to the segment to which it has been attached to, and the lengths of the proximal and medial segments. The method aims for simplifying the calibration procedure by estimating these parameters from simple voluntary hand movements. Traditional orientation estimators use the magnetometer for a drift free heading estimate, which is valid for a homogenous magnetic field, but could result in large deteriorated orientation estimates if the field is disturbed. Our approach estimates the relative poses solely using inertial sensors and is therefore invulnerable for hazardous magnetic environments. Different experiments were performed using similar hardware as described in chapters 2 and 3. The results demonstrate the potential of the approach taken as the estimation error of various parameter values were within 1 percent.
Chapter 5 presents a solution to estimate the full pose (3D position and 3D orientation) of the hand with respect to the sternum of the body using inertial sensors, magnetometers and a permanent magnet. Contrary to the previous chapter, magnetometers are used but not for estimating the heading from the earth magnetic field. We inferred the position of a permanent neodymium magnet by associating the magnetometer output to the static field induced by the magnet, which are in close vicinity to each other. The magnetic field strength, which is proportional to the dimensions of the magnet, is chosen such that magnetometers were able to pick up the field at distances up to 30 cm away from the permanent magnet. The human body is permeable for magnetic fields, which is very beneficial for measuring the kinematics of articulated structures, such as the arm, the hand and fingers. Furthermore, the use of a permanent magnet instead of an electromagnet provides the freedom of attaching it to small and poorly accessible spots as no external interfacing or powering is required. Experiments were performed by instrumenting the trunk with Inertial Measurement Units (IMUs) and magnetometers and attaching an IMU and a permanent magnet to the subject’s hand. A complex task in which simultaneous movements of both trunk and hand was performed, resulted in an average Root Mean Square (RMS) position difference of 19.4 ± 2.2 mm with respect to an optical reference system, whereas the relative trunk-hand and global trunk orientation error was 2.3 ± 0.9 and 8.6 ± 8.7 deg respectively.
Chapter 6 concerns the second research objective which is about the assessment of the physical interaction between the human hand and environmental objects. Dedicated sensors have been applied to measure 3D interaction forces for biomedical purposes. This hardware has been combined with the inertial hardware presented in the previous chapters and attached to the finger and thumb tips to measure interaction forces and finger motions simultaneously. The system is a first attempt to quantify the interactions of the hand with the environment without instrumenting the environment itself. A specific condition has been investigated in which the subject applied forces to different passive environmental objects and manipulated and moved these objects at the same time. The force and motion measurements enabled the estimation of the most dominant object characteristics. Experiments were conducted in which the weight of two mass like objects and the stiffness of a spring like object were estimated with an accuracy of 19.7 ± 10.6% and 29.3 ± 18.9% for a small (0.28 kg) and larger weight (0.44 kg) respectively, and 14.8 ± 9.6% for the spring object.
