UTFacultiesEEMCSEventsPhD Defence Zhicheng Yang | Hand-finger pose tracking using inertial and magnetic sensors

PhD Defence Zhicheng Yang | Hand-finger pose tracking using inertial and magnetic sensors

Hand-finger pose tracking using inertial and magnetic sensors

Due to the COVID-19 crisis the PhD defence of Zhicheng Yang will take place (partly) online.

The PhD defence can be followed by a live stream.

Zhicheng Yang is a PhD student in the research group Biomedical Signals and Systems (BSS). His supervisors are prof.dr.ir. P.H. Veltink from the Faculty of Electrical Engineering, Mathematics and Computer Science (EEMCS) and prof.dr. Y. Shenggang from the NorthWestern Polytechnical University (NPU), China.

Hand-finger motion tracking during daily life is often used in rehabilitation for diagnostic and human-computer interaction etc. Traditionally, optical tracking systems (OTSs) are used. However, these systems are restricted to lab environments with expensive cameras and data acquisition systems.  Ambulatory tracking out of a lab, using inertial sensors and magnetometers, is becoming increasingly popular to obtain insight in daily life.

There are two main disadvantages of the inertial and magnetometer-based tracking system. Firstly, the most existing methods are limited by high number of sensors, thus not satisfying the requirement of minimum obtrusiveness. Secondly, most existing methods require an accurate kinematic hand and finger model. From these two disadvantages, the goal of this thesis was derived: developing a minimally obtrusive inertial and magnetic sensing system that can be used in an ambulatory setting without kinematic information. The thesis was a joint work between Northwestern Polytechnical University (NPU) and University of Twente (UT), which has two parts. The first part was mainly about calibration of magnetometers, which provides basis for the follow-up work related to the localization with a magnetometer. This part was finished at NPU.  The second part was to estimate the fingertip pose with few inertial sensors, magnetometers and a magnet, which was finished at UT.

The first part is addressed in Chapter 2 and 3. Chapter 2 presents a hybrid calibration method for the gradiometer or magnetometer array with more than two magnetometers. The first magnetometer was calibrated with `scalar calibration' method, then errors from the magnetometer itself and misalignment error between magnetometers were calibrated together with linear method. The calibration efficiency can be greatly improved when the number of magnetometers is large. Chapter 3 presents a calibration method when the magnetometer is equipped on a carrier and its movement is restricted in a small range. When the magnetometer is restricted in a small range, the calibration data is insufficient and traditional calibration methods will fail. We exploited an improved truncated singular value decomposition method to obtain the error parameters and solved the divergent problem. The deviation of magnetometer output norm reduced a lot after calibration, which was verified when a magnetometer was fixed on a car.

The second part is addressed in Chapter 4, 5, 6 and 7. Chapter 4 presents a method that improves the OTS-based orientation estimation performance by fusing gyroscope information, providing a better orientation reference for following sections. The disadvantages of OTS-based orientation estimation are line-of-sight and wrong identification of mark problems.  Moreover, when the tracking object is small, such as a fingertip, the orientation error can be large. The excellent dynamic performance of the gyroscope improves the orientation accuracy. The OTS-based orientation error was reduced from  0.39°±0.16° to 0.23°±0.02° ,  when the distance between marker was 13mm. Besides, the proposed method filled the orientation data during “line of sight” period and corrected the orientation estimates when OTS markers were wrongly identified. Chapter 5 presents a method to estimate fingertip orientation relative to the hand only with inertial sensors.  “designated event” (when the hand moves as whole object) was used to compensate the drift. During the “designated event”, the dorsal side of the hand and fingertips share approximate angular velocity and acceleration. The results showed, the orientation error was smaller than 10 degrees when the “designated event” was partially available in a functional water-drinking task. Chapter 6 presented the position estimation of fingertips relative to the hand, with one magnetometer on the fingertip and one magnet on the dorsal side of the hand. We made assumptions that geomagnetic field is a disturbance compared with the magnet and the finger orientation relative to the hand is known. The finger position relative to the hand was estimated with Levenberg-Marquardt method. The experiment based on action research arm test resulted in median distance error between thumb and index finger of 9.6%. Chapter 7 presents the results that fused method in chapter 5 and 6. In chapter 7, the fingertip orientation was obtained based on inertial sensors with the method from chapter 5, rather than the orientation from the OTS (used in Chapter 6). Compared with the orientation from the OTS, the orientation from the inertial sensors may contained larger errors. The experiment results show: For whole hand rotation and functional grasping or writing experiments, the estimated errors of index fingertip position and orientation relative to the hand were 8.0 ~ 9.8 mm and 5.7°~11.27°.