UTFacultiesETEventsPhD Defence Hao Chen | Automatic segmentation and motion analysis of the knee joint based on MRI and 4DCT images

PhD Defence Hao Chen | Automatic segmentation and motion analysis of the knee joint based on MRI and 4DCT images

automatic segmentation and motion analysis of the knee joint based on mri and 4dct images

Hao Chen is a PhD student in the research group Biomechanical Engineering. His supervisors are prof.dr.ir. N.J.J. Verdonschot from the Faculty of Engineering Technology and prof.dr. Y. Kang from the Sino-Dutch Biomedical and Information Engineering School, Northeastern University, China.

Lower limbs disorders, such as osteoarthritis (OA), knee bursitis, meniscal lesions, are musculoskeletal diseases that affect the hips, knees and legs, and usually lead to chronic pain conditions and limited mobility. Knee joint pains can interfere with daily activities, from sports to climbing stairs to walking. In this sense, a relatively minor knee defect can have a great impact on everyday life and create negative conditions for further health deterioration. In the past decades, rapid advances in medical imaging and increased understanding of musculoskeletal mechanics, and as a result diagnosis and effective treatment of knee joint disorders have improved substantially. Nevertheless, the available tools for researchers, radiologists and orthopaedic surgeons to assess the biomechanical condition of patients still lack in delivering a quantitative analysis of muscle and ligament ruptures, cartilage deterioration and patella misalignment, forcing the clinicians to rely mostly on their own subjective judgement which can lead sub-optimal care. One of the main bottlenecks in delivering quantitative digital models of the knee joint is the fast and accurate segmentation of the various knee structures from medical images. Manual segmentation is intrinsically subjective and time-consuming despite the developments in software, and accurate segmentation requires both clinical and user interface experience. As a part of the ERC BioMechTools project, the work described in this thesis is dedicated to advancing existing methods and developing new methods for robust, accurate and maximally automated segmentation and clinical measure determination relevant to pathology assessment in the knee joint.

One of the strengths of MRI is its capability to render different contrast of human tissue. To bridge the gap between clinical MRI protocols developed for fast, reliable acquisition and qualitative assessment and requirements of finite element research, segmentation frameworks for two clinical MRI protocols were developed: Proton Density Weighted and fat suppressed T1/T2 Weighted MRI. Chapter 2 describes a non-supervised 3D local intensity clustering based level set method to determine the bone structure of the knee from PDW weighted MRI.  The algorithm first determines the trabecular bone areas using 3D local intensity clustering based level set method, together with correcting any slow varying inhomogeneity in MRI image and then estimates the thin cortical bone boundary with sub-pixel accuracy from intensity profiles along normal vectors originating from the trabecular bone shape. An average dice score of 0.96 was achieved in this study, together with segmentation time of 250 seconds.

In fat suppressed T1/T2 weighted MRI images of the knee, the trabecular bone is translucent, requiring a different approach. In Chapter 3 a deep neural network was employed to extract the bone structures using the public dataset, made available with the grand challenge SKI10 (http://www.ski10.org/). The extensive comparisons of different networks showed that a simple U-Net architecture with fined tuned hyper-parameters can achieve state-of-the-art results, but still required improvement on specific pathological data sets deviation from the training data. A resampling strategy and the introduction of adversarial losses further improved the bone segmentation by enlarging the contextual information of the network and ensuring the shape consistency. An average dice score of 0.98 and 0.88 for bone and cartilage segmentation respectively was achieved by using the proposed deep learning method, in an inference time of around 60 s, using a Nvidia 1080 Ti graphical card. 

Despite the considerable improvement in magnetic resonance imaging techniques of the lower extremity over the last decades, computed tomography (CT) remains an often-preferred image modality for knee joint related disease. Although the bone geometry can be clearly identified by the CT image, cartilage is hardly visible, which prevents providing the diagnostic potential of cartilage status and construction of 3D knee joint models from CT images. In Chapter 4, an alternative method to predict the status of cartilage degeneration based on the bone geometry only using a posterior shape model was proposed. The obtained accuracy of 0.64 mm for femur and 0.58 mm for tibia outperformed the average cartilage thickness distribution method in the tibiofemoral contact area. This provides both clinicians and modelling researchers with more information on the influence of cartilage status on knee joint kinematics, giving the clinician more basis to assess the need for further testing (MRI) and modelling researchers a spatially more resolved distribution of cartilage thickness to optimize their models.

Another focus of this thesis was to automate the knee joint motion analysis, generating clinically related parameters from CT scans. From the various biomechanical parameters described in literature, a fully automated workflow for extraction of the commonly used tibial tubercle – trochlear groove (TT-TG) distance was developed. First, a simulation sensitivity analysis was done on the influence of the alignment between the leg axis and the craniocaudal axis of the CT scanner on TT-TG determination. Chapter 5 showed how minor disagreement between the leg and craniocaudal axis of the CT scanner can have a significant impact on the TT-TG distance calculation (1 mm per degree) and suggested a necessity of pre-correction of the axial cross section before manual measurements.

Subsequently, since the extra works to compensate the deviation of the CT axial plane from the anatomical axial plane is not trivial, an automatic determination of the anatomical coordinate frame (Chapter 6) was designed to automate the correction of disagreement of the planes. Furthermore, according to the experiments in Chapter 5 and Chapter 6, the intra- and inter- observations of the key landmarks of TT-TG distance were significant to clinical measure outcomes, and the algorithm for the determination of these landmarks was designed. Overall, the proposed semi-automatic workflow provides more robust results and was able to successfully discriminate patients from healthy subjects in terms of previously described normal and pathological TT-TG distance. However, the workflow described in Chapter 6 still requires considerable manual input, hampering the application to larger patient groups and robustness of the method.

To reduce the required manual input, a deep learning based knee joint segmentation was developed for 3D and 4D CT of the knee joint (Chapter 7). A mean DSC for femur, tibia, and patella of

,

, and

, respectively was achieved. Meanwhile, the difference of TT-TG distance between the manual segmentation and automatic segmentation was relatively small (

 mm), compared with the manual intra-observation errors of

 mm and the manual inter-observation of

 mm. Overall, a fully automatic workflow is proposed in this thesis capable of accurate and robust assessment of knee joint functioning based on static and a novel dynamic TT-TG distance, validated on an in-house acquired data sets of 4 subjects. The total processing time for extraction of these measures from static and dynamic CT is around 30 minutes.  

In conclusion, as a part of the BioMechTools project, several novel image analysis techniques are presented in this thesis for quantitative analysis of the knee joint from MRI and CT images. The automatic segmentation of bone and cartilage from PDW and fat-suppressed T1/T2 weighted MRI advances their diagnostic merit and provides a starting point for subject-specific finite element models. Automatic captures of bone shapes and landmarks from static and dynamic CT provides the possibility of standardised assessment of kinematic knee joint functioning, increasing the diagnostic merit of the TT-TG distance. However, for successful implementation into the clinic, further efforts will have to be made to confirm the clinical findings in a larger patient group and a user-friendly GUI will have to be developed to make these methods available to clinical personnel in an easy to use and straightforward fashion.

 

Lower limbs disorders, such as osteoarthritis (OA), knee bursitis, meniscal lesions, are musculoskeletal diseases that affect the hips, knees and legs, and usually lead to chronic pain conditions and limited mobility. Knee joint pains can interfere with daily activities, from sports to climbing stairs to walking. In this sense, a relatively minor knee defect can have a great impact on everyday life and create negative conditions for further health deterioration. In the past decades, rapid advances in medical imaging and increased understanding of musculoskeletal mechanics, and as a result diagnosis and effective treatment of knee joint disorders have improved substantially. Nevertheless, the available tools for researchers, radiologists and orthopaedic surgeons to assess the biomechanical condition of patients still lack in delivering a quantitative analysis of muscle and ligament ruptures, cartilage deterioration and patella misalignment, forcing the clinicians to rely mostly on their own subjective judgement which can lead sub-optimal care. One of the main bottlenecks in delivering quantitative digital models of the knee joint is the fast and accurate segmentation of the various knee structures from medical images. Manual segmentation is intrinsically subjective and time-consuming despite the developments in software, and accurate segmentation requires both clinical and user interface experience. As a part of the ERC BioMechTools project, the work described in this thesis is dedicated to advancing existing methods and developing new methods for robust, accurate and maximally automated segmentation and clinical measure determination relevant to pathology assessment in the knee joint.

One of the strengths of MRI is its capability to render different contrast of human tissue. To bridge the gap between clinical MRI protocols developed for fast, reliable acquisition and qualitative assessment and requirements of finite element research, segmentation frameworks for two clinical MRI protocols were developed: Proton Density Weighted and fat suppressed T1/T2 Weighted MRI. Chapter 2 describes a non-supervised 3D local intensity clustering based level set method to determine the bone structure of the knee from PDW weighted MRI.  The algorithm first determines the trabecular bone areas using 3D local intensity clustering based level set method, together with correcting any slow varying inhomogeneity in MRI image and then estimates the thin cortical bone boundary with sub-pixel accuracy from intensity profiles along normal vectors originating from the trabecular bone shape. An average dice score of 0.96 was achieved in this study, together with segmentation time of 250 seconds.

In fat suppressed T1/T2 weighted MRI images of the knee, the trabecular bone is translucent, requiring a different approach. In Chapter 3 a deep neural network was employed to extract the bone structures using the public dataset, made available with the grand challenge SKI10 (http://www.ski10.org/). The extensive comparisons of different networks showed that a simple U-Net architecture with fined tuned hyper-parameters can achieve state-of-the-art results, but still required improvement on specific pathological data sets deviation from the training data. A resampling strategy and the introduction of adversarial losses further improved the bone segmentation by enlarging the contextual information of the network and ensuring the shape consistency. An average dice score of 0.98 and 0.88 for bone and cartilage segmentation respectively was achieved by using the proposed deep learning method, in an inference time of around 60 s, using a Nvidia 1080 Ti graphical card. 

Despite the considerable improvement in magnetic resonance imaging techniques of the lower extremity over the last decades, computed tomography (CT) remains an often-preferred image modality for knee joint related disease. Although the bone geometry can be clearly identified by the CT image, cartilage is hardly visible, which prevents providing the diagnostic potential of cartilage status and construction of 3D knee joint models from CT images. In Chapter 4, an alternative method to predict the status of cartilage degeneration based on the bone geometry only using a posterior shape model was proposed. The obtained accuracy of 0.64 mm for femur and 0.58 mm for tibia outperformed the average cartilage thickness distribution method in the tibiofemoral contact area. This provides both clinicians and modelling researchers with more information on the influence of cartilage status on knee joint kinematics, giving the clinician more basis to assess the need for further testing (MRI) and modelling researchers a spatially more resolved distribution of cartilage thickness to optimize their models.

Another focus of this thesis was to automate the knee joint motion analysis, generating clinically related parameters from CT scans. From the various biomechanical parameters described in literature, a fully automated workflow for extraction of the commonly used tibial tubercle – trochlear groove (TT-TG) distance was developed. First, a simulation sensitivity analysis was done on the influence of the alignment between the leg axis and the craniocaudal axis of the CT scanner on TT-TG determination. Chapter 5 showed how minor disagreement between the leg and craniocaudal axis of the CT scanner can have a significant impact on the TT-TG distance calculation (1 mm per degree) and suggested a necessity of pre-correction of the axial cross section before manual measurements.

Subsequently, since the extra works to compensate the deviation of the CT axial plane from the anatomical axial plane is not trivial, an automatic determination of the anatomical coordinate frame (Chapter 6) was designed to automate the correction of disagreement of the planes. Furthermore, according to the experiments in Chapter 5 and Chapter 6, the intra- and inter- observations of the key landmarks of TT-TG distance were significant to clinical measure outcomes, and the algorithm for the determination of these landmarks was designed. Overall, the proposed semi-automatic workflow provides more robust results and was able to successfully discriminate patients from healthy subjects in terms of previously described normal and pathological TT-TG distance. However, the workflow described in Chapter 6 still requires considerable manual input, hampering the application to larger patient groups and robustness of the method.

To reduce the required manual input, a deep learning based knee joint segmentation was developed for 3D and 4D CT of the knee joint (Chapter 7). A mean DSC for femur, tibia, and patella of

,

, and

, respectively was achieved. Meanwhile, the difference of TT-TG distance between the manual segmentation and automatic segmentation was relatively small (

 mm), compared with the manual intra-observation errors of

 mm and the manual inter-observation of

 mm. Overall, a fully automatic workflow is proposed in this thesis capable of accurate and robust assessment of knee joint functioning based on static and a novel dynamic TT-TG distance, validated on an in-house acquired data sets of 4 subjects. The total processing time for extraction of these measures from static and dynamic CT is around 30 minutes.  

In conclusion, as a part of the BioMechTools project, several novel image analysis techniques are presented in this thesis for quantitative analysis of the knee joint from MRI and CT images. The automatic segmentation of bone and cartilage from PDW and fat-suppressed T1/T2 weighted MRI advances their diagnostic merit and provides a starting point for subject-specific finite element models. Automatic captures of bone shapes and landmarks from static and dynamic CT provides the possibility of standardised assessment of kinematic knee joint functioning, increasing the diagnostic merit of the TT-TG distance. However, for successful implementation into the clinic, further efforts will have to be made to confirm the clinical findings in a larger patient group and a user-friendly GUI will have to be developed to make these methods available to clinical personnel in an easy to use and straightforward fashion.

 

Lower limbs disorders, such as osteoarthritis (OA), knee bursitis, meniscal lesions, are musculoskeletal diseases that affect the hips, knees and legs, and usually lead to chronic pain conditions and limited mobility. Knee joint pains can interfere with daily activities, from sports to climbing stairs to walking. In this sense, a relatively minor knee defect can have a great impact on everyday life and create negative conditions for further health deterioration. In the past decades, rapid advances in medical imaging and increased understanding of musculoskeletal mechanics, and as a result diagnosis and effective treatment of knee joint disorders have improved substantially. Nevertheless, the available tools for researchers, radiologists and orthopaedic surgeons to assess the biomechanical condition of patients still lack in delivering a quantitative analysis of muscle and ligament ruptures, cartilage deterioration and patella misalignment, forcing the clinicians to rely mostly on their own subjective judgement which can lead sub-optimal care. One of the main bottlenecks in delivering quantitative digital models of the knee joint is the fast and accurate segmentation of the various knee structures from medical images. Manual segmentation is intrinsically subjective and time-consuming despite the developments in software, and accurate segmentation requires both clinical and user interface experience. As a part of the ERC BioMechTools project, the work described in this thesis is dedicated to advancing existing methods and developing new methods for robust, accurate and maximally automated segmentation and clinical measure determination relevant to pathology assessment in the knee joint.

One of the strengths of MRI is its capability to render different contrast of human tissue. To bridge the gap between clinical MRI protocols developed for fast, reliable acquisition and qualitative assessment and requirements of finite element research, segmentation frameworks for two clinical MRI protocols were developed: Proton Density Weighted and fat suppressed T1/T2 Weighted MRI. Chapter 2 describes a non-supervised 3D local intensity clustering based level set method to determine the bone structure of the knee from PDW weighted MRI.  The algorithm first determines the trabecular bone areas using 3D local intensity clustering based level set method, together with correcting any slow varying inhomogeneity in MRI image and then estimates the thin cortical bone boundary with sub-pixel accuracy from intensity profiles along normal vectors originating from the trabecular bone shape. An average dice score of 0.96 was achieved in this study, together with segmentation time of 250 seconds.

In fat suppressed T1/T2 weighted MRI images of the knee, the trabecular bone is translucent, requiring a different approach. In Chapter 3 a deep neural network was employed to extract the bone structures using the public dataset, made available with the grand challenge SKI10 (http://www.ski10.org/). The extensive comparisons of different networks showed that a simple U-Net architecture with fined tuned hyper-parameters can achieve state-of-the-art results, but still required improvement on specific pathological data sets deviation from the training data. A resampling strategy and the introduction of adversarial losses further improved the bone segmentation by enlarging the contextual information of the network and ensuring the shape consistency. An average dice score of 0.98 and 0.88 for bone and cartilage segmentation respectively was achieved by using the proposed deep learning method, in an inference time of around 60 s, using a Nvidia 1080 Ti graphical card. 

Despite the considerable improvement in magnetic resonance imaging techniques of the lower extremity over the last decades, computed tomography (CT) remains an often-preferred image modality for knee joint related disease. Although the bone geometry can be clearly identified by the CT image, cartilage is hardly visible, which prevents providing the diagnostic potential of cartilage status and construction of 3D knee joint models from CT images. In Chapter 4, an alternative method to predict the status of cartilage degeneration based on the bone geometry only using a posterior shape model was proposed. The obtained accuracy of 0.64 mm for femur and 0.58 mm for tibia outperformed the average cartilage thickness distribution method in the tibiofemoral contact area. This provides both clinicians and modelling researchers with more information on the influence of cartilage status on knee joint kinematics, giving the clinician more basis to assess the need for further testing (MRI) and modelling researchers a spatially more resolved distribution of cartilage thickness to optimize their models.

Another focus of this thesis was to automate the knee joint motion analysis, generating clinically related parameters from CT scans. From the various biomechanical parameters described in literature, a fully automated workflow for extraction of the commonly used tibial tubercle – trochlear groove (TT-TG) distance was developed. First, a simulation sensitivity analysis was done on the influence of the alignment between the leg axis and the craniocaudal axis of the CT scanner on TT-TG determination. Chapter 5 showed how minor disagreement between the leg and craniocaudal axis of the CT scanner can have a significant impact on the TT-TG distance calculation (1 mm per degree) and suggested a necessity of pre-correction of the axial cross section before manual measurements.

Subsequently, since the extra works to compensate the deviation of the CT axial plane from the anatomical axial plane is not trivial, an automatic determination of the anatomical coordinate frame (Chapter 6) was designed to automate the correction of disagreement of the planes. Furthermore, according to the experiments in Chapter 5 and Chapter 6, the intra- and inter- observations of the key landmarks of TT-TG distance were significant to clinical measure outcomes, and the algorithm for the determination of these landmarks was designed. Overall, the proposed semi-automatic workflow provides more robust results and was able to successfully discriminate patients from healthy subjects in terms of previously described normal and pathological TT-TG distance. However, the workflow described in Chapter 6 still requires considerable manual input, hampering the application to larger patient groups and robustness of the method.

To reduce the required manual input, a deep learning based knee joint segmentation was developed for 3D and 4D CT of the knee joint (Chapter 7). A mean DSC for femur, tibia, and patella of

,

, and

, respectively was achieved. Meanwhile, the difference of TT-TG distance between the manual segmentation and automatic segmentation was relatively small (

 mm), compared with the manual intra-observation errors of

 mm and the manual inter-observation of

 mm. Overall, a fully automatic workflow is proposed in this thesis capable of accurate and robust assessment of knee joint functioning based on static and a novel dynamic TT-TG distance, validated on an in-house acquired data sets of 4 subjects. The total processing time for extraction of these measures from static and dynamic CT is around 30 minutes.  

In conclusion, as a part of the BioMechTools project, several novel image analysis techniques are presented in this thesis for quantitative analysis of the knee joint from MRI and CT images. The automatic segmentation of bone and cartilage from PDW and fat-suppressed T1/T2 weighted MRI advances their diagnostic merit and provides a starting point for subject-specific finite element models. Automatic captures of bone shapes and landmarks from static and dynamic CT provides the possibility of standardised assessment of kinematic knee joint functioning, increasing the diagnostic merit of the TT-TG distance. However, for successful implementation into the clinic, further efforts will have to be made to confirm the clinical findings in a larger patient group and a user-friendly GUI will have to be developed to make these methods available to clinical personnel in an easy to use and straightforward fashion.