MASTERÂ Assignment
The effect of data imbalances on Membership Inference Attacks in Federated Learning
Type : Master M-CS
Period: May, 2024 - October, 2024
Student : Dartel, B. van (Bram, Student M-CS)
Date Final project: October 28, 2024
Supervisors:
Abstract:
During previous years, Federated Learning has gained the interest of many parties who collaboratively want to train a Machine Learning model over privacy-sensitive data, as sharing the underlying training dataset is not needed. Whilst privacy is assumed, as no data is being shared, it has been shown that the setup is susceptible to Membership Inference Attacks, resulting in a leakage of information about the underlying dataset. Previous research has focused on understanding the impact of different setups and parameters of federated learning on attack accuracy, whilst data imbalances between clients have received less attention. Therefore, this research expands the state of the art by studying the impact of data imbalances in label space and feature space on a membership inference attack in federated learning. We define two metrics to measure this degree of non-IIDness in label space and feature space and use this on synthetic data to get an overview of the impact of this imbalance on the attack accuracy of a membership inference attack. Furthermore, we use several real-world datasets on which we use different data splits over the clients to show the impact between balanced and imbalanced data in a real-world setting on the attack accuracy. This research shows that the attack accuracy within federated learning is significantly higher for balanced datasets over unbalanced datasets, while most current research benchmarks their attack on balanced datasets.