Systems bias in federated learning

Data parties typically vary significantly in data quality, hardware resources, and stability, which results in challenges such as increased training times, higher resource costs, sub-par model performance and biased training. They result in hard systems challenges, and many existing works tend to address each of these challenges in isolation. Specifically, the bias in hardware and data consequently causes biased models. Additional challenges are introduced when party dropouts are considered. While the "Stragglers Management" chapter focuses mostly on the impact of stragglers, this chapter focuses on the impact of biasness. We take a look at how factors such as device dropouts, biased device data, biased participation, etc. affect the FL process from a systems perspective. We present a characterization study that empirically demonstrates how these challenges together impact important performance metrics such as model error, fairness, cost, and training time, and why it is important to consider them together instead of in isolation. We then talk about a method called DCFair which is a framework that comprehensively considers the multiple aforementioned important challenges of practical FL systems. Discussions on the characterization study and possible solutions are useful to gain a much deeper understanding of the inter-dependency of systems properties in Federated Learning.