Introduction to federated learning systems

In this chapter, we introduce federated learning from a systems perspective. We go into the details of the different federated learning scenarios that have different system design considerations. We first introduce two most common but quite different federated learning scenarios, namely cross-device federated learning and cross-silo federated learning. Cross-device federated learning typically involves a significant number of parties (e.g., thousands to millions), who are usually less reliable and equipped with mobile or IoT devices that have various computing and communication capabilities. In cross-silo federated learning, the parties are usually a small number of organizations with ample computing power and reliable communications. We first describe the two very different problems that each of them address. We then describe the architectural differences between the two and their corresponding training steps. We also discuss the unique systems challenges that arise due to these properties and give a brief description of current works that have talked about these problems in detail.