Skip to main content

Research Repository

Advanced Search

FedSim: similarity guided model aggregation for federated learning.

Palihawadana, Chamath; Wiratunga, Nirmalie; Wijekoon, Anjana; Kalutarage, Harsha

Authors

Harsha Kalutarage



Abstract

Federated Learning (FL) is a distributed machine learning approach in which clients contribute to learning a global model in a privacy preserved manner. Effective aggregation of client models is essential to create a generalised global model. To what extent a client is generalisable and contributing to this aggregation can be ascertained by analysing inter-client relationships. We use similarity between clients to model such relationships. We explore how similarity knowledge can be inferred from comparing client gradients, instead of inferring similarity on the basis of client data which violates the privacy-preserving constraint in FL. The similarity-guided FedSim algorithm, introduced in this paper, decomposes FL aggregation into local and global steps. Clients with similar gradients are clustered to provide local aggregations, which thereafter can be globally aggregated to ensure better coverage whilst reducing variance. Our comparative study also investigates the applicability of FedSim in both real-world datasets and on synthetic datasets where statistical heterogeneity can be controlled and studied systematically. A comparative study of FedSim with state-of-the-art FL baselines, FedAvg and FedProx, clearly shows significant performance gains. Our findings confirm that by exploiting latent inter-client similarities, FedSim’s performance is significantly better and more stable compared to both these baselines.

Citation

PALIHAWADANA, C., WIRATUNGA, N., WIJEKOON, A. and KALUTARAGE, H. [2021]. FedSim: similarity guided model aggregation for federated learning. Neurocomputing [online], special issue: distributed machine learning, optimization and applications, (accepted).

Journal Article Type Article
Acceptance Date Sep 17, 2021
Deposit Date Sep 17, 2021
Journal Neurocomputing
Print ISSN 0925-2312
Publisher Elsevier
Peer Reviewed Peer Reviewed
Keywords Federated learning; Model aggregation; Similarity; Clustering
Public URL https://rgu-repository.worktribe.com/output/1457270