Mr Chamath Palihawadana c.palihawadana@rgu.ac.uk
Research Assistant
FedSim: similarity guided model aggregation for federated learning.
Palihawadana, Chamath; Wiratunga, Nirmalie; Wijekoon, Anjana; Kalutarage, Harsha
Authors
Professor Nirmalie Wiratunga n.wiratunga@rgu.ac.uk
Associate Dean for Research
Anjana Wijekoon
Dr Harsha Kalutarage h.kalutarage@rgu.ac.uk
Associate Professor
Abstract
Federated Learning (FL) is a distributed machine learning approach in which clients contribute to learning a global model in a privacy preserved manner. Effective aggregation of client models is essential to create a generalised global model. To what extent a client is generalisable and contributing to this aggregation can be ascertained by analysing inter-client relationships. We use similarity between clients to model such relationships. We explore how similarity knowledge can be inferred from comparing client gradients, instead of inferring similarity on the basis of client data which violates the privacy-preserving constraint in FL. The similarity-guided FedSim algorithm, introduced in this paper, decomposes FL aggregation into local and global steps. Clients with similar gradients are clustered to provide local aggregations, which thereafter can be globally aggregated to ensure better coverage whilst reducing variance. Our comparative study also investigates the applicability of FedSim in both real-world datasets and on synthetic datasets where statistical heterogeneity can be controlled and studied systematically. A comparative study of FedSim with state-of-the-art FL baselines, FedAvg and FedProx, clearly shows significant performance gains. Our findings confirm that by exploiting latent inter-client similarities, FedSim’s performance is significantly better and more stable compared to both these baselines.
Citation
PALIHAWADANA, C., WIRATUNGA, N., WIJEKOON, A. and KALUTARAGE, H. 2022. FedSim: similarity guided model aggregation for federated learning. Neurocomputing [online], 483: distributed machine learning, optimization and applications, pages 432-445. Available from: https://doi.org/10.1016/j.neucom.2021.08.141
Journal Article Type | Article |
---|---|
Acceptance Date | Aug 30, 2021 |
Online Publication Date | Nov 2, 2021 |
Publication Date | Apr 28, 2022 |
Deposit Date | Sep 17, 2021 |
Publicly Available Date | Nov 3, 2022 |
Journal | Neurocomputing |
Print ISSN | 0925-2312 |
Electronic ISSN | 1872-8286 |
Publisher | Elsevier |
Peer Reviewed | Peer Reviewed |
Volume | 483 |
Pages | 432-445 |
DOI | https://doi.org/10.1016/j.neucom.2021.08.141 |
Keywords | Federated learning; Model aggregation; Similarity; Clustering |
Public URL | https://rgu-repository.worktribe.com/output/1457270 |
Files
PALIHAWADANA 2022 FedSim (AAM)
(1 Mb)
PDF
Publisher Licence URL
https://creativecommons.org/licenses/by-nc-nd/4.0/
You might also like
Clood CBR: towards microservices oriented case-based reasoning.
(2020)
Presentation / Conference Contribution
Counterfactual explanations for student outcome prediction with Moodle footprints.
(2021)
Presentation / Conference Contribution
DisCERN: discovering counterfactual explanations using relevance features from neighbourhoods.
(2021)
Presentation / Conference Contribution
Actionable feature discovery in counterfactuals using feature relevance explainers.
(2021)
Presentation / Conference Contribution
CBR driven interactive explainable AI.
(2023)
Presentation / Conference Contribution
Downloadable Citations
About OpenAIR@RGU
Administrator e-mail: publications@rgu.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search