Towards Opportunistic Federated Learning Using Independent Subnetwork Training

Abstract

Enabling federated learning in opportunistic networks unlocks the potential for machine learning in challenging environments like disaster zones and remote regions. However, the divergent models induced by dynamic node encounters, combined with complete parameter overlap in model-homogeneous training lead to catastrophic interference, which disrupts training progress. Furthermore, when whole models must be transmitted, nodes with shorter contact duration are limited from participating in the training process. To address these challenges, we propose a different approach for training neural networks in opportunistic settings that leverages independent subnetworks and sequential training. We partition the original neural network into non-overlapping subnetworks and assign each to a unique node. These subnetworks are then trained and exchanged repeatedly during node encounters, exposing them further to diverse datasets. As a consequence, we achieve parallel and conflict-free progress while minimizing participation costs. Our experiments demonstrate that continuous training and subnetwork accumulation foster the development of a more robust model. Moreover, by utilizing pre-trained backbones as feature extractors, we achieve a test accuracy of 75.06% on MEDIC's disaster damage severity assessment task, demonstrating that the approach can be adopted in resource-constrained and dynamic scenarios in the real world.

Links

BibTeX

code:references.bib

@inproceedings{bib:victor_ParticipatoryFL_SmartComp2024,

author={Romero II, Victor and Matsui, Tomokazu and Matsuda, Yuki and Suwa, Hirohiko and Yasumoto, Keiichi},

title={Towards Opportunistic Federated Learning Using Independent Subnetwork Training},

booktitle={The 10th International Conference on Smart Computing (SmartComp '24)},

pages={174--181},

year={2024},

doi={10.1109/SMARTCOMP61445.2024.00044},

}

https://scrapbox.io/files/68cb5dae455a9078552ef651.png

Category

Conference

Keywords

Collaborating Organization