FedML: FedAvg accuracy stucks under 50
I am training Fedavg to get the benchmark accuracy with the given parameters. But, the accuracy is stuck under 50.
Here is all my code:
!git clone https://github.com/FedML-AI/FedML
cd /content/FedML/fedml_experiments/standalone/fedavg
!python main_fedavg.py --model mobilenet --dataset cifar10 --data_dir ./../../../data/cifar10 --partition_method hetero --comm_round 100 --epochs 20 --batch_size 64 --lr 0.001
I suppose to get over 80% accuracy at least according to these benchmark results.
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 39 (17 by maintainers)
@hangxu0304 @AbdulMoqeet Hi, I find some possible reasons for this bug: Please check these codes: Original version: https://github.com/FedML-AI/FedML/blob/50d8a45d27675343a7b05a9b31279f6764d3f2ad/fedml_api/standalone/fedavg/fedavg_trainer.py#L45
Current version: https://github.com/FedML-AI/FedML/blob/8ccc24cf2c01b868988f5d5bd65f1666cf5526bc/fedml_api/standalone/fedavg/fedavg_api.py#L64
In the original version, the global model is deepcopied and loaded into clients. However, in the current version, the local client just load the global model (without deepcopy). So maybe the local training in every client will update the global model ?
I’m not completely sure that the bug is cased by this. Could you please change the current codes (make a deepcopy of global model) and to see if the result is correct?