wandb: [Bug]: wandb.login() hangs indefinitely under certain conditions
Describe the bug
Problem: It seems that wandb.login() sometimes hangs forever. I’ve observed it happen under these conditions (not comprehensive):
- Multiple independent processes are simultaneously trying to use wandb.login() with the same key & project.
- Or when I’m training hundreds of models sequentially, it is likely to hang on one of them & obstruct the whole process.
Expected Behavior: Handle login requests successfully, and if that is not possible due to limitations of the plan (idk if this is true) then simply raise an error.
My Setup:
- wandb version = 0.19.3
- OS = Linux
- Plan: Free
P.S. wandb.login(timeout) arg says: “timeout | (int, optional) Number of seconds to wait for user input.”
I don’t know what this means, but I’d like to know if this timeout also applies to waiting for the server to respond, so that a workaround could be wandb.login(key=key, timeout=X)?
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 1
- Comments: 16 (7 by maintainers)
Hey I’ve got another update, it turns out that %autoreload indeed can trigger a W&B hang inside a jupyter notebook. This might not represent the entire scope of the bug but I thought I’d provide the traceback to aid debugging:
auto_reload_WB_traceback.txt