wandb: [Bug]: wandb.login() hangs indefinitely under certain conditions

Describe the bug

Problem: It seems that wandb.login() sometimes hangs forever. I’ve observed it happen under these conditions (not comprehensive):

  • Multiple independent processes are simultaneously trying to use wandb.login() with the same key & project.
  • Or when I’m training hundreds of models sequentially, it is likely to hang on one of them & obstruct the whole process.

Expected Behavior: Handle login requests successfully, and if that is not possible due to limitations of the plan (idk if this is true) then simply raise an error.

My Setup:

  • wandb version = 0.19.3
  • OS = Linux
  • Plan: Free

P.S. wandb.login(timeout) arg says: “timeout | (int, optional) Number of seconds to wait for user input.” I don’t know what this means, but I’d like to know if this timeout also applies to waiting for the server to respond, so that a workaround could be wandb.login(key=key, timeout=X)?

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 1
  • Comments: 16 (7 by maintainers)

Most upvoted comments

Hey I’ve got another update, it turns out that %autoreload indeed can trigger a W&B hang inside a jupyter notebook. This might not represent the entire scope of the bug but I thought I’d provide the traceback to aid debugging:

auto_reload_WB_traceback.txt