wandb: runs in loop don't finish

Weights and Biases version: 0.8.36
Python version: 3.7
Operating System: Ubuntu 16.04.6

The problem is if I run many runs in a loop, every runs will last to the end of program but not the beginning of its next run.

according to the advice of #951, I tried

import time
for _ in range(2):
    run = wandb.init(reinit=True)
    with run:
        run.log({'metric':10})
    time.sleep(10)

But still I got this

Question

How can I make the run finished when init a new run ? (so it can record correct exec time)
Does this mean if I run 15 run in a loop, I will get 15 threads exist at the same time ?

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 15 (6 by maintainers)

Most upvoted comments

We’ll follow up here when it’s fixed.

vanpelt on Jun 9, 2020

The preferred syntax for the loop would be:

import wandb
for x in range(10):
    run = wandb.init(project="runs-from-for-loop", reinit=True)
    for y in range (100):
        run.log({"metric": x+y})
    run.join()

or:

for x in range(10):
    run = wandb.init(project="runs-from-for-loop", reinit=True)
    with run:
        for y in range (100):
            run.log({"metric": x+y})

raubitsj on Sep 1, 2020

@raubitsj Thanks a tonne for the update - just confirmed that it is working.

onoderat on Sep 1, 2020

FYI: This is fixed in our pre-release (soon to be released) version: %pip install --upgrade --pre wandb

raubitsj on Sep 1, 2020