vscode-dvc: Experiments table not updating while experiments are running

The metrics in the Experiments table are not following the results of multiple parallel experiment runs even though dvc exp show does show the different updated metrics. The Experiments table mainly contains the original base commit metrics.

This issue may be because it seems that the extension does not call dvc exp show. Instead, I see lots of dvc stage list commands in the “DVC” output window.

[version: 1.0.43, 2023-08-18T00:34:42.565Z, pid: 36047] > dvc stage list - INITIALIZED
[version: 1.0.43, 2023-08-18T00:34:43.435Z, pid: 36047] > dvc stage list - COMPLETED (912ms)
[version: 1.0.43, 2023-08-18T00:34:43.473Z, pid: 36050] > dvc stage list - INITIALIZED
[version: 1.0.43, 2023-08-18T00:34:43.487Z, pid: 35221] > dvc data status --granular --unchanged --json - COMPLETED (141986ms)
[version: 1.0.43, 2023-08-18T00:34:47.993Z, pid: 36050] > dvc stage list - COMPLETED (4556ms)
[version: 1.0.43, 2023-08-18T00:34:48.026Z, pid: 36054] > dvc stage list - INITIALIZED
[version: 1.0.43, 2023-08-18T00:34:49.929Z, pid: 36054] > dvc stage list - COMPLETED (1935ms)
[version: 1.0.43, 2023-08-18T00:34:49.981Z, pid: 36073] > dvc stage list - INITIALIZED
[version: 1.0.43, 2023-08-18T00:34:50.667Z, pid: 36073] > dvc stage list - COMPLETED (737ms)
[version: 1.0.43, 2023-08-18T00:34:50.715Z, pid: 36076] > dvc stage list - INITIALIZED
[version: 1.0.43, 2023-08-18T00:34:51.405Z, pid: 36076] > dvc stage list - COMPLETED (736ms)
[version: 1.0.43, 2023-08-18T00:34:51.452Z, pid: 36079] > dvc stage list - INITIALIZED
[version: 1.0.43, 2023-08-18T00:34:52.142Z, pid: 36079] > dvc stage list - COMPLETED (735ms)
[version: 1.0.43, 2023-08-18T00:34:52.196Z, pid: 36082] > dvc stage list - INITIALIZED
[version: 1.0.43, 2023-08-18T00:34:52.857Z, pid: 36082] > dvc stage list - COMPLETED (713ms)
[version: 1.0.43, 2023-08-18T00:34:52.900Z, pid: 36085] > dvc stage list - INITIALIZED
[version: 1.0.43, 2023-08-18T00:34:53.456Z, pid: 36085] > dvc stage list - COMPLETED (597ms)
[version: 1.0.43, 2023-08-18T00:34:53.496Z, pid: 36091] > dvc stage list - INITIALIZED
[version: 1.0.43, 2023-08-18T00:34:54.151Z, pid: 36091] > dvc stage list - COMPLETED (694ms)
[version: 1.0.43, 2023-08-18T00:34:54.199Z, pid: 36108] > dvc stage list - INITIALIZED
[version: 1.0.43, 2023-08-18T00:34:54.906Z, pid: 36108] > dvc stage list - COMPLETED (754ms)
[version: 1.0.43, 2023-08-18T00:34:54.953Z, pid: 36111] > dvc stage list - INITIALIZED
[version: 1.0.43, 2023-08-18T00:34:55.649Z, pid: 36111] > dvc stage list - COMPLETED (742ms)
[version: 1.0.43, 2023-08-18T00:34:55.696Z, pid: 36114] > dvc stage list - INITIALIZED
[version: 1.0.43, 2023-08-18T00:34:56.421Z, pid: 36114] > dvc stage list - COMPLETED (770ms)
[version: 1.0.43, 2023-08-18T00:34:56.466Z, pid: 36117] > dvc stage list - INITIALIZED
[version: 1.0.43, 2023-08-18T00:34:57.171Z, pid: 36117] > dvc stage list - COMPLETED (749ms)
[version: 1.0.43, 2023-08-18T00:34:57.223Z, pid: 36120] > dvc stage list - INITIALIZED
[version: 1.0.43, 2023-08-18T00:34:57.986Z, pid: 36120] > dvc stage list - COMPLETED (814ms)
[version: 1.0.43, 2023-08-18T00:34:58.026Z, pid: 36123] > dvc stage list - INITIALIZED
[version: 1.0.43, 2023-08-18T00:34:58.730Z, pid: 36123] > dvc stage list - COMPLETED (744ms)
[version: 1.0.43, 2023-08-18T00:34:58.768Z, pid: 36129] > dvc stage list - INITIALIZED
[version: 1.0.43, 2023-08-18T00:34:59.465Z, pid: 36129] > dvc stage list - COMPLETED (735ms)
[version: 1.0.43, 2023-08-18T00:34:59.520Z, pid: 36146] > dvc stage list - INITIALIZED
[version: 1.0.43, 2023-08-18T00:35:00.265Z, pid: 36146] > dvc stage list - COMPLETED (799ms)

There are plenty more of these; some failed with a validation of the dvc.lock file, e.g.,

[version: 1.0.43, 2023-08-18T00:35:35.341Z, pid: 36431] > dvc stage list - INITIALIZED
[version: 1.0.43, 2023-08-18T00:35:36.371Z, pid: 36431] > dvc stage list - FAILED with code 255 (1063ms)
'./dvc.lock' validation failed: 13 errors.



extra keys not allowed, in create_index_table, line 2, column 3

    1 create_index_table:                                                       

    2   cmd: python -m scripts.create_index_table

...

The latest commands shown at the moment are:

[version: 1.0.43, 2023-08-18T00:38:57.834Z, pid: 37297] > dvc stage list - INITIALIZED
[version: 1.0.43, 2023-08-18T00:38:58.575Z, pid: 37297] > dvc stage list - COMPLETED (772ms)
[version: 1.0.43, 2023-08-18T00:38:58.606Z, pid: 37300] > dvc stage list - INITIALIZED
[version: 1.0.43, 2023-08-18T00:39:01.712Z, pid: 37300] > dvc stage list - COMPLETED (3136ms)
[version: 1.0.43, 2023-08-18T07:59:14.480Z, pid: 52404] > dvc data status --granular --unchanged --json - INITIALIZED
[version: 1.0.43, 2023-08-18T07:59:14.509Z, pid: 52405] > git ls-files --others --exclude-standard - INITIALIZED
[version: 1.0.43, 2023-08-18T07:59:14.541Z, pid: 52406] > git ls-files --others --exclude-standard --directory --no-empty-directory - INITIALIZED
[version: 1.0.43, 2023-08-18T07:59:14.560Z, pid: 52405] > git ls-files --others --exclude-standard - COMPLETED (79ms)
[version: 1.0.43, 2023-08-18T07:59:14.592Z, pid: 52406] > git ls-files --others --exclude-standard --directory --no-empty-directory - COMPLETED (82ms)
[version: 1.0.43, 2023-08-18T08:01:36.953Z, pid: 52404] > dvc data status --granular --unchanged --json - COMPLETED (142502ms)
[version: 1.0.43, 2023-08-18T08:01:41.251Z, pid: 52496] > dvc data status --granular --unchanged --json - INITIALIZED
[version: 1.0.43, 2023-08-18T08:01:41.311Z, pid: 52497] > git ls-files --others --exclude-standard - INITIALIZED
[version: 1.0.43, 2023-08-18T08:01:41.374Z, pid: 52498] > git ls-files --others --exclude-standard --directory --no-empty-directory - INITIALIZED
[version: 1.0.43, 2023-08-18T08:01:41.379Z, pid: 52497] > git ls-files --others --exclude-standard - COMPLETED (127ms)
[version: 1.0.43, 2023-08-18T08:01:41.428Z, pid: 52498] > git ls-files --others --exclude-standard --directory --no-empty-directory - COMPLETED (117ms)
[version: 1.0.43, 2023-08-18T08:03:58.170Z, pid: 52496] > dvc data status --granular --unchanged --json - COMPLETED (136975ms)

But dvc exp show does not appear even though this is after several hours of these experiments running.

About this issue

Original URL
State: closed
Created 10 months ago
Comments: 26 (12 by maintainers)

Most upvoted comments

For anyone else that comes across this issue the min required versions for the fix are:

DVC: 3.17.0
DVCLive: 2.16.0
Ext: 1.0.49

mattseddon on Aug 29, 2023

Following the suggestion by @julieg18 to upgrade DVC to see if the other issue persists, I updated DVC to 3.15.2.

Not sure if that did anything or something else, but the Experiments table just updated. I see in the “DVC” output window:

[version: 1.0.43, 2023-08-18T10:19:04.300Z, pid: 58052] > dvc exp show --rev c7c4d83d358e06b0bca753cbcddffb777fdf140c --rev f7b9c12f505cb6498ab651df4be1447db610c76b --rev 9aca8f642eee1b3f816b4d4c587f525ad6540bc0 --rev c29975b5be3e984ede48359ff07c80d60f6013a0 --rev dc93a51d1b6cc33881e36acedf3ccfc60de0b3ad --rev 4717d23247ecd1ecc5422b81e5ef122f693809f2 --rev 0819967d5197d490b590c41e1227a45e65c4285e --json - COMPLETED (17228ms)

aschuh-hf on Aug 18, 2023