vscode-dvc: Exp run fails when launched via Experiments tab, but runs via command line
See https://www.loom.com/share/bbefd727e9044467b56cd1891d9e4452
When I launch an experiment via the Experiments tab, it fails because of an import error.
Launching the same experiment via the command line works fine.
It looks like there’s a difference in environment variables.
Is it possible that the extension is not using my virtual environment?
I am using pipenv
to manage the virtual env in this project.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 2
- Comments: 33 (22 by maintainers)
We made a decision a long time ago not to try and recreate the logic in the Python extension which is used to “auto-activate” virtual environments. There is an ongoing here: https://github.com/microsoft/vscode-python/issues/11039#issuecomment-1140596237 where they have further advocated for using the path to the interpreter instead of activating the environment. However, we know that does not meet all of our needs.
Currently, the Python extension does not actually expose the activation details. We don’t know the details of the expected activation file.
Yes, it is open source. It would be a significant amount of effort. Even finding the specific part of the code is difficult.
I will raise an issue to have the activation command added to their API.
There is a fair bit of logic there. I would start in
extension/src/cli/options.ts
.This is next on my list of things to fix.
I have not tested yet but I am hoping that when
pipenv
is involved we can use the samerun
command as we should be using forconda
. I.e for running experiments we should be able to simply callpipenv run dvc exp run
. This should hook us into the correct virtual environment and do the same thing as sourcing theactivate
file. I would only do this for running experiments (and leave the rest of the logic as is) because there is an overhead to starting up the environment/pipenv.quick test:
Remove sys hack from project and then run
Seems to work!
We should be able to update the setup wizard to ask the user which package manager they use (
conda
,virtualenv
orvenv
, orpipenv
) and adjust the way we call Python accordingly 👍🏻.Sounds like a hack, not sure virtualenv file made to be modified. I think I would either use relative paths in imports (I think should be possible, right?). It will be easier to use projects I think. Need to do a bit of reserach here …
But, I understand the problem finally, and ideally it should be working no matter if we use it in the demo projects or not, since VS Code terminal supports it.
@mattseddon could you please point me to the code that runs DVC commands? how does it interact with MS Python extension?
I believe it works now because of a recent workaround I made (specifically to address this issue): https://github.com/iterative/magnetic-tiles-defect/commit/8ccc5fb2b5dd450a61b7205065b1d050df116ce4
Before, I used to set
PYTHONPATH
as part of the virtual env activation script: https://github.com/iterative/magnetic-tiles-defect/commit/357a0a13fabb2a54f1cb0fc82c991c7c2ad1a16e The extension did not pick up this env var.