poetry: [virtualenvs] configuration not working with pyproject.toml

poetry.toml is now mentioned in the documentation. There is no plan on adding poetry settings control to pyproject.toml file.

I see your point and partially agree with the argument. Nevertheless, I believe there’s at least one action item for this issue: better document poetry.toml. I read through the docs for a while and found nothing on this issue (pyproject.toml vs poetry.toml). Not even on stackoverflow I found much about it (the most I got was someone mentioning poetry.toml existed, but it was such an old post that I simply thought it was a deprecated Poetry configuration file).

For me, this is really a pain because VS Code doesn’t work with the default poetry venv dir (there are issues open on this) and VS Code users HAVE to change venv’s directory in order to use poetry venv. In order to avoid that my team has problems with poetry and VS Code, I looked for a way to enforce poetry to install venv in the project’s root everytime (that way, VS Code would always find the venv folder). This being said, I can see that lots of developers will spend considerable time looking for an appropriate solution for their professional projects and not finding proper documentation and relying on trial and error or workarounds. I believe this justifies adding some documentation on poetry.toml and it’s usage, what configs could go there and so on.

Lastly, an opinion on the separation of files: we have black, isort and many other tool configurations under pyproject.toml -> [tools.<tool-name>]. I don’t see why we can’t have a project setting (where to install venv, for this specific project) in the same file (pyproject.toml) under different keys in order to avoid increasing the number of files. 🤔

See this example of a all-in-one configuration file (there are many other examples).

Just to be clear, I really appreciate your help and opinion… this is a IMO response only, based on bad UX regarding the documentation (not Poetry itself, which is so awesome that I’m trying to change my peers mind to use it in our project) 😄

The poetry.toml contains the local configuration of the poetry application. One should not check-in this file into a repository, because these configuration are just personal flavors.

It’s just a personal opinion that these are personal opinions. Other teams can decide otherwise.

pyproject.toml contains the configuration for black,isort,mypy, pytest,… It contains the information which python-version the virtualenv needs to be instantiated for, and what packages needs to be installed in the virtualenv, so why not support these extra settings?

We are happy to get rid of pytest.ini, .pylintrc,… and a plentitude of other files that can be consolidated in pyproject.toml

Now we are forced to create a poetry.toml, add it to the repo, just to have all our teammembers a consistent setup with an in-project virtualenv.

On some arcane setups where virtual environments need to be managed with another tool (see conda), it would be really helpful to have a way of overriding the virtual environment settings from the pyptoject.toml file, as disabling virtual environment creation is not a user preference but a must.

@finswimmer, would it be possible to preserve the current behavior and allow custom overrides? The priority order would look like this:

1. Global configuration.
2. Overrides from pyproject.toml.
3. Local configuration from poetry.toml.

Now we are forced to create a poetry.toml, add it to the repo, just to have all our teammembers a consistent setup with an in-project virtualenv.

To add to this, I can only assume you guys don’t have a bunch of junior developers on your team. Having a consistent development setup is incredibly important to aid beginners and writing instructions. It would be great to ensure all of them have the venv folder at the same location.

Hello @lcbm,

the reason why the local configs are not in pyproject.toml but in a separate poetry.toml is that one developer should not force another how poetry is configured. This would automatically the case if these local config would be in the pyproject.toml.

But it’s a good idea to document the existens of this file and would fit at best in the Local Configuration section of the docs.

PR is welcome 😃

fin swimmer

Hello @0x2b3bfa0,

if poetry detects that it is already running in a virtual environment - also if it is created by conda - it will not create another venv. Does this fit to your use case?

fin swimmer

Hi @finswimmer

I quite agree with @0x2b3bfa0 's suggestion. So I can have a default virtualenv setting in pyproject.toml(this makes perfect sense for the purpose of pyproject.toml), then if developer prefers a different venv setting then it can be overwritten by poetry.toml, this case poetry.toml can be excluded from version control.

@finswimmer I would like to add another point to maybe revisit this decision.

Recently Poetry implemented another config system-site-packages (#1393 and #3711), which not only allows for saving space, but also make currently incompatible packages (such as those not supporting PEP-517) to work seamlessly with Poetry as well (so we don’t need to specify them on dependencies and still rely on them as a dependency on host).

This setting alone will turn the developer’s environment consistent and therefore it should be set on a per-project basis IMHO. This was an example, but other settings might be introduced in the future that developers would like to “enforce” across their teams for sake of consistency/environment reproducibility.

Perhaps, as an option for future-proofing, it’s worth considering allowing users to set these properties at the project-level as well (on pyproject.toml), being overridden by those in global, then by those in local scope (so we don’t break compatibility). What do you think?

Hello @0x2b3bfa0, if poetry detects that it is already running in a virtual environment - also if it is created by conda - it will not create another venv. Does this fit to your use case? fin swimmer

Hi @finswimmer

I quite agree with @0x2b3bfa0 's suggestion. So I can have a default virtualenv setting in pyproject.toml(this makes perfect sense for the purpose of pyproject.toml), then if developer prefers a different venv setting then it can be overwritten by poetry.toml, this case poetry.toml can be excluded from version control.

This is precisely what needs to be done.

I assume you’re talking about enabling poetry configuration at the project level. In that case, not that I’m aware of, but maybe due to the issue title and tags, this is not getting enough visibility.

To help the devs (and to better clarify the request) we can push this as a feature request in a separate issue.

Could I ask what the difference between having the local config in poetry.toml and pyproject.toml is, on this dimension?

The pyproject.toml contains information about your package, like metadata and dependencies. Anything that is needed to build the package.

The poetry.toml contains the local configuration of the poetry application. One should not check-in this file into a repository, because these configuration are just personal flavors.

To the best of my knowledge, using a key called [virtualenvs] has never been supported, nor has overriding config options in the pyproject.toml – the poetry settings are used to handle processing for the pyproject.toml, so it makes sense that they are separate.