poetry: peotry version doesn't bump the value in __version__
$ poetry version 0.1.1
Bumping version from 0.1.0 to 0.1.1
$ grep version pyproject.toml
version = "0.1.1"
$ grep version */__init__.py
src/__init__.py:__version__ = '0.1.0'
$ poetry --version
Poetry 0.9.0
I don’t know if it’s intended or not. A way to do that safely is to parse the root __init__.py
, detect __version__
, back it up, extract the ast, bump the version string and replace it, then extract the new ast and compare the result. If both ast are the same, except for the version, the file semantics have been preserved. Otherwise, rollback the change and display an error message stating we can’t bump the version safely.
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Reactions: 50
- Comments: 75 (18 by maintainers)
Commits related to this issue
- fix: get project version and description. this will be improved after poetry fixed issue https://github.com/sdispater/poetry/issues/144 — committed to httprunner/httprunner by debugtalk 5 years ago
- Update version strings in project files. (#144) Added a dry-run option to version command and logic to replace current version strings with next version in: - pyproject.toml - python source ( assignm... — committed to JnyJny/poetry by JnyJny 5 years ago
- Calculate version number based on pyproject.toml During `poetry install`, Poetry installs the project as if it were a package itself. You can then pull the version from the package metadata. It's a l... — committed to br3ndonland/test3 by br3ndonland 4 years ago
- Calculate version number based on pyproject.toml br3ndonland/test3@227ab28 During `poetry install`, Poetry installs the project as if it were a package itself. You can then pull the version from the... — committed to br3ndonland/algorithms by br3ndonland 4 years ago
- fix: get project version and description. this will be improved after poetry fixed issue https://github.com/sdispater/poetry/issues/144 — committed to TanakitIntR/httprunner4 by TanakitIntR in 7 years
- (7/x) Move package version from setup.py to pyproject.toml We now specify the package version in a single place: pyproject.toml so it's not necessary to dynamically figure it out in setup.py. Copped... — committed to databricks/dbt-databricks by susodapop 8 months ago
- (1/x) Create pyproject.toml and move package name from setup.py Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com> (2/x) Move package description from setup.py Signed-off-by: Jesse W... — committed to databricks/dbt-databricks by susodapop 8 months ago
I really wish that the
version
field in the.toml
ended up superseding the__version__
string in__init__.py
. This would reduce the burden of having to keep multiple places in the code in sync.you may use that
If this is the recommended solution, then
poetry new
should be updated to use it. Currently, thepoetry new
command adds__version__
to__init__.py
, but thenpoetry version
doesn’t update it. This incompatibility between the two commands is confusing for me, and I imagine other newcomers.Currently workaround can be to use
bump2version
package (maintained fork ofbumpversion
).For my package
poetry_demo
with defaultpoetry_demo/__init__.py
andtests/test_poetry_demo.py
content I have following.bumpversion.cfg
file (use ofsetup.cfg
file name is also possible):To bump the version:
$ bump2version patch
and all version instances will become 0.1.1.Provided configuration will also commit the modification and create tag. This (and many more features) are configurable.
A little problem is, that currently it requires extra configuration file
.bumpversion.cfg
, but there is an issue https://github.com/c4urself/bump2version/issues/42 planning to usepyproject.toml
The tool seems rather mature, has many features and is used for quite some time.
To implement bumping versions in poetry, it may serve at least as nice inspiration.
Personally I think storing a
__version__
in__init__.py
seems duplicative. You can usepkg_resources
to retrieve the version for the installed package using:If used with:
This will work even when developing locally (it requires the egg-info).
At the moment this is not possible no.
poetry version
will only bump theversion
inpyproject.toml
. This is not intended and that’s definitely something I plan on adding.Setuptoos has solved this problem for 2 years:
https://setuptools.readthedocs.io/en/latest/setuptools.html#configuring-setup-using-setup-cfg-files
You can put all metadata in setup.cfg instead of setup.py. For the version, you can specify:
Which will load the version from the package directly, this way you have it in one place only. It works as long as you has setuptools >= 30.3 (current version is 38.5.1) so basically any package with a setup.py can already benefit from that without using any additional tool.
Le 27/05/2018 à 07:28, Victor-Nicolae Savu a écrit :
Reading this thread, it seems to me that for now it’s best to keep both the
pyproject.toml
version and thepackage.__version__
.So my process is just to stop me accidentally forgetting to update one.
I’ve got a simple test that checks that those two are in alignment and fails if not.
Seems to do the job without having to resort to some of the convoluted solutions above which may be hard to maintain
Closing this as
importlib.metadata
is the way to go. A possible pattern to support python ❤️.8 and >=3.8 I described here.I have 2 cents to spend here as well:
I stopped a long time ago to write any metadata in
__init__.py
. Others explained it or explain it better in other comments or blog articles: metadata has no place in the source code. I don’t want any__author__
,__version__
or else in__init__.py
. This information:pkg_resources
orimportlib.metadata
.Maybe writing
__version__
in__init__.py
was/is very common, but it’s not a standard thing. In the end, I don’t thinkpoetry
should bother too much to support bumping__version__
in__init__.py
.Solution to get version at runtime:
Solution to get version when building Sphinx documentation:
I’ll give you my (very opinionated) two cents.
My background is Java, but I recently find myself in a department of Python developers. For me, this is a massive culture shock - I assumed that a popular language like Python would have a mature ecosystem of tools, standard patterns for CI/CD, but that’s not what I have found.
Nevermind; when life gives you lemons, make lemonade 🍋😄
The first problem I must solve is packaging, so I need:
Obviously those two systems must agree on how to canonically identify a package, i.e. a coordinate scheme for artifacts [ namespace, name, version ].
Repository options in Python seem limited to Artifactory and
pypiserver
. I infer thatpypiserver
’s coordinate system must be some de facto standard. So, I must now find a package manager which can deploy topypiserver
.If you do some Googling, or read Hacker News, everyone is saying, “use Poetry, that’s the project to watch”.
So, I tried Poetry, and those people are correct - it’s already far superior to the alternatives, so good job 👍
However, I immediately ran into this issue #144 , and this prevents me enforcing Poetry as the standard Python packaging tool across my company.
I am that user!
You have presented some options @vlcinsky very clearly, which I appreciate.
I understand that you are constrained by things outside of Poetry, you have compatibility concerns, and so on. However, your suggestions don’t address the real problem in my opinion, which is this:
Only the package manager can define a package’s version, and it must be canonical.
Whatever this
__init__.py
nonsense is, get rid of it.Yes, I know, this file has some semantics for the Python interpreter, and is necessary in certain circumstances… I really don’t care, that file has no business trying to define the version of my package, versioning is the responsibility of the package manager and no-one else.
Most of the suggestions in this thread are about incrementing a version, which is interesting, but futile if there isn’t a single canonical version we can think about incrementing.
Now that doesn’t work when the file system is not available, or when the project file is not shiped.
Le 27/05/2018 à 20:21, jgirardet a écrit :
@pmav99 you can’t do that unless you also ship
pyproject.toml
with your WHEEL for instance.My 2 cents: I was quite used to add a
__version__
in all my__init__.py
and use it as the “truth”. As I wanted to give poetry a go, I started to read quite a few discussions/issues/PR here and a bit elsewhere and stumbled upon #1036, there the solution given by @chrahunt seems indeed to be the way to go. I tested quickly this with a single module and a package projects and it’s working as intended (for >3.8.3 or backports of course):After testing this, I have to admit I see the sense in not having any more any metadata in the code and use this standard library import if needed (cli version or whatever).
Just wanted to comment as this read session helped me quite a bit 😃
edit: added toml based info when in dev
Another potential solution: imitate setuptools_scm (see last part of https://github.com/pypa/setuptools_scm#pyprojecttoml-usage) and let the developer specify something like
(Add
_version.py
to your SCM’s ignore file so you don’t check it in)Then you can import it like
No external deps necessary.
Apart from poetry itself,
__version__
bumping should be implemented in the scaffold project that is being created bypoetry new
too. In that particular case, I think it would make sense to stick to the standard library instead of adding a toml depenedency, so something like this could be used:Depending on your build process, one solution is to use a shell script for bumping the version instead of running
poetry version
explicitly. I prefer using makefiles, so I added these two rules to mine to solve this problem.This command can be run as follows:
make version=1.2.3 version
If you dislike specifying the version(type) that way, this could just as easily be implemented as a shell script such as:
Thoughts on poetry tooling for versioning
Table of Contents
poetry
is becoming the tool simplifying python package development.However, to succeed in wider range, it must fit into automated build processes and this includes managing package versions.
Risk: When poetry fails to meet expectations (about version management) of a developer, we are likely to loose such user.
Requirements to version management
--version
flagMeeting all these requirements is in fact impossible. There will be always space for well defined and respected procedures in package version management. The solution shall try to make these procedures as simple as possible anyway.
Existing options
Manually edited source code
Most packages use this approach.
Version bumping tool
Tools like bump2version provide commands to bump version using simple call updating content of arbitrary number of files. Sometime the tool may even attempt to commit and tag the changes.
Note: currently poetry supports bumping version in
pyproject.toml
, so it partially falls into this category. Anyway, other features (edit content of other files) is not provided (yet).SCM commit/tag based versioning
Tools like pbr or setuptools_scm read version related information from commits and tags. When commit is tagged as v1.2.3, such package version is built. Following commits get incremented versions (using suffixes such as
dev{distance}
etc).Note: While there are multiple SCM systems (git, hg, …), some tools work only with git and this seems to be generally more and more acceptable decision.
Poetry options for package version management
Here are some options, poetry could take in regards to package version management.
No version bumping support
Skip version command, leave user with their own choice of external tools such as
bump2version
.Bump version on predefined locations
Bumping would update version in
pyproject.toml
and in another file (likemodule/__init__.py
).Adopt commit/tag based versioning
Adopt
pbr
orsetuptools_scm
approach and make it simpler.Support poetry versioning plugins
To be flexible enough, use plugins (similarly as
pytest
is using them) for versioning.It would require poetry to:
pyproject.toml
and add information about (single) version bumping plugin to use, probably allowing also extra section for plugin specific configuration parameters.There are packages which can make plugin integration rather easy, such as pluggy.
Summary
First two options (“No version bumping support” and “Bump version on predefined locations”) are not ideal, as most builds result in packages declaring the same version regardless of having different codebase. This could be partially solved by rule to bump to dev version after each release, but this is quite inflexible and not easy to follow.
I would be very happy with “Adopt commit/tag based versioning” option. Anyway, I am not sure, if such solutions seems enough acceptable to wider audience.
Last option (“Support poetry versioning plugins”) seem to be the most flexible one. It adds some complexity to poetry source code, but on the other hand it could outsource version bumping effort to other parties and it also has the potential to “please everybody”.
Proposed next actions:
In my case reading from
pyproject.toml
does not make sense as I ship a CLI andmyapp version
has to be able to answer to this question withoutpyproject.toml
.FWIW I’m using this:
Works fine 👍
One thing to keep in mind with the automatisms like that is that they will access the disk on importing the library. Code execution on import is not that uncommon in Python, but not desireable to keep down import times and minimize side effects.
Does the pyproject.toml file even ship with wheels?
Using
pkg_resources
also introduces an install-time dependency on setuptools, no?I have published poetry-bumpversion plugin to update version in other files when poetry upgrades the package version. To update
__version__
just install the plugin and add the following to yourpyproject.toml
file.I’m coming in here in Jan 2022, and I’m actually just looking for the canonical answer. Where is the single source of truth spot to put the version string (or where in the docs as poetry version doesn’t seem to be the place).
I ran across this: https://pythonrepo.com/repo/tiangolo-poetry-version-plugin-python-package-dependency-management
which seems to show that others have this issue and are solving it with plugins but if I missed this a pointer in the right direction is appreciated.
It doesn’t work with vendoring, or anything using sys.path.append. And it’s way harder to type in an interpretter. Who remember that ?
Le 30/05/2018 à 19:54, Bert JW Regeer a écrit :
Cross-posting my comment from: https://github.com/python-poetry/poetry/pull/2366#issuecomment-849615115 as it seems it’s relevant here too.
Sorry if it feels like spam, but I think people subscribed to this issue might not be subscribed to that one and could miss this.
I just made a plugin (available with Poetry
1.2.0a1
and above) to read the version from an__init__.py
file:or from a git tag, set with a GitHub release or with:
I thought this could be useful for others here too.
https://github.com/tiangolo/poetry-version-plugin
I’d also like some clarification. I’m not sure what to follow. There are still several different plugins floating around:
I’m fine to use
importlib.metadata
, so long as I can also use it for local packages I am developing (develop = true
). Has anyone tested this?Let me summarize (not expressing my opinion, only my understanding of opinions above – including mine; please correct any omissions or misunderstandings and I will edit accordingly). Specs:
Solutions (scm issue dealt with separately):
init.__version__
maintained by poetry: it’s a duplication but it is managed. In case of conflict pyproject.toml wins,__version__
is overwritterninit.__version__
initialized at load time using importlib.metadata or alternate mechanism: checks the first three boxesAs to the alternate mechanism to initialize
__version__
, some people are wary of an additional import so here are three options__init__
As to SCM:
version = {use_scm = true, scm=git}
or some such@vanyakosmos But based on @finswimmer’s comment, couldn’t you just use this logic to achieve that?
Again, extracting from pyproject.toml doesn’t help at run-time, because the pyproject file is not installed.
Another take on extracting the version from the “pyproject.toml” file:
The
version
variable will beset toNone
in case the file is broken.Bonus: basic tests for it!
This plugin also has support for updating
__version__
: https://github.com/mtkennerly/poetry-dynamic-versioningI also think this is something very important.
Another example is when you integrate it with a tool like Sentry, you are willing to know for which version it applies. Currently, I couldn’t find a proper way to include it except writing a
__version__
string in the__init__.py
@adithyabsk This is not guaranteed and safe. This would cause “pyproject.toml” to be sent to the “site-packages” folder. If another project has the same import configuration, it will replace the “pyproject.toml” of the other project.
Any solution that requires adding a python dependency to my project to read a simple version string is a non-solution. This includes
setuptools
and especiallypkg_resources
, which is very slow.IMHO, version semantics and bumping and SCMs could all be considered secondary features. The most important thing is to ensure that the version in
pyproject.toml
drives/generates the version found inmypackage.__version__
.I have the impression, the discussion here goes beyond the aim of poetry:
“Packaging” is the important word. Not CI/CD nor Deployment.
If you package your project, the version number is integrated into it. Then use
importlib_metadata
(since python 3.8 as build-inimportlib.metadata
) to access the version number during run time if needed. There is no need to have the version number in any additional file.With
poetry version
one have a cli to manipulate the version number inpyproject.toml
(which is used when building the package through poetry). So integrating this into a pipeline shouldn’t be that hard, should it?Just my two cents 😃
I’m with @vlcinsky on this one. The best I’ve found so far to support git tag driven versions is https://pypi.org/project/setuptools-scm/.
I also switched to
importlib.metadata
But I had to change my release workflow, before it was fully automated as I was able to choose in advance the version number during development.My Take Away for
importlib.metadata
Pros: No more version number in files No more required to bump/commit/push a new version during development or release workflow
Version is only available in git tags and inside package relatively easy to automate for patches releases even with branch protection
Cons: hard to automated for minor and major release
Bonus
Here is the github action snippet code to trigger a package publish for version 1.0.0 My Workflow
I use manually gh cli
gh release create 1.0.0 --generate-notes
or directly from github.com (it generates the same event)@williamcanin ~you can distribute the pyproject.toml if you would like, as in the option is there to do it.~
EDIT: this is actually incorrect as per: #144 (comment)
Fascinating reply from @sdispater in January to a qustion about getting poetry to use setuptools_scm “as a plugin or as direct integration” on twitter:
Any news on this matter?
I am right now checking out importlib_metadata in 3.7 (backport of importlib.metadata suggested by @dmontagu ) It seems to pick the contents of pyproject, not
__init__.py
after a couple of tests. The peps that describe metadata are 214, 314, 345 and 566. Pep 8 is about formatting, as we all know, and its mention of__version__
is cursory, in an example. 214 mentions two files, PKG_INFO and setup.py. I wonder where this use of putting some metadata in init.py comes from (I follow it too, without much thought; not questioning it’s widespread). The optional pep396 (status: deferred) is the closest I could find. It statesIn fact, proper version management was one of two reasons we postponed using poetry and moved back to using
pbr
. The other reason was usability with private repositories which we need in our continuous integration process.I really like the workflow, where the version is assigned by adding tag to git repo, I even plan to adopt usage of reno for generating release notes.
I wonder, if similar workflow could be possible with poetry. Taking into account, that pyproject.toml includes explicit package version, this does not seem trivial. One idea is to allow version having alternative value
git-tag
and in such a case to generate the value from git tag (the stylepbr
does that seems working well).I utilized the latter strategy (looking for a list of defined version files in
pyproject.toml
) when I created AutoPub, a tool that enables project maintainers to release new package versions to PyPI by merging pull requests. AutoPub needs to increment all relevant version numbers before publishing new packages to PyPI, and while I was hoping Poetry could handle this, I eventually solved it within AutoPub itself.First, AutoPub looks for a
tool.autopub.version-strings
key and creates aVERSION_STRINGS
list of the version file paths contained within, if any.AutoPub defines an
update_version_strings
function that increments version numbers. This function is used to update the relevant version string(s) if any version files are defined.Perhaps these code samples could be useful when a solution is devised for Poetry. I would love to remove that code and rely on Poetry to comprehensively handle version string updates in all the relevant files.
The reasons I think this latter approach is preferred relative to others include the following:
@JnyJny If I understand correctly, your current approach for text files is just replacing the version string wherever it occurs in any file with a matching extension. I feel like that has a relatively high potential for false positives, e.g. if you list required versions for dependencies anywhere in your documentation.
I haven’t dug into your implementation for python files at all but in principle I think it would be less likely to cause problems there (given you can analyze code more deeply than documentation, and code is less likely to include unrelated version strings).
Also, I think this would be a nice quality-of-life improvement even if it was only applied to
__init__.py
s (and given the potential for false positives, that may be preferable anyway).I would also like to see
poetry version
bump the version in:pyproject.toml
<package_name>/__init__.py
tests/*.py
# any tests which have a version literal in themSince
pyproject.toml
is already handled, it seems something like using astor might be a good choice for safely modifying python source.