twine: twine should fail more gracefully when encountering a metadata version it doesn't support

(It’s hard to provide a full reproduce here, so if any of the below isn’t clear enough, apologies, but it’s a first shot:)

I’m uploading a package (a wheel specifically) generated in the “standard way” with new setuptools and using twine upload, but twine dies by simply saying:

⊙  /Users/Julian/Development/Magnetic/Floodgate/.tox/pypy/bin/pypy -m twine upload --repository-url http://127.0.0.1:53385 --username floodgate-unittest-user --password 'some password' /var/folders/kh/mscfrt910qb_qb8wqrz7t_7w0000gn/T/tmp9TRY2p/floodgate_unittest-0.1.0-py2-none-any.whl                                                               Julian@Macnetic
Uploading distributions to http://127.0.0.1:53385
TypeError: must be convertible to a buffer, not None

Stepping through with a debugger, it looks like what’s failing is that metadata.name is None in PackageFile.from_filename, which pkg_resources.safe_str can’t cope with (because it regexes it for some reason).

(So that sounds like possibly one improvement, that None shouldn’t make it all the way down there)

The way it gets to be None seems to be that Distribution.parse calls _getHeaderAttrs to look up a callback to use for extracting metadata for a particular setuptools version, but if the version isn’t present, that function returns an empty list, and no metadata is parsed (so name is None).

Seems like another nice improvement would be failing loudly if the setuptools metadata version isn’t supported.

About this issue

Original URL
State: closed
Created 6 years ago
Reactions: 1
Comments: 18 (4 by maintainers)

Most upvoted comments

Also note that some package index implementations do not support receiving Metadata 2.1 packaged uploads - an example is the current version of Artifactory (see https://www.jfrog.com/jira/browse/RTFACT-16189 tracking the issue).

In this case there is a response from the server but all the user gets as notification is the HTTP response code of 400, despite there being a useful server response revealing the issue.

Current output…

HTTPError: 400 Client Error: Bad Request for url: https://artifactory.example.com/artifactory/api/pypi/internal-repo

The useful information that is discarded is…

{
  "errors" : [ {
    "status" : 400,
    "message" : "No enum constant org.jfrog.repomd.pypi.model.PypiMetadata.MetadataVersion.v2_1"
  } ]
}

I’m running a forked version which does a resp.raise_for_status() and outputs the body when it encounters an issue as a work-around until it’s fixed upstream, but I thought I’d mention it here for anyone else who may be encountering it.

+10

goodtune on Apr 24, 2018

For me I only had to pin wheel==0.30.0 the latest setuptools (which is 40.0.0 at this moment) worked just fine.

CCardosoDev on Aug 16, 2018

You will need to pin setuptools==35.5.2 and wheel==0.30.0 for the workaround until artifactory support 2.1.

minghuijiang on Jun 22, 2018

@ljohnston I hit the same thing. If you build your sdist / wheel using setuptools < 38.6.0 it should work fine.

http://setuptools.readthedocs.io/en/latest/history.html#v38-6-0

gatesn on Jun 11, 2018

You will need to pin setuptools==35.5.2 and wheel==0.30.0 for the workaround until artifactory support 2.1.

At the time of writing 35.5.2 is not available. In the proposed list i see 35.0.2 is working for me (together with wheels==0.30.0)

florent-vial on Feb 12, 2019

@sigmavirus24 https://github.com/pypa/twine/blob/fa55c0c5e9dd81a0b67f82bf0574437fbfc0d9a4/setup.py#L23

di on Apr 10, 2018