poetry: Can not exclude package data file only for the wheel

There seems to be an issue when trying to exclude package data files of the wheel only.

Example project here: https://github.com/sinoroc/poetry-gh-2015

Project file structure:

$ tree Thing
Thing
├── CHANGELOG.rst
├── LICENSE.txt
├── pyproject.toml
├── README.rst
├── test
│   └── test_unit.py
└── thing
    ├── data
    │   ├── file.all
    │   ├── file.bin
    │   ├── file.not
    │   └── file.src
    └── __init__.py

All files are in the git repository except *.bin data files (supposed to be a build artifact).

We want the *.src data files and CHANGELOG.rst in the sdist only. We want *.bin data files in the wheel only. We want *.all data files in both the sdist and the wheel. We do not want any of the .not data files in the distributions.

We also want the test package, but only in the sdist.

.gitignore

/thing/data/*.bin

pyproject.toml:

packages = [
    { include = 'thing' },
    { include = 'test', format = 'sdist' }
]
include = [
    { path = 'CHANGELOG.rst', format = 'sdist' },
    { path = 'thing/data/*.bin', format = 'wheel' },
    { path = 'thing/data/*.src', format = 'sdist' },
]
exclude = [
    { path = 'CHANGELOG.rst', format = 'wheel' },
    { path = 'thing/data/*.src', format = 'wheel' },
    'thing/data/*.not',
]

Content of the sdist:

$ python3 -m tarfile -l dist/Thing-0.1.0.tar.gz 
Thing-0.1.0/CHANGELOG.rst 
Thing-0.1.0/LICENSE.txt 
Thing-0.1.0/README.rst 
Thing-0.1.0/pyproject.toml 
Thing-0.1.0/test/test_unit.py 
Thing-0.1.0/thing/__init__.py 
Thing-0.1.0/thing/data/file.all 
Thing-0.1.0/thing/data/file.src 
Thing-0.1.0/setup.py 
Thing-0.1.0/PKG-INFO

Content of the wheel:

$ python3 -m zipfile -l dist/Thing-0.1.0-py3-none-any.whl 
File Name                                             Modified             Size
thing/__init__.py                              1980-01-01 00:00:00            0
thing/data/file.all                            1980-01-01 00:00:00            0
thing/data/file.bin                            1980-01-01 00:00:00            0
thing/data/file.src                            1980-01-01 00:00:00            0
thing-0.1.0.dist-info/LICENSE.txt              1980-01-01 00:00:00            8
thing-0.1.0.dist-info/WHEEL                    2016-01-01 00:00:00           83
thing-0.1.0.dist-info/METADATA                 2016-01-01 00:00:00         1515
thing-0.1.0.dist-info/RECORD                   2016-01-01 00:00:00          577

Somehow the thing/data/file.src appears in the wheel, which is unexpected and not what we want.

Example project: https://github.com/sinoroc/poetry-gh-2015

About this issue

Original URL
State: open
Created 4 years ago
Reactions: 5
Comments: 15 (1 by maintainers)

Most upvoted comments

exclude = [
    { path = 'CHANGELOG.rst', format = 'wheel' },
    { path = 'thing/data/*.src', format = 'wheel' },
]

At the moment this syntax isn’t support. exclude must be a list of path(globs). We should add the ability to work just like include.

fin swimmer

finswimmer on Nov 20, 2020

packages determines the base files that get included regardless of your distribution format

But, packages also allows to select target distribution formats. It’s also OK if there are 2 ways to achieve the same thing. It’s a bit confusing though… Because as far as I understood, packages and include achieve the same things, except that packages also accepts from.

Yeah I guess the maintainers will be better at explaining this than me. I only just worked from my own understanding based on trial and error. Maybe the functionality of these needs better documentation.

zoj613 on Nov 28, 2020

Is there any way to make the exclude command work without any hacks? doing exclude=["dir/*.pyx"] does nothing. It includes everything into the wheel. The previous workaround I suggested causes any shared libraries to copied outside of the package’s folder after install.

zoj613 on Feb 28, 2021

@sinoroc Here is how I would write the pyproject.toml

name = 'thing'
...

packages = [
	{ include = 'thing/*.py' },
	{ include = 'thing/**/*.all' }
]
include = [
    { path = 'CHANGELOG.rst', format = 'sdist' },
    { path = 'thing/data/*.src', format = 'sdist' },
	{ path = 'test', format = 'sdist' },

    { path = 'thing/data/*.bin', format = 'wheel' },
]
exclude = ['thing/**/*.not']

This approached worked for me. Basically, include everything thats required by both distributions in packages, then use include to selectively add files based on format. The use exclude to ignore files in both distributions.

zoj613 on Nov 27, 2020