astropy: Bad hardlink in 3.1 source tarball

Thanks for all the work put into this version; unfortunately I did not have time to test the release candidates. Unpacking the source (on Linux) results in the following error:

> tar xf astropy-3.1.tar.gz
tar: astropy-3.1/astropy/CITATION: Cannot hard link to `astropy-3.1/CITATION': Operation not permitted
tar: Exiting with failure status due to previous errors

Everything except for the link in astropy/ is present, so setup.py build subsequently fails with

Traceback (most recent call last):
  File "setup.py", line 26, in <module>
    import astropy
  File "/beegfs/home/hd/hd_hd/hd_qc666/build/python/astropy-3.1/astropy/__init__.py", line 274, in <module>
    __citation__ = __bibtex__ = _get_bibtex()
  File "/beegfs/home/hd/hd_hd/hd_qc666/build/python/astropy-3.1/astropy/__init__.py", line 267, in _get_bibtex
    with open(citation_file, 'r') as citation:
FileNotFoundError: [Errno 2] No such file or directory: '/beegfs/home/hd/hd_hd/hd_qc129/build/python/astropy-3.1/astropy/CITATION'

This requires manually running ln -s ../CITATION from the ./astropy/ subdir before the build proceeds.

About this issue

Original URL
State: closed
Created 6 years ago
Comments: 18 (18 by maintainers)

Most upvoted comments

@plim, sorry haven’t done so yet. I no longer have access to the cluster using BeeGFS and thus cannot test anything, but I checked that the 3.2.3 tarball still contains the hardlink. So assuming setup still requires both instances of CITATION, I suppose I can document the current state of things and the workaround of manually creating the link. Unfortunately I cannot investigate now if the same problem pops up when installing from source using pip, and how to fix the link in that case - installing explicitly from the GitHub release should probably do.

Wondering if known_issues.rst is actually the best place to put this, or rather the Building from Source section in install.rst…

dhomeier on Nov 2, 2019

That’s in fact strange - I’ve now tested this with the 3.1 and 3.1.1 releases and the latest HEAD on Linux, and setup.py sdist actually lets this always end up as a soft link in the tarball, although it reports tons of messages about creating hard links…

making hard links in astropy-3.2.dev23748...
hard linking .astropy-root -> astropy-3.2.dev23748
hard linking CHANGES.rst -> astropy-3.2.dev23748
hard linking CITATION -> astropy-3.2.dev23748
hard linking LICENSE.rst -> astropy-3.2.dev23748
hard linking README.rst -> astropy-3.2.dev23748
hard linking ah_bootstrap.py -> astropy-3.2.dev23748
hard linking pip-requirements -> astropy-3.2.dev23748
hard linking setup.cfg -> astropy-3.2.dev23748
hard linking setup.py -> astropy-3.2.dev23748
hard linking astropy/CITATION -> astropy-3.2.dev23748/astropy

when only CITATION comes out as a (soft) link at all:

-rw-r-----  1 derek 342746 Jan 16 23:39 CHANGES.rst
lrwxrwxrwx  1 derek     16 Dec  6 10:01 CITATION -> astropy/CITATION

Thus my speculation that pypi.org is creating their tarballs somewhat differently. Maybe distutils just has a different concept of hard link than Unix in general.

But anyway, not much we can do about the actual thing either way, so I’ll go ahead at last with a PR to document this in known_issues.rst.

dhomeier on Jan 16, 2019

I think so. In principle only the person or bot creating the tarball would have to know the latter solution, and otherwise anyone building from source would likely figure out how to manually create the link themselves, but it cannot hurt to have it documented.

dhomeier on Dec 19, 2018

Looks like creating the tarball with --hard-dereference will produce an actual copy of the file in astropy/CITATION; that should at least keep the issue from the user. The hard link actually appears to be only a problem of the specific cluster file system, not with Unix/Linux in general.

dhomeier on Dec 19, 2018