astropy: Unit parsing errors produce misleading advice in certain cases
Description
When certain units cannot be parsed, a specific message is printed that can be misleading in certain cases. If a user follows the advice, the result is an exception.
Expected behavior
The warning about defining a new unit in order to allow it to be parsed should take into account units that are already defined. In addition, astropy.units
should go ahead and parse the unit anyway, if it is already defined in astropy.units
.
How to Reproduce
- Set up a FITS file.
>>> import numpy as np
>>> import astropy.units as u
>>> from astropy.io import fits
>>> from astropy.table import Table, QTable
>>> rng = np.random.default_rng()
>>> targetid = rng.integers(0, 2**64, (20, ), dtype=np.uint64)
>>> mag = 1e9 * rng.random((20, ), dtype=np.float32)
>>> columns = fits.ColDefs([fits.Column(name='TARGETID', format='K', bzero=2**63, array=targetid),
>>> fits.Column(name='FLUX', format='E', array=mag)])
>>> hdu0 = fits.PrimaryHDU()
>>> hdu0.header.append(('EXTNAME', 'PRIMARY', 'extension name'))
>>> hdu0.add_checksum()
>>> hdu1 = fits.BinTableHDU.from_columns(columns, name='BINTABLE', uint=True)
>>> hdu1.header.insert('TFORM2', ('TUNIT2', 'nanomaggy', 'Units for FLUX'), after=True)
>>> hdu1.add_checksum()
>>> hdulist = fits.HDUList([hdu0, hdu1])
>>> hdulist.writeto('demonstrate_maggy.fits', overwrite=True)
- Try to read the FITS file.
>>> t = Table.read('demonstrate_maggy.fits', hdu='BINTABLE')
WARNING: UnitsWarning: 'nanomaggy' did not parse as fits unit: At col 0, Unit 'nanomaggy' not supported by the FITS standard. If this is meant to be a custom unit, define it with 'u.def_unit'. To have it recognized inside a file reader or other code, enable it with 'u.add_enabled_units'. For details, see https://docs.astropy.org/en/latest/units/combining_and_defining.html [astropy.units.core]
- Great, I’ll define the unit then!
>>> nanomaggy = u.def_unit('nanomaggy', 1.0e-9 * 3631*u.Jy)
>>> u.add_enabled_units( [nanomaggy,])
Traceback (most recent call last):
File "/Users/weaver/Documents/Code/git/desihub/desiutil/demonstrate_maggy.py", line 27, in <module>
u.add_enabled_units( [nanomaggy,])
File "/Users/weaver/Documents/local/products/venv/desiconda/lib/python3.10/site-packages/astropy/units/core.py", line 469, in add_enabled_units
get_current_unit_registry().add_enabled_units(units)
File "/Users/weaver/Documents/local/products/venv/desiconda/lib/python3.10/site-packages/astropy/units/core.py", line 221, in add_enabled_units
raise ValueError(
ValueError: Object with name 'nanomaggy' already exists in namespace. Filter the set of units to avoid name clashes before enabling them.
- OK, so if
nanomaggy
is already defined, why didn’t you say so? I can understand the FITS warning, butastropy.units
clearly has the capability to parse it anyway.
Versions
import platform; print(platform.platform())
import sys; print("Python", sys.version)
import astropy; print("astropy", astropy.__version__)
import numpy; print("Numpy", numpy.__version__)
import erfa; print("pyerfa", erfa.__version__)
import scipy; print("Scipy", scipy.__version__)
import matplotlib; print("Matplotlib", matplotlib.__version__)
macOS-13.5.2-x86_64-i386-64bit
Python 3.10.10 (main, Feb 10 2023, 08:40:50) [Clang 14.0.0 (clang-1400.0.29.202)]
astropy 5.2.1
Numpy 1.22.4
pyerfa 2.0.0.1
Scipy 1.8.1
Matplotlib 3.6.2
About this issue
- Original URL
- State: open
- Created 10 months ago
- Comments: 16 (16 by maintainers)
Taking a look now. Let me delve into the secondary bug at first (I’m running a very slightly simplified and self-contained version of the MWE, reproduced hereafter for convenience)
printing
t1
andt2
reveals thatt1
seems to have lost units metadata:so I guess that’s another symptom of the suspected bug, and maybe more easily exploitable than the difference between getting a warning or not. At a high level, the difference is that, when passed a filename,
read_table_fits
will first read it as aHDUList
, then recurse https://github.com/astropy/astropy/blob/48a792f9d8dc659f428e9c6f0e9e9964a1b60682/astropy/io/fits/connect.py#L234-L252 In the nested call, the following special-case code will be run https://github.com/astropy/astropy/blob/48a792f9d8dc659f428e9c6f0e9e9964a1b60682/astropy/io/fits/connect.py#L184 None of this happens if you just pass in aBinTableHDU
, as is evident from the first listing. Deeper investigation is needed, but I think this bug (and discussion) deserves its own issue, so I’ll open a separate one. I also intend to come back and study the main bug reported here more closely.update: xref #16077
I guess the “connect” logic in
io.fits
toTable
could catch this warning and and throw another warning that is more useful? 🤔An additional test:
nanomaggy
withnanofluxy
.nanofluxy
toastropy.units
:Interesting. I also want to test what happens with some other unit that isn’t already defined, I’ll give that a try tomorrow.
Correct,
nanomaggy
is not part of the FITS standard. Nevertheless, it is defined byastropy.units.photometric
. I do think it is fair to issue a warning about a unit not being in the FITS standard.However, it’s the second part of the warning that is misleading here, and that’s why I’m reporting this.