astropy: HDUList.header returns incorrect values for .keys() and .items()

Description

HDUList.header returns incorrect values for .keys() and .items()

In the example below, hdulist[0].header, shows a key/value pair of

OBJECT = ‘TILEID: 1106’

hdulist[0].header['OBJECT'] returns the value from the key/value pair.

However, both hdulist[0].header.keys() and hdulist[0].header.items() indicate a key of ‘OBJECT.TILEID’. Using the returned key from .keys() or .items() returns a bogus value:

hdulist[0].header['OBJECT.TILEID'] => 1106.0

Expected behavior

Expect ‘OBJECT’ but not ‘OBJECT.TILEID’ to be listed in header.keys() and header.items().

Actual behavior

Got bad values for a key in hdulist[0].header. Seems that the value of a key/value became part of the key. See “Reproduce”

Steps to Reproduce

>>> import astropy.io.fits
>>> hdulist = astropy.io.fits.open("https://astroarchive.noirlab.edu/api/retrieve/00b228b9384ab0e358fd12b3aa1586d0/")
>>> 'OBJECT' in hdulist[0].header        # => True
>>> 'OBJECT' in hdulist[0].header.keys() # => False  # but used as key next
>>> hdulist[0].header['OBJECT']          # => 'TILEID: 1106'
>>> hdulist[0].header['OBJECT.TILEID']   # => 1106.0   # BOGUS key and value
>>> hdulist[0].header                    # Shows 'OBJECT' but not 'OBJECT.TILEID'

System Details

Linux-5.4.0-113-generic-x86_64-with-glibc2.31 Python 3.9.7 (default, Sep 6 2021, 18:25:07) [GCC 9.3.0] Numpy 1.22.4 pyerfa 2.0.0.1 astropy 5.1 Scipy 1.8.1 Matplotlib 3.5.2

About this issue

Original URL
State: open
Created 2 years ago
Comments: 20 (11 by maintainers)

Most upvoted comments

@dhomeier Your direct pip install from hash worked. Yes, your branch resolves my problem. Thanks! I’ll leave my monkey-patch fix in my code until your change works its way into a production release.

pothiers on May 27, 2022

Treat this as a proposed simple fix for this bug.

To get around the items(), keys() bug I have monkey-patched my code with the following. If this change (or equivalent) is applied to Astropy, I’ll remove my monkey-patch and all should continue to work the same.

def NEW_items(self):
    """Like :meth:`dict.items`."""

    for k in self._rvkc_indices:
        v = self.get('OBJECT')
        yield k, None if v == UNDEFINED else v

    for card in self._cards:
        yield card.keyword, None if card.value == UNDEFINED else card.value


def NEW_keys(self):
    """
    Like :meth:`dict.keys`--iterating directly over the `Header`
    instance has the same behavior.
    """

    for k in self._rvkc_indices:
        yield k

    for card in self._cards:
        yield card.keyword

astropy.io.fits.header.Header.items = NEW_items
astropy.io.fits.header.Header.keys = NEW_keys

pothiers on May 27, 2022