astropy: Table.read(filename) is 10 times slower than Table(fits.getdata(filename))

Hey table team –

I have a ~1.5 GB FITS file with ~100 columns, and noticed the following:

Is this expected, or a known performance issue? I didn’t see a reference to this in the docs or currently open issues, but may have missed it…

This is with Astropy v3.0.1.

About this issue

Original URL
State: closed
Created 6 years ago
Comments: 43 (43 by maintainers)

Most upvoted comments

See #7422 for a fix.

taldcroft on May 3, 2018

@adrn - Could you upload your file to somewhere?

bsipocz on May 1, 2018

Tagging myself because I might be of help. Would be useful to have a self-contained test case, with data

manodeep on May 1, 2018

Maybe it is something to do with this FITS file? fitsverify says it’s ok.

BTW: there are no string columns. Here are all the datatypes in this file: ['>f4', '>f8', '>i2', '>i4', '>i8']

I’ll upload the file somewhere…

adrn on Apr 24, 2018

Weiiiiiirddddd

>>> %timeit Table.read('big_table.fits')
805 ms ± 30.1 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
>>> %timeit Table(fits.getdata('big_table.fits', 1))
3.35 s ± 74.1 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

adrn on Apr 24, 2018