lightkurve: CBVCorrector fails?

Problem description

CBVCorrector fails executing a minimal example from the tutorial.

Example

Input Script

import lightkurve as lk
from lightkurve.correctors import CBVCorrector

import numpy as np
import matplotlib.pyplot as plt
import matplotlib as mpl

print(lk.__version__)
lc = lk.search_lightcurve('TIC 737578805',sector=32,exptime=120)[0].download().remove_nans()
print('downloaded')

cbvCorrector = CBVCorrector(lc)
print('initialized corrector - using download')

# run cbv correction
# Select which CBVs to use in the correction
cbv_type = ['SingleScale', 'Spike']

# Select which CBV indices to use
# Use the first 8 SingleScale and all Spike CBVS
cbv_indices = [np.arange(1,9), 'ALL']

# Perform the correction
print('Correcting!')
cbvCorrector.correct(cbv_type=cbv_type, cbv_indices=cbv_indices);

Output

(lk) d-i184-54-49:lightkurve benjaminpope$ python lktest.py
2.0.10
WARNING: dropping mask in Quantity column 'flux': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'flux_err': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'centroid_col': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'centroid_row': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'sap_flux': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'sap_flux_err': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'sap_bkg': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'sap_bkg_err': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'pdcsap_flux': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'pdcsap_flux_err': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'psf_centr1': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'psf_centr1_err': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'psf_centr2': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'psf_centr2_err': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'mom_centr1': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'mom_centr1_err': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'mom_centr2': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'mom_centr2_err': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'pos_corr1': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'pos_corr2': masked Quantity not supported [astropy.table.table]
downloaded
initialized corrector - using download
Correcting!
WARNING: dropping mask in Quantity column 'flux': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'flux_err': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'centroid_col': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'centroid_row': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'sap_flux': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'sap_flux_err': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'sap_bkg': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'sap_bkg_err': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'pdcsap_flux': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'pdcsap_flux_err': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'psf_centr1': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'psf_centr1_err': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'psf_centr2': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'psf_centr2_err': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'mom_centr1': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'mom_centr1_err': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'mom_centr2': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'mom_centr2_err': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'pos_corr1': masked Quantity not supported [astropy.table.table]
WARNING: dropping mask in Quantity column 'pos_corr2': masked Quantity not supported [astropy.table.table]
WARNING: File may have been truncated: actual file length (458752) is smaller than the expected size (1895040) [astropy.io.fits.file]
Traceback (most recent call last):
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/astropy/timeseries/sampled.py", line 366, in read
    return super().read(filename, format=format, *args, **kwargs)
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/astropy/table/connect.py", line 61, in __call__
    out = registry.read(cls, *args, **kwargs)
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/astropy/io/registry.py", line 527, in read
    data = reader(*args, **kwargs)
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/lightkurve/io/tess.py", line 34, in read_tess_lightcurve
    lc = read_generic_lightcurve(filename, flux_column=flux_column, time_format="btjd")
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/lightkurve/io/generic.py", line 40, in read_generic_lightcurve
    tab = Table.read(hdulist[ext], format="fits")
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/astropy/table/connect.py", line 61, in __call__
    out = registry.read(cls, *args, **kwargs)
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/astropy/io/registry.py", line 527, in read
    data = reader(*args, **kwargs)
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/astropy/io/fits/connect.py", line 234, in read_table_fits
    data = table.data
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/astropy/utils/decorators.py", line 767, in __get__
    val = self.fget(obj)
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/astropy/io/fits/hdu/table.py", line 402, in data
    data = self._get_tbdata()
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/astropy/io/fits/hdu/table.py", line 172, in _get_tbdata
    raw_data = self._get_raw_data(self._nrows, columns.dtype,
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/astropy/io/fits/hdu/base.py", line 529, in _get_raw_data
    return self._file.readarray(offset=offset, dtype=code, shape=shape)
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/astropy/io/fits/file.py", line 324, in readarray
    return np.ndarray(shape=shape, dtype=dtype, offset=offset,
TypeError: buffer is too small for requested array

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "lktest.py", line 30, in <module>
    cbvCorrector.correct(cbv_type=cbv_type, cbv_indices=cbv_indices);
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/lightkurve/correctors/cbvcorrector.py", line 458, in correct
    minimize_result = minimize_scalar(self._goodness_metric_obj_fun, method='Bounded',
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/scipy/optimize/_minimize.py", line 804, in minimize_scalar
    return _minimize_scalar_bounded(fun, bounds, args, **options)
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/scipy/optimize/optimize.py", line 2024, in _minimize_scalar_bounded
    fx = func(x, *args)
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/lightkurve/correctors/cbvcorrector.py", line 811, in _goodness_metric_obj_fun
    underMetric = self.under_fitting_metric()
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/lightkurve/correctors/cbvcorrector.py", line 606, in under_fitting_metric
    metric = underfit_metric_neighbors (corrected_lc, 
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/lightkurve/correctors/metrics.py", line 174, in underfit_metric_neighbors
    lc_neighborhood, lc_neighborhood_flux = _download_and_preprocess_neighbors(
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/memoization/caching/plain_cache.py", line 45, in wrapper
    result = user_function(*args, **kwargs)
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/lightkurve/correctors/metrics.py", line 333, in _download_and_preprocess_neighbors
    lcfCol = search.download_all(flux_column=flux_column)
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/lightkurve/utils.py", line 555, in wrapper
    return f(*args, **kwargs)
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/lightkurve/search.py", line 423, in download_all
    self._download_one(
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/lightkurve/search.py", line 290, in _download_one
    return read(path, quality_bitmask=quality_bitmask, **kwargs)
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/lightkurve/io/read.py", line 92, in read
    return TessLightCurve.read(path_or_url, format="tess", **kwargs)
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/lightkurve/lightcurve.py", line 3078, in read
    return super().read(*args, **kwargs)
  File "/Users/benjaminpope/opt/anaconda3/envs/lk/lib/python3.8/site-packages/astropy/timeseries/sampled.py", line 373, in read
    raise ValueError("``time_column`` should be provided since the default Table readers are being used.")
ValueError: ``time_column`` should be provided since the default Table readers are being used.

Expected behavior

Expected to replicate tutorial at https://docs.lightkurve.org/tutorials/2-creating-light-curves/2-3-how-to-use-cbvcorrector.html .

Environment

All in Python 3.8 fresh install from Conda, in a fresh environment with no other packages installed other than a pip install of lightkurve. I’m separately testing a fork of the dev version for CBV IO in #1122, and a similar bug happens there.

• platform: OSX
• lightkurve version: 2.0.10
• installation method: pip