qlib: TypeError: control character 'delimiter' cannot be a newline
๐ Bug Description
Traceback (most recent call last):
File โD:\dev\anaconda3\envs\qlib\lib\site-packages\IPython\core\interactiveshell.pyโ, line 3398, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File โ<ipython-input-23-054cc63f4876>โ, line 1, in <cell line: 1>
e.calendar(start_time=โ2010-01-01โ, end_time=โ2017-12-31โ, freq=โdayโ)
File โD:\dev\pyqlib\qlib\qlib\data\data.pyโ, line 90, in calendar
_calendar, _calendar_index = self._get_calendar(freq, future)
File โD:\dev\pyqlib\qlib\qlib\data\data.pyโ, line 171, in _get_calendar
_calendar = np.array(self.load_calendar(freq, future))
File โD:\dev\pyqlib\qlib\qlib\data\data.pyโ, line 657, in load_calendar
backend_obj = self.backend_obj(freq=freq, future=future).data
File โD:\dev\pyqlib\qlib\qlib\data\storage\file_storage.pyโ, line 129, in data
H[โcโ][key] = self._read_calendar()
File โD:\dev\pyqlib\qlib\qlib\data\storage\file_storage.pyโ, line 111, in _read_calendar
for x in np.loadtxt(fp, str, skiprows=skip_rows, max_rows=n_rows, delimiter=โ\nโ, encoding=โutf-8โ)
File โD:\dev\anaconda3\envs\qlib\lib\site-packages\numpy\lib\npyio.pyโ, line 1301, in loadtxt
arr = _read(fname, dtype=dtype, comment=comment, delimiter=delimiter,
File โD:\dev\anaconda3\envs\qlib\lib\site-packages\numpy\lib\npyio.pyโ, line 1006, in _read
next_arr = _load_from_filelike(
TypeError: control character โdelimiterโ cannot be a newline (\r or \n).
To Reproduce
Steps to reproduce the behavior: 1.download the data by get_data.py firstly 2.write a simple demo qlib.init(provider_uri=โD:/dev/pyqlib/data/cn_data/โ)
from qlib.data import D
D.calendar(start_time='2010-01-01', end_time='2017-12-31', freq='day')[:5]
Expected Behavior
Screenshot
Environment
Note: User could run cd scripts && python collect_info.py all under project directory to get system information
and paste them here directly.
- Qlib version: 0.8.6
- Python version: 3.8
- OS (
Windows,Linux,MacOS):Windows - Commit number (optional, please provide it if you are using the dev version):
Additional Notes
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 16
This has been fixed in the latest code https://github.com/microsoft/qlib/blob/c34051c1ced920e365f8167e2bba5302a6f08318/qlib/data/storage/file_storage.py#L109
The previous solution does not work for high-frequent data. It is fixed here.
https://github.com/microsoft/qlib/pull/1169/commits/54231b1ea7ffb1d91435b8262dbfd00af770a1fe
It will be merged to the main branch in the short future.