yfinance: 0.2.29 bug: OperationalError('unable to open database file')

Describe bug

fail when downloading array, work ok in 0.2.28

Simple code that reproduces your problem

from pandas_datareader import data as pdr
import yfinance as yf
from numpy import isnan

##yf.pdr_override()

def loadtickers(tickers,start="2005-1-1",end=None,renamefield={},prev=None):
    for i in range(0,2):
        try:
            df = yf.download(tickers, start, end, auto_adjust=True,ignore_tz = True)
            df.columns = ['_'.join(((str.lower(col[1]) if col[1] not in renamefield.keys() else str.lower(renamefield[col[1]]), str.lower(col[0])))) for col in df.columns]
            for t in tickers:
                if t in renamefield.keys():
                    t=renamefield[t]
                print("{}_return".format(t).lower(),"{}_close".format(t).lower() )
                df["{}_return".format(t).lower()] = df["{}_close".format(t).lower()].pct_change().fillna(0)
            if prev is not None:
                df=df.join(prev,how='left')
            return df
        except Exception as e:
            print(e)
            continue
        break
    return

p1=loadtickers(["SPY","TLT"],start="2004-12-31",renamefield={"^VXTLT":"VXTLT","^TNX":"TNX",})

Debug log

DEBUG Entering download() DEBUG:yfinance:Entering download() DEBUG Disabling multithreading because DEBUG logging enabled DEBUG:yfinance: Disabling multithreading because DEBUG logging enabled DEBUG Entering history() DEBUG:yfinance: Entering history() DEBUG Entering history() DEBUG:yfinance: Entering history() DEBUG Entering history() DEBUG:yfinance: Entering history() DEBUG Entering history() DEBUG:yfinance: Entering history() DEBUG Entering history() DEBUG:yfinance: Entering history() DEBUG Entering history() DEBUG:yfinance: Entering history() DEBUG Entering history() DEBUG:yfinance: Entering history() DEBUG Entering history() DEBUG:yfinance: Entering history() DEBUG Entering history() DEBUG:yfinance: Entering history() DEBUG Entering history() DEBUG:yfinance: Entering history() DEBUG Entering history() DEBUG:yfinance: Entering history() DEBUG Entering history() DEBUG:yfinance: Entering history() DEBUG Entering history() DEBUG:yfinance: Entering history() DEBUG Entering history() DEBUG:yfinance: Entering history() DEBUG Entering history() DEBUG:yfinance: Entering history() DEBUG Entering history() DEBUG:yfinance: Entering history() DEBUG Entering history() DEBUG:yfinance: Entering history() DEBUG Entering history() DEBUG:yfinance: Entering history() DEBUG Entering history() DEBUG:yfinance: Entering history() DEBUG Entering history() DEBUG:yfinance: Entering history() DEBUG Entering history() DEBUG:yfinance: Entering history() DEBUG Entering history() DEBUG:yfinance: Entering history() DEBUG Entering history() DEBUG:yfinance: Entering history() DEBUG Entering history() DEBUG:yfinance: Entering history() ERROR
24 Failed downloads: ERROR:yfinance: 24 Failed downloads: ERROR [‘TMF’, ‘TMV’, ‘VBINX’, ‘SHV’, ‘^GVZ’, ‘VXZ’, ‘^VIX’, ‘SGOV’, ‘TLT’, ‘QQQ’, ‘^VXN’, ‘GLL’, ‘SPHD’, ‘SPY’, ‘TQQQ’, ‘QLD’, ‘SPLV’, ‘TBF’, ‘IAU’, ‘^VXTLT’, ‘GLD’, ‘XLU’, ‘VXX’, ‘UGL’]: OperationalError(‘unable to open database file’) ERROR:yfinance: [‘TMF’, ‘TMV’, ‘VBINX’, ‘SHV’, ‘^GVZ’, ‘VXZ’, ‘^VIX’, ‘SGOV’, ‘TLT’, ‘QQQ’, ‘^VXN’, ‘GLL’, ‘SPHD’, ‘SPY’, ‘TQQQ’, ‘QLD’, ‘SPLV’, ‘TBF’, ‘IAU’, ‘^VXTLT’, ‘GLD’, ‘XLU’, ‘VXX’, ‘UGL’]: OperationalError(‘unable to open database file’) DEBUG [‘TMF’, ‘TMV’, ‘VBINX’, ‘SHV’, ‘^GVZ’, ‘VXZ’, ‘^VIX’, ‘SGOV’, ‘TLT’, ‘QQQ’, ‘^VXN’, ‘GLL’, ‘SPHD’, ‘SPY’, ‘TQQQ’, ‘QLD’, ‘SPLV’, ‘TBF’, ‘IAU’, ‘^VXTLT’, ‘GLD’, ‘XLU’, ‘VXX’, ‘UGL’]: Traceback (most recent call last): File “/usr/local/lib/python3.10/dist-packages/peewee.py”, line 3192, in connect self._state.set_connection(self._connect()) File “/usr/local/lib/python3.10/dist-packages/peewee.py”, line 3536, in _connect conn = sqlite3.connect(self.database, timeout=self._timeout, sqlite3.OperationalError: unable to open database file

      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "/usr/local/lib/python3.10/dist-packages/yfinance/multi.py", line 275, in _download_one
          data = Ticker(ticker, session=session).history(
        File "/usr/local/lib/python3.10/dist-packages/yfinance/utils.py", line 114, in wrapper
          result = func(*args, **kwargs)
        File "/usr/local/lib/python3.10/dist-packages/yfinance/base.py", line 148, in history
          tz = self._get_ticker_tz(proxy, timeout)
        File "/usr/local/lib/python3.10/dist-packages/yfinance/base.py", line 1644, in _get_ticker_tz
          cache = utils.get_tz_cache()
        File "/usr/local/lib/python3.10/dist-packages/yfinance/utils.py", line 985, in get_tz_cache
          _tz_cache = _TzCache()
        File "/usr/local/lib/python3.10/dist-packages/yfinance/utils.py", line 926, in __init__
          db.connect()
        File "/usr/local/lib/python3.10/dist-packages/peewee.py", line 3191, in connect
          with __exception_wrapper__:
        File "/usr/local/lib/python3.10/dist-packages/peewee.py", line 3019, in __exit__
          reraise(new_type, new_type(exc_value, *exc_args), traceback)
        File "/usr/local/lib/python3.10/dist-packages/peewee.py", line 192, in reraise
          raise value.with_traceback(tb)
        File "/usr/local/lib/python3.10/dist-packages/peewee.py", line 3192, in connect
          self._state.set_connection(self._connect())
        File "/usr/local/lib/python3.10/dist-packages/peewee.py", line 3536, in _connect
          conn = sqlite3.connect(self.database, timeout=self._timeout,
      peewee.OperationalError: unable to open database file

DEBUG:yfinance: [‘TMF’, ‘TMV’, ‘VBINX’, ‘SHV’, ‘^GVZ’, ‘VXZ’, ‘^VIX’, ‘SGOV’, ‘TLT’, ‘QQQ’, ‘^VXN’, ‘GLL’, ‘SPHD’, ‘SPY’, ‘TQQQ’, ‘QLD’, ‘SPLV’, ‘TBF’, ‘IAU’, ‘^VXTLT’, ‘GLD’, ‘XLU’, ‘VXX’, ‘UGL’]: Traceback (most recent call last): File “/usr/local/lib/python3.10/dist-packages/peewee.py”, line 3192, in connect self._state.set_connection(self._connect()) File “/usr/local/lib/python3.10/dist-packages/peewee.py”, line 3536, in _connect conn = sqlite3.connect(self.database, timeout=self._timeout, sqlite3.OperationalError: unable to open database file

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File “/usr/local/lib/python3.10/dist-packages/yfinance/multi.py”, line 275, in _download_one data = Ticker(ticker, session=session).history( File “/usr/local/lib/python3.10/dist-packages/yfinance/utils.py”, line 114, in wrapper result = func(*args, **kwargs) File “/usr/local/lib/python3.10/dist-packages/yfinance/base.py”, line 148, in history tz = self._get_ticker_tz(proxy, timeout) File “/usr/local/lib/python3.10/dist-packages/yfinance/base.py”, line 1644, in _get_ticker_tz cache = utils.get_tz_cache() File “/usr/local/lib/python3.10/dist-packages/yfinance/utils.py”, line 985, in get_tz_cache _tz_cache = _TzCache() File “/usr/local/lib/python3.10/dist-packages/yfinance/utils.py”, line 926, in init db.connect() File “/usr/local/lib/python3.10/dist-packages/peewee.py”, line 3191, in connect with exception_wrapper: File “/usr/local/lib/python3.10/dist-packages/peewee.py”, line 3019, in exit reraise(new_type, new_type(exc_value, *exc_args), traceback) File “/usr/local/lib/python3.10/dist-packages/peewee.py”, line 192, in reraise raise value.with_traceback(tb) File “/usr/local/lib/python3.10/dist-packages/peewee.py”, line 3192, in connect self._state.set_connection(self._connect()) File “/usr/local/lib/python3.10/dist-packages/peewee.py”, line 3536, in _connect conn = sqlite3.connect(self.database, timeout=self._timeout, peewee.OperationalError: unable to open database file

DEBUG Exiting download() DEBUG:yfinance:Exiting download() 2023-09-22-15-59 tickers.pkl loading fresh 2023-09-22-15-59 tickers.pkl spy_return spy_close tlt_return tlt_close iau_return iau_close gld_return gld_close qqq_return qqq_close vix_return vix_close vxtlt_return vxtlt_close gvz_return gvz_close qld_return qld_close vxz_return vxz_close tbf_return tbf_close vxn_return vxn_close vxx_return vxx_close tmf_return tmf_close tmv_return tmv_close splv_return splv_close sphd_return sphd_close vbinx_return vbinx_close xlu_return xlu_close tqqq_return tqqq_close ugl_return ugl_close gll_return gll_close sgov_return sgov_close shv_return shv_close

Bad data proof

No response

`yfinance` version

0.2.29

Python version

No response

Operating system

No response

About this issue

Original URL
State: closed
Created 9 months ago
Reactions: 3
Comments: 25

Most upvoted comments

Not sure if this is related, but I am having a similar issue when using yf.download, I am getting a sqlite3.OperationalError: unable to open database file error. I am running on Windows and I believe it may have something to do with the cache path folder not existing.

If I manually create the py-yfinance folder in C:\Users\Himel\AppData\Local\, I no longer get the exception.

Himel55 on Sep 23, 2023

In my case (windows using Git Bash), I found that this is due to the fact that peewee.py is trying to create a db at %LocalAppData%/py-yfinance/tkr-tz.db and since sqlite3.connect cannot force create the py-yfinance folder this fails.

I manually created the %LocalAppData%/py-yfinance/ folder and was able to bypass this issue, in my instance.

Edit: I see it’s been identified already here

j-nx on Sep 23, 2023

Can I get more confirmation that my refactor hasn’t introduced bugs? Then can get a release out before markets open.

ValueRaider on Sep 24, 2023

I can confirm the error on my Linux environment. The peewee library does not create the py-yfinance folder. Tried your fix @ValueRaider and it worked.

Ke293-x2Ek-Qe-7-aE-B on Sep 23, 2023

@enzbus how many CPUs on your mac, vs how many on system running pipeline?

ValueRaider on Sep 22, 2023

Confirmed. Fails on my CI pipeline in python <=3.10, works on my mac with 3.11.

enzbus on Sep 22, 2023

I can’t debug until tomorrow, but here’s summary in case someone else can.

Previous version was 99% thread-safe. In rare cases the thread-safety failed.

Latest version switches to peewee module. Supposed to be 100% thread-safe, and I tested on few threads. Maybe it’s configured wrong?

ValueRaider on Sep 22, 2023

Run again debug logging enabled. Like the bug report template instructed.

ValueRaider on Sep 22, 2023

i added, tell me of this is enough, it’s fail with 0.2.29, but ok with 0.2.28