yfinance: Tutorial crashes

Describe bug

This tutorial code crashes now:

import yfinance
ticker = yfinance.Ticker("MSFT")
ticker.info

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[1], line 3
      1 import yfinance
      2 ticker = yfinance.Ticker("MSFT")
----> 3 ticker.info

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/ticker.py:142, in Ticker.info(self)
    140 @property
    141 def info(self) -> dict:
--> 142     return self.get_info()

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/base.py:1736, in TickerBase.get_info(self, proxy)
   1734 def get_info(self, proxy=None) -> dict:
   1735     self._quote.proxy = proxy
-> 1736     data = self._quote.info
   1737     return data

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/scrapers/quote.py:572, in Quote.info(self)
    569 @property
    570 def info(self) -> dict:
    571     if self._info is None:
--> 572         self._fetch(self.proxy)
    573         self._fetch_complementary(self.proxy)
    575     return self._info

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/scrapers/quote.py:602, in Quote._fetch(self, proxy)
    600 modules = ','.join(modules)
    601 params_dict = {"modules": modules, "ssl": "true"}
--> 602 result = self._data.get_raw_json(
    603     _BASIC_URL_ + f"/{self._symbol}", params=params_dict, proxy=proxy
    604 )
    605 result["quoteSummary"]["result"][0]["symbol"] = self._symbol
    606 query1_info = next(
    607     (info for info in result.get("quoteSummary", {}).get("result", []) if info["symbol"] == self._symbol),
    608     None,
    609 )

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/data.py:395, in YfData.get_raw_json(self, url, user_agent_headers, params, proxy, timeout)
    393 def get_raw_json(self, url, user_agent_headers=None, params=None, proxy=None, timeout=30):
    394     utils.get_yf_logger().debug(f'get_raw_json(): {url}')
--> 395     response = self.get(url, user_agent_headers=user_agent_headers, params=params, proxy=proxy, timeout=timeout)
    396     response.raise_for_status()
    397     return response.json()

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/utils.py:108, in log_indent_decorator.<locals>.wrapper(*args, **kwargs)
    105 logger.debug(f'Entering {func.__name__}()')
    107 with IndentationContext():
--> 108     result = func(*args, **kwargs)
    110 logger.debug(f'Exiting {func.__name__}()')
    111 return result

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/data.py:346, in YfData.get(self, url, user_agent_headers, params, proxy, timeout)
    343 if 'crumb' in params:
    344     raise Exception("Don't manually add 'crumb' to params dict, let data.py handle it")
--> 346 cookie, crumb, strategy = self._get_cookie_and_crumb()
    347 if crumb is not None:
    348     crumbs = {'crumb': crumb}

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/utils.py:108, in log_indent_decorator.<locals>.wrapper(*args, **kwargs)
    105 logger.debug(f'Entering {func.__name__}()')
    107 with IndentationContext():
--> 108     result = func(*args, **kwargs)
    110 logger.debug(f'Exiting {func.__name__}()')
    111 return result

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/data.py:322, in YfData._get_cookie_and_crumb(self, proxy, timeout)
    319         cookie, crumb = self._get_cookie_and_crumb_basic(proxy, timeout)
    320 else:
    321     # Fallback strategy
--> 322     cookie, crumb = self._get_cookie_and_crumb_basic(proxy, timeout)
    323     if cookie is None or crumb is None:
    324         # Fail
    325         self._set_cookie_strategy('csrf', have_lock=True)

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/utils.py:108, in log_indent_decorator.<locals>.wrapper(*args, **kwargs)
    105 logger.debug(f'Entering {func.__name__}()')
    107 with IndentationContext():
--> 108     result = func(*args, **kwargs)
    110 logger.debug(f'Exiting {func.__name__}()')
    111 return result

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/data.py:213, in YfData._get_cookie_and_crumb_basic(self, proxy, timeout)
    211 @utils.log_indent_decorator
    212 def _get_cookie_and_crumb_basic(self, proxy, timeout):
--> 213     cookie = self._get_cookie_basic(proxy, timeout)
    214     crumb = self._get_crumb_basic(proxy, timeout)
    215     return cookie, crumb

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/data.py:157, in YfData._get_cookie_basic(self, proxy, timeout)
    154     utils.get_yf_logger().debug('reusing cookie')
    155     return self._cookie
--> 157 self._cookie = self._load_cookie_basic()
    158 if self._cookie is not None:
    159     return self._cookie

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/data.py:143, in YfData._load_cookie_basic(self)
    142 def _load_cookie_basic(self):
--> 143     cookie_dict = cache.get_cookie_cache().lookup('basic')
    144     if cookie_dict is None:
    145         return None

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/cache.py:362, in _CookieCache.lookup(self, strategy)
    360     data =  _CookieSchema.get(_CookieSchema.strategy == strategy)
    361     cookie = _pkl.loads(data.cookie_bytes)
--> 362     return {'cookie':cookie, 'age':_datetime.datetime.now()-data.fetch_date}
    363 except _CookieSchema.DoesNotExist:
    364     return None

TypeError: unsupported operand type(s) for -: 'datetime.datetime' and 'str'

Simple code that reproduces your problem

See above

Debug log

DEBUG    get_raw_json(): https://query2.finance.yahoo.com/v10/finance/quoteSummary/MSFT
DEBUG    Entering get()
DEBUG     url=https://query2.finance.yahoo.com/v10/finance/quoteSummary/MSFT
DEBUG     params={'modules': 'financialData,quoteType,defaultKeyStatistics,assetProfile,summaryDetail', 'ssl': 'true'}
DEBUG     Entering _get_cookie_and_crumb()
DEBUG      cookie_mode = 'basic'
DEBUG      Entering _get_cookie_and_crumb_basic()
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[3], line 3
      1 import yfinance
      2 ticker = yfinance.Ticker("MSFT")
----> 3 ticker.info

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/ticker.py:142, in Ticker.info(self)
    140 @property
    141 def info(self) -> dict:
--> 142     return self.get_info()

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/base.py:1736, in TickerBase.get_info(self, proxy)
   1734 def get_info(self, proxy=None) -> dict:
   1735     self._quote.proxy = proxy
-> 1736     data = self._quote.info
   1737     return data

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/scrapers/quote.py:572, in Quote.info(self)
    569 @property
    570 def info(self) -> dict:
    571     if self._info is None:
--> 572         self._fetch(self.proxy)
    573         self._fetch_complementary(self.proxy)
    575     return self._info

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/scrapers/quote.py:602, in Quote._fetch(self, proxy)
    600 modules = ','.join(modules)
    601 params_dict = {"modules": modules, "ssl": "true"}
--> 602 result = self._data.get_raw_json(
    603     _BASIC_URL_ + f"/{self._symbol}", params=params_dict, proxy=proxy
    604 )
    605 result["quoteSummary"]["result"][0]["symbol"] = self._symbol
    606 query1_info = next(
    607     (info for info in result.get("quoteSummary", {}).get("result", []) if info["symbol"] == self._symbol),
    608     None,
    609 )

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/data.py:395, in YfData.get_raw_json(self, url, user_agent_headers, params, proxy, timeout)
    393 def get_raw_json(self, url, user_agent_headers=None, params=None, proxy=None, timeout=30):
    394     utils.get_yf_logger().debug(f'get_raw_json(): {url}')
--> 395     response = self.get(url, user_agent_headers=user_agent_headers, params=params, proxy=proxy, timeout=timeout)
    396     response.raise_for_status()
    397     return response.json()

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/utils.py:108, in log_indent_decorator.<locals>.wrapper(*args, **kwargs)
    105 logger.debug(f'Entering {func.__name__}()')
    107 with IndentationContext():
--> 108     result = func(*args, **kwargs)
    110 logger.debug(f'Exiting {func.__name__}()')
    111 return result

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/data.py:346, in YfData.get(self, url, user_agent_headers, params, proxy, timeout)
    343 if 'crumb' in params:
    344     raise Exception("Don't manually add 'crumb' to params dict, let data.py handle it")
--> 346 cookie, crumb, strategy = self._get_cookie_and_crumb()
    347 if crumb is not None:
    348     crumbs = {'crumb': crumb}

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/utils.py:108, in log_indent_decorator.<locals>.wrapper(*args, **kwargs)
    105 logger.debug(f'Entering {func.__name__}()')
    107 with IndentationContext():
--> 108     result = func(*args, **kwargs)
    110 logger.debug(f'Exiting {func.__name__}()')
    111 return result

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/data.py:322, in YfData._get_cookie_and_crumb(self, proxy, timeout)
    319         cookie, crumb = self._get_cookie_and_crumb_basic(proxy, timeout)
    320 else:
    321     # Fallback strategy
--> 322     cookie, crumb = self._get_cookie_and_crumb_basic(proxy, timeout)
    323     if cookie is None or crumb is None:
    324         # Fail
    325         self._set_cookie_strategy('csrf', have_lock=True)

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/utils.py:108, in log_indent_decorator.<locals>.wrapper(*args, **kwargs)
    105 logger.debug(f'Entering {func.__name__}()')
    107 with IndentationContext():
--> 108     result = func(*args, **kwargs)
    110 logger.debug(f'Exiting {func.__name__}()')
    111 return result

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/data.py:213, in YfData._get_cookie_and_crumb_basic(self, proxy, timeout)
    211 @utils.log_indent_decorator
    212 def _get_cookie_and_crumb_basic(self, proxy, timeout):
--> 213     cookie = self._get_cookie_basic(proxy, timeout)
    214     crumb = self._get_crumb_basic(proxy, timeout)
    215     return cookie, crumb

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/data.py:157, in YfData._get_cookie_basic(self, proxy, timeout)
    154     utils.get_yf_logger().debug('reusing cookie')
    155     return self._cookie
--> 157 self._cookie = self._load_cookie_basic()
    158 if self._cookie is not None:
    159     return self._cookie

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/data.py:143, in YfData._load_cookie_basic(self)
    142 def _load_cookie_basic(self):
--> 143     cookie_dict = cache.get_cookie_cache().lookup('basic')
    144     if cookie_dict is None:
    145         return None

File /opt/homebrew/Caskroom/mambaforge/base/envs/work/lib/python3.11/site-packages/yfinance/cache.py:362, in _CookieCache.lookup(self, strategy)
    360     data =  _CookieSchema.get(_CookieSchema.strategy == strategy)
    361     cookie = _pkl.loads(data.cookie_bytes)
--> 362     return {'cookie':cookie, 'age':_datetime.datetime.now()-data.fetch_date}
    363 except _CookieSchema.DoesNotExist:
    364     return None

TypeError: unsupported operand type(s) for -: 'datetime.datetime' and 'str'

Bad data proof

No response

`yfinance` version

0.2.32 (pip installed directly from Github GIT repository)

Python version

3.11.6

Operating system

MacOS Sonoma 14.1.1

About this issue

Original URL
State: closed
Created 7 months ago
Comments: 23

Most upvoted comments

Branch hotfix/cookie-cache-date has a workaround, try it.

It works ! Now that you have identified the culprit being the storage format in Sqlite, I can complete the missing pieces as I know this part well: 1- Sqlite has no true DATETIME storage. Even if the field type “seems” to exist, it’s just for Sqlite an alias for a string storage. 2-The mapping from a Python datetime to the stored string is performed by the Python sqlite3 driver. I don’t know if peewee relies on it or not but, based on the bug you elucidated, it seems it does replicate the same issue described below. 3-The sqlite3 driver offers (until Python 3.12, this will be removed in a future release) a default mapping that is the one you were expecting (with a space between the date and the time). 4-But I had in my dependencies tree a third-party library which enforces a strict ISO8601 mapping to make sure all stored dates can be parsed following the ISO norm in its own Sqlite database. The ISO norm uses a T between the date and the time

Note: the Python documentation is confusing because it states that its default mapping is ISO-compliant. But this is wrong. Using a space is not ISO compliant and is a tolerance stemming from RFC 3339, not supported by all tools. Strict ISO forbids the space, and requires a T.

The future for Python may favor strict ISO as the documentation announces default mappings will be removed, and offers as a compensation mapping examples strictly based on ISO (and not based on the RFC 3339 tolerance): https://docs.python.org/3/library/sqlite3.html#default-adapters-and-converters-deprecated https://docs.python.org/3/library/sqlite3.html#sqlite3-adapter-converter-recipes

So your workaround may be going in the right direction.

dmoklaf on Dec 21, 2023