yahooquery: Failed to get crumb - too many 429 error responses
Getting the above error consistently. Just started this morning - 6-May-25 EDT
To Reproduce
from yahooquery import Ticker
import UserAgent
# Generate some data
syms = 'ntc,goog,qcom,msft,gsg,meta,aapl,rtx,avgo,asml,bac,jpm,KKr,OHi,tgt,hon,nvda,ups,hack,cost,amzn,abbv,smtc'.upper().split(',')
userAgent = UserAgent().random
tics = Ticker(syms, user_agent=userAgent)
hist = tics.price
print(hist)
```**Expected behavior**
List of the price data
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**
-Running on Windows 11, WSL2 - Ubuntu 22.04.5 LTS
``
Users of yfinance are seeing the same problem. One suggestion there is to import from curl+cffi:
from curl_cffi import requests
import yfinance as yf
session = requests.Session(impersonate="chrome")
ticker = yf.Ticker('...', session=session)
Could this be done in yahooquery too?
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 5
- Comments: 52
Commits related to this issue
- fix: solve 429 HTTP error Solution from https://github.com/dpguthrie/yahooquery/issues/315#issuecomment-2857525226 — committed to gmasse/xbar-plugin-yf-stock-ticker by gmasse 2 years ago
@aggieed97 you could do something similar, just initiate the session and pass it along to the Ticker constructor
I’ve just published a new version of ybankinplay — a fork based on yahooquery that fixes several critical issues and improves compatibility:
The project is open-source and available on GitHub: github.com/jirisarri10/ybankinplay
To install it, just run:
pip install ybankinplay
It works just like yahooquery, so you can drop it in seamlessly. Feel free to open issues or share suggestions — I’d love to hear your feedback!
Un saludo desde España 🇪🇸
Javier Irisarri
Yes, a simple fix. He has too many options. I went into the source (ybankinplay/utils/impersonate.py) and change it read: IMPERSONATE= [ “chrome99”, “chrome100”, “chrome101”, “chrome104”, “chrome107”, “chrome110”, “chrome116”, “chrome119”, “chrome120”, “chrome123”, “chrome124”, “chrome131”, “chrome133a”, “chrome99_android”, “chrome131_android”, “safari15_3”, “safari15_5”, “safari17_0”, “safari18_0”, “safari17_2_ios”, “safari18_0_ios”, “edge99”, “edge101” ]
This is working fine for me.
Charlie
Hello,
I have been facing issues using the yahooquery library to retrieve data from Yahoo Finance. The error I keep getting is:
Failed to obtain crumb. Ability to retrieve data will be significantly limited.
After some investigation, I found that the library is not able to retrieve the “crumb” correctly for making requests to Yahoo Finance. However, I found a temporary solution:
It seems the issue lies with how the library handles “crumb” retrieval. Since this manual method works, it would be helpful if the library could implement a way to retrieve the “crumb” in a similar manner.
Is there any plan to address this in a future version of yahooquery? Or, if possible, could the documentation be updated to show how to manually retrieve the “crumb”?
Thank you for your work on the library; it’s been really useful!
Best regards,
it is fixed!!! a new release updated!!! thanks
With deep gratitude, I want to sincerely thank this user and all those who have contributed—and continue to contribute—their time, knowledge, and dedication to this project; without your generosity and commitment, none of this would be possible.
Dear @jirisarri10 and all others who contributed here. Thanks a LOT! Without you, it would take me ages to figure out! Ahmet
i just uploaded a new release, fixed IMPERSONATE!!! Thanks. You can write in jirisarri/ybankinplay en Github.
resulted in
Used a fresh project in python 3.12 only with venv and requirements loaded.
The warning I identified in ybankinplay, seem easy to fix:
ybankinplay/ticker.py (lines 1331-1334):
and the fix (mentioned also above in the context of yahooquery in the respective file but different line):
ybankinplay/utils/__init__.py, line 159has_live_indice = index_utc[-1] >= last_trade - pd.Timedelta(2, "s")The good news, is that ybankinplay can be installed on python 3.13 avoiding the lxml conflicts yahooquery had for P3.13 (Thanks!)
Many thanks. At Sunday i hope to upgrade pandas support. But is not there Pandas 3.0? I think the last versión pandas is 2.2.3.
chfiii,
I tried ybankinplay and I received an error: ImpersonateError(‘Impersonating edge135 is not supported’). I am using macOS and did not have the Edge browser installed.
I installed Edge browser and it seems to not give that same error anymore, but it still fails and the warning “Some characters could not be decoded, and were replaced with REPLACEMENT CHARACTER.”
Do you know a fix?
Thanks for ybankinplay by the way!
Javier:
Checkout the Issue I just opened on the ybankinplay project. It needs a pair of simple fixes in ticker.py to handle Pandas 3 warnings.
Thanks for your packaging up all the needed changes to the yaahooquery project. It seems to have been abandoned.
Charlie
@james-stevens Thank you for your workaround. For me it is working fine. Even the performance is as usual. For me the problem is solved.
We’re getting a lot here essentially for free. Just compare it with the API limits of paid plans like AlphaVantage and try to replicate this level of performance.
I only pull about 20 Tickers every 5 mins & didn’t see any performance hit for this work-around, but my service just runs in the background records the results to Prometheus, so my UI wouldn’t see any hit from an extra second or two. Prometheus is currently reporting 2.1s per poll - I poll the
priceandmajor_holdersfor all tickers.The history of both YahooFinance & YahooQuery is a constant arms race against Yahoo blocking or throttling access.
It’s a shame Yahoo don’t offer a consumer grade subscription at a reasonable price - I’d be happy to pay, but they only seem to have plans priced for Pros - tho I’ve not really researched it recently as YahooQuery does the job!!
Actually, the workaround is quite simple. Use curl_cffi package and add a session.
Unfortunately, the download speed drops by 90%. I’m also curious—will the rate limit eventually be lifted, or is this workaround likely to be detected and blocked again?