yfinance: HTTPError: 401 Client Error: Unauthorized for url

I was able to fix it by upgrading yfinance: pip install --upgrade yfinance hope it works for someone

What error I extracted sucessfully data that needed for my report with version 24

@decadecity Your experience is uniquely interesting, because 0.2.23 should only be returning “summaryDetail” module = ~37 values instead of ~130

Anyway I’ve created a PR #1603

@ValueRaider Apologies, I misunderstood you: I thought that was your WIP branch but as there was no PR raised for it I thought you were still looking for a fix. My bad.

As the info dict from 0.2.23 contatines all the data I’m after I don’t know what we’re expecting to see here. I’ve run your branch locally and this is what I get:

>>> import yfinance as yf
>>> yf.Ticker('AAPL').info
{'address1': 'One Apple Park Way', 'city': 'Cupertino', 'state': 'CA', 'zip': '95014', 'country': 'United States', 'phone': '408 996 1010', 'website': 'https://www.apple.com', 'industry': 'Consumer Electronics', 'industryDisp': 'Consumer Electronics', 'sector': 'Technology', 'longBusinessSummary': 'Apple Inc. designs, manufactures, and markets smartphones, personal computers, tablets, wearables, and accessories worldwide. The company offers iPhone, a line of smartphones; Mac, a line of personal computers; iPad, a line of multi-purpose tablets; and wearables, home, and accessories comprising AirPods, Apple TV, Apple Watch, Beats products, and HomePod. It also provides AppleCare support and cloud services; and operates various platforms, including the App Store that allow customers to discover and download applications and digital content, such as books, music, video, games, and podcasts. In addition, the company offers various services, such as Apple Arcade, a game subscription service; Apple Fitness+, a personalized fitness service; Apple Music, which offers users a curated listening experience with on-demand radio stations; Apple News+, a subscription news and magazine service; Apple TV+, which offers exclusive original content; Apple Card, a co-branded credit card; and Apple Pay, a cashless payment service, as well as licenses its intellectual property. The company serves consumers, and small and mid-sized businesses; and the education, enterprise, and government markets. It distributes third-party applications for its products through the App Store. The company also sells its products through its retail and online stores, and direct sales force; and third-party cellular network carriers, wholesalers, retailers, and resellers. Apple Inc. was incorporated in 1977 and is headquartered in Cupertino, California.', 'fullTimeEmployees': 164000, 'companyOfficers': [{'maxAge': 1, 'name': 'Mr. Timothy D. Cook', 'age': 61, 'title': 'CEO & Director', 'yearBorn': 1961, 'fiscalYear': 2022, 'totalPay': 16425933, 'exercisedValue': 0, 'unexercisedValue': 0}, {'maxAge': 1, 'name': 'Mr. Luca  Maestri', 'age': 59, 'title': 'CFO & Sr. VP', 'yearBorn': 1963, 'fiscalYear': 2022, 'totalPay': 5019783, 'exercisedValue': 0, 'unexercisedValue': 0}, {'maxAge': 1, 'name': 'Mr. Jeffrey E. Williams', 'age': 58, 'title': 'Chief Operating Officer', 'yearBorn': 1964, 'fiscalYear': 2022, 'totalPay': 5018337, 'exercisedValue': 0, 'unexercisedValue': 0}, {'maxAge': 1, 'name': 'Ms. Katherine L. Adams', 'age': 58, 'title': 'Sr. VP, Gen. Counsel & Sec.', 'yearBorn': 1964, 'fiscalYear': 2022, 'totalPay': 5015208, 'exercisedValue': 0, 'unexercisedValue': 0}, {'maxAge': 1, 'name': "Ms. Deirdre  O'Brien", 'age': 55, 'title': 'Sr. VP of Retail', 'yearBorn': 1967, 'fiscalYear': 2022, 'totalPay': 5019783, 'exercisedValue': 0, 'unexercisedValue': 0}, {'maxAge': 1, 'name': 'Mr. Chris  Kondo', 'title': 'Sr. Director of Corp. Accounting', 'exercisedValue': 0, 'unexercisedValue': 0}, {'maxAge': 1, 'name': 'Mr. James  Wilson', 'title': 'Chief Technology Officer', 'exercisedValue': 0, 'unexercisedValue': 0}, {'maxAge': 1, 'name': 'Ms. Mary  Demby', 'title': 'Chief Information Officer', 'exercisedValue': 0, 'unexercisedValue': 0}, {'maxAge': 1, 'name': 'Ms. Nancy  Paxton', 'title': 'Sr. Director of Investor Relations & Treasury', 'exercisedValue': 0, 'unexercisedValue': 0}, {'maxAge': 1, 'name': 'Mr. Greg  Joswiak', 'title': 'Sr. VP of Worldwide Marketing', 'exercisedValue': 0, 'unexercisedValue': 0}], 'auditRisk': 4, 'boardRisk': 1, 'compensationRisk': 5, 'shareHolderRightsRisk': 1, 'overallRisk': 1, 'governanceEpochDate': 1688169600, 'compensationAsOfEpochDate': 1672444800, 'maxAge': 86400, 'priceHint': 2, 'previousClose': 190.54, 'open': 190.23, 'dayLow': 190.1, 'dayHigh': 191.175, 'regularMarketPreviousClose': 190.54, 'regularMarketOpen': 190.23, 'regularMarketDayLow': 190.1, 'regularMarketDayHigh': 191.175, 'dividendRate': 0.96, 'dividendYield': 0.0050999997, 'exDividendDate': 1683849600, 'payoutRatio': 0.1559, 'fiveYearAvgDividendYield': 0.89, 'beta': 1.292349, 'trailingPE': 32.227657, 'forwardPE': 29.044073, 'volume': 8928348, 'regularMarketVolume': 8928348, 'averageVolume': 56677663, 'averageVolume10days': 51027070, 'averageDailyVolume10Day': 51027070, 'bid': 190.73, 'ask': 190.74, 'bidSize': 2200, 'askSize': 900, 'marketCap': 3005911924736, 'fiftyTwoWeekLow': 124.17, 'fiftyTwoWeekHigh': 194.48, 'priceToSalesTrailing12Months': 7.8056374, 'fiftyDayAverage': 180.4268, 'twoHundredDayAverage': 156.0571, 'trailingAnnualDividendRate': 0.92, 'trailingAnnualDividendYield': 0.0048283827, 'currency': 'USD', 'enterpriseValue': 3038578737152, 'profitMargins': 0.24493, 'floatShares': 15711872289, 'sharesOutstanding': 15728700416, 'sharesShort': 117870227, 'sharesShortPriorMonth': 117654782, 'sharesShortPreviousMonthDate': 1685491200, 'dateShortInterest': 1688083200, 'sharesPercentSharesOut': 0.0075, 'heldPercentInsiders': 0.00071000005, 'heldPercentInstitutions': 0.61232, 'shortRatio': 1.87, 'shortPercentOfFloat': 0.0075, 'impliedSharesOutstanding': 15728700416, 'bookValue': 3.953, 'priceToBook': 48.34556, 'lastFiscalYearEnd': 1663977600, 'nextFiscalYearEnd': 1695513600, 'mostRecentQuarter': 1680307200, 'earningsQuarterlyGrowth': -0.034, 'netIncomeToCommon': 94321000448, 'trailingEps': 5.93, 'forwardEps': 6.58, 'pegRatio': 4.0, 'lastSplitFactor': '4:1', 'lastSplitDate': 1598832000, 'enterpriseToRevenue': 7.89, 'enterpriseToEbitda': 24.547, '52WeekChange': 26.882862, 'SandP52WeekChange': 16.74484, 'lastDividendValue': 0.24, 'lastDividendDate': 1683849600, 'exchange': 'NMS', 'quoteType': 'EQUITY', 'symbol': 'AAPL', 'underlyingSymbol': 'AAPL', 'shortName': 'Apple Inc.', 'longName': 'Apple Inc.', 'firstTradeDateEpochUtc': 345479400, 'timeZoneFullName': 'America/New_York', 'timeZoneShortName': 'EDT', 'uuid': '8b10e4ae-9eeb-3684-921a-9ab27e4d87aa', 'messageBoardId': 'finmb_24937', 'gmtOffSetMilliseconds': -14400000, 'currentPrice': 191.11, 'targetHighPrice': 240.0, 'targetLowPrice': 140.0, 'targetMeanPrice': 188.98, 'targetMedianPrice': 190.0, 'recommendationMean': 2.0, 'recommendationKey': 'buy', 'numberOfAnalystOpinions': 39, 'totalCash': 55872000000, 'totalCashPerShare': 3.552, 'ebitda': 123788001280, 'totalDebt': 109614997504, 'quickRatio': 0.764, 'currentRatio': 0.94, 'totalRevenue': 385095008256, 'debtToEquity': 176.349, 'revenuePerShare': 24.116, 'returnOnAssets': 0.20559, 'returnOnEquity': 1.4560499, 'grossProfits': 170782000000, 'freeCashflow': 83796623360, 'operatingCashflow': 109583998976, 'earningsGrowth': 0.0, 'revenueGrowth': -0.025, 'grossMargins': 0.43181, 'ebitdaMargins': 0.32145, 'operatingMargins': 0.29163, 'financialCurrency': 'USD', 'trailingPegRatio': 2.618}

@emmaai My branch does that, see https://github.com/ranaroussi/yfinance/compare/main...hotfix/info-missing-values . And for me this appears to fix, many more values now, I just wanted confirmation in case still problems.

Why are people fixing this instead of reading the thread? I’ve already fixed it, just waiting for someone to verify.

Am I muted?

coz the code on the branch doesn’t work re missing info values, at least for me… https://github.com/ranaroussi/yfinance/issues/1592#issuecomment-1635869193 from @decadecity is the right answer and it took only one line fix params_dict["modules"] = modules

Why are people fixing this instead of reading the thread? I’ve already fixed it, just waiting for someone to verify.

Am I muted?

How’s this:

import urllib.parse

modules = ['summaryProfile', 'financialData', 'quoteType',
                     'defaultKeyStatistics', 'assetProfile', 'summaryDetail']
urllib.parse.urlencode([('ssl', 'true')] + [('module', m) for m in modules])

Results in:

'ssl=true&module=summaryProfile&module=financialData&module=quoteType&module=defaultKeyStatistics&module=assetProfile&module=summaryDetail'

May be we should use this: params_dict = ‘ssl=true’ for m in modules: params_dict += f’&modules={m}’

@ValueRaider I used curl to check, and I was able to retrieve the data.

To get the crumb: curl --header "Cookie: cookie_that_i_extracted_from_the_browser" https://query1.finance.yahoo.com/v1/test/getcrumb

To get the data: curl --header "Cookie: cookie_that_i_extracted_from_the_browser" https://query1.finance.yahoo.com/v10/finance/quoteSummary?modules=price,summaryDetail&crumb=crumb_that_i_just_got

@pbenaim You’re right, info is missing data. We should have tested more. Can people try Git branch hotfix/info-missing-values - looks good to me, but worth checking

@ValueRaider @signifer-geo v0.2.23 is working for me 👍🏻

@ValueRaider Initially yes but once I’d used that to figure out the minimum information required I was able to do it anonymously in a private browsing window with a completely clean session.

I got the cookie by visiting finances.yahoo.com in a browser and clicked through the consent dialogue which sends a POST to consent.yahoo.com which returns a 302 with the Set-Cookie for the A1. I got the crumb from window.YAHOO.context.crumb (which I found from your comment).

With those two I could then put them into a command line curl and it returned the data:

curl 'https://query2.finance.yahoo.com/v10/finance/quoteSummary/AAPL?modules=summaryProfile%2CfinancialData%2CquoteType%2CdefaultKeyStatistics%2CassetProfile%2CsummaryDetail&ssl=true&crumb={{ crumb }}' -H 'cookie: A1={{ A1 cookie }}'

I’ve not looked into how to do this in Python.

@lapis42 @decadecity Could you 2 elaborate on how you obtained and provided cookie & crumb?

@ValueRaider Yes, I was able to retrieve the data after providing crumb with cookie.

Done a bit of digging around with curl and browser requests and as a minimum it seems like it needs the A1 cookie and the crumb URL parameter.

@decadecity I have code that can extract crumb, might help. I’ve never used it in requests.

import requests, re, json
def get_crumb():
    response = requests.get("https://finance.yahoo.com")
    pat = re.compile(r'window\.YAHOO\.context = ({.*?});', re.DOTALL)
    match = re.search(pat, response.text)
    if match:
        js_dict = json.loads(match.group(1))
        return js_dict.get('crumb')
    return None

If I use the URL in the original message

Using curl I get:

{"finance":{"result":null,"error":{"code":"Unauthorized","description":"Invalid Cookie"}}}

If I do the same in a browser that has a logged in session to Yahoo! finance it returns:

{"finance":{"result":null,"error":{"code":"Unauthorized","description":"Invalid Crumb"}}}

Well obviously this is a problem. Let’s keep thread tidy and constructive now.

@lapis42 You know that because you have it working with cookie and crumb?

It seems “v10 quoteSummary” also needs cookie and crumb.