HowLongToBeat-PythonAPI: All searches returning None (all requests returning 404)
I was hoping to integrate this into Goodplays, unfortunately I haven’t been able to get it working. Every search returns None, apparently due to a 404.
For example:
>>> from howlongtobeatpy import HowLongToBeat
>>> HowLongToBeat(0.0).search("Outer Wilds")
returns None. When I modify HTMLRequests.py to spit out the response text on non-200 responses I get a 404 page:
<!DOCTYPE html><html lang="en"><head><meta charSet="utf-8"/><meta name="viewport" content="width=device-width"/><title>HowLongToBeat.com | Game Lengths, Backlogs and more!</title><meta name="theme-color" content="#000000"/><meta name="description" content="How long are your favorite video games? HowLongToBeat has the answer. Create a backlog, submit your game times and compete with your friends!"/><meta name="robots" content="noodp, noydir"/><meta name="thumbnail" content="https://howlongtobeat.com/img/hltb_brand2.png"/><link rel="canonical" href="https://howlongtobeat.comundefined"/><meta property="twitter:url" content="https://howlongtobeat.comundefined"/><meta property="og:url" content="https://howlongtobeat.comundefined"/><meta property="og:title" content="HowLongToBeat.com | Game Lengths, Backlogs and more!"/><meta property="og:type" content="website"/><meta property="og:image" content="https://howlongtobeat.com/img/hltb_brand2.png"/><meta property="og:description" content="How long are your favorite video games? HowLongToBeat has the answer. Create a backlog, submit your game times and compete with your friends!"/><meta name="twitter:card" content="summary"/><meta name="twitter:description" content="How long are your favorite video games? HowLongToBeat has the answer. Create a backlog, submit your game times and compete with your friends!"/><meta property="twitter:domain" content="howlongtobeat.com"/><meta name="twitter:site" content="@HowLongToBeat"/><meta name="twitter:image" content="https://howlongtobeat.com/img/hltb_brand2.png"/><meta name="next-head-count" content="19"/><link rel="apple-touch-icon" sizes="57x57" href="https://howlongtobeat.com/img/icons/apple-touch-icon-57x57.png"/><link rel="apple-touch-icon" sizes="60x60" href="https://howlongtobeat.com/img/icons/apple-touch-icon-60x60.png"/><link rel="apple-touch-icon" sizes="72x72" href="https://howlongtobeat.com/img/icons/apple-touch-icon-72x72.png"/><link rel="apple-touch-icon" sizes="76x76" href="https://howlongtobeat.com/img/icons/apple-touch-icon-76x76.png"/><link rel="apple-touch-icon" sizes="114x114" href="https://howlongtobeat.com/img/icons/apple-touch-icon-114x114.png"/><link rel="apple-touch-icon" sizes="120x120" href="https://howlongtobeat.com/img/icons/apple-touch-icon-120x120.png"/><link rel="apple-touch-icon" sizes="144x144" href="https://howlongtobeat.com/img/icons/apple-touch-icon-144x144.png"/><link rel="apple-touch-icon" sizes="152x152" href="https://howlongtobeat.com/img/icons/apple-touch-icon-152x152.png"/><link rel="apple-touch-icon" sizes="180x180" href="https://howlongtobeat.com/img/icons/apple-touch-icon-180x180.png"/><link rel="icon" type="image/png" href="https://howlongtobeat.com/img/icons/favicon-32x32.png" sizes="32x32"/><link rel="icon" type="image/png" href="https://howlongtobeat.com/img/icons/android-chrome-192x192.png" sizes="192x192"/><link rel="icon" type="image/png" href="https://howlongtobeat.com/img/icons/favicon-96x96.png" sizes="96x96"/><link rel="icon" type="image/png" href="https://howlongtobeat.com/img/icons/favicon-16x16.png" sizes="16x16"/><link rel="manifest" href="/manifest.json"/><link rel="preconnect" href="https://howlongtobeat.com"/><link rel="preload" as="script" href="https://cdn.ziffstatic.com/pg/howlongtobeat.js"/><script type="text/javascript" id="pogo" src="https://cdn.ziffstatic.com/pg/howlongtobeat.js" async=""></script><link rel="stylesheet" href="https://cdn.ziffstatic.com/pg/howlongtobeat.css"/><link rel="preload" as="script" href="https://cdn.ziffstatic.com/pg/howlongtobeat.prebid.js"/><script crossorigin="true" src="https://cdn.ziffstatic.com/jst/zdconsent.js" async=""></script><link rel="preload" href="/_next/static/css/228c4b801d7021a9.css" as="style"/><link rel="stylesheet" href="/_next/static/css/228c4b801d7021a9.css" data-n-g=""/><noscript data-n-css=""></noscript><script defer="" nomodule="" src="/_next/static/chunks/polyfills-c67a75d1b6f99dc8.js"></script><script src="/_next/static/chunks/webpack-2efae08fba7c9b69.js" defer=""></script><script src="/_next/static/chunks/framework-d51ece3d757c7ed2.js" defer=""></script><script src="/_next/static/chunks/main-c908647eeec19f62.js" defer=""></script><script src="/_next/static/chunks/pages/_app-c2db2970f545cae6.js" defer=""></script><script src="/_next/static/chunks/pages/404-c06b07ed3ab0773c.js" defer=""></script><script src="/_next/static/JCeqWkOAftOy_JRL-i2FJ/_buildManifest.js" defer=""></script><script src="/_next/static/JCeqWkOAftOy_JRL-i2FJ/_ssgManifest.js" defer=""></script></head><body><div id="__next"><script>!function(){try{var d=document.documentElement,n='data-theme',s='setAttribute';var e=localStorage.getItem('theme');if('system'===e||(!e&&true)){var t='(prefers-color-scheme: dark)',m=window.matchMedia(t);if(m.media!==t||m.matches){d.style.colorScheme = 'dark';d[s](n,'dark')}else{d.style.colorScheme = 'light';d[s](n,'light')}}else if(e){d[s](n,e|| '')}if(e==='light'||e==='dark')d.style.colorScheme=e}catch(e){}}()</script><div class="Layout_container__V2eEE"><header class="MainNavigation_header__WuiTa"><nav class="MainNavigation_nav__LkHHd"><a class="MainNavigation_brand__8YjKY" aria-label="HowLongToBeat" href="/"></a><ul class="MainNavigation_list__xBZrm"><li><a href="/forum">Forum</a></li><li><a href="/stats">Stats</a></li><li><a href="/submit">Submit</a></li></ul><ul class="MainNavigation_login__KE7zX"><li><a class="text_primary" href="/login">Login</a></li><li class="MainNavigation_join_link__4bsgx"><a class="mobile_hide text_primary" href="/login/signup">Join</a></li></ul><div class="MainNavigation_search__kw6St"><input class="MainNavigation_search_box__jDUWW back_form" aria-label="Search" tabindex="2" type="text" placeholder="Search Your Favorite Games..." autoComplete="off"/></div></nav></header><main class="Layout_main__NgJgX"><div class="back_dark" id="global_site" style="display:block;border-top:1px solid transparent"><div class="contain_out"><div class="contain_in index_padding"><div class="content_100 center"><div class="global_padding_big"><h1 class="global_padding"><span class="mobile_hide">Error</span> 404 - Not Found</h1></div></div><div class="content_100 center"><img src="https://howlongtobeat.com/img/404/pong.gif" style="width:100%" alt="404"/></div><div class="content_100 center"><p class="in">Sorry! The page you are looking for does not exist. Try going back or visiting a different link.</p></div></div></div></div></main><footer class="Footer_footer__2MMdT back_primary"><div class="Footer_footer_inside__UfjFE"><div class="Footer_footer_links__GA8Gc"><h3>HowLongToBeat</h3><ul><li><a href="/feedback">Contact Us</a></li><li><a href="/conduct">Code of Conduct</a></li><li><a href="/privacy">Privacy Policy</a></li></ul></div><div class="Footer_footer_links__GA8Gc"><h3>Social</h3><ul><li><a href="https://discord.gg/v5F26Dk" target="_blank" rel="noreferrer">Discord</a></li><li><a href="https://facebook.com/HowLongToBeat/" target="_blank" rel="noreferrer">Facebook</a></li><li><a href="https://twitter.com/HowLongToBeat/" target="_blank" rel="noreferrer">Twitter</a></li></ul></div><div class="Footer_footer_copyright__TDPg5"><h3>© 2022 Ziff Davis, LLC</h3><ul><li>Powered By Community, Built With Love</li><li><a href="#" class="showConsentTool"><img alt="AdChoices Icon" src="https://c.evidon.com/pub/icong1.png" class="evidon-consent-link-image" style="vertical-align:middle" width="14" height="18"/> <!-- -->AdChoices</a></li><li></li></ul></div></div></footer></div></div><script id="__NEXT_DATA__" type="application/json">{"props":{"pageProps":{"pageMetadata":{"noTopAd":true}},"__N_SSG":true},"page":"/404","query":{},"buildId":"JCeqWkOAftOy_JRL-i2FJ","isFallback":false,"gsp":true,"scriptLoader":[]}</script></body></html>
(I replicated this by manually posting against the URL.)
From what I can tell, it looks like they’ve changed the posting endpoint to https://www.howlongtobeat.com/api/search, reformatted it slightly, and are actually returning JSON now, which should make parsing it a hell of a lot easier. I haven’t been following this closely, but perhaps this change corresponds to the GamePass integration they’ve just implemented.
Unfortunately, I’m unable to get it to actually authorize (everything I post to the endpoint gives me a 401). For example:
>>> import requests
>>> requests.post('https://www.howlongtobeat.com/api/search', json={'searchType': 'games', 'searchTerms': ['OUTER', 'WILDS'], 'searchPage': 1, 'size': 20}, headers={'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.102 Safari/537.36 OPR/90.0.4480.54', 'content-type': 'application/json', 'origin': 'https://howlongtobeat.com', 'referrer': 'https://howlongtobeat.com/', 'accept': '*/*'})
<Response [401]>
Python version:
Python 3.6.10 (default, Dec 19 2019, 23:04:32)
[GCC 5.4.0 20160609] on linux
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 18 (11 by maintainers)
Version 1.0.1 is now available!
https://github.com/ScrappyCocco/HowLongToBeat-PythonAPI/releases/tag/1.0.1
For any problem feel free to open an issue, a discussion or even a pull request! The code is easier to edit now that is all JSON
Thank you for using this API
Will update this issue when the new release is live
Got it, I’ll start to work on it today and will update this Issue as soon as I have a working example
@Kononkov1998 @kparal @spurll so it would be ok to completely redo the parsing and then have all of this in the result struct?
This would require more end-user management about what values to pick, but the API would just expose all the values HLTB gives us, so it’s up to the api-user what values to pick and how to convert them (time is in second but it would be up to the user how to convert and how to approximate it)
This would also make this API code much easier as I just parse the JSON to put the values in the struct, so it would be easy to add/remove or edit a new/removed value; while keep track of labels, conversions and such could become a problem as different users can have different needs