core: MyQ fails to setup with 401 or 429 error
The problem
HA 2023.10.1 worked ok for a few days but now similar failure when trying a reload. See error log below
What version of Home Assistant Core has the issue?
2023.10.1
What was the last working version of Home Assistant Core?
No response
What type of installation are you running?
Home Assistant OS
Integration causing the issue
MyQ
Link to integration documentation on our website
No response
Diagnostics information
Logger: homeassistant.components.myq Source: helpers/update_coordinator.py:322 Integration: MyQ (documentation, issues) First occurred: 10:19:24 AM (1 occurrences) Last logged: 10:19:24 AM
Error fetching myq devices data: Error trying to re-authenticate to myQ service: Error requesting data from https://partner-identity.myq-cloud.com/connect/token: 401 - Unauthorized
Example YAML snippet
No response
Anything in the logs that might be useful for us?
Same issue as before but now with 401 instead of 403
Additional information
No response
About this issue
- Original URL
- State: closed
- Created 9 months ago
- Reactions: 156
- Comments: 237 (36 by maintainers)
Commits related to this issue
- Add changes from issue below to fix myq login https://github.com/home-assistant/core/issues/101763 — committed to dailow/Python-MyQ by dailow 9 months ago
- Add changes from issue below to fix myq login https://github.com/home-assistant/core/issues/101763 — committed to dailow/Python-MyQ by dailow 9 months ago
- fix: myq api auth (#22) MyQ recently updated their auth flow process. Updated the myq package dependency to account for changes and now bumping here to consume those updates. Hopefully it sticks an... — committed to brchri/tesla-youq by brchri 9 months ago
Hello everyone - I am closing this issue as it has been resolved with #101852 and will be included in 2023.10.2 #101871 I would highly recommend not doing any kind of manual solutions - and instead just wait for 10.2 to come out which will probably be later today.
I won’t ask for this thread to be locked, but please restrain from commenting so others can see this
As a reminder - please upvote the original post if you are having the same problem, do not add extra comments stating that you have the same problem - instead upvote the original post. I get buried in notifications and it is extremely difficult to separate helpful information and just chatter.
I don’t have a ton of time this week to try to look at it, but I will try my best. As a reminder to everyone - I don’t own one of these devices, and I am always looking for a new codeowner, I have had one user step forward and attempt to diagnose some problems, I will give them as much support as I can with my limited time this week and if they aren’t able to figure it out, I will try to solve what I can with my limited time
Going to see if I can capture what the iOS app is doing on login, there must be some detail that’s missing.
How did you manage to install HomeAssistant while being unable to read?
My MYQ just went 429 a minute ago
Fixed by updating api.py so that the call to get the OAuth token more closely represents a valid OAuth token call. Changes highlighted in yellow:
IOS_CGI_MYQ:
)*/*
client_id
,client_secret
, andscope
form fields since they’re not supposed to be sent on the get token call in the first place[0]
to the line extracting thecode
form field to get the code as a string rather than as an arrayAs text:
This might be a little easier.
Restart HA
Okay I got it working. The key I think is adding the following params to the login call:
Also changing the login POST call from
/Account/Login
to /Account/LoginWithEmail` is probably key. I’ll post a link to my updated Go package for reference when I’m finished with it. Shouldn’t be too far out, assuming it doesn’t break again.Did something change yesterday? I went to close my garage door in HA and it showed disconnected. I restarted HA and checked the MyQ app (on IOS) and that still works. I signed out of MyQ in HA and now when i try and sign back in it says “Failed to Connect”
Was working perfect after the core update till this morning
Ok, verified – the login code as it existed in Homebridge’s api_myq (and in my C# gist from earlier) works with three changes:
Brand
andUnifiedFlowRequested
), into the login POST even though they’re not in the HTMLReturnUrl
argument is in the HTML as a hidden field, but the code was previously written to exclude that from the login POST, so it was just a matter of no longer excluding itIOS_CGI_MYQ
toANDROID_CGI_MYQ
; and the callback url fromcom.myqops://ios
tocom.myqops://android
.Updated gist of C# code is here. Since it’s so close to the existing code this should be straightforward to integrate into python_myq too.
It is simultaneously a super annoying thing and a super fun thing. Feel free to message me on discord (Same username as here in HA discord) if you have any questions or if I can help you in anyway. Like I said I don’t have a ton of time right now to really dive into things, but I have enough time to answer questions
(Thrusting hips violently) “Corporate Greed!!”
I wouldn’t draw any conclusions from testing like that. They have Cloudflare’s bot protection on pretty sharply, so as soon as you start getting 429 errors, all bets are off until the heat on you cools down. The 429s aren’t really traditional rate limiting; it’s Cloudflare detecting a potentially automated access and throwing up a challenge page with a 429 status code.
There isn’t really a good workaround for that for Home Assistant, because even if you do an initial authentication request from a browser and grab some cookies to plug into HA (like some other integrations do), those cookies are only good for a relatively short period of time before Cloudflare will start challenging again; so unless you want to be authenticating manually every few hours it’s not worth doing.
Honestly, at this point, just tell everyone you know not to buy any MyQ devices, give them one star reviews on every online storefront you can find; and then wire up a Shelly or a ratdgo or some other roll-your-own solution and give up on using MyQ automation entirely. Short of a significant policy change with MyQ management, that’s going to be the only reliable solution.
@mstberto same for me… I disabled the integration when the “problem” reappeared on October 17. I just saw your post, re-enabled the MyQ integration. I’m back in business. until the next time 😉
edit 1: (one day later) it’s back!!! 429 errors. I’ve disable the MyQ integration and ordered 2 of these: https://www.athom.tech/blank-1/garage-door-opener-for-esphome … nice packaging, pre-flashed, No blinking warning light when closing.
I’ll follow along to see how this saga turns out, but my garage doors will work reliably from HA in the meantime.
Mine was not working yesterday but appears to be working again today.
Now that the patch has been merged, you should be able to just run this command from your HA install/container.
I’m running HA OS, not the docker. Follow instructions above to install Terminal.
I just did the above and I’m back online with MyQ as before.
Here are the steps I used to successfully sign in and use my garage door in the MyQ integration. Someone will probably make an easier guide with better commands but I’m not super well-verse with the CLI. I tried to make this as literal and noob-friendly as possible since I know there’s a lot of normies reading this thread wanting help.
Update: Simpler Method
Advanced SSH & Web Terminal Add-On
in the Official Add-On StoreInfo Page
of the Advanced SSH & Web Terminal, switch the “Protection Mode” switch to theOFF
position. (You can turn this back on when you’re done.)Open Web UI
(it may take a moment for the UI to actually load)docker exec -it homeassistant /bin/bash
pip install https://github.com/Python-MyQ/Python-MyQ/releases/download/v3.1.13/python_myq-3.1.13-py3-none-any.whl
My original comment
Advanced SSH & Web Terminal Add-On
in the Official Add-On Store.Info Page
of the Advanced SSH & Web Terminal, switch the “Protection Mode” switch to theOFF
position. (You can turn this back on when you’re done)Open Web UI
(it may take a moment for the UI to actually load)docker exec -it homeassistant bash
cd /usr/local/lib/python3.11/site-packages/pymyq/
vi api.py
i
on your keyboard to enter “insert mode”.401
. (it should be blank) (this will take awhile to scroll that far down)401
and402
respectively:data.update({"brand": "myq"})
data.update({"UnifiedFlowRequested": "True"})
Esc
key on your keyboard.:w
(literally hitshift + colon
and then hitw
) and then pressEnter
on your keyboard. This will write the changes you made to storage.:x
and then pressEnter
on your keyboard to exit the VI editor.vi const.py
.i
on your keyboard to enter “insert mode”.3
.3
fromOAUTH_CLIENT_ID = "IOS_CGI_MYQ"
toOAUTH_CLIENT_ID = "ANDROID_CGI_MYQ"
7
.7
fromOAUTH_REDIRECT_URI = "com.myqops://ios"
toOAUTH_REDIRECT_URI = "com.myqops://android"
Esc
key on your keyboard.:w
and then pressEnter
on your keyboard.:x
and then pressEnter
on your keyboard to exit the VI editor.You will now be able to sign into MyQ using the MyQ integration.
Long Live 3rd Party Integrations
Reverse engineering Android apps isn’t in my wheelhouse but no better time than the present to learn something new. I’m taking a stab at it, but don’t let that discourage anyone who actually already knows how to do this to do it instead.
I’m sorry to hear that. If it’s that critical for you then I would suggest reviewing the thread and going with one of the mentioned local only options. It’s probably better for you to prepare well ahead of time instead of MyQ completely pulling the plug and you scrambling because the integration is permanently broken. No idea if that’ll ever happen, but I’d personally prepare for that potential.
Yeah, but now they can double-dip by charging you for the garage door opener AND access via the app - it’s what they’re doing with Tesla, and what they’re doing with IFTTT. Their argument will be “You can ALWAYS just open the MyQ app and open/close for free, and you can ALWAYS fall back to the clicker/remote you put in your car.” If you want access to the API for any other reason, you have to pay.
The fact is that the HA integration (along with NodeRed, Homebridge, etc) are all reverse-engineered capabilities to get something for free. It’s only a matter of time before Chamberlain puts the API behind a tokenized URL and you have to subscribe to use it. I am personally not skilled enough in programming/coding to resolve this issue, so i’m putting my efforts into an alternate solution (ratgdo). I only use MyQ in HA to monitor status from work, and an automation to automatically close when no motion is detected in the garage after a period of time.
They’d be happy to charge us, like they do every other integration.
Can confirm the homebridge myq-api implementation works. There’s some stuff that needs to change in python-myq around authentication to mirror it – namely:
Python-myq also does some automatic user-agent generation that the homebridge implementation doesn’t do, but as far as I can tell, passing one doesn’t cause a failure.
I poked a bit at my local homeassistant to see if there was a quick fix that would get things working without being too impactful on the code and couldn’t get success, and I’m neither fluent enough in Python to do it well, nor set up for easy homeassistant dev/debugging work to tackle a bigger effort, so I’ll leave this in the hands of someone better positioned to work on it with hopefully the info above is a good starting point.
Also note that because they seem to require weird cookie behavior and non-standard arguments to the get token endpoint, tools that do standard OAuth stuff, like Postman, no longer work to log into the API (they did earlier today before the last round of API breakage). It must be done non-standard.
(And if it’s helpful, during my investigation I implemented a straightforward working login flow in C#, gist here.)
It doesn’t seem to be specific to HomeAssistant version. I rolled back to 2023.9.3 and the issue persists.
@home-assistant rename MyQ fails to setup with 401 or 429 error
The latest batch of 429 errors are from the https://partner-identity.myq-cloud.com/connect/authorize endpoint and appear to be Cloudflare anti-bot challenges, which require cookies and javascript to get past. It might be necessary to kick out to a browser to get past these.
Same. Things looked OK but went to 429 when I ‘reloaded’ the integration
My homebridge has the same issue
You’d want to add the params (data updates) here and replace the
url
line hereedit: Use @justinlindh 's PR below, apparently that’s a more up to date repo than pymyq.
can someone post where the lines in the api.py go so the rest of us can test this out please?
Can confirm that I’ve got this working after adding the changes that @brchri and @drysart mentioned above.
My changes are as follows (I can submit a PR later, if nobody else beats me to it):
const.py
:api.py
I can work on a proper PR later tonight, if nobody beats me to it (feel free to).
I was finally able to intercept the token call. So a couple of things to note about the current successful auth flow via (Android) app:
/connect/authorize
returns a 302 toLocation: /Account/Login
, which when followed returns a200
. I believe the subsequent form submission used to call to this/Account/Login
path, but now it’s posting to/Account/LoginWithEmail
/connect/token
POST call:Note that the
client_secret
is not present in this body. Also, the following request header was added:idk if that’s significant or sensitive.
So far I’m still getting a 401 on the
/connect/token
call despite setting the same parameters, so I’m still troubleshooting, but thought this info might help someone else that’s also looking into it.Note: I am not debugging against the pymyq package, so some of my assumptions about how that package behaves may be a bit off.
Blackstone bought Chaimberland a couple years ago. Anything they buy turns into a paywalled pile of useless dog crap.
Most likely. I grabbed it from a Postman login request to MyQ.
I don’t think it will. The 429s were arising in the first place because the login logic basically spams the service with retries when the login fails (it retries 5 times, then homeassistant restarts the integration which causes it to do it again right away). Without that issue, the API usage should be well below the rate limits.
And as an aside, it seems that MyQ is slowly tightening their API validations; the code needs a thorough once-over to make sure it’s actually compliant with OAuth properly and it isn’t doing any other extraneous/incorrect stuff; because given the history of this integration over the past month or so, any further deviations from the spec lurking in the code are likely to result in additional breakage in the future.
And also to add, if you apply this fix you may continue to receive 429 errors until your rate limiting on the MyQ API (caused by the broken code previously running and having sent too many requests to the API and getting your IP temporarily blocked for a while). Those issues should clear up within an hour or two.
I am having the same issue as well. I recently hooked an ESP32 up to my garage door opener and added 2 reed switches to the garage door so I can control it locally. Very glad I did it.
FYI go ahead and disable your MyQ addons until this is resolved.
@NukeThemTillTheyGlow and @matthewrosner, I recommend you start a new GitHub issue for not being able to get the integration running after installation.
I believe this will be part of 10.2… Not released yet… Hopefully later today.
@home-assistant close
I’m guessing this is maybe part of the issue - the repo needs a new maintainer …
https://github.com/Python-MyQ/Python-MyQ/issues/15
Bonjour; I have as well the same issue: I’ve tried to install (pip install Python-MyQ==v3.1.13) Restrated HA ent tried to reconfigure MyQ integration, no sucess, 401 error
Logger: pymyq.api Source: components/myq/config_flow.py:37 First occurred: 09:45:29 (2 occurrences) Last logged: 09:51:52
Authentication failed: Error requesting data from https://partner-identity.myq-cloud.com/connect/token: 401 - Unauthorized
As a dumy test, I’ve tried to connect to https://partner-identity.myq-cloud.com I get there a menu proposing “to secure page” --> https://partner-identity.myq-cloud.com/Account/LoginWithEmail?returnUrl=%2FHome%2FSecure My user name and password works fine on this page so the account is valid. From Chrome, accesing ttps://partner-identity.myq-cloud.com/connect/token give an error {“error”:“invalid_request”}
Egh. I was getting the 429. But my account was locked out. Tried to reset the pw and myq freaked out and never sent me an email, but locked me out of the password reset as well without sending the email. pure comedy.
thank you for that info, will disable in the meantime
429 means “Too many requests” and it looks like they implement a 15-minute cooldown period. So try in 15.
You may just need to wait for the cooldown period to expire. The 429 error should tell you how long you have to wait. It may start working after that.
Thank you guys for all your hard work!!! You really make this a great community to be a part of.
I did see some stuff in the app’s decompilation when I was digging through it that looked like functionality for them to forcibly deprecate old versions of the app, which makes me fearful they’re going to pursue a policy of aggressively requiring app updates that include protocol changes. Time will tell if and how often they’re going to fire that gun.
Sorry for the crappy details in comment above. The working fix is posted here: https://github.com/Python-MyQ/Python-MyQ/pull/22
I’m unsure whether the Android specific changes are needed. I’m going to evaluate that now, before asking for merge.
Right, it’s a POST with the body as the populated form fields, via hidden input params. Here’s my full list of changes. I just hit my 429 throttling so I can’t test further atm. 😒
Correct, and for further clarity, they are not headers, they are body params.
Don’t bother, it doesn’t work anymore. Furthermore, the Homebridge and Node-Red integrations are also now failing.
FWIW I collected ios calls to myq using the charles proxy app. I’m not sure if it helps much. I redacted the cookie and response code but I’ll include them in a DM to a maintainer if need-be
Login Call:
HTTP/1.1 302 Found Date: Wed, 11 Oct 2023 17:11:52 GMT Content-Length: 0 Connection: keep-alive Cache-Control: no-cache pragma: no-cache expires: Thu, 01 Jan 1970 00:00:00 GMT location: /connect/authorize/callback?redirect_uri=com.myqops%3A%2F%2Fios&code_challenge=[REDACTED]&scope=MyQ_Residential%20offline_access&client_id=IOS_CGI_MYQ&acr_values=unified_flow%3Av1%20brand%3Amyq%20app_version%3A5.242.0.38913&code_challenge_method=S256&prompt=login&response_type=code&ui_locales=en-US&suppressed_prompt=login set-cookie: idsrv=[REDACTED COOKIE TOKEN]; path=/; secure; samesite=none; httponly request-context: appId=cid-v1:0e843709-4fd8-4c8a-a52a-7eb45ba4930b x-xss-protection: 1; mode=block myq-correlationid: fb95fc49-9db0-44d3-a18e-966494621d24 strict-transport-security: max-age=15724800; includeSubDomains strict-transport-security: max-age=63072000 CF-Cache-Status: DYNAMIC Server: cloudflare CF-RAY: 8148b98af9b253bb-ATL alt-svc: h3=“:443”; ma=86400
Callback Call:
HTTP/1.1 302 Found Date: Wed, 11 Oct 2023 17:11:53 GMT Content-Length: 0 Connection: keep-alive location: com.myqops://ios?code=[REDACTED RESPONSE CODE]&scope=MyQ_Residential%20offline_access&iss=https%3A%2F%2Fpartner-identity.myq-cloud.com Cache-Control: no-store, no-cache, max-age=0 pragma: no-cache request-context: appId=cid-v1:0e843709-4fd8-4c8a-a52a-7eb45ba4930b x-xss-protection: 1; mode=block myq-correlationid: b695cbe2-70f3-4f0f-ac1e-ebd041ee09ff strict-transport-security: max-age=15724800; includeSubDomains strict-transport-security: max-age=63072000 CF-Cache-Status: DYNAMIC Server: cloudflare CF-RAY: 8148b990daaa53bb-ATL alt-svc: h3=“:443”; ma=86400
Nope, the C# code no longer works for login.
@Lash-L I’m not sure region domain cycling is going to fix the problem anymore, it looks like they may have updated the east and west regions to the newer code that doesn’t let us in. The homebridge myq-api no longer works either.
I have a feeling this is going to boil down to someone doing some new captures from the iOS or Android apps to see what’s required. I spent a little bit of time on it yesterday but I don’t have the tools to do it right so I was limited to just looking at decompiled obfuscated code from the Android app and trying to make sense of it, and that only goes so far. It does look like they’re sending additional HTTP headers to the get token endpoint (
MyQApplicationId
,BrandId
,App-Version
, and they do send aUser-Agent
header); but I couldn’t find the values they’re passing. I wouldn’t be surprised if they’re checking those headers now.https://github.com/Python-MyQ/Python-MyQ/tree/auth_fixes
This branch may be a good starting point for you - it was what I was able to do on my lunch break, but I’m timed out so I can’t test it further.
I did add domain cycling in the past, but I made it rather limited. I moved it to request so that it is now global for all requests that are made.
I’m timed out right now, so I can’t further test it and I’m almost done w/ my lunch so I need to get back to work
cc @jiffyjaff as well
@brchri
I “regionalized” as follows: const.py … added -east OAUTH_BASE_URI = “https://partner-identity-east.myq-cloud.com”
then I got the follow error messages at initialization time:
Logger: homeassistant.config_entries Source: components/myq/init.py:42 First occurred: 11:16:56 AM (1 occurrences) Last logged: 11:16:56 AM
Error setting up entry for myq Traceback (most recent call last): File “/usr/src/homeassistant/homeassistant/config_entries.py”, line 388, in async_setup result = await component.async_setup_entry(hass, self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File “/usr/src/homeassistant/homeassistant/components/myq/init.py”, line 42, in async_setup_entry myq = await pymyq.login(conf[CONF_USERNAME], conf[CONF_PASSWORD], websession) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File “/usr/local/lib/python3.11/site-packages/pymyq/api.py”, line 594, in login await api.authenticate(wait=True) File “/usr/local/lib/python3.11/site-packages/pymyq/api.py”, line 480, in authenticate await self._authentication_task File “/usr/local/lib/python3.11/site-packages/pymyq/api.py”, line 493, in _authenticate token, expires = await self._oauth_authenticate() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File “/usr/local/lib/python3.11/site-packages/pymyq/api.py”, line 421, in _oauth_authenticate “code”: parse_qs(urlsplit(redirect_url).query).get(“code”, “”)[0], ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^ IndexError: string index out of range
Restored const.py to OAUTH_BASE_URI = “https://partner-identity.myq-cloud.com”
(now I’m back to the 403 errors. (the only change I made was adding -east as shown above. honest.)
edit 1: now I see that regionalization stopped working for you…
Hear Hear!!
They would sell more of these boxes if they also opened up a local API to talk to the devices on your local network from your HA server rather than force the cloud service API dependency. I can’t image it would be any more or less security. If your cloud account can be hacked your garage can be opened from anywhere in the world. If you protect your home network, someone would have to break into your home network to talk to the device locally to open your door.
Yeah the endpoint was just updated again, my previously working Postman OAuth login isn’t working anymore either and failing with a 403. If you’re getting 429 it’s because of the previously mentioned issue that when login fails, homeassistant repeatedly hits the endpoint until it gets rate-limited.
Can confirm this resolved my issue as well.
cover.garage_door
was unavailable, logs reported 401 error)./usr/local/lib/python3.11/site-packages/pymyq/api.py
(your installation may vary)docker exec
’d into my HA container, backed up the file, opened the file in vi, found the code string referenced above within the fileYou guys are the best. Keep up the great work!
@Lash-L See https://github.com/Python-MyQ/Python-MyQ/pull/18
It’s just base64 encoded
IOS_CGI_MYQ:
so I assume it’s what the IOS app uses.I applied drysart’s change and am still throttled, but maybe it just needs to clear.
I am on 2023.9.2 but the latest MyQ code and it is failing
Do you have a link handy explaining how this gets wired up/flashed? I’d like to do the same.
Edit: Found this. It should work for me.
I lost connect in the last 30 minutes due to 429 Too Many Request but phone app still works:
Config entry ‘xxx@xxx.com’ for myq integration not ready yet: Got 429 error - stopping request until 2023-10-10 16:30:38.802025. there were 1 request; Retrying in background