alexa_media_player: aiohttp.client_exceptions.InvalidUrlClientError: /errors/validateCaptcha - solved by alexapy v1.28.2 but needs manual manifest.json edit

IMPORTANT: Please search the issues, including closed issues, and the FAQ before opening a new issue. The template is mandatory; failure to use it will result in issue closure.

Describe the bug

When updating to HA 2024.8.0 the alexa_media_player integration unfortunately broke for me. Was working fine on HA 2024.7.x. I think this has to do with a version bump of aiohttp.

Following error in the logs:
  File "/usr/local/lib/python3.12/site-packages/alexapy/alexalogin.py", line 687, in login
    post_resp = await self._session.post(

aiohttp.client_exceptions.InvalidUrlClientError: /errors/validateCaptcha

seems to me like the InvalidUrlClientError needs to be catched and a re-auth flow needs to be initialized.

To Reproduce

  1. Go to ‘…’
  2. Click on ‘…’
  3. Scroll down to ‘…’
  4. See error

Expected behavior

Initialize a re-auth flow if needed.

Screenshots

System details

  • Home-assistant (version): HA 2024.8.0
  • alexa_media (version from const.py or HA startup): v4.12.5
  • alexapy (version from pip show alexapy or HA startup):
  • Amazon 2FA is enabled (y/n). <!—We will not debug login issues if unanswered—>: YES 2FA enabled Logs Please provide logs.
Logger: homeassistant.config_entries
Source: config_entries.py:604
First occurred: 10:16:34 PM (1 occurrences)
Last logged: 10:16:34 PM
Error setting up entry Amazon Alexa for alexa_media

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 604, in async_setup
    result = await component.async_setup_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/alexa_media/__init__.py", line 359, in async_setup_entry
    await login.login(cookies=await login.load_cookie())
  File "/usr/local/lib/python3.12/site-packages/alexapy/helpers.py", line 137, in wrapper
    return await func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/alexapy/alexalogin.py", line 687, in login
    post_resp = await self._session.post(
                ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 576, in _request
    raise err_exc_cls(url)
aiohttp.client_exceptions.InvalidUrlClientError: /errors/validateCaptcha

Additional context

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 51
  • Comments: 174 (1 by maintainers)

Most upvoted comments

Downgraded back to 2024.7.4 and it’s working again, so something definitely broke this in 2024.8

For anyone having this same issue, you can rollback your HA version by using Advanced SSH Terminal and inputting

ha core update --version=2024.7.4

Error log on 2024.8.1:

Logger: alexapy.helpers Source: custom_components/alexa_media/__init__.py:359 integration: Alexa Media Player ([documentation](https://github.com/alandtse/alexa_media_player/wiki), [issues](https://github.com/alandtse/alexa_media_player/issues)) First occurred: 5:30:17 PM (1 occurrences) Last logged: 5:30:17 PM alexalogin.login((<alexapy.alexalogin.AlexaLogin object at 0x7f67c5c18170>,), {'cookies': {}}): An error occurred accessing AlexaAPI: An exception of type InvalidUrlClientError occurred. Arguments: (URL('/errors/validateCaptcha'),)

Also experiencing this issue. I’ve tried uninstalling and reinstalling multiple times. Not sure about how/where to apply the patch mentioned above.

The patch was originally mentioned here

ssh to HA

docker exec -it $(docker ps -f name=homeassistant -q) bash to get to the container level of HA

cd /usr/local/lib/python/3.12/site-packages/alexapy/

create the fix_cookies.patch file containing:

--- /tmp/alexalogin.py.orig
+++ /tmp/alexalogin.py.modified
@@ -1251,10 +1251,11 @@
                     # Strings are returned within quotations, strip them
                     else item["Value"][1:-1]
                 )
-                for name, value in item.items():
-                    if name in ["Name", "Value"]:
-                        continue
-                    raw_cookie[cookie_name][name] = f"{value}; Domain={domain}"
+                raw_cookie[cookie_name]["domain"] = domain
+                raw_cookie[cookie_name]["path"] = item["Path"]
+                raw_cookie[cookie_name]["secure"] = item["Secure"]
+                raw_cookie[cookie_name]["expires"] = item["Expires"]
+                raw_cookie[cookie_name]["httpOnly"] = item["HttpOnly"]                
                 # _LOGGER.debug("updating jar with cookie %s", raw_cookie)
                 self._session.cookie_jar.update_cookies(raw_cookie, URL(domain))
             _LOGGER.debug(

Then run:

apk --no-cache add patch
patch "/usr/local/lib/python3.12/site-packages/alexapy/alexalogin.py" < "fix_cookies.patch"

For anyone having this same issue, you can rollback your HA version by using Advanced SSH Terminal and inputting ha core update --version=2024.7.4

Thanks for this. I rolled mine back and can confirm its now working! Definitely keeping this version until I see a fix.

Same here…

Even if you uninstall the Integration you can’t install it.

will there be a official bug-fix / update for this issue?

It has be from alexapy. I only see the workaround so far. Tomorrow I will double check and if still none, I’ll fork it and submit a PR…

For anyone having this same issue, you can rollback your HA version by using Advanced SSH Terminal and inputting

ha core update --version=2024.7.4

Thanks for this. I rolled mine back and can confirm its now working! Definitely keeping this version until I see a fix.

Modifying manifest.json requirements key like this solved it for me:

"requirements": ["alexapy>=1.28.2", "packaging>=20.3", "wrapt>=1.14.0"],

Thanks for the help.

THANK YOU, THANK YOU, THANK YOU! Fixed for me as well!

The patch is no longer needed as alexapy 1.28.2 contains it.

image

1, Update AMP to v4.12.6 2. Edit manifest.json line 11:

"requirements": ["alexapy==1.28.2", "packaging>=20.3", "wrapt>=1.14.0"],
  1. Restart HA…

@micium update HA to 2024, update the custom_components/alexa_media/manifest.json file to alexapy==1.28.2, restart HA

I expect it’ll be at least HA 2024.8.2 or higher before all that happens but we’ll see. So if you install HA 2024.8.1 next week, it may all break again as the patch will be gone…

When you see AMP release notes listing a new alexapy version higher than v1.28.1 it should be safe

Modifying manifest.json requirements key like this solved it for me:

"requirements": ["alexapy>=1.28.2", "packaging>=20.3", "wrapt>=1.14.0"],

Thanks for the help.

I’d like to thank everyone for their tireless work to fix this bug. Please let us all know when the new AMP release (listing a new alexapy version higher than v1.28.1) is safe to upgrade. This integration is one of my most used in my HA instance. I have several service “notify” calls in various automations to audibly notify via alexa decices right thru my house… It’s by far one of my favourite things … Once we get a fix, will service calls just be automatically become “actions” (or whatever they are called), or will we have to recode all our automations? Thanks!

With the new .8.x version all services have been renamed to actions, so don’t worry about that. I’m using alexa integration the same way as you do. So looking forward for a ‘real’ fix too. Now nothing else to do than to wait.

Thanks to all participants for helping. 😄

The integrations for Amazon are back again. I can’t add a new one (yes, I restarted multiple times), but that’s OK. I can wait for a fix. 👍

@danielbrunt57 and @EzeCuervo I am overly thankful and joyfull for you both coming and helping a beginner that was totally lost. (It must be frustrating 😃) I think I have a better understanding of this and I can’t thank you enough. I hope yall have a wonderful rest of your day (or night)!

Also experiencing this issue. I’ve tried uninstalling and reinstalling multiple times. Not sure about how/where to apply the patch mentioned above.

The patch was originally mentioned here

ssh to HA

docker exec -it $(docker ps -f name=homeassistant -q) bash to get to the container level of HA

cd /usr/local/lib/python3.12/site-packages/alexapy/

create the fix_cookies.patch file containing:

--- /tmp/alexalogin.py.orig
+++ /tmp/alexalogin.py.modified
@@ -1251,10 +1251,11 @@
                     # Strings are returned within quotations, strip them
                     else item["Value"][1:-1]
                 )
-                for name, value in item.items():
-                    if name in ["Name", "Value"]:
-                        continue
-                    raw_cookie[cookie_name][name] = f"{value}; Domain={domain}"
+                raw_cookie[cookie_name]["domain"] = domain
+                raw_cookie[cookie_name]["path"] = item["Path"]
+                raw_cookie[cookie_name]["secure"] = item["Secure"]
+                raw_cookie[cookie_name]["expires"] = item["Expires"]
+                raw_cookie[cookie_name]["httpOnly"] = item["HttpOnly"]                
                 # _LOGGER.debug("updating jar with cookie %s", raw_cookie)
                 self._session.cookie_jar.update_cookies(raw_cookie, URL(domain))
             _LOGGER.debug(

Then run:

apk --no-cache add patch
patch "/usr/local/lib/python3.12/site-packages/alexapy/alexalogin.py" < "fix_cookies.patch"

Thanks, this solved the problem for me!

😃

Did I do a typo above?? It’s /python3.12/ not /python/3.12/

Yep. Then the apk command and the cd command

Downgraded back to 2024.7.4 and it’s working again, so something definitely broke this in 2024.8

It seems to me, that it may have to do with the unification of Services into Actions that something broke. 2024.7.4 put Services back.

No it looks like it’s an alexapy issue as mentioned in other threads see: https://github.com/alandtse/alexa_media_player/issues/2407#issuecomment-2272879665

Downgraded back to 2024.7.4 and it’s working again, so something definitely broke this in 2024.8

It seems to me, that it may have to do with the unification of Services into Actions that something broke. 2024.7.4 put Services back.

And add to this a password reset.

@hardwareadictos

  • Delete integration config entry
  • Delete /config/.storage/alexa_media.<your_email>.pickle file
    • AMP PR #2462 has been merged and v4.12.8 now includes its part in the new sequence to remove the cookiefile (pickle file in .storage) but alexapy merge request !380 needs to merge with a version bump and a new corresponding manifest.json (and another AMP version bump) for the remove cookiefile call to work.
  • Clear browser cookies
  • Restart HA
  • Verify your correct amazon account credentials
  • Verify Amazon 2SV (Two Step Verification) is enabled
  • Reinstall AMP integration - Note: Do not use private browsing mode!

This looks fixed in the newly released 4.12.7

Also experiencing this issue. I’ve tried uninstalling and reinstalling multiple times. Not sure about how/where to apply the patch mentioned above.

The patch was originally mentioned here

ssh to HA

docker exec -it $(docker ps -f name=homeassistant -q) bash to get to the container level of HA

cd /usr/local/lib/python3.12/site-packages/alexapy/

create the fix_cookies.patch file containing:

--- /tmp/alexalogin.py.orig
+++ /tmp/alexalogin.py.modified
@@ -1251,10 +1251,11 @@
                     # Strings are returned within quotations, strip them
                     else item["Value"][1:-1]
                 )
-                for name, value in item.items():
-                    if name in ["Name", "Value"]:
-                        continue
-                    raw_cookie[cookie_name][name] = f"{value}; Domain={domain}"
+                raw_cookie[cookie_name]["domain"] = domain
+                raw_cookie[cookie_name]["path"] = item["Path"]
+                raw_cookie[cookie_name]["secure"] = item["Secure"]
+                raw_cookie[cookie_name]["expires"] = item["Expires"]
+                raw_cookie[cookie_name]["httpOnly"] = item["HttpOnly"]                
                 # _LOGGER.debug("updating jar with cookie %s", raw_cookie)
                 self._session.cookie_jar.update_cookies(raw_cookie, URL(domain))
             _LOGGER.debug(

Then run:

apk --no-cache add patch
patch "/usr/local/lib/python3.12/site-packages/alexapy/alexalogin.py" < "fix_cookies.patch"

The is the one that did it for me. I’m running HAOS and thought I could just run the patch in the terminal. Turns out that I HAD to run the patch in the homeassistant docker container. Once I ran the patch in the right place and restarted HA, all my Echos showed up again.

Thanks community!

Just install HA 2024.8 and all Updates of Alexa Media Player (AMP). Then change the manifest file. It should work then.

The files you are searching for an in the homeassistant docker file.

just needs this PR to be merged for solving the issue worldwide #2423

… and resolved the https://github.com/alandtse/alexa_media_player/issues/2429 after the manifest.json fix all devices are available, no msg at HA startup, but the notify service/action is not allowed, with auth problem
I did a roolback to 2024.7.4 and all work fine

this is the error when i try doing TTS

Logger: alexapy.helpers Source: runner.py:190 First occurred: 1:27:50 pm (3 occurrences) Last logged: 1:27:50 pm

alexaapi.run_behavior((<alexapy.alexaapi.AlexaAPI object at 0x7f0f287e60>, {‘@type’: ‘com.amazon.alexa.behaviors.model.OpaquePayloadOperationNode’, ‘type’: ‘Alexa.Speak’, ‘operationPayload’: {‘deviceType’: ‘A2U21SRK4QGSE1’, ‘deviceSerialNumber’: ‘GAX2’, ‘locale’: ‘en-gb’, ‘customerId’: 'A6WN’, ‘textToSpeak’: ‘hello’, ‘target’: {‘customerId’: 'A6WN’, ‘devices’: [{‘deviceSerialNumber’: 'GAX2’, ‘deviceTypeId’: ‘A2U21SRK4QGSE1’}]}, ‘skillId’: ‘amzn1.ask.1p.saysomething’}}), {‘queue_delay’: 1.5}): An error occurred accessing AlexaAPI: An exception of type AlexapyLoginError occurred. Arguments: (‘Login error detected; not contacting API’,) alexaapi.send_sequence((<alexapy.alexaapi.AlexaAPI object at 0x7f0f287e60>, ‘Alexa.Speak’), {‘customer_id’: ‘A29W5ZO0ZXT6WN’, ‘textToSpeak’: ‘hello’, ‘target’: {‘customerId’: 'A6WN’, ‘devices’: [{‘deviceSerialNumber’: 'G******AX2’, ‘deviceTypeId’: ‘A2U21SRK4QGSE1’}]}, ‘skillId’: ‘amzn1.ask.1p.saysomething’, ‘queue_delay’: 1.5}): An error occurred accessing AlexaAPI: An exception of type AlexapyLoginError occurred. Arguments: (‘Login error detected; not contacting API’,) alexaapi.send_tts((<alexapy.alexaapi.AlexaAPI object at 0x7f0f287e60>, ‘hello’), {‘customer_id’: ‘A29W5ZO0ZXT6WN’, ‘queue_delay’: 1.5}): An error occurred accessing AlexaAPI: An exception of type AlexapyLoginError occurred. Arguments: (‘Login error detected; not contacting API’,)

I updated Alexa Integration with latest version and used the line from above:

"requirements": ["alexapy>=1.28.2", "packaging>=20.3", "wrapt>=1.14.0"],

restart HA and up and running

Before that I did the patch part this morning.

Can someone please summarize the workaround steps to fix this problem, starting from HA 2024.7.4 and AMP 4.12.6, to allow the HA update to 2024.8.0 with AMP integration working well ? thanks

I’d like to thank everyone for their tireless work to fix this bug. Please let us all know when the new AMP release (listing a new alexapy version higher than v1.28.1) is safe to upgrade. This integration is one of my most used in my HA instance. I have several service “notify” calls in various automations to audibly notify via alexa devices right thru my house… It’s by far one of my favourite things … Once we get a fix, will service calls just automatically become “actions” (or whatever they are called), or will we have to recode all our automations? Thanks!

will there be a official bug-fix / update for this issue?

It has be from alexapy. I only see the workaround so far. Tomorrow I will double check and if still none, I’ll fork it and submit a PR…

If there is an update through HACS that addresses this will I have to do anything with the patch you helped install?

It’s not HACS or AMP per-se. alexpy is a python dependency that comes from GitLab If and when a fix is pulled into alexapy I suspect it will be bumped to v1.28.2 followed by a boring poetry session (poetry.lock stuff which I don’t understand yet) and then a PR for AMP to update manifest.json with a new alexapy version. That’s when it’ll all happen magically in the background when you update to a new AMP version. Clear as mud, right??

oh now after 2nd restart it works!

rm fix_cookies.patch and start over with vi…

Your contents are incomplete. Should be:

--- /tmp/alexalogin.py.orig
+++ /tmp/alexalogin.py.modified
@@ -1251,10 +1251,11 @@
                     # Strings are returned within quotations, strip them
                     else item["Value"][1:-1]
                 )
-                for name, value in item.items():
-                    if name in ["Name", "Value"]:
-                        continue
-                    raw_cookie[cookie_name][name] = f"{value}; Domain={domain}"
+                raw_cookie[cookie_name]["domain"] = domain
+                raw_cookie[cookie_name]["path"] = item["Path"]
+                raw_cookie[cookie_name]["secure"] = item["Secure"]
+                raw_cookie[cookie_name]["expires"] = item["Expires"]
+                raw_cookie[cookie_name]["httpOnly"] = item["HttpOnly"]                
                 # _LOGGER.debug("updating jar with cookie %s", raw_cookie)
                 self._session.cookie_jar.update_cookies(raw_cookie, URL(domain))
             _LOGGER.debug(

I just removed my comments @danielbrunt57, go ahead! hahaha

Now copy the contents of the patch but don’t paste just yet!

The vi fix_cookies.patch and a blank window opens. Press i to enter insert mode Now paste and the press Escape to exit insert mode Then QQ (uppercase) to write and exit vi (alternatively :wq!)

image I did the cd command and this is what I see

do i just paste the command where the cursor is currently

Ok, in there, start with the docker exec -it $(docker ps -f name=homeassistant -q) bash command

I ran the command and I see…

image

Just redownloaded and running v4.12.6… I re-configured and am having the same error.

The patch was originally mentioned #2407 (comment) ssh to HA docker exec -it $(docker ps -f name=homeassistant -q) bash to get to the container level of HA cd /usr/local/lib/python3.12/site-packages/alexapy/ create the fix_cookies.patch file containing:

--- /tmp/alexalogin.py.orig
+++ /tmp/alexalogin.py.modified
@@ -1251,10 +1251,11 @@
                     # Strings are returned within quotations, strip them
                     else item["Value"][1:-1]
                 )
-                for name, value in item.items():
-                    if name in ["Name", "Value"]:
-                        continue
-                    raw_cookie[cookie_name][name] = f"{value}; Domain={domain}"
+                raw_cookie[cookie_name]["domain"] = domain
+                raw_cookie[cookie_name]["path"] = item["Path"]
+                raw_cookie[cookie_name]["secure"] = item["Secure"]
+                raw_cookie[cookie_name]["expires"] = item["Expires"]
+                raw_cookie[cookie_name]["httpOnly"] = item["HttpOnly"]                
                 # _LOGGER.debug("updating jar with cookie %s", raw_cookie)
                 self._session.cookie_jar.update_cookies(raw_cookie, URL(domain))
             _LOGGER.debug(

Then run:

apk --no-cache add patch
patch "/usr/local/lib/python3.12/site-packages/alexapy/alexalogin.py" < "fix_cookies.patch"

How do I do this?? Sorry, I’m pretty new to home assistant.

You can connect to your HA using the terminal add-on inside HA. Go to Settings > Add-ons and Install “Advanced SSH & Web Terminal”.

After install, you have to disable the Protection Mode.

image

Then open the terminal and follow all these steps, you can create the file with the command: vi fix_cookies.patch

After rebooting whole machine not just HA I get other errors an Notifitcation to reauthenticate against alexa. But how to re auth?

Logger: homeassistant.config_entries Quelle: config_entries.py:604 Erstmals aufgetreten: 07:17:18 (3 Vorkommnisse) Zuletzt protokolliert: 07:19:23

Error setting up entry my@domain.de - amazon.de for alexa_media Traceback (most recent call last):

File “/config/custom_components/alexa_media/config_flow.py”, line 578, in async_step_reauth reauth_schema = self._update_schema_defaults() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File “/config/custom_components/alexa_media/config_flow.py”, line 815, in _update_schema_defaults CONF_HASS_URL, default=self.config.get(CONF_HASS_URL, hass_url) ^^^^^^^^ NameError: name ‘hass_url’ is not defined`

You need to apply PR #2417 just merged intp master and released as v4.12.6…

+1 Broken on 2024.8. Waiting for an update. I don’t understand the patch thing?

aiohttp.client_exceptions.InvalidUrlClientError: /errors/validateCaptcha

Somehow alexapy is attempting to send a post request to the url “/error/validateCaptcha”, clearly a malformed url, which is why the exception is being raised by aiohttp. Could be an uncaught issue on how alexapy finds that URL, could be that amazon updated some page, a combination of these, etc.

https://github.com/aio-libs/aiohttp/pull/8158/files#diff-f334e752b4894ef951105572ab8b195aeb8db90eb6e48b1dfbd9a01da4c854f5R530

It looks like this PR added the exception where its raising but the comment in that PR implies it would fail anyways later