core: Growatt server error after upgrade

The problem

Hello,

Growatt server is not working anymore after upgrade to 5.3

Logger: aiohttp.server
Source: components/growatt_server/config_flow.py:50
First occurred: 11:01:53 AM (7 occurrences)
Last logged: 11:09:06 AM

Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1040, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 414, in connect
    self.sock = ssl_wrap_socket(
  File "/usr/local/lib/python3.9/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "/usr/local/lib/python3.9/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/local/lib/python3.9/ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "/usr/local/lib/python3.9/ssl.py", line 1040, in _create
    self.do_handshake()
  File "/usr/local/lib/python3.9/ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 440, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 785, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.9/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='server.growatt.com', port=443): Max retries exceeded with url: /newLoginAPI.do (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/aiohttp/web_protocol.py", line 435, in _handle_request
    resp = await request_handler(request)
  File "/usr/local/lib/python3.9/site-packages/aiohttp/web_app.py", line 504, in _handle
    resp = await handler(request)
  File "/usr/local/lib/python3.9/site-packages/aiohttp/web_middlewares.py", line 117, in impl
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 60, in security_filter_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 100, in forwarded_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 28, in request_context_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 79, in ban_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 219, in auth_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 137, in handle
    result = await result
  File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 205, in post
    return await super().post(request, flow_id)
  File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 62, in wrapper
    result = await method(view, request, *args, **kwargs)
  File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 109, in post
    result = await self._flow_mgr.async_configure(flow_id, data)
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 260, in async_configure
    result = await self._async_handle_step(
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 335, in _async_handle_step
    result: FlowResult = await getattr(flow, method)(user_input)
  File "/usr/src/homeassistant/homeassistant/components/growatt_server/config_flow.py", line 50, in async_step_user
    login_response = await self.hass.async_add_executor_job(
  File "/usr/local/lib/python3.9/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.9/site-packages/growattServer/__init__.py", line 115, in login
    response = self.session.post(self.get_url('newLoginAPI.do'), data={
  File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 577, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 517, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='server.growatt.com', port=443): Max retries exceeded with url: /newLoginAPI.do (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)')))

What version of Home Assistant Core has the issue?

core-5.3

What was the last working version of Home Assistant Core?

core-4,7

What type of installation are you running?

Home Assistant Container

Integration causing the issue

Growatt

Link to integration documentation on our website

https://www.home-assistant.io/integrations/growatt_server

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 127 (23 by maintainers)

Commits related to this issue

Most upvoted comments

Correct, I’ll be submitting the fix in about 20-30 minutes’ time. It’ll be linked to this ticket so you’ll all see once it’s merged etc. tiny bit more patience and we’ll be there.

any idea when the integration will be fixed without hacks. just working out of the box by installing the Growatt integration on home assistant frontend. I also can’t reinstall it

As soon as this PR is merged & released by Indy: https://github.com/indykoning/PyPi_GrowattServer/pull/29 I can then raise the PR for HomeAssistant, at which point it will be in the next nightly build. The hardest part is getting Indy to merge it. If everyone wants to go and comment on that PR saying “it works for me in home-assistant” then it might accelerate it, but there’s no guarantees. As with all opensource projects people work on them when they have time. I just happened to have a couple of hours spare last night and was able to submit the fix. It’s still up to Indy as the owner to release it.

Hi Everyone,

I’ve seen that this chain has got a bit busy with people saying they have the issue but they’re not reading up far enough to see the workaround/hotfix that I suggested yesterday.

Here’s a direct link to it: https://github.com/home-assistant/core/issues/71577#issuecomment-1155749243

Ultimately, if you just change that file to use the new Login URL it will start working.

This issue isn’t a bug in HomeAssistant or the Library that we use, the Growatt Servers that we speak to were updated to use a new URL for logins yesterday, therefore we have to update our library to match it. Growatt don’t provide any official support for their API so it’s completely out of our control.

I have submitted a pull-request to the library that myself and @indykoning support. Once he’s approved that and it has been released I can submit the update to home-assistant to use the new library version and it will be fixed in the latest build for everyone. Until then it’s a case of either waiting until the fix comes out, or, attempting the hotfix/patch that I’ve provided above.

Posted a more detailed update over on the https://github.com/indykoning/PyPi_GrowattServer/issues/28#issuecomment-1155738553 ticket however, for home-assistant specific stuff:

  1. I have temporarily hacked my copy of homeassistant to make this work with the fix that will be put into the upstream library
  2. I use a docker based installation so have performed the following by mounting files into the container, but the following steps should help people with Home Assistant OS (or other) installations.

File that needs to be changed: /usr/local/lib/python3.9/site-packages/growattServer/__init__.py @ line 115 Change: newLoginAPI.do to newTwoLoginAPI.do then restart home-assistant.

Please don’t ask me specifics for your installation as I’m not a home-assistant expert, I just know my way around the docker based installation. If there are people that get this work-around working on the OS or application version of home-assistant, please feel free to post better instructions.

Note - if you’re using a docker installation restarting home-assistant means restarting the container which will undo the change. I worked around this by mounting in a temporary directory over the top of the /usr/local/lib/python3.9/site-packages/growattServer/ directory with the hacked version of the library in it.

I hope that helps people temporarily until the upstream library has it’s fix published and we can integrate it into home-assistant. I will update accordingly and re-direct any new tickets that are logged with this issue to this one so that we have a single chain for this.

Same problem here, temporary workaround when using docker (works until you restart hass docker container):

in cli execute:

docker exec -it hass bash

inside docker container execute:

vi /usr/local/lib/python3.9/site-packages/growattServer/init.py

and modify line 115: Change: newLoginAPI.do to newTwoLoginAPI.do then restart home-assistant.

I agree that it would be very handy to be able to change this, I though we provided a drop-down in a recent version of the integration. In this specific instance I think that the Growatt server has gone down, but, the hass integration could be more graceful in handling it.

I’m currently at work, however I can look at some improvements this evening to see how we can better handle this scenario. In the meantime it looks like you’ve got some work-arounds going on.

For those that missed it, the release is out there in the wild: https://github.com/home-assistant/core/releases/tag/2022.6.7

Download and start using 2022.6.7 and hopefully all will be well again - no more hacks!

Hi All, this looks like it’s unrelated to latest changes to HomeAssistant (I’m running a much older version of home assistant and have the same issue). This looks like a change has been made on the server side of Growatt. I’ll report back shortly but this is likely to require a change to the library myself and @indykoning maintain which the Growatt Integration for homeassistant makes use of. Once we’ve worked out what has changed on the API I’ll see if we can issue a recommended change/hack that people can make until the official fix is live.

All, working for me now as well. I’ll postpone my evening hackathon tonight in that case, but I will take a look through the code to see what we can do to accommodate this in the integration. There’s a part of me that wants to say that this is a problem with growatt and not the integration, that said, it has happened a few times before so a little bit of preventative coding wouldn’t go amiss.

Thanks for the quick reply. I’ll give it a try. Unfortunately I have never done that before, so if someone can point me in the right direction, it would be much appreciated!

You can edit statistics under Developer -> Statistics in the Home Assistant UI.

If you have Luxtek inverters you can do exactly what you want to do. Search for Octolux, I’m not on that tariff so I’m currently in the process of automating the PV prediction to turn off/on the AC charge on the inverters. Ideally I’d like to be able to control the % based on SOC.

@saxon-viking: Since we are offtopic anyway, I have a question. Can I really contol battery charge via HA with Luxtec Hybrid Inverters? I cannot add a hard wired current sensor to my installation, so I am usinf Powerfox, which is also connected to by HA. It would be great if I could control bat charge/discharge via HA. Is there an HA integration for there inverters and would it run w/o hard wired current sensor?

Best J.

To be fair, it would also be a big advantage for the electric companies to have people charge up their solar inverter batteries when the electricity is plentiful, and then use it when there is not a lot of renewable energy.

If a lot of people had a battery, and did this, they would not have to run the coal or gas plants in the night as hard, because “everyone” would be running off of their own batteries.

Even better, when cars go fully V2G, imagine having a lot of EV’s as reservoir for electricity. Charge them in the daytime when sun and wind is abundant, and discharge them in the night when there are fewer renewables.

Hi, If you have Luxtek inverters you can do exactly what you want to do. Search for Octolux, I’m not on that tariff so I’m currently in the process of automating the PV prediction to turn off/on the AC charge on the inverters. Ideally I’d like to be able to control the % based on SOC.

It would have been done sooner but I messed up the config of the wifi dongles that can be changed to be controlled from HA. I wanted to go hard wired and found that the LAN dongles do not have the same open TCP ports that the WIFI dongles have. Every day is a school day!

Nice! How to install that fix?

with the next update of HA

Looking forward for this release 😉 …

Me 2 We use the integration extensively, to plan when to run home appliances and charge our EV, to minimize the electricity we have to actually buy from the grid 😃

Same here.

I’ve gone one step further (we have a system with storage batteries) and I’m in the process of writing a script which runs periodically to check the solar forecast for the next day for my location and then use it to charge the batteries during our off-peak window to the desired amount in order for us to use “cheap” electricity until we’re fully running from the solar the next day. It’s nearly ready, at which point I’ll be posting it on my github.

I have asked my supplier of the solar system something similar.

I also have a battery.

So, when it’s a really sunny day, and there is a bit of wind, electricity prices are usually low, but if there is a day, where its sunny and windy in the first part and not, in the last, the price will go up in the afternoon.

The problem is, my system will always match the power and use from my battery, but in some cases it would be better, if the battery was charged, to just buy cheap energy in the afternoon, and use the battery pack in the night, where prices might be higher.

Sadly there is no function to tell the inverter to put the electricity on the battery, instead of matching the current load, and there is no function to ask the inverter to just stop discharging the battery to save it for later where it is more expensive, and run off of cheap electricity from the grid.

The “Charge the battery from the grid” will almost never be a good deal here, as about 75% of the price of a kilowatt is taxes and VAT. (I know, thats lame, but living in a socialist country, this is what i must endure 😃 )

Nice! How to install that fix?

with the next update of HA

Looking forward for this release 😉 …

Me 2

We use the integration extensively, to plan when to run home appliances and charge our EV, to minimize the electricity we have to actually buy from the grid 😃

Same here.

I’ve gone one step further (we have a system with storage batteries) and I’m in the process of writing a script which runs periodically to check the solar forecast for the next day for my location and then use it to charge the batteries during our off-peak window to the desired amount in order for us to use “cheap” electricity until we’re fully running from the solar the next day. It’s nearly ready, at which point I’ll be posting it on my github.

Nice! How to install that fix?

with the next update of HA

Any guide to how one is to edit this file if running HAOS on a Raspberry Pi. I dont seem to have any of these libraries (visible) when i look at the file structure using the terminal program on my HA.

You would need to mount the 8th partition of HAOS from any LiveCD and go to the mount point and find in docker/overlays2/{LAYER_HASH}/diff/usr/local/lib/python3.9/site-packages/growattServer/init.py and edit it. I had that in 2 locations so it’s likely that one of the layers was out of date and hasn’t been cleaned up.

If that’s too difficult, you have to wait for official merge and update of the OS 😦.

Not even going to try that.

Thanks for the answer though, i will just have to wait then 😃

_

That looks good, but I am also unable to locate that file. I installed HA on Virtualization Station on my QNAP NAS and I cannot even find the growattServer nor a python directory 😦

I have the same, can’t find the growatt directory. Search all files. No idea, where it is…

Well, I found out myself a different solution. In Docker container, started BASH shell , used VI to change the file. Saved and restarted the container.

Thanks for all hints.

Great that a fix is to arrive. But in meantime, how can I do this

I use a docker based installation so have performed the following by mounting files into the container,

I use docker on Synology NAS, but would like to get a bit more detailed instruction how to do this. I have a patched version of the specific file, but how can I mount this ?
Thanks for help.

Posted a more detailed update over on the indykoning/PyPi_GrowattServer#28 (comment) ticket however, for home-assistant specific stuff:

  1. I have temporarily hacked my copy of homeassistant to make this work with the fix that will be put into the upstream library
  2. I use a docker based installation so have performed the following by mounting files into the container, but the following steps should help people with Home Assistant OS (or other) installations.

File that needs to be changed: /usr/local/lib/python3.9/site-packages/growattServer/__init__.py @ line 115 Change: newLoginAPI.do to newTwoLoginAPI.do then restart home-assistant.

Please don’t ask me specifics for your installation as I’m not a home-assistant expert, I just know my way around the docker based installation. If there are people that get this work-around working on the OS or application version of home-assistant, please feel free to post better instructions.

Note - if you’re using a docker installation restarting home-assistant means restarting the container which will undo the change. I worked around this by mounting in a temporary directory over the top of the /usr/local/lib/python3.9/site-packages/growattServer/ directory with the hacked version of the library in it.

I hope that helps people temporarily until the upstream library has it’s fix published and we can integrate it into home-assistant. I will update accordingly and re-direct any new tickets that are logged with this issue to this one so that we have a single chain for this.

Thanks for posting the quick fix, it seems to work again, i used portainer to connect to the docker container and edit the file

Yeah Growatt are a complete anomaly, it wouldn’t surprise me if the page just randomly re-appears tomorrow when China’s timezones means that someone is in an office and realises. Likewise, if wouldn’t surprise me at all if it never came back!

I’m thinking I can implement some logic to say “if the first URL doesn’t work try a fallback”. I haven’t looked at that code for a while so I need to remember what the existing logic is.

Awesome guys, workaround worked for me ! A code change would be very welcome, just so we don’t have to mess around the core config files manually (and then could break things). Kind of weird on the Growatt side though, that they have 2 different infra structures pointing to the same data. Here’s a view DNS wise:

dig server-api.growatt.com

; <<>> DiG 9.10.6 <<>> server-api.growatt.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 7202 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 1232 ;; QUESTION SECTION: ;server-api.growatt.com. IN A

;; ANSWER SECTION: server-api.growatt.com. 145 IN A 47.91.67.221

;; Query time: 89 msec ;; SERVER: 1.1.1.1#53(1.1.1.1) ;; WHEN: Mon May 09 12:20:26 CEST 2022 ;; MSG SIZE rcvd: 67

And the official one (which is used in the Growatt integration into HA):

dig server.growatt.com

; <<>> DiG 9.10.6 <<>> server.growatt.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 31723 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 1232 ;; QUESTION SECTION: ;server.growatt.com. IN A

;; ANSWER SECTION: server.growatt.com. 548 IN CNAME server.growatt.com.w.kunlunea.com. server.growatt.com.w.kunlunea.com. 60 IN A 8.48.85.219

;; Query time: 300 msec ;; SERVER: 8.8.8.8#53(8.8.8.8) ;; WHEN: Mon May 09 12:20:20 CEST 2022 ;; MSG SIZE rcvd: 110

Same here since +/- 8:30 AM CET, IOS APP and https://server-api.growatt.com are working. I have changed the URL manually in the core.config_entries and after a restart the integration was working again. My guess is that the mobile apps are using the api url. Maybe this url should be added in the config as well, first wait till the main URL is back again. I have contacted service via mail, let’s wait for their reaction

Thanks for contacting them!!

Same here! can´t find core.config_entries

Same here since +/- 8:30 AM CET, IOS APP and https://server-api.growatt.com are working.

I have changed the URL manually in the core.config_entries and after a restart the integration was working again.

image

My guess is that the mobile apps are using the api url. Maybe this url should be added in the config as well, first wait till the main URL is back again. I have contacted service via mail, let’s wait for their reaction