core: Growatt integration sensors unavailable
The problem
I noticed that the solar production was null in my energy dashboard while there was still electricity returned to the grid. Then I inspected the Growatt sensors and they were all unavailable. I reloaded the Growatt integration without succes. Neither did ‘Quick reload’. After a complete ‘Restart Home Assistant’ the sensors became available again. But after a few hours the problem returned.
What version of Home Assistant Core has the issue?
core-2023.9.2
What was the last working version of Home Assistant Core?
core-2023.9.2
What type of installation are you running?
Home Assistant OS
Integration causing the issue
Growatt
Link to integration documentation on our website
https://www.home-assistant.io/integrations/growatt_server
Diagnostics information
Growatt integration has no option to download diagnostic data
Example YAML snippet
No response
Anything in the logs that might be useful for us?
Logger: homeassistant.components.sensor
Source: components/growatt_server/sensor.py:43
Integration: Sensor (documentation, issues)
First occurred: 20:21:39 (1 occurrences)
Last logged: 20:21:39
Error while setting up growatt_server platform for sensor
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 359, in _async_setup_platform
await asyncio.shield(task)
File "/usr/src/homeassistant/homeassistant/components/growatt_server/sensor.py", line 87, in async_setup_entry
devices, plant_id = await hass.async_add_executor_job(get_device_list, api, config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/growatt_server/sensor.py", line 43, in get_device_list
login_response = api.login(config[CONF_USERNAME], config[CONF_PASSWORD])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/growattServer/__init__.py", line 132, in login
data = json.loads(response.content.decode('utf-8'))['back']
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Additional information
My Growatt account is functioning normally at the following website https://server.growatt.com/index
About this issue
- Original URL
- State: closed
- Created 9 months ago
- Comments: 393 (50 by maintainers)
A few responses to some of the chat in this thread.
Got the following reply from NL support
`Our global servers have had some problems since this Tuesday, causing app logins to be affected as well. Our engineers are working hard to repair it, so please be patient for a few days, and check the system and the app later this week. We hope the problem will be solved later this week. If it still has problems later, please email me.
Met vriendelijke groet/Kind regards,
Yalin Wang
Growatt New Energy B.V.`
For what it’s worth, for anyone being a bit overwhelmed by the many options and alternatives to this integration (all having been mentioned many times in this thread), I tried to give an overview on what’s available, what I picked and why, and how it was configured here: https://www.splitbrain.org/blog/2023-11/03-growatt_and_home_assistant
Hi Everyone (fwiw I’m the original maintainer of this integration),
Sorry for the delay in replying to all these comments/bugs. I see people have been trying all sorts to work around the various issues with Growatt. To be clear, I haven’t had the time to read through all these comments etc so apologies if I repeat anything that has been said before.
Here’s where I stand on things:
Some extra notes/detail:
My current setup:
My planned setup:
Like I said, I really need to pull this integration from Core to avoid the pain everyone has been suffering here. Grott has been working really well for me and if I can get the last bit working with it pivoting based on what’s available I think it’s a great (free) solution. (Grottserver even has a primitive feature for pushing configuration to the inverter as well).
I’m sorry if I haven’t answered people’s questions, there are about 15 different chains going on around this and I’m trying to divert them all to this one as it’s the biggest.
Hopefully this has been useful information at least.
Hi Everyone,
As the submitter of this defect I will now close this topic. Thanks for all your contributions. As muppet3000 clearly explained, the changes made by Growatt to it’s server requires extensive rework which is much more then just updating some server urls: https://github.com/home-assistant/core/issues/100874#issuecomment-1739911377
There are at least two alternative solutions mentioned in this thread:
Go with Grott, which requires setting up some additional services see: https://github.com/muppet3000/homeassistant-grott
Read out your Growatt locally using some ESP device and read out the UART bus using a RS485 https://esphome.io/components/sensor/growatt_solar.html
I walked that second route: rspring/Esphome-Growatt
Thanks
This solution will remain open until I remove the integration from HA Core. At which point the conversation can continue over on the HACS repo I posted above, but I don’t maintain that either anymore because the API is impossible to work with. One of my missions is to try and work with the maintainer of grott to make it more human-friendly!
Take a read of my overview/comment above: https://github.com/home-assistant/core/issues/100874#issuecomment-1739911377
grottserver has the option to do this, but you have to know precisely what you’re doing e.g. what registers etc. to write to. I’m planning on taking a look at implementing that functionality as part of the HA integration next (after I’ve implemented the dual system I was describing above)
This chain has been impossible for me to follow, so I’ve pretty much given up on trying to follow it.
For anyone wanting help getting Grott setup with (or without for that matter) my integration, then I’ll happily do it in individual requests/issues in my github for the grott-homeassistant integration here: https://github.com/muppet3000/homeassistant-grott
Here a new (?) API documentation: https://www.showdoc.com.cn/262556420217021/1494053950115877
Just to add my 2 pennies… it’s an issue with Growatt’s API server. They seem to have issues with their AWS setup and the WAF is causing issues thinking every request is suspicious. My guess is they have been messing about with their backend and fluffed something up.
https://server.growatt.com/ is even having various issues with 500 server errors and plants cant be found. The shinephone phone app will not even login. I have raised a support call with their UK support team.
There is nothing we can do while ever the integration uses the http://server-api.growatt.com/ address and have to wait for Growatt to sort it.
How can change this in the Intergration for all users?
I have problems too. I just sent an email to Growatt to find out the reason for the block on https://server-api.growatt.com/.
If you want to get nicely formatted output rather than the stock MQTT stuff including stat names and correct entity types you can use my Grott integration for that as well: https://github.com/muppet3000/homeassistant-grott
It still needs to have Grott running and pushing the data to MQTT it just means you get nicely formatted data.
I found my issue with not all the values showing from the inverter / MQTT that were showing in Shine.
Originally I used ‘default’ but not all the entities were published to MQTT. I had a look on the side of my inverter for the model and its an SPH, changed the grot.ini to SPH inverter type, restarted Grot, deleted the Grot Integration I had configured and re-added it, then once the Inveter published again, 5 mins later, all 66 entities were published 😃
Now I have 66 entitites rather than 32 entities , including Battery and export to grid.
Well, this thread becomes quite long, so long that I’m lost to be honest. One basic question:
Is this issue going to be solved or not ? (and can we declare this integration dead and I have to look for other options like workarounds given here) I’m willing to wait but if this is never ever going to be solved then well…
All these comments about switching to the other api service or whether they’re going to re-instate the old api ‘service’ is pointless in the context of this thread. It was never an officially supported API, we just spoofed as if we were the android app, and they’ve found a way to stop that happening, because it was never supported.
So, I received the following message
If you have your own monitoring platform and want to obtain the data from Growatt, please use openapi.growatt.com.
On Mon, Oct 2, 2023, 12:42 PM Sweenylein @.***> wrote:
That makes the app working back again but the HA integration is still having issues connecting
They definitely have some sort of IP restriction in place. I couldn’t access with the app today, yet changing to mobile data or using a VPN it would load instantly.
Latest response from Growatt Netherlands this morning:
**_There is still a problem with the Growatt servers at the moment. Growatt is aware of this and is working hard on a solution. This nuisance is expected to continue until the weekend.
During this period it may happen that logging in is not possible, data is not displayed or not fully displayed in the app or on the website, or that data loggers lose their connection.
We thank you for your understanding._**
They could also just expose a local API on their dataloggers so HA wont hit their servers…
Try to remove the “,” at your Last entry at your server_urls section
The old API worked fine Yesterday, today there is Just 0,1kwh, although in reality there are 10kwh (i am tracking the growatt with a Shelly 1pm plus). Since 9 o clock there are no solar values anymore at the old API.
Well I’ll be darned! I thought might be the case, but given that this integration is a native hacs integration, that it would be the most complete and effective solution, but ideally, the HA inclusion needs to ask the user for an online/offline deployment choice is made, then tumbles out into GROTT or growatt_server relevantly,
Right? I need to get a few thoughts from users before trying to make that happen though…
Indeed, who can update the Integration, such that the new URL is recognized?
I had the same issue, i also dont run HA on a Pi. Thanks to all the replies above, I combined the two solutions and got it working like this.
save it.
restart HA
Now re-add the Growatt Integration, except this time you should see a cloud icon and the local integration icon
This shows your local integration has been recognised
Login using your Shine Tool Creds
Growatt works again.
Thanks everyone for commenting above to work out this solution.
The same issue persists, but https://server.growatt.com/ works fine. The problem lies with the Growatt integration; it doesn’t load data, and the ShinePhone application has the same issue - I cannot log in. However, when I log in via a mobile network, it works without any problems on my mobile phone. I suspect there might be a limitation on the number of login attempts from a specific IP address.
That’s exactly what’s done here: https://github.com/muppet3000/homeassistant-grott However it requires extra steps to set up.
Just use modbus and all is happy, then also all your data stays local 😃
That’s not what I meant - Log it in the github for my hacs version of the integration here: https://github.com/muppet3000/homeassistant-growatt_server_api/issues
For anyone still clinging on to this thread hoping that the old integration may one day work again, I’ve been informed of a URL that seems to work (but I have no idea for how long). I’ve released a new version of my HACS version of the integration here: https://github.com/muppet3000/homeassistant-growatt_server_api which seems to work (for now).
Thank you for author and all support from github.
Now instalation is very easy and its working fine. I will recomend grott in the Poland (on fb group)
That’s the thing, here’s the list, I don 't see it.
But in Growatt app I do have Load consumption daily/total and import from grid daily/total and data is proxied through Grott, so it must be available somewhere, lol EDIT: eacDischarge today looks like load consumption today I just noticed, This could be the import from grid values. Still I miss them as well as in the mqtt device as in muppet3000’s integration.
Any missing values that appear in the grott logs but aren’t shown by my integration, just log a ticket over here: https://github.com/muppet3000/homeassistant-grott/issues
This is the newly packaged adddon. There is a 2.7 branch aka stable and the 2.8 branch who is the develop branch.
The one you are using is going to become the 2.8 series. The difference is there is support for the spa and min series of inverter.
Thx, that did it. But still missing import-from-grid daily/total and consumption-daily/total entities. Inverter is Growatt SPF-5000es
@yvesdemoerloose did you enable the top option in de grott addon called: Enable grott Mqtt?
This is needed for the integration to find the ‘device’. After installing the integration click on Add Item leave it empty (it will show a + sign). After a few minutes it will find the device.
Hi,
If you are using hassos (The native version of HA) you can use the addons. I have packaged grott with autoconfiguration of MQTT (using the mqtt addon in HA). You can find the repo at this address: https://github.com/egguy/grott-home-assistant-add-on/ It is compatible with @muppet3000 integration.
@muppet3000 I’ll start with reading your tutorial again and let you know if anything can be improved. Love this community 😃
If you do make a video, please let me know and I’ll add it to the documentation. If you want to help improve the documentation, I’ll happily accept changes & PRs to my repo which seems to be the best ‘tutorial’ so far: https://github.com/muppet3000/homeassistant-grott/blob/main/docs/prerequisites.md
This is the bit I need to update the documentation to reflect. It’s on my list!
Added that as well, to have both, good job! For anyone who wants to use this: You have to make sure to enable this in the grott add-on to get the datalogger automatically:
Enable grott MQTT Allow grott to send MQTT messages. If your are using the grott_ha extension, you don’t need this. If you are using the grott integration, you need to enable it.
Ok so it means this is all for today 😄 I have night. Thanks
Exactly the one I used.
Use this addon, https://github.com/egguy/grott-home-assistant-add-on
That ain’t nothing, In Europe we were almost forced to take covid shots 😉 LOL
Sorry, but can someone moderate this topic? Its not helpful to find a diffrent solution instead of fixing this issue in the current integration.
After reading (most) of this thread, I am so glad I switched to local monitoring (ESP-32 + MAX-485 + ESPHome). I have the ESPHome code available to anyone who wants to go that route. Based on: https://esphome.io/components/sensor/growatt_solar.html
That’s very kind, thank you. To be clear, I didn’t make grott, that was another developer. My integration just nicely visualises all the data from grott in Home Assistant for people.
If it’s the same as in the web interface, the password is growatt + today’s date in the format YYYYMMDD
so today would be
growatt20231004
If i were you, i would just call the ISP to port forward the Grott server to the internet:
so it should look something like this
A lot of different setup could be put in place but I guess this is the simplest solution. i would suggest setting up some firewall rules on the Grott server to only allow data from LAN and the external wan address of your off-site inverter, just in case.
I assumed there was already a server/router or anything on site. if not, things indeed get more complicated. Perhaps forwarding the grott port could work. Renting a cheap VPS for this could allow you to bypass the ISP shenanigans. Good luck 👍🏻
The shinelanbox part of this isn’t necessary, in fact it would replace whatever you’re currently using e.g. the wifi stick. All you need to do is configure your existing device to point to grott.
Has anyone established from Growatt if they are going to reinstate the Server-API.growatt.com option, as all other options seem to be a load of work with potential to break the link to the official reporting.
Hopefully it’s ok for me to post this. For all of us that now have missing Kwh, there is a easy way to ‘repair’ this. You don’t even have to use SQL for this to insert the data but use the UI. Make sure to make a backup before you start doing this and check the outcome after you fixed 1 day. do this at your own risk 😃 You do need to put all the missing Kwh on the last day you have received data. I’ll post screenshots to make it very simple. First make sure you don’t delete Growatt in the dashboard!!!
I’m running Grott on an old raspberry pi (rp 1 model b). Doesn’t seem to use too much at all.
It works differently depending on whether you’re viewing it from a browser vs android app vs python client the server responds differently
Log a ticket in the https://github.com/muppet3000/homeassistant-grott repo and I’ll assist you there.
Just to respond to a few comments I’ve seen/direct emails I’ve had.
I did not want to give up on this integration, I spent a load of effort on it at the beginning of the year moving it into HACS so I could support it better. The decision has been taken out of my hands though, Growatt do not have a supported API. The work that Indy and I did to make the python library was entirely through reverse engineering and that is prone to being changed without any notice/support from Growatt. Therefore, I moved to the alternative - remove the growatt servers from the loop.
I appreciate that for the many grott seems daunting and complicated and the documentation isn’t great, however, it does work and I’m doing my best to assist the developer to improve it and to provide additional documentation myself on my integration’s page as to how to set it up. That is a far better & more productive use of my time than fighting against the ever-changing Growatt api.
I hope people can understand that. I’d love to support this method, it’s simple, it worked and I even had plans to add configuration, but it has been taken out of my hands.
Grott is WAY to complicated for me - perhaps rather than close this issue we can keep it open whilst a solution is looked for???
Now, even server.growatt.com stop working here. That’s really terrible.
Just to respond to this comment specifically - This Integration: https://github.com/muppet3000/homeassistant-grott depends on you having grott (https://github.com/johanmeijer/grott) running. The Integration for home assistant handles all the nice stuff like mapping ‘proper’ data values from MQTT into the nice format that HA provides including specifying the correct energy types etc.
Works flawlessly Edit: Oh wait, i use this: https://github.com/johanmeijer/grott
Not as soon as it needs to log in again. If you remove it from Tibber and try to add it again you get an error
I like when Github bug reports generate this much interest. I have nothing new to report, 9.3 with no settings changed has not connected in the past three days.
So to sum up, the options right now seem to be:
EDIT: @saurjk got a response from Growatt customer support further down in the comments, quoting from them:
So I will definitely be waiting this out for now
Do nothing and wait until the integration is updated to include more or different server addresses Pro: project maintainer will be aware of the issue and might have mitigation strategies we can not assess from here, does not endanger accumulated energy data if dataset provided by Growatt remains the same - there are reports that changing the server URL also changes the names of sensors because the other API url reports back different sensor names Con: no timescale, several possible points of failure (Growatt and integration)
Do something: fork integration to custom integration, change variables for server URL there Pro: might be fastest way to mitigate issue and get data back Con: needs somewhat solid knowledge of how to fork and edit files; if custom integration is not replaced by “regular” integration after fix is deployed, custom integration will not pick up changes and improvements to baseline plugin; accumulated energy data might be lost if sensor names change because of different data provided
Do something: edit files in HASSIO core Pro: you’re living in the fast lane Con: I seriously wouldn’t
Me, personally, having forked a different custom component to “fix a bug” in another integration and losing all the gathered data associated with that integration I will accept that my HA-related knowledge of forking and fixing is not sufficient and I will sit this out waiting for either option 1 (Growatt gets their stuff in order) or option 2 (integration gets updated) to come to pass.
Other options, unrelated to this integration and not possible in every use case:
Assuming you’re running a low kW (the famous “Balkonkraftwerk”) that plugs into Schuko, you could put a wifi plug with power output/input reading where your Growatt meets your power circuits and use that value as the wattage output sensor (might need to flip the negative value). Pro: you’re no longer beholden to the whims of Growatt Con: you only get 1 reading (the wattage) and you have to invest in a wifi plug and you have to have the required infrastructure in place
If all else fails, try and read the data stick locally without going through the cloud. Might need reflashing, might need some sort of local proxying, have not yet looked into this because 1. you will likely lose your accumulated energy data and 2. I’d rather not.
It is possible - https://github.com/johanmeijer/grott
I only performed till step 6 and after that the integration worked with the new (deprecated) url again. When you remove your existing integration and add it again, you will lose your sensors and also your database history. Therefore a empty Energy Dashboard of your Growatt Inverter.
My problem is solved to, only needed to delete te existing intregratie, added again, only i got net Indenty’s… But a lot more then before.
Thank you that was the Problem. Now it works fine for me again 😃
your manifest.json is missing a comma
You have a missing comma in your manifest.json
At the end of the “requirements” line, add a comma and restart HA
Be sure to remove the existing Growatt integration first
Thank you for your clear instructions on the solution. I have zero experience with code and could follow the step-by-step guide you created
I will only mention why removing ‘https://server.growatt.com/’ from DEPRECATED_URLS is required.
Confirmed. Changing the URL seems to work! (The question is for how long?).
When I connect my Growatt inverter to my Domoticz server (HTTP / HTTPS), I immediately have all the information available and it works.
This integration looks at server.growatt.com.
There are only problems in combination with HA.