core: AirVisual making too many requests.

Yeah… Someone claimed victory a bit too early here. Of course it was working this morning, it was a new day, thus, the counter started from 0 again… I raised a ticket with AirVisual on day 0 (5 days ago) but no answer yet. I think we should accept the fact that they might have decided to reduce the number of calls to 500 a day and consequentially either allow the user to set a custom interval_time or to patch the code.

Heard back from IQAir today:

Hi Aaron,

We cannot check the API call amount in previous months. In November, you made 607 calls this month by far and 105 calls today. We didn’t change the call limits which are still as below. We can check again at the end of this month to see how many calls they make in a month in total. At the same time, if they see the error message again, they can send us a screenshot for troubleshooting. Thanks.

/Minute: 5 /Day: 500 /Month: 10,000

Best regards,

Edgar

My response:

Hello Edgar,

Thank you for the reply. Despite the numbers you reflect below (which are below the limits), even this morning, I am still receiving “call_per_day_limit_reached” responses from the API.

Will keep everyone posted as I hear more.

Submitted a helpdesk ticket to AirVisual; they said it may be up to two business days before I hear back.

All keys stopped counting at 502 calls… Anyhow, the entities in HA are never available…

Can confirm the same error, 117.1 and 117.2, entities are unavailable, the API key looks ok.

Tried to reload the AirVisual integration, no effect. Probably some failsafe threshold was added in place at HA, before reaching the airvisual?

This issue report needs more information to be actionable – specifically:

• What do DEBUG-level logs for AirVisual say?

I just removed the integration hoping to be able to recover it by re-configuring it but it didn’t work. I’m nuw unable to configure it again because of the limit error. Now I get this error, which is basically the same error, just in a different shape.

Logger: aiohttp.server Source: components/airvisual/config_flow.py:129 First occurred: November 2, 2020, 10:48:51 PM (4 occurrences) Last logged: 10:21:49 AM

Error handling request Traceback (most recent call last): File “/usr/local/lib/python3.8/site-packages/aiohttp/web_protocol.py”, line 418, in start resp = await task File “/usr/local/lib/python3.8/site-packages/aiohttp/web_app.py”, line 458, in _handle resp = await handler(request) File “/usr/local/lib/python3.8/site-packages/aiohttp/web_middlewares.py”, line 119, in impl return await handler(request) File “/usr/src/homeassistant/homeassistant/components/http/request_context.py”, line 18, in request_context_middleware return await handler(request) File “/usr/src/homeassistant/homeassistant/components/http/ban.py”, line 72, in ban_middleware return await handler(request) File “/usr/src/homeassistant/homeassistant/components/http/auth.py”, line 127, in auth_middleware return await handler(request) File “/usr/src/homeassistant/homeassistant/components/http/view.py”, line 129, in handle result = await result File “/usr/src/homeassistant/homeassistant/components/config/config_entries.py”, line 169, in post return await super().post(request, flow_id) File “/usr/src/homeassistant/homeassistant/components/http/data_validator.py”, line 60, in wrapper result = await method(view, request, *args, **kwargs) File “/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py”, line 106, in post result = await self._flow_mgr.async_configure(flow_id, data) File “/usr/src/homeassistant/homeassistant/data_entry_flow.py”, line 153, in async_configure result = await self._async_handle_step(flow, cur_step[“step_id”], user_input) File “/usr/src/homeassistant/homeassistant/data_entry_flow.py”, line 201, in _async_handle_step result: Dict = await getattr(flow, method)(user_input) File “/usr/src/homeassistant/homeassistant/components/airvisual/config_flow.py”, line 110, in async_step_geography return await self.async_step_geography_finish( File “/usr/src/homeassistant/homeassistant/components/airvisual/config_flow.py”, line 129, in async_step_geography_finish await cloud_api.air_quality.nearest_city() File “/usr/local/lib/python3.8/site-packages/pyairvisual/air_quality.py”, line 39, in nearest_city return await self._nearest(“city”, latitude, longitude) File “/usr/local/lib/python3.8/site-packages/pyairvisual/air_quality.py”, line 23, in _nearest data: dict = await self.request(“get”, f"nearest{kind}", params=params) File “/usr/local/lib/python3.8/site-packages/pyairvisual/cloud_api.py”, line 71, in _request raise_on_data_error(data) File “/usr/local/lib/python3.8/site-packages/pyairvisual/errors.py”, line 81, in raise_on_data_error raise error(message) pyairvisual.errors.AirVisualError: call_per_day_limit_reached

• How many instances of the AirVisual API are configured?

Just one. (Ehm, actually 0 at the present moment.)

• How many calls does your AirVisual API dashboard show? How does that number increase in a 5-to-10-minute span?

Unable to tell because the Integration stopped working…

all sensors showing unavailable since a couple of days, not sure if this coincided with updating to HA 117.x. This is all we get, I have but 1 instance configured:

added to the error in the issue above, this is shown in the logs for all sensors:

2020-11-03 10:15:35 ERROR (MainThread) [homeassistant.components.sensor] Error adding entities for domain sensor with platform airvisual
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 314, in async_add_entities
    await asyncio.gather(*tasks)
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 505, in _async_add_entity
    await entity.add_to_platform_finish()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 530, in add_to_platform_finish
    await self.async_added_to_hass()
  File "/usr/src/homeassistant/homeassistant/components/airvisual/__init__.py", line 419, in async_added_to_hass
    self.update_from_latest_data()
  File "/usr/src/homeassistant/homeassistant/components/airvisual/sensor.py", line 168, in update_from_latest_data
    data = self.coordinator.data["current"]["pollution"]
TypeError: 'NoneType' object is not subscriptable

more logs:

2020-11-03 10:23:44 DEBUG (MainThread) [homeassistant.components.airvisual] Leveling API key usage (6redacted5): 1 consumers, 5 minutes between updates
2020-11-03 10:23:44 DEBUG (MainThread) [homeassistant.components.airvisual] Updating interval for coordinator: {long/lat redacted}, 0:05:00
2020-11-03 10:23:45 ERROR (MainThread) [homeassistant.components.airvisual] Error fetching {long/lat redacted} data: Error while retrieving data: call_per_day_limit_reached
2020-11-03 10:23:45 DEBUG (MainThread) [homeassistant.components.airvisual] Finished fetching {long/lat redacted} data in 1.078 seconds
2020-11-03 10:23:45 ERROR (MainThread) [homeassistant.components.sensor] Error adding entities for domain sensor with platform airvisual
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 314, in async_add_entities
    await asyncio.gather(*tasks)
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 505, in _async_add_entity
    await entity.add_to_platform_finish()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 530, in add_to_platform_finish
    await self.async_added_to_hass()
  File "/usr/src/homeassistant/homeassistant/components/airvisual/__init__.py", line 419, in async_added_to_hass
    self.update_from_latest_data()
  File "/usr/src/homeassistant/homeassistant/components/airvisual/sensor.py", line 168, in update_from_latest_data
    data = self.coordinator.data["current"]["pollution"]
TypeError: 'NoneType' object is not subscriptable
2020-11-03 10:23:45 ERROR (MainThread) [homeassistant.components.sensor] Error while setting up airvisual platform for sensor
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 205, in _async_setup_platform
    await asyncio.gather(*pending)
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 314, in async_add_entities
    await asyncio.gather(*tasks)
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 505, in _async_add_entity
    await entity.add_to_platform_finish()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 530, in add_to_platform_finish
    await self.async_added_to_hass()
  File "/usr/src/homeassistant/homeassistant/components/airvisual/__init__.py", line 419, in async_added_to_hass
    self.update_from_latest_data()
  File "/usr/src/homeassistant/homeassistant/components/airvisual/sensor.py", line 168, in update_from_latest_data
    data = self.coordinator.data["current"]["pollution"]
TypeError: 'NoneType' object is not subscriptable
2020-11-03 10:23:45 DEBUG (MainThread) [homeassistant.components.airvisual] Finished fetching {long/lat redacted} data in 0.321 seconds
2020-11-03 10:23:45 ERROR (MainThread) [homeassistant.components.sensor] airvisual: Error on device update!
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 357, in _async_add_entity
    await entity.async_device_update(warning=False)
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 466, in async_device_update
    await task
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 235, in async_update
    await self.coordinator.async_request_refresh()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 126, in async_request_refresh
    await self._debounced_refresh.async_call()
  File "/usr/src/homeassistant/homeassistant/helpers/debounce.py", line 73, in async_call
    await self.hass.async_add_hass_job(self._job)  # type: ignore
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 193, in async_refresh
    update_callback()
  File "/usr/src/homeassistant/homeassistant/components/airvisual/__init__.py", line 414, in update
    self.update_from_latest_data()
  File "/usr/src/homeassistant/homeassistant/components/airvisual/sensor.py", line 168, in update_from_latest_data
    data = self.coordinator.data["current"]["pollution"]
TypeError: 'NoneType' object is not subscriptable

airvisual dashboard on https://www.iqair.com/dashboard/api:

I got the same message:

This API call limit issue has been fixed. Please try again and let us know if there is any further questions.

Let’s leave this issue open until the end of the week; if everyone is looking good, we’ll close.

@Ndrinta To answer your question:

Do you know if there is a way to check when every single call is made by the integration? Should I set a deeper log level?

…you can do something like this in your configuration.yaml:

logger:
  default: info
  logs:
    homeassistant.components.airvisual: debug

I think there’s something going on on their end. The most frequent API ping interval used by the HASS integration is 1 call per 4.7 minutes (based on a monthly calculation); that amounts to 304 calls per day, which is well under the represented daily limit.

Something doesn’t add up here, If there are too many requests then it should work for a period of time until we hit their limit but it doesn’t work at all at any time of the day. It worked for a day or so then stopped again and zero changes were made by @bachya which makes me think that it is something they have/are doing with their API.

let’s hope, because the newly created Api key has worked for exactly 1 day … hit the same error once again just now. gr…

Same here. On 0.117.4

I am also getting this for one or two days now. I have 2 instances with the same API.

It’s only this error on repeat in my case

2020-11-03 10:52:34 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 119, in _handle_refresh_interval await self.async_refresh()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 193, in async_refresh update_callback()
  File "/usr/src/homeassistant/homeassistant/components/airvisual/__init__.py", line 414, in update self.update_from_latest_data()
  File "/usr/src/homeassistant/homeassistant/components/airvisual/sensor.py", line 168, in update_from_latest_data data = self.coordinator.data["current"]["pollution"]
TypeError: 'NoneType' object is not subscriptable
2020-11-03 10:52:35 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 119, in _handle_refresh_interval await self.async_refresh()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 193, in async_refresh update_callback()
  File "/usr/src/homeassistant/homeassistant/components/airvisual/__init__.py", line 414, in update self.update_from_latest_data()
  File "/usr/src/homeassistant/homeassistant/components/airvisual/sensor.py", line 168, in update_from_latest_data data = self.coordinator.data["current"]["pollution"]
TypeError: 'NoneType' object is not subscriptable

Same thing happened to me. Testing API with curl returns {"status":"fail","data":{"message":"call_per_day_limit_reached"}}

I’m not sure how to find DEBUG-level logs, can you share details? I have one instance of AirVisual API The dashboard shows 502 requests in both a day and in a month.

use this to set debug level: