garmin-grafana: [BUG] Intraday data not imported for historical data older than 6 months

HR and sleep data in the past does not import for any year other than 2025. Example below for 2 days crossing the year boundary 2024-12-31 and 2025-01-01. Only 2025 data is imported

command:

docker compose run --rm -e MANUAL_START_DATE=2024-12-31 -e MANUAL_END_DATE=2025-01-01 -e FETCH_SELECTION=heartrate garmin-fetch-data

Log

[+] Creating 1/1
 ✔ Container influxdb  Running                                                                                                    0.0s 


2025-05-02 11:20:15,161 - INFO - Trying to login to Garmin Connect using token data from directory '~/.garminconnect'...
2025-05-02 11:20:15,845 - INFO - login to Garmin Connect successful using stored session tokens.
2025-05-02 11:20:15,846 - INFO - Fetching data for the given period in reverse chronological order
2025-05-02 11:20:19,024 - INFO - Success : Updated device last sync time
2025-05-02 11:20:19,040 - INFO - Success : updated influxDB database with new points
2025-05-02 11:20:19,266 - INFO - Success : **Fetching intraday Heart Rate for date 2025-01-01**
2025-05-02 11:20:19,344 - INFO - Success : updated influxDB database with new points
2025-05-02 11:20:19,344 - INFO - Success : Fetched all available health metrics for date 2025-01-01 (skipped any if unavailable)
2025-05-02 11:20:19,344 - INFO - Waiting : for 5 seconds
2025-05-02 11:20:24,511 - INFO - Success : Fetched all available health metrics for date 2024-12-31 (skipped any if unavailable)
2025-05-02 11:20:24,511 - INFO - Waiting : for 5 seconds
2025-05-02 11:20:29,512 - INFO - Bulk update success : Fetched all available health metrics for date range 2024-12-31 to 2025-01-01

Are you using docker?

  • Yes

Are you using a VPN?

  • No.

Did you read the README and tried to Troubleshoot?

  • I have manually ran the garminconnect API garmin.get_rhr_day(cdate="2024-12-31") for that date to confirm I have HR data for that day

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 41 (22 by maintainers)

Commits related to this issue

Most upvoted comments

Hi @brunothesatellite @alanosullivancs @lolokraus

This has been implemented with the latest commit https://github.com/arpanghosh8453/garmin-grafana/commit/7fcbf5e60548103ccccb7e8650ab6804156dce8f

Now, you can essentially use REQUEST_INTRADAY_DATA_REFRESH=True in the compose file under the garmin-fetch-data container ENV variables to request data refresh automatically. Please note that using this will automatically pause the script for 24 hours as soon as you hit the rate limit for the refresh request (as that is enforced by the Garmin’s servers) and will resume fetching the data thereafter. This refresh request is only performed and applicable on dates older than 6 months (180 days) from current date - as that is the archival limit for Garmin database. This will allow you to fetch your long term data in intraday level, but you need to run the script for about 2 months without any break (or shutdown) in order to fetch 5 years equivalent of data (assuming the rate limit remains 30 refresh request per day). If the script exits or you shut down the machine running it, please note the last day for which the data is fetched (fetching is done in reverse chronological order) and set that as the MANUAL_END_DATE when restarting the bulk fetch.

Hope that helps. I will keep this issue open for now, although this is the best we can do here.

Hi @alanosullivancs, Thank you so much for reporting this back. Can you please tell me how long it took to actually get the refreshed data after you get the "SUBMITTED" response (is that under 10 seconds)? and you do see the "COMPLETE" status message if you request the same date after a while (within 7 days I think), right? Can you check this once for me?

I can possibly add a ENV variable REQUEST_INTRADAY_DATA_REFRESH which can handle this. When set to True (by default False), it will pause the whole script for 24 hours once it hits the "status": "DENIED". We can’t bypass this limit, because it’s enforced by Garmin’s servers, but we can have this option if someone actually wants to fetch past historical intraday data. with the current limit, we can fetch 5 years (+ 6 months free) worth of historical data if we run the script with the said modification (REQUEST_INTRADAY_DATA_REFRESH=True) for 2 months continuously, and that does not seem impossible to me. The flag REQUEST_INTRADAY_DATA_REFRESH is not implemented yet, but let me know your thoughts on this. It will save you from automating this with a script yourself.

it can be re-populated within the app and then is available once more in the API! I wonder is there a way to trigger the endpoint that regenerates the data to make the historical import more feasible than instead of having to navigate each day for several years and regenerate each day?

Hey @brunothesatellite @alanosullivancs

Found a potential solution from this thread on python-garminconnect. This is a known and reported issue which I was not aware of.

The refresh endpoint is wellness-service/wellness/epoch/request/YYYY-MM-DD (found from here)

Image

Can @alanosullivancs make a request like garmin.connectapi("wellness-service/wellness/epoch/request/2024-08-20", method="POST") followed by the garmin.get_heart_rates(cdate="2024-08-20") to see if you get the data (without manually refreshing/reloading the date from the app itself?

If this workaround works, I will add automatic refresh before data pulling as you requested. Also I will increase the periodic fetch so that Garmin servers don’t get bombarded with too many requests. We should be nice on their servers.

I am sorry @alanosullivancs for not looking at your message pictures and commenting something is wrong with your account. I was actually on a hike and I was responding solely based on the message previews I got while walking, the images did not load. They say that the reloading is limited per day and they pull the data from a cold storage - so it’s expensive. I think there is no way we can bypass that limit and it’s enforced on the server. would it be best to pause the whole import for 24 hours after the refresh limit is reached or we should import the data whatever is available? I am confused on decision here. I believe for long term visualization you do not need the Intraday data. and the avg data is available for anytime, so maybe we should continue fetching and put a notice that intraday data refresh requires running the bulk update on a smaller date range over multiple days - there is no workaround. Of course, if you start using this script early on, that won’t be a problem as all of your older data will be in the local database itself.

waiting for your response and opinion. Thank you. Just got home and started investigating this.

Same issue with sleep data:

Image

Image

Image

you can close the thread @arpanghosh8453 but there is nothing “wrong” with my account. see my screenshots. the data is not there at that API endpoint until I force refresh the days. Garmin are limiting access to it intentionally

@alanosullivancs you can try the route @matin suggested on https://github.com/arpanghosh8453/garmin-grafana/issues/57#issuecomment-2836942759 to identify the request endpoints when you triggure a refresh from the app for older data.

Ok, got it. There is an issue indeed:

Image

Image

Image

I don’t find where to change the date for this specific panel, could you point out where to apply the modification?

@brunothesatellite The top right dropdown where it says last 30 days, if you open that you will see a date picker. Pick a date range older, and see if you see the intra day HR data for that old range in the top HR panel.

@alanosullivancs I understood that after I wrote that comment. It might be an universal problem. I don’t have enough data to test this on my end. So I requested @brunothesatellite to test this out for a specific past date older than current year for the intraday HR data (top most panel) to know if their data was imported successfully or not.

If they have the sane problem, then we want to keep this open to let other users know the current limitations of Garmin API.

As the data can be refreshed within the app, maybe they archive it and only pull on request. We need to figure out the endpoint that triggures the refresh.

you can close the thread @arpanghosh8453 but there is nothing “wrong” with my account. see my screenshots. the data is not there at that API endpoint until I force refresh the days. Garmin are limiting access to it intentionally