carbon-aware-sdk: [Bug]: Too much data generated by default when polling `/emissions/bylocation` endpoint

Contact Details

GitHub/Slack: Willmish

What happened?

When calling GET on the /emissions/bylocation endpoint, and only providing Location and not a time frame, by default the SDK requests data for a 7 days in 5 minute intervals (hundreds of data).
When calling the endpoints with the same time and toTime , SDK will respond with 2 values: before time and after toTime

This doesn’t seem like a healthy default for the SDK.

Instead, I suggest:

the SDK by default asks for 20-50 data points in 5 minute intervals, going from most recent to a few hours before current time.
SDK should not give values after toTime, only up to - this way we will also receive a single value when time is the same as toTime

Example Swagger UI query which produces a lot of data: Screenshot 2022-07-27 at 16 33 52

client

WebAPI (Default)

Relevant log output

No response

Code of Conduct

I agree to follow this project’s Code of Conduct

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 20 (18 by maintainers)

Most upvoted comments

Hi @NickVilimek sorry for the delay.

I agree with you that if someone provides a start AND end time, then it should calculate and override the default limit.

My thoughts were (especially for new people using this) not to be overwhelmed by results. So I would have expected that we could have some default values:

no start time => defaults to now
no end time => stops when results limit is reached
pagesize or nbResults (as no page really) => 20 (use -1 or less than zero for no limit? so it is clear when it has been overridden)
location => I expect is the only required parameter or would we want to default this too?
interval => defaulted to 5 minutes

So for example for different scenarios (with at least location):

provide start time only => defaults nb of results or end time from nb of results
provide start and end time => override nb of results to match start and end time requested
provide nothing more => 20 first result for given location from now
provide 50 for nb of results => 50 first results for given location from now

(Generally, I would say start time, end time and interval would be priority over to override nb or results if that makes sense)

@NickVilimek does that make sense, and would that be ok? ( I believe the default value of less than zero is important to ensure we can do this as low level as possible so it applies across CLI and WebAPI, no?)

Let me know if above is not clear enough or misses key scenarios…

danuw on Aug 16, 2022