sidekiq-statistic: timeslist does not expire
The timeslist should be expired after some amount of time and (I’d suggest) the times aggregated into a much more compact form. The Web UI could store hourly aggregates per worker.
If I were implementing this, I’d compact the timeslists hourly. The compact form could store:
execution count
average time
standard deviation
98th percentile
etc
So instead of a list of thousands of numbers, it would be converted into a hash with 4-6 numbers per hour. You could expire the hashes after one month, giving the user a nice recent history to compare.
About this issue
- Original URL
- State: closed
- Created 9 years ago
- Reactions: 2
- Comments: 16 (5 by maintainers)
Links to this issue
Commits related to this issue
- Remove sidekiq-statistic gem. * We don't really use it and there is an on-going issue with the gem not expiring keys in a Redis list which hasn't been fully resolved. https://github.com/davydo... — committed to discourse/discourse by tgxworld 7 years ago
- Remove sidekiq-statistic gem. * We don't really use it and there is an on-going issue with the gem not expiring keys in a Redis list which hasn't been fully resolved. https://github.com/davydo... — committed to chrisbeach/discourse by tgxworld 7 years ago
- Removes sidekiq-statistic as it can be a memory leaks https://github.com/davydovanton/sidekiq-statistic/issues/73 — committed to sul-dlss-deprecated/discovery-dispatcher by mejackreed 6 years ago
@davydovanton I’ve put this gem on the “troublesome” list and removed it from the Related Projects wiki page since this bug hasn’t been fixed.
Sorry for the delay @drn.
I was actually referring to this piece of code: https://github.com/davydovanton/sidekiq-statistic/blob/master/lib/sidekiq/statistic/statistic/metrics/store.rb#L52
It was added in the past and since then we haven’t seen activities here, that’s the reason I closed the issue.
We should probably have other approaches to handle it and I really think we can discuss further, if that’s the case we may want to create a new issue to change what we actually have.
+1, Redis killed by the Linux Out of Memory Killer.
As a workaround, I clear all the timeslist on a regular basis like this:
@MrJoy This did seem to address the issue we were having, yes. It safely drops (very) old history once an upper bound is reached, so perhaps not the most elegant solution, but works for us while still rendering the tool very useful.