thanos: ConsistencyDelayMetaFilter should filter on the block upload time and not the block creation time

The ConsistencyDelayMetaFilter filter currently filters out blocks based on the ULID time, which is the time when the block has been created: https://github.com/thanos-io/thanos/blob/cfced59c652aca5c311baa22b519e107a20ba422/pkg/block/fetcher.go#L727

This means that if that actual consistency delay is the configured delay minus the upload time. If the delay is X but the block upload time is > X, the consistency delay is effectively ignored.

To correct approach would be checking the consistency delay against the “block upload completed timestamp” and, to do so, checking the meta.json creation time could be fine. However, object stores stores the “updated time” and not the “creation time”, so I’m wondering how safe would be using it. Thoughts?

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 22 (14 by maintainers)

Most upvoted comments

I can’t find this ticket ;p So let’s keep this.

Definitely todo. We did not have just yet modified Time API in objstore that’s the reason why. Modified will be just enough I think, even though creation would be better here. The difficulty it so add working implementation of mod time to all implementations now.

We started some work in some old PRs. I think we also thought getting the Size, ModTime in one method Attr which is common pattern.

bwplotka on May 11, 2020