minio-go: Object.Read doesn't return error on truncated response
Thanos and Cortex are using the Minio client as S3 client. From time to time we’ve some reports of bugs that looks like if we get a partial/truncated response from the server which is not treated as an error from our application.
I’ve tried to simulate a scenario (which I don’t know how much realistic could be) where the server returns a body response smaller than the Content-Length. The Minio Object.Read() returns no error once all response has been consumed while, as comparison, the Google GCS client does return io.ErrUnexpectedEOF.
I’ve added a test in Thanos to show it: https://github.com/thanos-io/thanos/pull/3795
Questions:
- Why the Minio client doesn’t check if the received response size matches the
Content-Length? - The scenario described shouldn’t be reported as an error by the Minio client?
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 2
- Comments: 16 (12 by maintainers)
Commits related to this issue
- fix: make sure getObject returns error on truncated responses this PR also adds tests to handle these cases specially fixes #1451 — committed to harshavardhana/minio-go by harshavardhana 3 years ago
- fix: make sure getObject returns error on truncated responses this PR also adds tests to handle these cases specially fixes #1451 — committed to harshavardhana/minio-go by harshavardhana 3 years ago
- fix: make sure getObject returns error on truncated responses (#1452) this PR also adds tests to handle these cases specially fixes #1451 — committed to minio/minio-go by harshavardhana 3 years ago
- fix: make sure getObject returns proper error multiple reads As a continuation of work from #1452, when object sizes are bigger, we should calculate totalRead and match with expected object size. fi... — committed to harshavardhana/minio-go by harshavardhana 3 years ago
- fix: make sure getObject returns proper error multiple reads (#1453) As a continuation of work from #1452, when object sizes are bigger, we should calculate totalRead and match with expected object... — committed to minio/minio-go by harshavardhana 3 years ago
Will take a look @pracucci