minio-java: Listing objects fails with SignatureDoesNotMatch if > than 1000 objects

The minio java client 2.0.1 gives me a very strange behaviour as soon as there are more than 1000 objects returned by the listObjects call.

  • It lists all objects up to the 1000
  • Then it throws the attached error complaining about the signature
  • I tested it with the mcclient, which works like a charm and lists all objects, thus the issue is very likley in the java client code
  • The error does not depend at all on the content.
Error occurred: error occured
ErrorResponse(code=SignatureDoesNotMatch, message=The request signature we calculated does not match the signature you provided. Check your key and signing method., bucketName=null, objectName=null, resource=/media, requestId=3L137, hostId=3L137)
request={method=GET, url=http://goldbox:9000/media?marker=Videos/Movies/2016/some/path/file.jpg, headers=Host: goldbox:9000
User-Agent: Minio (x86_64; x86_64) minio-java/dev
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date: 20160912T110657Z
Authorization: AWS4-HMAC-SHA256 Credential=P6BE2YPLY0FPHIQ5CVZV/20160912/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=6975bb3352dfc687317cae9e8e66353ae0d219516885db1cb7d584b6905eaf47
}
response={code=403, headers=Accept-Ranges: bytes
Server: Minio/DEVELOPMENT.GOGET (linux; amd64)
Vary: Origin
X-Amz-Request-Id: OR86OFVUFWR6V5DE
Date: Mon, 12 Sep 2016 11:06:15 GMT
Content-Type: text/xml; charset=utf-8
Transfer-Encoding: chunked
OkHttp-Selected-Protocol: http/1.1
OkHttp-Sent-Millis: 1473678417911
OkHttp-Received-Millis: 1473678417913
}

Is there some magical barrier which gets hit when there are more than 1000 items returned?

About this issue

  • Original URL
  • State: closed
  • Created 8 years ago
  • Comments: 17 (17 by maintainers)

Commits related to this issue

Most upvoted comments

@IsNull Thanks for finding this bug. I just sent a fix for this issue. Please verify whether it fixes the issue your side.

That is great, thank you! Lucky that I uploaded the test data^^

I found the problem. Signer.java does double encoding for spaces ie %20 becomes %2520. Fixing it now.