fake-gcs-server: invalid character '\x1f' looking for beginning of value when uploading via java apis

Hi, I’m trying to create a bucket in the fake gcs server using the java apis(which don’t seem to be documented, happy to contribute with some docs once I get this working) but getting

 invalid character '\x1f' looking for beginning of value
[info]   at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:229)
[info]   at com.google.cloud.storage.spi.v1.HttpStorageRpc.create(HttpStorageRpc.java:272)
[info]   at com.google.cloud.storage.StorageImpl$2.call(StorageImpl.java:121)
[info]   at com.google.cloud.storage.StorageImpl$2.call(StorageImpl.java:118)
[info]   at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105)
[info]   at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
[info]   at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
[info]   at com.google.cloud.storage.StorageImpl.create(StorageImpl.java:117)

This is the code:

    val transportFactory = new HttpTransportFactory {
      override def create(): HttpTransport =
        new NetHttpTransport.Builder().doNotValidateCertificate().build()
    }

    val storage = StorageOptions.newBuilder
      .setCredentials(NoCredentials.getInstance)
      .setTransportOptions(HttpTransportOptions.newBuilder().setHttpTransportFactory(transportFactory).build())
      .setHost(s"https://$host:$port")
      .setProjectId(projectId)
      .build()
      .getService

    storage.create(BucketInfo.of(bucketName))

Can’t really figure out what I’m doing wrong, does anybody have any idea?

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 3
Comments: 28 (9 by maintainers)

Most upvoted comments

@fsouza (@flylo ), I got it.

https://github.com/dnatic09/Fake-Gcs-Test

Just clone and run mvn test .

One test will pass (rigged CURL). One test will fail due to GZIP (using library).

dnatic09 on Dec 24, 2020

I can make a reproducer overnight. I will message back when it is ready.

dnatic09 on Dec 23, 2020

@flylo , I ran into the same problem. The funny thing is that you get an NPE, but the file is actually in the fake GCS server and can be pulled. I already tested it. We should definitely help fix it, but there is definitely a path forward here.

dnatic09 on Dec 10, 2020

@fsouza , a follow-up. I simply hacked the debugger and set this.disableGzipContent to true. The request WORKED! So, the Fake GCS server is failing to decode gzipped content. Either we need to find a way to disable gzip encoding in the SDK or the GCS server needs to parse the Content-Encoding HTTP header and decompress the data. Can we do this?

dnatic09 on Dec 9, 2020