azure-sdk-for-java: [BUG] ListBlobs failing with Azurite

Describe the bug Cannot use Azurite for local development as listBlobs() (in BlobContainerClient) fails.

Exception or Stack Trace local:

[WARNING]
com.azure.storage.blob.models.BlobStorageException: Status code 400, (empty body)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0 (Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance (NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance (DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance (Constructor.java:423)
    at com.azure.core.http.rest.RestProxy.instantiateUnexpectedException (RestProxy.java:357)
    at com.azure.core.http.rest.RestProxy.lambda$ensureExpectedStatus$6 (RestProxy.java:422)
    at reactor.core.publisher.MonoDefer.subscribe (MonoDefer.java:44)
    at reactor.core.publisher.Mono.subscribe (Mono.java:4087)
    at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onComplete (FluxSwitchIfEmpty.java:75)
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.onComplete (MonoFlatMap.java:174)
    at reactor.core.publisher.FluxMap$MapSubscriber.onComplete (FluxMap.java:136)
    at reactor.core.publisher.MonoProcessor$NextInner.onComplete (MonoProcessor.java:518)
    at reactor.core.publisher.MonoProcessor.onNext (MonoProcessor.java:308)
    at reactor.core.publisher.MonoProcessor.onComplete (MonoProcessor.java:265)
    at reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onComplete (FluxDoFinally.java:138)
    at reactor.core.publisher.FluxHandle$HandleSubscriber.onComplete (FluxHandle.java:206)
    at reactor.core.publisher.FluxMap$MapConditionalSubscriber.onComplete (FluxMap.java:262)
    at reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onComplete (FluxDoFinally.java:138)
    at reactor.core.publisher.FluxFilterFuseable$FilterFuseableConditionalSubscriber.onComplete (FluxFilterFuseable.java:384)
    at reactor.core.publisher.Operators$MonoSubscriber.complete (Operators.java:1593)
    at reactor.core.publisher.MonoReduceSeed$ReduceSeedSubscriber.onComplete (MonoReduceSeed.java:156)
    at reactor.core.publisher.FluxMap$MapSubscriber.onComplete (FluxMap.java:136)
    at reactor.netty.channel.FluxReceive.terminateReceiver (FluxReceive.java:397)
    at reactor.netty.channel.FluxReceive.drainReceiver (FluxReceive.java:197)
    at reactor.netty.channel.FluxReceive.onInboundComplete (FluxReceive.java:345)
    at reactor.netty.channel.ChannelOperations.onInboundComplete (ChannelOperations.java:363)
    at reactor.netty.channel.ChannelOperations.terminate (ChannelOperations.java:412)
    at reactor.netty.http.client.HttpClientOperations.onInboundNext (HttpClientOperations.java:556)
    at reactor.netty.channel.ChannelOperationsHandler.channelRead (ChannelOperationsHandler.java:91)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead (AbstractChannelHandlerContext.java:374)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead (AbstractChannelHandlerContext.java:360)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead (AbstractChannelHandlerContext.java:352)
    at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead (CombinedChannelDuplexHandler.java:438)
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead (ByteToMessageDecoder.java:328)
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead (ByteToMessageDecoder.java:302)
    at io.netty.channel.CombinedChannelDuplexHandler.channelRead (CombinedChannelDuplexHandler.java:253)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead (AbstractChannelHandlerContext.java:374)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead (AbstractChannelHandlerContext.java:360)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead (AbstractChannelHandlerContext.java:352)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead (DefaultChannelPipeline.java:1422)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead (AbstractChannelHandlerContext.java:374)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead (AbstractChannelHandlerContext.java:360)
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead (DefaultChannelPipeline.java:931)
    at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady (AbstractEpollStreamChannel.java:792)
    at io.netty.channel.epoll.EpollEventLoop.processReady (EpollEventLoop.java:483)
    at io.netty.channel.epoll.EpollEventLoop.run (EpollEventLoop.java:383)
    at io.netty.util.concurrent.SingleThreadEventExecutor$6.run (SingleThreadEventExecutor.java:1044)
    at io.netty.util.internal.ThreadExecutorMap$2.run (ThreadExecutorMap.java:74)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run (FastThreadLocalRunnable.java:30)
    at java.lang.Thread.run (Thread.java:748)

Azurite:

127.0.0.1 - - [18/Dec/2019:14:29:13 +0000] "GET /devstoreaccount1/mycontainer?restype=container HTTP/1.1" 404 -
127.0.0.1 - - [18/Dec/2019:14:29:14 +0000] "PUT /devstoreaccount1/mycontainer?restype=container HTTP/1.1" 201 -
127.0.0.1 - - [18/Dec/2019:14:29:14 +0000] "GET /devstoreaccount1/mycontainer?restype=container HTTP/1.1" 200 -
127.0.0.1 - - [18/Dec/2019:14:29:14 +0000] "GET /devstoreaccount1/mycontainer?include=&restype=container&comp=list HTTP/1.1" 400 -

To Reproduce

Create quickstart mvn project, following the Azure Java Quickstart guide: https://docs.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-java

~/apache-maven-3.6.3/bin/mvn archetype:generate -DgroupId=com.blobs.listex -DartifactId=blob-listex-v12 -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4 -DinteractiveMode=false

Add pom.xml dependencies:

    <dependency>
      <groupId>com.azure</groupId>
      <artifactId>azure-storage-blob</artifactId>
      <version>12.1.0</version>
    </dependency>
    <dependency>
       <groupId>org.slf4j</groupId>
       <artifactId>slf4j-api</artifactId>
       <version>1.7.5</version>
       </dependency>
   <dependency>
       <groupId>org.slf4j</groupId>
       <artifactId>slf4j-simple</artifactId>
       <version>1.6.4</version>
   </dependency>

Start Azurite:

azurite

Edit App.java, then build and execute code snippet.

~/apache-maven-3.6.3/bin/mvn compile
~/apache-maven-3.6.3/bin/mvn package
~/apache-maven-3.6.3/bin/mvn exec:java -Dexec.mainClass="com.blobs.listex.App" -Dexec.cleanupDaemonThreads=false

Code Snippet App.java

package com.blobs.listex;

import com.azure.storage.blob.*;
import com.azure.storage.blob.models.*;
import java.io.*;

public class App
{
    public static void main( String[] args ) throws NoSuchFieldException {

       BlobContainerClient container = new BlobServiceClientBuilder()
                .connectionString("UseDevelopmentStorage=true;")
                .buildClient()
                .getBlobContainerClient("mycontainer");
       if (!container.exists()) container.create();
       System.out.println("exists: " + container.exists());

        // List the blob(s) in the container.
        for (BlobItem blobItem : container.listBlobs()) {
                System.out.println("\t" + blobItem.getName());
        }
    }
}

Expected behavior List all blobs currently in the container, if any, without crashing.

Setup (please complete the following information):

  • OS: Red Hat Enterprise Linux Server 7.4 (Maipo), OpenJDK 1.8.0
  • IDE : N/A (compiling and executing via mvn)
  • Version of the Library used: Azure Storage Blob 12.1.0, Azurite current master branch

Additional context Looking at the Azurite output as well as the List Blobs REST API (https://docs.microsoft.com/en-us/rest/api/storageservices/list-blobs), it looks like the List Blobs request may be malformed, especially when compared to the Get Container requests before.

Issue #6140 indicated that Azurite functionality may have some holes, and ListBlobs() did not seem to be included in the Azurite Simple Scenario testing in #6156.

Information Checklist Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

  • Bug Description Added
  • Repro Steps Added
  • Setup information Added

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Reactions: 2
  • Comments: 15 (10 by maintainers)

Commits related to this issue

Most upvoted comments

Thank you for contributing this fix. We are looking at it now.

I guess that many developers trying to integrate Azurite would benefit from such a change/fix. It would be very kind to have this issue resolved.

I am not that pro, but maybe I can help a bit by providing a little PR which changes the blob container client behavior for listing blob items?