okhttp: Http2Stream to have independent errors for request and response streams

I am using minio for Object storage. Minio is a golang project using go1.12 as the base go version. The server is HTTP2 enabled. I use using minio java client which uses okhttp3 . when i try to store the object in a bucket in multi threaded mode i get okhttp3.internal.http2.StreamResetException: stream was reset: NO_ERROR. Refer https://github.com/minio/minio/issues/7501 .

okhttp3.OkHttpClient uses http2.0 by default which causes such issue. If i forcefuly set the protocol to 1.1 ,it works fine. Refer PR https://github.com/minio/minio-java/pull/766.

I find there is some issue with Go1.12 or okhttp3 as i found this issue in the community. Refer https://github.com/square/okhttp/issues/3955 The only solution provided there was to revert to HTTP1_1 .

Setting the protocol to 1.1 is not the solution as i am not using the functionality of HTTP2.0 enabled server. Below is the code which replicates the problem

import io.minio.MinioClient;
class PutObjectRunnable implements Runnable {
   MinioClient client;
   String bucketName;
   String filename;

   public PutObjectRunnable(MinioClient client, String bucketName, String filename) {
       this.client = client;
       this.bucketName = bucketName;
       this.filename = filename;
   }

   public void run() {
       StringBuffer traceBuffer = new StringBuffer();

       try {

           client.putObject(bucketName, filename, filename);

       } catch (Exception e) {
           System.err.print(traceBuffer.toString());
           e.printStackTrace();
       }
   }
}


public class ThreadedPutObject {
    public static void main(String args[]) throws Exception {
	try{
        MinioClient client = new MinioClient("https://play.min.io:9000", "Q3AM3UQ867SPQQA43P2F", "zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG", true);
        client.traceOn(System.out);
        long startTime = System.currentTimeMillis();
		 boolean found = minioClient.bucketExists("my-bucketname");
      if (found) {
        System.out.println("my-bucketname already exists");
      } else {
        // Create bucket 'my-bucketname'.
        minioClient.makeBucket("my-bucketname");
        System.out.println("my-bucketname is created successfully");
      }        
      Thread[] threads = new Thread[7];
      String[] location = new String[]{"<<PATH_TO_OBJECT_1>>",
                                       "<<PATH_TO_OBJECT_2>>",
                                       "<<PATH_TO_OBJECT_3>>",
                                       "<<PATH_TO_OBJECT_4>>",
                                       "<<PATH_TO_OBJECT_5>>",
                                       "<<PATH_TO_OBJECT_6>>",	
                                       "<<PATH_TO_OBJECT_7>>" };

      for (int i = 0; i < 7; i++) {
            PutObjectRunnable pr = new PutObjectRunnable(client, "my-bucketname",location[i] );
          threads[i] = new Thread(pr);
      }
      for (int i = 0; i < 7; i++) {
        threads[i].start();
      }
            // Waiting for threads to complete.
      for (int i = 0; i < 7; i++) {
         threads[i].join();
          System.out.println(i);
        }
        // All threads are completed.
        } catch (Exception e) {
            throw e;
        }
        System.out.println("uploaded");
    }
}


About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 34 (9 by maintainers)

Commits related to this issue

Most upvoted comments

@swankjesse I would like to point out that I have lots of users who are waiting for a fix on this (https://github.com/PhilippC/keepass2android/issues/747), so if it is possible in any way to keep the milestone fixed to 4.5 I’d be really happy, as I have seen this being moved from 4.3 to 4.4 and now 4.5. Anyway: Thanks for your hard work! I really appreciate it and I totally understand that features are shifted to later milestones, so please don’t see this as criticism. I just want to make sure you are aware that this is a real issue for real people 😃

Can fix.

@swankjesse I am running into this issue as well (https://github.com/PhilippC/keepass2android/issues/747). I can reproduce it with the following code:

public class ConnectionInfo {
	public String URL;
	public String username;
	public String password;
}


private OkHttpClient getClient(ConnectionInfo ci) throws NoSuchAlgorithmException, KeyManagementException, KeyStoreException {

	Log.d("KP2AJ","building okhttp client");
	OkHttpClient.Builder builder = new OkHttpClient.Builder();
	final Map<String, CachingAuthenticator> authCache = new ConcurrentHashMap<>();

	com.burgstaller.okhttp.digest.Credentials credentials = new com.burgstaller.okhttp.digest.Credentials(ci.username, ci.password);
	final BasicAuthenticator basicAuthenticator = new BasicAuthenticator(credentials);
	final DigestAuthenticator digestAuthenticator = new DigestAuthenticator(credentials);

	DispatchingAuthenticator authenticator = new DispatchingAuthenticator.Builder()
			.with("digest", digestAuthenticator)
			.with("basic", basicAuthenticator)
			.build();

	builder = builder.authenticator(new CachingAuthenticatorDecorator(authenticator, authCache))
			.pingInterval(1, TimeUnit.SECONDS)
			.writeTimeout(100, TimeUnit.SECONDS)
			.connectTimeout(100, TimeUnit.SECONDS)
			.readTimeout(100, TimeUnit.SECONDS)
			.addInterceptor(new AuthenticationCacheInterceptor(authCache));

	OkHttpClient client =  builder.build();


	return client;
}

class PerformOkHttpTestTask extends AsyncTask<Object, Void, Void> {


	@Override
	protected Void doInBackground(Object... params) {

		android.util.Log.d("KP2AJ", "upload starting");

		ConnectionInfo ci = new ConnectionInfo();
		ci.password = "xxx";
		ci.URL = "https://...";
		ci.username = "userX";

		byte[] data = new byte[3000000];

		Request request = null;
		try {
			request = new Request.Builder()
					.url(new URL(ci.URL))
					.put(RequestBody.create(MediaType.parse("application/binary"), data))
					.build();


			Response response = null;

			response = getClient(ci).newCall(request).execute();
			if((response.code() < 200)
					|| (response.code() >= 300))
			{
				if (response.code() == 404)
					throw new FileNotFoundException();
				throw new Exception("Received unexpected response: " + response.toString());
			}
			android.util.Log.d("KP2AJ", "upload ok");


		} catch (Exception e) {
			e.printStackTrace();
		}

		return null;
	}
}

 @Override 
protected void onCreate(Bundle savedInstanceState) {
	super.onCreate(savedInstanceState);
	setContentView(R.layout.activity_main);
	
	findViewById(R.id.button1).setOnClickListener(new OnClickListener() {
        public void onClick(View v) {
        	new PerformOkHttpTestTask().execute();
        }
    });

Note that this issue also occurs without the pingInterval and read/write/connectTimeout calls on the builder. Note that it does not occur if I upload less data (e.g. 3000 bytes instead of 3MB).

The server I am testing this with is an Server version: Apache/2.4.25 (Raspbian) Server built: 2019-10-13T15:43:54 Nextcloud Version 16.0.3

My build.gradle has

 dependencies {
     compile 'com.squareup.okhttp3:okhttp:4.2.2'
     compile 'com.burgstaller:okhttp-digest:1.15'
 }

As many of my users (of Keepass2Android) are waiting for a fix to this: Is there anything I can change in my code or a workaround you can suggest?

Is the sample code enough or do you need a full Android Studio project?

Thanks a lot!

@michaelbannister yeah, I wanna cut a 4.x release very soon. I’ll make sure to include this fix.