conscrypt: Severe memory leak on failed connection

Here is a test case that demonstrates out-of-control memory allocation when a socket fails to connect.

The following code repeatedly attempts to connect to https://localhost:60452, a port on which no process is listening.

Without Conscrypt, a JVM with -Xmx512M will use no more than approx. 700MB res mem, even after millions of connections have been attempted.

With Conscrypt (the CI build based on commit 52f3cf1 from January 28, 2021), the process exceeds 5GB of res mem within seconds.

Compilation and running instructions: javac -cp conscrypt-openjdk-2.5.0-SNAPSHOT-29jan2021-linux-x86_64.jar Main12.java java -Xmx512M -cp conscrypt-openjdk-2.5.0-SNAPSHOT-29jan2021-linux-x86_64.jar:. Main12 (you may need to increase your open files limit with e.g. ulimit -n 100000)

Main12.java source:

import org.conscrypt.OpenSSLProvider;
import java.net.HttpURLConnection;
import java.net.URL;
import java.security.Security;
import java.util.*;
import java.util.concurrent.atomic.AtomicInteger;

public class Main12 {

  private static void attemptConnectionToNonListeningPort() {
    try {
      URL urlObj = new URL("https://localhost:60452"); // nothing will be listening on this port
      HttpURLConnection conn = (HttpURLConnection) urlObj.openConnection();
      conn.setConnectTimeout(10);
      conn.setReadTimeout(10);
      conn.setAllowUserInteraction(false);
      conn.setRequestMethod("GET");
      conn.setUseCaches(false);
      conn.setDoInput(true);
      conn.setDoOutput(true);
      conn.connect();
    }
    catch (Exception e) {
    }
  }

  public static void main(String[] args) throws Exception {

    System.out.println("pid: " + ProcessHandle.current().pid());

    Scanner scanner = new Scanner(System.in);

    System.out.println("Use conscrypt? (y/n)");
    boolean useConscrypt = scanner.nextLine().toLowerCase().startsWith("y");

    if(useConscrypt) Security.insertProviderAt(new OpenSSLProvider(), 1);

    AtomicInteger count = new AtomicInteger(0);

    Thread progressThread = new Thread(()->{
      while(true) {
        try {
          Thread.sleep(1000);
          Process p = Runtime.getRuntime().exec("ps -o rss " + ProcessHandle.current().pid());
          Scanner s = new Scanner(p.getInputStream()).useDelimiter("\\A");
          s.nextLine();
          System.out.println("Connection attempts: " + count.get() + ", Res mem (MB): " + (Integer.parseInt(s.nextLine().trim()) / 1024));
          p.destroy();
        }
        catch (Exception e) {
          e.printStackTrace();
        }
      }
    });
    progressThread.setDaemon(true);
    progressThread.start();

    int threads = 200;
    for(int i=0; i<threads; i++) {
      new Thread(() -> {
        while (true) {
          attemptConnectionToNonListeningPort();
          count.incrementAndGet();
        }
      }).start();
    }

  }

}

Sample output:

pid: 4284
Use conscrypt? (y/n)
y
Connection attempts: 4460, Res mem (MB): 205
Connection attempts: 27684, Res mem (MB): 642
Connection attempts: 45630, Res mem (MB): 926
Connection attempts: 101401, Res mem (MB): 1795
Connection attempts: 132191, Res mem (MB): 2174
Connection attempts: 157731, Res mem (MB): 2456
Connection attempts: 196982, Res mem (MB): 2909
Connection attempts: 232997, Res mem (MB): 3563
Connection attempts: 257751, Res mem (MB): 3856
Connection attempts: 289832, Res mem (MB): 4227
Connection attempts: 346709, Res mem (MB): 4935
Connection attempts: 392016, Res mem (MB): 5525