amazon-sqs-java-messaging-lib: java.lang.IllegalStateException: Connection pool shut down

I am using amazon-sqs-java-messaging-lib-1.0.8.jar. When I am sending a payload to SQS I am getting java.lang.IllegalStateException: Connection pool shut down exception.

Describe the bug When sending payload to SQS Queue getting below exception intermittently java.lang.IllegalStateException: Connection pool shut down org.apache.http.util.Asserts.check(Asserts.java:34) org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:191) …n.PoolingHttpClientConnectionManager.requestConnection(PoolingHttpClientConnectionManager.java:267) sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:498) …ttp.conn.ClientConnectionManagerFactory$Handler.invoke(ClientConnectionManagerFactory.java:76) com.amazonaws.http.conn.$Proxy191.requestConnection(Unknown Source) org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:176) org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) …g.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) …rg.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) …rg.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) …mazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72) …ttp.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1297) …ws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1113) …zonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:770) …http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744) …mazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726) …onaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:686) …p.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668) com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532) com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512) com.amazonaws.services.sqs.AmazonSQSClient.doInvoke(AmazonSQSClient.java:2215) com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2182) com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2171) …zonaws.services.sqs.AmazonSQSClient.executeGetQueueUrl(AmazonSQSClient.java:1163) com.amazonaws.services.sqs.AmazonSQSClient.getQueueUrl(AmazonSQSClient.java:1136) …amessaging.AmazonSQSMessagingClientWrapper.getQueueUrl(AmazonSQSMessagingClientWrapper.java:294) …amessaging.AmazonSQSMessagingClientWrapper.getQueueUrl(AmazonSQSMessagingClientWrapper.java:265) com.amazon.sqs.javamessaging.SQSSession.createQueue(SQSSession.java:636) …rt.destination.DynamicDestinationResolver.resolveQueue(DynamicDestinationResolver.java:85) …tion.DynamicDestinationResolver.resolveDestinationName(DynamicDestinationResolver.java:59) …tination.JmsDestinationAccessor.resolveDestinationName(JmsDestinationAccessor.java:115) org.springframework.jms.core.JmsTemplate.lambda$send$3(JmsTemplate.java:585) org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:504) org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:584)

Expected Behavior Ideally, I should get 200 and should be able to send the payload successfully to SQS.

Current Behavior Getting java.lang.IllegalStateException: Connection pool shut down Exception.

Your Environment amazon-sqs-java-messaging-lib-1.0.8.jar aws-java-sdk-sqs-1.11.832.jar Java 11 Linux

public JmsConfigurataion(JobsProperties sqsProperties) {

    EndpointConfiguration endpointConfiguration;
    endpointConfiguration = new EndpointConfiguration(
        endPoint + queueName, region);
    SQSConnectionFactory = connectionFactory = new SQSConnectionFactory(new ProviderConfiguration(),
        AmazonSQSClientBuilder.standard().withCredentials(
            new AWSStaticCredentialsProvider(new BasicAWSCredentials(
                accessKey(),
                secretKey())))
            .withEndpointConfiguration(endpointConfiguration).build());
}

@Override
@Bean(name = "jmsJobsContainerFactory")
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() {

    return super.jmsListenerContainerFactory();
}

@Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() {
    DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
    factory.setConnectionFactory(this.connectionFactory);
    factory.setDestinationResolver(new DynamicDestinationResolver());
    factory.setConcurrency("3-10");
    factory.setMessageConverter(messageConverter());
    factory.setErrorHandler(new ErrorHandler() {
        @Override
        public void handleError(Throwable t) {
            logger.error(t.getMessage(), t);
        }
    });
    return factory;
}

@Bean(name = "jmsJobsTemplate")
public JmsTemplate defaultJmsTemplate() {
    JmsTemplate jmsTemplate = new JmsTemplate(this.connectionFactory);
    jmsTemplate.setMessageConverter(messageConverter());
    return jmsTemplate;

}

Link - https://github.com/aws/aws-sdk-java/issues/2497

About this issue

Original URL
State: open
Created 3 years ago
Reactions: 5
Comments: 15

Most upvoted comments

I made this workaround which seems to work for my use case:

I run all my SQS calls through something like this: the new SQSClient method just creates a new client. this is like a wrapper over the AWS class with a retry built in. whatToThrow just converts everything to a RuntimeException.

public SendMessageResponse sendMessage(SendMessageRequest request) {
		return runAWSCall(() -> getSQSClient().sendMessage(request));
	}

private <T> T runAWSCall(Callable<T> toCall) {
		try {
			return toCall.call();
		} catch (IllegalStateException | Error illegalStateException) {
			// new client and retry ONCE only
			sqsClient = newSQSClient();
			try {
				return toCall.call();
			} catch (Exception e) {
				throw whatToThrow(e);
			}
		} catch (Exception e) {
			throw whatToThrow(e);
		}

	}

You can also specify more connections for the SQS connection pool:

		private SqsClient newSQSClient() {

		SqsClient sqs = SqsClient.builder()
				.region(Region.of(region))
				.credentialsProvider(() -> AwsBasicCredentials.create(awsId, awsKey))
				.httpClient(ApacheHttpClient.builder().maxConnections(150).build())
				.build();

		return sqs;
	}

I have a similar wrapper over the S3Client too.

more info here: https://blog.adebski.com/posts/apache-http-client-shutting-down/ I got the idea from the link above: "one of the possible solution could be:

Implement a wrapper around the client libraries that use Apache HTTP under the hood. That wrapper would handle the java.lang.Errors and replace old clients with new clients."

lucaalex87 on Sep 8, 2023

I am facing the same problem. Has anyone found a solution?

geheim01 on Jul 8, 2021