aws-glue-schema-registry: Http client conflicts

Hi,

I’m getting the error below when integrating schema registry in our project.

Caused by: software.amazon.awssdk.core.exception.SdkClientException: Multiple HTTP implementations 
were found on the classpath. To avoid non-deterministic loading implementations, please explicitly provide 
an HTTP client via the client builders, set the software.amazon.awssdk.http.service.impl system property 
with the FQCN of the HTTP service to use as the default, or remove all but one HTTP implementation 
from the classpath

I cannot resolve this with the directions above (with our current httpClient), and believe the issue is that UrlConnectionHttpClient is used explicitly in the builder below.

GlueClientBuilder glueClientBuilder = GlueClient
                .builder()
                .credentialsProvider(credentialsProvider)
                .overrideConfiguration(overrideConfiguration)
                .httpClient(UrlConnectionHttpClient.builder().build())
                .region(Region.of(glueSchemaRegistryConfiguration.getRegion()));

source

About this issue

Original URL
State: open
Created 3 years ago
Reactions: 11
Comments: 17 (6 by maintainers)

Commits related to this issue

Upgrade to the latest deps * Exclude `apache-client` from the `amazon-kinesis-client` to avoid conflict with an `url-connection-client` which is hard-coded to the `aws-glue-schema-registry`. See more... — committed to spring-cloud/spring-cloud-stream-binder-aws-kinesis by artembilan a year ago

Most upvoted comments

I reopened the issue; I feel like the fact that it is explicitly specified here in this dependency should not affect the rest of the codebase having glue-schema-registry as a dependency. Instead of having to specify it explicitly everywhere else, glue-schema-registry should also use the service loader approach and support several http clients.

samu-developments on Jan 20, 2022

@blacktooth Is there any chance to prioritize this given the issue has existed since 2021?

jhchee on Jul 20, 2023

This actually breaks the WebIdentityTokenFileCredentialsProvider. While other SDKv2 client builders provide override to specify the HTTP clients, there is no way to do so with WebIdentityTokenFileCredentialsProvider. This breaks using service accounts in IAM roles in Kubernetes/Containerized environments. Moreover, the preferred usage with the rest of the AWS SDK is to use the default provider chain; so we should not be forced to override that. The Glue clients should use the Apache client, as it the default with the rest of the AWS SDK.

singhbaljit on Oct 12, 2023

It seems like it was added with this PR: https://github.com/awslabs/aws-glue-schema-registry/pull/49

@mohitpali is the UrlConnectionHttpClient needed for the JSON support? Have this dependency in the library conflicts with being able to use the ProfileCredentialProvider in the default configuration when this dependency is in the classpath.

callado4 on Jan 6, 2022