aws-sdk-kotlin: Binary incompatibility with ktor-io 2.x results in NoSuchMethodError exceptions at runtime
Describe the bug
When using response.body?.toByteArray(), I get the following errors :
23:47:40.748 [DefaultDispatcher-worker-5] DEBUG httpTraceMiddleware - service: S3; operation: GetObject; - HttpResponse: 200: OK
23:47:40.767 [DefaultDispatcher-worker-5] ERROR ktor.application - Websocket handler failed
java.lang.NoSuchMethodError: 'void io.ktor.utils.io.bits.Memory.copyTo-iAfECsU(java.nio.ByteBuffer, java.nio.ByteBuffer, long, long, long)'
at aws.smithy.kotlin.runtime.io.SdkByteBufferKt.readFully-aPcrCvc(SdkByteBuffer.kt:377)
at aws.sdk.kotlin.runtime.http.engine.crt.SegmentKt.copyTo(Segment.kt:21)
at aws.sdk.kotlin.runtime.http.engine.crt.AbstractBufferedReadChannel.readAsMuchAsPossible(AbstractBufferedReadChannel.kt:113)
at aws.sdk.kotlin.runtime.http.engine.crt.AbstractBufferedReadChannel.readRemaining$suspendImpl(AbstractBufferedReadChannel.kt:97)
at aws.sdk.kotlin.runtime.http.engine.crt.AbstractBufferedReadChannel.readRemaining(AbstractBufferedReadChannel.kt)
at aws.smithy.kotlin.runtime.io.SdkByteReadChannel$DefaultImpls.readRemaining$default(SdkByteReadChannelJVM.kt:35)
at aws.smithy.kotlin.runtime.content.ByteStreamKt.consumeStream(ByteStream.kt:67)
at aws.smithy.kotlin.runtime.content.ByteStreamKt.toByteArray(ByteStream.kt:83)
Expected behavior
We are able to read the response body.
Current behavior
Trying to read the response body throws an exception java.lang.NoSuchMethodError
Steps to Reproduce
Here is the code to reproduce (taken from the doc pretty much as-is)
import aws.sdk.kotlin.services.s3.S3Client
import aws.sdk.kotlin.services.s3.model.GetObjectRequest
import aws.smithy.kotlin.runtime.content.toByteArray
import kotlinx.coroutines.async
private fun downloadImageFromS3(imageLocation: String) =
async {
val request = createS3GetObjectRequest(imageLocation)
var imageBytes: ByteArray? = null
S3Client(s3Configuration).use { s3 ->
s3.getObject(request) { response ->
imageBytes = response.body?.toByteArray()
}
}
imageBytes
}
private fun createS3GetObjectRequest(imageLocation: String) =
GetObjectRequest {
key = imageLocation
bucket = s3BucketName
}
Possible Solution
No response
Context
I am running Ktor 2.0.0-beta-1.
The error still occurs after deleting all the libraries downloaded, deleting gradle cache then re-downloading them via gradle sync.
Here is a screenshot of the dependencies that the S3 lib downloaded

AWS Kotlin SDK version used
0.9.5-beta
Platform (JVM/JS/Native)
JVM 11
Operating System and version
Ubuntu 20.04.3 LTS
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 2
- Comments: 24 (3 by maintainers)
Commits related to this issue
- fix(rt): fix readFully when using ktor 2.0 this change fixes reading bytes content under ktor 2.0 see https://github.com/awslabs/aws-sdk-kotlin/issues/480 — committed to AlexandreBrown/smithy-kotlin by AlexandreBrown 3 years ago
- fix(rt): bump smithy-kotlin to 0.10.0 Upgrading smithy-kotlin to 0.10.0 which includes an upgrade to ktor-2.x. This will fix awslabs/aws-sdk-kotlin#480 but as noted in the ticket may reverse the iss... — committed to awslabs/aws-sdk-kotlin by aajtodd 2 years ago
- * fix(rt): bump smithy-kotlin to 0.10.0 (#613) Upgrading smithy-kotlin to 0.10.0 which includes an upgrade to ktor-2.x. This will fix awslabs/aws-sdk-kotlin#480 but as noted in the ticket may rev... — committed to awslabs/aws-sdk-kotlin by aajtodd 2 years ago
This will be available in the next release (
0.16.0-beta) today or tomorrow hopefully.Please fix this soon, my application relies on it. I’m too far in on my project to revert back to last ktor version
Not at present no. The fix requires us to either upgrade (which will reverse the issue and cause binary incompatibility with 1.x users of ktor) OR shade
ktor-io:1.xinto the SDK.Our current plan, as Ian noted, is to look into upgrading to 2.0 in the near term and see what feedback we get to determine if we need to do anything further.
Yes Ktor 2.0 has been officially released and we’re now evaluating moving the SDK’s underlying version from 1.x to 2.0. Until that work is complete, you will likely still see errors when mixing the AWS SDK for Kotlin and Ktor 2.x.
The
0.16.0release which upgrades to ktor 2.0.1 has made it’s way to maven central. This build also includes a changeover to KMP builds/artifacts. If you are using Gradle the Kotlin plugin (whether you use multiplatform or not) should take care of this for you, if not you may need to depend on the-jvmversion (e.g. s3-jvm)of specific services. We of course still only support the JVM target at this time though.NOTE: We accidentally published this without the
-betaqualifier due to a miss in some new publishing infrastructure. Future dev preview releases will once again use this qualifier.Is there an ETA on this effort? I just upgraded to Ktor 2 without realizing the consequences. The aws-sdk-kotlin isn’t 100% vital for my application. My options are wait for a Ktor 2.x compatible version, or switch back to the AWS Java SDK. If this is fixed soon, my preference is leave things broken and not get notifications for a few days.
Agreed, we’ll leave this issue open for tracking, performing the upgrade/shading, and documenting the final outcomes.
@aajtodd I agree with you, it’s important to consider the impact of the fix on 1.x users.
It’s a good thing you notified JB about it as this seems like an issue that can affect a lot of libraries like you said.
We will keep an eye for any update from your part on this as this blocks us from downloading S3 objects for now.
Thanks