moshi: Moshi fails to process Kotlin 1.6 metadata

1.13.0 is out. Thanks for your patience.

Thanks. I’m going to close this as it’s already fixed on master, we’ll do a release by or around the 1.6 release

Koltin 1.6 is out, it’s possible to make a new release with the issue fixed? Thanks

Hey folks, we owe a bit of an explanation. Apologies for the radio silence.

• Yes, we are late on a release here. Square folks are combing through PRs to catch up (it’s been mostly me since the last release, I don’t work at Square nor do I control releases). They are almost done, but I can’t offer any concrete ETA other than “soon”.
• Yes this kotlinx-metadata stability issue is a problem. We, perhaps ironically in this case, shade any unstable APIs we use in order to guarantee future compatibility. I filed the issue above after talking with JB folks and hopeful it’ll go stable soon. Unfortunately this is sort of similar to ASM for Java where it can’t make guarantees about being able to read future versions, only backward compatible releases. When it’s stable at least, folks will be able to unblock locally by just forcing a newer version of it since we won’t have to shade it anymore.
• The next Moshi release will use kotlinx-metadata 0.4.0, which will support both Kotlin 1.6 as well as Kotlin 1.7 later down the line (it guarantees full backward compatibility + one future release), so that will buy more runway.
• We close issues when they are fixed on main, not when they are released.

As far as being blocked - as others have mentioned already, you aren’t truly blocked unless you’re maintaining other published libraries that depend on this. Here are some patterns we use to avoid being bottlenecked on external dependencies.

1. Download the latest snapshot jars (link in the README) or build from source (see RELEASING.md or ci.yml to see how we build artifacts)
2. You can check these into your repo or upload them to an internal artifactory/nexus/etc instance (change the version in the pom.xml file if you do upload). We do this with Moshi in our Slack repo today.

If you check the jars into your repo, you can put them in a standalone project like so

// In a standalone project like `tmp-moshi`
dependencies {
    implementation(files('libs/moshi.jar'))
}

Then add this to your root build.gradle file to substitute remote Moshi dependencies with that project

subprojects {
  configurations.configureEach {
    resolutionStrategy {
        dependencySubstitution {
            substitute(module("com.squareup.moshi:moshi")).apply {
              using(project("tmp-moshi")) // Your local project holding the moshi jars
              because("Moshi 1.13.0 is not released yet, this is a local cut from HEAD")
            }
            // Repeat for any other moshi artifacts you use (code gen, kotlin reflect, adapters, etc)
        }
    }
  }
}

If you upload jars to an artifactory, simply just use a newer version like 1.13.0-yourcompany and force that version instead. This whole process takes me around 30 seconds to do when we take early cuts of external libraries, so hope it’s as easy elsewhere 🤞.

I hope this is helpful and thanks for bearing with us.

@malloth Go ahead. Or maybe you can just do this “quick hotfix” by yourself on fork and use your own release? You can release it to public and be the hero we all deserve! Either way - stop making GH toxic place

Any way around this bug? I can not wait until new Moshi is released (it takes forever to prepare a simple bugfix for 1.12.1)

I’m thinking about switching to Kotlin Serialization because Moshi team is not being able to release a quick hotfix. So lame!

Jetpack Compose 1.1.0-beta04 now needed Kotlin 1.6.0. Moshi needs to publish a fix soon.

Also reproduced in my multi-module project, based on clean architecture. Presentation layer depends on Framework layer that has all retrofit/okhttp/moshi dependencies (including Codegen). I’m using hilt di modules to provide retrofit clients in Framework layer.

After updating kotlin to 1.6.0 I’m getting this error at build time:

Caused by: java.lang.IllegalStateException: Could not parse metadata! This should only happen if you're using Kotlin <1.1. at com.squareup.moshi.kotlinpoet.metadata.KotlinPoetMetadata.readKotlinClassMetadata(KotlinPoetMetadata.kt:89) at com.squareup.moshi.kotlinpoet.metadata.KotlinPoetMetadata.toImmutableKmClass(KotlinPoetMetadata.kt:119) at com.squareup.moshi.kotlin.codegen.MoshiCachedClassInspector.toImmutableKmClass(MoshiCachedClassInspector.kt:37) at com.squareup.moshi.kotlin.codegen.MetadataKt.targetType(metadata.kt:139) at com.squareup.moshi.kotlin.codegen.JsonClassCodegenProcessor.adapterGenerator(JsonClassCodegenProcessor.kt:148)

The only way to get it working is to revert “org.jetbrains.kotlin:kotlin-gradle-plugin” dep from 1.6.0 to 1.5.31.

So i hope Moshi team could fix this as fast as possible, because this issue won’t let me use the latest version of kotlin.

Other projects are now starting to mandate the use of Kotlin 1.6.0 to be able to remain current (get bug fixes, new features, etc.), e.g.: https://kotest.io/docs/changelog.html#kotlin-16-is-now-the-minimum-supported-version

What is the timeline for a release containing the fix please?

I get Records requires ASM8 when I use 1.13.0 and having Java jdk-11.0.12.jdk

I think that some frustration is understandable on both sides of this issue. People are blocked and don’t know how long it will take, there is no hint about that. On the other side moshi team is bombarded with questions like “When are you going to release this?”. This is open source so noone of us pays moshi team for maintaing this repo, everyone can release this on their own from fork. However this require some effort and time and before doing this it would be nice to know if it makes sense. If there will be an official release in few days, maybe 2 weeks then personally I will wait. Can we have such information? Everyone can then make his own decision what to do next and not spam this thread, release fork, wait or migrate to something else.

Correct, please take up issues with other tools on their respective issue trackers. One thing this release has revealed is some tools incorrectly read class files out of META-INF, which is a bug on their side and not something Moshi can fix.

for me it worked when i upgraded: Moshi -> 1.14.0 Kotlin -> 1.7.0

@lsuski Totally agree with that. I have problem with the attitude and unnecessary insults, but I do understand the problem.

No amount of frustration justifies being toxic.

Because we’re not ready for another release yet (we have more in the pipeline for 1.13.0) and you can easily unblock yourselves by building from source or use snapshots if you need it sooner.

…or just use a snapshot version.

@nokite Well, Zac Sweers said it in plain text above, and everybody could star the issue on YouTrack. It currently has “Priority: Major” and “State: In Progress”, which gives me some hope that things will improve eventually.

The ideal scenario would be that Jetbrains make this an actual stable API (https://youtrack.jetbrains.com/issue/KT-48011) or didn’t impose this reading limit since it’s technically just protobufs under the hood and should be compatible to read.

Is there a specific reason not to release this earlier, i.e. now? As I said, this blocks the Kotlin 1.5.31 update for people like me who want to opt-in for sealed whens.

Happens to me in a Gradle build as well, as soon as I set the languageVersion to “1.6” (to make use of the sealed when preview in Kotlin 1.5.30):

Caused by: java.lang.IllegalStateException: Could not parse metadata! This should only happen if you're using Kotlin <1.1.
        at com.squareup.moshi.kotlinpoet.metadata.KotlinPoetMetadata.readKotlinClassMetadata(KotlinPoetMetadata.kt:89)
        at com.squareup.moshi.kotlinpoet.metadata.KotlinPoetMetadata.toImmutableKmClass(KotlinPoetMetadata.kt:119)
        at com.squareup.moshi.kotlin.codegen.MoshiCachedClassInspector.toImmutableKmClass(MoshiCachedClassInspector.kt:37)
        at com.squareup.moshi.kotlin.codegen.MetadataKt.targetType(metadata.kt:139)
        at com.squareup.moshi.kotlin.codegen.JsonClassCodegenProcessor.adapterGenerator(JsonClassCodegenProcessor.kt:148)
        at com.squareup.moshi.kotlin.codegen.JsonClassCodegenProcessor.process(JsonClassCodegenProcessor.kt:114)
        at org.jetbrains.kotlin.kapt3.base.incremental.IncrementalProcessor.process(incrementalProcessors.kt:90)
        at org.jetbrains.kotlin.kapt3.base.ProcessorWrapper.process(annotationProcessing.kt:188)
        at jdk.compiler/com.sun.tools.javac.processing.JavacProcessingEnvironment.callProcessor(JavacProcessingEnvironment.java:980)
        ... 41 more

And yes, kotlinx-metadata is already silently updated to 0.3.0 for me as well and there doesn’t seem to be a newer version of this library available.

+--- com.squareup.moshi:moshi-kotlin-codegen -> 1.12.0
|    +--- com.squareup.moshi:moshi:1.12.0
|    |    +--- com.squareup.okio:okio:2.10.0
|    |    |    +--- org.jetbrains.kotlin:kotlin-stdlib:1.4.20 -> 1.5.30 (*)
|    |    |    \--- org.jetbrains.kotlin:kotlin-stdlib-common:1.4.20 -> 1.5.30
|    |    \--- org.jetbrains.kotlin:kotlin-stdlib:1.4.31 -> 1.5.30 (*)
|    +--- org.jetbrains.kotlin:kotlin-reflect:1.4.31 -> 1.5.30
|    |    \--- org.jetbrains.kotlin:kotlin-stdlib:1.5.30 (*)
|    +--- com.squareup:kotlinpoet:1.8.0
|    |    +--- org.jetbrains.kotlin:kotlin-reflect:1.4.31 -> 1.5.30 (*)
|    |    \--- org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.4.31 -> 1.5.30 (*)
|    +--- com.squareup:kotlinpoet-classinspector-elements:1.8.0
|    |    +--- com.squareup:kotlinpoet-metadata-specs:1.8.0
|    |    |    +--- com.squareup:kotlinpoet:1.8.0 (*)
|    |    |    +--- com.squareup:kotlinpoet-metadata:1.8.0
|    |    |    |    +--- org.jetbrains.kotlinx:kotlinx-metadata-jvm:0.2.0 -> 0.3.0 (*)
|    |    |    |    \--- org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.4.31 -> 1.5.30 (*)
|    |    |    \--- org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.4.31 -> 1.5.30 (*)
|    |    +--- com.google.guava:guava:29.0-jre (*)
|    |    \--- org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.4.31 -> 1.5.30 (*)

This seems to be the upstream tracking bug: https://youtrack.jetbrains.com/issue/KT-47945