ksp: KSP 1.9.21-1.0.15 leaking memory and causing OOMs
KSP 1.9.21-1.0.15 has started leaking memory and causing OOM issues when being used in a persistent worker environment. I suspect it’s related to https://github.com/google/ksp/commit/85f9cac5b3f53eab368f19e3296430add0fadbcb where instances are being placed into a global static list.
About this issue
- Original URL
- State: closed
- Created 7 months ago
- Reactions: 1
- Comments: 31 (10 by maintainers)
Commits related to this issue
- Clear both KSObjectCacheManager instances https://github.com/google/ksp/issues/1653 — committed to Bencodes/ksp by Bencodes 6 months ago
- Clear both KSObjectCacheManager instances https://github.com/google/ksp/issues/1653 — committed to google/ksp by Bencodes 6 months ago
- Clear both KSObjectCacheManager instances https://github.com/google/ksp/issues/1653 (cherry picked from commit 92d5905a863a8c796e55714d0f9da5e1f125f027) — committed to google/ksp by Bencodes 6 months ago
Fix is shipped with 1.9.22-1.0.17
Dug a bit deeper just to triple-check the memory leaking theory using
jmap -dump:format=b,file=out.hprof PIDlooking at 3 cases:Kotlin 1.9.20 + KSP 1.9.20-1.0.14 building all apps
Observed memory stays around 1.5gb - 2.5gb for most of the build and never seems to exceed 3gb. I managed to grab a dump that’s about 2.5gb in size.
Kotlin 1.9.21 + KSP 1.9.21-1.0.15 building all apps
Observed memory pretty quickly grows to more than 17gb. Running
jmapclears a lot of this memory out resulting in a dump between 3.5gb to 4.5gb, but the memory does climb back up to 17gb pretty quickly.The KSP worker also eventually locks up during the build once it hits the memory cap and has nothing else to consume.
Kotlin 1.9.21 + KSP 1.9.21-1.0.15 + reflection hack to clear wipe the memory
Observed memory consumption remains pretty low. Hands around 3.5 - 4.5gb. Running
jmapalso clears out some of this memory resulting in a dump of about 3gb. The observed memory as the build continues remains around 3.5 - 4.5gb.we are no longer seeing OOMs on our Buck or Bazel builds on
1.9.21-1.0.17-SNAPSHOTas @Bencodes previously noted, memory usage on newer KSP versions has increased which will require us to reconfigure some of our max memory & thread values
I confirmed that
1.9.21-1.0.17-SNAPSHOTfixes the OOM issue for Anvil 👍@jbarr21 Thanks for checking, can you file another issue to track since it looks like a different issue to me?
@neetopia I’m fine with that.