sqldelight: verifySqlDelightMigration hangs

SQLDelight Version

2.0.0

Application Operating System

Android

Describe the Bug

When executing the verifySqlDelightMigration tasks it seems to run forever. Yesterday I had it actually running in the background it and was going for almost 5 hours. Just today I noticed this too:

Calculating task graph as no configuration cache is available for tasks: verifySqlDelightMigration
<-------------> 0% EXECUTING [5m 52s]
> :database:verifyCommonMainQueryWrapperMigration

My initial guess is that it’s somehow a Gradle regression from a previous update that I did. I can also reliably reproduce this; what would be the best way to find out what is exactly is causing this?

I’m using Gradle 8.4

Stacktrace

No response

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 13
  • Comments: 15 (4 by maintainers)

Most upvoted comments

There might be two separate bugs here, in my case it happens with Java 17.

Maybe it would be possible to parameterize this setting so that you can decide at what level you want to run the verification? I’ll try to prepare PR.

It’s been a while, but the need to upgrade SQLDelight to 2.x has come up again in my project. The migration verification hanging issue is blocking the upgrade, so I’ve created a PR with changes that optionally allow reducing the migration verification strictness to the level that was used by default in SQLDelight 1.x — which, in my case, resolves the problem.

I’ve finally had some time to track down the root cause and I believe this task is able to finish after all it’s just that every combined PRIMARY KEY using two columns exponentially slows this down. Here is a public reproducer: vanniktech/playground#201 If I execute the verify task ./gradlew verifySqlDelightMigration --info --rerun-tasks as is on the branch it takes on my computer ~33 seconds. This should never take this long. We’re talking about 6 tables and a total of only 9 columns.

  1. If I delete FitnessProfileEquipment.sq
  2. Delete the database via find . -name "*.db" -exec rm {} \;
  3. Create a new database file to verify against ./gradlew generateCommonMainQueryWrapperSchema
  4. Rerun the task ./gradlew verifySqlDelightMigration --info --rerun-tasks

It only takes 1 second.

In my case, it turned out that foreign keys slowed down the verification dramatically (unable to complete within a few hours). I removed the foreign keys from my schema as a test and the verification passed within a few seconds…

I started digging deeper and after analyzing what VerifyMigrationTask was doing, I found that underneath it was using SchemaCrawler, which version of the dependency was bumped after a few years.

I think it is something in the changes of this library that changes in the behaviour of VerifyMigrationTask.

What I have managed to figure out so far is that the level of detail in database comparison is determined by the SchemaInfoLevel set to maximum in CatalogDatabase. I tested migration verification with foreign keys for which verification could not be completed earlier and with the minimum level, and the verification passed in a few seconds!

Maybe it would be possible to parameterize this setting so that you can decide at what level you want to run the verification? I’ll try to prepare PR.

I’m facing the same issue with a table that has an FK to another table and that table also as a FK. In my case the error I got from the verifyDatabaseMigration was:

* What went wrong:
Execution failed for task ':farore:repositories:verifyCommonMainTestSecurev2Migration'.
> A failure occurred while executing app.cash.sqldelight.gradle.VerifyMigrationTask$VerifyMigrationAction
   > Java heap space: failed reallocation of scalar replaced objects

Hey is there an update on this?

It suddenly started to hang for me too. I tried to revert my project to a very old commit that I’m sure was working fine, but it was hanging as well.

Then I remembered that I updated JDK on my computer from v20 to v21 recently. I then manually installed Java 20 (brew install --cask temurin20 + manually set JAVA_HOME) and suddenly verifySqlDelightMigration started to work fine again.

So I’m pretty sure there’s something in Java 21 that makes it hang.

I’ve finally had some time to track down the root cause and I believe this task is able to finish after all it’s just that every combined PRIMARY KEY using two columns exponentially slows this down.

Here is a public reproducer: vanniktech/playground#201

If I execute the verify task ./gradlew verifySqlDelightMigration --info --rerun-tasks as is on the branch it takes on my computer ~33 seconds. This should never take this long. We’re talking about 6 tables and a total of only 9 columns.

  1. If I delete FitnessProfileEquipment.sq
  2. Delete the database via find . -name "*.db" -exec rm {} \;
  3. Create a new database file to verify against ./gradlew generateCommonMainQueryWrapperSchema
  4. Rerun the task ./gradlew verifySqlDelightMigration --info --rerun-tasks

It only takes 1 second.

In my case, it turned out that foreign keys slowed down the verification dramatically (unable to complete within a few hours). I removed the foreign keys from my schema as a test and the verification passed within a few seconds…

I started digging deeper and after analyzing what VerifyMigrationTask was doing, I found that underneath it was using SchemaCrawler, which version of the dependency was bumped after a few years.

I think it is something in the changes of this library that changes in the behaviour of VerifyMigrationTask.

What I have managed to figure out so far is that the level of detail in database comparison is determined by the SchemaInfoLevel set to maximum in CatalogDatabase. I tested migration verification with foreign keys for which verification could not be completed earlier and with the minimum level, and the verification passed in a few seconds!

Maybe it would be possible to parameterize this setting so that you can decide at what level you want to run the verification? I’ll try to prepare PR.