efcore: How to do a missing migration check in CI

Looks like we started talking about two separate issues here.

React to this comment with 🚀 if you’re interested in a command that checks whether there are any migrations that haven’t been applied to a database (i.e. if you need to run dotnet ef database update)

React with 👀 if you’re interested in a command that checks there are any model changes that haven’t yet been added to a migration (i.e. if you need to run dotnet ef migrations add)

React with both if you want both.

if you are ok with using some internal EF Core this does what you’re asking for:

[Fact]
public void Should_not_have_missing_migrations()
{
    var builder = new DesignTimeServicesBuilder(typeof(MyDbContext).Assembly, Assembly.GetExecutingAssembly(), new OperationReporter(null), Array.Empty<string>());
    var provider = builder.Build(GetDbContext());
    var dependencies = provider.GetRequiredService<MigrationsScaffolderDependencies>();
    var modelSnapshot = dependencies.MigrationsAssembly.ModelSnapshot;
    var model = dependencies.SnapshotModelProcessor.Process(modelSnapshot?.Model);
    var relationalModel = model?.GetRelationalModel();

    var hasDifferences = dependencies.MigrationsModelDiffer.HasDifferences(relationalModel, dependencies.Model.GetRelationalModel());

    hasDifferences.Should().BeFalse();
}

@maxkoshevoi We discussed this in triage; there isn’t an easy way to do this now, but we agree that it would be useful.

Notes from triage:

• Issue #22105 covers warning for pending migrations and possible API surface to support this
  • With API, a unit test could be added to check there are no pending migrations
  • It would be useful to check for pending migrations in migrations list
• Using this issue to track command-line experience for asking specifically if there are pending migrations
  • A command that returns zero if there are no pending migrations and non-zero if there are would be useful in the C.I. where the return value is used to stop/continue the process.

Note from triage. This issue is still tracking having a command-line experience for asking specifically if there are pending migrations A command that returns zero if there are no pending migrations and non-zero if there are would be useful in the C.I. where the return value is used to stop/continue the process.

This command would be something like dotnet ef migrations check-pending.

Most people I’ve seen write a unit test:

[Fact]
public void No_pending_model_changes()
{
    using var context = new MyDbContext();

    var modelDiffer = context.GetService<IMigrationsModelDiffer>();

    var migrationsAssembly = context.GetService<IMigrationsAssembly>();
    var modelInitializer = context.GetService<IModelRuntimeInitializer>();
    var snapshotModel = migrationsAssembly.ModelSnapshot?.Model;
    if (snapshotModel is IMutableModel mutableModel)
        snapshotModel = mutableModel.FinalizeModel();
    if (snapshotModel is not null)
        snapshotModel = modelInitializer.Initialize(snapshotModel);

    var designTimeModel = context.GetService<IDesignTimeModel>();

    var pendingModelChanges = modelDiffer.HasDifferences(
        snapshotModel?.GetRelationalModel(),
        designTimeModel.Model.GetRelationalModel())

    Assert.Empty(pendingModelChanges);
}

#22105 is also about adding a simpler API (something like context.Database.HasPendingModelChanges()) to do this.

It gives you structed list of migrations with applied property. You can use it to find pending migrations

The issue here is not that migration hasn’t been applied, but that it hasn’t been created (there’re some model changes that are not reflected in DbContextModelSnapshot.cs), so I don’t think your solution is applicable here.

I’ve created a PR adding the check-pending command if people subscribed here have any notes! #31164

I’ve started some work on implementing this but would need sign off from the team before making a PR.

A command that returns zero if there are no pending migrations and non-zero if there are would be useful in the C.I. where the return value is used to stop/continue the process.

Based on @ajcvickers comment it looks like the team was more leaning towards this being it’s own command but from a simplicity standpoint it would be easier if it were an optional switch on the migrations add command. The experience for using such a switch would be like:

dotnet ef migrations add Test --fail-on-empty

The weirdness of this usage is that a lot of us want it to fail if a migration with actual operations is created. It’s possible to invert that check based on exit code in a CI pipeline but generally it’s more annoying. The other way would be to fail on there being actual operations but I personally haven’t thought of a name that I like. Things I have thought about: --require-empty, --verify-no-migrations, and the one thrown out in the issue body --verify-snapshot-up-to-date. I’m completely open to suggestions.

If we were to make it it’s own command the design I have thought about is:

dotnet ef migrations verify --no-changes

It feels a little extra to have it’s own command for this to me since the code for it would essentially need emulate getting to this part of the code like add. I’m open to any design but would love to start a discussion here about it because I’m serious about implementing this if a PR would be welcome.

if you are ok with using some internal EF Core this does what you’re asking for:

[Fact]
public void Should_not_have_missing_migrations()
{
    var builder = new DesignTimeServicesBuilder(typeof(MyDbContext).Assembly, Assembly.GetExecutingAssembly(), new OperationReporter(null), Array.Empty<string>());
    var provider = builder.Build(GetDbContext());
    var dependencies = provider.GetRequiredService<MigrationsScaffolderDependencies>();
    var modelSnapshot = dependencies.MigrationsAssembly.ModelSnapshot;
    var model = dependencies.SnapshotModelProcessor.Process(modelSnapshot?.Model);
    var relationalModel = model?.GetRelationalModel();

    var hasDifferences = dependencies.MigrationsModelDiffer.HasDifferences(relationalModel, dependencies.Model.GetRelationalModel());

    hasDifferences.Should().BeFalse();
}

For anyone using this, if the type ‘DesignTimeServicesBuilder’ can not be found, make sure “Microsoft.EntityFrameworkCore.Design” is not marked as a private asset in your .csproj file. (It is by default)

@aboryczko Thanks btw!

Checking if a migration is missing in your PR pipeline seems like a no-brainer to me, hoping ef core will get native support for this.

My GitHub Action to ensure that no new migrations are needed

  migration-test:
    name: Migration Test
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2

      - name: Setup .NET
        uses: actions/setup-dotnet@v1
        with:
          dotnet-version: '6.0.x'

      - name: Install EF Core CLI
        run: dotnet tool install --global dotnet-ef

      - name: Generate Migrations
        run: dotnet ef migrations add ${{ github.sha }} <your-params>

      - name: Check Created Migration File Against Regex
        run: |
            if [[ ! $(find <your-migration-folder> -name "*${{ github.sha }}.cs" | xargs grep -E -zo 'protected\s+override\s+void\s+Up\(.*\)\s*\{\s*\}\s*protected\s+override\s+void\s+Down\(.*\)\s*\{\s*\}') ]]; then
                echo "Unapplied Migration Found. Make sure you have run 'dotnet ef migration add ... and committed the generated migration file. See https://learn.microsoft.com/en-us/ef/core/managing-schemas/migrations/?tabs=dotnet-core-cli for more information."
                exit 1
            fi

FYI, you can test for unapplied migrations today using grep:

dotnet ef migrations list | grep "(Pending)"

Correct.

after starting to write a response, I think I’m understanding now; let me confirm: you’re saying there’s no need to connect to a specific database, because all the information needed to determine “is a new migration needed?” can be determined from code alone? that feels right, and I’ll test that now.

@BenMakesGames yes, you need to create the context. I use something more complex to get the context via IServiceProvider in a base test class, but what you have written should work fine. For a unit test this would be enough:

var builder = new DesignTimeServicesBuilder(typeof(MyDbContext).Assembly, Assembly.GetExecutingAssembly(), new OperationReporter(null), Array.Empty<string>());
var optionsBuilder = new DbContextOptionsBuilder<MyDbContext>().UseSqlServer();
var dbContext = new MyDbContext(optionsBuilder.Options);
var provider = builder.Build(dbContext);
...

of course if you use a different Database Provider you should change UseSqlServer() to something else.