aspnetcore: On upgrade to .NET 7, Data Protection throws "Payload was invalid" error when unprotecting values from .NET 6

Is there an existing issue for this?

  • I have searched the existing issues

Describe the bug

We use Data Protection to encrypt sensitive values in long-term storage. While testing .NET 7.0, we found that our app was unable to decrypt payloads encrypted by .NET 6.0, even when the same keys were available and the same purpose was used. This means that upgrading to .NET 7 causes users to be logged out of our site (since the auth cookie cannot be validated) and our app is no longer able to decrypt data which it had previously encrypted and stored in durable storage (database, etc.).

I was able to isolate this into a simple repro console app: https://github.com/anurse/dataprotection-repro

The app runs on both net6.0 and net7.0 and can protect/unprotect values. Running .\ReproApp protect "Some string" on net6.0 produces a Base64 protected value. Passing that same payload into .\ReproApp unprotect on net6.0 successfully unprotects the string. However, passing it in to the same .\ReproApp unprotect command in the same app when running on net7.0 fails with the following error:

Unhandled exception. System.Security.Cryptography.CryptographicException: The payload was invalid. For more information go to http://aka.ms/dataprotectionwarning
   at Microsoft.AspNetCore.DataProtection.Managed.ManagedAuthenticatedEncryptor.Decrypt(ArraySegment`1 protectedPayload, ArraySegment`1 additionalAuthenticatedData)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.UnprotectCore(Byte[] protectedData, Boolean allowOperationsOnRevokedKeys, UnprotectStatus& status)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.Unprotect(Byte[] protectedData)
   at Microsoft.AspNetCore.DataProtection.DataProtectionCommonExtensions.Unprotect(IDataProtector protector, String protectedData)
   at Program.Main(String[] args) in /Users/anurse/code/anurse/dataprotection-repro/ReproApp/Program.cs:line 35
   at Program.<Main>(String[] args)

Further debugging traced this down to this line:

https://github.com/dotnet/aspnetcore/blob/8f0d440d4251ae479966b65c7b331841adb70aa6/src/DataProtection/DataProtection/src/Managed/ManagedAuthenticatedEncryptor.cs#L235-L240

The MAC check appears to be failing. I wasn’t using a Debug build, so I wasn’t able to check if the MAC was actually different, or if this is a bug in the equality comparison.

Expected Behavior

When running the run-repro script in the provided repository, the output should be something like this:

... build output ...

Protecting: 'This is a test message'
*** Protecting with .NET 6.0 ... ***
Protected string: <trimmed>
Unprotecting with .NET 6.0 ...
This is a test message
Unprotecting with .NET 7.0 ...
This is a test message
*** Protecting with .NET 7.0 ... ***
Protected string: <trimmed>
Unprotecting with .NET 6.0 ...
This is a test message
Unprotecting with .NET 7.0 ...
This is a test message

The actual behavior is that an exception is thrown when unprotecting with a different major version than the data was originally protected with:

... build output ...

Protecting: 'This is a test message'
*** Protecting with .NET 6.0 ... ***
Protected string: <trimmed>
Unprotecting with .NET 6.0 ...
This is a test message
Unprotecting with .NET 7.0 ...
Unhandled exception. System.Security.Cryptography.CryptographicException: The payload was invalid. For more information go to http://aka.ms/dataprotectionwarning
   at Microsoft.AspNetCore.DataProtection.Managed.ManagedAuthenticatedEncryptor.Decrypt(ArraySegment`1 protectedPayload, ArraySegment`1 additionalAuthenticatedData)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.UnprotectCore(Byte[] protectedData, Boolean allowOperationsOnRevokedKeys, UnprotectStatus& status)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.Unprotect(Byte[] protectedData)
   at Microsoft.AspNetCore.DataProtection.DataProtectionCommonExtensions.Unprotect(IDataProtector protector, String protectedData)
   at Program.Main(String[] args) in /Users/anurse/code/anurse/dataprotection-repro/ReproApp/Program.cs:line 35
   at Program.<Main>(String[] args)
*** Protecting with .NET 7.0 ... ***
Protected string: <trimmed>
Unprotecting with .NET 6.0 ...
Unhandled exception. System.Security.Cryptography.CryptographicException: The payload was invalid. For more information go to http://aka.ms/dataprotectionwarning
   at Microsoft.AspNetCore.DataProtection.Managed.ManagedAuthenticatedEncryptor.Decrypt(ArraySegment`1 protectedPayload, ArraySegment`1 additionalAuthenticatedData)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.UnprotectCore(Byte[] protectedData, Boolean allowOperationsOnRevokedKeys, UnprotectStatus& status)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.Unprotect(Byte[] protectedData)
   at Microsoft.AspNetCore.DataProtection.DataProtectionCommonExtensions.Unprotect(IDataProtector protector, String protectedData)
   at Program.Main(String[] args) in /Users/anurse/code/anurse/dataprotection-repro/ReproApp/Program.cs:line 35
   at Program.<Main>(String[] args)
Unprotecting with .NET 7.0 ...
This is a test message

Steps To Reproduce

  1. Clone the repro repo: http://github.com/anurse/dataprotection-repro
  2. Ensure you have .NET 6 and .NET 7 installed
  3. Mac/Linux: Run ./run-repro script
  4. Windows:
    1. Run dotnet run --project .\ReproApp --framework net6.0 -- protect "a test string"
    2. Copy the output string
    3. Run dotnet run --project .\ReproApp --framework net6.0 -- unprotect "<paste>"
    4. Run dotnet run --project .\ReproApp --framework net7.0 -- unprotect "<paste>"

Exceptions (if any)

Unhandled exception. System.Security.Cryptography.CryptographicException: The payload was invalid. For more information go to http://aka.ms/dataprotectionwarning
   at Microsoft.AspNetCore.DataProtection.Managed.ManagedAuthenticatedEncryptor.Decrypt(ArraySegment`1 protectedPayload, ArraySegment`1 additionalAuthenticatedData)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.UnprotectCore(Byte[] protectedData, Boolean allowOperationsOnRevokedKeys, UnprotectStatus& status)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.Unprotect(Byte[] protectedData)
   at Microsoft.AspNetCore.DataProtection.DataProtectionCommonExtensions.Unprotect(IDataProtector protector, String protectedData)
   at Program.Main(String[] args) in /Users/anurse/code/anurse/dataprotection-repro/ReproApp/Program.cs:line 35
   at Program.<Main>(String[] args)

.NET Version

7.0.100-preview.7.22377.5

Anything else?

Since multiple .NET runtimes are involved, here’s my dotnet --info output:

.NET SDK:
 Version:   7.0.100-preview.7.22377.5
 Commit:    ba310d9309

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  12.0
 OS Platform: Darwin
 RID:         osx.12-arm64
 Base Path:   /usr/local/share/dotnet/sdk/7.0.100-preview.7.22377.5/

Host:
  Version:      7.0.0-preview.7.22375.6
  Architecture: arm64
  Commit:       eecb028078

.NET SDKs installed:
  6.0.301 [/usr/local/share/dotnet/sdk]
  6.0.400 [/usr/local/share/dotnet/sdk]
  7.0.100-preview.7.22377.5 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.6 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.8 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.0-preview.7.22376.6 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.6 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.8 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.0-preview.7.22375.6 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Reactions: 3
  • Comments: 27 (23 by maintainers)

Most upvoted comments

Just to comment on the code from @frabe1579 above for future readers, Microsoft documents a slightly different approach:

var trimmedContentRootPath = Environment.ContentRootPath.TrimEnd(Path.DirectorySeparatorChar);

services.AddDataProtection()
    .SetApplicationName(trimmedContentRootPath)
    ...

It is documented in the “Warning” section here:

https://learn.microsoft.com/en-us/aspnet/core/security/data-protection/configuration/overview?tabs=aspnetcore2x&view=aspnetcore-7.0#setapplicationname

+4
altenstedt on Dec 20, 2022

Ok, yes, this looks like https://github.com/dotnet/aspnetcore/issues/40964 + https://github.com/dotnet/aspnetcore/pull/41849. We changed ContentRoot between 5 and 6 and then again between 6 and 7 to try to “fix” the 5 -> 6 change. We had to choose between breaking people using HostBuilder and people using WebApplicationBuilder, and we chose the latter. It’s an unfortunate situation - if we’re right that this is the issue, the workaround would be to manually set your Discriminator to be the same as it was in 6 when you upgrade your app to 7. We should communicate this very loudly for 7 - @halter73 @blowdart where is the right place to do that?

(Of course, I’ll play with the repro app to confirm this suspicion, but all signs are currently pointing towards it)

+2
wtgodbe on Sep 8, 2022

@adityamandaleeka this should get triaged asap. If upgrading breaks data protection with an existing keyring that’s bad.

+2
blowdart on Sep 1, 2022

Same killing problem here, my app now has a / at the end of ApplicationDiscriminator. Solved, for now, with this:

      services.PostConfigure<DataProtectionOptions>(p => {
        if (p.ApplicationDiscriminator != null && (p.ApplicationDiscriminator.EndsWith("/") || p.ApplicationDiscriminator.EndsWith("\\")))
          p.ApplicationDiscriminator = p.ApplicationDiscriminator[..^1];
      });
+1
frabe1579 on Nov 23, 2022

@anurse thanks for the report! I’m investigating this now

+1
wtgodbe on Sep 6, 2022
Read more comments on GitHub
← aspnetcore: On Linux, failing to bind the address on Start generates a coredump
aspnetcore: @onevent:preventDefault does not trigger when event is not already registered →