FusionCache: [BUG] .NET8 support (as well as previous versions) is probably broken starting from v2
I stumbled upon NU1605 error (package downgrade detected) in my project (which targets .NET8 SDK) after installing FusionCache v2.
<TargetFrameworks>netstandard2.0;netcoreapp3.1;net6.0;net7.0;net8.0</TargetFrameworks>
but finding also:
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="9.0.0" />
</ItemGroup>
Reason: Version 9 of “Microsoft.Extensions.Caching.Memory” NuGet package (as well as others) is causing NU1603 error in projects targeting .NET8 SDK (and probably 7, 6, etc.).
Arguments:
- it is probably not right to reference NuGet packages of .NET9 while declaring support only for .NET8 max.
- OR we are missing conditional ItemGroup in csproj files, like:
<ItemGroup Condition="'$(TargetFramework)' == 'net8.0'">
<PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="8.0.1" />
<PackageReference Include="System.Collections.Immutable" Version="8.0.0" /> <!-- not sure if this one is needed -->
</ItemGroup>
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 2
- Comments: 52
Ok so I read the things and watched the videos, thought about it both in general and in the particular case of FusionCache.
I’ll now try to summarize the main points about this kerfuffle, step by step in a logical order, trying to stick to the things that are strictly related here to avoid confusion with secondary stuff.
Here we go.
What we mean when we say “.NET”
Seemingly silly to specify this, but just to be all on the same page about the details and their nuance.
So .NET as a whole is a product, composed of multiple things, and the things that are relevant to this discussion are:
Microsoft.Extensions.Caching.Abstractions)Each .NET “main version” like 7, 8 and 9 corresponds NOT ONLY to the runtime, but ALSO to the related core packages that comes out with tha .NET product: for example, when .NET v9 came out, it was accompanied with v9 core packages.
This does NOT (necessarily) mean that each v9 core package requires v9 of the .NET runtime: since core packages are “just” packages, they are frequently backward compatible with older TFMs.
A TFM (Target Framework Moniker) is a string used to identify a version of .NET “product” (meaning, as a whole). Somewhat nitpicking here, but not only a specific .NET product version (a.k.a: a “.NET release” like we see in November each year), but also a “.NET standard”. What is a .NET standard? Like it’s said in the official docs it is:
Package versions
Now anyway, by targeting the .NET 8 TFM we basically get a dependency on all the related core packages from that version, but we can also reference those same packages (even in the same version) directly.
For example,
System.Text.Jsonv9 does multi-targeting, meaning it targets multiple TFMs at the same time, including .NET Standard 2.0, which means it can also run on the very old .NET Framework 4.X from around 2018:As we can see when targeting .NET Standard 2.0 there are a lot of packages referenced, including
System.IO.Pipelinesv9 andSystem.Text.Encodings.Webv9, but when targeting .NET 8 these 2 are the only one there. Why? Because by targeting .NET 8 the other packages are already part of what .NET 8 is, just like when targeting .NET 9 nothing is there because these 2 packages have been added to the definition of what .NET 9 is, so it’s not necessary anymore to directly reference them.The main (but not necessarily the only) reason for multi-targeting a library is to have less dependencies declared in the manifest (even though they are there anyway at runtime because they are part of that runtime version itself).
Support Policy: STS/LTS
Each major .NET product version (so runtime + core packages) coming out in November each year has a corresponding support policy, either STS or LTS: an STS version is supported for 18 months, while an LTS is supported for 36 months (3 years).
For various reasons (eg: primarily costs associated with long term support of multiple versions) Microsoft decided that each major version of .NET would alternate between STS and LTS, so for example .NET 6 is LTS, .NET 7 is STS, .NET 8 is LTS and the recently released .NET 9 is STS.
One thing to pay attention to is that the support policy is for the entire product: so NOT ONLY for the .NET runtime, but also for the core packages.
This means that companies that, for whatever reason, only allows their tech people to use an LTS version, are forcing them to stay on an older tech stack: again, not just the runtime, but also core libs/packages, and so on. And you’d be surprised to know how many times companies don’t even know that even by disallowing, for example, .NET 9 they are not in fact actually blocking all the v9 core packages, and they’ll get updated anyway without nobody even noticing.
Package Version Lifting
Now let’s say we create a console app for .NET 8: this means we are implicitly referencing the v8 core packages, right? But what happens when we then add a reference to a package that, in turn, references a core package in v9?
An automatic “lift” happens, and now our console app is referencing v9 of that package, along with all its dependencies (probably v9 themselves).
The good @DamianEdwards clearly explains all of this in a video, where at the 24m40s mark starts with “ok, this is an incredibly complicated question…”:
I think his explanation is complex (because the thing itself is complex, and cannot simplified more) but masterfully clear in the way it’s explained. Really, go watch it, it’s that good.
So anyway, can the automatic lift always work? No, there’s one case where this process breaks, and it’s when our app has a direct dependency to the same package.
Basically, if for example in our app we directly reference v8 of
System.Text.Jsonand then we also reference aFoopackage that, in turn, references v9 ofSystem.Text.Json, the system cannot auto-lift versions, because we are being explicit about the version ofSystem.Text.Jsonwe want (v8) but we are ALSO saying that we want to work withFoowhich in turn requiresSystem.Text.Jsonv9.In cases like this we’ll receive an error like
"Detected package downgrade: System.Text.Json from 9.0.0 to 8.0.0"or similar.So, what can we do? We need to either:
Foo(or use an older version without theSystem.Text.Jsonv9 requirement)So, what should library authors do?
Library authors: what versions to target?
Now, library authors like myself need to decide what to target: apart from multi-targeting or not, we need to pick a baseline minimum version.
Of course we can target whatever minimum version we want: let’s keep in mind it’s our own project, our choice, etc.
But.
But let’s see what we may do to give good support to people and companies that (good or bad that it is, and I personally think it’s at least very limiting) want to stick to LTS only versions.
While doing so, let’s also say that a reasonable thing in this regard is to support only the last LTS (right now this means .NET 8): everything more than that (eg: also target .NET 6 or below) is of course possible, but let’s say not strictly “necessary”.
I can see 2 different options:
Oh, and in the 2nd option above I said I “highest version” because instead of sticking to the baseline
v8.0.0of the LTS package, we can/should stick to the highest version (with the same major) since regarding the STS/LTS part it doesn’t change anything, and because of things like CVEs like this:Anyway with both options users would still be able to directly upgrade core package references to v9 or more, and everything should (in theory) work correctly, so there’s also freedom there.
But what if v9 core packages have new overloads that are more optimized? We library authors may have optimized code-paths via some
#IFdirectives, but still that would enable the new stuff NOT when referencing v9 of the core packages, BUT ONLY when running on .NET v9, which is different, so a lot of potential good would be left on the table.And what if instead v9 core packages have totally new stuff, with new types to use and maybe return from our public methods? It becomes harder to opt-in to those things via
#IFdirectives.Now, unless v9 core packages have some new features strictly needed by the library, “Last LTS” can be the reasonable thing to do, imho.
Ok so, can I do this for FusionCache v2?
Eh…
Ok so, FusionCache v2?
In theory I may easily change the references to v8 core packages for FusionCache, and in fact this is what I basically ended up doing in the past (meaning: generally sticking to the latest LTS), except… this time I can’t, because of this feature.
You see, the HybridCache abstractions (eg:
public abstract class HybridCacheand related basic options classes) that 3rd parties like myself can implement, have been defined in v9 of the coreMicrosoft.Extensions.Caching.Abstractionspackage.If I try to get back to v8 core packages the project would not compile, that’s it.
The 2nd option above is also not an option, because by multi-targeting .NET 8 with v8 core packages and .NET 9 with v9 core packages, it would not compile for .NET 8 for the reasons above.
Sure, I can go with
#IFdirectives and exclude “everything HybridCache” for .NET 8 and below, but… that would limit people targeting .NET 8 that have no problems at all updating to the latest core packages.Case in point, currently I have about 27K downloads of
v2+v2.1: even considering downloads for demos or little tests, we are talking about thousands of people already using it, including HIBP, Dometrain and others, happily using it as-is without any issue.Another option I’m thinking about is that, in theory, I can stick to v8 core packages for the main package and create a new package that contains just the HybridCache adapter, where I can reference v9 packages. Want the HybridCache-compatible stuff? Use the extra package, but with v9 core packages. Don’t want that? Use the main package only so that, if you want, you can stick to v8 core packages.
I even already have such package, see here:
Before going GA with v2 though, the extra package became “useless” and I emptied+deprecated it, see here for why.
The problem in doing the extra package thing anyway is that this would be a breaking change, and everybody already happily using FusionCache v2 will then update and have problems.
On the other hand, I can simply stay like it is right now: user wanting the new stuff can update their core packages, or stay with the v1.4.1 version. The problem is that v2, apart from the HybridCache thing, contains A LOT of wonderful stuff like Tagging and Clear support, which are honestly huge.
And I’d really really like to give those features to the widest possible group of users 🥲
Oh, also: I’m working on adding support for the new (optional)
IBufferDistributedCacheinteface, to reduce allocations even more, but that is also defined inMicrosoft.Extensions.Caching.Abstractionsv9. This measn I’d have the same problem of not being able to reference it in the main package, if I go with the “v8-only in main package” route.Conclusions (actually, not yet)
So anyway, I’m still thinking about what to do here, pondering pros and cons.
Meanwhile, please let me know what you think: did I miss something? Did I say something wrong? I’d like to know before making a decision.
Will update as soon as I get to a definitive answer.
Thanks!
PS: all of this will become basically useless as soon as .NET 10 (LTS) will be out in November. Yeah, I know, 9 months, but still.
Any kind of change now would break users who have already upgraded. I would be uncomfortable doing that.
This issue has 1 upvote, while V2 has 27k downloads on nuget. 1.3.0 has almost 500k downloads. If this is the reason people aren’t upgrading to V2 I would expect a lot more activity. On this issue.
I’ve been thinking about this, and to me this is a have your cake and eat it too situation. There is this sentiment that STS is big and scary, while LTS is warm, fuzzy and stable. So they don’t want to leave LTS because
it doesn't make sense to race for new features, but then do want to upgrade to a major version of a 3rd party library (made by one person) that came out a month ago, with tons of changes, new features and new dependencies to facilitate those features.In this case, and same as the OTEL case there is a justifiable reason on why the dependency had to be lifted. It is required because you use the feature that it provides. It’s not a
bigger number bigger bettersituation. I think the same thing could be the best approach here, the next time this is about to happen try to limit the required version, but leave it as it is for now.This is the point where I say Could be wrong though. 😅 It’s your decision.
Ok fellas, now that my weekly daily job + commute is off the table, I’m reading all the backlog of comments, links and whatnot, here and on social.
First thing that came up from a comment on Bluesky is this video, where at the 24m40s mark @DamianEdwards starts by saying “ok, this is an incredibly complicated question…”
So yeah, will watch some videos, read some docs and csproj files posted here and will get back to you all with my thoughts.
Wish me luck.
I think we should look at the ones I posted.
Serilog replicates the .NET versioning schema exactly for these reason. Users of the .NET 8 should use the 8.* packages, as explained here.
Opentelemetry needs the latest versions of System.Diagnostics, but in any case, the decision of bumping all the dependencies to the latest version was also somewhat controversial
Considering that .NET 8 is LTS, that’s not as big of a problem. And in any case, they don’t force STS packages to the LTS users.
That is irrelevant. Microsoft offers LTS versions for a reason. Some companies mandate the use of LTS.
In any case it seems that nuget guidances in this respect, so the library authors should depend on versions above the TFM ONLY if they need it, so this topic should be settled: do not force upgrades if it’s not needed.
Can OP try to upgrade all direct references of
Microsoft.*packages to9.*? And if not, why not? Afaik you should be upgrading these across major version as well. Don’t confuse the package major version with current latest .Net version. They are versioned like this for convenience only, there is actually no runtime blockers as evidenced by the supported frameworks table on nuget.orgAlso, why are you locking to the .Net 8 SDK? The SDK itself should be the latest always. The oldest SDK you can need today is .Net 6, but that’s because 9 dropped support for building 6. If you develop for .Net 8 today, you need two things. 9 SDK and 8 runtime. This is not relevant to the issue, I’m just curious what is the justification.
UPDATE 2025-02-23
So I tried the FusionCache.Core package route, basically:
v8core packagesv9core packages that are needed (no need for an extra package for that)Then I tried compiling the main projects and… issues.
The first one is that I’m using the
AddException()method onActivity(for observability and friends), and that has been introduced only in later versions of theSystem.Diagnostics.DiagnosticSourcepackage.So I had to look for the official source code for that and recreate it locally as a different ext method, but a couple of changes ot make it compile. Mmmmh.
Then the main projects compiled.
So I tried compiling the entire solution (with extra projects like tests, the Simulator, ec) and… again, issues.
It was not compiling because some projects were only referencing a satellite package, and that reference would transitively bring in the FusionCache package: by changing the packages refs they are now bringing in the Core package instead of the main one. Problem is, those projects were already using the HybridCache adapter in a way or another, and that was now missing, resulting in a surprise compilation error. This is something that would realistically happen to some of the thousands users out there, too. Then there are some issues with the git history, but I suppose I can work them out. Also having the main package containing just a 10 lines class (the adapter) feels quite strange.
All in all I’m not so sure about the Core package approach for now.
I’m now trying the time travel approach, meaning creating a
2.0.1package aligned feature-wise with2.1, will update.@alexaka1 counting stars and downloads is not the right strategy as we all know what might happen with the numbers: https://www.youtube.com/watch?v=PiT-441KR3s …
As the OP, I am happy to accept the “ship has sailed” conclusion with a hope to re-join the official package at “net10.0” TFM.
Glad we’ve seen the problem instead of pretending it doesn’t exist - there is lot of fruit for thoughts. Giving a HUGE ⭐ to @jodydonetti for this answer
The issue is no longer a blocker as I’ve successfully re-built the library for “net8.0” without HybridCache (once again, the link to the backporting patch is above in this thread).
First and foremost, thank you @jodydonetti for your outstanding work. I’ve been following this closely as I’m dealing with the same conundrum as a framework author.
Just my two cents on the matter - unrelated to FusionCache.
Our in-house framework has several dependencies on libraries like MySql.Data, MySqlConnector, MongoDb, SQL Server Client, npgsql, Kafka, OpenTelemetry and of course Microsoft’s Platform Extensions. I’m sure I missed some. No, I definitely missed some - the expanded dependency tree is bound to surprise me if I were to examine each package recursively.
It’s becoming borderline impossible to match the TFM’s major version with an arbitrarily complex dependency tree.
I’ve been trying to abide by the “major versions must match” rule but I’m seeing a general ecosystem shift towards whatever new version is out for updates.
Unsurprisingly, this implies a non-zero probability of introducing breaking changes that creep up transitively; for example, some Microsoft Platform Extension (can’t remember which off the top of my head) had a different nullability annotation that caused a compilation error. It’s nothing to write home about, I can fix that for my own codebase - but I can’t do the same for consumers of my framework, which will get a breaking change for free, as it were. Long story short, this has a real implication on how we approach versioning as library and framework authors.
In all honesty, I wish Microsoft kept the Platform Extension updates in line with the TFM, so that the whole ecosystem would follow suit.
However, I’m afraid that particular ship has sailed, and whatever the decision, some are bound to be disappointed. I for one will attempt to introduce dependencies with versions >= TFM since I believe I’ll be forced to in the short term.
Again, just my 2c - thank you all for the great comments so far.
Hey, @jodydonetti
I run my smoke unit-tests in regard to FusionCache (removal by tags, failsafe) - no errors or behavior changes with v2.0.1
Thank you!
Ok so a way to make it kinda work may be to take the code that depends on the HybridCache abstractions and move it back from the main package (where the dependencies can then get back to v8 core packages) to the separate package (where the dependencies will be on the v9 core packages).
This will make FusionCache v2 LTS-friendly, even though I don’t know all the ramifications of downgrading the dependencies from v9 core packages to v8, I’m pretty sure it will lead to surprises for some people out there.
Having said that, there’s another bigger problem: people that are already using the
AsHybridCache()method on the builder would not compile anymore, because they are not referencing the extra package, and will not know what to do. Yes, release notes & so son, but I cannot count on that for a minor version (otherwise I would need to release FusionCache v3 to make such a breaking change, and… yeah, no thanks).One way to avoid this (still theoretical, I’d have to try and extensively test it) would be to split the related code: I can leave the
AsHybridCache()method declaration in the main package, but have the actual inner implementation in the extra package being injected at startup. Also, in theAsHybridCache(), there may be a check for which when the method is called, if the implementation has not been injected, it will throw an exception with some explanations about adding the missing package etc.But this, for people already using the
AsHybridCache()method, would lead to them:This would be a runtime surprise, the explanation probably a complex one to follow, and all of this hassle for something that is now a core a dependency that will stay in .NET forever (the
HybridCacheabstract class is basically like theIDistributedCacheinterface).Also, the extra package (which contains like 20 lines of code for the adapter to a thing that is now part of core .NET) will then be required with all future versions of .NET, even after .NET 10 (LTS) will be out, which is not great.
I’m still open to ideas, but overall I think the supposed move would be quite big all things considered, and probably not worth the effort.
One last (crazy) option would be to release a new v2 version, higher than
v2.0.0but lower than the currentv2.1.0(so, maybev2.0.1?) that targets v8 core packages and does not contain the HybridCache adapter: this version would not be updated with new stuff moving forward, but will contain Tagging + Clear, basically av2“slim” in a way, that would serve as a “bridge” until .NET 10 comes out.But again, is it really worth the trouble, confusion, etc? Mmmh, I’d say no.
Let me know your opinions.
Hi @juantxorena
Because as stated core packages of a certain major version (eg:
v8.X) are covered by the same support policy as the related .NET product version (eg: .NET 8), and since .NET 8 is LTS (long term supported) they updated the core packages too.So “simply upgrade packages without changing the TFM” is both technically correct (meaning: it in fact works) but also an incomplete statement (meaning: it inadvertently implies a switch from LTS-supported packages to STS-supported packages).
Agree?
No, not that they should use, but that they can use if they want to stick to LTS-only stuff. Now, you may be thinking “well if they can update packages to
v9why would they not also update .NET tov9?” but that may be for other reasons like some tooling around the .NET runtime/sdk may not be updated to .NET 9 (been there, sadly), like in the build pipelines, etc.On this I agree: if a company want to use LTS-only stuff (both .NET itself and core packages) they should ideally be able to do that, but as stated it’s more complicated than this because of the need to natively support HybridCache.
Will update in a moment with some news, which we can discuss. Stay tuned.
Thanks @rcollina for sharing you experience here, it adds to the mix of knowledge to try and get to some overall guidance, all things considered.
And yes: when implemented, the compatibility matrix with multiple TFMs can become quite crazy, quite fast 🥲
Again, really appreciate it.
@andriibratanin
I did the experiment, this would be my solution:
Case 2 fixed:
Case 4 fixed:
This is what I meant by multi TFM.
From the very first post: “it is probably not right to reference NuGet packages of .NET9 while declaring support only for .NET8 max.”
They are not multi-TFM, it is just a copy-paste in release notes. Please consider doing the experiment yourself from this post: https://github.com/ZiggyCreatures/FusionCache/issues/377#issuecomment-2638219692
With all due respect @andriibratanin, you upgraded to a major version of FusionCache, and experienced an incompatible change that you disagree with.
And to reiterate it is NOT an incompatible change, because the package still works on
net8.0and lower. Why would the 9.x packages multi TFM net8.0 if they were meant for net9.0 only? I will continue to push back on the LTS vs STS stuff, because it is just silly and just a marketing gimmick. You are suggesting that LTS is more stable, which is just simply not true. It is just a gimmick, and Microsoft should get all the criticism for perpetuating this model.For me, “just upgrade” sounds strange, because it cancels the idea of versioning at all:
DependencyInjection.Abstractionsis the package you must reference when creating your own DLLs (you will have this dependency because of DI registrations and you cannot “hope” someone else already references it) - this is the reason of my “requirement by code”Upgrades are actually not as simple as we try to imagine them - they take time: to make the decision, to read release notes, to re-compile. For example, do you know that “FluentAssertions” are no longer free starting from their version 8? “Just upgrade” policy may lead you to very unpleasant consequences one day…
Personally, it is not hard to check-out the repo and re-compile it targeting v8 - kudos to Jody and the community - the code is covered with tests very good. The fun fact is I am 99% sure it will compile and will not have any issues…
So, yes:
Happy have started this topic - we’ve already learned alot and, probably, gave another nudge to MS for clear guidances… 😉
Believe .NET9 will fallback to closest target-framework which is NET8. Just like NET48-project will fallback to NET461.
When I compile a NET9-project that depend on
NLog.Extensions.Loggingthen output-folder contains the NET8-assembly (and not NetStandard2)For clarity on .NET major releases:
https://dotnet.microsoft.com/en-us/platform/support/policy
I’ll add: looking for example at the Microsoft.Extensions.Caching.Hybrid package, they ( @mgravell basically) are doing the same thing there:
Another example is System.Text.Json, and they do the same:
Maybe is the direct reference I mentioned? Or maybe something specific about CPM (central package management)?
I’ll keep looking into this.