roslyn: RC1 regression: dependencies no longer work correctly in source generators across

Hi @roji. I apologize for the delay on this. I was able to reproduce the problem. In the binlog I see the following arguments passed to the compiler:

/analyzer:C:\Users\rikki\.nuget\packages\scriban.signed\4.0.1\lib\netstandard2.0\Scriban.Signed.dll
/analyzer:C:\Users\rikki\src\roji-test\TestSourceGenerator\bin\Debug\netstandard2.0\TestSourceGenerator.dll

(below I use the terms “analyzer” and “source generator” interchangeably because the loading mechanism for them is the same)

The problem here is that we introduced a new AssemblyLoadContext-based analyzer loading mechanism when compiling on .NET Core. Basically, now when we build on .NET Core we make a few deep assumptions:

1. Any given analyzer has all of its dependencies (aside from framework and Microsoft.CodeAnalysis.* stuff) right next to it in the same directory. This has been a requirement for nuget packaging of analyzers for some time.
2. Every analyzer and analyzer dependency has its full path passed to the compiler via the /analyzer option.

The design of our ALC-based system makes the assumption about (1) into a requirement, when before it was perhaps just an assumption, at least in the ProjectReference scenario. When a dependency is in a different folder, the analyzer’s ALC just won’t find it any more.

From talking with @jmarolf I found you can set the msbuild property <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies> in the analyzer project to ensure that its dependencies get copied to the output directory. However this doesn’t solve the problem because msbuild still ends up using a path to the dependency which is located within the nuget cache. I think if we could get msbuild to use the paths to the copied dependencies in the output directory then we would be all good. This might be a solution, but I’m not yet certain whether there’s a better one.

To fix this scenario in the short term I think we need to figure out what changes should be made to the analyzer project and consuming project to meet the compiler’s new invariants. I will follow up when I have an answer for what that change looks like. (It’s possible that the compiler’s invariants or discovery mechanism should be adjusted, but I’m less certain of that.) cc @chsienki @jaredpar @jkoritzinsky

In the medium term: it seems like ProjectReferences to analyzers are not a well tested scenario in the SDK, and that a bigger investment may be warranted to make the experience much more straightforward and productive.

@roji Agreed. On our side we’ve considered introducing a feature flag for .NET 6.0 which would simply force loading all analyzers and dependencies in a single ALC (maybe using whatever ALC the compiler is loaded into, or maybe one per analyzer loader.) This would fix your scenario but break other scenarios. So the consuming project would probably need to set one or the other value for the flag depending on their constraints. If the previous scheme in preview7 worked in your project then this feature flag would probably get you up and running again.

There are some other alternatives that I haven’t fully thought through yet, but at any rate I agree that we should do something here for .NET 6 to ensure people aren’t just left out in the cold.

I had a situation alike in some regards. I have a source generator with a project reference to a utility lib. The utility lib is used in a VSIX extenstion and in a source generator in webapi project. To make this work i had to reference both the utility project and the source generator project as analyser references in the webapi project, and everything works. Now the source generator project picks up the util project without errors.

<ItemGroup>
   ...
    <ProjectReference Include="..\XXX.Utils\XXX.Utils.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false"/> 
    <ProjectReference Include="..\XXX\XXX.Generator.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false" />
...
  </ItemGroup>

This is affecting me too, although in my case the source generator depends on another project reference rather than a package reference. I didn’t actually notice until recently either since it only happens on the command line and not with Visual Studio.

---

@RikkiGibson Thanks for the detailed explanation for what’s going on. (It also helped to look through the PR which caused this https://github.com/dotnet/roslyn/pull/55098/)

I ended coming up with a workaround sort-of similar to what you mentioned in https://github.com/dotnet/roslyn/issues/56442#issuecomment-923379101. It identifies Analyzer items which came from a project reference and copies them to a project-specific intermediate directory and tells Roslyn to load them from there instead: (This target goes in the analyzer consumer.)

<Target Name="Roslyn56442Workaround" AfterTargets="ResolveProjectReferences" BeforeTargets="CoreCompile">
  <Error Message="IntermediateOutputPath is empty." Condition="'$(IntermediateOutputPath)' == '' and '$(IntermediateAnalyzersPath)' == ''" />
  <PropertyGroup>
    <IntermediateAnalyzersPath Condition="'$(IntermediateAnalyzersPath)' == ''">$(IntermediateOutputPath)collected-analyzers\</IntermediateAnalyzersPath>
    <IntermediateAnalyzersPath Condition="!HasTrailingSlash('$(IntermediateAnalyzersPath)')">$(IntermediateAnalyzersPath)\</IntermediateAnalyzersPath>
  </PropertyGroup>

  <!-- Identify all analyzers which originated from a ProjectReference and remove them -->
  <ItemGroup>
    <_AnalyzerFromProjectReference Include="@(Analyzer)" Condition="'%(Analyzer.ReferenceSourceTarget)' == 'ProjectReference'" />
    <Analyzer Remove="@(_AnalyzerFromProjectReference)" />
  </ItemGroup>
  
  <!-- Copy the analyzers to a common directory -->
  <Copy SourceFiles="@(_AnalyzerFromProjectReference)" DestinationFolder="$(IntermediateAnalyzersPath)" SkipUnchangedFiles="true" />
  
  <!-- Re-add the analyzers using the copied analyzer instead -->
  <ItemGroup>
    <Analyzer Include="@(_AnalyzerFromProjectReference->'$(IntermediateAnalyzersPath)%(Filename)%(Extension)')" OriginalProjectReferenceOutputPath="%(Identity)" />
  </ItemGroup>
</Target>

---

Another idea I had while looking over the new loader logic that seems like it should be possible: Rather than use a single ALC per directory, maybe the concept of an “analyzer context” can be introduced. If the analyzer context is not explicitly specified, it defaults to the directory which contains the analyzer (or maybe the NuGet package ID which provided the analyzer.) Some MSBuild logic similar to above could make all ProjectReference analyzers default going into their own context.

We are currently running with https://github.com/dotnet/roslyn/issues/56442#issuecomment-926044063

I’m curious, why is it working fine in Visual Studio (2022 preview 4.1)? Do analyzers/generators resolve their own dependencies differently?

I don’t know for sure whether 2022 preview 4.1 corresponds to .NET 6 RC1, but to clarify:

Analyzers and generators use the same loading/dependency resolution mechanism. However, we do different things depending on whether the compiler is running on .NET (Core) or .NET Framework. The above issue isn’t applicable when the compiler is running on .NET Framework (when you hit “build” in msbuild.exe or in VS or when you’re just using the editor). It’s only applicable when the compiler is running on .NET/.NET Core (for example when you use ‘dotnet build’ on the command line to build).

I believe setting <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies> in your project copies all dependencies to the output directory so you shouldn’t need to do that in your workaround @PathogenDavid you just need to remap the analyzers to point to that directory.