sdk: SDK doesnt publish exes correctly when used as a project reference

Often a group of exes need to be built and deployed together. This is common in applications like Roslyn where we ship several exes as a single unit: csc, vbc and VBCSCompiler.

Rather than building each project separately and then copying the outputs together we create deployment projects. Essentially dummy exes projects that just reference all of the exes that we need via project reference elements

<ProjectReference Include="..\..\src\Compilers\CSharp\csc\csc.csproj" />
<ProjectReference Include="..\..\src\Compilers\VisualBasic\vbc\vbc.csproj" />
<ProjectReference Include="..\..\src\Compilers\Server\VBCSCompiler\VBCSCompiler.csproj" />

This approach works fine in desktop MSBuild as it will correctly deploy all of the runtime assets. The same approach does not work on SDK projects though as the publish step fails to include critical files like deps.json and runtimeconfig.json. That completely breaks this approach.

If this isn’t meant to be supported by SDK that would be unfortunate but understandable. I would expect an error though positively asserting that this case is not supported. Today everything builds and publishes without error but the output is completely unusable.

Repo:

Clone https://github.com/jaredpar/roslyn and checkout the branch repro/group-exe. Then run the following commands:

> dotnet restore build/ToolsetPackages/BaseToolset.csproj
> dotnet restore build/ToolsetPackages/CoreToolset.csproj
> dotnet restore build/Toolset/CoreToolset.csproj
> dotnet publish -o ${HOME}/temp/test --framework netcoreapp2.0 build/Toolset/CoreToolset.csproj

The directory ${HOME}/temp/test should contain the deps.json / runtimeconfig.json for csc, vbc and VBCSCompiler but it does not. That means the output of publish is unusable.

CC @khyperia

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Reactions: 25
  • Comments: 35 (22 by maintainers)

Commits related to this issue

Most upvoted comments

OK the reason why the proposed workaround doesn’t work in your repro is because you don’t have any other content items in the lib that are copied over. Please try this one:

  <Target Name="AddRuntimeDependenciesToContent"
          Condition="'$(TargetFrameworkIdentifier)' == '.NETCoreApp'"
          BeforeTargets="GetCopyToOutputDirectoryItems"
          DependsOnTargets="GenerateBuildDependencyFile;
                            GenerateBuildRuntimeConfigurationFiles">
    <ItemGroup>
      <ContentWithTargetPath Include="$(ProjectDepsFilePath)"
                            Condition="'$(GenerateDependencyFile)' == 'true'"
                            CopyToOutputDirectory="PreserveNewest"
                            TargetPath="$(ProjectDepsFileName)" />
      <ContentWithTargetPath Include="$(ProjectRuntimeConfigFilePath)"
                            Condition="'$(GenerateRuntimeConfigurationFiles)' == 'true'"
                            CopyToOutputDirectory="PreserveNewest"
                            TargetPath="$(ProjectRuntimeConfigFileName)" />
    </ItemGroup>
  </Target>

Here’s a potential workaround that includes the deps.json and runtimeconfig.json in the files that referenced projects will copy.

  <!-- Include MSBuild.deps.json and MSBuild.runtimeconfig.json in ContentWithTargetPath so they will be copied to the output folder of projects
       that reference this one. -->
  <Target Name="AddRuntimeDependenciesToContent" Condition=" '$(TargetFrameworkIdentifier)' == '.NETCoreApp'" BeforeTargets="GetCopyToOutputDirectoryItems">
    <ItemGroup>
      <ContentWithTargetPath Include="$(ProjectDepsFilePath)" CopyToOutputDirectory="PreserveNewest" TargetPath="$(ProjectDepsFileName)" />

      <ContentWithTargetPath Include="$(ProjectRuntimeConfigFilePath)" CopyToOutputDirectory="PreserveNewest" TargetPath="$(ProjectRuntimeConfigFileName)" />
    </ItemGroup>
  </Target>

It should be in .NET SDK 5.0.200, which will ship with VS 16.9

VS 16.9.0 is out and it works! Thanks for the fix.

@MartyIX $(TargetFrameworkIdentifier) is still .NETCoreApp in .NET 5.

Note that the workaround doesn’t handle a self-contained publish correctly: the dependency won’t be published in self-contained mode.

To reproduce:

dotnet new console -n ConsoleAppA
dotnet new console -n ConsoleAppB
dotnet add ConsoleAppA reference ConsoleAppB

Paste the workaround to ConsoleAppB, then:

dotnet publish ConsoleAppA -r win-x64

Now diff the two .deps.json files in ConsoleAppA\bin\Debug\netcoreapp3.1\win-x64\publish:

  • The runtimeTarget of ConsoleAppA is .NETCoreApp,Version=v3.1/win-x64 and it has all the runtime dependencies included
  • The runtimeTarget of ConsoleAppB is .NETCoreApp,Version=v3.1 and the runtime dependencies are not listed

This can be fixed by setting the RuntimeIdentifier or RuntimeIdentifiers property in ConsoleAppB.


I’m currently using a… less subtle workaround though as I haven’t seen this issue before, and I don’t really want to set the RuntimeIdentifier property in the project. I’m posting it here in case it may help someone else:

In ConsoleAppA, add this:

<Target Name="AddExeDependencyForBuild" BeforeTargets="CopyFilesToOutputDirectory">
  <MSBuild Projects="..\ConsoleAppB\ConsoleAppB.csproj"
           Targets="Publish"
           Properties="PublishDir=$([System.IO.Path]::GetFullPath($(OutDir)));PublishReadyToRun=false"
           BuildInParallel="$(BuildInParallel)" />
</Target>

<Target Name="AddExeDependencyForPublish" BeforeTargets="CopyFilesToPublishDirectory">
  <MSBuild Projects="..\ConsoleAppB\ConsoleAppB.csproj"
           Targets="Publish"
           Properties="PublishDir=$([System.IO.Path]::GetFullPath($(PublishDir)))"
           BuildInParallel="$(BuildInParallel)" />
</Target>

The drawback of this approach is that the project gets built twice.

I suppose you can combine both workarounds this way:

  • Set Condition="'$(RuntimeIdentifier)' == ''" on the AddRuntimeDependenciesToContent target
  • Delete the AddExeDependencyForBuild target (only keep AddExeDependencyForPublish)

Here’s yet another fix I tried, it’s to be used with the original one. Add this to ConsoleAppA:

<Target Name="ForceRidAgnosticToFalse" AfterTargets="_GetProjectReferenceTargetFrameworkProperties">
  <ItemGroup>
    <_MSBuildProjectReferenceExistent Update="@(_MSBuildProjectReferenceExistent)"
                                      UndefineProperties="$([MSBuild]::ValueOrDefault('%(UndefineProperties)', '').Replace(';RuntimeIdentifier', ''))" />
  </ItemGroup>
</Target>

This is not great either, as it depends on and modifies “internal” data (targets and items whose names start with an underscore).


Anyway, nothing I posted here is particularly pleasant… I’d be grateful if the SDK was fixed to be able to handle exe references correctly. 😉

The same approach does not work on SDK projects though as the publish step fails to include critical files like deps.json and runtimeconfig.json.

Just to note, that the problem appear not only on publish step, but on simple build too, see #11207.

. In order to accomplish this I think you could change your aggregator project to have a list of projects that it needs to publish and for you to invoke publish in each of them separately.

That doesn’t quite get the behavior we want though. Part of the advantage to using the <ProjectReference> route is that MSBuild hepls ensure we have consistent dependencies across the projects. If any dependencies are wrong the standard warnings around conflicts will come into play.

This is important because the versions need to be consistent in order to ensure the exes will run correctly when all their dependencies are dumped into a single directory.

If that’s the case, you are right, we do not support that. P2P references are treated as libraries and not applications

In that case why do the app.config files get deployed at all? Thats only relevant when running the application. I would assume, possibly wrong, that app.config, runtimeconfig.json and deps.json would all be deployed or none.

We’re running into challenges because of this too. It would be really nice if this could get resolved.

However, I also noticed that some dependencies are not copied for the referenced project as well.

Dependencies shouldn’t be copied. Dependencies should be allowed to flow transitively into the referenced project. That’s what @jaredpar was specifically calling out as something he wanted to accomplish with this strategy here https://github.com/dotnet/sdk/issues/1675#issuecomment-338283234.

In this case System.Security.Permissions wasn’t copied because your referencing project is an ASPNetCore project and AspNetCore’s shared framework includes System.Security.Permissions.dll. You’d run into the same problem when referencing any package which is in the AspNetCore.App shared framework and not NetCore.App shared framework.