runtime: GenerateRuntimeGraph fails when building runtime in source-build mode on Alpine

Description

The dotnet/runtime build fails when running on Alpine in source-build mode with a portable configuration:

  /runtime/artifacts/source-build/self/src/src/libraries/Microsoft.NETCore.Platforms/src/Microsoft.NETCore.Platforms.csproj(63,5): error MSB4018: The "GenerateRuntimeGraph" task failed unexpectedly. [TargetFramework=net8.0]
  /runtime/artifacts/source-build/self/src/src/libraries/Microsoft.NETCore.Platforms/src/Microsoft.NETCore.Platforms.csproj(63,5): error MSB4018: System.InvalidOperationException: AdditionalRuntimeIdentifier x64 was specified, which could not be found in any existing RuntimeGroup, and no parent was specified. [TargetFramework=net8.0]
  /runtime/artifacts/source-build/self/src/src/libraries/Microsoft.NETCore.Platforms/src/Microsoft.NETCore.Platforms.csproj(63,5): error MSB4018:    at Microsoft.NETCore.Platforms.BuildTasks.RuntimeGroupCollection.AddRuntimeIdentifier(RID rid, String parent) in /runtime/artifacts/source-build/self/src/src/libraries/Microsoft.NETCore.Platforms/src/RuntimeGroupCollection.cs:line 116 [TargetFramework=net8.0]
  /runtime/artifacts/source-build/self/src/src/libraries/Microsoft.NETCore.Platforms/src/Microsoft.NETCore.Platforms.csproj(63,5): error MSB4018:    at Microsoft.NETCore.Platforms.BuildTasks.RuntimeGroupCollection.AddRuntimeIdentifier(RID rid, String parent) in /runtime/artifacts/source-build/self/src/src/libraries/Microsoft.NETCore.Platforms/src/RuntimeGroupCollection.cs:line 140 [TargetFramework=net8.0]
  /runtime/artifacts/source-build/self/src/src/libraries/Microsoft.NETCore.Platforms/src/Microsoft.NETCore.Platforms.csproj(63,5): error MSB4018:    at Microsoft.NETCore.Platforms.BuildTasks.RuntimeGroupCollection.AddRuntimeIdentifier(RID rid, String parent) in /runtime/artifacts/source-build/self/src/src/libraries/Microsoft.NETCore.Platforms/src/RuntimeGroupCollection.cs:line 140 [TargetFramework=net8.0]
  /runtime/artifacts/source-build/self/src/src/libraries/Microsoft.NETCore.Platforms/src/Microsoft.NETCore.Platforms.csproj(63,5): error MSB4018:    at Microsoft.NETCore.Platforms.BuildTasks.RuntimeGroupCollection.AddRuntimeIdentifier(String runtimeIdentifier, String parent) in /runtime/artifacts/source-build/self/src/src/libraries/Microsoft.NETCore.Platforms/src/RuntimeGroupCollection.cs:line 39 [TargetFramework=net8.0]
  /runtime/artifacts/source-build/self/src/src/libraries/Microsoft.NETCore.Platforms/src/Microsoft.NETCore.Platforms.csproj(63,5): error MSB4018:    at Microsoft.NETCore.Platforms.BuildTasks.GenerateRuntimeGraph.AddRuntimeIdentifiers(ICollection`1 runtimeGroups) in /runtime/artifacts/source-build/self/src/src/libraries/Microsoft.NETCore.Platforms/src/GenerateRuntimeGraph.cs:line 327 [TargetFramework=net8.0]
  /runtime/artifacts/source-build/self/src/src/libraries/Microsoft.NETCore.Platforms/src/Microsoft.NETCore.Platforms.csproj(63,5): error MSB4018:    at Microsoft.NETCore.Platforms.BuildTasks.GenerateRuntimeGraph.Execute() in /runtime/artifacts/source-build/self/src/src/libraries/Microsoft.NETCore.Platforms/src/GenerateRuntimeGraph.cs:line 157 [TargetFramework=net8.0]
  /runtime/artifacts/source-build/self/src/src/libraries/Microsoft.NETCore.Platforms/src/Microsoft.NETCore.Platforms.csproj(63,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [TargetFramework=net8.0]
  /runtime/artifacts/source-build/self/src/src/libraries/Microsoft.NETCore.Platforms/src/Microsoft.NETCore.Platforms.csproj(63,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [TargetFramework=net8.0]

The problem is most visible with .NET 6, because dotnet/runtime is built in portable mode as part of the end-to-end source-build process.

With .NET 7 and .NET 8, source-build only builds dotnet/runtime in non-portable mode, so this problem doesn’t appear there out of the box. It only appears when building dotnet/runtime by itself, in a configuration that’s not used by source-build.

Reproduction Steps

Create an alpine container using a Dockerfile

# Dockerfile suitable for building .NET on Alpine

FROM alpine:3.17

RUN apk update && \
    apk add \
    bash \
    binutils \
    clang \
    cmake \
    git \
    gcc \
    icu-dev \
    krb5-dev \
    llvm \
    libstdc++ \
    linux-headers \
    lttng-ust-dev \
    make \
    musl-dev \
    openssl-dev \
    python3 \
    zlib-dev \

Then git clone https://github.com/dotnet/runtime in the container and try and build runtime in source-build mode in the appropriate branch:

main:

./build.sh -p:TargetRid=linux-musl-x64 -c Release --restore --build --pack /p:ArcadeBuildFromSource=true -bl

release/7.0 and release/6.0

./build.sh -c Release --restore --build --pack /p:ArcadeBuildFromSource=true -bl

Expected behavior

Build works

Actual behavior

Build fails

Regression?

No response

Known Workarounds

No response

Configuration

No response

Other information

No response

About this issue

Original URL
State: open
Created a year ago
Comments: 30 (30 by maintainers)

Most upvoted comments

Tag v7.0.4 and v8.0.0-preview.2.23128.3 of runtime with the following command:

@ayakael because source-build uses a non-portable rid, it won’t encounter this parse issue. To make this command work, you can probably unblock yourself by explicitly adding /p:AdditionalRuntimeIdentifiers= to it.

tmds on Apr 6, 2023

I extracted the RID parsing code, and passed linux-musl-x64 through it:

https://dotnetfiddle.net/3ej5qT

As you can see, rid.BaseRID and rid.Architecture comes out as linux (expected linux-musl) and musl (expected x64), respectively. The code just doesn’t have a case when a BaseRID has a - character, unless there’s a version number tracked using . .

A quick fix is to skip adding OutputRid to AdditionalRuntimeIdentifiers when RuntimeOS is equal to linux-musl:

diff --git a/src/runtime/src/libraries/Microsoft.NETCore.Platforms/src/Microsoft.NETCore.Platforms.csproj b/src/runtime/src/libraries/Microsoft.NETCore.Platforms/src/Microsoft.NETCore.Platfor
ms.csproj
index 1df893388..6a1591035 100644
--- a/src/runtime/src/libraries/Microsoft.NETCore.Platforms/src/Microsoft.NETCore.Platforms.csproj
+++ b/src/runtime/src/libraries/Microsoft.NETCore.Platforms/src/Microsoft.NETCore.Platforms.csproj
@@ -23,7 +23,9 @@
     <_generateRuntimeGraphTargetFramework Condition="'$(MSBuildRuntimeType)' != 'core'">net472</_generateRuntimeGraphTargetFramework>
     <_generateRuntimeGraphTask>$([MSBuild]::NormalizePath('$(BaseOutputPath)', $(Configuration), '$(_generateRuntimeGraphTargetFramework)', '$(AssemblyName).dll'))</_generateRuntimeGraphTask>
     <!-- When building from source, ensure the RID we're building for is part of the RID graph -->
-    <AdditionalRuntimeIdentifiers Condition="'$(DotNetBuildFromSource)' == 'true'">$(AdditionalRuntimeIdentifiers);$(OutputRID)</AdditionalRuntimeIdentifiers>
+  <!-- Skips 'linux-musl' and 'linux-bionic' as 'RID.Parse' doesn't know how to parse 'BaseRID' containing a '-' character 
+       See https://github.com/dotnet/runtime/issues/84127 -->
+    <AdditionalRuntimeIdentifiers Condition="'$(DotNetBuildFromSource)' == 'true' AND ( '$(RuntimeOS)' != 'linux-musl' OR '$(RuntimeOS)' != 'linux-bionic' )">$(AdditionalRuntimeIdentifiers);$(OutputRid)</AdditionalRuntimeIdentifiers>
   </PropertyGroup>
 
   <ItemGroup Condition="'$(TargetFrameworkIdentifier)' == '.NETFramework'">

An actual fix is adding in a case for parsing BaseRID that contains a - characters. You would probably have to add logics to src/libraries/Microsoft.NETCore.Platforms/src/RID.cs to be able to parse supported architectures, as to extract that from RIDs that don’t contain a version string.

ayakael on Apr 5, 2023

Later on, I’d expect this fast path to be hit:

I assume the path doesn’t get hit because the knownRIDs didn’t get parsed but constructed. So it looks like: BaseRID = "linux-musl", Architecture = "x64", Qualifier = "". And then fails equality against BaseRID = "linux", Architecture = "musl", Qualifier = "x64".

Because we know the additional rids won’t have a qualifier, my preferred option would be:

Parse(string, noQualifier = true)

Otherwise we still have issue when an additional runtime identifier has - in its os and is not yet known.

tmds on Apr 6, 2023

Is it supposed to work with just TargetRid specified like this?

TargetRid names the output of the build. You can set it to anything you want.

I’m not sure if you actually need to set something because there is Alpine handling here:

https://github.com/dotnet/runtime/blob/e579ccb60acb044453f2cf1907482b165658488e/Directory.Build.props#L198

If it’s needed, you need to set /p:RuntimeOS=linux-musl:

https://github.com/dotnet/runtime/blob/e579ccb60acb044453f2cf1907482b165658488e/eng/SourceBuild.props#L34-L35

My goal was to build runtime in portable mode

I’m not sure what you want to build. source-build (/p:ArcadeBuildFromSource=true) implies non-portable.

tmds on Apr 4, 2023

I am not sure I follow. I ran into this issue when building dotnet/runtime standalone.

SourceBuild.props no longer worked standalone with the changes for the non-portable build. That is fixed on main.

If you want to make it work standalone on 6.0 as-is, you can add /p:BaseOS=.

Could you point to the fix for this so I can backport?

I think that would be https://github.com/dotnet/runtime/pull/80901 and https://github.com/dotnet/runtime/pull/81497

tmds on Mar 30, 2023