sdk: Access denied for NuGet package source while build dotnet/cli 3.1.300 Tag

Seems like https://github.com/dotnet/cli/commit/7c1fa3d6c7ce4082a4c9974643536f821b91800c#diff-a7c26590e27fcbb61c69ff2564beb1fa changed the feeds from darc-pub-dotnet-* to darc-int-dotnet-*, which I assume made them go from public to internal. As the change is made by @dotnet-bot, I can’t think of any other reason for this change than an oversight and that this must be a bug.

Simply removing the internal feeds from NuGet.config just leads to a swath of other problems saying Source control information is not available - the generated source link is empty. I don’t even know where to begin looking for the source of this problem, so some assistance would be helpful.

Trying to build a version from before https://github.com/dotnet/cli/commit/7c1fa3d6c7ce4082a4c9974643536f821b91800c, such as v3.1.201, ends up printing an almost endless amount of the similar error message to the console:

Retrying 'FindPackagesByIdAsync' for source 'https://pkgs.dev.azure.com/dnceng/9ee6d478-d288-47f7-aacc-f6e6d082ae6d/_packaging/2160f577-8dc0-4c31-83d2-8d0e0da8208c/nuget/v3/flat2/runtime.native.system/index.json'.
  An error occurred while sending the request.
    Problem with the SSL CA cert (path? access rights?)

Finally failing and halting on the following error:

.dotnet/sdk/3.1.200-preview-014946/NuGet.targets(124,5): error : Too many open files [Microsoft.DotNet.Cli.sln]

Trying to access one of the faulty feeds ends up returning the following JSON-wrapped exception:

{
    "$id": "1",
    "innerException": null,
    "message": "Can't find the package 'runtime.native.system' in feed 'dotnet3.1'.",
    "typeName": "Microsoft.VisualStudio.Services.NuGet.WebApi.Exceptions.PackageNotFoundException, Microsoft.VisualStudio.Services.NuGet.WebApi",
    "typeKey": "PackageNotFoundException",
    "errorCode": 0,
    "eventId": 3000
}

So currently, it seems impossible to build any version of .NET Core from source unless you’re hired by Microsoft and have access to their internal NuGet feeds. That’s very, very unfortunate. A fix would be highly appreciated. 🙏

With Homebrew/homebrew-core#60929 merged, this can be considered fixed for me at least.

But before dotnet/source-build#1685 is resolved, it’s hard to tell. Is it possible to have that fast-tracked somehow, @dagood?

Replied at https://github.com/dotnet/source-build/issues/1685#issuecomment-675476767.

Seen some build servers have an executable bash script in path named dotnet that sets up needed environment variables for that session only and passes along all arguments to “native” binary. Could that be a possibility?

Since the formula is for both the SDK and the runtime, I’m not sure which version number takes precedence.

Using the SDK version is correct:

• There might be a new version of the SDK released without a new runtime version
• Multiple versions of the SDK use the same version of the runtimes (e.g. 3.1.106 and 3.1.302 both use 3.1.6)

We have some info about packaging recommendations at https://docs.microsoft.com/en-us/dotnet/core/distribution-packaging, but it actually gets this specific piece of info wrong so I’ve filed https://github.com/dotnet/docs/pull/19921 to fix it.

To me, it feels like using the dotnet/source-build repository for source builds is the best option. The only thing that seems to be missing is https://github.com/dotnet/source-build/issues/1646#issuecomment-668267153.

Building the sdk and cli repositories require too much source modifications and other local “hacks” in order to get the build to work and seems to go against the goals of the repositories. Since the goal of the source-build repository is to provide source builds, that feels like the absolute best option to me.

What I did to get around the issue was remove the internal darc feeds and the build worked fine for me afterwards. I would really like a better fix then me making changes to the repo before doing a build.

I was building in iTerm2, running./build.sh --configuration Release on a downloaded .tar.gz, as that’s the preferable way to build from source in Homebrew. It’s possible to build from a cloned Git repository, but that means the formula becomes “head only” and has to be installed with the --HEAD argument. I’m not yet sure how it will affect dependent formulae, but it doesn’t sound like the recommended path to take.

There is another option:

• NuGet has a recent change that allows for inaccessible feeds to not fail the restore. Because of how we specify package versions, this option (RestoreIgnoreFailedSources=true), should be safe to use. It looks like it’s a tad buggy at the moment:

https://github.com/NuGet/Home/issues/9765

The downside of SetupNuGetSources.ps1 is:

• It alters local files in the enlistment
• It’s not currently integrated into build.ps1 (and not all places use the common build.ps1), so users would need to explicitly call it.