sdk: Build error "An item with the same key has already been added."

“dotnet build” throws an error when project.lock.json contains assemblies with the same name in different packages. While this seems logical on one side, it is quite hard to debug these issues.

My biggest concern is that this is a common problem in Microsoft packages as well. Packages like “Microsoft.Net.Http” and “System.Net.Http” contain assemblies with the same name. AFAIK people should move to the newer “System.Net.Http” package but I’m afraid that’s not always under our control. Some 3rd party libraries might still use “Microsoft.Net.Http” and others might already use “System.Net.Http”.

Steps to reproduce

These are two sample projects that fail:

{
  "version": "1.0.0-*",
  "compilationOptions": {
    "warningsAsErrors": true
  },

  "dependencies": {
    "Microsoft.Bcl.Immutable": "1.0.34",
    "Microsoft.AspNet.Mvc": "6.0.0-rc1-final"
  },

  "frameworks": {
    "net46": { }
  }
}

Error: An item with the same key has already been added. Key: (System.Collections.Immutable, neutral, b03f5f7f11d50a3a)

{
  "version": "1.0.0-*",
  "compilationOptions": {
    "warningsAsErrors": true
  },

  "dependencies": {
    "Hyak.Common": "1.1.0",
    "System.Net.Http.Formatting.Extension": "5.2.3"
  },

  "frameworks": {
    "net46": { }
  }
}

Error: An item with the same key has already been added. Key: (System.Net.Http.Extensions, neutral, b03f5f7f11d50a3a)

Expected behavior

I’m not sure if these are exactly the same libraries and a binding redirect would solve the problem. I’ve seen that there are already some other issues (eg dotnet/sdk#4805 ) on that topic.

If it can’t be solved with a binding redirect, maybe it can be resolved by renaming the libraries?

If this can’t be fixed, how should we proceed in such cases? If the vendor of the library doesn’t change his dependencies we might end up in a blocking situation.

Environment data

dotnet --version output:

.NET Command Line Tools (1.0.0-beta-001718)

Product Information:
 Version:     1.0.0-beta-001718
 Commit Sha:  5b0afe7880

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.10586
 OS Platform: Windows
 Runtime Id:  win10-x86

About this issue

Original URL
State: closed
Created 8 years ago
Reactions: 2
Comments: 27 (3 by maintainers)

Commits related to this issue

Search SDK: Fixing Timeout and removing unneeded projects from the solution Fixes #2604 Now when you call searchClient.Indexex.GetClient(), it will propagate the Timeout value set on the searchClien... — committed to brjohnstmsft/azure-sdk-for-net by brjohnstmsft 7 years ago
Merge pull request #1860 from tannergooding/master Updating SignToolData to include the produced nuget packages — committed to wli3/cli by tannergooding 6 years ago

Most upvoted comments

I found another solution that seems to fix this problem, do a:

dotnet restore --no-cache

+23

SergeySagan on Nov 16, 2016

I start hitting this on VS Update 3.

D:\GitHub\azure-sdk-for-net\tools>dotnet --info
.NET Command Line Tools (1.0.0-preview1-002702)

Product Information:
 Version:     1.0.0-preview1-002702
 Commit Sha:  6cde21225e

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.10586
 OS Platform: Windows
 RID:         win10-x64

D:\GitHub\azure-sdk-for-net\tools>NuGet.exe update -self
Checking for updates from https://www.nuget.org/api/v2/.
Currently running NuGet.exe 3.5.0-beta2-1456.
NuGet.exe is up to date.

How to repro:

clone https://github.com/hovsepm/azure-sdk-for-net/
switch to AutoRest branch
open src\ResourceManagement\Authorization\Authorization.sln
build

Actual: Error An item with the same key has already been added. Key: Microsoft.Azure.Test.HttpRecorder TestFramework C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\DotNet\Microsoft.DotNet.Common.Targets 241

Expected: Build successful.

+15

hovsepm on Jun 21, 2016

Any progress on this? This is blocking me in a rather bad way at the moment.

+13

fearthecowboy on Mar 17, 2016

I’m having this issue as well. My current workaround has been to delete project.lock.json and rebuild the project.

andyjansson on Sep 20, 2016

You must have not have deleted all of the lock files (as it is the root src of the issue, deleting the files is the solution).

Make sure Visual Studio is closed then browse to the solution directory and do a search for *.lock.json and delete ALL the files that come up (may take a minute depending on sln size). Now when you open the project Visual Studio will rebuild all those files.

ByronAP on Dec 12, 2016

@Serjster Exactly! I was just about to write the same.

The project.lock.json generated with dotnet restore works. This lets the project compile in VS, emitting no "same key has already been added" errors.
The project.lock.json generated with Visual Studio 2015 Update 3 (e.g. after deleting the project.lock.json file manually) does not work. This will emit the "same key has already been added." errors.

For the AWS .NET SDK the projects.lock.json generated by dotnet restore is 140kB smaller than the one generated by Visual Studio.

I’ve attached both files here, maybe somebody can quickly spot the reason why this happens.

Anybody knows how VS is creating the lock file differently than the cli? Shouldn’t it use the same cli command?

/Edit: Visual Studio is issuing dotnet restore someRestoreDgFile.dg. In my case, it’s "C:\Program Files\dotnet\dotnet.exe" restore "e:\aws-sdk-net\sdk\.vs\restore.dg. Exactly this last file parameter is the reason for dotnet restore creating the bigger project.lock.json file. Without the dg file parameter, the project.lock.json is smaller and presumably without duplicates.

The .dg file is a reference mapping file for project-to-project references and it gets created by Visual Studio.

derFunk on Oct 17, 2016

@davidfowl Besides having the same issue @xatabhk has, I also can’t build using visual studio. I am able to build some libraries using dotnet build, but building the same library from inside visual studio gives a plethora of errors mostly of the format

The type 'IList<>' is defined in an assembly that is not referenced. You must add a reference to assembly 'System.Runtime, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.

Where IList<> can be object or other basic classes. Might this be the same issue?

I have added an issue report to the aspnet/mvc repository

MariovanZeist on Mar 17, 2016

Same issue here. Whenever I add a nuget package to the solution, or add a reference from one project to another, multiple projects stop building with this error. The fix is to delete project.lock.json in those projects, but it’s annoying to have to keep doing it all the time. Using VS 2015 update3 and tools 1.0.0-preview2-final.

andrewslavin on Dec 14, 2016

why is M$ not promoting or pushing preview3 anywhere? REF: https://www.microsoft.com/net/core#windowsvs2015 which specs to preview2 and VS update3 which specs to preview2 as well.

Even the latest release is Preview 2-1-003177 I only see 1 ref to a 3.* build in releases

ByronAP on Dec 10, 2016

one thing you can try doing is to clear out your nuget cache… this seems to have solved my problem… at least for now.

Download nuget.exe: https://github.com/NuGet/Home/releases
nuget.exe locals all -clear from the folder where you downloaded above (I put mine in C:\Program Files (x86)\NuGet\Visual Studio 2015 )
Run dotnet restore in your VS Package Manager Console

SergeySagan on Nov 16, 2016

Why is this bug closed? Like louislewis2, I still experience this bug. Only workaround working was the dotnet restore --no-cache suggested by Serjster. But that is a workaround … please fix this annoying bug!

gimlichael on Nov 13, 2016

The only solution that has worked for me to remove these annoying errors is to remove solution project references from the project in the error, as in go to Solution Explorer, right click on References in the project specified in the error, click on Add Reference, then uncheck all checkboxes in the Projects/Solution. Then ‘dotnet restore’ in the ‘Package Manager Console’. After that’s done, go back and readd those references the same way you removed them. Rebuild the solution. That seems to fix the errors, but is super annoying.

SergeySagan on Oct 5, 2016

@davidfowl Is there a workaround for now?

flagbug on Mar 16, 2016

I am also having the same issue, also have all the latest updates installed.

louislewis2 on Oct 21, 2016

Is this fixed? I’m hitting the same issue as @hovsepm mentioned in the above post.

dragonfly91 on Aug 10, 2016