sdk: GenerateDepsFile: The process cannot access the file '...\MyProject.deps.json' because it is being used by another process.
UPDATE: While writing this, rather long, post – I’ve managed to produce a small reproduction example, see bottom
We’ve recently upgraded our builds to run on faster hardware with more cpu cores, and are now ~100% of the time seeing the below error in a number of repositories. The error here is from Windows, but our CI servers run on linux and have the same error (see logs below).
We run the following commandlines:
“C:\Program Files\dotnet\dotnet.exe” restore C:\Project\MyProject.sln “C:\Program Files\dotnet\dotnet.exe” build C:\Project\MyProject.sln --configuration Debug --framework netstandard2.0 --no-restore
C:\Program Files\dotnet\sdk\2.2.101\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(129,5): error MSB4018
The "GenerateDepsFile" task failed unexpectedly. [C:\Project\MyProject.csproj]
System.IO.IOException: The process cannot access the file 'C:\Project\bin\Debug\netstandard2.0\MyProject.deps.json' because it is being used by another process. [C:\Project\MyProject.csproj]
at System.IO.FileStream.ValidateFileHandle(SafeFileHandle fileHandle) [C:\Project\MyProject.csproj]
at System.IO.FileStream.CreateFileOpenHandle(FileMode mode, FileShare share, FileOptions options) [C:\Project\MyProject.csproj]
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options) [C:\Project\MyProject.csproj]
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize) [C:\Project\MyProject.csproj]
at System.IO.File.Create(String path) [C:\Project\MyProject.csproj]
at Microsoft.NET.Build.Tasks.GenerateDepsFile.ExecuteCore() [C:\Project\MyProject.csproj]
at Microsoft.NET.Build.Tasks.TaskBase.Execute() [C:\Project\MyProject.csproj]
at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [C:\Project\MyProject.csproj]
at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [C:\Project\MyProject.csproj]
Observations:
- This fails on my local Windows desktop roughly 33% of the time. Rerunning the build (no cleanup) always succeeds (seems to be a race condition, so HW speeds play a role). On our Linux CI servers, it fails 100% of the time currently
- The deps file in question is always the same file for a given repository / builds. The file is for a shared library between multiple projects.
- A plain
dotnet build MyProject.sln -f netstandard2.0(no build script) also exhibits this issue. - A
dotnet build MyProject.slnseemingly always succeeds (locally and on CI server), at least for three consecutive attempts right now. - All the projects in the
MyProject.slnhavenetstandard2.0- but a number of them have more targets (ie. netcoreapp) - ~We have more projects, f.ex. for tests, but they’re in
MyProject-Tests.sln. These projects only targetnetcoreapp2.x- we’ve split the solutions both for speed when developing and to be able to build artifacts without compiling the tests (since we can’t build certain projects from the solution only … that’s another issue entirely)~ (not relevant) - Limiting the build to one process (msbuild:
/m:1) seems to “solve” the problem. - At work (inaccessible currently), I performed some Procmon captures to see the difference between successful and failing runs
- It turns out, that two
dotnet.exeprocesses attempt to create the.deps.jsonfiles - When succeeding, there is a clear seperation between two CreateFile() calls, where the first creates the file, and the second call (from the second PID) finds the file already there and does nothing (not even reads it)
- When failing, the two processes make calls overlapping each other, leading to both of them discovering the file as missing, and both trying to create it (one obviously failing, producing the stacktrace above) – classical concurrency issue.
- It turns out, that two
So.
- It should be that when two projects reference a third, shared, project … that this shared one is built exactly once … right?.. How come two attempts are made at writing the same
.deps.jsonfiles? - How come this seemingly works fine, when building for all target frameworks?
Versions:
- Local, Windows 10 x64,
dotnet 2.2.101 - Linux CI, docker,
dotnet 2.2.103(we build our own docker images using the dockerfile from the basemicrosoft/dotnet:2-sdkimages)
Reproduction
Project: ci-stresstest-master.zip
In it, I have a solution with 5 projects, all empty (no actual code), but with a targeting setup that mimicks a very small subset of our projects. I’ve then created four seperate CI builds that did the following:
- log (succeeds)
dotnet restore ci-stresstest.sln+dotnet build ci-stresstest.sln --no-restore - log (succeeds)
dotnet build ci-stresstest.sln - log (fails)
dotnet restore ci-stresstest.sln+dotnet build ci-stresstest.sln -f netstandard2.0 --no-restore - log (fails)
dotnet build ci-stresstest.sln -f netstandard2.0
Notes on repro.
- We run a vanilla docker setup for CI, except for one thing:
- We mount a shared directory for nuget packages, to be able to share downloaded packages between builds
See also #2076
About this issue
- Original URL
- State: open
- Created 5 years ago
- Reactions: 28
- Comments: 23 (6 by maintainers)
Commits related to this issue
- Update dependencies from https://github.com/dotnet/cli build 20190915.1 (#2902) - Microsoft.DotNet.Cli.Runtime - 5.0.100-alpha1.19465.1 — committed to dotnet/sdk by dotnet-maestro[bot] 5 years ago
- Remove .sln files after cloning dependencies We are only using the project (.csproj) themselves. Apparently TargetFramework definitions in solution files can cause problems while publishing projects:... — committed to mabevtech/keycloak-poc by mabevtech 2 years ago
- Removed .NET Core 3.1 from anvil-csharp-logging frameworks This branch is currently seeing intermittent build errors, which seem to be related to concurrent framework builds (see https://github.com/d... — committed to tjvezina/anvil-csharp-core by tjvezina 2 years ago
After switching/adding on TargetFrameworks (
net6.0;net7.0),msbuildimmediately starting giving file access concurency warnings with retries, and frequently started failing entirely. (similar symptoms stated above)In this instance we only had a single solution file, and single project, targeting >1 SDK for build.
The only workaround that completely resolved the issue was to drop down CPU count to 1 to disable any parallel execution.
^ after doing this, ALL warnings were immediately eliminated and build hasn’t failed since
I am getting this error if I simply call
dotnet build Main.slnwithout specifying the target framework. So I can’t believe in this “core problem” explanation.@LordMike Thank you for mentioning this! There seems to be a problem with the
--frameworkparameter when the projects just target a single framework version. This is still happening for me as of .NET 7.Works (with
<TargetFrameworks>net6.0;net7.0</TargetFrameworks>in the CSPROJs):dotnet build --no-restore -c Release -f net7.0 Solution.slnWorks (with<TargetFramework>net7.0</TargetFramework>in the CSPROJs):dotnet build --no-restore -c Release Solution.slnDOES NOT WORK (with<TargetFramework>net7.0</TargetFramework>in the CSPROJs):dotnet build --no-restore -c Release -f net7.0 Solution.slnI’ve started facing this issue on CI (TeamCity) after introducing Source Generators to my project.
The workaround for me was to add
dotnet restorestep beforedotnet build. Sometimes there is still some file access issue duringdotnet build, but it gets retried and the build passes:So clearly there is some race condition with file access.
I’m currently facing this issue with the latest net6.0 sdk in the following build scenario:
dotnet build xxx.sln /p:Configuration=Debug --framework netstandard2.1The error occurs (on the first netstandard-only project) in ~30% of all builds.
FWIW, I’ve experienced similar issue with
GenerateDepsFilephase using .net sdk 3.1.301 - in my case one way to mitigate the issue is to explicitly build the project in question first (usingdotnet build) before building projects that depend on it.For our current use case, we wanted to optimize our builds a bit, because we’re targeting two frameworks to allow our developers to test the modules they’re working (they have a small program.cs, and a shared library to actually run code – only available to them when in netcoreapp2.x).
So our builds on the CI would be building for both coreapp and netstandard. Targeting just netstandard (since everything in the slution supports that anyways) did this and shaved a few seconds of our builds.
We’ve workarounded by not targeting the specific framework, since we found that it was just a few seconds…
However. I’m not at all sure why this is an issue… All of the projects in question target netstandard2.0…
While trying to reproduce a failure on Windows with Procmon, I’m met with this. I’ve noticed sporadic errors like this before, but never enough of them to make an impression. Given the above, I think there might be a lot of concurrency issues in msbuild/dotnet build …
Other locking/access errors I’ve seen:
The process cannot access the file ‘…\Shared02.dll’ because it is being used by another process.
git clean -xdf) between builds produces this roughly once every two builds.Log
Could not read state file “obj\…\TopLib01.csprojAssemblyReference.cache”. The process cannot access the file ‘…\TopLib01.csprojAssemblyReference.cache’ because it is being used by another process.
Log
Currently I am thinking about workaround to simply change build script part from something like
To something like
There can be more tries if needed. I already did that trick with MSBuild in year 2013 or 2014 when we were unable to get reliable build of website project (ASP.NET). We had 4 tries back then because 2 tries were not enough. P.S. The problem with website project was different.
We’ve run into another similar (but not the exact same) race condition:
I’m not sure if this is caused by a bug in the build system or if there is somehow a conflict in our projects that could be resolved by fixing the project definition. We use standard project layouts with simple SDK style csproj projects.
In the latter case, it would be amazing if the build system could detect the sources of these conflicts and warn or error on them.
Hi,
I’ve a case where there is no specifying of TargetFramework at the solution level where the
dotnet buildbehaves similar in some cases where TargetFramework at the project level refers tonetstandard2.0.This behavior has just emerged without any notice in
dotnet core 2.0. I’ve also tried to upgrade todotnet core 2.1, but in vein.In my case, there seems to be a race condition from my hardware setup, due to the
dotnet buildleaves few spawneddotnetprocesses.Thank you.
Regards Henrik
The core problem here is that you’re specifying a TargetFramework at the solution level. As you note, it works without that, while building all of the TargetFrameworks specified at the individual project level.
We hope to improve the experience around this – at least providing an informative error, but maybe also doing something smarter. But at the moment, I’m not quite sure what that would be.
Can you describe why you’re specifying a TargetFramework and what you expect to happen in that build, as distinct from the no-TF-specified solution build?