msbuild: Wildcard expansion is silently disabled when a glob expansion encounters an I/O exception
This was reported as a Connect issue.
The repro is simple:
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<MyItem Include="MyDir\**\*.*" Exclude="MyDir\node_modules\**\*.*;MyDir\tmp\**\*.*" />
</ItemGroup>
<Target Name="TestInputs">
<Warning Text="Inputs = @(MyItem)" />
</Target>
</Project>
Where there are files in MyDir\node_modules that exceed MAX_PATH. That returns:
C:\Users\raines\Downloads>msbuild test.proj
Microsoft (R) Build Engine version 14.0.24720.0
Copyright (C) Microsoft Corporation. All rights reserved.
Build started 12/16/2015 12:46:15 PM.
Project "C:\Users\raines\Downloads\test.proj" on node 1 (default targets).
C:\Users\raines\Downloads\test.proj(8,3): warning : Inputs = MyDir\**\*.*
Done Building Project "C:\Users\raines\Downloads\test.proj" (default targets).
Build succeeded.
Note that the wildcards weren’t actually expanded.
The original issue mentions an expectation that the Exclude element would prevent this from happening because that directory and all its contents are excluded. The reason that doesn’t work is because we build the Include list fully first, then build the Exclude list, then do a subtraction. The failure occurs when building the Include list, so Exclude has no effect.
About this issue
- Original URL
- State: open
- Created 9 years ago
- Reactions: 9
- Comments: 34 (14 by maintainers)
Commits related to this issue
- Use newer npm on AppVeyor (as a workaround for https://github.com/dotnet/cli/issues/6561 and https://github.com/Microsoft/msbuild/issues/406), and run correct test command — committed to reactjs/React.NET by Daniel15 7 years ago
- Upgrade to Visual Studio 2017 + csproj tooling (#406) * Upgrade to Visual Studio 2017 + csproj tooling * Include workaround for https://github.com/NuGet/Home/issues/4337 (explicitly specify versio... — committed to reactjs/React.NET by Daniel15 7 years ago
- Exclude node_modules This fixes building within VS 2017 community. node_modules can sometimes contain paths that are very long, which can trigger an msbuild bug https://github.com/Microsoft/msbuild/... — committed to dustinsoftware/React.NET by dustinsoftware 7 years ago
- Exclude node_modules (#442) This fixes building within VS 2017 community. node_modules can sometimes contain paths that are very long, which can trigger an msbuild bug https://github.com/Microso... — committed to reactjs/React.NET by dustinsoftware 7 years ago
- Update dependencies from https://github.com/dotnet/toolset build 20200331.1 (#406) - Microsoft.Dotnet.Toolset.Internal - 3.1.202-servicing.20181.1 Dependency coherency updates - Microsoft.DotNet.Cl... — committed to radical/msbuild by dotnet-maestro[bot] 4 years ago
- Upgrade to Visual Studio 2017 + csproj tooling (#406) * Upgrade to Visual Studio 2017 + csproj tooling * Include workaround for https://github.com/NuGet/Home/issues/4337 (explicitly specify versio... — committed to toptaldev92/React.NET by Daniel15 7 years ago
- Exclude node_modules (#442) This fixes building within VS 2017 community. node_modules can sometimes contain paths that are very long, which can trigger an msbuild bug https://github.com/Microso... — committed to toptaldev92/React.NET by toptaldev92 7 years ago
Yes, in any case where the long-path support works, which is still not everywhere.
This issue is still open because any other I/O exception still triggers the silent assume-they-meant-a-literal-string-with-
*-in-it behavior. I’ll rename for clarity.If MSBuild can detect when an item group evaluation has reached MAX_PATH, at least log a normal or diagnostic log saying “ItemGroup evaluation for X reached MAX_PATH at C:\longpath… and will be discarded” so that the user sees this is happening to them.
We ran into this issue internally due to a misbehaving program that generated a very long path which then causes MSBuild to suddenly stop building inexplicably because it couldn’t expand a dirs.proj with an
<ProjectReferences Include="src/**/*.csproj">. I wish I could get back a day of trying to figure why my build suddenly broke 😃i am totally blocked by this. I am also on osx and having the same results as @RehanSaeed looks like msbuild / dotnet core are not actually ready for unix 😕
@Daniel15 Thanks for the suggestion. Just tried that and this is what I found:
The longest path from the project causing the build error is 110 characters:
The longest path from another project in the solution is 118 characters:
These paths are well short of the 260 character max path, unless I’m missing something about Linux or MacOS where I’m running the
dotnet buildcommand.Let’s reboot this issue given there is support beyond Win32 MAX PATH now in .NET
cc @jaredpar
@RehanSaeed for what its worth, it seems like some issue with the hidden keys folder thats generated when launching debug session.
to solve this: i created a keys folder (not hidden), and then within that a file with the same filename as the one that had been generated in the hidden .keys folder. i left the file empty.
now every time i build / launch it works fine…
@shift-evgeny
Excludeshould now behave as you expect, thanks to improvements made for lazy item evaluation (#770). MSBuild now considers the exclude list while recursing through the include list and will stop when it hits an excluded directory.This should already be available in VS “15” Preview 5.
I’m not closing this issue because the bad doesn’t-expand-wildcards behavior persists if you don’t have an exclude. But at least the obvious workaround for
node_modulesworks now.