roslyn: editorconfig c# language rules doesn't trigger build failure

upvote! We added a .editorconfig file to our solution and added some settings to compile as errors to find legacy code that needed to be updated. When compiled the build succeeded with no errors.

Currently IDE code style rules do not apply during builds. We are actively working to make these analyzer available during compilation scenarios, at which point errors would cause the build to fail.

CodeStyle analyzer assemblies are now directly inserted into the .NET5 SDK (will need VS2019 16.8 Preview4 or later and .NET5 RC2 SDK or later, once both of these are released). You will not need to install any special analyzer NuGet package to enable code style enforcement in build if you are on the required .NET SDK and VS. Also note that you don’t need your project TargetFramework to be net5 or later to enable this feature, you just need to be on the .NET5 SDK.

Official documentation with the steps to enable build time code style enforcement is at https://docs.microsoft.com/dotnet/fundamentals/productivity/code-analysis#code-style-analysis.

I am going to close this issue as no further feature work is required, please file separate Rolsyn issue(s) for anything that does not work as expected and doc issues at the above page if you need more clarity on documentation.

@chucker You have 2 options to enforce code style analyzers on build:

1. Set MSBuild property EnforceCodeStyleInBuild to true in your project file OR
2. Add a package reference to Microsoft.CodeAnalysis.CSharp.CodeStyle. With package reference, you need to ensure that you use the required minimum version required by the package. For example, version 3.8.0 requires at least VS2019 16.8.

@ElteHupkes That comment is outdated. Starting with the 16.3 release, can now set the severity of IDE0055 (or any other rule) via a .editorconfig file.

# IDE0055: Fix formatting
dotnet_diagnostic.IDE0055.severity = error

Hey folks, any progress on this? This would be extremely useful

@mavasani I can’t get this to work either with the latest Visual Studio version (16.4.2) and I don’t see anything particularly in the documentation you linked. Could you clarify what the current state is?

Having static analyzers that only work when you have a file open makes them rather pointless since you get the biggest benefits from automatically checking these things during CI.

EDIT: For people arriving in the future, before doing anything with this comment, be sure to read the one directly below it.

@klausenbusk A comment under a related issue indicated that you also need a .ruleset file, and with that I actually got this to work for code formatting. So to summarize:

• Create a .editorconfig with the rules you’d like
• Create a CodeFormatting.ruleset in your project (name is arbitrary). My contents are:

<RuleSet Name="CodeAnalysis Ruleset" Description="This ruleset sets the severity for the Microsot.CodeAnalysis.CSharp.CodeStyle package, so it breaks on build" ToolsVersion="10.0">
    <!-- Even though the IDE will correctly show errors for rule violations in .editorconfig,
         we need to explicitly set the severity in a ruleset file to get build errors for it:
         https://github.com/dotnet/roslyn/issues/30541 -->
    <!-- It doesn't really seem to matter what I put in the AnalyzerId and RuleNamespace fields,
         as long as the rule ID is correct. -->
    <Rules AnalyzerId="Microsoft.CodeAnalysis.CSharp.CodeStyle" RuleNamespace="Microsoft.CodeQuality.Analyzers">
        <!-- IDE0055 is the formatting analyzer from Microsoft.CodeAnalysis.CSharp.CodeStyle. -->
        <Rule Id="IDE0055" Action="Error" />
    </Rules>
</RuleSet>

• Add it to your .csproj file like:

<PropertyGroup>
    <CodeAnalysisRuleSet>CodeFormatting.ruleset</CodeAnalysisRuleSet>
</PropertyGroup>

With that, my build has started to fail on code formatting errors. You could of course also configure it as Action="Warning" to just throw a warning. For showing these errors / warnings in the build it seems to respect the severity level in .editorconfig, so say you had:

csharp_new_line_before_open_brace = all:hint

Then it would not affect the build.

EDIT: By that I mean, it only shows the error level defined in the .ruleset for the properties configured as errors. If you have Action="Error" with a prop=value:warning it won’t show up as a warning. Not ideal but somewhat workable for things you really don’t want in your build.

These are the package versions I have installed:

<PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="3.7.0" />
<PackageReference Include="Microsoft.CodeQuality.Analyzers" Version="3.0.0"/>
<PackageReference Include="Microsoft.CodeAnalysis.FxCopAnalyzers" Version="3.0.0" />

And my .editorconfig has

[*.cs]

# IDE0051: Remove unused private members
dotnet_diagnostic.IDE0051.severity = error

Yet unused private members do not trigger build failures. Have I missed a step?

(I even tried creating a brand new console project with latest versions of everything and the same behaviour is observed)

I also tried this from package manager:

PM> Install-Package Microsoft.CodeAnalysis.CSharp.CodeStyle -Version 3.8.0-1.20330.5 -Source https://dotnet.myget.org/F/roslyn/api/v3/index.json
The 'Source' parameter is not respected for the transitive package management based project(s) ATest. The enabled sources in your NuGet configuration will be used. 
Restoring packages for C:\Dev\ATest\ATest.csproj...
  GET https://api.nuget.org/v3-flatcontainer/microsoft.codeanalysis.csharp.codestyle/index.json
  NotFound https://api.nuget.org/v3-flatcontainer/microsoft.codeanalysis.csharp.codestyle/index.json 889ms
Install-Package : NU1101: Unable to find package Microsoft.CodeAnalysis.CSharp.CodeStyle. No packages 
exist with this id in source(s): Microsoft Visual Studio Offline Packages, nuget.org

This appears to succeed:

dotnet add package Microsoft.CodeAnalysis.CSharp.CodeStyle --version 3.8.0-1.20330.5 --source https://dotnet.myget.org/F/roslyn/api/v3/index.json

But then fails to build with a large number of errors:

CSC : error CS8032: An instance of analyzer Microsoft.CodeAnalysis.UseSystemHashCode.UseSystemHashCodeDiagnosticAnalyzer cannot be created from C:\dev\.nuget\packages\microsoft.codeanalysis.csharp.codestyle\3.8.0-1.20330.5\analyzers\dotnet\cs\Microsoft.CodeAnalysis.CodeStyle.dll : Could not load file or assembly 'Microsoft.CodeAnalysis, Version=3.6.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'. The system cannot find the file specified..

BUT…

Using version 3.7.0-1.20208.4 did actually work!

Is there a timeline for having this published on nuget.org?

Is there a complete list of IDEXXX rules

You can specify the following entries to escalate all code style rules in the package to default to a warning and that should enable all of them for build.

[*.cs]
dotnet_analyzer_diagnostic.category-Style.severity = warning
dotnet_analyzer_diagnostic.category-CodeQuality.severity = warning

You can then selectively downgrade specific rules to a lower severity (none/silent/suggestion) as per your preference. For example, see https://github.com/dotnet/roslyn-analyzers/blob/master/.editorconfig#L185-L239

Update:

1. We have ported nearly half of the IDE analyzers to a pre-release C# CodeStyle NuGet package (VB one is here).
2. Adding a NuGet package reference to the above will ensure the ported IDE/editorconfig rules are enforced on build/CI. NOTE: You need to be on VS2019 16.3 or later for the NuGet package to work.
3. Work is in progress to port the remainder of the rules within the next few weeks.

@sharwell - do you think we are good to close this issue or should we wait till all rules are ported?

@jmarolf I believe @kindermannhubert is requesting that we move all the in-built IDE analyzers into a separate assembly which can be consumed via a NuGet package, just like regular third party analyzers. Currently, the analyzers in the IDE layer (i.e. MS.CA.Features.dll) have lot of internal dependencies, primary ones being the options infrastructure which is tied to the Workspace. Hence these cannot be consumed through any NuGet package or even referencing MS.CA.Features.dll directly as an Analyzer item group in your project. @sharwell has created a separate CodeStyle project and NuGet package within Roslyn.sln, which currently only has the formatting analyzer, but is consumable as an analyzer NuGet package. He is also working on porting individual IDE style analyzers into this package, so then can be consumed as a NuGet package and enabled in CI - .editorconfig support in compiler is a critical step, and now that it is in place, we hope to start tackling one analyzer at a time to move it into this package.

In my case with .net5 I was forced to remove Microsoft.CodeAnalysis.CSharp.CodeStyle and enable EnforceCodeStyleInBuild. Wit net3.1 the package worked. Thanks all!

@mavasani I see now. I had the following in my .editorconfig file:

csharp_prefer_braces = true : error

which showed the IDE0011 error in my Error List so I assumed the build would fail. After adding the following:

dotnet_diagnostic.IDE0011.severity = error

the build does fail.

Thanks for explaining.

@RehanSaeed Setting default severity for all analyzer diagnostics is documented here: https://docs.microsoft.com/dotnet/fundamentals/code-analysis/configuration-options#scope

Ah ok, I had considered trying to upgrade VS, good to know. When this is published on nuget.org will it require the latest VS?

However, what is not clear is which part of Microsoft’s recommended code quality analyzers i.e. Microsoft.CodeAnalysis.FxCopAnalyzers integrate with the build.

All analyzers shipped in Microsoft.CodeAnalysis.FxCopAnalyzers work at build time.

Errors only seem to make themselves known some of the time whilst coding, many of them do not show at build time

Would you be able to identify a standalone repro and open a separate issue with actual and expected behaviors? Seems like you are hitting a potential bug in how editorconfig based severity configuration is affecting diagnostics in IDE typing and/or build time. It is pretty likely not an issue with the analyzer package itself, as the analyzer packages do not control the final severity of reported diagnostics, whether or not they execute at different analysis phases in IDE/command line, etc.

In a previous role we used Microsoft.CodeAnalysis.FxCopAnalyzers but with ruleset configuration and we had errors working at design time and build time reliably.

This strongly suggests you are hitting some bug in editorconfig based severity configuration support added to the Roslyn compiler and/or IDE. I would suggest filing an issue.

@RehanSaeed I don’t see any dotnet_diagnostic.ruleId.severity entries in your .editorconfig, so none of your code style rules will execute on CI/command line build, they will only run in the IDE live analysis. The core issue is explained in https://github.com/dotnet/roslyn/issues/44201. We are trying to work out a solution where users do not have to manually map code styles to IDExxxx diagnostics and add dotnet_diagnostic entries for those to get build enforcement from this NuGet package. Until then, you would need to add these to your .editorconfig to get build enforcement.

You can also try the .editorconfig starter project that I created. All rules are configured:

https://github.com/RehanSaeed/EditorConfig

I just played around with the code style analyzers (Microsoft.CodeAnalysis.CSharp.CodeStyle) and it seems it is only working with a configured ruleset file. Does this make sense?

No, the rules work fine with .editorconfig as well. However, you will need to explicitly set severity of each IDExxxx rule ID associated with a code style by using the dotnet_diagnostic.ruleId.severity = warning format entry as explained here: https://docs.microsoft.com/en-us/visualstudio/code-quality/use-roslyn-analyzers?view=vs-2019#set-rule-severity-in-an-editorconfig-file. We understand it is a tedious task to map code styles to IDExxxx diagnostic IDs, so we are planning to implement a UI that will do this automatically for you: https://github.com/dotnet/roslyn/issues/40365#issuecomment-657685800. Tagging @mikadumont @jmarolf

Will it work with VSCode?

The analyzer package is identical to any other third party analyzer package. It should work wherever you can run third party analyzers.

There are some design issues that need to be tackled to lay the foundation before the analyzers can be moved to a NuGet package: https://github.com/dotnet/roslyn/issues/38480