clang-power-tools: Command Line too long

The command line generated appears to exceed a threshold

Perhaps it would be better to generate a command file, to avoid command length limitations

The command line is too long.
Write-Err : Cannot bind argument to parameter 'msg' because it is an empty string.
At C:\Users\GeorgeRu\AppData\Local\Microsoft\VisualStudio\15.0_13f2e2ac\Extensions\43s1b10b.dwz\clang-build.ps1:567 
char:15
+     Write-Err ($compileResult.Output)
+               ~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (:) [Write-Err], ParameterBindingValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationErrorEmptyStringNotAllowed,Write-Err

About this issue

Original URL
State: closed
Created 6 years ago
Comments: 19 (6 by maintainers)

Most upvoted comments

Hello guys,

I had some time and looked into why Tidy does not work. The mentioned “YAML” error indeed means, Tidy does not treat the “–config” option the same way as Clang does.

To put it short: for a Clang run, we can put the whole command line into a file and call Clang with “–config filename” For a Tidy run, we have a command line like this: clang-tidy.exe tidy-params – params-passed-from-tidy-to-clang

So the command line includes the Tidy params as well as the params passed on from Tidy to Clang, separated by “–”.

You guessed the solution yet? Split the Tidy command line at the “–” separator and pass the left side to Tidy, and pass the right side as in a file to Clang.

I have tested this and it works for me with LLVM6, VS2017 with Clang and Tidy runs.

To patch your installation, please:

find and backup the clang-build.ps1 file
find the Run-ClangJobs function
find the code block which starts with: if ($aUseParallelCompile)
Copy-paste this code BEFORE the above block:

    $tmp = [System.IO.Path]::GetTempFileName()
    Write-Verbose "Params saved to $tmp"

    if($job.FilePath -like '*tidy*')
    {
      #Tidy: split parameters: tidy params and clang params
      $splitparams = $job.ArgumentList -split "--"
      $splitparams[1] -replace '\\', '/' | Out-File $tmp
      #pass clang params as file
      $job.ArgumentList = ($splitparams[0] + " -- --config $tmp")
    }
    else
    {
      #Clang: save options to config file
      $job.ArgumentList -replace '\\', '/' | Out-File $tmp
      $job.ArgumentList = "--config $tmp"
    }

Now you should be able to run Clang compile and Tidy using the PowerTools Plugin.

I am not a PowerShell programmer so maybe it is not elegant, not optimal, but it works. It checks if we are doing a Clang or a Tidy run. For a Clang run, the command line is simply saved. For a Tidy run, the string will be split and the left side goes to Tidy, the rest to Clang.

I hope the developers will look into this and give us a fix soon. Have fun guys.

ghost on Jul 16, 2018

Excellent solution, @dzolee.

At first, we were skeptical of this approach, since it means creating and deleting thousands of temporary files, one per cpp. We don’t want to slow down Clang Power Tools and we also want to avoid having two (or more) ways of doing things. So we decided to test for it…

We ran a multitude of compilation tests on our test repository, consisting of around 140 projects and 8000 CPP files. To our surprise, we found that the temporary-file creation and deletion additional cost is practically zero.

In the majority of tests the compilation was slightly faster (around 1%) than the baseline of the control group. There appears to be a tiny time cost associated with a long command line, perhaps from CMD.EXE command line parsing. The tests were ran on the same machine, in identical conditions, on both SSDs and HDDs, yielding similar results.

We are very pleased with the results and are happy to inform you that starting with Clang Power Tools v3.8 (i.e. next release) we’ll move to using config files for giving arguments to clang.

Thank you all for your very insightful and helpful feedback.

hero101111 on Jul 23, 2018

Sadly, we don’t currently have a working solution for this. As @grrussel pointed out, cmd.exe invocation rules dictate a 8192 character limit for the command-line. This applies to powershell.exe invocations as well.

A command file would likely not help us since it is bound to the same 8192 character limitation. That cmd file would still need to invoke clang++.exe which would mean the same thing.

As a workaround, we could explore invoking the CreateProcess() Wi32 API function manually from our Powershell code. MSDN documents a 32768 commandline character limit for that. That would put us at a comfortable 4x above the current limit. Still not a general solution to the problem.

We welcome any ideas on how to overcome this basic limitation.

hero101111 on Jul 11, 2018

Hello there I also have this problem, it is unfortunately preventing me from using the VS plugin.

I have a relatively big VS solution, with ~600 projects, and many projects are dependent on others. I used to compile single files with the tool, for the syntax only.

Can you please come up with a solution? Like the other commenter said, move the parameters into a file?

thanks

ghost on Jul 11, 2018