vscode-cmake-tools: Can't run CTest tests in parallel
Brief Issue Summary
Not sure if this is some project misconfiguration on my side, but CTest tests are not run in parallel. For every test <test_name> in my project, I see this in my output tab:
[proc] Executing command: /opt/homebrew/bin/ctest -j10 -C Debug -T test --output-on-failure -R <test_name>
While it actually goes through the motions of running in parallel (by passing -j10), because it filters the list of tests down to a single one (since <test_name> is the name of one single, specific test), there is no opportunity to actually run in parallel.
I get this behavior by running it from the “Testing” pane in the left-side of the VS Code screen: either clicking the button whose tooltip is “Run Tests” and runs all tests, clicking on the “play”-shaped button next to CTest or the same button next to my project folder under CTest. The behavior is also the same if I choose “CMake: Run Tests” or “Test: Run All Tests” from the command palette.
I can’t see how this could be the intended behavior, as there’s even a setting “Cmake > Ctest: Parallel Jobs” in the extension – which BTW is set to 0 in my case, and so is “Cmake: Parallel Jobs”; indeed you can see above I have a -j10 in my CTest command line. Could I get some pointers about what I might be doing wrong, perhaps a setting or something in my CMakeLists.txt that is blocking parallel tests? If by chance this is intended behavior, I’d like to turn this into a feature request to get tests to run in parallel as this would speed up my workflow a lot.
CMake Tools Diagnostics
No response
Debug Log
No response
Additional Information
No response
About this issue
- Original URL
- State: closed
- Created a year ago
- Reactions: 33
- Comments: 33 (13 by maintainers)
@snehara99 thanks, now it works as you explained. In my case, 6.5k tests were executed in 37 seconds and I still wonder why it’s so slow comparing to running a test binary by hand which takes only 2 seconds (the whole suite has 10 binaries). But I guess it’s related to
ctestitself, not to the extension.I for one am finding the Project Status View’s interface a bit unintuitive: the “[All tests]” entry under “Tests” doesn’t actually have any run options, and the “Play” button at the top appears to be equivalent to launching the selected target under the “launch” section. So it’s very easy to click “All tests” and hit “play” to run them, resulting in it running something else entirely. Possibly a “Tests” button at the top would help.
Personally, I don’t tend to use Project Status View anyway - I want to run my tests off the Testing pane. If there’s a one-click option to run all the tests via the CMake pane or the status bar, that’s nice, but that’s not primarily where I expect to be running the tests from - and I want them to work in parallel there too. Pretty please? 😃
It seems to me this should be labelled “regression”, then, instead of “feature request”?
Parallel execution of tests executed by CTest worked with previous version 1.13.45.
It does not work with version 1.14.30. It seems this version runs each test one by one instead of letting CTest schedule the order of execution of tests and use parallel execution when possible. It is visible in logs of #3126