nyc: `--all` does not work for pre-instrumented files

@mrchief @novemberborn @ljharb would definitely love to support this use-case better – maybe we can brainstorm something in this thread … nyc is supposed to work like magic for everything.

not true that nothing ever becomes stale, but this ticket won’t get flagged stale again.

Made this very simple repo to reproduce this problem: https://github.com/mrchief/nyc-all-demo

However, in the above repo, adding

"require": [
      "babel-register"
    ]

to nyc makes it work (just like the documentation says). So I think there is something else that’s messing up things (besides the fact that babel-register may have to be loaded twice).

I’m going to see if I can make things break (where requiring babel-register won’t help either).

Ok, I did it! The whole thing is a mess, but that’s been the whole nyc/istanbul/remap story so far anyway 😦

Here’s the changes I had to make if they’re useful to anyone:

https://github.com/Dart-Code/Dart-Code/commit/3ec4973dde00335033aad2820d29a4c2b7b7a518 https://github.com/Dart-Code/Dart-Code/commit/c9ee2a56e6041ebfb11a8ccffd44aed010b746ff

It would be much nicer if nyc report could just take an --all and read the files itself, injecting fake zeros for anything that didn’t have coverage, but I’m just happy that this finally works (it’s been weeks since I first started trying to get coverage set up!).

We’d have to parse the file to extract the coverage header. Loading the file may lead to unexpected code execution.

We’re actually already doing this with the --all functionality, it does a noop require and extracts and stores the coverage header from the file, if I’m recalling correctly. I’m not 100% sure why @ljharb’s approach would not be working; Perhaps the issue is that the transpiled code is in an excluded folder, e.g,. /dist?

It’s probably not a huge amount of work to better support @ljharb’s use-case, but it might take some fiddling to get source-maps, etc,. working as expected. Here’s the logic that handles --all:

https://github.com/istanbuljs/nyc/blob/master/index.js#L173

Note how it swaps out the require statement for an empty function, so that code doesn’t execute; any ways, would love supporting @ljharb’s use-case if anyone feels like digging in … there’s even a new slack I’ve launched for coordinating work.

Did anyone get a solution to this? I’m facing the same issue. My project is a VS Code extension written in TypeScript. Because of how VS Code loads the tests, nyc can’t instrument them on the fly so I’m doing this:

1. Compile the TS to JS
2. Pre-instrument the JS with nyc instrument
3. In my test teardown, write __COVERAGE__ to disk
4. Use istandbul-remapper to fix up source mapping back to ts
5. Run nyc report to generate lcov

The issue is that any file that wasn’t loaded in the test run doesn’t get included when I write __COVERAGE__ to disk and therefore doesn’t show in the report. The missing files are instrumented.

I’ve got CodeClimate set up and as things are, if someone sends a PR that loads a file that wasn’t loaded before, it blames them for adding a ton of uncovered code (when in reality it was already uncovered, just missing from the report).

Any suggestions welcome!