vitest: V8 coverage provider does not properly report react coverage, istanbul does

Describe the bug

I have a project (in deno) that works using my own library; thankfully i got testing working after a lot of back and forth. I am now trying to get coverage to work properly. After failing all day i tried to use istanbul instead of v8 just for the hell of it and for some reason that works?

The bug: Using V8 instead of istanbul as a provider for coverage makes all lines “miss”. The output between the two are inconsistent. The project i mean can be found here: https://gitlab.zweieuro.at/wui/projects/vivaci-ui-deno (branch “vitest-bug”)

Command: deno task test:coverage How to reproduce: Using v8 as a provider will print:

Task test:coverage deno -A --node-modules-dir=auto npm:vitest run --coverage -c ./vite.config.ts

 RUN  v3.0.8 /home/zweieuro/Documents/uni/2023_SS/WUI/html_UI/vivaci_ui_deno/client
      Coverage enabled with v8

 ✓ tests/ScoreDisplay.test.tsx (3 tests) 33ms
 ✓ tests/App.test.tsx (2 tests) 30ms

 Test Files  2 passed (2)
      Tests  5 passed (5)
   Start at  16:55:22
   Duration  4.18s (transform 3.54s, setup 5.23s, collect 1.54s, tests 63ms, environment 471ms, prepare 76ms)

 % Coverage report from v8
--------------------|---------|----------|---------|---------|-------------------
File                | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
--------------------|---------|----------|---------|---------|-------------------
All files           |       0 |       20 |      20 |       0 |                   
 App.tsx            |       0 |        0 |       0 |       0 | 1-57              
 HighscoreInput.tsx |       0 |        0 |       0 |       0 | 1-64              
 LifeIcon.tsx       |       0 |      100 |     100 |       0 | 2-15              
 ScoreDisplay.tsx   |       0 |        0 |       0 |       0 | 1-71              
 main.tsx           |       0 |        0 |       0 |       0 | 1-11              
--------------------|---------|----------|---------|---------|-------------------

Using istanbul instead:

Task test:coverage deno -A --node-modules-dir=auto npm:vitest run --coverage -c ./vite.config.ts

 RUN  v3.0.8 /home/zweieuro/Documents/uni/2023_SS/WUI/html_UI/vivaci_ui_deno/client
      Coverage enabled with istanbul

 ✓ tests/ScoreDisplay.test.tsx (3 tests) 30ms
 ✓ tests/App.test.tsx (2 tests) 24ms

 Test Files  2 passed (2)
      Tests  5 passed (5)
   Start at  16:55:58
   Duration  4.18s (transform 3.61s, setup 5.26s, collect 1.68s, tests 54ms, environment 473ms, prepare 76ms)

 % Coverage report from istanbul
--------------------|---------|----------|---------|---------|-------------------
File                | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
--------------------|---------|----------|---------|---------|-------------------
All files           |   77.14 |     62.5 |   78.57 |   77.14 |                   
 App.tsx            |     100 |      100 |     100 |     100 |                   
 HighscoreInput.tsx |   44.44 |       40 |      25 |   44.44 | 6,47-58           
 LifeIcon.tsx       |     100 |      100 |     100 |     100 |                   
 ScoreDisplay.tsx   |   89.47 |    66.66 |     100 |   89.47 | 8,28              
 main.tsx           |       0 |      100 |     100 |       0 | 7                 
--------------------|---------|----------|---------|---------|-------------------

I do not have full coverage, but this output makes sense.

I made a discussion which eventually lead to this but started somewhere very different: https://github.com/vitest-dev/vitest/discussions/7617

Note: The server is my private one, it may be a bit slow at times

BIG NOTE: I am using deno! which uses npm behind the scenes but i can only select package manager npm here!

Reproduction

! I am assuming you got deno installed

git clone https://gitlab.zweieuro.at/wui/projects/vivaci-ui-deno.git
cd vivaci-ui-deno
git checkout vitest-bug
deno task test:coverage

# Now change the reporter to v8

deno task test:coverage

you will end up with different output

System Info

System:
    OS: Linux 6.12 Arch Linux
    CPU: (16) x64 AMD Ryzen 7 7840U w/ Radeon  780M Graphics
    Memory: 15.68 GB / 30.54 GB
    Container: Yes
    Shell: 5.2.37 - /usr/bin/bash
  Binaries:
    Node: 22.14.0 - ~/.nvm/versions/node/v22.14.0/bin/node
    npm: 10.9.2 - ~/.nvm/versions/node/v22.14.0/bin/npm

Used Package Manager

npm

Validations

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 1
  • Comments: 15 (7 by maintainers)

Most upvoted comments

This fix alone doesn’t fix Deno compatibility.

Deno seems to add paths instead of file URLs in ScriptCoverage. Looks like bug to me. There’s related issue in:

We could add bunch of if(isDeno) checks but I’d like to avoid that. We don’t do that for Bun either, intentionally.

{
  result: [
    {
      scriptId: '407',
      url: '/Users/x/repros/deno/main.test.ts',
      functions: [Array],
      startOffset: 185
    },
    {
      scriptId: '409',
      url: '/Users/x/repros/deno/main.ts',
      functions: [Array],
      startOffset: 185
    }
  ]
}

If I manually flip these to file URLs, V8 coverage starts to work with Deno. Source maps look good too.

{
  result: [
    {
      ...
-     url: '/Users/x/repros/deno/main.test.ts',
+     url: 'file:///Users/x/repros/deno/main.test.ts',
    },
    {
      ...
-     url: '/Users/x/repros/deno/main.ts',
+     url: 'file:///Users/x/repros/deno/main.ts',
    }
  ]
}