monocart-reporter: [BUG] Attachments not found

Hi,

It seems attachments are not being fetched properly depending on how you setup your playwright outputDir and monocart-reporter outputFile.

If I setup my playwright.config.ts as such:

const _testsDir = "./tests";
const _testsOutputBaseDir = path.join(_testsDir, "test-results");
const _testReportersOutputBaseDir = path.join(_testsOutputBaseDir, "reporters");
const _htmlReporterDir = path.join(_testReportersOutputBaseDir, "html");
const _monoCartReporterDir = path.join(_testReportersOutputBaseDir, "monocart");
const _monoCartReporterOutputFile = path.join(_monoCartReporterDir, "index.html");

export default defineConfig({
    ...
    outputDir: `${_testsOutputBaseDir}`,
    reporter: [
        [
            "monocart-reporter",
            {
                name: "Playwright Monocart Report",
                outputFile: _monoCartReporterOutputFile,
                coverage: {
                    sourceFilter: (sourceName: any) => sourceName.search(/src\//) !== -1,
                },
            },
        ],
    ],
    ...
});

And have an automatic fixture as such:

const { addCoverageReport, attachCoverageReport } = require("monocart-reporter");

export { expect } from "@playwright/test";

interface CodeCoverageFixture {
    autoTestFixture: void;
}

export const test = baseTest.extend<CodeCoverageFixture>({
    autoTestFixture: [
        async ({ page }, use): Promise<void> => {
            // coverage API is chromium only
            if (test.info().project.name === "chromium") {
                await Promise.all([
                    page.coverage.startJSCoverage(),
                    page.coverage.startCSSCoverage(),
                ]);
            }

            await use();

            if (test.info().project.name === "chromium") {
                const [jsCoverage, cssCoverage] = await Promise.all([
                    page.coverage.stopJSCoverage(),
                    page.coverage.stopCSSCoverage(),
                ]);
                const coverageList = [...jsCoverage, ...cssCoverage];
                await addCoverageReport(coverageList, test.info());
                const report = await attachCoverageReport(coverageList, test.info());
            }
        },
        {
            auto: true,
        },
    ],
});

Then I run a test that produces a snapshot and I have this output:

[MCR] coverage: tests/test-results/coverage-8aefe567157b61ce7446/index.html Coverage Report - basic test
[MCR] coverage: tests/test-results/reporters/monocart/coverage/index.html Coverage Report - Playwright Monocart Report (global)
[MCR] json report: tests/test-results/reporters/monocart/index.json
[MCR] html report: tests/test-results/reporters/monocart/index.html
[MCR] view report: npx monocart show-report tests/test-results/reporters/monocart/index.html

Notice that both the HTML report and the global coverage report share the same base dir tests/test-results/reporters/monocart/ and as such I can navigate from the index.html to the global code coverage report:

  • I can go from http://localhost:8090/index.html to http://localhost:8090/coverage/index.html

However the screenshot attachments and the code coverage attachments display as not found. When I try to access the test code coverage it tries to go to http://localhost:8090/coverage-8aefe567157b61ce7446/index.html and because the path coverage-8aefe567157b61ce7446/index.html does not exist at tests/test-results/reporters/monocart it can’t find the file. That path is at tests/test-results/.

The same happens for the screenshots. If I try to go to a screenshot it tries to go to http://localhost:8090/find-institution-example-basic-test-chromium/basic-test-1-expected.png and it cant find the path /find-institution-example-basic-test-chromium/basic-test-1-expected.png at tests/test-results/reporters/monocart so it fails. The path is at tests/test-results/.

It seems to me that the attachments path uses playwrights outputDirto store data instead of the directory relative to where the monocart html report is at which is indicated byoutputFile` and then there’s a mismatch in where files should be ? Do you think this is a bug or am I misusing the configuration?

Let me know if you need more info.

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 16 (7 by maintainers)

Most upvoted comments

I can confirm it’s working as expected. Awesome 👍

image

This issue morphed from attachments not found, to improving the images. I should have created another issue for that to make things clearer. Apologies for that.

Regarding the original purpose of this issue: I think you can close this since I understand how the paths must be configured for the reporter to pick up files. The only improvement you might consider is trying to detect when the user has misconfigured the paths and put a warning message on the output.

The tooltip idea is great. For the phrasing I would do something like:

  • For quick comparison of the image, click and hold on the image. Depending on which tab you are on, the image will swap to a different one. For example: to compare the actual image with the expected one, select the Actual tab, click and hold the mouse on the image and it will show the image from the Expected tab. Releasing the mouse will swap back to the image from the Actual tab.

I did something a bit more verbose because I think this behaviour is not that common so I felt better to add an example.