test-reporter: Test report doesn't appear in expected place in GitHub UI

Hi,

thanks a lot for this Action 👍

We currently have multiple GitHub Actions Workflows (main.yaml , lint-yaml.yaml etc ).

When using the action inside of the main.yaml, the test results appear under the lint-yaml results 🤔

Not 100% sure what is causing it

image

our step, within main.yaml looks like

    - name: Report PR Test results
      uses: dorny/test-reporter@v1
      if: ${{ github.event_name == 'pull_request' && (success() || failure()) }}
      with:
        name: jest Tests
        path: reports/jest-*.xml
        reporter: jest-junit

Here is a (sanitized) output of the step :

 1s
Run dorny/test-reporter@v1
  with:
    name: jest Tests
    path: reports/jest-*.xml
    reporter: jest-junit
    list-suites: all
    list-tests: all
    max-annotations: 10
    fail-on-error: true
    token: ***
  env:
     // snipped
Action was triggered by pull_request: using SHA from head of source branch
Check runs will be created with SHA=8d1d74409581269df26b4df491b55819597d9501
Listing all files tracked by git
  /usr/bin/git ls-files -z
  // ...
Found 190 files tracked by GitHub
Using test report parser 'jest-junit'
Creating test report jest Tests
  Processing test results from reports/jest-junit.xml
  Creating report summary
  Generating check run summary
  Creating annotations
  Creating check run with conclusion success
  Check run create response: 201
  Check run URL: https://api.github.com/repos/XXXX/YYYY/check-runs/2005347522
  Check run HTML: https://github.com/XXXX/YYYY/runs/2005347522

About this issue

  • Original URL
  • State: open
  • Created 6 years ago
  • Reactions: 32
  • Comments: 33 (3 by maintainers)

Commits related to this issue

Most upvoted comments

Maybe the new job summary (https://github.blog/2022-05-09-supercharging-github-actions-with-job-summaries/) can be used to:

  1. display they success/fail count like in the blog post
  2. display link to the check run in order to easier find it?

+1 for option to publish as job summary, instead of check

Job summaries are already used on main. We just need a new release.

Two comments on this.

  1. When not using the job summary, but a check run to also get the annotations, then this workaround might help to at least get the check run at a defined place and not added to a random workflow. As also for this action you can configure the token to be used, the work-around should probably also work for this action, though I did not try it yet.

  2. Maybe an approach like https://github.com/EnricoMi/publish-unit-test-result-action uses could slightly improve the situation when using the check run (use-actions-summary: 'false'). That is to create the check run, but additionally add a job summary that links to the check like can be seen for example here: https://github.com/jEdit-editor/jEdit/actions/runs/14906258648. With that you can at least easily navigate to the check run even if it is added to a random workflow.

@jozefizso / @dorny Could we cut a new release to resolve this issue?

Hello, if I understand it correctly, the #383 added support for reporting the results to the job summary in GHA.

Although the check may still be attributed to a different job run and/or workflow, the job summary should be a workaround for the problem.

Please, check if the dorny/test-reporter@v2 or dorny/test-reporter@v2.0.0 or dorny/test-reporter@ 6e6a65b7a0bd2c9197df7d0ae36ac5cee784230c action works for you.

https://github.com/dorny/test-reporter/releases/tag/v2.0.0

Hi @william-wallace-simplisafe and @tbutler-qontigo, this is an open source project with source code available and if you have a bugfix for this issue you are welcome to provide pull request for it.

@jozefizso I believe people have mentioned that the fix is already in main but there has not been a new release since May 8, 2024.

Hi @jozefizso As others have reported in this thread, the ability to publish to job summary is already in main - there just isn’t a release for it as yet. I was referring to those calls for a release for this existing implementation that would work around the problem rather than asking about a new change to directly address this open issue. Thanks

Any updates regarding the new release?

@jozefizso or @ritchxu, would one of you feel comfortable publishing a new release?

can we get the ‘Check run HTML’ URL as output of this report??

This also happen when you have a schedule trigger workflow. The test report is not attached to each of the run.

Just a quick thought (and sorry if this has been discussed before):

Instead of creating a check, couldn’t you use the same mechanism that https://github.com/actions/upload-artifact uses (under the assumption that you can upload other file types than .zip files)?

This would solve the problem in this issue. It would also solve the problem that this action requires the checks: write permission (which is not available for pull requests from forks).

I understand that this change would (most likely) worsen the UX of this action - but on the other hand, I’d argue that you don’t need to view the test report that often (usually only if some test has failed), so this change could be ok.

Unfortunately as @lhotari already commented - it’s a GitHub “feature”. Another post on this is here: https://github.community/t/specify-check-suite-when-creating-a-checkrun/118380

Until it’s fixed on GitHub side, the best I can do is to mention it in README.