unity-test-runner: The test runner gets stuck in the "Testing in playmode" phase

Bug description

The test runner gets stuck in the “Testing in playmode” phase. I have tried ubuntu-latest and windows-2022 docker images.

CI configuration is almost the same as described in https://game.ci/docs/github/getting-started

name: CI

on:
  workflow_dispatch: { }
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  test:
    name: Unity Tests Workflow
    runs-on: ubuntu-latest
    timeout-minutes: 60

    strategy:
      matrix:
        unityVersion:
          - 2020.3.19f1

    steps:

      # Checkout (without LFS)
      - name: Checkout Repository
        uses: actions/checkout@v2

      # Git LFS
      - name: Create LFS file list
        run: git lfs ls-files -l | cut -d' ' -f1 | sort > .lfs-assets-id

      - name: Restore LFS cache
        uses: actions/cache@v2
        id: lfs-cache
        with:
          path: .git/lfs
          key: ${{ runner.os }}-lfs-${{ hashFiles('.lfs-assets-id') }}

      - name: Git LFS Pull
        run: |
          git lfs pull
          git add .
          git reset --hard

      # Cache
      - uses: actions/cache@v2
        with:
          path: Library
          key: Library-${{ hashFiles('Assets/**', 'Packages/**', 'ProjectSettings/**') }}
          restore-keys: |
            Library-

      # Test
      - name: Run Tests
        uses: game-ci/unity-test-runner@v2
        id: tests
        env:
          UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
        with:
          gitPrivateToken: ${{ secrets.GIT_PRIVATE_TOKEN }}
          githubToken: ${{ secrets.GITHUB_TOKEN }}

      # Test Results
      - name: Upload Test Results
        uses: actions/upload-artifact@v2
        if: always()
        with:
          name: Test Restuls
          path: ${{ steps.tests.outputs.artifactsPath }}

Run Tests logs:

Run game-ci/unity-test-runner@v2
/usr/bin/docker run --workdir /github/workspace --rm --env UNITY_LICENSE --env UNITY_LICENSE_FILE --env UNITY_EMAIL --env UNITY_PASSWORD --env UNITY_SERIAL --env UNITY_VERSION=2020.3.19f1 --env PROJECT_PATH=. --env CUSTOM_PARAMETERS= --env TEST_PLATFORMS=playmode;editmode;COMBINE_RESULTS --env COVERAGE_OPTIONS=generateAdditionalMetrics;generateHtmlReport;generateBadgeReport --env COVERAGE_RESULTS_PATH=CodeCoverage --env ARTIFACTS_PATH=artifacts --env GITHUB_REF --env GITHUB_SHA --env GITHUB_REPOSITORY --env GITHUB_ACTOR --env GITHUB_WORKFLOW --env GITHUB_HEAD_REF --env GITHUB_BASE_REF --env GITHUB_EVENT_NAME --env GITHUB_WORKSPACE=/github/workspace --env GITHUB_ACTION --env GITHUB_EVENT_PATH --env RUNNER_OS --env RUNNER_TOOL_CACHE --env RUNNER_TEMP --env RUNNER_WORKSPACE --env GIT_PRIVATE_TOKEN=*** --env CHOWN_FILES_TO= --volume /home/runner/work/_temp/_github_home:/root:z --volume /home/runner/work/_temp/_github_workflow:/github/workflow:z --volume /home/runner/work/bird-game/bird-game:/github/works...
Unable to find image 'unityci/editor:ubuntu-2020.3.19f1-linux-il2cpp-1' locally
ubuntu-2020.3.19f1-linux-il2cpp-1: Pulling from unityci/editor
08a6abff8943: Pulling fs layer
17bc3eeef474: Pulling fs layer
7525af58113d: Pulling fs layer
643424925c07: Pulling fs layer
a64283161285: Pulling fs layer
6f8bb29b92de: Pulling fs layer
f93bd952912f: Pulling fs layer
2f3943063d01: Pulling fs layer
b94b5611506d: Pulling fs layer
13f2bc070dfd: Pulling fs layer
4f4fb700ef54: Pulling fs layer
7d1b8006d6d1: Pulling fs layer
643424925c07: Waiting
a64283161285: Waiting
6f8bb29b92de: Waiting
f93bd952912f: Waiting
2f3943063d01: Waiting
b94b5611506d: Waiting
13f2bc070dfd: Waiting
4f4fb700ef54: Waiting
7d1b8006d6d1: Waiting
17bc3eeef474: Verifying Checksum
17bc3eeef474: Download complete
08a6abff8943: Verifying Checksum
08a6abff8943: Download complete
643424925c07: Verifying Checksum
643424925c07: Download complete
a64283161285: Verifying Checksum
a64283161285: Download complete
6f8bb29b92de: Verifying Checksum
6f8bb29b92de: Download complete
f93bd952912f: Verifying Checksum
f93bd952912f: Download complete
b94b5611506d: Verifying Checksum
b94b5611506d: Download complete
13f2bc070dfd: Verifying Checksum
13f2bc070dfd: Download complete
4f4fb700ef54: Verifying Checksum
4f4fb700ef54: Download complete
7525af58113d: Verifying Checksum
7525af58113d: Download complete
08a6abff8943: Pull complete
7d1b8006d6d1: Verifying Checksum
7d1b8006d6d1: Download complete
17bc3eeef474: Pull complete
7525af58113d: Pull complete
643424925c07: Pull complete
a64283161285: Pull complete
6f8bb29b92de: Pull complete
f93bd952912f: Pull complete
2f3943063d01: Verifying Checksum
2f3943063d01: Download complete
2f3943063d01: Pull complete
b94b5611506d: Pull complete
13f2bc070dfd: Pull complete
4f4fb700ef54: Pull complete
7d1b8006d6d1: Pull complete
Digest: sha256:d8e7a364522199122b17c750851d5f1e467015b849695188f49e37944520bd49
Status: Downloaded newer image for unityci/editor:ubuntu-2020.3.19f1-linux-il2cpp-1
Changing to "/github/workspace/_activate-license" directory.
/github/workspace/_activate-license /github/workspace
Requesting activation (personal license)
Activation complete.
/github/workspace
/steps/set_gitcredential.sh: line 1: #!/usr/bin/env: No such file or directory
GIT_PRIVATE_TOKEN is set configuring git credentials
---------- git config --list -------------
filter.lfs.smudge=git-lfs smudge -- %f
filter.lfs.process=git-lfs filter-process
filter.lfs.required=true
filter.lfs.clean=git-lfs clean -- %f
credential.helper=store
url.https://github.com/.insteadof=ssh://git@github.com/
url.https://github.com/.insteadof=git@github.com
***github.com/.insteadof=https://github.com/
***github.com/.insteadof=ssh://git@github.com/
***github.com/.insteadof=git@github.com:
---------- git config --list --show-origin -------------
file:/etc/gitconfig	filter.lfs.smudge=git-lfs smudge -- %f
file:/etc/gitconfig	filter.lfs.process=git-lfs filter-process
file:/etc/gitconfig	filter.lfs.required=true
file:/etc/gitconfig	filter.lfs.clean=git-lfs clean -- %f
file:/root/.gitconfig	credential.helper=store
file:/root/.gitconfig	url.https://github.com/.insteadof=ssh://git@github.com/
file:/root/.gitconfig	url.https://github.com/.insteadof=git@github.com
file:/root/.gitconfig	***github.com/.insteadof=https://github.com/
file:/root/.gitconfig	***github.com/.insteadof=ssh://git@github.com/
file:/root/.gitconfig	***github.com/.insteadof=git@github.com:
Using project path "/github/workspace/.".
Using artifacts path "artifacts" to save test results.
Using coverage results path "CodeCoverage" to save test coverage results.
Using custom parameters .
Using Unity version "2020.3.19f1" to test.
###########################
#    Artifacts folder     #
###########################
Creating "/github/workspace/artifacts" if it does not exist.
###########################
#    Project directory    #
###########################
total 56K
drwxr-xr-x 9 1001  121 4.0K Jul  1 06:15 .
drwxr-xr-x 4 root root 4.0K Jul  1 06:15 ..
-rw-r--r-- 1 1001  121  217 Jul  1 06:12 .editorconfig
drwxr-xr-x 9 1001  121 4.0K Jul  1 06:12 .git
-rw-r--r-- 1 1001  121 1.9K Jul  1 06:12 .gitattributes
drwxr-xr-x 3 1001  121 4.0K Jul  1 06:12 .github
-rw-r--r-- 1 1001  121 1.3K Jul  1 06:12 .gitignore
drwxr-xr-x 6 1001  121 4.0K Jul  1 06:12 Assets
-rw-r--r-- 1 1001  121  766 Jul  1 06:12 CHARK.BirdGame.Client.csproj.DotSettings
-rw-r--r-- 1 1001  121  766 Jul  1 06:12 CHARK.BirdGame.Editor.csproj.DotSettings
drwxr-xr-x 2 1001  121 4.0K Jul  1 06:12 Packages
drwxr-xr-x 2 1001  121 4.0K Jul  1 06:12 ProjectSettings
drwxr-xr-x 2 root root 4.0K Jul  1 06:15 _activate-license
drwxr-xr-x 2 root root 4.0K Jul  1 06:15 artifacts
###########################
#   Testing in playmode  #
###########################

I used secret var ACTIONS_STEP_DEBUG = true for enabling verbose logs.

Where could be the problem?

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 43 (5 by maintainers)

Commits related to this issue

Most upvoted comments

I guess this could be interesting: https://docs.unity3d.com/2022.3/Documentation/Manual/CLIBatchmodeCoroutines.html TLDR: Not all Coroutines are supported in batch mode, most notably: WaitForEndOfFrame . Which might be part of this issue for some.

just found solution in my case, just changing uses: game-ci/unity-test-runner@v2 to uses: game-ci/unity-test-runner@v2.0.0 maybe there some run test code in new game-ci version that I can’t used in game-ci test runner

@ugnelis Yeah, I’m also using game-ci for a while and I’m super happy with it.

@asyasmi what output do you get?

Creating "/github/workspace/ubuntu-latest_2021.3.21f1_StandaloneLinux64_artifacts" if it does not exist.

###########################
#    Project directory    #
###########################

total 56K
drwxr-xr-x 11 1001  123 4.0K May 19 10:54 .
drwxr-xr-x  4 root root 4.0K May 19 10:54 ..
drwxr-xr-x  2 1001  123 4.0K May 19 10:52 .ci
drwxr-xr-x  8 1001  123 4.0K May 19 10:52 .git
-rw-r--r--  1 1001  123  210 May 19 10:52 .gitattributes
drwxr-xr-x  3 1001  123 4.0K May 19 10:52 .github
-rw-r--r--  1 1001  123 1.2K May 19 10:52 .gitignore
drwxr-xr-x  4 1001  123 4.0K May 19 10:52 Assets
drwxr-xr-x  2 1001  123 4.0K May 19 10:52 Packages
drwxr-xr-x  2 1001  123 4.0K May 19 10:52 ProjectSettings
-rw-r--r--  1 1001  123   51 May 19 10:52 README.md
drwxr-xr-x  2 root root 4.0K May 19 10:54 _activate-license
drwxr-xr-x  2 root root 4.0K May 19 10:54 ubuntu-latest_2021.3.21f1_StandaloneLinux64_artifacts

###########################
#   Testing in editmode  #
###########################

that’s all before the cancellation of the job 😦

Regarding timeout-minutes - we had no timeout at all, it still never for out of this state. But I think this time I got it.

UPD: Eventually that were our tests. The issue was with tests, one of them eventually was broken and just hung a process and logs were not shown before they were done and tests were never done, so… The job ended in this limbo.

So I’d really be waiting for https://github.com/game-ci/cli/issues/12 to be realized.

No problem. I am not sure what is wrong. The maintainer should answer this question. FIY, I have tested v2.0.0, v2.0.1, and v2.0.2 but only v2.0.0 works as intended

Looks like one of the releases caused this. It would be extremely helpful if you could help us find out which one.

Do the versions v2.0.1 and v2.0.2 work for you too?

I have tested v2.0.1 and v2.0.2. Both of these versions get stuck in the “Testing in playmode” phase