runner: Error messages when removing a self-hosted runner

Describe the bug When removing a self-hosted runner, I get the following error messages:

ldd: ./bin/libSystem.Security.Cryptography.Native.OpenSsl.so: No such file or directory
ldd: ./bin/libSystem.IO.Compression.Native.so: No such file or directory
# Runner removal
√ Runner removed successfully
√ Removed .credentials
√ Removed .runner
An error occurred: Access denied. System:ServiceIdentity;DDDDDDDD-DDDD-DDDD-DDDD-DDDDDDDDDDDD needs View permissions to perform the action.

The removal seems to succeed though, as refreshing the page removes the self-hosted runner from the list.

To Reproduce Steps to reproduce the behavior:

  1. Get the removal line from “Settings” tab of the repository, then “Actions” tab, then the three-dot menu next to the self-hosted runner name, and “Remove”
  2. Run the removal line ./config.sh remove --token XXX
  3. See error

Expected behavior A clear and concise description of what you expected to happen.

Runner Version and Platform

Version of your runner? Sorry I forgot to check, but in both cases they were downloaded from the official download links as given out in “Add runner”

OS of the machine running the runner? Linux x86_64. This has happened twice: with Ubuntu Linux 20.04 LTS and CentOS 8.

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 4
  • Comments: 19 (8 by maintainers)

Most upvoted comments

I am experiencing same issue with actions-runner-controller on AWS EKS after trying force remove the runner. All of my runner pods are keep created and terminated itself within 2 minutes. Any updates on this or workaroud to avoid terminating the runner?

Facing the same issue via clicking the Force removal of self-runner

I am seeing runners not picking up jobs, staying idle, then exiting with this

An error occurred: Access denied. System:ServiceIdentity;DDDDDDDD-DDDD-DDDD-DDDD-DDDDDDDDDDDD needs View permissions to perform the action.
Runner listener exit with retryable error, re-launch runner in 5 seconds.
Restarting runner...

√ Connected to GitHub

Failed to create a session. The runner registration has been deleted from the server, please re-configure.
Runner listener exit with terminated error, stop the service, no retry needed.
Exiting runner...
2023-04-02 04:27:45.294  NOTICE --- Runner init exited. Exiting this process with code 0 so that the container and the pod is GC'ed Kubernetes soon.

Any update on this issue? I’m facing this issue daily with enterprise-level ephemeral self-hosted runners (containerized)… It is blocking us to implement proper runner autoscaling

Is there any plan to fix this issue? We’re facing several of these errors daily as we’re relying on ephemeral self-hosted runners…

2021-12-22 03:16:40Z: Job CLANGARM64 completed with result: Canceled
An error occurred: Access denied. System:ServiceIdentity;DDDDDDDD-DDDD-DDDD-DDDD-DDDDDDDDDDDD needs View permissions to perform the action.

Also of note is that it reports that the job result was Canceled, but the job was not canceled.

[2021-12-22 03:15:55Z INFO JobDispatcher] Successfully renew job request 554, job is valid till 12/22/2021 3:25:55 AM
[2021-12-22 03:16:35Z ERR  GitHubActionsService] GET request to https://pipelines.actions.githubusercontent.com/yCmu0F2oGfbA9DkO6Byr4wKOkszFHnzaBFmAngWq8HAMcu3T9a/_apis/distributedtask/pools/1/messages?sessionId=a2055832-e3bc-4d2c-9226-ea321e364000&lastMessageId=1 failed. HTTP Status: Forbidden, AFD Ref: Ref A: 08A8A04A5F8C4272A9D3805928073430 Ref B: ASHEDGE1213 Ref C: 2021-12-22T03:16:35Z
[2021-12-22 03:16:35Z ERR  MessageListener] Catch exception during get next message.
[2021-12-22 03:16:35Z ERR  MessageListener] GitHub.DistributedTask.WebApi.AccessDeniedException: Access denied. System:ServiceIdentity;DDDDDDDD-DDDD-DDDD-DDDD-DDDDDDDDDDDD needs View permissions to perform the action.
   at GitHub.Services.WebApi.VssHttpClientBase.HandleResponseAsync(HttpResponseMessage response, CancellationToken cancellationToken)
   at GitHub.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
   at GitHub.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken)
   at GitHub.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpMethod method, IEnumerable`1 additionalHeaders, Guid locationId, Object routeValues, ApiResourceVersion version, HttpContent content, IEnumerable`1 queryParameters, Object userState, CancellationToken cancellationToken)
   at GitHub.Runner.Listener.MessageListener.GetNextMessageAsync(CancellationToken token)
[2021-12-22 03:16:35Z INFO MessageListener] Non-retriable exception: Access denied. System:ServiceIdentity;DDDDDDDD-DDDD-DDDD-DDDD-DDDDDDDDDDDD needs View permissions to perform the action.
[2021-12-22 03:16:35Z INFO JobDispatcher] Shutting down JobDispatcher. Make sure all WorkerDispatcher has finished.
[2021-12-22 03:16:35Z INFO JobDispatcher] Ensure WorkerDispather for job d4d6bd6c-c8bd-59d0-ecca-16324ffb3d87 run to finish, cancel any running job.
[2021-12-22 03:16:35Z INFO JobDispatcher] Send job cancellation message to worker for job d4d6bd6c-c8bd-59d0-ecca-16324ffb3d87.
[2021-12-22 03:16:35Z INFO ProcessChannel] Sending message of length 0, with hash 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'
[2021-12-22 03:16:40Z INFO ProcessInvokerWrapper] Scan all processes to find relationship between all processes.
[2021-12-22 03:16:40Z INFO ProcessInvokerWrapper] Find all child processes of process '6652'.
[2021-12-22 03:16:40Z INFO ProcessInvokerWrapper] Need kill all child processes trees before kill process '6652'.
[2021-12-22 03:16:40Z INFO ProcessInvokerWrapper] Child process '6740' needs be killed first.
[2021-12-22 03:16:40Z INFO ProcessInvokerWrapper] Find all child processes of process '6740'.
[2021-12-22 03:16:40Z INFO ProcessInvokerWrapper] Kill process '6740'.
[2021-12-22 03:16:40Z INFO ProcessInvokerWrapper] Kill process '6652'.
[2021-12-22 03:16:40Z INFO ProcessInvokerWrapper] Finished process 6652 with exit code 100, and elapsed time 02:24:48.8044470.
[2021-12-22 03:16:40Z INFO JobDispatcher] finish job request for job d4d6bd6c-c8bd-59d0-ecca-16324ffb3d87 with result: Canceled
[2021-12-22 03:16:40Z INFO Terminal] WRITE LINE: 2021-12-22 03:16:40Z: Job CLANGARM64 completed with result: Canceled
[2021-12-22 03:16:40Z INFO JobDispatcher] Stop renew job request for job d4d6bd6c-c8bd-59d0-ecca-16324ffb3d87.
[2021-12-22 03:16:40Z INFO JobDispatcher] job renew has been canceled, stop renew job request 554.
[2021-12-22 03:16:40Z INFO JobNotification] Entering JobCompleted Notification
[2021-12-22 03:16:40Z INFO JobNotification] Entering EndMonitor
[2021-12-22 03:16:40Z INFO JobDispatcher] Fire signal for one time used runner.
[2021-12-22 03:16:40Z ERR  GitHubActionsService] DELETE request to https://pipelines.actions.githubusercontent.com/yCmu0F2oGfbA9DkO6Byr4wKOkszFHnzaBFmAngWq8HAMcu3T9a/_apis/distributedtask/pools/1/sessions/a2055832-e3bc-4d2c-9226-ea321e364000 failed. HTTP Status: Forbidden, AFD Ref: Ref A: 9082B93F2F8D401B97858A82DA5ACD1B Ref B: ASHEDGE1213 Ref C: 2021-12-22T03:16:40Z
[2021-12-22 03:16:40Z INFO Runner] Ignore any exception during DeleteSession for an ephemeral runner. GitHub.DistributedTask.WebApi.AccessDeniedException: Access denied. System:ServiceIdentity;DDDDDDDD-DDDD-DDDD-DDDD-DDDDDDDDDDDD needs View permissions to perform the action.
   at GitHub.Services.WebApi.VssHttpClientBase.HandleResponseAsync(HttpResponseMessage response, CancellationToken cancellationToken)
   at GitHub.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
   at GitHub.Services.WebApi.VssHttpClientBase.SendAsync(HttpMethod method, Guid locationId, Object routeValues, ApiResourceVersion version, HttpContent content, IEnumerable`1 queryParameters, Object userState, CancellationToken cancellationToken)
   at GitHub.DistributedTask.WebApi.TaskAgentHttpClientBase.DeleteAgentSessionAsync(Int32 poolId, Guid sessionId, Object userState, CancellationToken cancellationToken)
   at GitHub.Runner.Listener.MessageListener.DeleteSessionAsync()
   at GitHub.Runner.Listener.Runner.RunAsync(RunnerSettings settings, Boolean runOnce)
[2021-12-22 03:16:40Z ERR  Terminal] WRITE ERROR: An error occurred: Access denied. System:ServiceIdentity;DDDDDDDD-DDDD-DDDD-DDDD-DDDDDDDDDDDD needs View permissions to perform the action.
[2021-12-22 03:16:40Z ERR  Listener] GitHub.DistributedTask.WebApi.AccessDeniedException: Access denied. System:ServiceIdentity;DDDDDDDD-DDDD-DDDD-DDDD-DDDDDDDDDDDD needs View permissions to perform the action.
   at GitHub.Services.WebApi.VssHttpClientBase.HandleResponseAsync(HttpResponseMessage response, CancellationToken cancellationToken)
   at GitHub.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
   at GitHub.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken)
   at GitHub.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpMethod method, IEnumerable`1 additionalHeaders, Guid locationId, Object routeValues, ApiResourceVersion version, HttpContent content, IEnumerable`1 queryParameters, Object userState, CancellationToken cancellationToken)
   at GitHub.Runner.Listener.MessageListener.GetNextMessageAsync(CancellationToken token)
   at GitHub.Runner.Listener.MessageListener.GetNextMessageAsync(CancellationToken token)
   at GitHub.Runner.Listener.Runner.RunAsync(RunnerSettings settings, Boolean runOnce)
   at GitHub.Runner.Listener.Runner.RunAsync(RunnerSettings settings, Boolean runOnce)
   at GitHub.Runner.Listener.Runner.RunAsync(RunnerSettings settings, Boolean runOnce)
   at GitHub.Runner.Listener.Runner.ExecuteCommand(CommandSettings command)
   at GitHub.Runner.Listener.Program.MainAsync(IHostContext context, String[] args)

The ephemeral runner is removed from the org, but the .runner and .credentials are still present on the runner itself, whereas before this started happening those were removed when the ephemeral runner shut down.

I’ve started getting this error when an ephemeral runner on Windows finishes.

@wyphan @TingluoHuang I’m also experiencing the same error message.