runner: Downloading runner update fails: "An error occurred: Access to the path is denied"

Our self-hosted runner which runs in OpenShift is stuck in an update failure loop:

+++ hostname
++ ./config.sh --name actions-runner-1612996517-594b54548b-h8dqt --token ACD2A5V7C4TNQPYWFUVVPZDAERW36 --url https://github.com/redhat-actions --work _work --labels quay.io/redhat-github-actions/runner:v1.0.0, --unattended --replace
ldd: ./bin/libSystem.Security.Cryptography.Native.OpenSsl.so: No such file or directory
ldd: ./bin/libSystem.IO.Compression.Native.so: No such file or directory

--------------------------------------------------------------------------------
|        ____ _ _   _   _       _          _        _   _                      |
|       / ___(_) |_| | | |_   _| |__      / \   ___| |_(_) ___  _ __  ___      |
|      | |  _| | __| |_| | | | | '_ \    / _ \ / __| __| |/ _ \| '_ \/ __|     |
|      | |_| | | |_|  _  | |_| | |_) |  / ___ \ (__| |_| | (_) | | | \__ \     |
|       \____|_|\__|_| |_|\__,_|_.__/  /_/   \_\___|\__|_|\___/|_| |_|___/     |
|                                                                              |
|                       Self-hosted runner registration                        |
|                                                                              |
--------------------------------------------------------------------------------

# Authentication


√ Connected to GitHub

# Runner Registration



√ Runner successfully added
√ Runner connection is good

# Runner settings


√ Settings Saved.

++ set +x
++ svc_pid=68
++ wait 68
++ ./bin/runsvc.sh ''
.path=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Starting Runner listener with startup type: service
Started listener process
Started running service

√ Connected to GitHub

2021-02-10 22:35:42Z: Listening for Jobs
Runner update in progress, do not shutdown runner.
Downloading 2.277.1 runner
An error occurred: Access to the path is denied.
Runner listener exited with error code 2
Runner listener exit with retryable error, re-launch runner in 5 seconds.
Starting Runner listener with startup type: service
Started listener process

√ Connected to GitHub

… which is repeating indefinitely

As far as we can tell, are the permissions are set up correctly. The current user ‘runner’ is part of the root group and is able to write to _diag, _work, etc.

[runner@actions-runner-1612996517-594b54548b-h8dqt ~]$ pwd
/home/runner
[runner@actions-runner-1612996517-594b54548b-h8dqt ~]$ ls -l
total 72
-rw-rw-r--. 1 runner root     9 Feb  7 17:14 RUNNER_VERSION.txt
drwxr-xr-x. 2 runner root  4096 Feb 10 22:42 _diag
drwxr-xr-x. 3 runner root    21 Feb 10 22:41 _work
drwxrwxr-x. 4 runner root 12288 Jan 21 19:30 bin
drwxr-xr-x. 4 runner root 12288 Feb 10 22:35 bin.2.277.1
-rwxrwxr-x. 1 runner root  2416 Jan 21 19:28 config.sh
-rwxr-xr-x. 1 runner root  2492 Feb  7 17:12 entrypoint.sh
-rwxrwxr-x. 1 runner root   646 Jan 21 19:28 env.sh
drwxrwxr-x. 4 runner root    41 Jan 21 19:29 externals
drwxr-xr-x. 4 runner root    41 Feb 10 22:36 externals.2.277.1
-rwxrwxr-x. 1 runner root  1144 Feb  7 17:12 get-runner-release.sh
-rwxrwxr-x. 1 runner root  1598 Feb 10 22:42 run.sh
-rwxr-xr-x. 1 runner root  4706 Feb 10 22:35 svc.sh
-rwxr-xr-x. 1 runner root  1326 Feb  7 17:12 uid.sh

You can see the updater was able to download and extract the 2.277.1 directories but seems to have failed after that step.

Logs:

[runner@actions-runner-1613010216-776c59b66c-5bpd9 ~]$ cat _diag/Runner_20210211-022453-utc.log 
[2021-02-11 02:24:53Z INFO HostContext] No proxy settings were found based on environmental variables (http_proxy/https_proxy/HTTP_PROXY/HTTPS_PROXY)
[2021-02-11 02:24:53Z INFO HostContext] Well known directory 'Bin': '/home/runner/bin'
[2021-02-11 02:24:53Z INFO HostContext] Well known directory 'Root': '/home/runner'
[2021-02-11 02:24:53Z INFO HostContext] Well known config file 'Credentials': '/home/runner/.credentials'
[2021-02-11 02:24:54Z INFO Listener] Runner is built for Linux (X64) - linux-x64.
[2021-02-11 02:24:54Z INFO Listener] RuntimeInformation: Linux 4.18.0-193.41.1.el8_2.x86_64 #1 SMP Wed Jan 13 11:33:33 EST 2021.
[2021-02-11 02:24:54Z INFO Listener] Version: 2.276.1
[2021-02-11 02:24:54Z INFO Listener] Commit: 5effa808be96eae7f2e08d713426b8d056bddff1
[2021-02-11 02:24:54Z INFO Listener] Culture: 
[2021-02-11 02:24:54Z INFO Listener] UI Culture: 
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Bin': '/home/runner/bin'
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Root': '/home/runner'
[2021-02-11 02:24:54Z INFO Listener] Validating directory permissions for: '/home/runner'
[2021-02-11 02:24:54Z INFO CommandLineParser] Parse
[2021-02-11 02:24:54Z INFO CommandLineParser] Parsing 3 args
[2021-02-11 02:24:54Z INFO CommandLineParser] parsing argument
[2021-02-11 02:24:54Z INFO CommandLineParser] HasArgs: False
[2021-02-11 02:24:54Z INFO CommandLineParser] Adding Command: run
[2021-02-11 02:24:54Z INFO CommandLineParser] parsing argument
[2021-02-11 02:24:54Z INFO CommandLineParser] HasArgs: True
[2021-02-11 02:24:54Z INFO CommandLineParser] arg: startuptype
[2021-02-11 02:24:54Z INFO CommandLineParser] parsing argument
[2021-02-11 02:24:54Z INFO CommandLineParser] HasArgs: True
[2021-02-11 02:24:54Z INFO CommandLineParser] Adding option 'startuptype': 'service'
[2021-02-11 02:24:54Z INFO Listener] Arguments parsed
[2021-02-11 02:24:54Z INFO Runner] ExecuteCommand
[2021-02-11 02:24:54Z INFO ConfigurationStore] currentAssemblyLocation: /home/runner/bin/Runner.Listener.dll
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Bin': '/home/runner/bin'
[2021-02-11 02:24:54Z INFO ConfigurationStore] binPath: /home/runner/bin
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Bin': '/home/runner/bin'
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Root': '/home/runner'
[2021-02-11 02:24:54Z INFO ConfigurationStore] RootFolder: /home/runner
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Bin': '/home/runner/bin'
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Root': '/home/runner'
[2021-02-11 02:24:54Z INFO HostContext] Well known config file 'Runner': '/home/runner/.runner'
[2021-02-11 02:24:54Z INFO ConfigurationStore] ConfigFilePath: /home/runner/.runner
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Bin': '/home/runner/bin'
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Root': '/home/runner'
[2021-02-11 02:24:54Z INFO HostContext] Well known config file 'Credentials': '/home/runner/.credentials'
[2021-02-11 02:24:54Z INFO ConfigurationStore] CredFilePath: /home/runner/.credentials
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Bin': '/home/runner/bin'
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Root': '/home/runner'
[2021-02-11 02:24:54Z INFO HostContext] Well known config file 'MigratedCredentials': '/home/runner/.credentials_migrated'
[2021-02-11 02:24:54Z INFO ConfigurationStore] MigratedCredFilePath: /home/runner/.credentials_migrated
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Bin': '/home/runner/bin'
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Root': '/home/runner'
[2021-02-11 02:24:54Z INFO HostContext] Well known config file 'Service': '/home/runner/.service'
[2021-02-11 02:24:54Z INFO ConfigurationStore] ServiceConfigFilePath: /home/runner/.service
[2021-02-11 02:24:54Z INFO CommandSettings] Flag 'help': 'False'
[2021-02-11 02:24:54Z INFO CommandSettings] Flag 'version': 'False'
[2021-02-11 02:24:54Z INFO CommandSettings] Flag 'commit': 'False'
[2021-02-11 02:24:54Z INFO CommandSettings] Flag 'check': 'False'
[2021-02-11 02:24:54Z INFO CommandSettings] Command 'configure': 'False'
[2021-02-11 02:24:54Z INFO CommandSettings] Command 'remove': 'False'
[2021-02-11 02:24:54Z INFO CommandSettings] Command 'warmup': 'False'
[2021-02-11 02:24:54Z INFO ConfigurationManager] LoadSettings
[2021-02-11 02:24:54Z INFO ConfigurationStore] IsConfigured()
[2021-02-11 02:24:54Z INFO ConfigurationStore] IsConfigured: True
[2021-02-11 02:24:54Z INFO ConfigurationManager] Is configured: True
[2021-02-11 02:24:54Z INFO ConfigurationStore] Read setting file: 356 chars
[2021-02-11 02:24:54Z INFO ConfigurationManager] Settings Loaded
[2021-02-11 02:24:54Z INFO ConfigurationStore] IsServiceConfigured()
[2021-02-11 02:24:54Z INFO ConfigurationStore] IsServiceConfigured: False
[2021-02-11 02:24:54Z INFO CommandSettings] Command 'run': 'True'
[2021-02-11 02:24:54Z INFO ConfigurationStore] IsConfigured()
[2021-02-11 02:24:54Z INFO ConfigurationStore] IsConfigured: True
[2021-02-11 02:24:54Z INFO ConfigurationManager] Is configured: True
[2021-02-11 02:24:54Z INFO Runner] Set runner startup type - Service
[2021-02-11 02:24:54Z INFO CommandSettings] Flag 'once': 'False'
[2021-02-11 02:24:54Z INFO Runner] RunAsync
[2021-02-11 02:24:54Z INFO ConfigurationManager] LoadSettings
[2021-02-11 02:24:54Z INFO ConfigurationStore] IsConfigured()
[2021-02-11 02:24:54Z INFO ConfigurationStore] IsConfigured: True
[2021-02-11 02:24:54Z INFO ConfigurationManager] Is configured: True
[2021-02-11 02:24:54Z INFO ConfigurationManager] Settings Loaded
[2021-02-11 02:24:54Z INFO MessageListener] {
  "AgentId": 58,
  "AgentName": "actions-runner-1613010216-776c59b66c-5bpd9",
  "PoolId": 1,
  "PoolName": "Default",
  "ServerUrl": "https://pipelines.actions.githubusercontent.com/xKlIG5rz7cXPUF8ADvf26OsQl4rkLxWZkewUYDaGr662Vi4LZC",
  "GitHubUrl": "https://github.com/redhat-actions/self-hosted-runner-installer",
  "WorkFolder": "/home/runner/_work"
}
[2021-02-11 02:24:54Z INFO MessageListener] Loading Credentials
[2021-02-11 02:24:54Z INFO ConfigurationStore] HasCredentials()
[2021-02-11 02:24:54Z INFO ConfigurationStore] stored True
[2021-02-11 02:24:54Z INFO CredentialManager] GetCredentialProvider
[2021-02-11 02:24:54Z INFO CredentialManager] Creating type OAuth
[2021-02-11 02:24:54Z INFO CredentialManager] Creating credential type: OAuth
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Bin': '/home/runner/bin'
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Root': '/home/runner'
[2021-02-11 02:24:54Z INFO HostContext] Well known config file 'RSACredentials': '/home/runner/.credentials_rsaparams'
[2021-02-11 02:24:54Z INFO RSAFileKeyManager] Loading RSA key parameters from file /home/runner/.credentials_rsaparams
[2021-02-11 02:24:54Z INFO MessageListener] Attempt to create session.
[2021-02-11 02:24:54Z INFO MessageListener] Connecting to the Runner Server...
[2021-02-11 02:24:54Z INFO RunnerServer] Establish connection with 100 seconds timeout.
[2021-02-11 02:24:54Z INFO GitHubActionsService] Starting operation Location.GetConnectionData
[2021-02-11 02:24:54Z INFO RunnerServer] Establish connection with 60 seconds timeout.
[2021-02-11 02:24:54Z INFO GitHubActionsService] Starting operation Location.GetConnectionData
[2021-02-11 02:24:54Z INFO RunnerServer] Establish connection with 60 seconds timeout.
[2021-02-11 02:24:54Z INFO GitHubActionsService] Starting operation Location.GetConnectionData
[2021-02-11 02:24:54Z INFO GitHubActionsService] Finished operation Location.GetConnectionData
[2021-02-11 02:24:54Z INFO GitHubActionsService] Finished operation Location.GetConnectionData
[2021-02-11 02:24:54Z INFO GitHubActionsService] Finished operation Location.GetConnectionData
[2021-02-11 02:24:57Z INFO MessageListener] VssConnection created
[2021-02-11 02:24:57Z INFO Terminal] WRITE LINE: 
[2021-02-11 02:24:57Z INFO Terminal] WRITE LINE: 
[2021-02-11 02:24:57Z WARN GitHubActionsService] Authentication failed with status code 401.
WWW-Authenticate: Bearer
Strict-Transport-Security: max-age=2592000
X-TFS-ProcessId: 1b155fbb-97b6-4f7a-8689-7b1f5279585e
ActivityId: 3543c995-5eb9-446b-b117-7d8a87874b66
X-TFS-Session: 3148d865-3929-4900-bf15-a86d63dc4682
X-VSS-E2EID: 3e247701-609b-4a32-aaa5-584c133a57b7
X-VSS-SenderDeploymentId: 13a19993-c6bc-326c-afb4-32c5519f46f0
X-TFS-ServiceError: The+user+%27System%3aPublicAccess%3baaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa%27+is+not+authorized+to+access+this+resource.
X-VSS-S2STargetService: 0000005A-0000-8888-8000-000000000000/visualstudio.com
X-MSEdge-Ref: Ref A: 7462F6E1178640BE90DA1DDE3068B78D Ref B: ASHEDGE1406 Ref C: 2021-02-11T02:24:57Z
Date: Thu, 11 Feb 2021 02:24:57 GMT

[2021-02-11 02:24:57Z INFO RSAFileKeyManager] Loading RSA key parameters from file /home/runner/.credentials_rsaparams
[2021-02-11 02:24:57Z INFO RSAFileKeyManager] Loading RSA key parameters from file /home/runner/.credentials_rsaparams
[2021-02-11 02:24:57Z INFO GitHubActionsService] AAD Correlation ID for this token request: Unknown
[2021-02-11 02:24:57Z INFO MessageListener] Session created.
[2021-02-11 02:24:57Z INFO Terminal] WRITE LINE: 2021-02-11 02:24:57Z: Listening for Jobs
[2021-02-11 02:24:57Z INFO JobDispatcher] Set runner/worker IPC timeout to 30 seconds.
[runner@actions-runner-1613010216-776c59b66c-5bpd9 ~]$ cat _diag/Runner_20210211-022453-utc.log 
[2021-02-11 02:24:53Z INFO HostContext] No proxy settings were found based on environmental variables (http_proxy/https_proxy/HTTP_PROXY/HTTPS_PROXY)
[2021-02-11 02:24:53Z INFO HostContext] Well known directory 'Bin': '/home/runner/bin'
[2021-02-11 02:24:53Z INFO HostContext] Well known directory 'Root': '/home/runner'
[2021-02-11 02:24:53Z INFO HostContext] Well known config file 'Credentials': '/home/runner/.credentials'
[2021-02-11 02:24:54Z INFO Listener] Runner is built for Linux (X64) - linux-x64.
[2021-02-11 02:24:54Z INFO Listener] RuntimeInformation: Linux 4.18.0-193.41.1.el8_2.x86_64 #1 SMP Wed Jan 13 11:33:33 EST 2021.
[2021-02-11 02:24:54Z INFO Listener] Version: 2.276.1
[2021-02-11 02:24:54Z INFO Listener] Commit: 5effa808be96eae7f2e08d713426b8d056bddff1
[2021-02-11 02:24:54Z INFO Listener] Culture: 
[2021-02-11 02:24:54Z INFO Listener] UI Culture: 
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Bin': '/home/runner/bin'
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Root': '/home/runner'
[2021-02-11 02:24:54Z INFO Listener] Validating directory permissions for: '/home/runner'
[2021-02-11 02:24:54Z INFO CommandLineParser] Parse
[2021-02-11 02:24:54Z INFO CommandLineParser] Parsing 3 args
[2021-02-11 02:24:54Z INFO CommandLineParser] parsing argument
[2021-02-11 02:24:54Z INFO CommandLineParser] HasArgs: False
[2021-02-11 02:24:54Z INFO CommandLineParser] Adding Command: run
[2021-02-11 02:24:54Z INFO CommandLineParser] parsing argument
[2021-02-11 02:24:54Z INFO CommandLineParser] HasArgs: True
[2021-02-11 02:24:54Z INFO CommandLineParser] arg: startuptype
[2021-02-11 02:24:54Z INFO CommandLineParser] parsing argument
[2021-02-11 02:24:54Z INFO CommandLineParser] HasArgs: True
[2021-02-11 02:24:54Z INFO CommandLineParser] Adding option 'startuptype': 'service'
[2021-02-11 02:24:54Z INFO Listener] Arguments parsed
[2021-02-11 02:24:54Z INFO Runner] ExecuteCommand
[2021-02-11 02:24:54Z INFO ConfigurationStore] currentAssemblyLocation: /home/runner/bin/Runner.Listener.dll
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Bin': '/home/runner/bin'
[2021-02-11 02:24:54Z INFO ConfigurationStore] binPath: /home/runner/bin
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Bin': '/home/runner/bin'
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Root': '/home/runner'
[2021-02-11 02:24:54Z INFO ConfigurationStore] RootFolder: /home/runner
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Bin': '/home/runner/bin'
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Root': '/home/runner'
[2021-02-11 02:24:54Z INFO HostContext] Well known config file 'Runner': '/home/runner/.runner'
[2021-02-11 02:24:54Z INFO ConfigurationStore] ConfigFilePath: /home/runner/.runner
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Bin': '/home/runner/bin'
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Root': '/home/runner'
[2021-02-11 02:24:54Z INFO HostContext] Well known config file 'Credentials': '/home/runner/.credentials'
[2021-02-11 02:24:54Z INFO ConfigurationStore] CredFilePath: /home/runner/.credentials
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Bin': '/home/runner/bin'
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Root': '/home/runner'
[2021-02-11 02:24:54Z INFO HostContext] Well known config file 'MigratedCredentials': '/home/runner/.credentials_migrated'
[2021-02-11 02:24:54Z INFO ConfigurationStore] MigratedCredFilePath: /home/runner/.credentials_migrated
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Bin': '/home/runner/bin'
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Root': '/home/runner'
[2021-02-11 02:24:54Z INFO HostContext] Well known config file 'Service': '/home/runner/.service'
[2021-02-11 02:24:54Z INFO ConfigurationStore] ServiceConfigFilePath: /home/runner/.service
[2021-02-11 02:24:54Z INFO CommandSettings] Flag 'help': 'False'
[2021-02-11 02:24:54Z INFO CommandSettings] Flag 'version': 'False'
[2021-02-11 02:24:54Z INFO CommandSettings] Flag 'commit': 'False'
[2021-02-11 02:24:54Z INFO CommandSettings] Flag 'check': 'False'
[2021-02-11 02:24:54Z INFO CommandSettings] Command 'configure': 'False'
[2021-02-11 02:24:54Z INFO CommandSettings] Command 'remove': 'False'
[2021-02-11 02:24:54Z INFO CommandSettings] Command 'warmup': 'False'
[2021-02-11 02:24:54Z INFO ConfigurationManager] LoadSettings
[2021-02-11 02:24:54Z INFO ConfigurationStore] IsConfigured()
[2021-02-11 02:24:54Z INFO ConfigurationStore] IsConfigured: True
[2021-02-11 02:24:54Z INFO ConfigurationManager] Is configured: True
[2021-02-11 02:24:54Z INFO ConfigurationStore] Read setting file: 356 chars
[2021-02-11 02:24:54Z INFO ConfigurationManager] Settings Loaded
[2021-02-11 02:24:54Z INFO ConfigurationStore] IsServiceConfigured()
[2021-02-11 02:24:54Z INFO ConfigurationStore] IsServiceConfigured: False
[2021-02-11 02:24:54Z INFO CommandSettings] Command 'run': 'True'
[2021-02-11 02:24:54Z INFO ConfigurationStore] IsConfigured()
[2021-02-11 02:24:54Z INFO ConfigurationStore] IsConfigured: True
[2021-02-11 02:24:54Z INFO ConfigurationManager] Is configured: True
[2021-02-11 02:24:54Z INFO Runner] Set runner startup type - Service
[2021-02-11 02:24:54Z INFO CommandSettings] Flag 'once': 'False'
[2021-02-11 02:24:54Z INFO Runner] RunAsync
[2021-02-11 02:24:54Z INFO ConfigurationManager] LoadSettings
[2021-02-11 02:24:54Z INFO ConfigurationStore] IsConfigured()
[2021-02-11 02:24:54Z INFO ConfigurationStore] IsConfigured: True
[2021-02-11 02:24:54Z INFO ConfigurationManager] Is configured: True
[2021-02-11 02:24:54Z INFO ConfigurationManager] Settings Loaded
[2021-02-11 02:24:54Z INFO MessageListener] {
  "AgentId": 58,
  "AgentName": "actions-runner-1613010216-776c59b66c-5bpd9",
  "PoolId": 1,
  "PoolName": "Default",
  "ServerUrl": "https://pipelines.actions.githubusercontent.com/xKlIG5rz7cXPUF8ADvf26OsQl4rkLxWZkewUYDaGr662Vi4LZC",
  "GitHubUrl": "https://github.com/redhat-actions/self-hosted-runner-installer",
  "WorkFolder": "/home/runner/_work"
}
[2021-02-11 02:24:54Z INFO MessageListener] Loading Credentials
[2021-02-11 02:24:54Z INFO ConfigurationStore] HasCredentials()
[2021-02-11 02:24:54Z INFO ConfigurationStore] stored True
[2021-02-11 02:24:54Z INFO CredentialManager] GetCredentialProvider
[2021-02-11 02:24:54Z INFO CredentialManager] Creating type OAuth
[2021-02-11 02:24:54Z INFO CredentialManager] Creating credential type: OAuth
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Bin': '/home/runner/bin'
[2021-02-11 02:24:54Z INFO HostContext] Well known directory 'Root': '/home/runner'
[2021-02-11 02:24:54Z INFO HostContext] Well known config file 'RSACredentials': '/home/runner/.credentials_rsaparams'
[2021-02-11 02:24:54Z INFO RSAFileKeyManager] Loading RSA key parameters from file /home/runner/.credentials_rsaparams
[2021-02-11 02:24:54Z INFO MessageListener] Attempt to create session.
[2021-02-11 02:24:54Z INFO MessageListener] Connecting to the Runner Server...
[2021-02-11 02:24:54Z INFO RunnerServer] Establish connection with 100 seconds timeout.
[2021-02-11 02:24:54Z INFO GitHubActionsService] Starting operation Location.GetConnectionData
[2021-02-11 02:24:54Z INFO RunnerServer] Establish connection with 60 seconds timeout.
[2021-02-11 02:24:54Z INFO GitHubActionsService] Starting operation Location.GetConnectionData
[2021-02-11 02:24:54Z INFO RunnerServer] Establish connection with 60 seconds timeout.
[2021-02-11 02:24:54Z INFO GitHubActionsService] Starting operation Location.GetConnectionData
[2021-02-11 02:24:54Z INFO GitHubActionsService] Finished operation Location.GetConnectionData
[2021-02-11 02:24:54Z INFO GitHubActionsService] Finished operation Location.GetConnectionData
[2021-02-11 02:24:54Z INFO GitHubActionsService] Finished operation Location.GetConnectionData
[2021-02-11 02:24:57Z INFO MessageListener] VssConnection created
[2021-02-11 02:24:57Z INFO Terminal] WRITE LINE: 
[2021-02-11 02:24:57Z INFO Terminal] WRITE LINE: 
[2021-02-11 02:24:57Z WARN GitHubActionsService] Authentication failed with status code 401.
WWW-Authenticate: Bearer
Strict-Transport-Security: max-age=2592000
X-TFS-ProcessId: 1b155fbb-97b6-4f7a-8689-7b1f5279585e
ActivityId: 3543c995-5eb9-446b-b117-7d8a87874b66
X-TFS-Session: 3148d865-3929-4900-bf15-a86d63dc4682
X-VSS-E2EID: 3e247701-609b-4a32-aaa5-584c133a57b7
X-VSS-SenderDeploymentId: 13a19993-c6bc-326c-afb4-32c5519f46f0
X-TFS-ServiceError: The+user+%27System%3aPublicAccess%3baaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa%27+is+not+authorized+to+access+this+resource.
X-VSS-S2STargetService: 0000005A-0000-8888-8000-000000000000/visualstudio.com
X-MSEdge-Ref: Ref A: 7462F6E1178640BE90DA1DDE3068B78D Ref B: ASHEDGE1406 Ref C: 2021-02-11T02:24:57Z
Date: Thu, 11 Feb 2021 02:24:57 GMT

[2021-02-11 02:24:57Z INFO RSAFileKeyManager] Loading RSA key parameters from file /home/runner/.credentials_rsaparams
[2021-02-11 02:24:57Z INFO RSAFileKeyManager] Loading RSA key parameters from file /home/runner/.credentials_rsaparams
[2021-02-11 02:24:57Z INFO GitHubActionsService] AAD Correlation ID for this token request: Unknown
[2021-02-11 02:24:57Z INFO MessageListener] Session created.
[2021-02-11 02:24:57Z INFO Terminal] WRITE LINE: 2021-02-11 02:24:57Z: Listening for Jobs
[2021-02-11 02:24:57Z INFO JobDispatcher] Set runner/worker IPC timeout to 30 seconds.
[2021-02-11 02:25:00Z INFO RSAFileKeyManager] Loading RSA key parameters from file /home/runner/.credentials_rsaparams
[2021-02-11 02:25:00Z INFO MessageListener] Message '5' received from session '5ad0f557-7a63-46c9-b36b-121a3d875513'.
[2021-02-11 02:25:00Z INFO Runner] Refresh message received, kick-off selfupdate background process.
[2021-02-11 02:25:00Z INFO SelfUpdater] Version '2.277.1' of 'agent' package available in server.
[2021-02-11 02:25:00Z INFO SelfUpdater] Current running runner version is 2.276.1
[2021-02-11 02:25:00Z INFO SelfUpdater] An update is available.
[2021-02-11 02:25:00Z INFO Terminal] WRITE LINE: Runner update in progress, do not shutdown runner.
[2021-02-11 02:25:00Z INFO Terminal] WRITE LINE: Downloading 2.277.1 runner
[2021-02-11 02:25:00Z INFO HostContext] Well known directory 'Bin': '/home/runner/bin'
[2021-02-11 02:25:00Z INFO HostContext] Well known directory 'Root': '/home/runner'
[2021-02-11 02:25:00Z INFO HostContext] Well known directory 'Work': '/home/runner/_work'
[2021-02-11 02:25:00Z INFO SelfUpdater] Attempt 1: save latest runner into /home/runner/_work/_update/runner1.tar.gz.
[2021-02-11 02:25:00Z INFO SelfUpdater] Download runner: begin download
[2021-02-11 02:25:01Z INFO SelfUpdater] Download runner: finished download
[2021-02-11 02:25:01Z INFO SelfUpdater] Which: 'tar'
[2021-02-11 02:25:01Z INFO SelfUpdater] Location: '/usr/bin/tar'
[2021-02-11 02:25:01Z INFO ProcessInvokerWrapper] Starting process:
[2021-02-11 02:25:01Z INFO ProcessInvokerWrapper]   File name: '/usr/bin/tar'
[2021-02-11 02:25:01Z INFO ProcessInvokerWrapper]   Arguments: '-xzf "/home/runner/_work/_update/runner1.tar.gz"'
[2021-02-11 02:25:01Z INFO ProcessInvokerWrapper]   Working directory: '/home/runner/_work/_update'
[2021-02-11 02:25:01Z INFO ProcessInvokerWrapper]   Require exit code zero: 'False'
[2021-02-11 02:25:01Z INFO ProcessInvokerWrapper]   Encoding web name:  ; code page: ''
[2021-02-11 02:25:01Z INFO ProcessInvokerWrapper]   Force kill process on cancellation: 'False'
[2021-02-11 02:25:01Z INFO ProcessInvokerWrapper]   Redirected STDIN: 'False'
[2021-02-11 02:25:01Z INFO ProcessInvokerWrapper]   Persist current code page: 'False'
[2021-02-11 02:25:01Z INFO ProcessInvokerWrapper]   Keep redirected STDIN open: 'False'
[2021-02-11 02:25:01Z INFO ProcessInvokerWrapper]   High priority process: 'False'
[2021-02-11 02:25:01Z INFO ProcessInvokerWrapper] Failed to update oom_score_adj for PID: 287.
[2021-02-11 02:25:01Z INFO ProcessInvokerWrapper] System.UnauthorizedAccessException: Access to the path '/proc/287/oom_score_adj' is denied.
 ---> System.IO.IOException: Permission denied
   --- End of inner exception stack trace ---
   at System.IO.FileStream.WriteNative(ReadOnlySpan`1 source)
   at System.IO.FileStream.FlushWriteBuffer()
   at System.IO.FileStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at System.IO.StreamWriter.CloseStreamFromDispose(Boolean disposing)
   at System.IO.StreamWriter.Dispose(Boolean disposing)
   at System.IO.TextWriter.Dispose()
   at System.IO.File.WriteAllText(String path, String contents)
   at GitHub.Runner.Sdk.ProcessInvoker.WriteProcessOomScoreAdj(Int32 processId, Int32 oomScoreAdj)
[2021-02-11 02:25:01Z INFO ProcessInvokerWrapper] Process started with process id 287, waiting for process exit.
[2021-02-11 02:25:03Z INFO ProcessInvokerWrapper] STDOUT/STDERR stream read finished.
[2021-02-11 02:25:03Z INFO ProcessInvokerWrapper] STDOUT/STDERR stream read finished.
[2021-02-11 02:25:03Z INFO ProcessInvokerWrapper] Finished process 287 with exit code 0, and elapsed time 00:00:02.2407173.
[2021-02-11 02:25:03Z INFO SelfUpdater] Finished getting latest runner package at: /home/runner/_work/_update.
[2021-02-11 02:25:03Z INFO HostContext] Well known directory 'Bin': '/home/runner/bin'
[2021-02-11 02:25:03Z INFO HostContext] Well known directory 'Root': '/home/runner'
[2021-02-11 02:25:03Z INFO SelfUpdater] Copy /home/runner/_work/_update/bin to /home/runner/bin.2.277.1.
[2021-02-11 02:25:03Z INFO HostContext] Well known directory 'Bin': '/home/runner/bin'
[2021-02-11 02:25:03Z INFO HostContext] Well known directory 'Root': '/home/runner'
[2021-02-11 02:25:03Z INFO SelfUpdater] Copy /home/runner/_work/_update/externals to /home/runner/externals.2.277.1.
[2021-02-11 02:25:03Z INFO SelfUpdater] Copy any remaining .sh/.cmd files into runner root.
[2021-02-11 02:25:03Z INFO HostContext] Well known directory 'Bin': '/home/runner/bin'
[2021-02-11 02:25:03Z INFO HostContext] Well known directory 'Root': '/home/runner'
[2021-02-11 02:25:03Z INFO JobDispatcher] Shutting down JobDispatcher. Make sure all WorkerDispatcher has finished.
[2021-02-11 02:25:04Z ERR  Terminal] WRITE ERROR: An error occurred: Access to the path is denied.
[2021-02-11 02:25:04Z ERR  Listener] System.UnauthorizedAccessException: Access to the path is denied.
 ---> System.IO.IOException: Operation not permitted
   --- End of inner exception stack trace ---
   at System.IO.FileSystem.CopyFile(String sourceFullPath, String destFullPath, Boolean overwrite)
   at System.IO.FileInfo.CopyTo(String destFileName, Boolean overwrite)
   at GitHub.Runner.Listener.SelfUpdater.DownloadLatestRunner(CancellationToken token)
   at GitHub.Runner.Listener.SelfUpdater.SelfUpdate(AgentRefreshMessage updateMessage, IJobDispatcher jobDispatcher, Boolean restartInteractiveRunner, CancellationToken token)
   at GitHub.Runner.Listener.Runner.RunAsync(RunnerSettings settings, Boolean runOnce)
   at GitHub.Runner.Listener.Runner.RunAsync(RunnerSettings settings, Boolean runOnce)
   at GitHub.Runner.Listener.Runner.RunAsync(RunnerSettings settings, Boolean runOnce)
   at GitHub.Runner.Listener.Runner.ExecuteCommand(CommandSettings command)
   at GitHub.Runner.Listener.Program.MainAsync(IHostContext context, String[] args)
[2021-02-11 02:25:04Z ERR  Listener] #####################################################
[2021-02-11 02:25:04Z ERR  Listener] System.IO.IOException: Operation not permitted

Any idea what path may be missing permissions?

Or, could we add the path that is getting the “permission denied” error, to the error message?

Thank you

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 30 (4 by maintainers)

Most upvoted comments

I am having this issue, too. Using ARC in Kubernetes Mode… Can we Please Reopen this one? @hross

System.UnauthorizedAccessException: Access to the path '/home/runner/_work/_tool' is denied.
 ---> System.IO.IOException: Permission denied
   --- End of inner exception stack trace ---
   at System.IO.FileSystem.CreateDirectory(String fullPath)
   at System.IO.Directory.CreateDirectory(String path)
   at GitHub.Runner.Worker.JobRunner.RunAsync(AgentJobRequestMessage message, CancellationToken jobRequestCancellationToken)
   at GitHub.Runner.Worker.JobRunner.RunAsync(AgentJobRequestMessage message, CancellationToken jobRequestCancellationToken)
   at GitHub.Runner.Worker.Worker.RunAsync(String pipeIn, String pipeOut)
   at GitHub.Runner.Worker.Program.MainAsync(IHostContext context, String[] args)
+10
mrclrchtr on Oct 3, 2023

Still breaks for us on a regular basis. Please reopen.

+4
wcrowther-rentpath on Mar 16, 2023

@matthenry87 , Got it working. Found some SELinux related settings that were referred to in the old documentation

Added this to the spec:

      securityContext:
        seLinuxOptions:
          level: "s0"
          role: "system_r"
          type: "super_t"
          user: "system_u"

Excellent, I’ll give it a try soon. Thanks!

+3
matthenry87 on Nov 9, 2023

In my case the problem using the containerMode: type: kubernetes was happening because it was mounting the ephemeral volume to the _work dir with the permissions of the root user. I’ve solved by changing the securityContext on the template to 1001. So the volume now can be used by the “runner” user.

template:
  spec:
    securityContext:
      fsGroup: 1001
    containers:
    - name: runner
      image: ghcr.io/actions/actions-runner:2.311.0
      command: ["/home/runner/run.sh"]
+2
gabrielmuras on Oct 24, 2023

Thanks guys, just ran into the exact same issue and getting an active thread (less than an hour old) with a solution

+1
kevogabaa on Nov 10, 2023

I am having this issue - trying out the kubernetes containerMode for the first time.

Errors I get:

Error: Missing file at path: /runner/_work/_temp/_runner_hook_responses/<redacted>.json

Access to the path '/runner/_work/_temp/_github_workflow' is denied.

And when I shell into the pod, I cannot cd into /runner/_work or list it’s contents despite being the owner:

runner@k8s-container-mode:/$ cd /runner
runner@k8s-container-mode:/runner$ ls -la
total 104
drwxrwxrwx. 8 runner docker  4096 Apr 15 20:25 .
drwxr-xr-x. 1 root   root    4096 Apr 15 20:25 ..
-rw-r--r--. 1 runner runner   266 Apr 15 20:25 .credentials
-rw-------. 1 runner runner  1667 Apr 15 20:25 .credentials_rsaparams
-rw-r--r--. 1 runner runner    58 Apr 15 20:25 .env
-rw-r--r--. 1 runner runner   134 Apr 15 20:25 .path
-rw-r--r--. 1 runner runner   338 Apr 15 20:25 .runner
drwxr-xr-x. 4 runner runner  4096 Apr 15 20:25 _diag
drwxr-xr-x. 8 runner docker  4096 Apr 15 20:26 _work
drwxr-xr-x. 4 runner runner 16384 Apr 15 20:25 bin
-rwxr-xr-x. 1 runner runner  2458 Apr 15 20:25 config.sh
-rwxr-xr-x. 1 runner runner   646 Apr 15 20:25 env.sh
drwxr-xr-x. 6 runner runner  4096 Apr 15 20:25 externals
drwxr-xr-x. 2 runner runner  4096 Apr 15 20:25 externalstmp
drwxr-xr-x. 2 runner runner  4096 Apr 15 20:25 k8s
-rw-r--r--. 1 runner runner  1487 Apr 15 20:25 run-helper.cmd.template
-rwxr-xr-x. 1 runner runner  2522 Apr 15 20:25 run-helper.sh
-rwxr-xr-x. 1 runner runner  2522 Apr 15 20:25 run-helper.sh.template
-rwxr-xr-x. 1 runner runner  2537 Apr 15 20:25 run.sh
-rwxr-xr-x. 1 runner runner    65 Apr 15 20:25 safe_sleep.sh
-rwxr-xr-x. 1 runner runner  5222 Apr 15 20:25 svc.sh
runner@k8s-container-mode:/runner$ cd _work
bash: cd: _work: Permission denied
runner@k8s-container-mode:/runner$

I also just re-built my customized runner image on top of docker.io/summerwind/actions-runner:latest and made sure my runner’s ImagePullPolicy is set to Always.

The volume is successfully being attached (or else runner pod wouldn’t have come up).

@hross Can we please re-open this issue?

+1
matthenry87 on Apr 15, 2023
Read more comments on GitHub
← runner: `concurrency` with properties now causes "syntax error"
runner: Environment Secrets are not available on Reusable Workflow / Workflow Templates →