runner: `Load failed: 5: Input/output error` on ./svc.sh run | Mac Big Sur Issues?

Describe the bug Just deployed 2 new self-hosted runners on Mac Minis, I get the error Load failed: 5: Input/output error when I install and then start the runner as a service. Interestingly the service appears to start and the runner appears online on github.com Silicon Intel OS Version Big Sur 11.2.3

To Reproduce

Install the latest self-hosted runner (v2.278.0 at the time of writing) and setup the runner ./config.sh --url $URL --token $TOKEN
Install the runner as a service ./svc.sh install
Run the service ./svc.sh run

Expected behavior No error is posted

Runner Version and Platform

Silicon Intel OS Version Big Sur 11.2.3 Runner Version v2.278.0

What’s not working?

The runner appears to register with github and take work, currently testing to see if there is any functional issues, will update

Job Log Output

I’ve replaced our actual org and hostname values with $org and $hostname

starting actions.runner.$org.$hostname
Load failed: 5: Input/output error
status actions.runner.$org.$hostname:

/Users/runner/Library/LaunchAgents/actions.runner.$org.$hostname.plist

Started:
512 0 actions.runner.$org.$hostname

Runner and Worker’s Diagnostic Logs

If you google load failed 5 input/output error you will find lots of posts about this error on a wide array of software being installed as a service with lots of references to Big Sur. Something has changed in relation to Big Sur and services and the plist that gets auto generated by the service install script and / or the values within the plist no longer being valid for some reason.

Example Search Results https://developer.apple.com/forums/thread/665661 https://www.reddit.com/r/MacOS/comments/kbko61/launchctl_broken/ https://www.reddit.com/r/MacOS/comments/kbko61/launchctl_broken/gpv2to1/

EDIT

It looks like launchctl load | unload are now legacy commands, the svc.sh script probably needs looking to setup / run the service in a > Big sur compliant way?

https://ss64.com/osx/launchctl.html

Subcommands from the previous implementation of launchd are generally
available, though some may be unimplemented. Unimplemented subcommands
are documented as such.

load | unload [-wF] [-S sessiontype] [-D searchpath] paths ...
        Load the specified configuration files or directories of
        configuration files.  Jobs that are not on-demand will be started
        as soon as possible. All specified jobs will be loaded before any
        of them are allowed to start. Note that per-user configuration
        files (LaunchAgents) must be owned by root (if they are located
        in /Library/LaunchAgents) or the user loading them (if they are
        located in $HOME/Library/LaunchAgents).  All system-wide daemons
        (LaunchDaemons) must be owned by root. Configuration files must
        disallow group and world writes. These restrictions are in place
        for security reasons, as allowing writability to a launchd
        configuration file allows one to specify which executable will be
        launched.

About this issue

Original URL
State: open
Created 3 years ago
Reactions: 11
Comments: 25 (1 by maintainers)

Most upvoted comments

I had a similar sounding issue that does appear to be related to the launchctl update issues. From what I understand on https://babodee.wordpress.com/2016/04/09/launchctl-2-0-syntax/ using load is now replaced with bootstrap however that alone was not my issue.

The issue I found is that load and bootstrap both require the user to be in a gui session to configure them. I originally was only configuring over ssh. Running ./svc.sh start over vnc in a terminal session worked for me.

I would like to see the setup script modified to use the newer launchctl settings and configure it to run in the user/ domain instead of gui/ that way it can be configured solely over ssh.

+11

brandonbirdj on May 20, 2021

A workaround that worked for me is to install the plist file in /Library/LaunchDaemons and start the service with

sudo /bin/launchctl load /Library/LaunchDaemons/actions.runner.sebsto-xcodeinstall.plist

My plist is as follow

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>Label</key>
    <string>actions.runner.sebsto-xcodeinstall.ip-172-31-67-99</string>
    <key>ProgramArguments</key>
    <array>
      <string>/Users/ec2-user/actions-runner/runsvc.sh</string>
    </array>
    <key>UserName</key>
    <string>ec2-user</string>
    <key>GroupName</key>
    <string>staff</string>  
    <key>WorkingDirectory</key>
    <string>/Users/ec2-user/actions-runner</string>
    <key>RunAtLoad</key>
    <true/>    
    <key>StandardOutPath</key>
    <string>/Users/ec2-user/Library/Logs/actions.runner.sebsto-xcodeinstall.ip-172-31-67-99/stdout.log</string>
    <key>StandardErrorPath</key>
    <string>/Users/ec2-user/Library/Logs/actions.runner.sebsto-xcodeinstall.ip-172-31-67-99/stderr.log</string>
    <key>EnvironmentVariables</key>
    <dict> 
      <key>ACTIONS_RUNNER_SVC</key>
      <string>1</string>
    </dict>
<!--
    <key>ProcessType</key>
    <string>Interactive</string>
-->
    <key>SessionCreate</key>
    <true/>
  </dict>
</plist>

sebsto on Sep 5, 2022

It would be great if there were a purely ssh solution that didn’t depend on the GUI at all. In the meantime, this can be helpful: go to System Preferences -> Users and Groups -> Login Options -> Automatic Login, and set the ‘runner’ or ‘gha’ user to login automatically. Reboot. The user will login at boot time and the services will start.

sdarwin on Aug 2, 2022

This is also an issue with monterey on ec2 mac metal instance

Command: ./svc.sh start Result:

starting actions.runner.X
Load failed: 5: Input/output error
Try running `launchctl bootstrap` as root for richer errors.
status actions.runner.X:

/Users/ec2-user/Library/LaunchAgents/actions.runner.X.plist

Stopped

ec2-user@X actions-runner % launchctl list | grep action
(Didnt start)

c3-kaspesi on Jun 21, 2022

As a workaround, our team was able to run the following commands unattended on an AWS Mac EC2 instance. We put the following within the instance’s user-data. The runsvc.sh script start the agent but in the foreground but adding the & moves the process to the background. Not the most ideal but for now seems to work for us.

sudo su -- ec2-user ./svc.sh install
sudo su -- ec2-user ./runsvc.sh start &

jveldboom on Aug 20, 2021

@sebsto has the right idea. Real services start on boot, not on login so on macOS this means the service needs to be a LaunchDaemon and not a LaunchAgent

phatblat on Jun 16, 2023

@muojp this is unecessary. I have GitHub agent + LaunchDaemons configuration up and running and serving dozens of builds per day. This runs on a totally unattended Amazon EC2 Mac instance on which I never connected with a GUI.

Here is the launch daemon plist file I use

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>Label</key>
    <string>actions.runner.sebsto-xcodeinstall.xcodeinstall</string>
    <key>ProgramArguments</key>
    <array>
      <string>/Users/ec2-user/actions-runner-xcodeinstall/run.sh</string>
    </array>
    <key>UserName</key>
    <string>ec2-user</string>
    <key>WorkingDirectory</key>
    <string>/Users/ec2-user/actions-runner-xcodeinstall</string>
    <key>RunAtLoad</key>
    <true/>    
    <key>StandardOutPath</key>
    <string>/Users/ec2-user/Library/Logs/actions.runner.sebsto-xcodeinstall.xcodeinstall/stdout.log</string>
    <key>StandardErrorPath</key>
    <string>/Users/ec2-user/Library/Logs/actions.runner.sebsto-xcodeinstall.xcodeinstall/stderr.log</string>
    <key>EnvironmentVariables</key>
    <dict> 
      <key>ACTIONS_RUNNER_SVC</key>
      <string>1</string>
    </dict>
<!--
    <key>ProcessType</key>
    <string>Interactive</string>
-->
    <key>SessionCreate</key>
    <true/>
  </dict>
</plist>

And this is how I prepare the keychain

https://github.com/sebsto/amplify-ios-getting-started/blob/main/code/ci_actions/01_keychain.sh

sebsto on Nov 20, 2023

i’ve been using this to fix it:

sed -i -Ee ‘s;(launchctl) load -w;\1 bootstrap gui/$user_id;’ -e ‘s;(launchctl) unload;\1 bootout gui/$user_id;’ -e ‘s;(launchctl) list;\1 print gui/$user_id;’ svc.sh

jordanb-afs on May 2, 2022