cloudflared: `cloudflared service install` broken on macOS

Doc: https://github.com/cloudflare/cloudflare-docs/blob/production/products/cloudflare-one/src/content/connections/connect-apps/run-tunnel/run-as-service.md

Steps to reproduce

  1. sudo cloudflared service install
  2. tail -f /Library/Logs/com.cloudflare.cloudflared.err.log /Library/Logs/com.cloudflare.cloudflared.out.log
  3. Observe repeating line in error log Use `cloudflared tunnel run` to start tunnel MY_TUNNEL

Expected result:

Seeing the usual tunnel startup info in the logs

A fix that worked for me

Edit /Library/LaunchDaemons/com.cloudflare.cloudflared.plist

Original (runs /usr/local/bin/cloudflared)

                <array>
                        <string>/usr/local/bin/cloudflared</string>
                </array>

Fixed (runs /usr/local/bin/cloudflared tunnel run)

                <array>
                        <string>/usr/local/bin/cloudflared</string>
                        <string>tunnel</string>
                        <string>run</string>
                </array>

ref: https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/CreatingLaunchdJobs.html

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 8
  • Comments: 17 (2 by maintainers)

Commits related to this issue

Most upvoted comments

Additional instructions for .plist editing:

  1. Install service sudo cloudflared service install (you MUST use sudo, as the intention here is to run cloudflared at boot per the docs)
  2. Edit .plist file
  3. sudo launchctl stop com.cloudflare.cloudflared
  4. sudo launchctl unload /Library/LaunchDaemons/com.cloudflare.cloudflared.plist
  5. sudo launchctl load /Library/LaunchDaemons/com.cloudflare.cloudflared.plist
  6. The service should start upon launchctl load, but you can also sudo launchctl start com.cloudflare.cloudflared

To troubleshoot launchctl: launchctl dumpstate > myfile.txt then open file and search for string cloudflared – this is how I realized only the command cloudflared was executing and not cloudflared tunnel run.

It was frustrating to be told by Cloudflare Support that the product team did not consider this something requiring action to resolve. I also see the same intention in this PR response.

Our Argo Tunnel product team feedback: this is broken at higher level as there’s no common service mode that works across all features. We picked “client” mode for windows and macOS and the “tunnel” mode for Linux. Therefore the bug is that we had to pick, it is not the choices we made.

There won’t be a fix planned for this. What’ve you done is the workaround which you figured out already.

My last message to Cloudflare Support:

  1. I was not aware anything was modal – will the existence of “client” and “tunnel” modes be clearly documented?

  2. Has the product team considered adding a “–mode” flag to the “cloudflared service install” command? As well as making the OS/platform differences clear in the cli help?

I absolutely love the Argo Tunnel product, and use it in production on multiple projects. However the learning curve was extremely steep and full of trial-and-error, due to sharp edges such as this issue (#327). That learning curve is the reason I have not extended my use of Argo Tunnel and cloudflared to other teams in my organization.

Thank you!

I did a bit more to have it working as service/daemon on macOS, with help of @adamfeldman findings:

# setup
sudo mkdir /etc/cloudflared
sudo cp ~/.cloudflared/*.json /etc/cloudflared
sudo cp ~/.cloudflared/cert.pem /etc/cloudflared
sudo cp ~/.cloudflared/config.yml /etc/cloudflared
sudo sed -i.bak 's/\/Users\/'${USER}'\/\./\/etc\//g' /etc/cloudflared/config.yml

# check
sudo ls -la /etc/cloudflared

# install
sudo cloudflared service install

# check
sudo ls -la /Library/LaunchDaemons | grep cloudflared
sudo ls -la /Library/Logs | grep cloudflared
sudo tail /Library/Logs/com.cloudflare.cloudflared.err.log

# update
sudo launchctl unload /Library/LaunchDaemons/com.cloudflare.cloudflared.plist
sudo patch -p 1 /Library/LaunchDaemons/com.cloudflare.cloudflared.plist < cloudflared.plist.patch
sudo launchctl load -w /Library/LaunchDaemons/com.cloudflare.cloudflared.plist

# check
ps aux | grep cloudflared
sudo tail /Library/Logs/com.cloudflare.cloudflared.err.log

cloudflared.plist.patch:

--- /Library/LaunchDaemons/com.cloudflare.cloudflared.plist     2022-02-17 10:08:01.000000000 +0100
+++ /tmp/com.cloudflare.cloudflared.plist       2022-02-17 10:11:47.000000000 +0100
@@ -7,6 +7,8 @@
                <key>ProgramArguments</key>
                <array>
                        <string>/opt/homebrew/bin/cloudflared</string>
+                       <string>tunnel</string>
+                       <string>run</string>
                </array>
                <key>RunAtLoad</key>
                <true/>
@@ -22,4 +24,4 @@
                <key>ThrottleInterval</key>
                <integer>5</integer>
        </dict>
-</plist>
\ No newline at end of file
+</plist>

cloudflared version 2023.5.1 (built 2023-05-23-1847 UTC)

note: This is still an issue when you install as a service and required

  1. manual copy & paste of configurations (config.json, pem, {uuid}.json) from ~/.cloudflared/* to /etc/cloudflared/* << not sure this step is even doing anything, as suggested in the documentation. WEIRD.
  2. manual adding of “tunnel run” xml into plist with sudo.

Otherwise my log.err was full of

Use `cloudflared tunnel run` to start tunnel {UUID}
Use `cloudflared tunnel run` to start tunnel {UUID}
Use `cloudflared tunnel run` to start tunnel {UUID}
...

Until editing .plist manually according to the comment above. I edited the plist using sudo vi by hand.

Even the url given when running a tunnel is broken:

This is going to be fixed in the next release

As for this whole thread/issue, we’ll have it fixed in the scope of a big new feature. Stay tuned.

Thanks for your help @adamfeldman - a big help on something that I’d have expected Cloudflare to catch.

Also note (for other struggling with this) that the value of the first <string> item might be /opt/homebrew/bin/cloudflared if HomeBrew was used for the installation of Cloudflared as suggested in the official docs