cloudflared: Segmentation fault on raspberry pi 2 model b

Here: https://bin.equinox.io/a/4SUTAEmvqzB/cloudflared-2018.7.2-linux-arm.tar.gz

Recompiled binaries are available here https://hobin.ca/cloudflared/ . I can confirm that 2019.8.1 runs fine on my Raspberry Pi Zero. I installed using the normal method in the cloudflared docs but replaced

wget https://bin.equinox.io/c/VdrWdbjqyF/cloudflared-stable-linux-arm.tgz

with

wget https://hobin.ca/cloudflared/releases/2019.8.1/cloudflared_2019.8.1_arm.tar.gz

Hope this helps others.

EDIT: I’m just passing along a solution I found, I had no part in generating these files and don’t take any credit for them.

Same on Raspberry Pi Zero and Raspberry Pi Zero W.

It would be great if the author of those binaries shared how they got it to compile. I tried compiling it for my Raspberry Pi 1B, with my limited knowledge of golang, to no avail.

@Limeth Author of those binaries here. Building the binaries for ARMv6 is rather straightforward. You just have to make sure you have a C cross-compiler for ARM, such as gcc-arm-linux-gnueabi for Debian-based systems. Then set the following environment variables…

GOOS=linux
GOARCH=arm
GOARM=6
CGO_ENABLED=1
CC=arm-linux-gnueabi-gcc # This should be your ARM C cross-compiler

Now just build it…

go build -v "-ldflags=-X 'main.Version=${version}' -X 'main.BuildTime=${date}'" github.com/cloudflare/cloudflared/cmd/cloudflared

…where ${version} is the version of Cloudflared and ${date} is the current time.

Then just bundle the resulting executable however you want.

Thanks to @meliurwen for pointing me here from reddit.

Edit: Rather than building it with go build ..., you should be able to use their Makefile as well. I’m building it the way described above, however. Also worth noting, if you build on your Pi itself, you could probably skip the environment variables (the defaults target your build machine).

Last edit: Quick word on reproducibility, in case anyone is trying to reproduce these binaries byte-for-byte for the sake of auditing. Go builds are only reproducible if built with the same GOPATH and GOROOT. So those values for me (as of version 2019.8.3) are…

GOPATH=/home/hobindar/go
GOROOT=/usr/lib/go-1.13

You’ll obviously also need to use the same version and date string that I used. You can get this string by running cloudflared version, which will print out a line like cloudflared version foo (built bar). In this example, foo is the version string and bar is the date string.

More information about byte-for-byte reproducibility in Go can be found here: https://blog.filippo.io/reproducing-go-binaries-byte-by-byte/

Last edit for real: The following (mentioned in an earlier comment) will grab version 2019.8.1…

wget https://hobin.ca/cloudflared/releases/2019.8.1/cloudflared_2019.8.1_arm.tar.gz

You could, however, grab the latest (regardless of version) using…

wget https://hobin.ca/cloudflared/latest?type=tar

…replacing “tar” with the package type of your choice (i.e. “deb”, “rpm”, or “tar”).

This more closely mimics the behavior of the equinox link in their documentation (by grabbing latest).

Hi from Cloudflare. We’ve made some improvements to packaging and DoH issues. That said, we still haven’t been able to track down the cause of the issues with Raspberry Pi devices. I know it’s frustrating.

We have the PiHole DoH issues on our list, but don’t have an ETA yet when you could expect a fix. We’ll update this thread and the related ones when we do. Thanks for the patience and feedback - apologies for the delay here.

If anyone is interested, I have created a Dockerfile for building cloudflared on my Mac. https://github.com/reshnm/cloudflared-build

You can build cloudflared by calling ./build.sh <version> e.g. ./build.sh 2019.11.3

I see many people started to use the binaries linked by @cemtes and I’m a little worried. The author of those binaries (reddit username probably DTHCND) didn’t provide documentation to reproduce its build process.

Please be really careful and if possible restrain from using untrusted or unknown binary blobs.

Especially for tools like cloudflared, where one of its core features is privacy and distrust of the actors handling/forwarding your DNS requests between you and the Cloudflare’s servers. Using those binaries, unless for whatever reason you trust this person, makes the use of this tool almost (if not completely) pointless.

@sssilver Since there has been a new release, does that mean this issue is fixed?

Does not appear to have been fixed. I tried the current release today and it’s still segfaulting.

Who maintains this?

That would be me.

I mean, this is supposed to use a secure protocol to make queries, how safe is this build archive?

In my opinion, it’s definitely safe. I left a comment earlier in this thread about how to build it yourself. If you follow all the instructions in that comment, I believe the resulting binaries should be bit-for-bit identical, allowing you to verify that I did not do anything malicious in my builds. If you ever find they are not identical, let me know and we can investigate why.

The main page doesn’t makes me feel precisely comfortable.

Haha, yeah, I should probably change that page at some point. That’s just a page that I hacked together many years ago to try out viewport units, back when viewport units were a brand new thing (and while I was still a university student). I’ll replace the page when I think of something worthwhile to replace it with.

CC: @brbergami

Hi,

The next cloudflared release will be built with Go 1.11.

Exactly the same issue as well. It’s also being reported elsewhere. Reverting to 2018.7.2 works…

the issue is 2018.7.3

+1 for same issue on a RPi Model B; 2018-7.2 works but I keep randomly running into this (https://github.com/cloudflare/cloudflared/issues/23) same error with 2018-7.2. failed to connect to an HTTPS backend and then have to reboot the Pi.

Would love to get this fixed to try a later version of Cloudflared that hopefully resolves that problem 😃

Still happening in Pi 2 Model B

@xyzulu how do you revert to 2018.7.2? I can’t find the binaries for download.

Same issue on: Raspberry Pi Model B Rev 2 (/proc/cpuinfo)

On trying to compile from source, I think the golang version is too old in Debian Buster (reading some errors) - so I compiled the golang backports dpkg source version to ensure no ABI issue on the machine as I haven’t set up or tested a cross compiler yet. This took more than one attempt and several attempts for tests to pass, and they never did during the build process so I overrided to create the deb anyway and manually verified the failing tests (due to timeout). I probably should have checked if Raspbian did anything to that package prior to starting… too late!

go version go1.14 linux/arm

Then compiling cloudflared go install -work -v -buildmode=pie ./cmd/cloudflared

and I have a working executable… although I’m new to cloudflared, so I haven’t really got to the stage of testing anything yet.

Did anyone else do this? Are my build settings different from the release package? can we git bisect? (once I have a cross compiler working… this is far too slow on the Pi!)

I can confirm this issue still happens. I’m using a Zero W. Had to find a 2018 build that works. But if you diff, there’s a ton of commits added (which I guess includes features and bug fixes), can’t be 2 years and this still persist.

root@DietPi:~# uname -a
Linux DietPi 5.4.79+ #1373 Mon Nov 23 13:18:15 GMT 2020 armv6l GNU/Linux
root@DietPi:~# lsb_release -a
No LSB modules are available.
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 10 (buster)
Release:        10
Codename:       buster
root@DietPi:~# cat /proc/cpuinfo
processor       : 0
model name      : ARMv6-compatible processor rev 7 (v6l)
BogoMIPS        : 1423.06
Features        : half thumb fastmult vfp edsp java tls
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x0
CPU part        : 0xb76
CPU revision    : 7

Hardware        : BCM2835
Revision        : 9000c1
Serial          : 000000001ea488ff
Model           : Raspberry Pi Zero W Rev 1.1

Working build: b2b46a483505babd15743d3743728ebf5a1b82ed8e47093a2009ae6e48c83034

Still happening pi b+

Some problem raspberry pi zero W.

What Solution?

Update: I got it working! See complete steps below. 🙂

-bash: ./cloudflared: cannot execute binary file: Exec format error

version=2018.12.1
date=$(date)
GOOS=linux
GOARCH=arm
GOARM=6
CGO_ENABLED=1
CC=arm-linux-gnueabi-gcc

git clone git@github.com:cloudflare/cloudflared.git ~/go/src/github.com/cloudflare/cloudflared

go build -v "-ldflags=-X 'main.Version=${version}' -X 'main.BuildTime=${date}'" github.com/cloudflare/cloudflared/cmd/cloudflared

On Ubuntu 18.04, here’s how I managed to compile cloudflared for my Raspberry Pi Zero W:

Install Go for Ubuntu as described on the Golang wiki.

Check go version, it should print go version go1.13.3 linux/amd64.

Run the following:

# install required CC
sudo apt install gcc-arm-linux-gnueabi

# clone cloudflared source where Go can find it
git clone git@github.com:cloudflare/cloudflared.git ~/go/src/github.com/cloudflare/cloudflared

# build (will create cloudflared binary in current directory)
version=2018.12.1 date=$(date) GOOS=linux GOARCH=arm GOARM=6 CGO_ENABLED=1 CC=arm-linux-gnueabi-gcc go build -v "-ldflags=-X 'main.Version=${version}' -X 'main.BuildTime=${date}'" github.com/cloudflare/cloudflared/cmd/cloudflared

Then copy the cloudflared file to your Pi (e.g. with SSH) and run ./cloudflared -v to make sure it’s working.

My mistake before was that I forgot to export the env variables, or to pass them in on the same line where I run the go build command. So it was just building for Linux amd64 instead of for the Pi’s architecture. Thanks @hobindar!

I used go1.12.5 linux/arm. I think that was the latest at the time.

Edit: You will probably need to increase the swap size.

It’s possible to compile it on a Pi 1B, you need to install a recent version of the Go compiler, clone the cloudflared repository, checkout the release you want, and use the Makefile. See my comment above https://github.com/cloudflare/cloudflared/issues/38#issuecomment-504482367. The version from Raspbian may not be recent enough.

@meliurwen For what it’s worth, this was my first thought as well, so I’ve been keeping an eye on the system I installed them on over the last two weeks. So far they haven’t made a single network request outside of the Cloudflare DNS servers and they’re not secretly storing requests to disk. Obviously this is not a through audit, but based on that and a bit of background on the person who made them available, I feel they’re safe.

Obviously everyone should make up their own mind and do their own testing (for all you know I could be in on it) and I do agree with the point you’re making. If you do use these I would recommend setting up a system to monitor outbound network traffic so you can make sure they’re not doing anything nefarious.

@cemtes Thanks! Works great on my Pi B+!

Anyone have any method to get it working on RPi Zero? I tried downloading different versions and they work without giving a segmentation fault. But, if I restart my RPi or after a while, it just stops working.

I think the segfault issue is due to released cloudflared being built on a later ARM processor, or cross compiled. The latest current version (2019.5.0) works fine on a Pi 3 B+, whether it stops working after a while (#23), I don’t know, didn’t test it for long enough, but segfault on a Pi 1B (BCM2835). I installed go1.12.5 linux/arm on the Pi 1B and built from source, release 2019.5.0 plus a few commits (babcd9f), this appears to work, but stops working after a while (#23), it appears to run out of file handles.

Jun  3 06:33:14 jennifer cloudflared[261]: time="2019-06-03T06:33:14Z" level=error msg="failed to connect to an HTTPS backend \"https://1.1.1.1/dns-query\"" error="failed to perform an HTTPS request: Post https://1.1.1.1/dns-query: dial tcp 1.1.1.1:443: socket: too many open files"
Jun  3 06:33:14 jennifer cloudflared[261]: time="2019-06-03T06:33:14Z" level=error msg="failed to connect to an HTTPS backend \"https://1.0.0.1/dns-query\"" error="failed to perform an HTTPS request: Post https://1.0.0.1/dns-query: dial tcp 1.0.0.1:443: socket: too many open files"
Jun  3 06:33:14 jennifer cloudflared[261]: time="2019-06-03T06:33:14Z" level=error msg="failed to connect to an HTTPS backend \"https://1.1.1.1/dns-query\"" error="failed to perform an HTTPS request: Post https://1.1.1.1/dns-query: dial tcp 1.1.1.1:443: socket: too many open files"
Jun  3 06:33:14 jennifer cloudflared[261]: time="2019-06-03T06:33:14Z" level=error msg="failed to connect to an HTTPS backend \"https://1.0.0.1/dns-query\"" error="failed to perform an HTTPS request: Post https://1.0.0.1/dns-query: dial tcp 1.0.0.1:443: socket: too many open files"

Update: Having looked at the Makefile, I built version 2019.5.0 using it…

pi@jennifer:~ $ cd gocode/src/github.com/cloudflare/cloudflared/
pi@jennifer:~/gocode/src/github.com/cloudflare/cloudflared $ git checkout 2019.5.0
Note: checking out '2019.5.0'.
...
HEAD is now at 4bff1ef... Release 2019.5.0
pi@jennifer:~/gocode/src/github.com/cloudflare/cloudflared $ export PATH=$PATH:/usr/local/go/bin
pi@jennifer:~/gocode/src/github.com/cloudflare/cloudflared $ export GOPATH=~/gocode
pi@jennifer:~/gocode/src/github.com/cloudflare/cloudflared $ go clean
pi@jennifer:~/gocode/src/github.com/cloudflare/cloudflared $ make cloudflared
go build -v -ldflags='-X "main.Version=2019.5.0" -X "main.BuildTime=2019-06-03-1717 UTC"' github.com/cloudflare/cloudflared/cmd/cloudflared
pi@jennifer:~/gocode/src/github.com/cloudflare/cloudflared $ ./cloudflared -v
cloudflared version 2019.5.0 (built 2019-06-03-1717 UTC)

This version may be ok, it’s now been running for several hours including a Pi reboot and a router reboot.

cloudflared version 2019.4.1 (built 2019-04-19-2152 UTC) (official binary download)) Works on Raspberry Pi 3b+

+1 on all of the above, including falling back to 2018-7.2 works.

Same issue, same device

I have also reproduced this issue on my Raspberry Pi Model 1B (rev 2). Reverting to 2018.7.2 as xyzulu suggested works.