go: runtime/cgo: fails to build after updating to Mojave

What version of Go are you using (go version)?

go version go1.11 darwin/amd64
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/foo/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Volumes/DailySD/SD_Dev"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/fj/yby80m4x1pq1nzvfvd1hplhw0000gn/T/go-build110153853=/tmp/go-build -gno-record-gcc-switches -fno-common"

GCC

Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/c++/4.2.1
Apple LLVM version 10.0.0 (clang-1000.11.45.2)
Target: x86_64-apple-darwin18.0.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

What did you do?

update macOS to mojave, install go, vscode for Developement.

An error that did not get offended when go test was done within an existing project occurred. It is below.

# runtime/cgo
_cgo_export.c:3:10: fatal error: 'stdlib.h' file not found

If possible, provide a recipe for reproducing the error. A complete runnable program is good. A link on play.golang.org is best.

What is missing, please help me.

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 21
  • Comments: 59 (8 by maintainers)

Most upvoted comments

open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

Shortly the way it worked on macOS Catalina in my case:

export CPATH="/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/"
export CGO_ENABLED=1; export CC=gcc;

if someone facing this issue in catalina (10.15) try this. export SDKROOT="$(xcrun --sdk macosx --show-sdk-path)"

open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

This does the magic.

Found the solution with this one

if someone facing this issue in catalina (10.15) try this. export SDKROOT="$(xcrun --sdk macosx --show-sdk-path)"

Thanks a lot! This work for me!

FYI - Had the same issue, tried everything above

However, the file, mentioned in comments above…

/Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

seems to be no longer available, with the latest version of xcode, see the New Features section here https://developer.apple.com/documentation/xcode_release_notes/xcode_10_release_notes


Solution:

Running brew doctor, as per the instructions below…

Warning: Unbrewed header files were found in /usr/local/include.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.

Unexpected header files:
  /usr/local/include/abstract.h
  /usr/local/include/aes.h
  /usr/local/include/anchor.h
etc...

So, using the script provided in the link below, which simply moves the files one at a time to another folder, fixed the issue for me https://github.com/SOHU-Co/kafka-node/issues/881#issuecomment-396197724

Repeated below…

mkdir /tmp/includes brew doctor 2>&1 | grep “/usr/local/include” | awk ‘{$1=$1;print}’ | xargs -I _ mv _ /tmp/includes

You can try xcode-select --install

So what I conclude is that cgo defaults to using clang as the compiler, and since I have installed clang from Homebrew and added it to my PATH, this causes the error. Therefore, the problem lies with either Homebrew or LLVM – not Go – and somebody should open an issue against one of those projects.

@raxod502 Thanks for the advice. In this case, just unlinking LLVM temporally fixes this issue:

brew unlink llvm@4

That is what I thought. However, I just did some more testing and it appears that I was wrong.

  • The /usr/bin/gcc that ships with macOS can find the standard headers.
  • The /usr/bin/clang that ships with macOS can find the standard headers.
  • The /usr/local/bin/gcc-9 that is installed by Homebrew can find the standard headers.
  • The /usr/local/opt/llvm/bin/clang that is installed by Homebrew cannot find the standard headers.

(It did not occur to me that the behavior might be different between different compilers.)

So what I conclude is that cgo defaults to using clang as the compiler, and since I have installed clang from Homebrew and added it to my PATH, this causes the error. Therefore, the problem lies with either Homebrew or LLVM – not Go – and somebody should open an issue against one of those projects.

I’m sorry for spreading misinformation here. Thanks for your help.

Solution: go to https://developer.apple.com/download/more/ (Apple Developer account required) and download “Command Line Tools (macOS 10.14) for Xcode 10.2.1” (or equivalent for your OS version). Extract that package, which installs the macOS_SDK_headers_for_macOS_10.14.pkg package to /Library/Developer/CommandLineTools/Packages, which you can then extract separately. That finally gets you your /usr/include back.

It works for me because I’ve used clang in anaconda. (macOS Big Sur 11.1)

which clang

/Users/username/anaconda3/bin/clang

rm /Users/username/anaconda3/bin/clang
which clang

/usr/bin/clang

Removing brew llvm helped me. brew remove llvm I think it was not searching for the headers in the right spot.

I have the same issue and xcode-select --install did not solve the problem

It works for me (Big Sur 11.1)

xcode-select --install
brew remove llvm

It works for me because I’ve used clang in anaconda. (macOS Big Sur 11.1)

which clang

/Users/username/anaconda3/bin/clang

rm /Users/username/anaconda3/bin/clang
which clang

/usr/bin/clang

This fixed my issue. The path for Clang was coming up as /user/local/bin/clang, instead of /user/bin/clang, guessing because I had once used Homebrew to install a version of Clang.

on mojave 10.14.4, it works. command line> open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

Yes, there is really nothing that can be done in the Go tools. The Go tools never read system headers themselves, and they never read /usr/include. Instead, they invoke the C compiler, and expect the C compiler to read those headers. You can see the C compiler that they will invoke by using go env CC, and you can see precisely what the Go tool does by using go build -x.

if someone facing this issue in catalina (10.15) try this. export SDKROOT="$(xcrun --sdk macosx --show-sdk-path)"

Thanks a lot! This work for me!

set Environment Varibles in xcode clang target arguments:

SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk

worked for me!

reinstall x-code commandline tools :

sudo rm -rf  /Library/Developer/CommandLineTools
sudo xcode-select --install

and then

 open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

it work for me !

System headers have moved

Apple have changed the default location of the system headers with macOS 10.14, Mojave, and Xcode 10:

Xcode 10 Release Notes

The Command Line Tools package installs the macOS system headers inside the macOS SDK. Software that compiles with the installed tools will search for headers within the macOS SDK provided by either Xcode at:

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk

or the Command Line Tools at:

/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk

depending on which is selected using xcode-select. The command line tools will search the SDK for system headers by default. However, some software may fail to build correctly against the SDK and require macOS headers to be installed in the base system under /usr/include. If you are the maintainer of such software, we encourage you to update your project to work with the SDK or file a bug report for issues that are preventing you from doing so. As a workaround, an extra package is provided which will install the headers to the base system. In a future release, this package will no longer be provided. You can find this package at:

/Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

To make sure that you’re using the intended version of the command line tools, run xcode-select -s <path to Xcode> or xcode select -s /Library/Developer/CommandLineTools after installing.

open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

it works.

@ianlancetaylor I understand what you are saying, but I think Apple’s implication is that the Go build system needs to pass an extra -I argument to the C compiler with the path to the macOS SDK. Yes, that’s completely ridiculous – but what is the alternative? Every single user of Go on macOS needs to export C_INCLUDE_PATH, or …?

It is an unfortunate fact that the C compiler cannot find stdlib.h on macOS anymore, at least by default. So while it makes perfect sense to say “it is your own responsibility to make sure the compiler can find stdlib.h”, the sad truth is that this means every single Go developer on macOS would be required to take manual steps to get Go working properly.

An alternative solution would be for Homebrew to do something to fix this problem in general, but I don’t know what the options look like for that.

Unfortunately, that package appears to no longer exist as of a recent update. What can I do?

% ls -lA /usr/include
ls: /usr/include: No such file or directory
% ls -lA /Library/Developer/CommandLineTools
total 0
drwxr-xr-x  6 root  admin  192 Jun  4 09:08 Library
drwxr-xr-x  6 root  wheel  192 Jun  4 09:08 SDKs
drwxr-xr-x  7 root  admin  224 Apr  5 17:30 usr
% ls -lA /Library/Developer/CommandLineTools/Packages
ls: /Library/Developer/CommandLineTools/Packages: No such file or directory
% xcode-select --install
xcode-select: error: command line tools are already installed, use "Software Update" to install updates

@ianlancetaylor I totally am on your side here, but unfortunately Apple has made it clear they want every project to update their build tooling to work with the weird new SDK system:

some software may fail to build correctly against the SDK and require macOS headers to be installed in the base system under /usr/include. If you are the maintainer of such software, we encourage you to update your project to work with the SDK or file a bug report for issues that are preventing you from doing so.

So unfortunately I think this is a bug in Go, unless you want to drop support for macOS.

open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

This fix it for me!

open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

Nice

I was seeing the same issue but that was due to GCC, which I install using apk add build-base