pact-go: Library not loaded: libpact_ffi.dylib

Apologies in advance for a long post. This issue started happening to me seemingly from the latest MacOS update, but might’ve been caused by some other changes on the system. I’m wondering if this is local to my system or if others have experienced this too?

Basically just trying to run go test ... with pact-go causes issues loading libpact_ffi.dylib:

dyld[13198]: Library not loaded: libpact_ffi.dylib
  Referenced from: <097B1871-BF1B-3A20-B6CE-B3EE8D6C100B> /private/var/folders/vk/9xmgp9w16dj3ty41013x0zt80000gn/T/go-build92553240/b001/consumer.test
  Reason: tried: 'libpact_ffi.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibpact_ffi.dylib' (no such file), 'libpact_ffi.dylib' (no such file), '/Users/stanislav.vodetskyi/github/pact-go/consumer/libpact_ffi.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/stanislav.vodetskyi/github/pact-go/consumer/libpact_ffi.dylib' (no such file), '/Users/stanislav.vodetskyi/github/pact-go/consumer/libpact_ffi.dylib' (no such file)
FAIL    github.com/pact-foundation/pact-go/v2/consumer  0.474s

Note how the log doesn’t mention looking at /usr/local/lib or /usr/lib. I’m not 100% sure, but I suspect that previously DYLD_FALLBACK_LIBRARY_PATH was set to a default value of /usr/local/lib:/usr/lib, or dyld had these defaults hardcoded if the variable was unset, but not anymore. So it used to be able to find libpact_ffi.dylib in fallback path, but not anymore.

Some more details.

pact-go installs libpact_ffi.dylib under /usr/local/lib by default. pact-go also invokes install_name_tool -id libpact_ffi.dylib /usr/local/lib/libpact_ffi.dylib. This sets the id / install name of the lib to libpact_ffi.dylib. So whenever I link against this library (e.g. via go test -c ...) it looks like this (interestingly linking works ok, not sure which vars control the lookup paths there):

otool -L pact.test
pact.test:
        libpact_ffi.dylib (compatibility version 0.0.0, current version 0.0.0)

(not sure why the version is 0.0.0, looks weird and suspicious, but I’ve tried two latest libpact_ffi releases and both look like that) so when dyld actually links to it at runtime (and assuming dyld_library_path is not set), it will search by this name first (since it’s not absolute path, it’ll check current dir) or check fallback path.

Search by name fails, because our current work dir is not /usr/local/lib and there’s no such file. And fallback path is empty by default in my case, I suspect because of the latest MacOS update, so it doesn’t even try to search there. This corresponds to the error message I’m seeing, where it only searches current path (and some weird paths that ultimately look like current path + some security nonsense)

I was able to resolve it locally by:

• either ensure DYLD_FALLBACK_LIBRARY_PATH is set to /usr/local/lib - tricky to do for go test, since it requires using -exec flag, as described here - https://forum.golangbridge.org/t/go-test-with-cgo-on-macos-and-dyld-library-path/32274/2
• or calling install_name_tool -id /usr/local/lib/libpact_ffi.dylib /usr/local/lib/libpact_ffi.dylib - now when linking to this lib, the resulting binary would link to a full path, rather than just the file name, removing the need to search in fallback dir

I don’t have an older MacOS version to confirm if DYLD_FALLBACK_LIBRARY_PATH was set (or if dyld had a hardcoded value to use when it wasn’t set).

I can send a PR to pact-go to change install_name_tool invocation, but I’m not sure why it was setting the name without a full path in the first place, and since I don’t fully understand it, I’m worried I might break some use case unknown to me. Could it be that older pact ffi dylibs had a wrong id set and the call to install_name_tool is just to correct it?

One more thing that I’m still confused about is how it was supposed to work when custom --libDir value was passed to pact-go install - how would that custom library be found without modifying DYLD_* variables or providing a full path in install_name_tool? E.g. in the Makefile of pact-go itself. Is /tmp searched by linker and loader by default?

Software versions

• OS: e.g. Mac OSX 13.6 (22G120)
• Golang Version: go1.20.8 (also tried with go1.21)
• Golang environment:

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/stanislav.vodetskyi/Library/Caches/go-build"
GOENV="/Users/stanislav.vodetskyi/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/stanislav.vodetskyi/go/pkg/mod"
GONOPROXY="github.com/confluentinc/*"
GONOSUMDB="github.com/confluentinc/*"
GOOS="darwin"
GOPATH="/Users/stanislav.vodetskyi/go"
GOPRIVATE="github.com/confluentinc/*"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.20.8"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/vk/9xmgp9w16dj3ty41013x0zt80000gn/T/go-build2646031252=/tmp/go-build -gno-record-gcc-switches -fno-common"

Relevent log files

pact-go install is successful:

2023/09/27 00:17:15 [INFO] package libpact_ffi not found
2023/09/27 00:17:15 [INFO] downloading library from https://github.com/pact-foundation/pact-reference/releases/download/libpact_ffi-v0.4.5/libpact_ffi-osx-x86_64.dylib.gz to /usr/local/lib/libpact_ffi.dylib
&{}
2023/09/27 00:17:17 [DEBUG] obtaining hash for file /usr/local/lib/libpact_ffi.dylib
2023/09/27 00:17:17 [DEBUG] writing config {map[libpact_ffi:{libpact_ffi 0.4.5 5971854e21bf8b41db8b821a77caa268}]}
2023/09/27 00:17:17 [DEBUG] writing yaml config to file libraries:
  libpact_ffi:
    libname: libpact_ffi
    version: 0.4.5
    hash: 5971854e21bf8b41db8b821a77caa268

2023/09/27 00:17:17 [INFO] setting install_name on library libpact_ffi for osx
2023/09/27 00:17:17 [DEBUG] output from command []
2023/09/27 00:17:17 [INFO] package libpact_ffi found
2023/09/27 00:17:17 [INFO] checking version 0.4.5 of libpact_ffi against semver constraint >= 0.4.0, < 1.0.0
2023/09/27 00:17:17 [DEBUG] 0.4.5 satisfies constraints 0.4.5 >= 0.4.0, < 1.0.0
2023/09/27 00:17:17 [INFO] package libpact_ffi is correctly installed
2023/09/27 00:17:17 [DEBUG] skip checking ffi version() call because FFI not loaded. This is expected when running the 'pact-go' command