go: runtime: can't call from Go to c-shared Go library on Darwin

What version of Go are you using (go version)?

$ go version
go version go1.13.10 darwin/amd64

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/trankhai/Library/Caches/go-build"
GOENV="/Users/trankhai/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/trankhai/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/trankhai/Documents/GitHub/2hat_contest2/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/2_/9794qxcj4l56p3k1z5_y9v040000gn/T/go-build055049177=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I wrote a program that combines golang vs python. I call from golang to python and from python call to golang. It works great in Ubuntu but can’t work in macos.

Then run the program in macOS Catalina v10.15.3:

fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x6238648]

goroutine 20 [running, locked to thread]:
runtime.throw(0x62f3e2d, 0x2a)
	/usr/local/go/src/runtime/panic.go:774 +0x72 fp=0xc0000e71d8 sp=0xc0000e71a8 pc=0x620d382
runtime.sigpanic()
	/usr/local/go/src/runtime/signal_unix.go:378 +0x47c fp=0xc0000e7208 sp=0xc0000e71d8 pc=0x621ffbc
runtime.memmove(0x0, 0xc00006f800, 0x1)
	/usr/local/go/src/runtime/memmove_amd64.s:146 +0x108 fp=0xc0000e7210 sp=0xc0000e7208 pc=0x6238648
runtime.heapBitsSetType(0xc00006f800, 0x400, 0x400, 0x62d5640)
	/usr/local/go/src/runtime/mbitmap.go:1400 +0x45d fp=0xc0000e72e8 sp=0xc0000e7210 pc=0x61f66fd
runtime.mallocgc(0x400, 0x62d5640, 0xc0000d4001, 0x8)
	/usr/local/go/src/runtime/malloc.go:1052 +0x53e fp=0xc0000e7388 sp=0xc0000e72e8 pc=0x61ede1e
runtime.makeslice(0x62d5640, 0x8, 0x8, 0x0)
	/usr/local/go/src/runtime/slice.go:49 +0x6c fp=0xc0000e73b8 sp=0xc0000e7388 pc=0x6220b2c
sync.(*Pool).pinSlow(0x63bbce0, 0x0, 0x0)
	/usr/local/go/src/sync/pool.go:227 +0xfa fp=0xc0000e7458 sp=0xc0000e73b8 pc=0x623fefa
sync.(*Pool).pin(0x63bbce0, 0x1, 0xc0000e74d8)
	/usr/local/go/src/sync/pool.go:206 +0x5e fp=0xc0000e7480 sp=0xc0000e7458 pc=0x623fdce
sync.(*Pool).Get(0x63bbce0, 0x1c025ae5de0, 0xc0000e7558)
	/usr/local/go/src/sync/pool.go:128 +0x2f fp=0xc0000e74c8 sp=0xc0000e7480 pc=0x623fa8f
encoding/json.newEncodeState(0x1c01e2dad20)
	/usr/local/go/src/encoding/json/encode.go:286 +0x31 fp=0xc0000e74f0 sp=0xc0000e74c8 pc=0x629e151
encoding/json.Marshal(0x62ca280, 0x1c0000921e0, 0xc0000e75f8, 0x62ca280, 0x1c0000921e0, 0x40cdb01, 0xc0001121b0)
	/usr/local/go/src/encoding/json/encode.go:159 +0x26 fp=0xc0000e7568 sp=0xc0000e74f0 pc=0x629da66
main.GetTopics(0x60369b4, 0x4, 0x0)
	/Users/trankhai/Documents/GitHub/2hat_contest2/hybrid_go.go:46 +0xf9 fp=0xc0000e7618 sp=0xc0000e7568 pc=0x62ad369
main._cgoexpwrap_d079111cff13_GetTopics(0x60369b4, 0x4, 0x0)
	_cgo_gotypes.go:64 +0x72 fp=0xc0000e7678 sp=0xc0000e7618 pc=0x62acf32
runtime.call32(0x0, 0x7ffeefbff1b0, 0x7ffeefbff240, 0x18)
	/usr/local/go/src/runtime/asm_amd64.s:539 +0x3b fp=0xc0000e76a8 sp=0xc0000e7678 pc=0x62356cb
runtime.cgocallbackg1(0x0)
	/usr/local/go/src/runtime/cgocall.go:314 +0x1b7 fp=0xc0000e7790 sp=0xc0000e76a8 pc=0x61e5847
runtime.cgocallbackg(0x0)
	/usr/local/go/src/runtime/cgocall.go:191 +0xc1 fp=0xc0000e77f8 sp=0xc0000e7790 pc=0x61e55f1
runtime: unexpected return pc for runtime.cgocallback_gofunc called from 0x405ae42
stack: frame={sp:0xc0000e77f8, fp:0xc0000e7818} stack=[0xc0000e6000,0xc0000e8000)
000000c0000e76f8:  0000000000000006  00000000000001ff 
000000c0000e7708:  0000000000000008  000000c0000e76a8 
000000c0000e7718:  00000000061e57f5 <runtime.cgocallbackg1+357>  00000000062f5ba0 
000000c0000e7728:  0000000000000000  000000c0000e7980 
000000c0000e7738:  000000c0000e76ef  0000000000000002 
000000c0000e7748:  0000000000000019  000000c0000e7780 
000000c0000e7758:  0000000006215ed0 <runtime.exitsyscall+416>  000000c000078300 
000000c0000e7768:  0000000200000003  000000c000078300 
000000c0000e7778:  000000c000078300  000000c0000e77e8 
000000c0000e7788:  00000000061e55f1 <runtime.cgocallbackg+193>  0000000000000000 
000000c0000e7798:  0000000004005f65  0000000004004d9b 
000000c0000e77a8:  0000000000000000  0000000000000000 
000000c0000e77b8:  0000000000000000  0000000000000000 
000000c0000e77c8:  0000000000000000  0000000000000000 
000000c0000e77d8:  000000c0000e7820  000000c000078300 
000000c0000e77e8:  00007ffeefbff170  0000000006236c9b <runtime.cgocallback_gofunc+155> 
000000c0000e77f8: <0000000000000000  000000000458c5c0 
000000c0000e7808:  000000c0000e7820 !000000000405ae42 
000000c0000e7818: >0000000004004dbf  0000000004285c60 
000000c0000e7828:  000000c0000e7888  000000c0000e7878 
000000c0000e7838:  0000000004005405  000000000458c5c0 
000000c0000e7848:  000000c0000e7878  00000000040e455e 
000000c0000e7858:  0000000004285c60  000000c0000e7888 
000000c0000e7868:  000000c000026500  000000c0000e7888 
000000c0000e7878:  000000c0000e78c8  00000000040e5d26 
000000c0000e7888:  0000000006033e60  00000000057e0690 
000000c0000e7898:  00000000057e8398  0000000000000000 
000000c0000e78a8:  0000000000000000  00000000057e8398 
000000c0000e78b8:  00000000057e0690  0000000006033e60 
000000c0000e78c8:  000000c0000e7900  00000000040e5283 
000000c0000e78d8:  000000c0000d4050  000000c0000d4058 
000000c0000e78e8:  000000c0000d4068  000000c0000d4068 
000000c0000e78f8:  00000000057e8398  000000c0000e79e0 
000000c0000e7908:  000000000428334c  000000c0000d4050 
runtime.cgocallback_gofunc(0x4004dbf, 0x4285c60, 0xc0000e7888, 0xc0000e7878)
	/usr/local/go/src/runtime/asm_amd64.s:793 +0x9b fp=0xc0000e7818 sp=0xc0000e77f8 pc=0x6236c9b

goroutine 2 [force gc (idle)]:
runtime.gopark(0x62f5a30, 0x63bbbc0, 0x1411, 0x1)
	/usr/local/go/src/runtime/proc.go:304 +0xe0 fp=0x1c000044fb0 sp=0x1c000044f90 pc=0x620f0d0
runtime.goparkunlock(...)
	/usr/local/go/src/runtime/proc.go:310
runtime.forcegchelper()
	/usr/local/go/src/runtime/proc.go:253 +0xb7 fp=0x1c000044fe0 sp=0x1c000044fb0 pc=0x620ef87
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1357 +0x1 fp=0x1c000044fe8 sp=0x1c000044fe0 pc=0x62373c1
created by runtime.init.5
	/usr/local/go/src/runtime/proc.go:242 +0x35

goroutine 3 [GC sweep wait]:
runtime.gopark(0x62f5a30, 0x63bbcc0, 0x140c, 0x1)
	/usr/local/go/src/runtime/proc.go:304 +0xe0 fp=0x1c0000457a8 sp=0x1c000045788 pc=0x620f0d0
runtime.goparkunlock(...)
	/usr/local/go/src/runtime/proc.go:310
runtime.bgsweep(0x1c00006a000)
	/usr/local/go/src/runtime/mgcsweep.go:89 +0x131 fp=0x1c0000457d8 sp=0x1c0000457a8 pc=0x6202c01
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1357 +0x1 fp=0x1c0000457e0 sp=0x1c0000457d8 pc=0x62373c1
created by runtime.gcenable
	/usr/local/go/src/runtime/mgc.go:210 +0x5c

goroutine 4 [GC scavenge wait]:
runtime.gopark(0x62f5a30, 0x63bbe60, 0x140d, 0x1)
	/usr/local/go/src/runtime/proc.go:304 +0xe0 fp=0x1c000045f40 sp=0x1c000045f20 pc=0x620f0d0
runtime.goparkunlock(...)
	/usr/local/go/src/runtime/proc.go:310
runtime.bgscavenge(0x1c00006a000)
	/usr/local/go/src/runtime/mgcscavenge.go:374 +0x3b3 fp=0x1c000045fd8 sp=0x1c000045f40 pc=0x62024c3
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1357 +0x1 fp=0x1c000045fe0 sp=0x1c000045fd8 pc=0x62373c1
created by runtime.gcenable
	/usr/local/go/src/runtime/mgc.go:211 +0x7e

goroutine 18 [finalizer wait]:
runtime.gopark(0x62f5a30, 0x63d71a8, 0x1410, 0x1)
	/usr/local/go/src/runtime/proc.go:304 +0xe0 fp=0x1c000044758 sp=0x1c000044738 pc=0x620f0d0
runtime.goparkunlock(...)
	/usr/local/go/src/runtime/proc.go:310
runtime.runfinq()
	/usr/local/go/src/runtime/mfinal.go:175 +0xa3 fp=0x1c0000447e0 sp=0x1c000044758 pc=0x61f8be3
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1357 +0x1 fp=0x1c0000447e8 sp=0x1c0000447e0 pc=0x62373c1
created by runtime.createfing
	/usr/local/go/src/runtime/mfinal.go:156 +0x61

goroutine 19 [GC worker (idle)]:
runtime.gopark(0x62f58a8, 0x1c000016090, 0x1418, 0x0)
	/usr/local/go/src/runtime/proc.go:304 +0xe0 fp=0x1c000040760 sp=0x1c000040740 pc=0x620f0d0
runtime.gcBgMarkWorker(0x1c000024000)
	/usr/local/go/src/runtime/mgc.go:1846 +0xff fp=0x1c0000407d8 sp=0x1c000040760 pc=0x61fc46f
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1357 +0x1 fp=0x1c0000407e0 sp=0x1c0000407d8 pc=0x62373c1
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1794 +0x77

goroutine 34 [GC worker (idle)]:
runtime.gopark(0x62f58a8, 0x1c0000160a0, 0x1418, 0x0)
	/usr/local/go/src/runtime/proc.go:304 +0xe0 fp=0x1c0013c2760 sp=0x1c0013c2740 pc=0x620f0d0
runtime.gcBgMarkWorker(0x1c000026500)
	/usr/local/go/src/runtime/mgc.go:1846 +0xff fp=0x1c0013c27d8 sp=0x1c0013c2760 pc=0x61fc46f
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1357 +0x1 fp=0x1c0013c27e0 sp=0x1c0013c27d8 pc=0x62373c1
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1794 +0x77

goroutine 5 [GC worker (idle)]:
runtime.gopark(0x62f58a8, 0x1c0000160b0, 0x1418, 0x0)
	/usr/local/go/src/runtime/proc.go:304 +0xe0 fp=0x1c000046760 sp=0x1c000046740 pc=0x620f0d0
runtime.gcBgMarkWorker(0x1c000028a00)
	/usr/local/go/src/runtime/mgc.go:1846 +0xff fp=0x1c0000467d8 sp=0x1c000046760 pc=0x61fc46f
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1357 +0x1 fp=0x1c0000467e0 sp=0x1c0000467d8 pc=0x62373c1
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1794 +0x77

goroutine 6 [GC worker (idle)]:
runtime.gopark(0x62f58a8, 0x1c0000ac090, 0x1418, 0x0)
	/usr/local/go/src/runtime/proc.go:304 +0xe0 fp=0x1c000046f60 sp=0x1c000046f40 pc=0x620f0d0
runtime.gcBgMarkWorker(0x1c00002af00)
	/usr/local/go/src/runtime/mgc.go:1846 +0xff fp=0x1c000046fd8 sp=0x1c000046f60 pc=0x61fc46f
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1357 +0x1 fp=0x1c000046fe0 sp=0x1c000046fd8 pc=0x62373c1
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1794 +0x77

goroutine 35 [GC worker (idle)]:
runtime.gopark(0x62f58a8, 0x1c0013c8000, 0x1418, 0x0)
	/usr/local/go/src/runtime/proc.go:304 +0xe0 fp=0x1c0013c2f60 sp=0x1c0013c2f40 pc=0x620f0d0
runtime.gcBgMarkWorker(0x1c00002d400)
	/usr/local/go/src/runtime/mgc.go:1846 +0xff fp=0x1c0013c2fd8 sp=0x1c0013c2f60 pc=0x61fc46f
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1357 +0x1 fp=0x1c0013c2fe0 sp=0x1c0013c2fd8 pc=0x62373c1
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1794 +0x77

goroutine 36 [GC worker (idle)]:
runtime.gopark(0x62f58a8, 0x1c0000ac0a0, 0x1418, 0x0)
	/usr/local/go/src/runtime/proc.go:304 +0xe0 fp=0x1c0013c3760 sp=0x1c0013c3740 pc=0x620f0d0
runtime.gcBgMarkWorker(0x1c00002f900)
	/usr/local/go/src/runtime/mgc.go:1846 +0xff fp=0x1c0013c37d8 sp=0x1c0013c3760 pc=0x61fc46f
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1357 +0x1 fp=0x1c0013c37e0 sp=0x1c0013c37d8 pc=0x62373c1
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1794 +0x77

goroutine 7 [GC worker (idle)]:
runtime.gopark(0x62f58a8, 0x1c0000160c0, 0x1418, 0x0)
	/usr/local/go/src/runtime/proc.go:304 +0xe0 fp=0x1c000047760 sp=0x1c000047740 pc=0x620f0d0
runtime.gcBgMarkWorker(0x1c000032000)
	/usr/local/go/src/runtime/mgc.go:1846 +0xff fp=0x1c0000477d8 sp=0x1c000047760 pc=0x61fc46f
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1357 +0x1 fp=0x1c0000477e0 sp=0x1c0000477d8 pc=0x62373c1
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1794 +0x77

goroutine 8 [GC worker (idle)]:
runtime.gopark(0x62f58a8, 0x1c0013c8010, 0x1418, 0x0)
	/usr/local/go/src/runtime/proc.go:304 +0xe0 fp=0x1c000047f60 sp=0x1c000047f40 pc=0x620f0d0
runtime.gcBgMarkWorker(0x1c000034500)
	/usr/local/go/src/runtime/mgc.go:1846 +0xff fp=0x1c000047fd8 sp=0x1c000047f60 pc=0x61fc46f
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1357 +0x1 fp=0x1c000047fe0 sp=0x1c000047fd8 pc=0x62373c1
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1794 +0x77
exit status 2

What did you expect to see?

Program successfully

What did you see instead?

Program panic

About this issue

  • Original URL
  • State: open
  • Created 4 years ago
  • Reactions: 1
  • Comments: 32 (14 by maintainers)

Most upvoted comments

Here’s a simple reproducer.

foo.go:

package main

import "C"
import "os"

//export Foo
func Foo() {
     println(os.Stdout)
}
func main() {
}

main.go:

package main

/*
#include <dlfcn.h>
#cgo LDFLAGS: -ldl
void f() {
   void *x = dlopen("./libfoo.so", RTLD_LAZY);
   void (*Foo)() = (void(*)())dlsym(x, "Foo");
   Foo();
}
*/
import "C"

func main() {
     C.f()
}

$ go build -o libfoo.so -buildmode=c-shared foo.go
$ go run main.go

This prints 0x0 on a mac. It prints something nonzero on linux.

Something is wrong with the Go->C->Go path. When we get back to Go, we have an uninitialized environment.

+4
randall77 on Apr 29, 2020

cc @odeke-em @eliasnaur for ideas on a solution.

+2
networkimprov on Apr 30, 2020

I do not know. Without a way for us to reproduce it’s going to be hard to find out.

Please let me know after you’ve watched it, I will delete it because I don’t want to be public Demo

You can send it directly to @randall77 via email.

+2
cuonglm on Apr 27, 2020

I’m sorry, I don’t have any useful pointers. I’m not familiar with the details of shared libraries on macOS.

I want to clarify that this is not an intended use case. The -buildmode=c-shared option is intended to build a Go shared library that is opened by a program written in C (or C++ or some other non-Go language). Using -buildmode=c-shared is not intended for a Go library that is opened by a Go program. People who need Go shared libraries that are opened by Go programs are expected to use either -buildmode=plugin or -buildmode=shared, depending on what they are trying to do.

That said, I would not be surprised if those options also have trouble on macOS. My point in mentioning this is that if you want to work on fixing something here, I encourage you to focus on -buildmode=plugin or -buildmode=shared before you focus on -buildmode=c-shared.

+1
ianlancetaylor on Nov 7, 2020

@khaitranvan96kt I hope that someone will solve your problem. But it is hard in the general case, and I do not know of anybody working on it. Sorry.

+1
ianlancetaylor on May 1, 2020

Well that looks like a completely different panic trace. Something is definitely clobbered here, it’s crashing because log.std is nil, which can’t happen (unless Python is calling back into Go before Go has been initialized)? Looks like memory corruption.

my code runs well in ubuntu but not in macos, what’s the difference here?

I do not know. Without a way for us to reproduce it’s going to be hard to find out.

+1
randall77 on Apr 27, 2020
Read more comments on GitHub
← go: runtime: can't build go distribution caused by #16272 and #16570
go: runtime: cgo call to symbol from library loaded dynamically will panic with go 1.21.1 and ld >2.38 →