murmur3: Flaky "unsafe pointer conversion"-panics with -race and go1.14rc1

I cannot reproduce the panic below with a minimal example and it occurs very often, but neither all the time nor in the same test - so this report might be entirely unhelpful, but I am reporting it anyway in case it makes any sense to you:

When running the Syncthing database testsuite on go1.14rc1 with race detection enabled on linux/amd64 I get the following panic:

$ go test -v -race -short -count 1 github.com/syncthing/syncthing/lib/db/ 2>&1 | tee test.out
fatal error: checkptr: unsafe pointer conversion

goroutine 148 [running]:
runtime.throw(0xd106c0, 0x23)
	/media/ext4_data/Linux/source/go/src/runtime/panic.go:1112 +0x72 fp=0xc00022ba50 sp=0xc00022ba20 pc=0x4630d2
runtime.checkptrAlignment(0xc00011e3f1, 0xc40880, 0x1)
	/media/ext4_data/Linux/source/go/src/runtime/checkptr.go:13 +0xd0 fp=0xc00022ba80 sp=0xc00022ba50 pc=0x4348a0
github.com/spaolacci/murmur3.(*digest128).bmix(0xc000123c80, 0xc00011e3f1, 0x20, 0x2f, 0x0, 0x48, 0x58)
	/media/ext4_data/Coding/go/pkg/mod/github.com/spaolacci/murmur3@v1.1.0/murmur128.go:67 +0xc0 fp=0xc00022bae8 sp=0xc00022ba80 pc=0xb649f0
github.com/spaolacci/murmur3.(*digest).Write(0xc000123c80, 0xc00011e3f1, 0x20, 0x2f, 0xb64474, 0xc000123c80, 0xc000123c80)
	/media/ext4_data/Coding/go/pkg/mod/github.com/spaolacci/murmur3@v1.1.0/murmur.go:51 +0x127 fp=0xc00022bb98 sp=0xc00022bae8 pc=0xb63dd7
github.com/spaolacci/murmur3.(*digest128).Write(0xc000123c80, 0xc00011e3f1, 0x20, 0x2f, 0x10, 0x10, 0x21)
	<autogenerated>:1 +0x6a fp=0xc00022bbf8 sp=0xc00022bb98 pc=0xb6558a
github.com/willf/bloom.baseHashes(0xc00011e3f1, 0x20, 0x2f, 0x0, 0x0, 0x0, 0x0)
	/media/ext4_data/Coding/go/pkg/mod/github.com/willf/bloom@v2.0.3+incompatible/bloom.go:97 +0xb9 fp=0xc00022bc68 sp=0xc00022bbf8 pc=0xb6d179
github.com/willf/bloom.(*BloomFilter).Test(0xc000134700, 0xc00011e3f1, 0x20, 0x2f, 0xdf6fe0)
	/media/ext4_data/Coding/go/pkg/mod/github.com/willf/bloom@v2.0.3+incompatible/bloom.go:183 +0x6b fp=0xc00022bd20 sp=0xc00022bc68 pc=0xb6deab
github.com/syncthing/syncthing/lib/db.(*Lowlevel).gcBlocks(0xc000098870, 0x0, 0x0)
	/media/ext4_data/Coding/go/src/github.com/syncthing/syncthing/lib/db/lowlevel.go:570 +0x4d4 fp=0xc00022beb8 sp=0xc00022bd20 pc=0xb798b4
github.com/syncthing/syncthing/lib/db.(*Lowlevel).gcRunner(0xc000098870)
	/media/ext4_data/Coding/go/src/github.com/syncthing/syncthing/lib/db/lowlevel.go:483 +0x279 fp=0xc00022bfd8 sp=0xc00022beb8 pc=0xb78fb9
runtime.goexit()
	/media/ext4_data/Linux/source/go/src/runtime/asm_amd64.s:1375 +0x1 fp=0xc00022bfe0 sp=0xc00022bfd8 pc=0x4969c1
created by github.com/syncthing/syncthing/lib/db.NewLowlevel
	/media/ext4_data/Coding/go/src/github.com/syncthing/syncthing/lib/db/lowlevel.go:65 +0x36a

(rest of the backtrace does not include murmur3 nor bloom).

Do you have any ideas what this might be about? Thanks in advance for any help.

About this issue

  • Original URL
  • State: open
  • Created 4 years ago
  • Reactions: 8
  • Comments: 15

Commits related to this issue

Most upvoted comments

The fork of @twmb seems quite nice: https://github.com/twmb/murmur3

Fundamentally, the unsafe code needs changing. Feel free to take my patches: https://github.com/twmb/murmur3/compare/b0c3ada...HEAD

Note that using binary.LittleEndian (or my path) will resolve this repos tickets around issues across architectures, as well.