go-json: Concurrent map iteration and write exception

I’m using the library in an application where multiple go-routines are marshalling JSON (potentially at the same time). Although I’ve run a number of times successfully, today I got a concurrent map access exception from within the go-json library. Is the library intended to be thread-safe?

The stack trace given was:

goroutine 3287070 [running]:
runtime.throw({0x1830962, 0x10000000186d480})
        /usr/local/go/src/runtime/panic.go:1198 +0x71 fp=0xc04931da40 sp=0xc04931da10 pc=0x444e91
runtime.mapiternext(0x1787760)
        /usr/local/go/src/runtime/map.go:858 +0x4eb fp=0xc04931dab0 sp=0xc04931da40 pc=0x41f26b
runtime.mapiterinit(0x1, 0xc04931db00, 0xc0438baa80)
        /usr/local/go/src/runtime/map.go:848 +0x236 fp=0xc04931dad0 sp=0xc04931dab0 pc=0x41ed36
reflect.mapiterinit(0x1a95250, 0xc0438baa80)
        /usr/local/go/src/runtime/map.go:1340 +0x45 fp=0xc04931db10 sp=0xc04931dad0 pc=0x472065
github.com/goccy/go-json/internal/encoder/vm.Run(0xc0777cc000, {0xc0438baa80, 0x0, 0x6a80}, 0xc00013b1c0)
        /home/astock/go/pkg/mod/github.com/goccy/go-json@v0.8.1/internal/encoder/vm/vm.go:406 +0x2ad7 fp=0xc049320fc0 sp=0xc04931db10 pc=0x10a03d7
github.com/goccy/go-json.encodeRunCode(0x203012, {0xc0438baa80, 0x203012, 0xc04a82bb00}, 0xc0c545de80)
        /home/astock/go/pkg/mod/github.com/goccy/go-json@v0.8.1/encode.go:307 +0x68 fp=0xc049320ff8 sp=0xc049320fc0 pc=0x10ca968
github.com/goccy/go-json.encode(0xc0777cc000, {0x1755620, 0xc04a82bb00})
        /home/astock/go/pkg/mod/github.com/goccy/go-json@v0.8.1/encode.go:232 +0x21c fp=0xc049321078 sp=0xc049320ff8 pc=0x10ca85c
github.com/goccy/go-json.marshal({0x1755620, 0xc04a82bb00}, {0x0, 0x0, 0xc0cfc5d128})
        /home/astock/go/pkg/mod/github.com/goccy/go-json@v0.8.1/encode.go:147 +0xba fp=0xc0493210d8 sp=0xc049321078 pc=0x10ca51a
github.com/goccy/go-json.MarshalWithOption(...)
        /home/astock/go/pkg/mod/github.com/goccy/go-json@v0.8.1/json.go:186
github.com/goccy/go-json.Marshal({0x1755620, 0xc04a82bb00})
        /home/astock/go/pkg/mod/github.com/goccy/go-json@v0.8.1/json.go:171 +0x2a fp=0xc049321110 sp=0xc0493210d8 pc=0x10ca9ea```

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 17 (7 by maintainers)

Most upvoted comments

@Max-Cheng Thank you for your report !!

@watchforstock @OlegElizarov This problem may have been fixed by some modifications to the map encoder. If you encounter similar issues with v0.9.6 or later, please reopen this.

goccy on Mar 25, 2022

Ah, Looking at the stack trace, it seems that you are using json.MarshalNoEscape instead of json.Marshal . This does not escape the argument, so this API will cause problems depending on the timing of the GC. I want to know the pattern of bugs that occur in json.Marshal.

goccy on Jan 17, 2022

Thanks for following this up. After raising this, we got another (but different) concurrent access error while JSON marshalling. I now suspect that there was some object sharing between the goroutines and that may have been the cause of this error. We’ve put a fix into our codebase and will try this library again and see if we see the issue again.

watchforstock on Jan 9, 2022