protobuf: Nested composite fields sometimes not being marshaled
What version of protobuf and what language are you using?
package: google.golang.org/protobuf v1.25.0
protoc-gen-go: v1.25.0
libprotoc: 3.6.1
What did you do?
II have a system which, among other things, passes messages to multiple receivers. There’s simplified version of it, which should be sufficient to show how it works with protobuf messages:
schema.proto
syntax = "proto3";
package package;
option go_package = "proto.definitions/generated/package";
message Point {
sint32 x = 1;
sint32 y = 2;
}
message PointCommand {
Point src = 2;
}
message PointEvent {
Point src = 3;
}
code.go
package main
import (
"fmt"
"io"
"net"
"google.golang.org/protobuf/proto"
pd "proto.definitions/generated/package"
)
var watchers map[uint64]chan<- proto.Message
type Session struct {
writer io.Writer
}
func (sess *Session) Listen(conn net.Conn) {
sess.writer = conn
for {
// data []byte : read from conn
var cmd pd.PointCommand
if err := proto.Unmarshal(data, &cmd); err != nil {
panic(err)
}
if cmd.Src == nil {
panic(fmt.Errorf("nil src"))
}
evt := pd.PointEvent{
Src: cmd.Src,
}
for _, w := range watchers {
w <- &evt
}
}
}
func (sess *Session) Watch(ch <-chan proto.Message) {
for msg := range ch {
data, err := proto.Marshal(msg)
if err != nil {
panic(err)
}
// send data to sess.writer
}
}
func ListenAndServe(addr string) error {
l, _ := net.Listen("tcp", addr)
defer l.Close()
for {
conn, _ := l.Accept()
sess := new(Session)
{
var id uint64 // generate unique id
ch := make(chan proto.Message)
watchers[id] = ch
go sess.Watch(ch)
}
go sess.Listen(conn)
}
}
func main() {
go ListenAndServe(":xxxx")
<-make(chan struct{})
}
While most of the wrapping code wiped out, the key part is still there: message passed without any modifications after creation and then marshaled separately for different receivers. However, wire data may not have src field despite explicit checking.
User code of course was the main suspect and I ran application with race detector without any error output. In controlled environment, when src set to Point with default values, proto.Marshal produces expected serialization result with zero-length field value.
What did you expect to see?
Empty (but not nil) nested structs should always be marshaled as empty ones.
What did you see instead?
Sometimes marshaled data written to Writer has no serialized src field even though it can’t be nil.
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 22 (8 by maintainers)
If
PointEvent.Src = nilthen it prints nothing.1A 00is:field 3, zero length. Which makes sense because it has no non-default values.