protobuf: Nested composite fields sometimes not being marshaled

What version of protobuf and what language are you using? package: google.golang.org/protobuf v1.25.0 protoc-gen-go: v1.25.0 libprotoc: 3.6.1

What did you do?

II have a system which, among other things, passes messages to multiple receivers. There’s simplified version of it, which should be sufficient to show how it works with protobuf messages:

schema.proto
syntax = "proto3";

package package;
option go_package = "proto.definitions/generated/package";

message Point {
	sint32 x = 1;
	sint32 y = 2;
}

message PointCommand {
	Point src = 2;
}

message PointEvent {
	Point src = 3;
}
code.go
package main

import (
	"fmt"
	"io"
	"net"

	"google.golang.org/protobuf/proto"

	pd "proto.definitions/generated/package"
)

var watchers map[uint64]chan<- proto.Message

type Session struct {
	writer io.Writer
}

func (sess *Session) Listen(conn net.Conn) {
	sess.writer = conn

	for {
		// data []byte : read from conn

		var cmd pd.PointCommand
		if err := proto.Unmarshal(data, &cmd); err != nil {
			panic(err)
		}

		if cmd.Src == nil {
			panic(fmt.Errorf("nil src"))
		}

		evt := pd.PointEvent{
			Src: cmd.Src,
		}

		for _, w := range watchers {
			w <- &evt
		}
	}
}

func (sess *Session) Watch(ch <-chan proto.Message) {
	for msg := range ch {
		data, err := proto.Marshal(msg)
		if err != nil {
			panic(err)
		}

		// send data to sess.writer
	}
}

func ListenAndServe(addr string) error {
	l, _ := net.Listen("tcp", addr)
	defer l.Close()

	for {
		conn, _ := l.Accept()
		sess := new(Session)

		{
			var id uint64 // generate unique id
			ch := make(chan proto.Message)
			watchers[id] = ch
			go sess.Watch(ch)
		}

		go sess.Listen(conn)
	}
}

func main() {
	go ListenAndServe(":xxxx")
	<-make(chan struct{})
}

While most of the wrapping code wiped out, the key part is still there: message passed without any modifications after creation and then marshaled separately for different receivers. However, wire data may not have src field despite explicit checking.

User code of course was the main suspect and I ran application with race detector without any error output. In controlled environment, when src set to Point with default values, proto.Marshal produces expected serialization result with zero-length field value.

What did you expect to see?

Empty (but not nil) nested structs should always be marshaled as empty ones.

What did you see instead?

Sometimes marshaled data written to Writer has no serialized src field even though it can’t be nil.

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 22 (8 by maintainers)

Most upvoted comments

Not at all, since src is not nil, it should be 1A 02 0A 00 and not 1A 00 as per documentation.

If PointEvent.Src = nil then it prints nothing. 1A 00 is: field 3, zero length. Which makes sense because it has no non-default values.