protobuf: Rev of proto-gen-go to ProtoPackageIsVersion3 causing breakage

dep users should version their protoc-gen-go as a required binary, like so (in Gopkg.toml):

required = [
    "github.com/golang/protobuf/protoc-gen-go",
]

You can then install protoc-gen-go from vendor:

 $ go install ./vendor/github.com/golang/protobuf/protoc-gen-go

I’m not sure what the go modules equivalent workflow is.

@johanbrandhorst @edwarnicke @lavoiesl If any of you guys happen to be using go modules- this worked for me: go get github.com/golang/protobuf@8d0c54c1246661d9a51ca0ba455d22116d485eaa ^^^ that will update your go.mod to use the specific revision needed to avoid problems

I could fix the dependency issue in this way:

dep ensure
go install ./vendor/github.com/golang/protobuf/protoc-gen-go/
go install ./vendor/github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway/

Thanks @lenvs inspired me! I also solved the problem ! You may do that.

rm -rf vendor/github.com/golang/protobuf
cd vendor/github.com/golang/
git clone https://github.com/golang/protobuf.git
go install

Stay in branch master, don`t checkout to V1.2.0 !!!

This hack worked for me

cd $GOPATH/src/github.com/golang/protobuf/protoc-gen-go
git checkout v1.2.0
go install

problem solved. This is because proto-go is installed by go get -u github.com/golang/protobuf/protoc-gen-go, which is the newest master branch. you can switch to tag 1.2.0. reinstall proto-go, reprotoc all files will fix this problem.

I want to note some caveats with https://github.com/golang/protobuf/issues/763#issuecomment-452153894 suggestion:

• The issue that most people are running into is because they are building protoc-gen-go from head, but have a specific version of the proto package pinned.
• The suggestion above pins the proto package to a more recent version closer to today’s head, but doesn’t address underlying problem; that is, you are still building protoc-gen-go from head, but have a specific version of proto pinned.
• The README now contains instructions for how to pin a specific version of protoc-gen-go, which is a more stable path forward, than to still build the binary from head.
• Go 1.12 will make this much easier, since it will allow installing a Go binary at a specific revision. See golang/go#24250.

I’m also a bit embarrassed that our CI got caught by this too. go get -u github.com/golang/protobuf/protoc-gen-go was just too convenient.

Something that could be helpful coming from the golang/protobuf side is to put binaries at a well known stable location. In the repo release would be great, or something like the k8s release notes binaries section. Forcing users to build the tool is a bit of a complexity leak. go get handled this more or less transparently, until it didn’t.

@fertobar I might be wrong but I think that’s a grpc-gateway issue. Almost certainly the same problem as here though, you’ve got one version of your generator and another version of the runtime library. Please make sure you’re using the same version of both.

I have solve the problem ! I think its because the protoc-gen-go’s version not correct.

git clone https://github.com/golang/protobuf cd ~/protobuf/protoc-gen-go git chectout tags/v1.2.0 -b v1.2.0 go install

my environment is mac go1.11

I appreciate all the work you are doing, but I want to note that I find it highly disruptive to have released a breaking change on master without a new version.

go get -u does not support versioning so it’s cumbersome to require a specific version. It also happens to not be currently possible with an internal solution we have to manage such. I know we could add support, but it goes to show that this is a cumbersome breaking change.

I would recommend releasing a new version asap.

In the mean time, we have changed our projects to track the master branch, but this is not ideal either because in some other cases, the generator is installed in a docker image, so it is not always up-to-date.

Side note: Major version bump would probably be appropriate here. This is not a change where some people may be affected, it’s a hard version requirement, which completely breaks BC.

Personally, I find this change super disturbing but I understand that’s sometimes needed. I can’t find a way to make dep happy with grpc-ecosystem/grpc-gateway with this new change. Has anybody tried it ?

Errr, sorry, I mispoke. It’s not the -I flag. It’s the how the proto file to compile was specified when given to protoc.

For example:

$ protoc --go_out=paths=source_relative:. ptypes/any/any.proto
$ mv ptypes/any/any.pb.go ptypes/any/any1.pb.go
$ (cd ptypes && protoc --go_out=paths=source_relative:. any/any.proto)
$ mv ptypes/any/any.pb.go ptypes/any/any2.pb.go 
$ (cd ptypes/any && protoc --go_out=paths=source_relative:. any.proto)
$ mv ptypes/any/any.pb.go ptypes/any/any3.pb.go 
$ cat any*.pb.go | grep RegisterFile
func init() { proto.RegisterFile("ptypes/any/any.proto", fileDescriptor_any_fea4429e4270d3cd) }
func init() { proto.RegisterFile("any/any.proto", fileDescriptor_any_f2863f9fa4d3af49) }
func init() { proto.RegisterFile("any.proto", fileDescriptor_any_0a174ad2e4d7ef58) }

Notice that all three have different values?

This behavior is surprising, but is an artifact of how protobufs were originally designed. That is, they assume a single root where all source files universally lived under in a build system similar to Bazel. It’s not great for today’s world of independently distributed build processes, but such is the status quo.

steps by @lenvs fixed the ProtoPackageIsVersion3 issue, but I’m getting undefined: utilities.IOReaderFactory with grpc after switch to v1.2.0 . any help is appreciated

@DarrienG, how do you propose this be addressed? A major version bump is not the right answer:

• The existence of breakages is not what justifies a major version bump. For example, most projects follow (implicitly or explicitly) the Go1 compatibility policy and that document reserves the right for authors to add new struct fields. However, doing so can break people when depending on a struct being comparable that no longer became so when a field was added. Breakages like this have happened before with data structures in the standard library, yet the standard library did not regard the existence of breakages justification for v2 (i.e., they still broke those users).
• The change made is explicitly specified in our compatibility agreement as within our right to make. Thus, we are not required to produce a major version bump.
• A major version bump is a very costly change. It often means that types are no longer compatible, a possibly separate API that people need to know which to import, etc. We are working on a v2 of protobufs, and interestingly the change made was for the purposes of getting v1 and v2 to inter-operate to reduce fragmentation when both APIs exist. If a major version is issued, it should provide strong benefits to justify its costs and 8d0c54c1246661d9a51ca0ba455d22116d485eaa alone does not justify a v2.

I know this is open source and all, but please consider being cognizant of people’s setups.

We are cognizant of people’s setup within reason. I want to note that this change:

• does not break people who live at HEAD for both the proto package and protoc-gen-go. I personally think people who do this are risky, but we try hard not to break this situation, and 8d0c54c1246661d9a51ca0ba455d22116d485eaa upheld this expectation as it atomically changed both proto and protoc-gen-go.
• does not break people who live at a specific version of the proto package and protoc-gen-go. This is my recommended approach for a healthy CI system.
• does break people who live at a specific version of the proto package and a HEAD version of protoc-gen-go. It’s unfortunate that this situation breaks, but there is fundamentally no way that this combination of dependencies remain break-free since protoc-gen-go depends on proto.

While I agree that a build system should be hermetic and all, I think it’s silly this broke on a minor version. I would expect breakages in a X.minor.bugs release but not a major.X.bugs release.

Gopkg even assumes it is safe to do this, and if you don’t specify an =VERSION it would have auto-upgraded a number of setups that would have been broken even if people did specify 1.2.0.

I know this is open source and all, but please consider being cognizant of people’s setups.

I’m not sure what we can do on our end, this seems to be an issue really with the Go toolchain that there isn’t an easy way (or at least I don’t know of one) to build a binary at a specific revision.

For the time being, you can do something like this:

$ git clone git@github.com:golang/protobuf.git && (cd protobuf && git checkout v1.2.0 && go build -o $BIN_PATH/protoc-gen-go ./protoc-gen-go) && rm -r protobuf

Be sure to have $BIN_PATH be the output path where the binary should end up.