channel-pruning: Failed to include caffe_pb2

Hi Yihui,

I tried your code and met some problems.

After make -j8 and make pycaffe, I tried to python3 train.py, but found something wrong with protobuf. So I change the protobuf version but the problem was still not solved.

Here is the problem: When I tried protobuf 3.0.0(b1,b2,b3,b4) or 3.1.0 , the error message is:

Failed to include caffe_pb2, things might go wrong!
Traceback (most recent call last):
  File "/home/dutianyuan/anaconda3/lib/python3.5/site-packages/google/protobuf/internal/python_message.py", line 1087, in MergeFromString
    if self._InternalParse(serialized, 0, length) != length:
  File "/home/dutianyuan/anaconda3/lib/python3.5/site-packages/google/protobuf/internal/python_message.py", line 1109, in InternalParse
    (tag_bytes, new_pos) = local_ReadTag(buffer, pos)
  File "/home/dutianyuan/anaconda3/lib/python3.5/site-packages/google/protobuf/internal/decoder.py", line 181, in ReadTag
    while six.indexbytes(buffer, pos) & 0x80:
TypeError: unsupported operand type(s) for &: 'str' and 'int'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "train.py", line 19, in <module>
    from lib.net import Net, load_layer, caffe_test
  File "/mnt/lustre/dutianyuan/channel-pruning/lib/net.py", line 7, in <module>
    import caffe
  File "/mnt/lustre/dutianyuan/channel-pruning/caffe/python/caffe/__init__.py", line 4, in <module>
    from .proto.caffe_pb2 import TRAIN, TEST
  File "/mnt/lustre/dutianyuan/channel-pruning/caffe/python/caffe/proto/caffe_pb2.py", line 799, in <module>
    options=_descriptor._ParseOptions(descriptor_pb2.FieldOptions(), '\020\001')),
  File "/home/dutianyuan/anaconda3/lib/python3.5/site-packages/google/protobuf/descriptor.py", line 869, in _ParseOptions
    message.ParseFromString(string)
  File "/home/dutianyuan/anaconda3/lib/python3.5/site-packages/google/protobuf/message.py", line 185, in ParseFromString
    self.MergeFromString(serialized)
  File "/home/dutianyuan/anaconda3/lib/python3.5/site-packages/google/protobuf/internal/python_message.py", line 1093, in MergeFromString
    raise message_mod.DecodeError('Truncated message.')
google.protobuf.message.DecodeError: Truncated message.

and when I change protobuf to 3.2.0 / 3.3.0 / 3.4.0, the error message is

Traceback (most recent call last):
File "train.py", line 19, in <module>
  from lib.net import Net, load_layer, caffe_test 
File "/mnt/lustre/dutianyuan/channel-pruning/lib/net.py", line 7, in <module>
  import caffe
File "/mnt/lustre/dutianyuan/channel-pruning/caffe/python/caffe/__init__.py", line 4, in <module>
  from .proto.caffe_pb2 import TRAIN, TEST
File "/mnt/lustre/dutianyuan/channel-pruning/caffe/python/caffe/proto/caffe_pb2.py", line 17, in <module>
  serialized_pb='\n\x0b\x63\x61\x66\x66\x65.proto\x12\x05\x63\x61\x66\x66\x65\"\x1c\n\tBlobShape\x12\x0f\n\x03\x64im\x18\x01 
.....with a lot of \x......
  File "/home/dutianyuan/anaconda3/lib/python3.5/site-packages/google/protobuf/descriptor.py", line 824, in __new__
    return _message.default_pool.AddSerializedFile(serialized_pb)
TypeError: expected bytes, str found

When I first met this problem, my Python version is 3.4 and your default setting is with Python 3.5. So I install Python3.5 and Python3.6 by Anaconda, Python 3.6 by apt-get. No matter what version is, the problem was still not solved.

Hope you can show me the specific version of your coding environment! Thanks!

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Comments: 19 (4 by maintainers)

Most upvoted comments

@tianyuandu @yihui-he protobuf problem fixed, because of the c++ version pb not matching the python version, reinstall them has fixed. Thanks two guys!

@tianyuandu @LearnerInGithub Hello, I downloaded the latest version of caffe, installed anaconda, and installed both python 2.7 and 3.5. The protoc in usr/lib is version 2.5. If I don’t install(conda install) protoc 3.2 (or later) in python and use protoc2.5 , then caffe can be made all. But once protoc3.2 (or later) is installed in python, i cannot make all caffe. I hope you can provide some advice, thank you.