tensorflow: Restoring SavedModel in //tensorflow/c:c_api_test fails

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04
TensorFlow installed from (source or binary): source
TensorFlow version: ‘v1.2.1-0-gb4957ff’, ‘1.2.1’
Python version: 2.7.12
Bazel version: 0.4.5
CUDA/cuDNN version: No GPU
GPU model and memory: No GPU
Exact command to reproduce: bazel test //tensorflow/c:c_api_test

The problem:

While executing c_api_test from c module on a big endian machine, it fails while restoring SavedModel(tensorflow/cc/saved_model/testdata/half_plus_two/00000123). There is a warning displayed Reading a bundle with different endianness from the reader.

Similarly, other tests from cc, java module which read this SavedModel from testdata also fail due to endianness mismatch.

What would be appropriate way to handle this failure on big endian? Is there a way to convert the above SavedModel to Big Endian while reading it in the tests?

Test Logs:

[ RUN      ] CAPI.SavedModel
2017-06-29 09:04:33.999662: I tensorflow/cc/saved_model/loader.cc:226] Loading SavedModel from: <HOME>/.cache/bazel/_bazel_test/24685d064c07f7346b48c2d13ec3ad69/execroot/tensorflow/bazel-out/local-opt/bin/tensorflow/c/c_api_test.runfiles/org_tensorflow/tensorflow/cc/saved_model/testdata/half_plus_two/00000123
2017-06-29 09:04:34.039733: I tensorflow/cc/saved_model/loader.cc:145] Restoring SavedModel bundle.
2017-06-29 09:04:34.079006: W tensorflow/core/framework/op_kernel.cc:1158] Unimplemented: Reading a bundle with different endianness from the reader
2017-06-29 09:04:34.079311: W tensorflow/core/framework/op_kernel.cc:1158] Unimplemented: Reading a bundle with different endianness from the reader
2017-06-29 09:04:34.079363: W tensorflow/core/framework/op_kernel.cc:1158] Unimplemented: Reading a bundle with different endianness from the reader
2017-06-29 09:04:34.079708: I tensorflow/cc/saved_model/loader.cc:274] Loading SavedModel: fail. Took 136931 microseconds.
tensorflow/c/c_api_test.cc:1198: Failure
      Expected: TF_OK
      Which is: 0
To be equal to: TF_GetCode(s)
      Which is: 12
Reading a bundle with different endianness from the reader
         [[Node: save/RestoreV2 = RestoreV2[_output_shapes=[[]], dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_1, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]

About this issue

Original URL
State: closed
Created 7 years ago
Comments: 15 (12 by maintainers)

Most upvoted comments

I think the larger question raised by this is what our position should be regarding serialization formats. Regenerating the testdata would make this test pass, but we’d still have issues where say a SavedModel produced by a little endian machine will be unreadable on a big endian machine and vice-versa. Similarly, any tensors sent over RPCs across between machines with different architectures would likely fail.

A more principled fix would be something that perhaps includes the encoding format in the serialization and translates if necessary when deserializing. Something like that, I believe, would be the right way to go about ensuring TensorFlow is big-endian friendly.

That said, for specifically this test, you could regenerate a saved model in big-endian format in the testdata directory by running https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/saved_model/saved_model_half_plus_two.py , committing the output and changing c_api_test.cc to use one path or the other based on the architecture it’s running on.

Unassigning myself as both changes outlined above is not something that I, or anyone else on the TensorFlow team (I think), will be getting to in the near future. But we’d be happy to accept PRs.

asimshankar on Aug 9, 2017