tensorflow: Android demo app crashes when using quantized model obtained from graph_transform tool?

System information

  • Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Only edited ClassifierActivity.java to suit a custom model
  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Android smartphone / host machine: Ubuntu 16.04 LTS
  • TensorFlow installed from (source or binary): Source
  • TensorFlow version (use command below): 1.1
  • Bazel version (if compiling from source): 0.4.5
  • CUDA/cuDNN version: 8.0/5.1
  • GPU model and memory: GTX 860M

Describe the problem

I am currently following the guide in: http://nilhcem.com/android/custom-tensorflow-classifier

to train a custom classifier. However, I am using my own frozen graph that I obtained from my own training. What I noticed was when I used the quantized graph obtained through the graph_transform method, the app simply crashes without even running. In the guide, it is recommended to run this command on the inference graph:

bazel-bin/tensorflow/python/tools/optimize_for_inference \
  --input=/tf_files/retrained_graph.pb \
  --output=/tf_files/retrained_graph_optimized.pb \
  --input_names=Mul \
  --output_names=final_result

While I think it may be in conflict with the quantized graph transformations, I ran this command to test, and the app did crash. Here are the observations for all 4 permutations I tested:

  1. quantization + optimize_for_inference = app crash
  2. quantization only = app crash
  3. optimize_for_inference on frozen graph = app works
  4. frozen_graph only = app works

So my conclusion is the quantization operations within the quantized graph caused the app to fail.

Here is the quantization command I ran:

/home/kwotsin/tensorflow-android/tensorflow/bazel-bin/tensorflow/tools/graph_transforms/transform_graph \
--in_graph=./frozen_model_mobilenet.pb \
--out_graph=./quantized_model_mobilenet.pb \
--inputs='Placeholder_only' \
--outputs='MobileNet/Predictions/Softmax' \
--transforms='
  add_default_attributes
  strip_unused_nodes(type=float, shape="1,299,299,3")
  remove_nodes(op=Identity, op=CheckNumerics)
  fold_constants(ignore_errors=true)
  fold_batch_norms
  fold_old_batch_norms
  quantize_weights
  quantize_nodes
  strip_unused_nodes
  sort_by_execution_order'

And the java file I edited to build my APK:

  private static final int INPUT_SIZE = 224;
  private static final int IMAGE_MEAN = 128;
  private static final float IMAGE_STD = 128;
  private static final String INPUT_NAME = "Placeholder_only";
  private static final String OUTPUT_NAME = "MobileNet/Predictions/Softmax";

  private static final String MODEL_FILE = "file:///android_asset/quantized_model_mobilenet.pb";
  private static final String LABEL_FILE =
      "file:///android_asset/mobilenet_labels.txt";

Other than these edit, every other file is the same. I then imported this project in Android Studio, and ran the build apk option located on the top toolbar of Android Studio.

So far, existing tutorials I’ve seen (e.g. those from Pete Warden) use the quantize_graph method to run on mobile devices. Is graph_transform compatible for quantizing models for mobile devices yet?

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Comments: 28 (19 by maintainers)

Most upvoted comments

@kwotsin Accuracy is retained and the performance is also the same as original model…

I think the underlying issue here is that you need to specify the right input and output names when you call transform_graph too. In the code above, you’re using the default of Mul and final_result when you invoke the tool, but they should be input and MobileNet/Predictions/Softmax, like your application code.