java: TensorFlow.loadLibrary not working on Windows

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04 x86_64): Windows 10
TensorFlow installed from (source or binary): 2.4.1
TensorFlow version (use command below):
Java version (i.e., the output of java -version): 11
Java command line flags (e.g., GC parameters):
Python version (if transferring a model trained in Python):
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version:
GPU model and memory:

Describe the current behavior

I’m trying to use models which contain ops from tensorflow-text. To get them to work on Linux, I just took the .so files from the pip wheel and loaded them using TensorFlow.loadLibrary(). When I run the same code on Windows, using the binary files from the Windows pip wheel extracted in the same way (which are also .so files for some reason), they cannot be loaded. The example code below uses the universal-sentence-encoder-multilingual-large and works as expected when run from within the WSL2 Ubuntu 20.04 environment on my Windows 10 machine. Is there something else that needs to be done to get the libraries to load on Windows?

Describe the expected behavior

Code to reproduce the issue


public class Playground {

    public static void main(String[] args) {

        System.out.println(TensorFlow.version());

        File tensorflowTextBase = new File("../resources/tf_text/2.4.1/linux");

        File[] libraries = tensorflowTextBase.listFiles(f -> f.getName().endsWith(".so"));

        System.out.println("found " + libraries.length + " libraries");

        for (File lib : libraries) {
            try {
                OpList list = TensorFlow.loadLibrary(lib.getAbsolutePath());
                System.out.println("loaded " + list.getOpCount() + " ops from " + lib.getName());
            } catch (UnsatisfiedLinkError e) {
                System.err.println("could not load " + lib.getName());
            }
        }


        String text = "this is a test text";

        File file = new File("../resources/universal-sentence-encoder-multilingual-large_3/");
        System.out.println(file.getAbsolutePath());

        SavedModelBundle textEmbedding = SavedModelBundle.load(file.getAbsolutePath());

        try (TString textTensor = TString.tensorOf(NdArrays.vectorOfObjects(text))) {

            HashMap<String, Tensor> inputMap = new HashMap<>();
            inputMap.put("inputs", textTensor);

            Map<String, Tensor> resultMap = textEmbedding.call(inputMap);

            TFloat32 embedding = (TFloat32) resultMap.get("outputs");

            float[] embeddingArray = new float[512];
            FloatDataBuffer floatBuffer = DataBuffers.of(embeddingArray);
            embedding.read(floatBuffer);

            System.out.println(Arrays.toString(embeddingArray));

        }
    }
}

Other info / logs

Output from WSL2 Ubuntu 20.04

2.4.1
found 12 libraries
loaded 1 ops from _constrained_sequence_op.so
loaded 1 ops from _mst_ops.so
loaded 4 ops from _normalize_ops.so
loaded 1 ops from _regex_split_ops.so
loaded 7 ops from _sentencepiece_tokenizer.so
loaded 1 ops from _sentence_breaking_ops.so
loaded 1 ops from _split_merge_from_logits_tokenizer.so
loaded 1 ops from _split_merge_tokenizer.so
loaded 1 ops from _state_based_sentence_breaker_op.so
loaded 1 ops from _unicode_script_tokenizer.so
loaded 1 ops from _whitespace_tokenizer.so
loaded 1 ops from _wordpiece_tokenizer.so
/mnt/c/Users/Lucaro/Workspace/cineast/cineast-runtime/../resources/universal-sentence-encoder-multilingual-large_3
2022-01-06 13:12:35.253304: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:32] Reading SavedModel from: /mnt/c/Users/Lucaro/Workspace/cineast/cineast-runtime/../resources/universal-sentence-encoder-multilingual-large_3
2022-01-06 13:12:35.326120: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:55] Reading meta graph with tags { serve }
2022-01-06 13:12:35.326257: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:93] Reading SavedModel debug info (if present) from: /mnt/c/Users/Lucaro/Workspace/cineast/cineast-runtime/../resources/universal-sentence-encoder-multilingual-large_3
2022-01-06 13:12:35.326472: I external/org_tensorflow/tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-01-06 13:12:35.536663: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:206] Restoring SavedModel bundle.
2022-01-06 13:12:35.555468: I external/org_tensorflow/tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 3501000000 Hz
2022-01-06 13:12:37.667468: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:190] Running initialization op on SavedModel bundle at path: /mnt/c/Users/Lucaro/Workspace/cineast/cineast-runtime/../resources/universal-sentence-encoder-multilingual-large_3
2022-01-06 13:12:38.426469: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:277] SavedModel load for tags { serve }; Status: success: OK. Took 3173164 microseconds.
[-0.025447764, 0.05077521, -0.033019744, 0.078675404, 0.01637241, 0.06915054, -0.05788038, -0.023094412, -0.02089644, 0.02315649, 0.009380558, -0.021741828, -0.008661511, -0.0459719, 0.0042683827, -0.0513097, -0.01917885, -0.02274911
...

Output from Windows

2.4.1
found 12 libraries
could not load _constrained_sequence_op.so
could not load _mst_ops.so
could not load _normalize_ops.so
could not load _regex_split_ops.so
could not load _sentencepiece_tokenizer.so
could not load _sentence_breaking_ops.so
could not load _split_merge_from_logits_tokenizer.so
could not load _split_merge_tokenizer.so
could not load _state_based_sentence_breaker_op.so
could not load _unicode_script_tokenizer.so
could not load _whitespace_tokenizer.so
could not load _wordpiece_tokenizer.so
C:\Users\Lucaro\Workspace\cineast\resources\universal-sentence-encoder-multilingual-large_3
2022-01-06 13:30:06.993614: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:32] Reading SavedModel from: C:\Users\Lucaro\Workspace\cineast\resources\universal-sentence-encoder-multilingual-large_3
2022-01-06 13:30:07.024864: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:55] Reading meta graph with tags { serve }
2022-01-06 13:30:07.024895: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:93] Reading SavedModel debug info (if present) from: C:\Users\Lucaro\Workspace\cineast\resources\universal-sentence-encoder-multilingual-large_3
2022-01-06 13:30:07.025022: I external/org_tensorflow/tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-01-06 13:30:07.174499: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:206] Restoring SavedModel bundle.
2022-01-06 13:30:09.339210: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:190] Running initialization op on SavedModel bundle at path: C:\Users\Lucaro\Workspace\cineast\resources\universal-sentence-encoder-multilingual-large_3
2022-01-06 13:30:10.152906: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:277] SavedModel load for tags { serve }; Status: success: OK. Took 3159272 microseconds.
Exception in thread "main" org.tensorflow.exceptions.TensorFlowException: Op type not registered 'SentencepieceOp' in binary running on LUCARO-DESKTOP-. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) `tf.contrib.resampler` should be done before importing the graph, as contrib ops are lazily registered when the module is first accessed.
	at org.tensorflow.internal.c_api.AbstractTF_Status.throwExceptionIfNotOK(AbstractTF_Status.java:101)
	at org.tensorflow.SavedModelBundle.load(SavedModelBundle.java:418)
	at org.tensorflow.SavedModelBundle.access$000(SavedModelBundle.java:59)
	at org.tensorflow.SavedModelBundle$Loader.load(SavedModelBundle.java:68)
	at org.tensorflow.SavedModelBundle.load(SavedModelBundle.java:242)
	at org.vitrivr.cineast.standalone.Playground.main(Playground.java:45)

About this issue

Original URL
State: open
Created 2 years ago
Comments: 17 (6 by maintainers)

Most upvoted comments

Thanks for the dependency dump @lucaro , yes looks like it is _pywrap_tensorflow_internal.pyd that is playing its dark magic on Windows.

Like @saudet suggested (supposedly many times 😉 ), we can try building artifacts that links directly to this binary instead of the TensorFlow build we distribute. If that works, we could distribute that artifact as a new platform for TensorFlow Java (e.g. windows-py) to let people use it, at the cost of a (useless) additional dependency to CPython.

I understand you are not setup for building it, so we can probably create an additional setup in our CI build to test it out, but if you are eager to try it out now, you can look at this page for more guidance on how to prepare your environment to do it: https://github.com/tensorflow/java/blob/master/CONTRIBUTING.md

Please let me know if you plan to try it, thanks!

karllessard on Jan 7, 2022