machinelearning: ML.Net is not able to load Tensorflow GPU DLL...
I am trying work with TF GPU native dll so that I can speed up the scoring process. I replaced the tensorflow.dll in microsoft.ml.tensorflow.redist\0.5.0\runtimes\win-x64\native with gpu version but ML.Net throws DLL not found exception.
I did the same with TensorflowSharp and it loads the GPU version perfectly fine.
SOLVED
Updating To TensorFlow GPU DLL
If you want to use TF GPU dll during scoring follow the steps below
- In your ML.Net nuget, replace the
tensorflow.dllinmicrosoft.ml.tensorflow.redist\<ML.Net-Version>\runtimes\win-x64\nativewith gpu version (please keep the name same). - You can grap the prebuilt gpu dll from https://github.com/fo40225/tensorflow-windows-wheel.
- Update the graphics card drivers, CUDA and CuDNN according the Tensorflow gpu version you are using.
- TensorFlowScorer will automatically pick up the TF GPU version.
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 24 (13 by maintainers)
UPDATE: I was using CUDA 10.1 not 10.0, so it wasn’t working. Moving back to 10.0 and the tensor flow GPU dll loads ok.
However, I’m not seeing any GPU usage. I’m testing demo code based on https://docs.microsoft.com/en-us/dotnet/machine-learning/tutorials/image-classification
Thanks
Taking a look at your steps: I wouldn’t recommend that folks update the nuget package. The Nuget package cache is shared between all projects. Lots of parts of the build treat it as immutable.
Instead, to use the GPU binary, you can exclude the native assets from the Microsoft.ML.Tensorflow package reference
This will tell NuGet not to include the native tensorflow binaries (it applies transitively to the dependencies, and will thus apply to TensorFlow.Redist package).
Then they can “bring their own” copy of tensorflow however they choose. Simply referencing it as content is one way. If we want to officially support this scenario we could consider a packaging story for the GPU build (as well as an official way for someone to choose to use this).