onnxruntime: ValueError: DmlExecutionProvider does not contain a subset of available providers ['CPUExecutionProvider', 'DmlExecutionProvider']

When trying to set the provider to DmlExecutionProvider, it fails with the following error :

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
d:\Codes\fac_ver\python\test_beds\testbed_stuff.py in 
     325 providers = onnxruntime.get_available_providers()
     326 print(f'current device: {onnxruntime.get_device()}')
---> 327 sess.set_providers(providers[1])
     328 
     329 

~\Anaconda3\Lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py in set_providers(self, providers, provider_options)
     68         if not set(providers).issubset(C.get_available_providers()):
     69             raise ValueError("{} does not contain a subset of available providers {}".format(
---> 70                 providers, C.get_available_providers()))
     71 
     72         if provider_options:

ValueError: DmlExecutionProvider does not contain a subset of available providers ['CPUExecutionProvider', 'DmlExecutionProvider']

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10 1909
ONNX Runtime installed from (source or binary): built from source
ONNX Runtime version: 1.5.2
Python version: 3.7.4
Visual Studio version (if applicable): Microsoft Visual Studio Enterprise 2019 (3) Version 16.7.5
GCC/Compiler version (if compiling from source): Visual C++ v142
CUDA/cuDNN version: None
GPU model and memory: Intel® Iris® Pro Graphics 580

To Reproduce Run :

import numpy as np
import onnxruntime

model_path = 'r18_q_onnx.onnx'
img_input= np.random.randn(1, 3, 112, 112)
img_input= np.asarray(res, dtype=np.float32) 

sess = onnxruntime.InferenceSession(model_path)
ort_inputs = {sess.get_inputs()[0].name: img_input}
ort_outs = sess.run(None, ort_inputs)

providers = onnxruntime.get_available_providers()
print(f'available providers : {providers}')
print(f'current device: {onnxruntime.get_device()}')

sess.set_providers(providers[-1])
ort_outs = sess.run(None, ort_inputs)

Onnx model : https://gofile.io/d/GdkIeR

Expected behavior Should be able to execute this on GPU using DirectML?

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 15 (5 by maintainers)

Most upvoted comments

Any luck on finding out the culprit?

Update posted on the related Issue #50 in the DirectML repo.

wchao1115 on Nov 8, 2020

@Coderx7 let us look into this issue a bit more. DirectML operator coverage in onnxruntime is pretty extensive. It supports up to ONNX opset-12 in the v1.5 distribution, and definitely should not be slower than the CPU on most DNN models.

wchao1115 on Oct 31, 2020

OnnxRuntime can only work off what DML library can execute. Let me give one toy example: let us say there is a simple graph: nodeA -> node B -> node C. If DML tells OnnxRuntime that it can only execute nodeB. OnnxRuntime will assign node A and nodeC to CPU. So the final graph will look like this: nodeA -> copy to GPU -> node B -> copy to CPU -> nodeC. Two GPU <-> CPU copy nodes are now added and it may take away all the hardware acceleration gains of nodeB as well as add more overhead. So in theory, this “hardware accelerated” graph could perform worse than just nodeA -> nodeB -> nodeC all running on CPU with no device data copies involved. Keep in mind that the CPU EP is a pretty optimal backend by itself. It is multi-threaded and uses vectorized instructions to provide maximum perf gains as possible on CPU.

hariharans29 on Oct 31, 2020