onnxruntime: ONNX Runtime Test Dependency, Unable to Skip Tests
Describe the bug A clear and concise description of what the bug is.
ONNX Runtime test after installation has implicit dependency on scipy. Using --skip_onnx_tests was not able to skip all the test.
2019-11-19 18:18:59,488 Build [DEBUG] - Running subprocess in '/tmp/onnxruntime/build/Linux/Debug'
['/usr/bin/python3', '/tmp/onnxruntime/onnxruntime/test/onnx/gen_test_models.py', '--output_dir', 'test_models']
Traceback (most recent call last):
File "/tmp/onnxruntime/onnxruntime/test/onnx/gen_test_models.py", line 12, in <module>
from scipy.spatial import distance
ModuleNotFoundError: No module named 'scipy'
Traceback (most recent call last):
File "/tmp/onnxruntime/tools/ci_build/build.py", line 1043, in <module>
sys.exit(main())
File "/tmp/onnxruntime/tools/ci_build/build.py", line 987, in main
args.use_dnnlibrary)
File "/tmp/onnxruntime/tools/ci_build/build.py", line 610, in run_onnxruntime_tests
run_subprocess([sys.executable, os.path.join(source_dir,'onnxruntime','test','onnx','gen_test_models.py'),'--output_dir','test_models'], cwd=cwd)
File "/tmp/onnxruntime/tools/ci_build/build.py", line 196, in run_subprocess
return subprocess.run(args, cwd=cwd, check=True, stdout=stdout, stderr=stderr, env=my_env, shell=shell)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/onnxruntime/onnxruntime/test/onnx/gen_test_models.py', '--output_dir', 'test_models']' returned non-zero exit status 1.
Urgency If there are particular important use cases blocked by this or strict project-related timelines, please share more information and dates. If there are no hard deadlines, please specify none.
System information
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): 18.04
- ONNX Runtime installed from (source or binary): source
- ONNX Runtime version: rel-1.0.0
- Python version: 3.6.8
- Visual Studio version (if applicable):
- GCC/Compiler version (if compiling from source): 7.4.0
- CUDA/cuDNN version: 10.0/7.6
- GPU model and memory: V100
To Reproduce Describe steps/code to reproduce the behavior:
$ git clone https://github.com/microsoft/onnxruntime.git
$ cd onnxruntime/
# Select a branch for stable release
$ git checkout rel-1.0.0
# Update Git submodules
$ git submodule update --init --recursive
$ ./build.sh --cuda_home /usr/local/cuda --cudnn_home /usr/lib/x86_64-linux-gnu/ --parallel --use_full_protobuf --build_wheel
$ cd build/Linux/Debug/
$ python ../../../setup.py install
Expected behavior A clear and concise description of what you expected to happen.
Dependency should be handled by the installation package instead of the user. The user should not have seen such errors during installation.
Screenshots If applicable, add screenshots to help explain your problem.
Additional context Add any other context about the problem here. If the issue is about a particular model, please share the model details as well to facilitate debugging.
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 15 (2 by maintainers)
Commits related to this issue
- Various fixes coming out of discussions in #2436 - Add --skip_tests option to build.py based on github feedback - Add debug output at end of run_subprocess so it's clearer when the output is from... — committed to microsoft/onnxruntime by skottmckay 5 years ago
- Various fixes coming out of discussions in #2436 (#2497) - Add --skip_tests option to build.py based on github feedback - Add debug output at end of run_subprocess so it's clearer when the output ... — committed to microsoft/onnxruntime by skottmckay 5 years ago
It should work with python 3.6. setup.py however swallows any import errors which might be the cause of your issue. Seems pretty similar to this: https://github.com/intel-isl/Open3D/pull/1012/files
Do you have the ‘wheel’ package installed for python 3.6?
Regarding this:
The test isn’t actually failing, The first line is actually listing that there were no failed test cases. The following lines are a warning message from the next step in the testing process. I’ll see if a message can be inserted in between so it’s clearer that the script the ‘Failed Test Cases:’ is coming from completes.
Finally, would it be helpful to have a ‘–skip_tests’ parameter that skipped all tests?
I had the same question in #1938 this means the
--update --buildto skip tests in counter-intuitive when there is a--skip_onnx_testsoption.There are 3 phases: update, build and test. Default is to do all three. Running the onnx tests is just a part of all the potential tests.
Add ‘–update --build’ to skip all the tests.