cmr: cupy.cuda.compiler.CompileException: nvrtc: error: failed to load builtins
When you run the demo: python -m cmr.demo --name bird_net --num_train_epoch 500 --img_path cmr/demo_data/img1.jpg
I am getting this error:
/home/ubuntu/akshay/cmr/venv_cmr/local/lib/python2.7/site-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float
to np.floating
is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type
.
from ._conv import register_converters as _register_converters
Setting up model…
loading /home/ubuntu/akshay/cmr/nnutils/…/cachedir/snapshots/bird_net/pred_net_500.pth…
Traceback (most recent call last):
File “/usr/lib/python2.7/runpy.py”, line 174, in _run_module_as_main
“main”, fname, loader, pkg_name)
File “/usr/lib/python2.7/runpy.py”, line 72, in _run_code
exec code in run_globals
File “/home/ubuntu/akshay/cmr/demo.py”, line 119, in <module>
app.run(main)
File “/home/ubuntu/akshay/cmr/venv_cmr/local/lib/python2.7/site-packages/absl/app.py”, line 274, in run
_run_main(main, argv)
File “/home/ubuntu/akshay/cmr/venv_cmr/local/lib/python2.7/site-packages/absl/app.py”, line 238, in _run_main
sys.exit(main(argv))
File “/home/ubuntu/akshay/cmr/demo.py”, line 108, in main
outputs = predictor.predict(batch)
File “cmr/nnutils/predictor.py”, line 110, in predict
self.forward()
File “cmr/nnutils/predictor.py”, line 149, in forward
self.cam_pred)
File “cmr/nnutils/nmr.py”, line 183, in forward
return Render(self.renderer)(verts, faces)
File “cmr/nnutils/nmr.py”, line 114, in forward
masks = self.renderer.forward_mask(vs, fs)
File “cmr/nnutils/nmr.py”, line 50, in forward_mask
self.masks = self.renderer.render_silhouettes(self.vertices, self.faces)
File “build/bdist.linux-x86_64/egg/neural_renderer/renderer.py”, line 38, in render_silhouettes
File “/home/ubuntu/akshay/cmr/venv_cmr/local/lib/python2.7/site-packages/chainer/functions/array/concat.py”, line 90, in concat
y, = Concat(axis).apply(xs)
File “/home/ubuntu/akshay/cmr/venv_cmr/local/lib/python2.7/site-packages/chainer/function_node.py”, line 245, in apply
outputs = self.forward(in_data)
File “/home/ubuntu/akshay/cmr/venv_cmr/local/lib/python2.7/site-packages/chainer/functions/array/concat.py”, line 44, in forward
return xp.concatenate(xs, self.axis),
File “/home/ubuntu/akshay/cmr/venv_cmr/local/lib/python2.7/site-packages/cupy/manipulation/join.py”, line 49, in concatenate
return core.concatenate_method(tup, axis)
File “cupy/core/core.pyx”, line 2439, in cupy.core.core.concatenate_method
File “cupy/core/core.pyx”, line 2482, in cupy.core.core.concatenate_method
File “cupy/core/core.pyx”, line 2533, in cupy.core.core.concatenate
File “cupy/core/core.pyx”, line 1630, in cupy.core.core.ndarray.setitem
File “cupy/core/core.pyx”, line 3101, in cupy.core.core._scatter_op
File “cupy/core/elementwise.pxi”, line 823, in cupy.core.core.ufunc.call
File “cupy/util.pyx”, line 39, in cupy.util.memoize.decorator.ret
File “cupy/core/elementwise.pxi”, line 622, in cupy.core.core._get_ufunc_kernel
File “cupy/core/elementwise.pxi”, line 33, in cupy.core.core._get_simple_elementwise_kernel
File “cupy/core/carray.pxi”, line 170, in cupy.core.core.compile_with_cache
File “/home/ubuntu/akshay/cmr/venv_cmr/local/lib/python2.7/site-packages/cupy/cuda/compiler.py”, line 123, in compile_with_cache
base = _preprocess(‘’, options, arch)
File “/home/ubuntu/akshay/cmr/venv_cmr/local/lib/python2.7/site-packages/cupy/cuda/compiler.py”, line 86, in _preprocess
result = prog.compile(options)
File “/home/ubuntu/akshay/cmr/venv_cmr/local/lib/python2.7/site-packages/cupy/cuda/compiler.py”, line 233, in compile
raise CompileException(log, self.src, self.name, options)
cupy.cuda.compiler.CompileException: nvrtc: error: failed to load builtins
I tried the solution mentioned in this: https://groups.google.com/forum/#!topic/chainer-jp/GKNe5KY_fm0 yet it is not working
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Reactions: 1
- Comments: 17
Commits related to this issue
- Merge pull request #4 from daniilidis-group/devel Added Python 2 support, resolves #3 — committed to chenyuntc/cmr by nkolot 6 years ago
@charliememory did you use chainer 3.3.0 or the latest version? With chainer==3.3.0 I keep getting an error:
RuntimeError: CUDA environment is not correctly set up (see https://github.com/chainer/chainer#installation).cannot import name sqrt_fixed
I checkedchainer.backends.cuda.available
and it returned False. With the new version of chainer, I get the nvrtc error above even though I have set the CUDA_PATH and LD_LIBRARY_PATH as instructed in the cupy documentation.