cudf: RuntimeError: radix_sort: failed on 2nd step: cudaErrorInvalidValue: invalid argument

context: https://github.com/rapidsai/cuml/issues/4048#issuecomment-878747900

import cudf
import cupy
import pandas as pd

X = pd.read_csv("df.csv")
y = X['target'].copy()

X = cudf.from_pandas(X).fillna(0.0)
y = cudf.from_pandas(y).fillna(0.0)

ar = cupy.asarray(y).flatten()
ar.sort()

Traceback (most recent call last):
  File "testradix.py", line 12, in <module>
    ar.sort()
  File "cupy/_core/core.pyx", line 695, in cupy._core.core.ndarray.sort
  File "cupy/_core/core.pyx", line 713, in cupy._core.core.ndarray.sort
  File "cupy/_core/_routines_sorting.pyx", line 43, in cupy._core._routines_sorting._ndarray_sort
  File "cupy/cuda/thrust.pyx", line 75, in cupy.cuda.thrust.sort
RuntimeError: radix_sort: failed on 2nd step: cudaErrorInvalidValue: invalid argument

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 15 (5 by maintainers)

Most upvoted comments

For me this error turned out to have to do with cuda 11+ toolkit not working properly on sm35 gpus. Current workaround is uninstalling cupy that gets installed with rapids then install cuda 10.2 (alongside with cuda 11.+) then install cupy-cuda10.2 while having rapids venv active.