docling: Error building extension 'MultiScaleDeformableAttention' when running sample from web site.

Bug

When running the sample app, I get these errors:

Could not load the custom kernel for multi-scale deformable attention: Error building extension 'MultiScaleDeformableAttention': [1/2] /usr/bin/nvcc --generate-dependencies-with-compile --dependency-output ms_deform_attn_cuda.cuda.o.d -DTORCH_EXTENSION_NAME=MultiScaleDeformableAttention -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr -isystem /mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/torch/include -isystem /mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/torch/include/TH -isystem /mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/torch/include/THC -isystem /usr/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -std=c++17 -c /mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu -o ms_deform_attn_cuda.cuda.o 
FAILED: ms_deform_attn_cuda.cuda.o 
/usr/bin/nvcc --generate-dependencies-with-compile --dependency-output ms_deform_attn_cuda.cuda.o.d -DTORCH_EXTENSION_NAME=MultiScaleDeformableAttention -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr -isystem /mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/torch/include -isystem /mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/torch/include/TH -isystem /mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/torch/include/THC -isystem /usr/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -std=c++17 -c /mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu -o ms_deform_attn_cuda.cuda.o 
/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu:19:9: warning: #pragma once in main file
   19 | #pragma once
      |         ^~~~
/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu:19:9: warning: #pragma once in main file
   19 | #pragma once
      |         ^~~~
/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_im2col_cuda.cuh(261): warning #177-D: variable "q_col" was declared but never referenced
          detected during instantiation of "void ms_deformable_im2col_cuda(cudaStream_t, const scalar_t *, const int64_t *, const int64_t *, const scalar_t *, const scalar_t *, int, int, int, int, int, int, int, scalar_t *) [with scalar_t=double]" 
/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu(69): here

/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_im2col_cuda.cuh(762): warning #177-D: variable "q_col" was declared but never referenced
          detected during instantiation of "void ms_deformable_col2im_cuda(cudaStream_t, const scalar_t *, const scalar_t *, const int64_t *, const int64_t *, const scalar_t *, const scalar_t *, int, int, int, int, int, int, int, scalar_t *, scalar_t *, scalar_t *) [with scalar_t=double]" 
/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu(140): here

/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_im2col_cuda.cuh(872): warning #177-D: variable "q_col" was declared but never referenced
          detected during instantiation of "void ms_deformable_col2im_cuda(cudaStream_t, const scalar_t *, const scalar_t *, const int64_t *, const int64_t *, const scalar_t *, const scalar_t *, int, int, int, int, int, int, int, scalar_t *, scalar_t *, scalar_t *) [with scalar_t=double]" 
/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu(140): here

/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_im2col_cuda.cuh(331): warning #177-D: variable "q_col" was declared but never referenced
          detected during instantiation of "void ms_deformable_col2im_cuda(cudaStream_t, const scalar_t *, const scalar_t *, const int64_t *, const int64_t *, const scalar_t *, const scalar_t *, int, int, int, int, int, int, int, scalar_t *, scalar_t *, scalar_t *) [with scalar_t=double]" 
/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu(140): here

/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_im2col_cuda.cuh(436): warning #177-D: variable "q_col" was declared but never referenced
          detected during instantiation of "void ms_deformable_col2im_cuda(cudaStream_t, const scalar_t *, const scalar_t *, const int64_t *, const int64_t *, const scalar_t *, const scalar_t *, int, int, int, int, int, int, int, scalar_t *, scalar_t *, scalar_t *) [with scalar_t=double]" 
/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu(140): here

/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_im2col_cuda.cuh(544): warning #177-D: variable "q_col" was declared but never referenced
          detected during instantiation of "void ms_deformable_col2im_cuda(cudaStream_t, const scalar_t *, const scalar_t *, const int64_t *, const int64_t *, const scalar_t *, const scalar_t *, int, int, int, int, int, int, int, scalar_t *, scalar_t *, scalar_t *) [with scalar_t=double]" 
/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu(140): here

/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_im2col_cuda.cuh(649): warning #177-D: variable "q_col" was declared but never referenced
          detected during instantiation of "void ms_deformable_col2im_cuda(cudaStream_t, const scalar_t *, const scalar_t *, const int64_t *, const int64_t *, const scalar_t *, const scalar_t *, int, int, int, int, int, int, int, scalar_t *, scalar_t *, scalar_t *) [with scalar_t=double]" 
/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu(140): here

/usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
  435 |         function(_Functor&& __f)
      |                                                                                                                                                 ^ 
/usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
/usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
  530 |         operator=(_Functor&& __f)
      |                                                                                                                                                  ^ 
/usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
ninja: build stopped: subcommand failed.

…

Steps to reproduce

pip install docling

Then create a script with:

from docling.document_converter import DocumentConverter

source = "https://arxiv.org/pdf/2408.09869"  # PDF path or URL
converter = DocumentConverter()
result = converter.convert(source)
print(result.document.export_to_markdown())  # output: "### Docling Technical Report[...]"

Running this code produces the error above.

…

Docling version

Docling version: 2.12.0 Docling Core version: 2.10.0 Docling IBM Models version: 3.1.0 Docling Parse version: 3.0.0 …

Python version

Python 3.10.12

…

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 17 (3 by maintainers)

Most upvoted comments

@dromeuf what was your Ubuntu and CUDA versions? I tried this on a fresh container with latest cuda (from Nvidia’s image: Ubuntu 24.04 + CUDA 12.8.0) with no luck.

This might really be a bug in Transformers however as the source of the problem is the custom kernel in transformers/kernels/deformable_detr: https://github.com/huggingface/transformers/tree/7eecdf2a8650306ed5fbb6150c64f99f587e004d/src/transformers/kernels/deformable_detr/

Hello, this problem occurs reliably on a default install of Ubuntu 22.04 + docling from PyPI. Basically it’s the usual C++ being incompatible with C++ and CUDA being incompatible with CUDA nonsense. The issue and the solution are documented here: https://github.com/NVIDIA/nccl/issues/650

If reinstalling or updating CUDA doesn’t solve it (try that first) then this will:

sudo apt install gcc-10 g++-10
export CC=gcc-10 CXX=g++-10

The CUDA toolkit version installed system wide (and thus the nvcc version) is 11.5.1-1ubuntu1 while what torch pulls in is 12.4.127, which may have something to do with it, but in any case convincing nvcc to use GCC 10 solves the problem.

We never encounter (yet) such an error. I’m a bit suspicious about the CUDA and Pytorch versions. I would recommend making sure torch and cuda are compatible, e.g. using the install methods listed on https://pytorch.org/.

Uninstalled and reinstalled cuda-toolkit and that seemed to have fixed it. Thanks.

I got the same error,but i still can convert it to MarkDown.And this error may disappear after you update gcc version.

/lib/python3.9/site-packages/torch/include/c10/util/C++17.h:13:2: error: #error "You're trying to build PyTorch with a too old version of GCC. We need GCC 9 or later."