opencv_contrib: cv2.cudacodec.createVideoReader error when input is a video stream

System information (version)
  • OpenCV => 4.5.2
  • Operating System / Platform => linux
  • Compiler => gcc
Detailed description

I built opencv_contrib_python by myself. Here is the build information

Extra dependencies:          m pthread cudart_static dl rt nppc nppial nppicc nppidei nppif nppig nppim nppist nppisu nppitc npps cublas cudnn cufft -L/usr/local/cuda/lib64 -L/lib64
    3rdparty dependencies:

  OpenCV modules:
    To be built:                 calib3d core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudastereo cudawarping cudev dnn features2d flann highgui imgcodecs imgproc ml objdetect photo python3 stitching video videoio ximgproc
    Disabled:                    aruco bgsegm bioinspired ccalib datasets dnn_objdetect dnn_superres dpm face fuzzy hfs img_hash intensity_transform line_descriptor mcc optflow phase_unwrapping plot quality rapid reg rgbd saliency shape stereo structured_light superres surface_matching text tracking videostab wechat_qrcode world xfeatures2d xobjdetect xphoto
    Disabled by dependency:      cudaoptflow
    Unavailable:                 alphamat cnn_3dobj cvv freetype gapi hdf java julia matlab ovis python2 sfm ts viz
    Applications:                -
    Documentation:               NO
    Non-free algorithms:         NO

  GUI: 
    GTK+:                        NO

  Media I/O: 
    ZLib:                        /usr/local/lib/libz.so (ver 1.2.12)
    JPEG:                        /usr/local/lib/libjpeg.so (ver 90)
    WEBP:                        build (ver encoder: 0x020f)
    PNG:                         /usr/local/lib/libpng.so (ver 1.6.37)
    TIFF:                        build (ver 42 - 4.2.0)
    JPEG 2000:                   build (ver 2.4.0)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    DC1394:                      NO
    FFMPEG:                      YES
      avcodec:                   YES (58.91.100)
      avformat:                  YES (58.45.100)
      avutil:                    YES (56.51.100)
      swscale:                   YES (5.7.100)
      avresample:                NO
    GStreamer:                   NO
    v4l/v4l2:                    YES (linux/videodev2.h)

  Parallel framework:            pthreads

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    VA:                          NO
    Lapack:                      NO
    Eigen:                       NO
    Custom HAL:                  NO
    Protobuf:                    build (3.5.1)

  NVIDIA CUDA:                   YES (ver 11.0, CUFFT CUBLAS NVCUVID)
    NVIDIA GPU arch:             35 37 50 52 60 61 70 75 80
    NVIDIA PTX archs:

  cuDNN:                         YES (ver 8.1.1)

I want to use cuda to accelerate video decoding, but only found it worked on files. I want to use it on video stream.

Steps to reproduce
import cv2
url="rtsp://admin:admin@192.168.2.46/media/video1"

cpu:

cv2.VideoCapture(url).read() # worked

gpu:

cv2.cudacodec.createVideoReader(url).next_frame()

get

opencv_contrib/modules/cudacodec/src/video_reader.cpp:128: error: (-210:Unsupported format or combination of formats) Unsupported video source in function 'nextFrame'

but if the url is a video file both cpu and gpu worked.

Issue submission checklist
  • I report the issue, it’s not a question
  • I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
  • I updated to the latest OpenCV version and the issue is still there
  • There is reproducer code and related data files: videos, images, onnx, etc

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 36 (14 by maintainers)

Most upvoted comments

OK,just wait , i will update to opencv 4.6.0 which contains your MR and then test it next week.@cudawarped

Right,thank you.

Is nvidia-video-sdk the runtime dependency or compiling denpendency or both?

It is a compile time dependancy. As you have discovered if it cannot be found, NVCUVID is missing from the cmake output but you can still compile cv::cudacodec. However it is not implemented, because nvidia-video-sdk is not available, and you get the error messages you observed.

Also which nvidia-video-sdk is the real one which opencv need? I have downloaded nvidia-video-sdk and my nvidia driver have the so with the same name.

cp Video_Codec_SDK_11.1.5/Lib/linux/stubs/x86_64/* /usr/local/cuda/lib64
cp Video_Codec_SDK_11.1.5/Interface/* /usr/local/cuda/include

and build opencv then it works. But i found that Video_Codec_SDK_11.1.5 and my nvidia driver both have the libnvcuvid.so and libnvidia-encode.so. Same name, different size.

From the documentation:

Stub libraries (libnvcuvid.so and libnvidia-encode.so) have been included as part of the SDK package, in order to aid development of applications on systems where the NVIDIA driver has not been installed.

Meaning that, as you discovered, as long as CMake can find either the stubs or the driver libraries cv::cudacodec should be functional.

Also ffmpeg can compile only with https://github.com/FFmpeg/nv-codec-headers/tree/n11.1.5.1 and without nvidia-video-sdk But opencv need nvidia-video-sdk,nv-codec-headers is not useful for opencv . So what is the differece between them

I am not sure why this is important and I think this thread is getting off track. I would say with reference to the thread topic that it is probably a good idea to allow the user to disable the maxUnparsedPackets check when they are streaming and I will see if I can add this option to cudacodec::createVideoReader().

I would be surprised if you can get anywhere near that number without hitting 100% CPU usage. With VideoReader I can easily do 32 1080p h265 streams on GTX 10 series.

@cudawarped

If its easy try setting maxUnparsedPackets = 100; and re-building

It works ! WOW Amazing.

>>> cap.nextFrame()
(True, <cuda_GpuMat 0x7f06c5a84190>)
>>> cap.nextFrame()
(True, <cuda_GpuMat 0x7f06c5a841b0>)
>>> cap.nextFrame()
(True, <cuda_GpuMat 0x7f06c5a841d0>)
>>> cap.nextFrame()
(True, <cuda_GpuMat 0x7f06c5a84170>)
>>> cap.nextFrame()
(True, <cuda_GpuMat 0x7f06c5a84190>)
>>> cap.nextFrame()
(True, <cuda_GpuMat 0x7f06c5a841b0>)
>>> cap.nextFrame()
(True, <cuda_GpuMat 0x7f06c5a841d0>)