decord: Resource temporarily unavailable when calling VideoReader or VideoLoader multiple times

Hi Joshua,

I tried to call VideoReader() or VideoLoader() various times but it crashed in the middle. To reproduce the error, I provide the sample code below:

import decord
for i in range(250):
    print(i)
    #a = decord.VideoLoader(['/mnt/SSD/Kinetics_trimmed_videos_train_merge/7E60em35UUw.mp4'], shape=(8,340,256,3), interval=8, skip=0,shuffle=3)
    a = decord.VideoReader('/mnt/SSD/Kinetics_trimmed_videos_train_merge/7E60em35UUw.mp4')
    # a.reset()

The program shuts down when i is 230 or 231 with an error message saying:

Traceback (most recent call last):
  File "test_code.py", line 6, in <module>
    a = decord.VideoReader('/mnt/SSD/Kinetics_trimmed_videos_train_merge/7E60em35UUw.mp4')
  File "/home/yzhao/.local/lib/python3.6/site-packages/decord-0.0.1-py3.6.egg/decord/video_reader.py", line 21, in __init__
    uri, ctx.device_type, ctx.device_id, width, height)
  File "/home/yzhao/.local/lib/python3.6/site-packages/decord-0.0.1-py3.6.egg/decord/_ffi/_ctypes/function.py", line 175, in __call__
    ctypes.byref(ret_val), ctypes.byref(ret_tcode)))
  File "/home/yzhao/.local/lib/python3.6/site-packages/decord-0.0.1-py3.6.egg/decord/_ffi/base.py", line 62, in check_call
    raise DECORDError(py_str(_LIB.DECORDGetLastError()))
decord._ffi.base.DECORDError: Resource temporarily unavailable

For VideoLoader, the error message is similar:

Traceback (most recent call last):
  File "test_code.py", line 4, in <module>
    a = decord.VideoLoader(['/mnt/SSD/Kinetics_trimmed_videos_train_merge/7E60em35UUw.mp4'], shape=(8,340,256,3), interval=8, skip=0,shuffle=3)
  File "/home/yzhao/.local/lib/python3.6/site-packages/decord-0.0.1-py3.6.egg/decord/video_loader.py", line 24, in __init__
    uri, shape[0], shape[1], shape[2], shape[3], interval, skip, shuffle, prefetch)
  File "/home/yzhao/.local/lib/python3.6/site-packages/decord-0.0.1-py3.6.egg/decord/_ffi/_ctypes/function.py", line 175, in __call__
    ctypes.byref(ret_val), ctypes.byref(ret_tcode)))
  File "/home/yzhao/.local/lib/python3.6/site-packages/decord-0.0.1-py3.6.egg/decord/_ffi/base.py", line 62, in check_call
    raise DECORDError(py_str(_LIB.DECORDGetLastError()))
decord._ffi.base.DECORDError: Resource temporarily unavailable

The environment that I use is also attached for reference

Ubuntu 18.04
ffmpeg 3.4.6-0ubuntu0.18.04.1

Will appreciate it if you could look into this. Thanks!

About this issue

Original URL
State: open
Created 5 years ago
Comments: 23 (9 by maintainers)

Most upvoted comments

Hi @zhreshold

When working with video list (say Kinetics-700 has 500K videos), consistent memory increase is observed which indicates potential resource leak.

A simple test snippet would looks like:

import decord
video_list = ['xx00001.mp4', 'xx00002.mp4', ... 'xx99999.mp4']
for filename in video_list:
    print(i)
    a = decord.VideoReader(filename)

For the test code above, a server with ~256GB memory will be filled up after accessing around 300K videos. (Therefore it is not observed until I recently play with the full Kinetics-700 dataset) In my experiment with MMAction, the memory will be full after reading ~50K videos.

I tried to dig a little bit into the code (https://github.com/zhreshold/decord/blob/master/src/video/video_reader.cc#L79) and find if I do something like AVFormatContext *fmt_ctx = fmt_ctx_.get(); avformat_free_context(fmt_ctx);, a segmentation fault would raise. Would you kindly have a look? Thanks!

zhaoyue-zephyrus on Jul 9, 2019