SlowFast: Torchvision backend not working.

Facing the same error when using Pytorch 1.11.0 and torchvision 0.12.0. The exact error message is: RuntimeError: No such operator video_reader::probe_video_from_memory

Switching to pyav throws the following error: pyav with exception: unsupported operand type(s) for -: 'list' and 'int'

How do you check if torchvision is able to detect ffmpeg?

The culprit is due to the changes in commit 99a655bd533d7fddd7f79509e3dfaae811767b5c for adding unsupervised training. num_frames is now a list, and the torchvision implementation is updated, but not the pyav. See https://github.com/facebookresearch/SlowFast/blob/main/slowfast/datasets/decoder.py#L295-L337 vs. https://github.com/facebookresearch/SlowFast/blob/main/slowfast/datasets/decoder.py#L417-L429.

num_frames - 1 doesn’t work anymore since num_frames isn’t an int anymore. @feichtenhofer could you consider fixing this as it seems the torchvision backend doesn’t work without building torchvision from source? A lot of us are still using pyav as the backend.

I am using pytorch 1.12.0 and torchvision 0.13.0 and still receiving the error:

Failed to decode by torchvision with exception: '_OpNamespace' object has no attribute 'probe_video_from_memory'
[07/08 19:06:20][WARNING] kinetics.py:  372: Failed to decode video idx 217946 from /home/alp/Desktop/kinetics-dataset/k400/train/playing_ice_hockey/thF1B8Nb8Nw_000023_000033.mp4; trial 99

When I changed the backend to PyAV, now I receive the error:

Failed to decode by pyav with exception: unsupported operand type(s) for -: 'list' and 'int'
[07/08 19:08:14][WARNING] kinetics.py:  372: Failed to decode video idx 175784 from /home/alp/Desktop/kinetics-dataset/k400/train/playing_flute/5ShceiaaL7s_000011_000021.mp4; trial 99

Hence, there is still no fix to this issue in the latest versions of PyTorch and torchvision. However, the changes @dfan suggested in the pyav_decode() function in this pull request work, and now I am able to run the framework with PyAv backend.

Took a crack at fixing it. #541

I just ran into the same problem with pytorch1.6.0 torchvision 0.7.0, same output “Failed to decode by torchvision with exception: No such operator video_reader::probe_video_from_memory”, if I ran slowfast with pyav backends, it totally works. But I ran X3D with torchvision as data decode backends, it reports errors.

Hi everyone, now I am finally able to work with both torchvision and PyAV video backends. I created my own fork of slowfast with updated and corrected installation steps (INSTALL.md) here. I hope my documentation for the installation process helps you.

Hi @takatosp1, Is there any reason that you’re using the master branch of the TorchVision instead of the stable release? In the INSTALL.md it didn’t say that TorchVision has to be the master branch.

Also, I had no luck with trying to install TorchVision from source and TorchVision doesn’t have a nightly build on conda.

have you solved this problem already

[UPDATE]

After days of exploring, I found some clues why compiling the torchvision from source fails.

As suggested here, the torchvision doesn’t support compiling from source with MacOS or with Python 3.9. So, you should set up another new env.

Alternatively, simply removing the py39 checking also works well in my case (ubuntu + ffmpeg4.3 installed from conda). Hope this helps.

Please check out this pull request: https://github.com/facebookresearch/SlowFast/pull/541 I copied and pasted @dfan’s changes in the file slowfast/datasets/decoder.py

I have tried to build torchvision from source with ffmpeg4.0 (you need to make sure that the torchvision detects the ffmpeg). Then the problem of read_video_meta_data_from_memory can be solved.

Have you tried following the installation steps in my fork: https://github.com/alpargun/SlowFast/blob/main/INSTALL.md

Your instructions worked, thanks a lot!

@alpargun your solution work for me, thank you so much!

Recompiling torchvision from source still fails. Luckily, alpargun’s solution works well for me. Thanks so much.

Took a crack at fixing it. https://github.com/facebookresearch/SlowFast/pull/541

In the official torchvision, “Building with FFMPEG is disabled by default in the latest main. If you want to use the ‘video_reader’ backend, please compile torchvision from source”

I met this error too. I solved this problem by make the following change in configs/xx.yaml:

#  DECODING_BACKEND: torchvision  #comment this line