imageio-ffmpeg: Race condition: We had to kill ffmpeg to stop it.

I’m trying to encode several mp4s at a time, and all of the larger ones give me the message We had to kill ffmpeg to stop it. Most of the files for which this message is printed are corrupted. Looking at the code, there appears to be a race condition in there where it only gives ffmpeg 2.5s to clean up. Is this needed? Or is there a way to disable that?

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 39 (17 by maintainers)

Most upvoted comments

A new release is now available on Pypi.

almarklein on Apr 12, 2019

agreed.

I re-posted because someone requested a workaround, which typically aren’t long-term solutions. This can be added without waiting for a PR to get approved, merged, released and posted.

TinyTheBrontosaurus on Apr 10, 2019

Nevermind, I got the test to pass (see #14) by forcing the dtype to uint8.

djhoese on Apr 10, 2019

Ok I can add this and make a PR. Would timeout be a little too generic? Maybe ffmpeg_timeout?

djhoese on Apr 10, 2019

Simple modification to the included imageio-ffmpeg tests to reproduce:

import numpy as np

...

def test_write_big_frames():

    sizes = []
    for pixfmt, bpp in [("gray", 1), ("rgb24", 3), ("rgba", 4)]:
        # Prepare for writing
        gen = imageio_ffmpeg.write_frames(
            test_file2, (2048, 2048), pix_fmt_in=pixfmt)
        gen.send(None)  # seed
        for i in range(9):
            data = bytes((255 * np.random.rand(2048 * 2048 * bpp)).astype(int))
            gen.send(data)
        gen.close()
        with open(test_file2, "rb") as f:
            sizes.append(len(f.read()))
        # Check nframes
        nframes, nsecs = imageio_ffmpeg.count_frames_and_secs(test_file2)
        assert nframes == 9

NB: Only included numpy because generating the required random bytes with random in a list comprehension was much too slow!

This also highlights that the issue depends on having non-uniform data in the frames, which presumably causes ffmpeg to spend longer in the encoding…

jmetz on Apr 5, 2019

I’ve just run into this issue in a pretty lightweight scenario (low frame count - 35, but large input frame size of 2048x2048).

I upped the timeout to 10s, during which ffmpeg closed normally.

This is on the following machine

Linux 4.9.0-8-amd64 #1 SMP Debian 4.9.144-3 (2019-02-02) x86_64 GNU/Linux

It is not a particularly low-spec machine (8 cores, 32GB RAM).

jmetz on Apr 2, 2019