opencv: cv::VideoCapture::set does not work when using GStreamer

System information (version)

• OpenCV => 3.3.1
• Operating System / Platform => Windows 64 Bit
• Compiler => Visual Studio 2015

Detailed description

The end of CvCapture_GStreamer::open contains this code:

            status_ = gst_element_query_position(pipeline, FORMAT, &value_);
            if (!status_ || value_ != 0 || duration < 0)
            {
                CV_WARN(cv::format("Cannot query video position: status=%d value=%lld duration=%lld\n",
                        (int)status_, (long long int)value_, (long long int)duration).c_str());
                isPosFramesSupported = false;
                isPosFramesEmulated = true;
                emulatedFrameNumber = 0;
            }
            else
                isPosFramesSupported = true;

According to GStreamer reference manual, “this query will usually only work once the pipeline is prerolled (i.e. reached PAUSED or PLAYING state).”

So based on the documentation, OpenCV’s check, if querying the video position is possible, will usually fail, as it is done too early: immediately after opening the video, when the the pipeline is not yet prerolled. In my tests, it always failed, which makes isPosFramesEmulated = true. If later we want to set the video frame index via cv::VideoCapture::set(cv::CAP_PROP_POS_FRAMES, frameNumber) this call has no effect, because CvCapture_GStreamer::setProperty does not do anything if isPosFramesEmulated==true.

I tried the same using GStreamer directly. With the same videos, gst_element_query_position works without any problems as soon as the video is playing. Unfortunately, because of the flag isPosFramesEmulated OpenCV prevents to set the video frame index at any later time, though it would be perfectly possible.

Steps to reproduce

In tutorial project bg_sub, in function processVideo before code line ss << capture.get(CAP_PROP_POS_FRAMES); add a line capture.set(CAP_PROP_POS_FRAMES,1000); Set a breakpoint there and start debugging (passing “-vid testvideo.mpg” at command line). You will see that the subsequent get will return 1, which is not what we expect (it is CvCapture_GStreamer::emulatedFrameNumber). As explained above, this is caused by the OpenCV’s open method that makes isPosFramesEmulated = true. During open, this is written to stdout:

(cpp-tutorial-bg_sub.exe:13676): GStreamer-CRITICAL **: gst_query_set_position: assertion 'format == g_value_get_enum (gst_structure_id_get_value (s, GST_QUARK (FORMAT)))' failed
warning: Cannot query video position: status=1 value=-1 duration=18795
 (C:\opencv\modules\videoio\src\cap_gstreamer.cpp:949)

The first warning message is from GStreamer (GST_DEBUG=4), the second the subsequent message from OpenCV (see line 948 of cap_gstreamer.cpp).