VideoProcessingFramework: Wrong PTS values for some VP9 & HEVC sequences

Describe the bug PyNvDecoder provides wrong PTS values on some VP9 & HEVC sequences.

To Reproduce Steps to reproduce the behavior: 1.nv_dmx.DemuxSinglePacket(self.packet) 2. nv_dmx.LastPacketData(self.packet_data) 3. packet_data.pts field is broken

Expected behavior packet_data.pts to show correct frame PTS value

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

Windows, Linux
CUDA Version 11.2
Video Codec SDK Version 11.0
Python Version 3.7

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 28 (28 by maintainers)

Commits related to this issue

Seeking to stream begining in the demuxer ctor to address #195 — committed to NVIDIA/VideoProcessingFramework by rarzumanyan 3 years ago
Fixing samples: #195 — committed to NVIDIA/VideoProcessingFramework by rarzumanyan 3 years ago
Issue #195: adding builtin and standalone modes for decode sample — committed to NVIDIA/VideoProcessingFramework by rarzumanyan 3 years ago

Most upvoted comments

Hi @lferraz

Demuxer seek is different from decoder seek.

Demuxer can get any encoded frame at any time so if you seek for frame 7 you either get it or frame 0 (closest key frame in past). If you ask decoder for frame 7 it will jump to closest key frame (0) and will decode 7 more frames to give you frame 7 in display order.

Decoder seek may vary for different sequences. E. g. for hevc file I’ve asked decoder for frame 11. For some reason it was a “single shot” search but it the end of the day decoder has returned me 11th frame for display.

all the branches can be merged

Great news. Will do that in Monday. Have a great weekend!

rarzumanyan on Apr 30, 2021

@lferraz

Regarding the

In BUILTIN mode nvc.SeekMode.PREV_KEY_FRAME is not working fine. It is going to the EXACT_FRAME.

Decoder actually returns right frame.

I’ve seek for frame 11 in dec.InitMode.BUILTIN mode, then decoded it & saved to file. Also decoded whole file in dec.InitMode.BUILTIN mode and saved 11th frame with raw video player - frames are same.

Regarding the av_seek_frame() FFmpeg function which is used for seek - by default it seeks for key frames only (this is what they say in the doc) so AVSEEK_FLAG_ANY is set on exclusively in EXACT_FRAME mode. In PREV_KEY_FRAME mode, the AVSEEK_FLAG_BACKWARD flag is set to seek for closest previous key frame. I have no idea why does it search for the key frame sometimes and for exact frame in other cases but somehow it does the job. May be it watches the DBP dependencies between the frames?

rarzumanyan on Apr 30, 2021

@lferraz

You’ve got the exact frame that you’ve asked from demuxer. It’s you responsibility to check if it can be decoded. If you feed decoder with P or B frame first time or if inter-frame depencencies aren’t resolved there’s no guarantee the decoder will be able to reconstruct given frame. If you want to be sure, seek for closest key frame in the past.

rarzumanyan on Apr 30, 2021

Hi @lferraz Thanks for the detailed reply.

Regarding the

I got this error: Decoder can only seek to closest previous key frame

It’s not an error, it’s expected behavior. When decoder uses built-in demuxer it can’t just jump to arbitrary frame because inter-frame dependencies may not be resolved. In stand-alone mode you can seek with demuxer to any frames to your liking.

Regarding the other points - allow me some time to reproduce & test on my machine, I’ll update as I find solution.

rarzumanyan on Apr 30, 2021