librealsense: Number of frames in .bag file does not correspond to expected number of frames
-
Before opening a new issue, we wanted to provide you with some useful suggestions (Click “Preview” above for a better view):
- Consider checking out SDK examples.
- Have you looked in our documentations?
- Is you question a frequently asked one?
- Try searching our GitHub Issues (open and closed) for a similar issue.
-
All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven’t followed any of the suggestions above 😃
| Required Info | |
|---|---|
| Camera Model | D435I |
| Firmware Version | 05.12.12.100 |
| Operating System & Version | Win 10 |
| Platform | PC |
| SDK Version | 2.44.0.3073 |
| Language | python |
| Segment | others |
Issue Description
After upgrading from pyrealsense2==2.35.0.1791 to pyrealsense2==2.44.0.3073, the number of frames in the recorded bag file does not correspond the the expected number of frames in the bag file.
Here is an example code to illustrate the issue:
import time
import pyrealsense2 as rs2
camera_framerate = 30
row = 480
col = 848
def enable_all_streams(config):
config.enable_stream(
stream_type=rs2.stream.depth,
width=col,
height=row,
format=rs2.format.z16,
framerate=camera_framerate,
)
config.enable_stream(
stream_type=rs2.stream.color,
width=col,
height=row,
format=rs2.format.bgr8,
framerate=camera_framerate,
)
#Make recording
duration = 5
bag_path = r"c:\temp\recording.bag"
config = rs2.config()
pipeline = rs2.pipeline()
ctx = rs2.context()
devices = ctx.query_devices()
serial_number = devices[0].get_info(rs2.camera_info.serial_number)
config.enable_device(serial_number)
enable_all_streams(config)
config.enable_record_to_file(bag_path)
pipeline.start(config)
#Wait for 5 seconds
time.sleep(duration)
pipeline.stop()
#Read bag file
config = rs2.config()
pipeline = rs2.pipeline()
config.enable_device_from_file(bag_path, repeat_playback=False)
enable_all_streams(config)
pipeline_profile = pipeline.start(config)
playback = pipeline_profile.get_device().as_playback()
playback.set_real_time(False)
success = True
frame_number = 0
while success:
success, frameSet = pipeline.try_wait_for_frames(1000)
frame_number += 1
print(f"Numbers of frames read from the bag file: {frame_number} | Expected number of frames: {camera_framerate*duration}")
When using pyrealsense2==2.35.0.1791 the result is:
Numbers of frames read from the bag file: 136 | Expected number of frames: 150
with pyrealsense2==2.44.0.3073 the result is:
Numbers of frames read from the bag file: 270 | Expected number of frames: 150
The number of frames read from the bag file is doubled when using the 2.44.0.3073 version.
Have you changed something in the SDK that I should be aware of, or is this a bug?
Regards, Paul
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 27
@Nir-Az Thanks very much for your help! The Intel bug report DSO-17075 associated with this case has now been closed.
I can confirm this is definitely not an isolated case. I spent days trying to figure out what went wrong in my C++ code when I finally narrowed it down to differences between SDK versions 2.45 and 2.36. I examined the problem in more detail by looking at individual frame numbers and timestamps directly from the stream of the bag file, and this is what I found. It’s not exactly duplicating framesets. However there are duplicate frames across different framesets. In other words, with the newer SDKs each frameset staggers color and depth alternatively.
For example the original sequence of frameset grabs during wait_for_frames() or poll_for_frames() is: (c1,d1), (c2,d2), (c3,d3), … (c_n, d_n) where, c and d stand for color and depth frames. They are best matched according to timestamps.
With the new SDK version this is the sequence I get: (c1,d1), (c1, d2), (c2,d2), (c2,d3), (c3,d3), (c3, d4)… etc.
As you can see duplicate frames across framesets are alternatively staggered almost like some bad interpolation instead of properly matching frames together. This leads to approximately doubling of the frames as reported by psandal. Hope this information helps. I will try to post actual stats from my bag file to illustrate this issue.
Edit: Posting actual numbers from my tests: First with the 2.36 SDK. Frames are optimally matched. There are rare duplicates but over all things add up. Each frameset grab results in unique color and depth frames as you can see from the relative timestamps. (relative timestamps are calculated by subtracting the first frame’s timestamp from the current timestamp and dividing by 1000 to convert into seconds).
Frameset Depth_Frame Color_Frame Time_difference Frame: 0 Time1:0.01666 Time2:0.009694 Difference:0.006966 Frame: 1 Time1:0.033321 Time2:0.026494 Difference:0.006827 Frame: 2 Time1:0.049982 Time2:0.043294 Difference:0.006688 Frame: 3 Time1:0.066642 Time2:0.060094 Difference:0.006548 Frame: 4 Time1:0.083303 Time2:0.076894 Difference:0.006409 Frame: 5 Time1:0.099964 Time2:0.093694 Difference:0.00627 Frame: 6 Time1:0.116624 Time2:0.110494 Difference:0.00613 Frame: 7 Time1:0.133285 Time2:0.127294 Difference:0.005991 Frame: 8 Time1:0.149946 Time2:0.144094 Difference:0.005852 Frame: 9 Time1:0.166606 Time2:0.160894 Difference:0.005712 Frame: 10 Time1:0.183267 Time2:0.177694 Difference:0.005573
Now with the 2,45 SDK this is what I get:
Frameset Depth_Frame Color_Frame Time_difference Frame: 0 Time1:0.016661 Time2:0.009555 Difference:0.007106 Frame: 1 Time1:0.016661 Time2:0.026355 Difference:-0.009694 Frame: 2 Time1:0.016661 Time2:0.043155 Difference:-0.026494 Frame: 3 Time1:0.033321 Time2:0.043155 Difference:-0.009834 Frame: 4 Time1:0.033321 Time2:0.059955 Difference:-0.026634 Frame: 5 Time1:0.049982 Time2:0.059955 Difference:-0.009973 Frame: 6 Time1:0.049982 Time2:0.076755 Difference:-0.026773 Frame: 7 Time1:0.066643 Time2:0.076755 Difference:-0.010112 Frame: 8 Time1:0.066643 Time2:0.093555 Difference:-0.026912 Frame: 9 Time1:0.083303 Time2:0.093555 Difference:-0.010252 Frame: 10 Time1:0.083303 Time2:0.110355 Difference:-0.027052
Here you can see that going by frame timestamp as an ID, that each frameset is different but frames are usually staggered alternatively. Additionally, the color frame timestamps are not the same for some reason across the two different SDKs!
Hi @pasandal An official bug report about the duplicating frame-count when reading a bag file in pyrealsense2 on Windows 10 has been filed with Intel. This case should be left open whilst that bug report is active. Thanks again for reporting the issue! In the meantime, the best solution for you may be to continue using SDK 2.35.0 for your project.