librealsense: IR frames occasionally corrupt until hardware reset

---

Required Info
Camera Model	{ D415, D435, D455 }
Firmware Version	5.12.9.0 and 5.12.12.100
Operating System & Version	Windows 10, Raspbian 10
Kernel Version (Linux Only)	5.4.83-v7l+
Platform	x86 laptop, Raspberry Pi 4
SDK Version	2.36 and 2.43 }
Language	C/C++ }
Segment	Others

Issue Description

In #5209, we were told to open a new issue if the problem happens again.

On D415, D435, and D455, we occasionally see the IR frames getting corrupted, which produces a depth frame with mostly invalid depth pixels. Please see the reference images for the normal, expected operation of the sensor:

and the following images showing the problem case:

Unfortunately, due to software error by me, only one IR image was saved on disk. Note that in the error case the RGB image turns black as well.

This problem has happened with Realsense SDK versions 2.36 and 2.43 on Raspberry Pi 4 and Windows 10 (x86) using passive 5m Newnex USB cables (part number: U3SA01C12-050 recommended by Intel) with firmware versions 5.12.9.0 and 5.12.12.100. Official Realsense cables are yet to be tested at length, but we do see at least some type of invalid depth frames with the devices installed at our customer locations using the official Realsense USB cable. As of yet we don’t know what the IR frames look like in those scenarios, but I’ll try to include some of those in this issue when I get a chance to debug one.

When the problem happens, there are two solutions for fixing it:

• Unplugging the cable
• issuing rs2_hardware_reset. Does not always work with Raspberry 4 using the RSUSB backend.

Restarting the program and re-initializing the librealsense2 context is not enough to fix the issue. We also plan to test using uhubctl on RPI4 to disable power delivery to the USB ports to see if that’s enough to reset the Realsense.

The problem has happened both at night when our office is empty and devoid of light, and at day when we have people working with the lights on. No strong IR sources exist in the office, excepting perhaps the metal door that reflects some IR light back towards the sensors (if closed).

We can reproduce this issue by restarting our test program approximately once every minute, and then wait until we receive 100 frames in row with average depth frame “blackness” percentage larger than 85%. In that case we save depth, IR, and RGB images on disk and consider the bug to have been reproduced and make a note in our debugging folder. Our next step is to see if this issue also happens with the sensor running for several days with no program rebooting.

In the previous ticket where we discussed black depth frames, the leading theories involved Realsense temperature going too high as well as external strong IR pulses hitting the IR sensor. Neither of those events happens in our office which is air conditioned and has no sunlight.