Universal_Robots_ROS2_Driver: Pipeline producer overflowed!

It’s a little flaky. Before the fix that @urrsk proposed, sometimes I would see the issue … sometimes I would not. I did not see the issue at all in several hours after implementing the fix.

@kschwan I think it is related to what hardware you are running it on and how loaded it is. If you get one timestamp behind, you can not catch up with the current implementation of the ros2_control

I think this might be cause by the sleep in the ros2_control while loop. The read function in the client_library is by default blocking and waits on the robot’s hardware clock trigger sending a packet. Though, the main loop in ros2_control also tries to control the clock. So if the driver is too slow receiving the e.g. first packet, when it will never be fast enough to catch up due to the sleep. A way to verity that could be to set the update_rate in the e.g.ur_bringup/config/ur10e_update_rate.yaml to a very high value like 9999999 Hz. This will make the sleep be negligible. @kschwan, @GewoonMidas, @NebuDa or @christianlandgraf (or somebody else) verify this assumption?

I coould get it running as well with your hint 👍 (sorry for the previous message )

@urrsk your suggestion seemed to help, for me. No more “pipeline producer overflow” since I increased the update rate. Will keep you updated in case I see it again.

I think this might be cause by the sleep in the ros2_control while loop. The read function in the client_library is by default blocking and waits on the robot’s hardware clock trigger sending a packet. Though, the main loop in ros2_control also tries to control the clock. So if the driver is too slow receiving the e.g. first packet, when it will never be fast enough to catch up due to the sleep. A way to verity that could be to set the update_rate in the e.g.ur_bringup/config/ur10e_update_rate.yaml to a very high value like 9999999 Hz. This will make the sleep be negligible. @kschwan, @GewoonMidas, @NebuDa or @christianlandgraf (or somebody else) verify this assumption?

@urrsk : I have set the update Rate in the ros_ur_driver/install/ur_bringup/share/ur_bringup/config/ur_controllers.yaml very high like you intended (Foxy Banch causes the different file) and the bug dissapear. No more problems with “pipeline producer overflowed”. I was directly connected over Ethernet to a real UR10e. But sometimes I have some trouble with connection lost. I will investigate it maybe next week, with access to the robot.

@urrsk Thanks for the help. I quickly tested it but I still get the same error except for ros2_control_node is dying now. I’ll have a deeper look into it tomorrow, whether anything else has changed since last time.

I have tried another setup with Ubuntu directly installed and I got the same error with URSimulator. I have tried different update rates in ur_controllers.yaml but no change. Ubuntu 20.04 ROS2 Foxy from Debians Moveit2 source build ROS2 UR Driver Branch Foxy source build

Any other ideas?

I can try with a real UR5 and UR5e also, but I can’t promise when exactly

@livanov93 I saw that someone updated the update rate to 530 Hz. This has solved the problem for me. Thanks for the help!