rmw_fastrtps: Regression in Rolling: error "cannot publish data, at ./src/rmw_publish.cpp:66" during shutdown of binary
Bug report
In Rolling, I observe weird segfaults during shutdown of the test binary:
2024-05-26T18:55:07.8227778Z 12: [moveit_task_constructor_core-test-move-relative-1] [==========] 3 tests from 1 test suite ran. (138 ms total)
2024-05-26T18:55:07.8228588Z 12: [moveit_task_constructor_core-test-move-relative-1] [ PASSED ] 3 tests.
2024-05-26T18:55:07.8229520Z 12: [moveit_task_constructor_core-test-move-relative-1] cannot publish data, at ./src/rmw_publish.cpp:66 during '__function__'
2024-05-26T18:55:07.8230659Z 12: [moveit_task_constructor_core-test-move-relative-1] Fail in delete datareader, at ./src/rmw_service.cpp:89 during '__function__'
2024-05-26T18:55:07.8233380Z 12: [ERROR] [moveit_task_constructor_core-test-move-relative-1]: process has died [pid 5393, exit code -11, cmd '/home/runner/work/moveit_task_constructor/moveit_task_constructor/.work/target_ws/build/moveit_task_constructor_core/test/moveit_task_constructor_core-test-move-relative --ros-args --params-file /tmp/launch_params_ujuovdp5 --params-file /tmp/launch_params_htlg3d0o --params-file /tmp/launch_params_4g9zqee6 --params-file /tmp/launch_params_niamvcg5'].
Humble and Iron don’t exhibit this issue.
WORKAROUND: Bypassing normal cleanup routines by calling quick_exit().
Required Info:
- Operating System: Ubuntu 24.04
- Installation type: deb package
- Version or commit hash: 8.5.0
- DDS implementation:fastrtps
- Client library (if applicable): rclcpp
Steps to reproduce issue
git clone -b https://github.com/ros-planning/moveit_task_constructor.git src/mtc
(cd src/mtc; git checkout bb047c8)
docker run --rm -it -v $PWD/src:/root/ws_moveit/src moveit/moveit2:rolling-source
in docker:
colcon build --mixin rel-with-deb-info
python3 -m launch_testing.launch_test /root/ws_moveit/src/mtc/core/test/test.launch.py test_binary:=/root/ws_moveit/build/moveit_task_constructor_core/test/moveit_task_constructor_core-test-move-relative
Expected behavior
Binary shuts down cleanly after tests.
Actual behavior
Binary throws error message and segfaults:
cannot publish data, at ./src/rmw_publish.cpp:66 during '__function__'
Fail in delete datareader, at ./src/rmw_service.cpp:89 during '__function__'
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 17 (8 by maintainers)
I have created a PR ros2/rclcpp#2556 to fix the issue in Executor.
For fastdds, when the program ends, fastdds resources start to release by Destructor. Note that the destruction of the static executor happens after this process. This results in an exception when the executor tries to access fastdds resources during its destruction.
As for why cyclonedds doesn’t have this issue, I believe it’s because cyclonedds doesn’t release its resources when the program exits (fastdds use Destructor). Therefore, when the static executor is being destructed, it can still access those resources. These resources are eventually reclaimed by the operating system.
So I think this code should be changed. https://github.com/moveit/moveit_task_constructor/blob/bb047c894cd2a395a672abdd47ec76d709d1cb1e/core/src/introspection.cpp#L160
I just wanted to quickly check an idea. Putting a shutdown in a test isn’t really appropriate.
Yes. This is a problem.
It is used to gracefully shut down ROS 2 nodes and associated resources (The correct order of release).|
Yes, there are other reasons too. I’ll continue to investigate this.