walking-controllers: Closing the Walking Module may crash the yarprbotinterface

This happens quite seldom, but it happens.

I wonder what we can do at the application level that can ultimately lead the yarprobotinterface to crash.

cc @traversaro @GiulioRomualdi

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 15 (4 by maintainers)

Most upvoted comments

Not even those. Memory leak is when you call a new/malloc, and you never call delete/free. Double free is the exact opposite! 😄

Actually it seems that this pattern is quite widespared across icub-main, I will open a discussion in Community to discuss that.

See https://github.com/robotology/yarp/discussions/2533 .

I will open a PR to remove it.

Thanks, even if from the discussion in https://github.com/robotology/yarp/discussions/2533 it turns out that _port.getOutputCount() is inexpensive, I think that in such soft-real-time-oriented code we should prefer determinism over saving computational time based on external conditions.

Thanks for opening the issue on YARP, I wonder if this could be related to some of the old recurring crashing test (see https://github.com/robotology/yarp/issues/880). However, at the WBD level, I wonder what is the point of having _port.getOutputCount() in https://github.com/robotology/whole-body-estimators/blob/2ffe2d0c65a70b5435a4aa97882ab1857cb091a0/devices/wholeBodyDynamics/WholeBodyDynamicsDevice.cpp#L2497 at all.

This is a relic from the old wbd (see https://github.com/robotology/icub-main/blob/65366322b10adeaa52f94acc44411d35bb3b6988/src/modules/wholeBodyDynamics/observerThread.cpp#L1161) that I always copied in all the evolutions of WBD, in a rather cargo-cultish way. If getOutputCount is (as it appears from https://github.com/robotology/yarp/blob/9b52a71e176e23ebe42956cf5b23d8ae40b255f0/src/libYARP_os/src/yarp/os/impl/PortCore.cpp#L615) contacting all the connected ports to check if they are still alive, then it probably slows down the execution much more than any benefit that you could get by skipping the _port.prepare()/_port.write() calls.

Actually it seems that this pattern is quite widespared across icub-main, I will open a discussion in Community to discuss that.

I tested the new version today on the robot and seems to be working fine. Closing this. If it happens again we will keep investigating.

Removing getOutputCount seems to remove all the memory leaks.

It just happened once in simulation that WBD crashed after closing the WalkingModule. I guess this can cause in turn yarprobotinterface to crash. Here the output of the terminal:

[INFO] run phase starting...
[DEBUG] yarprobotinterface running happily
[INFO] |yarp.os.impl.PortCoreOutputUnit| Sending output from /wholeBodyDynamics/left_foot_front/cartesianEndEffectorWrench:o to /walking-coordinator/left_foot_front/cartesianEndEffectorWrench:i using tcp
[INFO] |yarp.os.impl.PortCoreOutputUnit| Sending output from /wholeBodyDynamics/left_foot_rear/cartesianEndEffectorWrench:o to /walking-coordinator/left_foot_rear/cartesianEndEffectorWrench:i using tcp
[INFO] |yarp.os.impl.PortCoreOutputUnit| Sending output from /wholeBodyDynamics/right_foot_front/cartesianEndEffectorWrench:o to /walking-coordinator/right_foot_front/cartesianEndEffectorWrench:i using tcp
[INFO] |yarp.os.impl.PortCoreOutputUnit| Sending output from /wholeBodyDynamics/right_foot_rear/cartesianEndEffectorWrench:o to /walking-coordinator/right_foot_rear/cartesianEndEffectorWrench:i using tcp
[INFO] |yarp.os.impl.PortCoreOutputUnit| Removing output from /wholeBodyDynamics/left_foot_front/cartesianEndEffectorWrench:o to /walking-coordinator/left_foot_front/cartesianEndEffectorWrench:i
[INFO] |yarp.os.impl.PortCoreOutputUnit| Removing output from /wholeBodyDynamics/left_foot_rear/cartesianEndEffectorWrench:o to /walking-coordinator/left_foot_rear/cartesianEndEffectorWrench:i
[INFO] |yarp.os.impl.PortCoreOutputUnit| Removing output from /wholeBodyDynamics/right_foot_front/cartesianEndEffectorWrench:o to /walking-coordinator/right_foot_front/cartesianEndEffectorWrench:i
free(): double free detected in tcache 2
Aborted (core dumped)

cc @prashanthr05

I am trying to remember if I have ever faced such an issue before, but in vain. I have only faced with issues reading the sensors, but with the network load of another device (base estimation) trying to read parallely the sensor buffers. Sometimes, it happens so that one of the two devices can’t read from the buffers at all anymore.

But a crash seems new to me.

I will open a PR to remove it.