rclpy: Memory leak in client when using python wrapper to send request to service

I found this problem in latest Galactic release, it is simple to reproduce, write a simple service(C++) and a client (Python), memory leak will definitly happen. It will not happen when using C++ in client.

  1. Data struct:
int32 seq
uint64 time
byte[1048576] input_tensor
---
int32 seq
uint64 time
byte[1048576] output_tensor

  1. Client side code:
import sys

from my_struct.srv import InferService
import rclpy
from rclpy.node import Node

import time


class MinimalClientAsync(Node):

    def __init__(self):
        super().__init__('minimal_client_async')
        self.cli = self.create_client(InferService, 'infer')
        while not self.cli.wait_for_service(timeout_sec=1.0):
            self.get_logger().info('service not available, waiting again...')
        self.req = InferService.Request()

    def send_request(self):
        self.req.seq = 2000
        self.req.time = 200
        self.future = self.cli.call_async(self.req)


def main(args=None):
    rclpy.init(args=args)

    minimal_client = MinimalClientAsync()
    for _ in range(100000):
        minimal_client.send_request()

        while rclpy.ok():
            rclpy.spin_once(minimal_client)
            if minimal_client.future.done():
                try:
                    response = minimal_client.future.result()
                except Exception as e:
                    minimal_client.get_logger().info(
                        'Service call failed %r' % (e,))
                else:
                    minimal_client.get_logger().info(
                        'Result of inference: resp seq %d' % (response.seq))
                break
            #time.sleep(0.5)


    minimal_client.destroy_node()
    rclpy.shutdown()


if __name__ == '__main__':
    main()

You can write a simple service code, just receive the request and do nothing and send a response to client.

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 16 (8 by maintainers)

Commits related to this issue

Most upvoted comments

problem confirmed, in the process space there is a lot of heap memory area mapped. as long as client/service running, virtual/physical memory increases. i created the reproducible sample program, https://github.com/fujitatomoya/ros2_test_prover/tree/master/prover_rclpy.

under colcon envirnoment,

colcon build --symlink-install --packages-select prover_interfaces prover_rclpy
source install/local_setup.bash
ros2 run prover_rclpy rclpy_server_822
ros2 run prover_rclpy rclpy_client_822

CC: @Barry-Xu-2018 @iuhilnehc-ynos could you take a look if you have time? i guess this is memory leak, if i am not mistaken…

You don’t need to do reset() manually, just let the unique_ptr with its destroy_ros_message_function do the magic.

Refer to https://github.com/ros2/rclpy/blob/691e4fbfcb4bd4cc2a01182a7dced3105a78200b/rclpy/src/rclpy/action_client.cpp#L102-L121

@iuhilnehc-ynos After deleting these two release lines, problem disappeared.

In my understanding, release function will take the owership of buffer from rclcpp to caller, right? but python wrapper does not get the unique pointer, so this buffer will never have a chance to be freed. Correct me if I am wrong!

I’d like to share something about this issue.

__convert_to_py(void * raw_ros_message) doesn’t own the raw_ros_message.

service server

Service::service_take_request
	auto taken_request = create_from_py(pyrequest_type);   // allocate a buffer
	...
	result_tuple[0] = convert_to_py(taken_request.get(), pyrequest_type);

	taken_request.release();  // Delete this line because this function have the responsibility to deallocate the buffer
	...

convert_to_py (using PyBytes_FromStringAndSize, Py_BuildValue, etc) copy data from the raw message instead of owning it.

same for service client