azure-sdk-for-python: Possible memory leak receiving messages in servicebus 0.50.3

Package Name: servicebus
Package Version: 0.50.3
Operating System: Linux Alpine/Debian, OSX
Python Version: 3.8.6

Describe the bug Apparently memory leaks from receiving messages.

To Reproduce Steps to reproduce the behavior:

create a queue client
create a receiver
receive (and ignore) a message
back to point 1

Expected behavior No relevant memory usage increase.

Code

Here a sample of the code I use to replicate. Notice I am not processing messages and I am running against an empty queue.

import os

from azure.servicebus import ServiceBusClient, ReceiveSettleMode


if __name__ == '__main__':

    service_bus_client = ServiceBusClient(service_namespace=os.environ['AZURE_BUS_NAMESPACE'],
                                          shared_access_key_name=os.environ['AZURE_BUS_KEY_NAME'],
                                          shared_access_key_value=os.environ['AZURE_BUS_KEY_VALUE'])
    queue_name = '-'.join([os.environ['AZURE_BUS_QUEUE'], 'low'])

    for _ in range (1000000):
        # ofc I could create the queue once, but this is closer to my actual code
        queue = service_bus_client.get_queue(queue_name=queue_name)
        with queue.get_receiver(mode=ReceiveSettleMode.PeekLock) as receiver:
            msg_batch = receiver.fetch_next(max_batch_size=1, timeout=1)
            # notice I do not really expect to receive any message, printing just to keep it busy....
            print(msg_batch)

Additional context I know that I am not running on the latest version, but I have the feeling the issue might be in the uamqp layer and therefore of interest for also the new versions as well.

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 22 (9 by maintainers)

Most upvoted comments

apologize for missing the thread here @alberto-delgado-maersk, and thanks @miroag for bringing it to my attention! I’ll investigate into the issue this week.

yunhaoling on Jul 13, 2021

Hi @yunhaoling, it seems there are still some memory issues in version 7.2.0 of the service bus python package.
I ran a similar version to the script below for around 6 hours, explicitly calling the GC every few iterations and the memory just kept growing.

        while passed_secs < time_limit_secs:
            # ofc I could create the queue once, but this is closer to my actual code
            with service_bus_client.get_queue_receiver(queue_name=queue_name) as receiver:
                msg_batch = receiver.receive_messages(max_message_count=1,
                                                    max_wait_time=3)
                # notice I do not really expect to receive any message, printing just to keep it busy....
                print(msg_batch)
                if cnt % 100 == 0:
                    print("Forcing GC to run")
                    gc.collect()

Below you can see also a plot of how the memory grows: single_service_bus

When instead I initialized the service bus client within the while loop above, we see that the memory increases at a much slower rate: restart_service_bus

Could you check if you see the same at your end?

alberto-delgado-maersk on Jun 4, 2021

@yunhaoling thanks for the clarification, we will migrate to 7 soon. Please keep me posted on the leak.

andrea-cassioli-maersk on Jan 21, 2021