azure-sdk-for-python: UAMQP Client fails to open underlying connections on azure functions

Context:

  • azure-servicebus v 7.8.1:
  • OS is from container mcr.microsoft.com/azure-functions/python:4-python3.9 (so debian variant)
  • Python 3.9

Describe the bug

Getting errors of the style

b'Invalid tlsio_state. Expected state is TLSIO_STATE_OPENING_UNDERLYING_IO.' 
(b'/project/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters
/tlsio_openssl.c':b'on_underlying_io_open_complete':849)

To Reproduce

This occurs randomly, but seems to happen mostly in the morning EST. I bother mentioning since we are running on azure functions and it may have something to do with the environment.

Expected behavior The connection is not supposed to hang for this long. We have retries setup to the default of 3, exponential backoff to .8 and maxi mum to two minutes. More importantly, I don’t expect that in a managed environment the resources are unavailable.

Screenshots

I attached a bigger log with all the errors. I’ve had to redact some stuff but it does not impact the global logs. azure_log.csv

Additional context We are running these in azure functions. We are using UAMQP over websockets. I doubt this would change anything considering the low-level nature of the errors, but I’m including it regardless.

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 18 (7 by maintainers)

Most upvoted comments

Hi! We have noticed and integrated. However, it seems that we now have issues with the socket connect timeout. I think this is starting to look like an azure service bus issue more than an API issue. We are monitoring to see reduced error rate but it seems like it is working for now. Thank you so much for your hard work!

sorry about the delay.

No worries, I was making sure the connection was still alive πŸ˜‰ Take the time needed πŸ˜ƒ

Thanks, got it @SamuelYvon. Ill update when I hear back so that we have a better idea of next steps.

@kashifkhan Done πŸ˜ƒ Please confirm reception πŸ˜ƒ