azure-sdk-for-net: [BUG] Invalid operation exception while sending batched messages to Service Bus Standard tier

Library name and version

Azure.Messaging.Servicebus 7.5.1

Describe the bug

I was sending batched messages to Azure Service Standard and encountered the following exception. Note that this exception is not reproducible consistently. My observation is that when the load on the ASB Standard is close to getting it throttled, this happens.

System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: Cannot open log for source 'Microsoft.ServiceBus'. You may not have write access.
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Azure.Messaging.ServiceBus.Amqp.AmqpSender.<SendBatchInternalAsync>d__19.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.ServiceBus.Amqp.AmqpSender.<>c.<<SendBatchAsync>b__18_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.ServiceBus.ServiceBusRetryPolicy.<>c__20`1.<<RunOperation>b__20_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at Azure.Messaging.ServiceBus.ServiceBusRetryPolicy.<RunOperation>d__21`2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Azure.Messaging.ServiceBus.ServiceBusRetryPolicy.<RunOperation>d__21`2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.ServiceBus.ServiceBusRetryPolicy.<RunOperation>d__20`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.ServiceBus.Amqp.AmqpSender.<SendBatchAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.ServiceBus.ServiceBusSender.<SendMessagesAsync>d__30.MoveNext()
   --- End of inner exception stack trace ---
---> (Inner Exception #0) System.InvalidOperationException: Cannot open log for source 'Microsoft.ServiceBus'. You may not have write access.
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Azure.Messaging.ServiceBus.Amqp.AmqpSender.<SendBatchInternalAsync>d__19.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.ServiceBus.Amqp.AmqpSender.<>c.<<SendBatchAsync>b__18_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.ServiceBus.ServiceBusRetryPolicy.<>c__20`1.<<RunOperation>b__20_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at Azure.Messaging.ServiceBus.ServiceBusRetryPolicy.<RunOperation>d__21`2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Azure.Messaging.ServiceBus.ServiceBusRetryPolicy.<RunOperation>d__21`2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.ServiceBus.ServiceBusRetryPolicy.<RunOperation>d__20`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.ServiceBus.Amqp.AmqpSender.<SendBatchAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.ServiceBus.ServiceBusSender.<SendMessagesAsync>d__30.MoveNext()<---

Expected behavior

There should not be any exceptions.

Actual behavior

Exception occured

System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: Cannot open log for source 'Microsoft.ServiceBus'. You may not have write access.
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Azure.Messaging.ServiceBus.Amqp.AmqpSender.<SendBatchInternalAsync>d__19.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.ServiceBus.Amqp.AmqpSender.<>c.<<SendBatchAsync>b__18_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.ServiceBus.ServiceBusRetryPolicy.<>c__20`1.<<RunOperation>b__20_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at Azure.Messaging.ServiceBus.ServiceBusRetryPolicy.<RunOperation>d__21`2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Azure.Messaging.ServiceBus.ServiceBusRetryPolicy.<RunOperation>d__21`2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.ServiceBus.ServiceBusRetryPolicy.<RunOperation>d__20`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.ServiceBus.Amqp.AmqpSender.<SendBatchAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.ServiceBus.ServiceBusSender.<SendMessagesAsync>d__30.MoveNext()
   --- End of inner exception stack trace ---
---> (Inner Exception #0) System.InvalidOperationException: Cannot open log for source 'Microsoft.ServiceBus'. You may not have write access.
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Azure.Messaging.ServiceBus.Amqp.AmqpSender.<SendBatchInternalAsync>d__19.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.ServiceBus.Amqp.AmqpSender.<>c.<<SendBatchAsync>b__18_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.ServiceBus.ServiceBusRetryPolicy.<>c__20`1.<<RunOperation>b__20_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at Azure.Messaging.ServiceBus.ServiceBusRetryPolicy.<RunOperation>d__21`2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Azure.Messaging.ServiceBus.ServiceBusRetryPolicy.<RunOperation>d__21`2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.ServiceBus.ServiceBusRetryPolicy.<RunOperation>d__20`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.ServiceBus.Amqp.AmqpSender.<SendBatchAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.ServiceBus.ServiceBusSender.<SendMessagesAsync>d__30.MoveNext()<---

Reproduction Steps

Used .net SDK to continuously send batched messages to ASB standard

Environment

.net 4.8

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 15 (6 by maintainers)

Most upvoted comments

Hi @arunprakashn. We’ve discussed internally and agree that this appears to be an error triggered by the service. It is currently under investigation.

+1
jsquire on Mar 8, 2022

@arunprakashn: Apologies for the confusion. I do not believe this is a local exception in your application or in the SDK. I’ll follow-up with @shankarsama offline for discussion.

+1
jsquire on Mar 4, 2022

Thank you @jsquire. Appreciate your work.

+1
srivathsanvlb on Feb 18, 2022

@arunprakashn: Thank you for your feedback. Tagging and routing to the team best able to assist.

+1
jsquire on Feb 17, 2022
Read more comments on GitHub
← azure-sdk-for-net: [BUG] Internal Error with AsPages Method not found Exception
azure-sdk-for-net: [BUG] KustoDatabaseCollection CreateOrUpdateAsync returning ResourceNotFound - false failure →