ApplicationInsights-dotnet: High CPU utilization on Azure Web Apps caused by QuickPulseTelemetryModule
- Runtime version: netcoreapp3.1, Microsoft.ApplicationInsights.AspNetCore 2.15.0
- Hosting environment: Azure Web App
Describe the bug
Sometimes without any reason, we run in the high 100% CPU utilization on one of our Azure Web Apps that can only be fixed by restarting the application.
The CPU profiler snapshot is taken from the problem Azure Apps shows the 3-4K! stale threads and this hot path
Microsoft.ApplicationInsights.Extensibility.PerfCounterCollector.QuickPulse.QuickPulseTelemetryModule.CollectionThreadWorker
We use the Adaptive sampling rate with default configuration AI.
We really count on your help, and ready to provide any additional diagnostic data by request
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 36 (15 by maintainers)
For anyone else facing similar issue: The way we and @tatarincev was able to confirm that the issue is locally reproing is by collecting SDK logs (https://docs.microsoft.com/en-us/azure/azure-monitor/app/asp-net-troubleshoot-no-data#troubleshooting-logs) and check if any module is initialized more than once. We picked DependencyTrackingModule as an example and checked for the text “Initializing DependencyTrackingModule” in the SDK logs. Its expected to occur once, but was occurring multiple times. This confirm the issue is repro-ed locally as well. (It won’t likely lead to 100% cpu in local, as the volume is low.)
Yes. The support ticket will be looked at following regular process. You should be getting updates in the ticket directly. We’ll post any learnings/bugs/issues back here for everyone else to see.
Means either your application is still in 2.13 of the SDK, or have another application(s) using 2.13 sending telemetry to this ikey.
Yes, we have. Here is the ticket number 120112325003975 where the diag sessions were attached.
Previously we used the AI 2.12 version so changes could be introduced between 2.12 and 2.15 versions. We also noticed that in Azure our applications were starting to generate an absolutely enormous qty of telemetry data with type Metrics for 2.15 version after update with the relative the same user traffic.
Can we somehow control/reduce/filter the quantity for telemetry data with type Metric? Any suggestion is acceptable. We can do the changes in code or in AI settings on the Azure portal