@surgupta-msft and I have been looking at this and I wanted to provide an update on what we’re planning.

First, let me explain what’s happening. A long time back when we first added App Insights support to Functions, we decided that we did not want the logging filters set in host.json to apply to the Live Metrics stream. The idea was that you could turn off all logging for an application but still jump on Live Metrics in the portal and see how things were behaving. Because Live Metrics is free, we didn’t think it’d matter much. We also didn’t think of the perf implications. In retrospect, this was likely a confusing decision, and it has caused some issues throughout the years.

But to be clear – if you add a filter to host.json and see those logs in Live Metrics, it does not necessarily mean they are also being persisted in App Insights. You’d need to go to the actual Logs in App Insights to check there. If you are seeing the filters not working as expected in the persisted layer, please let us know. Everything we’ve tested with the filters does, indeed, filter there correctly.

With a recent release of the bindings, Functions and WebJobs updated their dependencies on the Azure SDK. This new dependency introduced a wave of new logs at the Information level, resulting in a flood of logs with the “Azure.Core” category. A lot of reports came in that these are unfilterable – however for everything we see they are filterable, just not for the Live Metrics view.

This behavior isn’t something we can change by default until a new major version of the Functions host. So instead, we’re going to:

1. Introduce a new property that you can set in host.json (something like EnableLiveMetricsFilters) which will then allow the filters to be applied at the Live Metrics level. This would make log filtering in the Live Metrics view match those that are persisted. We suspect there’ll also be a perf benefit here although we’ve not measured it. Then, adding “Azure.Core”: “Warning” to the log filters should clearly show in both Live Metrics and the persisted logs.
2. Update our new project templates to enable this setting (along with a "Azure.Core": "Warning" log filter). That way it’s clear that these exist and it’s easy enough to switch them. But there’s been so much confusion around these that we really want them front-and-center.

We hope this addresses the issue and provides some background. Please let us know if something here doesn’t seem to addres the issue that you’ve been seeing in this thread.

We experienced the same issue. In addition to logging to Application Insights our functions are configured with Serilog to send to different sources. By setting the following configuration we were able to filter these out

Log.Logger = new LoggerConfiguration()
.MinimumLevel.Information()
.MinimumLevel.Override("Azure", LogEventLevel.Warning)
.CreateLogger();

I can only concur with what has been said already. I’m looking that log stream right now and all I see is

2023-04-12T06:18:58Z   [Information]   Request [133d8023-09bd-4cc0-938d-277314594baa] PUT https://...blob.core.windows.net/azure-webjobs-hosts/locks/.../host?comp=lease
x-ms-lease-action:renew
x-ms-lease-id:00000000-0000-0000-0000-000026dc43da
x-ms-version:2021-08-06
Accept:application/xml
x-ms-client-request-id:133d8023-09bd-4cc0-938d-277314594baa
x-ms-return-client-request-id:true
User-Agent:azsdk-net-Storage.Blobs/12.13.0,(.NET 6.0.15; Linux 5.10.102.2-microsoft-standard #1 SMP Mon Mar 7 17:36:34 UTC 2022)
x-ms-date:Wed, 12 Apr 2023 06:18:57 GMT
Authorization:REDACTED
client assembly: Azure.Storage.Blobs
2023-04-12T06:18:58Z   [Information]   Response [133d8023-09bd-4cc0-938d-277314594baa] 200 OK (00.0s)
ETag:"0x8DB3B1C02F3DCEF"
Server:Windows-Azure-Blob/1.0,Microsoft-HTTPAPI/2.0
x-ms-request-id:165bc704-901e-0053-1906-6d4ac9000000
x-ms-client-request-id:133d8023-09bd-4cc0-938d-277314594baa
x-ms-version:2021-08-06
x-ms-lease-id:00000000-0000-0000-0000-000026dc43da
Date:Wed, 12 Apr 2023 06:18:57 GMT
Content-Length:0
Last-Modified:Wed, 12 Apr 2023 06:06:09 GMT

Over and over and over again. That is 30 lines of noise every second or two.

The problem here is that the Azure.Core package is logging all requests with log level information.

https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/core/Azure.Core/src/Diagnostics/AzureCoreEventSource.cs#L57-L61

Log filtering doesn’t apply to the log stream, so there’s no point in trying to filter these out. The only thing we can do is to change the log level to warning in the log stream view.

What team is responsible for the log stream? Can we petition them to add more filtering capabilities so that you can more easily get away from this problem. The problem is really that all the polling the Azure Function Host does ruins the log stream feature for everyone. It is more or less unusable even for development work (never mind a production setting). It’s just too spammy. Can’t the log stream just work with category filters to? Not that it should pickup the host.json config but that it should allow you to filter out unwanted output.

I don’t suppose you know how to configure this for the default logger?

I’ve got the IOC installed and setup, but I’m at a lost as to how to configure the default logger to do what you’re doing.

Does the above work with Azure Functions? I don’t use Serilog and hoped for a solution that would involve configuration changes in the Host.json

@d4046949 We are using it in Azure Functions. All our attempts at changing the host.json file failed, no matter what namespace we defined. Already had the Serilog implementation in our apps, so it wasn’t a big change for us. After posting the last comment, we also had to add some additional namespaces to get rid of the extra logging from Durable Functions.