azure-functions-dotnet-worker: ILogger structured logging with Application Insights not working in .Net 5 Azure Function
.Net 5 Azure Function: Custom properties of ILogger generated messages are not showing up in Application Insights
Example:
logger.LogInformation("Example {ExampleValue}", 1001);
Expected in Application Insights log: customDimensions.prop__ExampleValue == 1001 customDimensions.prop__{OriginalFormat} == “Example {ExampleValue}”
However, the only thing that is logged is the message as string: message == “Example 1001”
This makes structured logging impossible and logging to Application Insights useless, as no proper filtering can be applied.
Other non-working examples:
using (_logger.BeginScope("ScopeTest"))
{
_logger.LogInformation("Scope logging test"); //Logs only the string, scope missing
}
Dictionary<string, string> properties = new Dictionary<string, string>()
{
{ "Prop1", "abc" },
{ "Prop2", "def" },
};
_logger.LogInformation("CustomProperties", properties); //Logs only the string, dictionary content missing completely
About this issue
- Original URL
- State: open
- Created 3 years ago
- Reactions: 16
- Comments: 35 (3 by maintainers)
Looks like this is still broken.
Probably should add this to the execution mode comparison document, so people (like me) don’t waste their time upgrading to .NET 7 and realize logging, custom metrics and dashboards that rely on structured logging get hopelessly broken.
if you like some pakour, see https://github.com/Azure/azure-functions-dotnet-worker/issues/1182 for some guidance on how to get some of it to work. Though probably not for production.
This has been a pretty big pain point for my team after migrating to dotnet-isolated so we could use .net 5. None of our expected customDimensions have been showing up so we have switched to including the dictionary values passing into the message instead so we can at least see what is going on. This breaks all of our aggregations in application insights for alerts and reporting.
Any suggestions or alternatives for working around this issue?
As a workaround, I thought of adding a custom logger factory to DI that would create an ILogger that doesn’t flow through the host GRPC channel and would connect directly to Insights. In theory it should work, but we would loose all console and stream logs in Azure and local dev and would be forced to rely on Insights for ANY diagnostics. If we try to keep the connection to GRPC, to also show in the stream logs, Insights would get duplicated entries with and without the structured information.
It should work, but I didn’t have the time to try and test it yet.
This NEEDS to be FIXED soon. It is a very serious bug. From the looks of it, I’m worried that it might only happen for .NET 7, but that would be a serious case of not giving enough importance to very important community feedback. This specific issue is already 7 months old, and it was not the first on this problem.
If the intention really is to have only isolated process in .NET 7, as the roadmap suggests, EVERYTHING that work in process will need to work in isolated, including structured logs.
Please make it available as a fix in 6, not only in 7!
It’s been about 2 months since my last post on this issue, and I don’t see any news about it being fixed or even being addressed/planned. I am about to put code in production with my workaround, even with the logs getting duplicated in order to be able to have good logs. ANY news on this would be really appreciated!
Ok, I was just able to test this and the structured logs did get recorded, but with some “missing” information and as you mentioned, my attempt at fixing the duplicated log to insights didn’t work.
Also, as far as I can tell, the minimum log level can only be set in code. If you call SetMinimumLevel(Trace) in your program.cs, it will log traces and debugs, but when you do that, it ignores the settings in configuration. That could be the reason we can’t properly configure this (the host process could be setting the configurations in code, wich overrides the settings in files and the portal)
The missing information are a few fields that are set in the logs that come from the host but not in the logs that come from the worker process: logLevel, ProcessId, HostInstanceId, cloud_RoleName, operation_Name, operation_id, operation_ParentId
There is probably a way to set this values in scopes or some other way. I didn’t investigate this yet.
The bad thing about switching to in-process would be:
The correct fix would be for the host process to properly support scopes flowing through the GRPC channel and also to respect the settings defined in the hosts files (log levels lower than Information and different log level by provider, for example). Another thing would be that the structured logs respected the key names we generate instead of attaching a “Prop_” prefix. If they do this to prevent name collision, they should add the prefix to their properties, not ours. We should have full control over how our logs are generated.
An alternative fix would be for the host to simply respect our settings. This way the settings above would’ve disabled the Function logs to Insights, allowing us to log the way we need without duplication. That could be the simpler way to fix this mess…
I have no idea how I missed this entire conversation; I apologize for the delay here.
The approach we’ve taken is to have logging go directly to App Insights from the worker (as was proposed here). This uses the App Insights logger with very minimal changes (no “prop_” appended, for example).
Give the newer package a try and see if it works how you’d expect: https://github.com/Azure/azure-functions-dotnet-worker/pull/944#issue-1282987627https://github.com/Azure/azure-functions-dotnet-worker/pull/944#issue-1282987627.
Is there a plan forward with this item?
@brettsam @fabiocav it doesn’t seem to be mentioned as much as the other App Insights issues, etc. and seems to have been lost in the ether 😕
Thanks for the workaround! This at least allows structured logging, although with duplicates. Anyhow, there are still too many things not working in regards of Application Insights. The dependency map shows the Function App isolated, calls from the Function App to other web-services or Azure Storage etc. are not visible at all… And no one seems to be working on these serious bugs. This is really frustrating. Overall the solution seem not to be enterprise ready.
@kelps No worries, totally understand, just thought I would ask 😄
Hope you have a Happy Christmas 🎄
I don’t have a public repo with this yet. Sorry. I’ll see if I can create a small repo on github with this setup when I get some time. As of this moment, my test code has other dependencies that I can’t put in a public repo.
Oh! @WestDiscGolf that is a good one. I forgot about the packages 😃 We have updated all the packages to the very latest versions. This is the one for adding appinsights: -Microsoft.Extensions.Logging.ApplicationInsights (2.19.0)
Let me know how it goes 😉
I’ve opened https://github.com/Azure/azure-functions-host/issues/7452 to support this in host. Once supported, we’ll work through this on the worker side.