NLog: Memory Leak with async target and usage of ${threadid}, GDC or MLDC in the layout.

NLog version: 4.5.10

Platform: .Net 4.5

Current NLog config

<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

  <targets async="true">
    <target name="logfile" xsi:type="File" fileName="log.txt">
      <layout xsi:type="CsvLayout">
        <withHeader>true</withHeader>
        <column name="Date" layout="${date:universalTime=True:format=o}" />
        <column name="Server" layout="${machinename}" />
        <column name="Logger" layout="${logger}" />
        <column name="Source" layout="${callsite}" />
        <column name="Level" layout="${level}" />
        <column name="Message" layout="${message}" />
        <column name="Exception" layout="${exception:format=ToString}" />
      </layout>
    </target>
    <target name="logfile_crash" xsi:type="File" fileName="log_crash.txt">
      <layout xsi:type="CsvLayout">
        <withHeader>true</withHeader>
        <column name="Date" layout="${date:universalTime=True:format=o}" />
        <column name="Server" layout="${machinename}" />
        <!--<column name="ThreadId" layout="${threadid}" />-->
        <column name="Application" layout="${mdlc:item=Application}" />
        <!--<column name="Version" layout="${gdc:item=Version}" />-->
        <column name="Logger" layout="${logger}" />
        <column name="Source" layout="${callsite}" />
        <column name="Level" layout="${level}" />
        <column name="Message" layout="${message}" />
        <column name="Exception" layout="${exception:format=ToString}" />
      </layout>
    </target>
  </targets>

  <rules>
    <logger name="PerfTest" minlevel="Info" writeTo="logfile" />
    <logger name="PerfTestCrash" minlevel="Info" writeTo="logfile_crash" />
  </rules>
</nlog>

Hi,

We encounter a memory leak in our application in production, and the exploitation of memory dump led us to suspect NLog. After many tests, we realized that using an asynchronous target with a ${threadid}, GDC or MLDC in the layout causes the leak. The current thread hang, the memory increases and the application finally crash.

We can reproduce quickly by logging large messages quickly.

You can find a test project here with the demonstration : https://github.com/staufour/LoggerPerfTest

Thanks for your help.

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 26 (19 by maintainers)

Most upvoted comments

@staufour Made some more optimizations of the CsvLayout, that should reduce the memory allocations:

https://ci.appveyor.com/project/nlog/nlog/build/4.5.0-8455/artifacts

Please try NLog.4.5.10.8455-PR2934.nupkg

update of the Nlog version

So you have tested with NLog.4.5.10.8455-PR2934.nupkg ?

Memory escalation with both (~3Go) and OutOfMemory crash with 150Ko logs.

Can only suggest that you use the following AsyncWrapper-configuration for targets receiving very large messages (This will reduce performance in handling small messages, but will prevent out-out-memory):

<default-wrapper xsi:type="AsyncWrapper" timeToSleepBetweenBatches="0" overflowAction="Block" queueLimit="150" batchSize="10" fullBatchSizeWriteLimit="10" />

queueLimit and batchSize can be increased if normal max-message-size is less than 4 Mbyte (gives better performance). And it is a very good idea to use keepFileOpen="true"

@staufour NLog 4.6.1 has now been released that includes several performance improvements to CsvLayout: https://www.nuget.org/packages/NLog