fluent-bit: [Winlog] - Fluent bit crash unexpectedly when it reads some channels

Bug Report

Describe the bug Fluent bit crash unexpectedly without any message

To Reproduce No exact way to reproduce… I parse multiple winlog channels and it was working for some time. If I disable winlog input everything is ok.

Expected behavior Fluentbit shows an error or continue to work

Screenshots

Your Environment

  • Version used: td-fluent-bit 1.7.1
  • Configuration:
[INPUT]
    Name         winlog
    Channels      Application
    Interval_Sec 3
    DB           winlog1.sqlite
    Tag          winevent.log

[INPUT]
    Name         winlog
    Channels      Microsoft-Windows-TerminalServices-RemoteConnectionManager/Operational
    Interval_Sec 3
    DB           winlog2.sqlite
    Tag          winevent.log

# I also tried only one input with a list of channels : 
#  Channels     System,Application,Windows PowerShell,Microsoft-Windows-Kernel-Boot/Operational,Microsoft-Windows-SMBServer/Security,Microsoft-Windows-SMBClient/Connectivity,Microsoft-Windows-TerminalServices-RemoteConnectionManager/Operational
# but it's not better and I have dupplicate logs with different sources... Fluent bit seems shuffle everything.

[FILTER]
    Name aws
    # Fake matching rule to disable it localy...
    Match *
    imds_version v2
    az true
    ec2_instance_id true
    ec2_instance_type true
    private_ip true
    ami_id true
    account_id false
    hostname false
    vpc_id false

[FILTER]
    Name record_modifier
    Match *
    Record hostname ${HOSTNAME}

# Parse nested JSON in StringInserts for win eventlog
[FILTER]
    Name parser
    Match winevent.*
    Key_Name StringInserts
    Reserve_Data True
    Parser json-data

# Trick for otel collector which not supports uppercase letter in config
[FILTER]
    Name modify
    Match winevent.*
    Rename EventType eventtype

[OUTPUT]
    name  stdout
    match debug

[OUTPUT]
    # https://github.com/fluent/fluent-bit-docs/blob/master/pipeline/outputs/forward.md
    Name       forward
    Match      *
    Host         xxxxxxxxxxx
  • Environment name and version (e.g. Kubernetes? What version?):
  • Server type and version:
  • Operating System and version:
  • Filters and plugins:

Additional context

Fluent Bit v1.7.1
* Copyright (C) 2019-2021 The Fluent Bit Authors
* Copyright (C) 2015-2018 Treasure Data
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2021/03/11 14:48:07] [ info] Configuration:
[2021/03/11 14:48:07] [ info]  flush time     | 5.000000 seconds
[2021/03/11 14:48:07] [ info]  grace          | 5 seconds
[2021/03/11 14:48:07] [ info]  daemon         | 0
[2021/03/11 14:48:07] [ info] ___________
[2021/03/11 14:48:07] [ info]  inputs:
[2021/03/11 14:48:07] [ info]      winlog
[2021/03/11 14:48:07] [ info]      winlog
[2021/03/11 14:48:07] [ info] ___________
[2021/03/11 14:48:07] [ info]  filters:
[2021/03/11 14:48:07] [ info]      aws.0
[2021/03/11 14:48:07] [ info]      record_modifier.1
[2021/03/11 14:48:07] [ info]      parser.2
[2021/03/11 14:48:07] [ info]      modify.3
[2021/03/11 14:48:07] [ info] ___________
[2021/03/11 14:48:07] [ info]  outputs:
[2021/03/11 14:48:07] [ info]      stdout.0
[2021/03/11 14:48:07] [ info]      forward.1
[2021/03/11 14:48:07] [ info] ___________
[2021/03/11 14:48:07] [ info]  collectors:
[2021/03/11 14:48:07] [ info] [engine] started (pid=32892)
[2021/03/11 14:48:07] [debug] [engine] coroutine stack size: 98302 bytes (96.0K)
[2021/03/11 14:48:07] [debug] [storage] [cio stream] new stream registered: winlog.0
[2021/03/11 14:48:07] [debug] [storage] [cio stream] new stream registered: winlog.1
[2021/03/11 14:48:07] [ info] [storage] version=1.1.0, initializing...
[2021/03/11 14:48:07] [ info] [storage] in-memory
[2021/03/11 14:48:07] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
[2021/03/11 14:48:07] [debug] [input:winlog:winlog.0] load channel<Application record=192696 time=1615161894>
[2021/03/11 14:48:07] [debug] [input:winlog:winlog.1] load channel<Microsoft-Windows-TerminalServices-RemoteConnectionManager/Operational record=193716 time=1615195888>
[...]
[2021/03/11 14:48:10] [debug] [input:winlog:winlog.0] read 513836 bytes from 'Application'
PS C:\Program Files\td-agent-bit> echo $LASTEXITCODE
-1073741819

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 29 (9 by maintainers)

Most upvoted comments

@edsiper I can’t seem to find 1.7.5 Windows binary available for download anywhere. Official site still lists only 1.7.4.

@jeremyje “now” 😃