fluent-bit: in_tcp is broken on Windows for 2.0.x (2.0.6)
Bug Report
Describe the bug The tcp input plugin is not processing json payloads correctly in 2.0.x. After the first payload json message it appears to hang and fluent-bit requires multiple ctrl-c to exit. This works as expected in the 1.x versions of fluent-bit.
To Reproduce fluent-bit executable is downloaded from https://fluentbit.io/releases/2.0/fluent-bit-2.0.6-win64.zip
Start fluent-bit with the tcp input and stdout plugin C:\fluent-bit-2.0.6-win64\bin>fluent-bit.exe -v -i tcp -o stdout
Now send a json payload to be processed by fluent-bit using chocolatey provided netcat using the command below
echo {"key 1": 123456789, "key 2": "abcdefg"} | C:\ProgramData\chocolatey\lib\netcat\tools\nc.exe -w 1 localhost 5170
Any succeeding payloads are not processed by the fluent-bit by e.g. by executing the command above a second time.
The output of the fluent-bit program is as follows. Note that the app does not exit gracefully after receiving the first SIGINT. One have to send additional signal to get the fluent-bit application to quit (using CTRL-C).
C:\fluent-bit-2.0.6-win64\bin>fluent-bit.exe -v -i tcp -o stdout
Fluent Bit v2.0.6
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io
[2023/01/10 15:56:57] [ info] Configuration:
[2023/01/10 15:56:57] [ info] flush time | 1.000000 seconds
[2023/01/10 15:56:57] [ info] grace | 5 seconds
[2023/01/10 15:56:57] [ info] daemon | 0
[2023/01/10 15:56:57] [ info] ___________
[2023/01/10 15:56:57] [ info] inputs:
[2023/01/10 15:56:57] [ info] tcp
[2023/01/10 15:56:57] [ info] ___________
[2023/01/10 15:56:57] [ info] filters:
[2023/01/10 15:56:57] [ info] ___________
[2023/01/10 15:56:57] [ info] outputs:
[2023/01/10 15:56:57] [ info] stdout.0
[2023/01/10 15:56:57] [ info] ___________
[2023/01/10 15:56:57] [ info] collectors:
[2023/01/10 15:56:57] [ info] [fluent bit] version=2.0.6, commit=211f841fe2, pid=4044
[2023/01/10 15:56:57] [debug] [engine] coroutine stack size: 98302 bytes (96.0K)
[2023/01/10 15:56:57] [ info] [storage] ver=1.3.0, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2023/01/10 15:56:57] [ info] [cmetrics] version=0.5.7
[2023/01/10 15:56:57] [ info] [ctraces ] version=0.2.5
[2023/01/10 15:56:57] [ info] [input:tcp:tcp.0] initializing
[2023/01/10 15:56:57] [ info] [input:tcp:tcp.0] storage_strategy='memory' (memory only)
[2023/01/10 15:56:57] [ info] [output:stdout:stdout.0] worker #0 started
[2023/01/10 15:56:57] [debug] [tcp:tcp.0] created event channels: read=636 write=640
[2023/01/10 15:56:57] [debug] [downstream] listening on 0.0.0.0:5170
[2023/01/10 15:56:57] [debug] [stdout:stdout.0] created event channels: read=648 write=652
[2023/01/10 15:56:57] [ info] [sp] stream processor started
[2023/01/10 15:57:10] [debug] [input chunk] update output instances with new chunk size diff=37
[2023/01/10 15:57:10] [debug] [task] created task=0000023344CB03D0 id=0 OK
[2023/01/10 15:57:10] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[0] tcp.0: [1673366230.367999500, {"key 1"=>123456789, "key 2"=>"abcdefg"}]
[2023/01/10 15:57:10] [debug] [out flush] cb_destroy coro_id=0
[2023/01/10 15:57:10] [debug] [task] destroy task=0000023344CB03D0 (task_id=0)
[2023/01/10 15:57:24] [engine] caught signal (SIGINT)
^C
Expected behavior Processing more than the first json payload.
Your Environment
- Version used:
C:\fluent-bit-2.0.6-win64\bin>fluent-bit.exe --version
Fluent Bit v2.0.6
Git commit: 211f841fe2190cf7decc6183bfe697873ec22ff6
- Operating System and version: Windows Server 2016 Datacenter version 1607 Can be reproduced on other windows version as well. Additional context Works flawless on fluent-bit 1.6.10. Has as well tested on fluent-bit 1.9.10 - See execution log below.
C:\fluent-bit-1.9.10-win64\bin>fluent-bit.exe -i tcp -o stdout
Fluent Bit v1.9.10
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io
[2023/01/10 15:54:46] [ info] [fluent bit] version=1.9.10, commit=760956f50c, pid=3032
[2023/01/10 15:54:46] [ info] [storage] version=1.3.0, type=memory-only, sync=normal, checksum=disabled, max_chunks_up=128
[2023/01/10 15:54:46] [ info] [cmetrics] version=0.3.7
[2023/01/10 15:54:46] [ info] [input:tcp:tcp.0] listening on 0.0.0.0:5170
[2023/01/10 15:54:46] [ info] [output:stdout:stdout.0] worker #0 started
[2023/01/10 15:54:46] [ info] [sp] stream processor started
[0] tcp.0: [1673366101.047399000, {"key 1"=>123456789, "key 2"=>"abcdefg"}]
[0] tcp.0: [1673366104.865521900, {"key 1"=>123456789, "key 2"=>"abcdefg"}]
[2023/01/10 15:55:10] [engine] caught signal (SIGINT)
[2023/01/10 15:55:10] [ warn] [engine] service will shutdown in max 5 seconds
[2023/01/10 15:55:11] [ info] [engine] service has stopped (0 pending tasks)
[2023/01/10 15:55:11] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2023/01/10 15:55:11] [ info] [output:stdout:stdout.0] thread worker #0 stopped
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 20 (10 by maintainers)
Commits related to this issue
- Upgrade fb-2.0.10 - See if windows tcp test works (https://github.com/fluent/fluent-bit/issues/6671 is resolved) — committed to newrelic/fluent-bit-package by luckslovez 8 months ago
- Upgrade fb-2.0.10 - See if windows tcp test works (https://github.com/fluent/fluent-bit/issues/6671 is resolved) — committed to newrelic/fluent-bit-package by luckslovez 8 months ago
- Upgrade fb-2.0.10 - See if windows tcp test works (https://github.com/fluent/fluent-bit/issues/6671 is resolved) — committed to newrelic/fluent-bit-package by luckslovez 8 months ago
Tested PR #7576 and confirmed the issue appears to be resolved with that change.