fluent-bit: [Windows] fluent-bit can not find log files grater then 2Gbytes
Bug Report
Describe the bug Using input plugin tail fluent-bit can not find log files grater then 2Gbytes
To Reproduce We have file sized 2147483648bytes = 2Gbytes
PS D:\td-agent-bit\logs> dir
Directory: D:\td-agent-bit\logs
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 28.05.2020 19:24 2147483648 events1.log
Starting fluent-bit
> .\fluent-bit.exe -c ..\conf\fluent-bit.conf
Fluent Bit v1.5.0
* Copyright (C) 2019-2020 The Fluent Bit Authors
* Copyright (C) 2015-2018 Treasure Data
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io
[2020/05/28 19:47:51] [ info] Configuration:
[2020/05/28 19:47:51] [ info] flush time | 5.000000 seconds
[2020/05/28 19:47:51] [ info] grace | 5 seconds
[2020/05/28 19:47:51] [ info] daemon | 0
[2020/05/28 19:47:51] [ info] ___________
[2020/05/28 19:47:51] [ info] inputs:
[2020/05/28 19:47:51] [ info] tail
[2020/05/28 19:47:51] [ info] ___________
[2020/05/28 19:47:51] [ info] filters:
[2020/05/28 19:47:51] [ info] ___________
[2020/05/28 19:47:51] [ info] outputs:
[2020/05/28 19:47:51] [ info] es.0
[2020/05/28 19:47:51] [ info] ___________
[2020/05/28 19:47:51] [ info] collectors:
[2020/05/28 19:47:51] [debug] [storage] [cio stream] new stream registered: tail.0
[2020/05/28 19:47:51] [ info] [storage] version=1.0.3, initializing...
[2020/05/28 19:47:51] [ info] [storage] in-memory
[2020/05/28 19:47:51] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
[2020/05/28 19:47:51] [ info] [engine] started (pid=7384)
[2020/05/28 19:47:51] [debug] [engine] coroutine stack size: 98302 bytes (96.0K)
[2020/05/28 19:47:51] [debug] [input:tail:tail.0] scanning path D:\td-agent-bit\logs\events1.log
[2020/05/28 19:47:51] [debug] [input:tail:tail.0] 0 files found for 'D:\td-agent-bit\logs\events1.log'
[2020/05/28 19:47:51] [trace] [input:tail:tail.0 at C:\projects\fluent-bit-2e87g\plugins\in_tail\tail.c:217] scan path: D:\td-agent-bit\logs\events1.log
[2020/05/28 19:47:51] [debug] [output:es:es.0] host=101.1.1.132 port=9200 uri=/_bulk index=firid-test-1 type=sid
[2020/05/28 19:47:51] [debug] [router] match rule tail.0:es.0
[2020/05/28 19:47:51] [ info] [sp] stream processor started
There is no files found.
If we reduce this file to 2147483647 bytes size output will be:
...
[2020/05/28 19:54:12] [debug] [input:tail:tail.0] 1 files found for 'D:\td-agent-bit\logs\events1.log'
[2020/05/28 19:54:12] [trace] [input:tail:tail.0 at C:\projects\fluent-bit-2e87g\plugins\in_tail\tail.c:217] scan path: D:\td-agent-bit\logs\events1.log
...
Expected behavior fluent-bit find and parse log files grater then 2Gbytes
Environment Windows Server 2016 Standart Tested on Fluent-bit 1.4.1 thue 1.5.0
[INPUT]
Name tail
Tag firid
Parser firid
Path D:\td-agent-bit\logs\events*.log
DB ..\db\firid.db
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 1
- Comments: 15 (10 by maintainers)
@AlekseyKalinin I have done adding 64-bit file support to
in_tail
plugin.This issue should be resolved once #2584 landed onto the main line. (I expect that the fix will be included in the next stable version: v1.6.0)