vector: Strategy device_and_inode does not work properly
Hi
vector version 0.8.2
I have few logs files, for example:
$ ls
logfile_json.log
logfile_json.1
logfile_json.2
logfile_json.3
When logfile_json.log reaching 100MB it will be renamed to logfile_json.1 as well as other log files to next number. Also, my vector config:
[sources.in_logs]
type = "file"
include = ["/var/log/myapp/logfile_json*"]
exclude = ["*gz"]
glob_minimum_cooldown = 1000
ignore_older = 86400
max_read_bytes = 4096
[sources.in_logs.fingerprinting]
strategy = "device_and_inode"
When vector starts - it detects all files and starts to read each of them and saves offsets properly. I checked inode number after renaming - new file gets new inode, renamed file saves his inode. Also, after renaming files vector detects changes and logs new_path and old_path properly.
However, vector does not read logs from those files, that were detected by device_and_inode strategy.
debug log level did not clarify situation, because renaming detection shows even in info log level
After changing strategy to checksum it works well.
Ask me to provide something else, if you will need more information.
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 15 (8 by maintainers)
We’ve been busy adding first-class integration with k8s. There’s a new
kubernetes_logssource, and we made sure to address the issue you’ve encountered here in that source. More specifically, we’re using a new implementation of the fingerprinter that checksums just the first line. We’re going to ship this in the next release (0.11).@Hoverbear it’s linux - timberio/vector:0.8.2-debian