fluent-bit: Unable to get k8s metadata from systemd input
I am attempting to get the Kubernetes filter metadata from journald logs and output to Elasticsearch. (From the thread https://github.com/fluent/fluent-bit/issues/332 I assume this is possible.)
We have an existing k8s setup with a number of pods logging to journald.
I created a new pod for fluent-bit:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: fluent-bit
spec:
replicas: 1
template:
metadata:
labels:
name: fluent-bit
spec:
containers:
- name: fluent-bit
image: fluent/fluent-bit:0.12.14
volumeMounts:
- name: varlog
mountPath: /var/log
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
- name: fluent-bit-config
mountPath: /fluent-bit/etc/
- name: fluent-bit-log
mountPath: /fluent-bit/log/
terminationGracePeriodSeconds: 10
volumes:
- name: varlog
hostPath:
path: /var/log
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
- name: fluent-bit-config
hostPath:
path: /var/Davra/fluent-bit
- name: fluent-bit-log
hostPath:
path: /data
This uses the following fluent-bit config:
# cat /var/Davra/fluent-bit/fluent-bit.conf
[SERVICE]
Flush 1
Daemon Off
Log_Level info
Log_File /fluent-bit/log/fluent-bit.log
Parsers_File parsers.conf
[INPUT]
Name systemd
Path /var/log/journal
Tag k8s.*
[FILTER]
Name kubernetes
Match k8s.*
[OUTPUT]
Name es
Match *
Host 192.168.10.203
Port 9200
Logstash_Format On
Retry_Limit False
The resultant docs in Elasticsearch show systemd metadata but no kubernetes metadata. If I flip my log-driver’s to use json-file and tail /var/log/containers/*.log on the input instead, I do see the kubernetes metadata.
So, should I be able to get kubernetes metadata from journald input and, if so, any ideas of what may be wrong here?
Thanks.
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 21 (11 by maintainers)
Commits related to this issue
- filter_kubernetes: fix metadata lookup when source is Systemd (#518) Signed-off-by: Eduardo Silva <eduardo@treasure-data.com> — committed to fluent/fluent-bit by edsiper 6 years ago
My expectation is to release it by the third week of this month:
https://github.com/fluent/fluent-bit/milestone/5
@garysmith-github please check the new release (0.13-dev:0.10)
https://github.com/fluent/fluent-bit-kubernetes-logging/tree/0.13-dev
I’ve found the root cause of the problem. I will provide an update shorlty
@garysmith-github hmm yeah there is a problem, parsing CONTAINER_NAME we should get:
likely the hash key is not set correctly, I will troubleshoot.