VictoriaLogs: systemd-journal-upload fails to send data to VictoriaLogs

Is your question request related to a specific component?

VictoriaLogs

Describe the question in detail

I configured systemd-journal-upload like described here: https://docs.victoriametrics.com/victorialogs/data-ingestion/journald/

systemd-journal-upload fails with the following error:

Upload to http://localhost:9428/insert/journald/upload failed with code 400: remoteAddr: "127.0.0.1:56100"; requestURI: /insert/journald/upload; cannot read journald protocol data: too big data size exceeding -journald.maxRequestSize=67108864 bytes

So my journal is too big. Can’t this be a chunked upload? Is there really no way of uploading an existing journal without throwing away all the previous logs?

Troubleshooting docs

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 26 (15 by maintainers)

Commits related to this issue

Most upvoted comments

thanks for data, i was able to reproduce it

@AndrewChubatiuk I work with @Zinurist . Here are some example logs in json format:

example.logs.tar.gz

it’s not yet ready, will let you know once it’s ready

Indeed on a fresh installation I need a much larger journal to reproduce. This is a ~1GB journal gzipped. Try the same curl reproducer as above.

Tried uploading, but the gzipped file appears to be too large to attach. I uploaded it to our company S3 https://victoriametrics9070.fra1.s3.croit.io/logs.journal.tar.gz

let me know if you feel uncomfortable downloading from there and we can figure something else out.

Just to make sure, I am running these options. Everything else is pretty much default: -journald.streamFields=_MACHINE_ID,_SYSTEMD_UNIT,SYSLOG_IDENTIFIER,_TRANSPORT everything else is pretty much default.