vector: Several AWS integrations are broken in v0.46.0
A note for the community
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
Problem
Hello,
I updated vector to v0.46.0 to use it with ‘aws_kinesis_firehose’ and when I validated the service I got an error that said the following:
'2025-04-09T08:05:08.405300Z ERROR vector::topology::builder: msg="Healthcheck failed." error=DescribeDeliveryStream failed: dispatch failure component_kind="sink" component_type="aws_kinesis_firehose" component_id=firehose_stream
x Health check for "firehose_stream" failed: DescribeDeliveryStream failed: dispatch failure'
The moment I downgrade vector to v0.45.0 the validation returns a ‘OK’ health check.
Configuration
sinks:
firehose_stream:
auth: {access_key_id: REDACTED, secret_access_key: REDACTED }
batch: {max_bytes: 250000, max_events: 500, timeout_secs: 1}
buffer: {max_size: 268435488, type: disk, when_full: block}
encoding: {codec: json}
healthcheck: {enabled: true}
inputs: [journald_aws]
region: eu-west-3
stream_name: redacted
type: aws_kinesis_firehose
Version
vector 0.46.0 (x86_64-unknown-linux-gnu a7429e2 2025-04-08 16:05:37.911631718)
Debug Output
Example Data
No response
Additional Context
No response
References
No response
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 14
- Comments: 24 (13 by maintainers)
We’re running into this as well with the
aws_kinesis_streamscomponent (so it is not limited to theaws_kinesis_firehosecomponent); upgrade to 0.46.0 results in healthcheck failure:And failure to push to the Kinesis stream:
Downgrading to 0.45.0 resolves this for us as well.
Sink config looks like:
I briefly enabled TRACE logging; there’s only one sequence that seems relevant (specifically the
encountered orchestrator errormessage):There’s a line in the block that I’ve redacted (
RuntimeComponents) which has a JSON-like blob that’s nearly 1600 lines when unfurled/formatted; I’ll need to review that carefully before providing it, but wanted to get the above notes into the conversation while I do that.I’ve upgraded a few of our deployments to 0.46.1 and can confirm I no longer see these errors and logs are flowing as-expected through the
aws_kinesis_streamscomponent. Thanks for quick fix!Ignore my comment; i missed the part where you bumped the deps. Had been looking at just after you reverted the endpoint PR.
Thank you for the custom build, I got it working with kineses, pod identity (aws eks), and assuming a role
@pront Thanks for the info & link. I can confirm that the fix candidate is working for me:
Thanks for this info. I was able to install cargo+git and then run:
Unfortunately, my server had too little memory to compile the
syncrate, so it eventually crashed, and I will have to try again with a larger server.@pront I’ll try that sha now. I rebased my other PR (locally) on top of your branch and validated it worked yesterday, but will explicitly test your branch to minimize variables.
Config I’m testing with:
Works as expected.