fluent-bit: "PutObject request failed" on all S3 uploads on 1.8.11 with windows server 2019
Bug Report
Describe the bug “PutObject request failed” on all S3 uploads on 1.8.11 with windows server 2019
To Reproduce
- Rubular link if applicable:
- Example log message if applicable:
[2022/01/19 10:37:17] [debug] [aws_credentials] Initialized EC2 Provider in standard chain
[2022/01/19 10:37:17] [debug] [aws_credentials] Sync called on the EC2 provider
[2022/01/19 10:37:17] [debug] [aws_credentials] Init called on the env provider
[2022/01/19 10:37:17] [debug] [aws_credentials] Init called on the EC2 IMDS provider
[2022/01/19 10:37:17] [debug] [aws_credentials] requesting credentials from EC2 IMDS
[2022/01/19 10:37:17] [debug] [http_client] not using http_proxy for header
[2022/01/19 10:37:17] [debug] [http_client] server 169.254.169.254:80 will close connection #1084
[2022/01/19 10:37:17] [debug] [aws_client] (null): http_do=0, HTTP Status: 401
[2022/01/19 10:37:17] [debug] [http_client] not using http_proxy for header
[2022/01/19 10:37:17] [debug] [http_client] server 169.254.169.254:80 will close connection #1084
[2022/01/19 10:37:17] [debug] [imds] using IMDSv2
[2022/01/19 10:37:17] [debug] [http_client] not using http_proxy for header
[2022/01/19 10:37:17] [debug] [http_client] server 169.254.169.254:80 will close connection #1104
[2022/01/19 10:37:17] [debug] [aws_credentials] Requesting credentials for instance role Server-Role
[2022/01/19 10:37:17] [debug] [imds] using IMDSv2
[2022/01/19 10:37:17] [debug] [http_client] not using http_proxy for header
[2022/01/19 10:37:17] [debug] [http_client] server 169.254.169.254:80 will close connection #1084
[2022/01/19 10:37:17] [ info] [output:s3:s3.0] Sending locally buffered data from previous executions to S3; buffer=/tmp/fluent-bit/s3/log-bucket
[2022/01/19 10:37:17] [debug] [upstream] connection #1100 failed to s3.ap-northeast-1.amazonaws.com:443
- Steps to reproduce the problem:
Expected behavior
Screenshots
Your Environment
- Version used: 1.8.11
- Configuration:
- Environment name and version (e.g. Kubernetes? What version?):
- Server type and version: ec2
- Operating System and version: windows 2019
- Filters and plugins:
Additional context
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 15 (9 by maintainers)
@PettitWesley It doesn’t work even if you specify the vpc endpoint. Even if you set ec2 elasticip, the upload to s3 fails. Linux(amazon linux, centos, ubunt) works perfectly on the same vpc.
I think this is the source of the problem somehow… these two must be related.
I think you need to set the
endpointoption on the S3 output to be your VPC endpoint DNS.https://docs.fluentbit.io/manual/pipeline/outputs/s3