cloudquery: Cannot fetch AWS s3
After running ./cloudquery fetch
, All information is fetched from AWS. But only the information of the s3 backet cannot fetch.
I have already got the read-only permission. I run aws s3 ls
. It cannot show all the backet. So I know it is not a permission issue. May I know how can I solve this issue?
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 17 (9 by maintainers)
Commits related to this issue
- chore: Synced file(s) with cloudquery/.github (#157) Synced local file(s) with [cloudquery/.github](https://github.com/cloudquery/.github). --- This PR was created automatically by the [repo-fi... — committed to cloudquery/cloudquery by cq-bot 2 years ago
- chore: Synced file(s) with cloudquery/.github (#157) Synced local file(s) with [cloudquery/.github](https://github.com/cloudquery/.github). --- This PR was created automatically by the [repo-fi... — committed to cloudquery/cloudquery by cq-bot 2 years ago
- fix(deps): Update module github.com/hashicorp/go-hclog to v1.2.2 (#157) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [github.com/hashicorp/go-hclo... — committed to cloudquery/cloudquery by cq-bot 2 years ago
- chore(main): Release v0.5.2 (#155) :robot: I have created a release *beep* *boop* --- ## [0.5.2](https://github.com/cloudquery/cq-provider-terraform/compare/v0.5.1...v0.5.2) (2022-08-07) ### Bug ... — committed to cloudquery/cloudquery by cq-bot 2 years ago
I have using the AWS provider v0.5.3. It seems to fix the s3 issue. But I run
./cloudquery fetch --dsn "host=localhost user=xxxxx password=xxxxx DB.name=postgres port=5432" --verbose
. It keeps the loop in 81/89. I checked the log, it seems related to aws_s3_bucket_grants.Great I can see the problem now, will create a fix later today.
After I installed the v0.14.0, It still does not work in my environment. Seems the same issue.
I think I found the reason to cause this issue. If the access column is “Error”, then it will cause this issue.
After execute
./cloudquery fetch --dsn "host=localhost user=postgres password=pass DB.name=postgres port=5432" --verbos
, the log is following below.