terraform-provider-aws: S3 settings on aws_dms_endpoint conflict with "extra_connection_attributes"
Community Note
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
- Please do not leave “+1” or “me too” comments, they generate extra noise for issue followers and do not help prioritize the request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
Terraform Version
v0.11.13
Affected Resource(s)
- aws_dms_endpoint.s3_settings
Terraform Configuration Files
resource "aws_dms_endpoint" "s3_raw" {
endpoint_id = "s3-raw"
engine_name = "s3"
endpoint_type = "target"
extra_connection_attributes = "dataFormat=parquet;"
s3_settings {
service_access_role_arn = "${aws_iam_role.role.arn}"
bucket_name = "${var.s3_bucket}"
bucket_folder = "${var.raw_data_path}/dms"
compression_type = "GZIP"
}
}
Expected Behavior
The “extra connection status” in the DMS endpoint should be:
bucketFolder=data/raw/dms;bucketName=MY_BUCKET_NAME;compressionType=GZIP;csvDelimiter=,;csvRowDelimiter=\n;dataFormat=parquet;
Actual Behavior
The “extra connection status” in the DMS endpoint is:
bucketFolder=data/raw/dms;bucketName=MY_BUCKET_NAME;compressionType=GZIP;csvDelimiter=,;csvRowDelimiter=\n;
Notice the lack of dataFormat=parquet
Steps to Reproduce
terraform apply
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 89
- Comments: 21 (5 by maintainers)
Commits related to this issue
- #8000 #8009 Extend DMS S3 settings to support updated API changes in aws-sdk-go. This is so that we can avoid using extra_connection_attributes. — committed to lyle-nel/terraform-provider-aws by deleted user 4 years ago
- #8000 #8009 Optional fields should not be set unless it has a default or is explicitly populated. — committed to lyle-nel/terraform-provider-aws by deleted user 4 years ago
@aeschright is this being actively worked on?
Hi all! đź‘‹ Just wanted to direct you to our public roadmap for this quarter (Nov-Jan) in which this item has been mentioned.
Due to the significant community interest in resolving this issue, we will be looking at merging existing contributions soon.
We appreciate all the contributions and feedback thus far.
anyone working on this? seem the fix has not passed QA for a while?
@rory-lamendola @mchudoba Please upvote the pull request if you want it prioritised. It is not a full solution, but it will serve most of the use-cases out there by simply extending the supported
s3_settingsattributes to cover attributes that would otherwise need to be defined inextra_connection_attributes.