aws-sdk-go: SerializationError: failed decoding EC2 Query response

Please fill out the sections below to help us address your issue.

Version of AWS SDK for Go?

1.10.33

Version of Go (`go version`)?

1.9.2

What issue did you see?

When calling describe instances for sa-east-1 we just started getting serialisation errors e.g.

describe instances for sa-east-1 call 1: SerializationError: failed decoding EC2 Query response 
caused by: flate: corrupt input before offset 5

Steps to reproduce

Perform a describe instances call to region sa-east-1

Its not clear at this point if this is an infrastructure issue with sa-east-1 but we’ve not seen this before.

About this issue

Original URL
State: closed
Created 7 years ago
Reactions: 1
Comments: 23 (9 by maintainers)

Most upvoted comments

Started receiving the same issue today. Using terraform:

" SerializationError: failed decoding Query response"

shows

+14

farrukh90 on Apr 4, 2019

Happening to me just now on aws_iam_role_policy; SerializationError: failed decoding Query response status code: 200, request id: <redacted> caused by: unexpected EOF

severity1 on Apr 5, 2019

^^ seeing the same error as above but for aws_iam_role_policy

gwong89 on Apr 5, 2019

we are experiencing the same issue in us-east-1. here is the piece of log that can help

DEBUG: Unmarshal Response iam/GetPolicyVersion failed, not retrying, error SerializationError: failed decoding Query response 2019-04-04T21:30:11.309-0500 [DEBUG] plugin.terraform-provider-aws_v2.4.0_x4: status code: 200, request id: 80a3a858-81e1-4730-99da-cc4224a88690 EBUG] plugin.terraform-provider-aws_v2.4.0_x4: caused by: unexpected EOF

tannirurahul on Apr 5, 2019

I have opened a new top level issue: #2549

bflad on Apr 5, 2019

We too are experiencing this in us-east-1 for IAM and here is part of the trace log:

2019-04-04T21:49:57.016-0500 [DEBUG] plugin.terraform-provider-aws_v2.4.0_x4: ---[ RESPONSE ]--------------------------------------
2019-04-04T21:49:57.016-0500 [DEBUG] plugin.terraform-provider-aws_v2.4.0_x4: HTTP/1.1 200 OK
2019-04-04T21:49:57.016-0500 [DEBUG] plugin.terraform-provider-aws_v2.4.0_x4: Connection: close
2019-04-04T21:49:57.016-0500 [DEBUG] plugin.terraform-provider-aws_v2.4.0_x4: Transfer-Encoding: chunked
2019-04-04T21:49:57.016-0500 [DEBUG] plugin.terraform-provider-aws_v2.4.0_x4: Content-Type: text/xml
2019-04-04T21:49:57.016-0500 [DEBUG] plugin.terraform-provider-aws_v2.4.0_x4: Date: Fri, 05 Apr 2019 02:49:56 GMT
2019-04-04T21:49:57.016-0500 [DEBUG] plugin.terraform-provider-aws_v2.4.0_x4: Vary: accept-encoding
2019-04-04T21:49:57.016-0500 [DEBUG] plugin.terraform-provider-aws_v2.4.0_x4: X-Amzn-Requestid: XXXXXXXXXXXXXXXXXXXX
2019-04-04T21:49:57.016-0500 [DEBUG] plugin.terraform-provider-aws_v2.4.0_x4: X-Amzn-Requestid: XXXXXXXXXXXXXXXXXXXX
2019-04-04T21:49:57.016-0500 [DEBUG] plugin.terraform-provider-aws_v2.4.0_x4: 
2019-04-04T21:49:57.016-0500 [DEBUG] plugin.terraform-provider-aws_v2.4.0_x4: 
2019-04-04T21:49:57.016-0500 [DEBUG] plugin.terraform-provider-aws_v2.4.0_x4: -----------------------------------------------------
2019-04-04T21:49:57.016-0500 [DEBUG] plugin.terraform-provider-aws_v2.4.0_x4: 2019/04/04 21:49:57 [DEBUG] [aws-sdk-go]
2019-04-04T21:49:57.016-0500 [DEBUG] plugin.terraform-provider-aws_v2.4.0_x4: 2019/04/04 21:49:57 [DEBUG] [aws-sdk-go] DEBUG: Unmarshal Response iam/GetRolePolicy failed, not retrying, error SerializationError: failed decoding Query response
2019-04-04T21:49:57.016-0500 [DEBUG] plugin.terraform-provider-aws_v2.4.0_x4: 	status code: 200, request id: XXXXXXXXXXXXXXXXXXXX
2019-04-04T21:49:57.016-0500 [DEBUG] plugin.terraform-provider-aws_v2.4.0_x4: caused by: unexpected EOF
2019/04/04 21:49:57 [ERROR] root: eval: *terraform.EvalRefresh, err: aws_iam_role_policy.XXXXXXXXXXXXXXXXXXXX: Error reading IAM policy XXXXXXXXXXXXXXXXXXXX from role XXXXXXXXXXXXXXXXXXXX: SerializationError: failed decoding Query response
	status code: 200, request id: XXXXXXXXXXXXXXXXXXXX
caused by: unexpected EOF
2019/04/04 21:49:57 [ERROR] root: eval: *terraform.EvalSequence, err: aws_iam_role_policy.XXXXXXXXXXXXXXXXXXXX: Error reading IAM policy XXXXXXXXXXXXXXXXXXXX from role XXXXXXXXXXXXXXXXXXXX: SerializationError: failed decoding Query response
	status code: 200, request id: XXXXXXXXXXXXXXXXXXXX
caused by: unexpected EOF
2019/04/04 21:49:57 [TRACE] [walkRefresh] Exiting eval tree: aws_iam_role_policy.XXXXXXXXXXXXXXXXXXXX

scsarver on Apr 5, 2019

Thanks for everyone’s patient. Looks like the issue has been solved 😃. I’ll go ahead and close this. If anyone else if having issues, please feel free to reopen.

xibz on Nov 5, 2017

@xibz Thank you. I can no longer see any failed requests on my end now, it was probably some transient issue in AWS infrastructure.

danielfm on Nov 4, 2017

@stevenh @danielfm @Luizm - I’ve went ahead and reached out to the service team letting them know that there is some issue with that region, specifically.

xibz on Nov 4, 2017