terraform-provider-aws: Since provider version 1.34.0 `terraform plan` is really slow

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave “+1” or “me too” comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

0.11.8

Problem

Not sure if it’s me or something under the hood. I was running 0.11.8/1.27.0 and upgraded provider to 1.35.0. Immediate after upgrade I ran terraform plan and it took very long time to finish. I blamed network at first, but after some time I ran it again with same result. I’ve downgraded plugin back to 1.27.0 and it is fast again … Not sure if it is isolated issue or something was introduced in one of the versions following 1.27.0 😕

I run it on the same subset of resources and state plan (remote S3). Also not sure if it’s network-slow or locally-slow

1.27.0 time terraform plan

real	0m43.978s
user	0m6.544s
sys	0m2.043s

1.35.0 time terraform plan

real	14m30.323s
user	0m8.626s
sys	0m3.522s

I’ll try to upgrade version-by-version and see when it starts happening for me.

About this issue

Original URL
State: closed
Created 6 years ago
Reactions: 18
Comments: 28 (12 by maintainers)

Most upvoted comments

The upstream Go issues mentioned above seem to indicate it was a Go 1.11 regression with DNS handling so it should be machine specific based on DNS configuration. The machine’s Go runtime shouldn’t matter unless you are manually building the provider and running that custom built provider.

It seems Go 1.11.1 is slated to include a fix, so we’ll keep this issue open to track the officially released version of the AWS provider that is built via that updated Go version.

bflad on Sep 26, 2018

Go 1.11.1 has been released, which should include this fix. I’ll switch our build processes to use the updated Go version and hopefully this will be part of the version 1.39.0 release.

bflad on Oct 1, 2018

I confirm version 1.39.0 has solved the issue on my side. Terraform is now working flawlessly, even with multiple DNS entries in /etc/resolv.conf.

supasteev0 on Oct 6, 2018

FYI, I am exactly in the same situation. Terraform 0.11.8 and provider.aws 1.35 is really slow. Any plan or apply action takes 10 times longer than before. Just downgraded to 1.33 as suggested by @rmldsky and execution time is now as fast as before.

supasteev0 on Sep 11, 2018

@nathanielks response fixed my probleme

jvolonda42 on May 20, 2019

I had this same problem. My issue was one of the entries in my DNS config wasn’t resolving. Emptying it out to be solely 1.1.1.1 has fixed the issue. terraform plan went from 3 minutes and 30 seconds to 20 seconds total! Thank you, folks!

nathanielks on Mar 27, 2019

The DNS issue starting in version 1.34.0 (built with Go 1.11.0) should now be resolved in version 1.39.0 (built with Go 1.11.1), released just now. 🎉

bflad on Oct 3, 2018

@rmldsky any update?

bflad on Sep 25, 2018

@bflad just FYI not to leave you hanging, I’ll get back on this issue, this week.

rmldsky on Sep 17, 2018

@rmldsky No, I’m on Linux Fedora 25.

supasteev0 on Sep 12, 2018

I’ve checked on a different machine, with fresh install of both terraform (0.11.8) and provider (1.35.0, I did test also 1.27.0 and 1.34.0) and to my surprise it works as expected. Only difference is an OS: macOS El Capitan (slow one) vs macOS High Sierra (where it works as expected).

I’ll try to check if there is anything I can do on the El Capitan machine to fix it. Any clues why it might be an issue on that platform?

rmldsky on Sep 11, 2018