copilot-cli: DescribeCertificate did not contain DomainValidationOptions after 10 tries

Hi there!

I tried to deploy my front-end service today and got the following error:

Received response status [FAILED] from custom resource. Message returned: DescribeCertificate did not contain DomainValidationOptions after 10 tries. (Log: /aws/lambda/xxx-CertificateValidationFunction-b9AmhJj5xZyy/2021/07/13/[$LATEST]de2507a6f95044e5a03a4f5d864416c0) (RequestId: 73db11eb-4028-4d9b-8645-c1f85e714122)

You can find below my manifest.yml:

type: Load Balanced Web Service

http:
  path: '/'
  healthcheck:
    path: '/hc'
    success_codes: '200'
    healthy_threshold: 3
    unhealthy_threshold: 2
    interval: 15s
    timeout: 3s
  stickiness: false

image:
  build: Dockerfile
  port: 4200

cpu: 256 
memory: 512 
count: 1
exec: true

network:
  vpc:
    placement: 'public'

environments:
  dev:
    count: 1
    http:
      alias: dev.myapp.domain.com
  staging:
    count: 1
    cpu: 1024
    http:
      alias: staging.myapp.domain.com
  prod:
    cpu: 512
    memory: 1024
    range: 1-3
    # requests: 10000
    # response_time: 2s
    cpu_percentage: 70
    memory_percentage: 80
    http:
      alias: myapp.domain.com

I’ve checked a few things that may help you guys try and repro this bug:

SSL certs are created and issued successfully although they don’t get deleted after the stack gets rolled back
Cloudformation seems to get stuck creating EnvControllerAction
The lambda functions used to validate the cert seems to be created successfully too

Any help would be much appreciated as this issue is impacting our ability to go live with our application using AWS Copilot.

Thanks!

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 15 (9 by maintainers)

Commits related to this issue

fix(lambda): handle gracefully for same alias default domain (#2607)  fixes #2602. Currently Copilot fails if users set the `alias` to be the same as the default d... — committed to aws/copilot-cli by iamhopaul123 3 years ago
fix(lambda): handle gracefully for same alias default domain (#2607)  fixes #2602. Currently Copilot fails if users set the `alias` to be the same as the default d... — committed to localstack/copilot-cli-local by iamhopaul123 3 years ago

Most upvoted comments

Hi @efekarakus and @iamhopaul123

I can confirm the issue is now fixed. Although I agree with @iamhopaul123 this wasn’t expected behaviour, I think the confusion was I thought the default aliases would carry the service name with them in the format of ${SvcName}.${EnvName}.${AppName}.${DomainName} which I’m sure I’ve seen in the docs somewhere. My mistake but I’m glad you guys added that extra check on your side. Thanks a lot again!

afgallo on Jul 13, 2021

The fix is now released in v1.9.0! 🎊

Lou1415926 on Jul 29, 2021

@afgallo We are tentatively looking at 7/29!

efekarakus on Jul 20, 2021

Hi @afgallo 👋

Sorry for the churn! You’re right the recommendation does end up resulting to only front-end.dev.myapp.domain.com 😞.
The PR #2607 does fix this issue, we could set the alias dev.myapp.domain.com and it works.

Few possible alternatives while waiting on the new fix:

Use different alias values for dev and staging environments:

environments:
  dev:
     http:
       alias: 'beta.myapp.domain.com'
  stage:
    http:
       alias: 'gamma.myapp.domain.com' # or 'preprod.myapp.domain.com'

This is not ideal but when we release v1.9.0 you can change it back to dev.myapp.domain.com.

You can build a local version of copilot by running make from the commit that has the fix: https://github.com/aws/copilot-cli/tree/d0d42cada0bdd3596b279a7c2be4993152c4f67f

efekarakus on Jul 14, 2021

Hi @afgallo. Glad the workaround worked for this issue! Totally understandable for setting the default aliases. It is an edge case that we should definitely consider (either support it or error out early). Thank you so much for reporting this issue!

iamhopaul123 on Jul 13, 2021

Gotcha thank you! It does look like it’s the certificate validation lambda that’s failing.
Which region is this happening in? I wonder if there is a transient issue with ACM.

Would you mind also using the AWS CLI: https://docs.aws.amazon.com/cli/latest/reference/acm/describe-certificate.html aws acm describe-certificate --certificate-arn <ssl cert arn> during the deployment to see the output (for DomainValidationOptions) and pasting it here for us 🙏

Apologies for the inconvenience 🙇

efekarakus on Jul 13, 2021