cloudformation-cli-python-plugin: Cloudformation fails with `Internal Failure` and no CloudWatch Logs
For some reason, after successfully submitting to CFN registry via the CLI with the command: cfn submit
, and successfully testing with sam local invoke TestEntrypoint --event test.json
- which launches the required resources in my AWS Account, creating the resource via the Cloudformation fails with the error Internal Failure
.
I tried to find related CloudWatch logs, but nothing is available? Any ideas I might be missing anything?
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 18 (5 by maintainers)
I’d recommend removing the looping logic and instead conditionally return
or
depending on the response from the API. You can set the deployment id in the callback context and that will be included in the next invocation request after the initial creation. You can then add some branching logic to execute your logic in the while loop if that is set in the callback context or simply try to create if it is not (the first invocation).
Our framework will handle reinvoking your resource handler. No need to try to stabilize on your own.
We got this and it turned out that our schema was invalid in a way that the registry wasn’t checking for but that failed deeper down in the service. I believe for us, it was that one of the properties used as
primaryIdentifier
had a value that was a list. Unfortunately, only the service team can actually find out, as these internal failures swallow the logs, which is a terrible customer experience.Agree this isn’t great. FWIW,
InternalFailure
is really just a mask exception for something we haven’t handled specifically. If there are no CloudWatch logs then it’s unlikely your type is even being invoked. Schema errors are the most likely culprit, but I’d have to look at a stack ARN to know for sure. If you have a support setup file an issue through that channel first.I think this will work for you
Great! Suggest you cut an issue to the main https://github.com/aws-cloudformation/cloudformation-cli repo showing that regex is a problem. We’ll have to fix the validation.
For the new issue, you should be able to see logs now? This error occurs if your type does not return a primaryIdentifier value back to CloudFormation within 1 minute. The way to fix this is to return a
ProgressEvent.progress
with yourResourceModel
populated with it’s primaryIdentifier as soon as you have it, and then continue stabilization.@rjlohan, I removed the Regex and it seems to work now. However, I get the error:
Resource timed out waiting for creation of physical resource
.@rjlohan, thanks. Below is the schema:
What may I be missing?