amazon-ecs-agent: ECS Exec Command Exit Code Ignored by AWS CLI

Summary

ECS Exec does not seem to pass through the exit code of command back to AWS CLI.

Description

When attempting to use aws ecs execute-command to execute a command on an ECS container the commands return code doesn’t pass back to the shell where the AWS CLI command was executed.

Expected Behavior

> aws ecs execute-command --region us-east-1 --cluster fargate-esc-cluster --task <task_arn> --container ubuntu --command "ls /tmp/folderthatdoesntexist" --interactive


The Session Manager plugin was installed successfully. Use the AWS CLI to start a session.


Starting session with SessionId: ecs-execute-command-123
ls: cannot access '/tmp/folderthatdoesntexist': No such file or directory


Exiting session with sessionId: ecs-execute-command-123.

> echo $?
1

Observed Behavior

> aws ecs execute-command --region us-east-1 --cluster fargate-esc-cluster --task <task_arn> --container ubuntu --command "ls /tmp/folderthatdoesntexist" --interactive


The Session Manager plugin was installed successfully. Use the AWS CLI to start a session.


Starting session with SessionId: ecs-execute-command-123
ls: cannot access '/tmp/folderthatdoesntexist': No such file or directory


Exiting session with sessionId: ecs-execute-command-123.

> echo $?
0

validating container side error code

> aws ecs execute-command --region us-east-1 --cluster fargate-esc-cluster --task <task_arn> --container ubuntu --command "sh -c 'ls -l /tmp/folderthatdoesntexist || echo $?'" --interactive


The Session Manager plugin was installed successfully. Use the AWS CLI to start a session.


Starting session with SessionId: ecs-execute-command-123
ls: cannot access '/tmp/folderthatdoesntexist': No such file or directory
130

Exiting session with sessionId: ecs-execute-command-123.
> echo $?
0

Environment Details

Testing with ECS Fargate. ECS Exec works great we are just trying to utilize the functionality in scripts and noticed that errors weren’t stopping execution.

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 9
  • Comments: 15 (4 by maintainers)

Most upvoted comments

@jkribeiro hit the nail on the head. It’s not at all expected behaviour that because SSM connected successfully then the exit code should be 0. At least add a flag like --exit-with-command-status or something similar to bubble up the error.

I don’t think a hack/workaround is enough to close this issue. It seems to be an important feature/fix. Is it possible to know from the maintainers if this feature/fix is included in the ECS EXEC backlog?

Having command output is very common when using Docker exec. The same behaviour should be considered for ECS EXEC.

docker exec -it a sh -c 'exit 4'
echo $?
4

👍 - I would love a way to see the exit code using execute-command. I wonder, when --non-interactive is enabled (right now I get a runtime error if I try to use that flag), will that enable seeing the exit code?

Please re-open if you have more questions.

@fierlion can we reopen this? Doing && echo ok'" | grep ok is not a solution, it swallows all output and leaves you blind (but with an exit code). I prefer a flag like trent-abc suggested.

Yes, please reopen this.

I was expecting ECS execute-command to have similar functionality to kubectl exec, oc rsh etc. but it is severely lacking and difficult to use with any CLI tools due to the lack of non-interactive mode and return values.

This issue absolutely needs to be reopened. The proposed solution is very far from being considered acceptable 😞

In this case the 0 exit code is the expected behavior for a successful ecs execute-command call. The interactive exec session mostly follows the ssm start-session pattern https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ssm/start-session.html It seems that your use case follows the ssm run-command pattern (see https://docs.aws.amazon.com/systems-manager/latest/userguide/command-exit-codes.html) This run-command like functionality isn’t currently supported by ecs execute-command.

So for now you might need to get creative. For the ls example above, you can try:

aws ecs execute-command \
--region us-east-1 \
--cluster fargate-esc-cluster \
--task <task_arn> \
--container ubuntu 
--interactive 
--command "/bin/sh -c 'ls /tmp/folderthatdoesntexist && echo ok'" | grep ok

The overall ecs execute-command | grep call will return nothing for a non-existent directory, and return ok for an available directory. This is just one (hacky!) way to surface the output of the specific call from the interactive session to your current shell session.

ecs execute-command is still pretty new and we’re constantly working to make it better.