argo-cd: Problem with Terminal and linux shell different from bash

Describe the bug

Tried the new most wanted feature, Terminal, it works with linux container with bash but got some problem with container Alpine based.

In screenshot, tried on busybox:stable container (it’s in loop to keep it running) got # after OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "bash": executable file not found in $PATH: unknown but cannot use it (cannot write)

Tried with some other container based on node:lts-alpine and got only black screen with blinking cursor.

Shell is working with same container using Rancher and Lens.

To Reproduce

Click on Terminal on Alpine based container

Expected behavior

Get shell from container and possibility to use it

Screenshots image

Version

2.4.0+91aefab

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 14
  • Comments: 29 (14 by maintainers)

Most upvoted comments

@crenshaw-dev turns out that the black screen it’s only on local k8s cluster where is located argocd and it’s because it was a lack of permission on k8s RBAC, in ClusterRole:argocd-server added:

- apiGroups:
  - ""
  resources:
  - pods/exec
  verbs:
  - create

After that I got the error OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "bash": executable file not found in $PATH: unknown same error in the screenshot in the first post, got # but not usable, so I tried what suggested by @arthurk.

I built argo with sh first and it works, but it opens only sh on all containers. It’s just a workaroud imho.

I found this: https://github.com/argoproj/argo-cd/blob/56eb09507e917de9c962a3249435d05f4dbf7e4b/server/application/terminal.go#L225

It’s exactly what happen!

I’ve upgraded to 2.4.6 and can confirm that the web terminal feature works well now

use nginx image instead, it works

@mvpwar thanks for your reply

I tried with nginx:1.19.1 and it works but with nginx:1.19.1-alpine not work, black screen with blinking cursor.

of course it could be a tmp workaround

I’m satisfied with a temporary workaround for now. This will solve a headache for 90% of use cases and will buy us time to figure out the 10%.

Making the list configurable is a win regardless, because people might want to add options that are not currently in the hardcoded list (e.g. ash).

It fails for our alpine based images too. It works when bash is installed via apk add bash

The error I get in argocd-server is:

{"level":"error","msg":"read message err: websocket: unexpected reserved bits 0x60","time":"2022-06-15T04:15:27Z"}

I think I actually found the real issue. Can someone test the PR to confirm? https://github.com/argoproj/argo-cd/pull/9895

I think I actually found the real issue. Can someone test the PR to confirm? #9895

I can confirm that your PR is solving the issue. Even that bash will fail, switch to sh is working and terminal is usable 🎉

Thanks!

image

I’m in favor of that change, and I agree that a configurable list makes sense. I think the config can go in ‘argocd-cmd-params-cm’.

It feels like this issue will be bit harder to figure out, as there are multiple libraries involved. My suggestion is to quick fix this issue, by temporary removing bash shell from list which would resolve this issue for most containers. When problem shell test is resolved we can put bash back. WDYT?

I don’t have any preference. Feel free to open a pr to change the order and ArgoCD maintainers can decide whether they are willing to accept this change or not. Again maybe a configmap for shells so that users can reorder the shells as they want would be more flexible.

It feels like this issue will be bit harder to figure out, as there are multiple libraries involved. My suggestion is to quick fix this issue, by temporary removing bash shell from list which would resolve this issue for most containers. When problem shell test is resolved we can put bash back. WDYT?

@crenshaw-dev I downloaded the code and added ash then builded the container and tried but didn’t work, same black screen with blinking cursor.

Hi @crenshaw-dev

After a while I got a 504 Gateway timeout as you can see on the screenshot

image

With other containers, not alpine, on same cluster all is working like a charm