cli: "gh codespace ssh" fails to connect during Codespace instance first boot lifetime
Describe the bug
After creating a Codespace instance, gh codespace ssh does not work until that
Codespace instance is stopped and restarted.
It is unclear whether connecting to the Codespace instance in VSCode contributes to or increases the frequency of this issue.
$ gh version
gh version 2.11.3 (2022-05-25)
https://github.com/cli/cli/releases/tag/v2.11.3
Steps to reproduce the behavior
- Create and start a Codespace instance, either:
- In the UI, and open that Codespace instance in VSCode, OR
- via
gh codespace create
- In an operating system terminal (not the VSCode terminal), run
gh codespace ssh -- -vvv - Select the Codespace instance to connect to
- Observe the error message:
kex_exchange_identification: banner line 0: unable to find user -i: no matching entries in passwd file - Ctrl+C to stop
gh codespace ssh -- -vvv - Run
gh codespace sshand select the same Codespace instance to connect to - Command hangs without connecting to the Codespace instance
- Ctrl+C to stop
gh codespace ssh - Stop the Codespace instance in the UI or by running
gh codespace stop - Restart the Codespace instance in the UI or via
gh codespace start - In an operating system terminal (not the VSCode terminal), run
gh codespace ssh -- -vvv - (Seemingly most of the time) Observe that the ssh connection was successful
Expected vs actual behavior
Expected behavior: gh codespace ssh successfully connects after initial
Codespace instance creation.
Actual behavior: gh codespace ssh frequently hangs (with error message if
-vvv is passed to ssh command) until the Codespace instance is stopped and
restarted. Restarting the Codespace instance once does not always avoid this
issue.
Logs
Example failure (with -vvv passed to ssh):
$ gh cs ssh -- -vvv
? Choose codespace: redacted/redacted: master
OpenSSH_8.6p1, LibreSSL 2.8.3
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no
files
debug1: /etc/ssh/ssh_config line 54: Applying options for *
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' ->
'/Users/smkent/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' ->
'/Users/smkent/.ssh/known_hosts2'
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug1: Connecting to localhost port 52794.
debug1: Connection established.
debug1: identity file /Users/smkent/.ssh/id_rsa type -1
debug1: identity file /Users/smkent/.ssh/id_rsa-cert type -1
debug1: identity file /Users/smkent/.ssh/id_dsa type -1
debug1: identity file /Users/smkent/.ssh/id_dsa-cert type -1
debug1: identity file /Users/smkent/.ssh/id_ecdsa type -1
debug1: identity file /Users/smkent/.ssh/id_ecdsa-cert type -1
debug1: identity file /Users/smkent/.ssh/id_ecdsa_sk type -1
debug1: identity file /Users/smkent/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /Users/smkent/.ssh/id_ed25519 type 3
debug1: identity file /Users/smkent/.ssh/id_ed25519-cert type -1
debug1: identity file /Users/smkent/.ssh/id_ed25519_sk type -1
debug1: identity file /Users/smkent/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /Users/smkent/.ssh/id_xmss type -1
debug1: identity file /Users/smkent/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.6
debug1: kex_exchange_identification: banner line 0: unable to find user -i: no matching entries in passwd file
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 1
- Comments: 17 (8 by maintainers)
@eljog yep, that seems to work, but this still seems like a bug:
Maybe updating the docs & error message should be a new ticket but this is likely impacting all users and should probably be higher than a p3.
This is a rather sudden breaking change –
gh codespace sshis ostensibly supported by the cli but just doesn’t work anymore by default. Seems to me that this should be higher priority, orgh codespace sshshould be deprecated (or at least print a helpful error message in this case!)I’m having the same issue - it looks like ssh install is failing in some form. After restarting the codespace I’m seeing the following:
Thank you for the feedback. We understand the inconvenience and have prioritized the work items for updating the error message and documentation. The fix will be made available soon.
SSH install was hitting conflicts/locks often on the initial connect, where async user commands such as postcreate-command or dotfile installation was running in parallel. Hence pre-installing SSH on the container brings in benefits such as:
As suggested above, adding
sshdfeature in thedevcontainer.jsonwould be the apt fix hereHi @eljog,
Could you describe what those issues and conflicts are?
We want to enable people SSH’ing into their Codespaces, and this was a feature that always worked for us and was promoted by GitHub itself as a viable way to work with Codespaces. If I understand correctly, we now have to add
"sshd": "latest"to our devcontainer.json config in order for this to work; however, there are certain issues and conflicts that it may cause? What are those issues?Thanks
With a recent change, SSH will not be auto-installed on codespaces anymore (this was done to avoid certain other issues and conflicts caused by SSH auto install). Users will now need to make sure SSH is installed on their codespace container. One easy way to do that is by adding
sshdfeature to yourdevcontainer.json(using following snippet)For the Github folks, it looks like there’s an
/workspaces/.codespaces/.persistedshare/installSSH.shscript that’s being run when we attempt to ssh into the instance via the CLI and it’s failing for some reason?Based on the
-e’s in the output, I’m guessing that it’s being run viashexplicitly which makes it ignore the shebang to run it underbash.echoundershdoesn’t know the-eflag.