libssh2: ssh-agent libssh2_agent_userauth error, works on 1.10.0

Describe the bug Hi there,

We’re excited about the rsa-sha2-256/512 support in libssh2, however we encountered a bug with ssh-agent support reproduced with the examples.

Everything works fine when using keys from disk, however when trying to auth to a server using libssh2_agent_userauth we get LIBSSH2_ERROR_PUBLICKEY_UNVERIFIED. The same exact code works with 1.10.0 (when accessing a server that accepts ssh-rsa).

See below for output from a few different scenarios, along with a dockerfile and patch used for testing.

Potentially related: #536 and #626

To Reproduce ./example/example-ssh2 <server-ip> git: success ./example/example-ssh2_agent <server-ip> git: error

See test patch below used to provide output, but reproducible without any patches.

Expected behavior libssh2_agent_userauth should auth successfully.

Version (please complete the following information):

• OS: Fedora 35
• libssh2 version: master (13ad7b2f5cd67e0dc843098ce19ce8b208368c29)

Additional context 1.10.0 example-ssh2 PubkeyAcceptedKeyTypes=ssh-ed25519,rsa-sha2-256,ssh-rsa (Success expected):

Connecting to git@172.17.0.2. pass: password
[libssh2] 0.572899 Key Ex: Sent KEX: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1,diffie-hellman-group-exchange-sha1
[libssh2] 0.572909 Key Ex: Sent HOSTKEY: ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519,ssh-rsa,ssh-dss
[libssh2] 0.572913 Key Ex: Sent CRYPT_CS: aes128-ctr,aes192-ctr,aes256-ctr,aes256-cbc,rijndael-cbc@lysator.liu.se,aes192-cbc,aes128-cbc,blowfish-cbc,arcfour128,arcfour,cast128-cbc,3des-cbc
[libssh2] 0.572915 Key Ex: Sent CRYPT_SC: aes128-ctr,aes192-ctr,aes256-ctr,aes256-cbc,rijndael-cbc@lysator.liu.se,aes192-cbc,aes128-cbc,blowfish-cbc,arcfour128,arcfour,cast128-cbc,3des-cbc
[libssh2] 0.572916 Key Ex: Sent MAC_CS: hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-sha1-96,hmac-md5,hmac-md5-96,hmac-ripemd160,hmac-ripemd160@openssh.com
[libssh2] 0.572918 Key Ex: Sent MAC_SC: hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-sha1-96,hmac-md5,hmac-md5-96,hmac-ripemd160,hmac-ripemd160@openssh.com
[libssh2] 0.572919 Key Ex: Sent COMP_CS: none
[libssh2] 0.572921 Key Ex: Sent COMP_SC: none
[libssh2] 0.572922 Key Ex: Sent LANG_CS: 
[libssh2] 0.572924 Key Ex: Sent LANG_SC: 
[libssh2] 0.573698 Key Ex: Agreed on KEX method: curve25519-sha256
[libssh2] 0.573704 Key Ex: Agreed on HOSTKEY method: ecdsa-sha2-nistp256
[libssh2] 0.573705 Key Ex: Agreed on CRYPT_CS method: aes128-ctr
[libssh2] 0.573706 Key Ex: Agreed on CRYPT_SC method: aes128-ctr
[libssh2] 0.573707 Key Ex: Agreed on MAC_CS method: hmac-sha2-256
[libssh2] 0.573708 Key Ex: Agreed on MAC_SC method: hmac-sha2-256
[libssh2] 0.573708 Key Ex: Agreed on COMP_CS method: none
[libssh2] 0.573709 Key Ex: Agreed on COMP_SC method: none
[libssh2] 0.573778 Key Ex: Initiating curve25519 SHA2
[libssh2] 0.577639 Key Ex: Server's MD5 Fingerprint: af:61:b0:ad:fa:6e:3b:b6:c4:0b:4a:ee:34:85:9a:f1
[libssh2] 0.577651 Key Ex: Server's SHA1 Fingerprint: 3d:eb:f3:d1:63:0b:22:5e:e6:a8:46:f0:61:88:f7:c0:50:6a:11:5c
[libssh2] 0.577655 Key Ex: Server's SHA256 Fingerprint: fDDUXe6r2q+TCFIhzZdck5R2zj1TaMiM/JL7gLsUhoA=
[libssh2] 0.577898 Key Ex: Received NEWKEYS message
[libssh2] 0.577903 Key Ex: session_id calculated
[libssh2] 0.577909 Key Ex: Client to Server IV and Key calculated
[libssh2] 0.577913 Key Ex: Server to Client IV and Key calculated
[libssh2] 0.577915 Key Ex: Client to Server HMAC Key calculated
[libssh2] 0.577917 Key Ex: Server to Client HMAC Key calculated
[libssh2] 0.577918 Key Ex: Client to Server compression initialized
[libssh2] 0.577918 Key Ex: Server to Client compression initialized
Fingerprint: 3D EB F3 D1 63 0B 22 5E E6 A8 46 F0 61 88 F7 C0 50 6A 11 5C 
[libssh2] 0.585770 Userauth: Permitted auth methods: publickey
Authentication methods: publickey
[libssh2] 0.585784 Userauth: Loading public key file: /home/ian/.ssh/pubs/id_rsa_test.pub
[libssh2] 0.585830 Userauth: Attempting publickey authentication
[libssh2] 0.586355 Userauth: Loading private key file: /home/ian/.ssh/id_rsa_test
[libssh2] 0.586436 Userauth: Computing RSA keys from private key data
[libssh2] 0.588955 Userauth: Attempting publickey authentication -- phase 2
[libssh2] 0.601704 Userauth: Publickey authentication successful
        Authentication by public key succeeded.
all done!

1.10.0 example-ssh2 PubkeyAcceptedKeyTypes=ssh-ed25519,rsa-sha2-256 (Failure expected):

Connecting to git@172.17.0.2. pass: password
[libssh2] 0.507683 Key Ex: Sent KEX: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1,diffie-hellman-group-exchange-sha1
[libssh2] 0.507699 Key Ex: Sent HOSTKEY: ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519,ssh-rsa,ssh-dss
[libssh2] 0.507701 Key Ex: Sent CRYPT_CS: aes128-ctr,aes192-ctr,aes256-ctr,aes256-cbc,rijndael-cbc@lysator.liu.se,aes192-cbc,aes128-cbc,blowfish-cbc,arcfour128,arcfour,cast128-cbc,3des-cbc
[libssh2] 0.507703 Key Ex: Sent CRYPT_SC: aes128-ctr,aes192-ctr,aes256-ctr,aes256-cbc,rijndael-cbc@lysator.liu.se,aes192-cbc,aes128-cbc,blowfish-cbc,arcfour128,arcfour,cast128-cbc,3des-cbc
[libssh2] 0.507704 Key Ex: Sent MAC_CS: hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-sha1-96,hmac-md5,hmac-md5-96,hmac-ripemd160,hmac-ripemd160@openssh.com
[libssh2] 0.507705 Key Ex: Sent MAC_SC: hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-sha1-96,hmac-md5,hmac-md5-96,hmac-ripemd160,hmac-ripemd160@openssh.com
[libssh2] 0.507707 Key Ex: Sent COMP_CS: none
[libssh2] 0.507708 Key Ex: Sent COMP_SC: none
[libssh2] 0.507708 Key Ex: Sent LANG_CS: 
[libssh2] 0.507709 Key Ex: Sent LANG_SC: 
[libssh2] 0.508503 Key Ex: Agreed on KEX method: curve25519-sha256
[libssh2] 0.508510 Key Ex: Agreed on HOSTKEY method: ecdsa-sha2-nistp256
[libssh2] 0.508511 Key Ex: Agreed on CRYPT_CS method: aes128-ctr
[libssh2] 0.508512 Key Ex: Agreed on CRYPT_SC method: aes128-ctr
[libssh2] 0.508513 Key Ex: Agreed on MAC_CS method: hmac-sha2-256
[libssh2] 0.508514 Key Ex: Agreed on MAC_SC method: hmac-sha2-256
[libssh2] 0.508515 Key Ex: Agreed on COMP_CS method: none
[libssh2] 0.508516 Key Ex: Agreed on COMP_SC method: none
[libssh2] 0.508600 Key Ex: Initiating curve25519 SHA2
[libssh2] 0.512767 Key Ex: Server's MD5 Fingerprint: af:61:b0:ad:fa:6e:3b:b6:c4:0b:4a:ee:34:85:9a:f1
[libssh2] 0.512779 Key Ex: Server's SHA1 Fingerprint: 3d:eb:f3:d1:63:0b:22:5e:e6:a8:46:f0:61:88:f7:c0:50:6a:11:5c
[libssh2] 0.512783 Key Ex: Server's SHA256 Fingerprint: fDDUXe6r2q+TCFIhzZdck5R2zj1TaMiM/JL7gLsUhoA=
[libssh2] 0.513041 Key Ex: Received NEWKEYS message
[libssh2] 0.513046 Key Ex: session_id calculated
[libssh2] 0.513055 Key Ex: Client to Server IV and Key calculated
[libssh2] 0.513059 Key Ex: Server to Client IV and Key calculated
[libssh2] 0.513063 Key Ex: Client to Server HMAC Key calculated
[libssh2] 0.513065 Key Ex: Server to Client HMAC Key calculated
[libssh2] 0.513067 Key Ex: Client to Server compression initialized
[libssh2] 0.513069 Key Ex: Server to Client compression initialized
Fingerprint: 3D EB F3 D1 63 0B 22 5E E6 A8 46 F0 61 88 F7 C0 50 6A 11 5C 
[libssh2] 0.518451 Userauth: Permitted auth methods: publickey
Authentication methods: publickey
[libssh2] 0.518460 Userauth: Loading public key file: /home/ian/.ssh/pubs/id_rsa_test.pub
[libssh2] 0.518494 Userauth: Attempting publickey authentication
        Authentication by public key failed!
        Error (-18): Username/PublicKey combination invalid
all done!

1.10.0 example-ssh2_agent PubkeyAcceptedKeyTypes=ssh-ed25519,rsa-sha2-256,ssh-rsa (Success expected):

[libssh2] 0.118770 Key Ex: Sent KEX: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1,diffie-hellman-group-exchange-sha1
[libssh2] 0.118786 Key Ex: Sent HOSTKEY: ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519,ssh-rsa,ssh-dss
[libssh2] 0.118788 Key Ex: Sent CRYPT_CS: aes128-ctr,aes192-ctr,aes256-ctr,aes256-cbc,rijndael-cbc@lysator.liu.se,aes192-cbc,aes128-cbc,blowfish-cbc,arcfour128,arcfour,cast128-cbc,3des-cbc
[libssh2] 0.118789 Key Ex: Sent CRYPT_SC: aes128-ctr,aes192-ctr,aes256-ctr,aes256-cbc,rijndael-cbc@lysator.liu.se,aes192-cbc,aes128-cbc,blowfish-cbc,arcfour128,arcfour,cast128-cbc,3des-cbc
[libssh2] 0.118790 Key Ex: Sent MAC_CS: hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-sha1-96,hmac-md5,hmac-md5-96,hmac-ripemd160,hmac-ripemd160@openssh.com
[libssh2] 0.118791 Key Ex: Sent MAC_SC: hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-sha1-96,hmac-md5,hmac-md5-96,hmac-ripemd160,hmac-ripemd160@openssh.com
[libssh2] 0.118792 Key Ex: Sent COMP_CS: none
[libssh2] 0.118793 Key Ex: Sent COMP_SC: none
[libssh2] 0.118794 Key Ex: Sent LANG_CS: 
[libssh2] 0.118795 Key Ex: Sent LANG_SC: 
[libssh2] 0.119483 Key Ex: Agreed on KEX method: curve25519-sha256
[libssh2] 0.119489 Key Ex: Agreed on HOSTKEY method: ecdsa-sha2-nistp256
[libssh2] 0.119490 Key Ex: Agreed on CRYPT_CS method: aes128-ctr
[libssh2] 0.119491 Key Ex: Agreed on CRYPT_SC method: aes128-ctr
[libssh2] 0.119492 Key Ex: Agreed on MAC_CS method: hmac-sha2-256
[libssh2] 0.119493 Key Ex: Agreed on MAC_SC method: hmac-sha2-256
[libssh2] 0.119494 Key Ex: Agreed on COMP_CS method: none
[libssh2] 0.119495 Key Ex: Agreed on COMP_SC method: none
[libssh2] 0.119563 Key Ex: Initiating curve25519 SHA2
[libssh2] 0.123372 Key Ex: Server's MD5 Fingerprint: af:61:b0:ad:fa:6e:3b:b6:c4:0b:4a:ee:34:85:9a:f1
[libssh2] 0.123386 Key Ex: Server's SHA1 Fingerprint: 3d:eb:f3:d1:63:0b:22:5e:e6:a8:46:f0:61:88:f7:c0:50:6a:11:5c
[libssh2] 0.123391 Key Ex: Server's SHA256 Fingerprint: fDDUXe6r2q+TCFIhzZdck5R2zj1TaMiM/JL7gLsUhoA=
[libssh2] 0.123632 Key Ex: Received NEWKEYS message
[libssh2] 0.123636 Key Ex: session_id calculated
[libssh2] 0.123643 Key Ex: Client to Server IV and Key calculated
[libssh2] 0.123646 Key Ex: Server to Client IV and Key calculated
[libssh2] 0.123648 Key Ex: Client to Server HMAC Key calculated
[libssh2] 0.123650 Key Ex: Server to Client HMAC Key calculated
[libssh2] 0.123651 Key Ex: Client to Server compression initialized
[libssh2] 0.123652 Key Ex: Server to Client compression initialized
Fingerprint: 3D EB F3 D1 63 0B 22 5E E6 A8 46 F0 61 88 F7 C0 50 6A 11 5C 
[libssh2] 0.131866 Userauth: Permitted auth methods: publickey
Authentication methods: publickey
[libssh2] 0.132304 Userauth: Attempting publickey authentication
[libssh2] 0.136196 Userauth: Attempting publickey authentication -- phase 2
[libssh2] 0.148798 Userauth: Publickey authentication successful
        Authentication with username git and public key ian@ianh-dell succeeded!
all done!

master (13ad7b2f5cd67e0dc843098ce19ce8b208368c29) example-ssh2 PubkeyAcceptedKeyTypes=ssh-ed25519,rsa-sha2-512,rsa-sha2-256 (Success expected):

Connecting to git@172.17.0.2. pass: password
[libssh2] 0.320826 Key Ex: Sent KEX: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1,diffie-hellman-group-exchange-sha1,ext-info-c
[libssh2] 0.320846 Key Ex: Sent HOSTKEY: ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ssh-dss
[libssh2] 0.320849 Key Ex: Sent CRYPT_CS: aes128-ctr,aes192-ctr,aes256-ctr,aes256-cbc,rijndael-cbc@lysator.liu.se,aes192-cbc,aes128-cbc,blowfish-cbc,arcfour128,arcfour,cast128-cbc,3des-cbc
[libssh2] 0.320850 Key Ex: Sent CRYPT_SC: aes128-ctr,aes192-ctr,aes256-ctr,aes256-cbc,rijndael-cbc@lysator.liu.se,aes192-cbc,aes128-cbc,blowfish-cbc,arcfour128,arcfour,cast128-cbc,3des-cbc
[libssh2] 0.320852 Key Ex: Sent MAC_CS: hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-sha1-96,hmac-md5,hmac-md5-96,hmac-ripemd160,hmac-ripemd160@openssh.com
[libssh2] 0.320854 Key Ex: Sent MAC_SC: hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-sha1-96,hmac-md5,hmac-md5-96,hmac-ripemd160,hmac-ripemd160@openssh.com
[libssh2] 0.320855 Key Ex: Sent COMP_CS: none
[libssh2] 0.320856 Key Ex: Sent COMP_SC: none
[libssh2] 0.320858 Key Ex: Sent LANG_CS: 
[libssh2] 0.320859 Key Ex: Sent LANG_SC: 
[libssh2] 0.321839 Key Ex: Agreed on KEX method: curve25519-sha256
[libssh2] 0.321849 Key Ex: Agreed on HOSTKEY method: ecdsa-sha2-nistp256
[libssh2] 0.321852 Key Ex: Agreed on CRYPT_CS method: aes128-ctr
[libssh2] 0.321853 Key Ex: Agreed on CRYPT_SC method: aes128-ctr
[libssh2] 0.321854 Key Ex: Agreed on MAC_CS method: hmac-sha2-256
[libssh2] 0.321857 Key Ex: Agreed on MAC_SC method: hmac-sha2-256
[libssh2] 0.321859 Key Ex: Agreed on COMP_CS method: none
[libssh2] 0.321861 Key Ex: Agreed on COMP_SC method: none
[libssh2] 0.321956 Key Ex: Initiating curve25519 SHA2
[libssh2] 0.327217 Key Ex: Server's MD5 Fingerprint: af:61:b0:ad:fa:6e:3b:b6:c4:0b:4a:ee:34:85:9a:f1
[libssh2] 0.327236 Key Ex: Server's SHA1 Fingerprint: 3d:eb:f3:d1:63:0b:22:5e:e6:a8:46:f0:61:88:f7:c0:50:6a:11:5c
[libssh2] 0.327246 Key Ex: Server's SHA256 Fingerprint: fDDUXe6r2q+TCFIhzZdck5R2zj1TaMiM/JL7gLsUhoA=
[libssh2] 0.327633 Key Ex: Received NEWKEYS message
[libssh2] 0.327641 Key Ex: session_id calculated
[libssh2] 0.327652 Key Ex: Client to Server IV and Key calculated
[libssh2] 0.327657 Key Ex: Server to Client IV and Key calculated
[libssh2] 0.327660 Key Ex: Client to Server HMAC Key calculated
[libssh2] 0.327663 Key Ex: Server to Client HMAC Key calculated
[libssh2] 0.327664 Key Ex: Client to Server compression initialized
[libssh2] 0.327665 Key Ex: Server to Client compression initialized
[libssh2] 0.327711 Key Ex: Server to Client extension server-sig-algs: ssh-ed25519,sk-ssh-ed25519@openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com,webauthn-sk-ecdsa-sha2-nistp256@openssh.com
Fingerprint: 3D EB F3 D1 63 0B 22 5E E6 A8 46 F0 61 88 F7 C0 50 6A 11 5C 
[libssh2] 0.333604 Userauth: Permitted auth methods: publickey
Authentication methods: publickey
[libssh2] 0.333623 Userauth: Loading public key file: /home/ian/.ssh/pubs/id_rsa_test.pub
[libssh2] 0.333687 Key Ex: Signing using rsa-sha2-512
[libssh2] 0.333692 Userauth: Attempting publickey authentication
[libssh2] 0.334412 Userauth: Loading private key file: /home/ian/.ssh/id_rsa_test
[libssh2] 0.334519 Userauth: Computing RSA keys from private key data
[libssh2] 0.338672 Userauth: Attempting publickey authentication -- phase 2
[libssh2] 0.345583 Userauth: Publickey authentication successful
        Authentication by public key succeeded.
all done!

master (13ad7b2f5cd67e0dc843098ce19ce8b208368c29) example-ssh2_agent PubkeyAcceptedKeyTypes=ssh-ed25519,rsa-sha2-512,rsa-sha2-256[,ssh-rsa] (Failure NOT expected):

[libssh2] 0.020891 Key Ex: Sent KEX: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1,diffie-hellman-group-exchange-sha1,ext-info-c
[libssh2] 0.020911 Key Ex: Sent HOSTKEY: ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ssh-dss
[libssh2] 0.020913 Key Ex: Sent CRYPT_CS: aes128-ctr,aes192-ctr,aes256-ctr,aes256-cbc,rijndael-cbc@lysator.liu.se,aes192-cbc,aes128-cbc,blowfish-cbc,arcfour128,arcfour,cast128-cbc,3des-cbc
[libssh2] 0.020914 Key Ex: Sent CRYPT_SC: aes128-ctr,aes192-ctr,aes256-ctr,aes256-cbc,rijndael-cbc@lysator.liu.se,aes192-cbc,aes128-cbc,blowfish-cbc,arcfour128,arcfour,cast128-cbc,3des-cbc
[libssh2] 0.020916 Key Ex: Sent MAC_CS: hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-sha1-96,hmac-md5,hmac-md5-96,hmac-ripemd160,hmac-ripemd160@openssh.com
[libssh2] 0.020917 Key Ex: Sent MAC_SC: hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-sha1-96,hmac-md5,hmac-md5-96,hmac-ripemd160,hmac-ripemd160@openssh.com
[libssh2] 0.020918 Key Ex: Sent COMP_CS: none
[libssh2] 0.020919 Key Ex: Sent COMP_SC: none
[libssh2] 0.020920 Key Ex: Sent LANG_CS: 
[libssh2] 0.020921 Key Ex: Sent LANG_SC: 
[libssh2] 0.021780 Key Ex: Agreed on KEX method: curve25519-sha256
[libssh2] 0.021788 Key Ex: Agreed on HOSTKEY method: ecdsa-sha2-nistp256
[libssh2] 0.021790 Key Ex: Agreed on CRYPT_CS method: aes128-ctr
[libssh2] 0.021791 Key Ex: Agreed on CRYPT_SC method: aes128-ctr
[libssh2] 0.021791 Key Ex: Agreed on MAC_CS method: hmac-sha2-256
[libssh2] 0.021793 Key Ex: Agreed on MAC_SC method: hmac-sha2-256
[libssh2] 0.021793 Key Ex: Agreed on COMP_CS method: none
[libssh2] 0.021794 Key Ex: Agreed on COMP_SC method: none
[libssh2] 0.021867 Key Ex: Initiating curve25519 SHA2
[libssh2] 0.025822 Key Ex: Server's MD5 Fingerprint: af:61:b0:ad:fa:6e:3b:b6:c4:0b:4a:ee:34:85:9a:f1
[libssh2] 0.025833 Key Ex: Server's SHA1 Fingerprint: 3d:eb:f3:d1:63:0b:22:5e:e6:a8:46:f0:61:88:f7:c0:50:6a:11:5c
[libssh2] 0.025838 Key Ex: Server's SHA256 Fingerprint: fDDUXe6r2q+TCFIhzZdck5R2zj1TaMiM/JL7gLsUhoA=
[libssh2] 0.026104 Key Ex: Received NEWKEYS message
[libssh2] 0.026109 Key Ex: session_id calculated
[libssh2] 0.026116 Key Ex: Client to Server IV and Key calculated
[libssh2] 0.026120 Key Ex: Server to Client IV and Key calculated
[libssh2] 0.026122 Key Ex: Client to Server HMAC Key calculated
[libssh2] 0.026124 Key Ex: Server to Client HMAC Key calculated
[libssh2] 0.026125 Key Ex: Client to Server compression initialized
[libssh2] 0.026126 Key Ex: Server to Client compression initialized
[libssh2] 0.026158 Key Ex: Server to Client extension server-sig-algs: ssh-ed25519,sk-ssh-ed25519@openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com,webauthn-sk-ecdsa-sha2-nistp256@openssh.com
Fingerprint: 3D EB F3 D1 63 0B 22 5E E6 A8 46 F0 61 88 F7 C0 50 6A 11 5C 
[libssh2] 0.032922 Userauth: Permitted auth methods: publickey
Authentication methods: publickey
[libssh2] 0.033701 Key Ex: Signing using rsa-sha2-512
[libssh2] 0.033712 Userauth: Attempting publickey authentication
[libssh2] 0.037304 Userauth: Attempting publickey authentication -- phase 2
        Authentication with username git and public key ian@ianh-dell failed!
        Error (-19): Invalid signature for supplied public key, or bad username/public key combination
Couldn't continue authentication
all done!

Dockerfile

FROM debian:stable

RUN apt update && apt -y install git openssh-server \
  && adduser --disabled-password --gecos "" git \
  && runuser -l git -c 'mkdir .ssh && chmod 700 .ssh && touch .ssh/authorized_keys && chmod 600 .ssh/authorized_keys' \
  && mkdir -p /srv/git/project.git && chown -R git:git /srv/git \
  && runuser -l git -c 'cd /srv/git/project.git && git init --bare' \
  && echo "PubkeyAcceptedKeyTypes=ssh-ed25519,rsa-sha2-512,rsa-sha2-256\nPasswordAuthentication no" >> /etc/ssh/sshd_config

COPY --chown=git:git authorized_keys /home/git/.ssh/authorized_keys

ENTRYPOINT service ssh restart && bash

# On host machine:
# Ensure authorized_keys file is in same directory as sshd.Dockerfile
# sudo docker build -t sshd-git -f sshd.Dockerfile . # build image
# sudo docker run --name sshd-git_1 --rm -it sshd-git # run container, interactive, remove on close
# sudo docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' sshd-git_1 # get IP address
# git remote is git@<ip>:/srv/git/project.git

# On container:
# modify accepted protocols (allow old clients to connect):
# sed -i "s/PubkeyAcceptedKeyTypes=.*/PubkeyAcceptedKeyTypes=ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa/g" /etc/ssh/sshd_config && service ssh restart

# NOTE: openssh-server doesn't seem to account for PubkeyAcceptedKeyTypes when sending server-sig-algs
# (after removing rsa-sha2-512 from PubkeyAcceptedKeyTypes, openssh-server still sends it in server-sig-algs. The client attempts to use rsa-sha2-512 and the server rejects it):
# https://lists.archive.carbon60.com/openssh/dev/73729?page=last

Testing patch

diff --git a/example/ssh2.c b/example/ssh2.c
index fa86f55..19a3fc1 100644
--- a/example/ssh2.c
+++ b/example/ssh2.c
@@ -37,8 +37,8 @@
 #include <ctype.h>
 
 
-const char *keyfile1 = "~/.ssh/id_rsa.pub";
-const char *keyfile2 = "~/.ssh/id_rsa";
+const char *keyfile1 = "/home/ian/.ssh/pubs/id_rsa_test.pub";
+const char *keyfile2 = "/home/ian/.ssh/id_rsa_test";
 const char *username = "username";
 const char *password = "password";
 
@@ -98,6 +98,7 @@ int main(int argc, char *argv[])
         password = argv[3];
     }
 
+    printf("Connecting to %s@%s. pass: %s\n", username, argc > 1 ? argv[1] : "localhost", password);
     rc = libssh2_init(0);
     if(rc != 0) {
         fprintf(stderr, "libssh2 initialization failed (%d)\n", rc);
@@ -122,6 +123,7 @@ int main(int argc, char *argv[])
      * banners, exchange keys, and setup crypto, compression, and MAC layers
      */
     session = libssh2_session_init();
+    libssh2_trace(session, LIBSSH2_TRACE_AUTH | LIBSSH2_TRACE_KEX);
     if(libssh2_session_handshake(session, sock)) {
         fprintf(stderr, "Failure establishing SSH session\n");
         return -1;
@@ -190,9 +192,14 @@ int main(int argc, char *argv[])
     }
     else if(auth_pw & 4) {
         /* Or by public key */
-        if(libssh2_userauth_publickey_fromfile(session, username, keyfile1,
-                                               keyfile2, password)) {
+        int rc = libssh2_userauth_publickey_fromfile(session, username, keyfile1,
+                                                     keyfile2, password);
+        if (rc)
+        {
             fprintf(stderr, "\tAuthentication by public key failed!\n");
+            char *msg;
+            libssh2_session_last_error(session, &msg, NULL, 0);
+            fprintf(stderr, "\tError (%d): %s\n", rc, msg);
             goto shutdown;
         }
         else {
diff --git a/example/ssh2_agent.c b/example/ssh2_agent.c
index 1cc5084..8b514ea 100644
--- a/example/ssh2_agent.c
+++ b/example/ssh2_agent.c
@@ -101,6 +101,7 @@ int main(int argc, char *argv[])
      * banners, exchange keys, and setup crypto, compression, and MAC layers
      */
     session = libssh2_session_init();
+    libssh2_trace(session, LIBSSH2_TRACE_AUTH | LIBSSH2_TRACE_KEX);
     if(libssh2_session_handshake(session, sock)) {
         fprintf(stderr, "Failure establishing SSH session\n");
         return 1;
@@ -153,10 +154,14 @@ int main(int argc, char *argv[])
             rc = 1;
             goto shutdown;
         }
-        if(libssh2_agent_userauth(agent, username, identity)) {
+        int rc = libssh2_agent_userauth(agent, username, identity);
+        if(rc) {
             fprintf(stderr, "\tAuthentication with username %s and "
                    "public key %s failed!\n",
                    username, identity->comment);
+            char *msg;
+            libssh2_session_last_error(session, &msg, NULL, 0);
+            fprintf(stderr, "\tError (%d): %s\n", rc, msg);
         }
         else {
             fprintf(stderr, "\tAuthentication with username %s and "

Please let me know if there’s anything else I can do to assist with debugging.