yubico-piv-tool: sign_and_send_pubkey: signing failed: agent refused operation (after some inactivity)

Remote ssh-server can’t verify my private key from YubiKey after thirty ~ fourty five minutes ssh-agent inactivity.

I use YubiKey 5C Nano under MacOS 11.5.2 (Apple M1) with lib from yubico-piv-tool-2.2.0-mac-arm64.pkg package. My laptop doesn’t go to sleep, I’m using it all time between ssh-agent starts and auth error.

Example, start ssh-agent:

Console one:

user@host1 ~ $ ssh-agent -d -a /Users/user/.ssh/ssh_auth_sock -P /usr/local/lib/libykcs11.dylib
SSH_AUTH_SOCK=/Users/user/.ssh/ssh_auth_sock; export SSH_AUTH_SOCK;
echo Agent pid 59899;
debug2: fd 3 setting O_NONBLOCK

Console two:

user@host1~ $ SSH_AUTH_SOCK=$HOME/.ssh/ssh_auth_sock ssh-add -s /usr/local/lib/libykcs11.dylib
Enter passphrase for PKCS#11:
Card added: /usr/local/lib/libykcs11.dylib

Console one:

...
debug3: fd 4 is O_NONBLOCK
debug1: process_message: socket 1 (fd=4) type 20
debug1: process_add_smartcard_key: add /usr/local/lib/libykcs11.dylib
debug1: pkcs11_start_helper: starting /usr/libexec/ssh-pkcs11-helper -vvv
debug1: process_add
debug1: provider /usr/local/lib/libykcs11.dylib: manufacturerID <Yubico (www.yubico.com)> cryptokiVersion 2.40 libraryDescription <PKCS#11 PIV Library (SP-800-73)> libraryVersion 2.20
debug1: provider /usr/local/lib/libykcs11.dylib slot 0: label <YubiKey PIV #10114264> manufacturerID <Yubico (www.yubico.com)> model <YubiKey YK5> serial <10114264> flags 0x40d
debug1: have 1 keys
debug1: have 2 keys
debug1: pkcs11_k11_free: parent 0x145b1a730 ptr 0x145b1a620 idx 0
debug1: pkcs11_provider_unref: 0x145804350 refcount 3
debug1: pkcs11_k11_free: parent 0x145b19ea0 ptr 0x145b19d90 idx 0
debug1: pkcs11_provider_unref: 0x145804350 refcount 3
debug1: pkcs11_k11_free: parent 0x145b1a930 ptr 0x145b198f0 idx 0
debug1: pkcs11_provider_unref: 0x145804350 refcount 3

# All right (mark one)
debug3: fd 4 is O_NONBLOCK
debug1: process_message: socket 1 (fd=4) type 11
debug1: process_message: socket 1 (fd=4) type 13
debug1: process_sign
debug1: check 0x145b19590 /usr/local/lib/libykcs11.dylib
debug1: pkcs11_check_obj_bool_attrib: provider 0x145804350 slot 0 object 86: attrib 514 = 0
debug1: pkcs11_k11_free: parent 0x144711900 ptr 0x0 idx 0

# All right too (mark two)
debug3: fd 4 is O_NONBLOCK
debug1: process_message: socket 1 (fd=4) type 11
debug1: process_message: socket 1 (fd=4) type 13
debug1: process_sign
debug1: check 0x145b19590 /usr/local/lib/libykcs11.dylib
debug1: pkcs11_check_obj_bool_attrib: provider 0x145804350 slot 0 object 86: attrib 514 = 0
debug1: pkcs11_k11_free: parent 0x144711900 ptr 0x0 idx 0

# Error after inactivity (mark three)
debug3: fd 4 is O_NONBLOCK
debug1: process_message: socket 1 (fd=4) type 11
debug1: process_message: socket 1 (fd=4) type 13
debug1: process_sign
debug1: check 0x145b19590 /usr/local/lib/libykcs11.dylib
debug1: pkcs11_check_obj_bool_attrib: provider 0x145804350 slot 0 object 86: attrib 514 = 0
C_Sign failed: 48
debug1: pkcs11_k11_free: parent 0x145805450 ptr 0x0 idx 0
process_sign_request2: sshkey_sign: error in libcrypto

# Same error again (mark four)
debug3: fd 4 is O_NONBLOCK
debug1: process_message: socket 1 (fd=4) type 11
debug1: process_message: socket 1 (fd=4) type 13
debug1: process_sign
debug1: check 0x145b19590 /usr/local/lib/libykcs11.dylib
debug1: pkcs11_check_obj_bool_attrib: provider 0x145804350 slot 0 object 86: attrib 514 = 0
C_Sign failed: 48
debug1: pkcs11_k11_free: parent 0x144711900 ptr 0x0 idx 0
process_sign_request2: sshkey_sign: error in libcrypto

Console three after ssh-agent (re)start:

user@host1 ~ $ ssh -v host3
...
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: Next authentication method: gssapi-keyex
debug1: No valid Key exchange context
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure.  Minor code may provide more information
No Kerberos credentials available: Disk quota exceeded

debug1: Unspecified GSS failure.  Minor code may provide more information
No Kerberos credentials available: Disk quota exceeded

debug1: Next authentication method: publickey
debug1: Offering RSA public key: /usr/local/lib/libykcs11.dylib
debug1: Server accepts key: pkalg rsa-sha2-512 blen 279
debug1: Authentication succeeded (publickey).
Authenticated to host3 ([10.1.1.1]:22).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
...
-bash-4.2$ hostname
host 3

Console three after some time (between MARK TWO and MARK THREE), I’m on the remote host and usging agent forwarding:

user@host2 ~ $ ssh -v host3
OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017
debug1: Reading configuration data /home/user/.ssh/config
debug1: /home/user/.ssh/config line 1: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 1: Applying options for *
debug1: Connecting to host3 [10.1.1.1] port 22.
debug1: Connection established.
debug1: key_load_public: No such file or directory
debug1: identity file /home/user/.ssh/id_rsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/user/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/user/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/user/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/user/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/user/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/user/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/user/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.4
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4
debug1: match: OpenSSH_7.4 pat OpenSSH* compat 0x04000000
debug1: Authenticating to host3:22 as 'user'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: curve25519-sha256 need=64 dh_need=64
debug1: kex: curve25519-sha256 need=64 dh_need=64
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:cyhMOsis5CrYnZ/U102a1pksG6DTJaprzVgySp182GE
debug1: Host 'host3' is known and matches the ECDSA host key.
debug1: Found key in /home/user/.ssh/known_hosts:2540
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: Next authentication method: gssapi-keyex
debug1: No valid Key exchange context
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure.  Minor code may provide more information
No Kerberos credentials available: Disk quota exceeded

debug1: Unspecified GSS failure.  Minor code may provide more information
No Kerberos credentials available: Disk quota exceeded

debug1: Next authentication method: publickey
debug1: Offering RSA public key: /usr/local/lib/libykcs11.dylib
debug1: Server accepts key: pkalg rsa-sha2-512 blen 279
sign_and_send_pubkey: signing failed: agent refused operation
debug1: Offering RSA public key: /usr/local/lib/libykcs11.dylib
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: Trying private key: /home/user/.ssh/id_rsa
debug1: Trying private key: /home/user/.ssh/id_dsa
debug1: Trying private key: /home/user/.ssh/id_ecdsa
debug1: Trying private key: /home/user/.ssh/id_ed25519
debug1: No more authentication methods to try.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic).

Command “ssh-add -l” always gives same results (during normal work and after failure)

user@host1 ~ $ ssh-add -l
2048 SHA256:dvIR9e14pNm5VUxVMzNjCMnBnD0I8wCj+PKvEbjymhA /usr/local/lib/libykcs11.dylib (RSA)
2048 SHA256:Xk7dUop5/qjwucmsUduwTtG9hgBmOE/jD3UJh+wqlVY /usr/local/lib/libykcs11.dylib (RSA)

About this issue

Original URL
State: closed
Created 3 years ago
Reactions: 3
Comments: 41 (14 by maintainers)

Links to this issue

FIDO2 SSH keys not working with ssh agent

Most upvoted comments

try running gpg-connect-agent updatestartuptty /bye. I experienced the same error but I dont know if it’s the same cause. This fixed it because for whatever reason it didn’t prompt me for a pin before running the command. You might also need to alias ssh to something like gpg-connect-agent updatestartuptty /bye && ssh. For me, it works across restarts and everything now.

+37

VixieTSQ on Apr 15, 2022

try running gpg-connect-agent updatestartuptty /bye. I experienced the same error but I dont know if it’s the same cause. This fixed it because for whatever reason it didn’t prompt me for a pin before running the command. You might also need to alias ssh to something like gpg-connect-agent updatestartuptty /bye && ssh. For me, it works across restarts and everything now.

Thanks a lot! This fixed my issue.

+11

yuanfang-chen on Jul 2, 2022

There might be an issue using always-auth keys with ssh, could you try using a different slot ? Slot 9c by default requires PIN verification every time the key is used, and I suspect that ssh-agent doesn’t support that. Slot 9a by default only requires PIN once, and might work better.

qpernil on May 18, 2022

Have same issue (i guess, plz sorry if it’s off topic): After some time of inactivity, ssh connection fails with
$ ssh user@host
sign_and_send_pubkey: signing failed for RSA "PIV AUTH pubkey" from agent: agent refused operation
user@host: Permission denied (publickey).
i tried to debug this, but don’t get the point of log output:
debug1: Offering public key: PIV AUTH pubkey RSA SHA256:+%sometoken% token agent
debug1: Server accepts key: PIV AUTH pubkey RSA SHA256:+%sometoken% token agent
sign_and_send_pubkey: signing failed for RSA "PIV AUTH pubkey" from agent: agent refused operation
debug1: pkcs11_k11_free: parent 0x6000003883c0 ptr 0x0 idx 0
Usually, i just run alias ssh-add -e /usr/local/lib/opensc-pkcs11.so; ansible-vault view ~/.ssh/.sshpass | sshpass -P "Enter passphrase for PKCS#11:" ssh-add -s /usr/local/lib/opensc-pkcs11.so but it’s kinda annoying 😄

You aren’t using library from a Yubico package. You have to update (or install) the Yubico pkg and use a yubico lib.

Link to the pkg https://developers.yubico.com/yubico-piv-tool/Release_Notes.html , look for the libykcs11.dylib inside and add it instead the OpenCS lib.

alexeyantropov on May 17, 2022

Have same issue (i guess, plz sorry if it’s off topic): After some time of inactivity, ssh connection fails with

$ ssh user@host
sign_and_send_pubkey: signing failed for RSA "PIV AUTH pubkey" from agent: agent refused operation
user@host: Permission denied (publickey).

i tried to debug this, but don’t get the point of log output:

debug1: Offering public key: PIV AUTH pubkey RSA SHA256:+%sometoken% token agent
debug1: Server accepts key: PIV AUTH pubkey RSA SHA256:+%sometoken% token agent
sign_and_send_pubkey: signing failed for RSA "PIV AUTH pubkey" from agent: agent refused operation
debug1: pkcs11_k11_free: parent 0x6000003883c0 ptr 0x0 idx 0

Usually, i just run alias ssh-add -e /usr/local/lib/opensc-pkcs11.so; ansible-vault view ~/.ssh/.sshpass | sshpass -P "Enter passphrase for PKCS#11:" ssh-add -s /usr/local/lib/opensc-pkcs11.so but it’s kinda annoying 😄

Tony-Sol on May 17, 2022

Closing this issue now as it seems to be mostly solved, please open a new issue if you still have problems.

qpernil on May 11, 2022

try running gpg-connect-agent updatestartuptty /bye. I experienced the same error but I dont know if it’s the same cause. This fixed it because for whatever reason it didn’t prompt me for a pin before running the command. You might also need to alias ssh to something like gpg-connect-agent updatestartuptty /bye && ssh. For me, it works across restarts and everything now.

Thank you so much! I’ve been running into this all day today and this fixed it!!!

Apollorion on May 3, 2022

try running gpg-connect-agent updatestartuptty /bye. I experienced the same error but I dont know if it’s the same cause. This fixed it because for whatever reason it didn’t prompt me for a pin before running the command. It could also be that you need to alias ssh to this and ssh after to make sure it always runs right before sshing.

I had a similar issue like OP and this fixed it for me, thank you @VixieTSQ

Nxtmaster10 on Apr 10, 2022

Very possible that this is related to #330.

If you get a chance @alexeyantropov, can you run your same test but with export YKCS11_DBG=1?

skevy on Oct 28, 2021