secretive: Authentication inconsistently fails against GitLab and Codewerks

I wanted to try this software so I generated a new keypair with it and copied the public key to my testserver. When trying to connect to it, a password prompt appears even after the publickey got acceptet (As seen in the log). This does not happen if I use the same setup with my ssh key in my .ssh folder.

debug1: Authentications that can continue: publickey,password
debug1: Offering public key: ecdsa-sha2-nistp256 ECDSA SHA256:RANDOMSTRING agent
debug1: Server accepts key: ecdsa-sha2-nistp256 ECDSA SHA256:RANDOMSTRING agent
debug1: Authentications that can continue: publickey,password
debug1: Offering public key: ecdsa-sha2-nistp256 ECDSA SHA256:RANDOMSTRING agent
debug1: Authentications that can continue: publickey,password
debug1: Trying private key: /Users/admin/.ssh/id_rsa
debug1: Trying private key: /Users/admin/.ssh/id_dsa
debug1: Trying private key: /Users/admin/.ssh/id_ecdsa
debug1: Trying private key: /Users/admin/.ssh/id_ed25519
debug1: Trying private key: /Users/admin/.ssh/id_xmss
debug1: Next authentication method: password
user@192.168.1.10's password:

If theres some kind of log I can provide let me know.