@BM5k figured it out for 10.12, which uses ssh-agent now (cool). Just

ssh-add ~/.ssh/<your rsa file>

And then you can brew update

FWIW I’ve had

[url "git@github.com:"]
    insteadOf = https://github.com/

In my ~/.gitconfig for ages and never had a problem until installing the 10.12 public beta.

There is some issue with your SSH setup. And you seem to be rewriting https://github.com/ remotes to git@github.com: as can be seen from the last output. Please check your ~/.gitconfig for a line similar to insteadOf = https://github.com/ that you must have added at some point in time (or some tool did for you) to your configuration. Please comment that out by prefixing the line with # and try again.

To maybe clarify a bit: I think if you look at your ~/.gitconfig you’ll find a section that looks like this:

[url "git@github.com:"]
        insteadOf = https://github.com/

This has the effect of transparently rewriting all your GitHub HTTPS URLs to SSH. And this would explain why you see this behavior. This change to ~/.gitconfig was either made by you or by some Git-related tool you have installed since you’ve last successfully ran brew update. Do you remember doing something like this in the past 7 weeks (that’s when your last updated Homebrew)?

This error can also occur the first time you interact with github when your ‘~/.ssh/known_hosts’ file doesn’t contain an entry for github.

A workaround is to do git pull or git clone in some other repo. You’ll be asked to add the github ip to your known hosts. Say yes. brew should work afterward.

Just to make sure, can you post the output of brew tap-info --installed, to see if these are all https or ssh remotes?

If those tap remotes are git@github.com: links, then this may just happen: when you use ssh connections, and it wants to use your private key, git will pop up a dialog prompting for a passphrase the first time you use it. When git is run from inside brew, though, those dialogs don’t happen and ssh key usage just fails (maybe because we’re running git headless instead of attached to a tty). This happens to me sometimes, and you see error messages just like this.

If this is the case, you can work around it by manually switching in to one of the taps, or any other local GitHub clone using the same ssh identity, and do a git update. That’ll produce the password prompt and store it for subsequent usage in your session, and brew update will be able to use it.

Or switch all the taps back to https remotes.