go: crypto/x509: "certificate is not standards compliant" on MacOS

Hello:

i can confirm that Amazon Neptune is affected :

 {"error": "x509: "*.xxxxxx.us-east-1.neptune.amazonaws.com" certificate is not standards compliant"}

what is the current recommend workaround?

We’ve had the same issue with connecting to AWS Elasticache Redis servers. Amazon will not support SCTs to avoid publishing customer cluster names in a public log. The connection previously worked fine in 1.17.

@jameskilroe hey! we at Pulumi dug deep into this after seeing a similar issue. You should be able to resolve it by restarting your machine. We found that should almost always fix the issue.

Workaround

We found that in every case where a user reported this issue, either of these were true:

1. The machine either had been recently reimaged, and not rebooted since.
2. The user had not restarted the machine in a substantial period of time.

In both cases, restarting the OS was the workaround.

Analysis

I’m going to summarize my colleague @kmosher’s analysis. In recent years Apple, Google, and others have added the following requirements for a TLS cert to be considered valid :

• It must be valid for no longer than 398 days
• It must be signed by multiple Certificate Transparency (CT) logs

On macOS, and in Safari and Go programs, the system service trustd is responsible for checking a certificate against certificate transparency (CT) log information cached by the OS.

Due to unknown reasons, trustd does not update the list of trusted certificate transparency (CT) logs it uses while the system is running. As a result, certificates signed against chains that are currently trusted (and may be listed in the file below) aren’t considered valid until a restart.

The trusted certificate transparency logs on macOS can be located here:

/System/Library/Security/Certificates.bundle/Contents/Resources/TrustedCTLogs.plist

And Apple publishes a JSON document for the OS to update from here:

https://valid.apple.com/ct/log_list/current_log_list.json

OpenShift Client 4.11.4 is affected as well.

oc login --token=sha256~xxx --server=https://api.xx.com
error: x509: “api.xx.com” certificate is not standards compliant

I just downgraded the OpenShift Client to 4.10.22 now it is working again.

Hi @AaronFriel

Thanks for the answer. I updated my machine to the latest OS to try and solve the problem and did restart my machine a few time this morning, but unfortunately, the problem persists.

I did notice that the files in my TrustedCTLogs.plist were last updated on 9 Feb (before my update). Is there any chance you know how to force and update? I did some googling but no obvious answers.

All round very frustrating!

To add: we (Sectigo) will be making a change to include SCTs even if the TLS Feature extension is present, hopefully helping to workaround this issue. @rs - @robstradling has told me the change should be implemented in the next day or so, and you should also hear from the Support team confirming the same.

@jameskilroe rather than looking at the date, diff it against the current list (https://valid.apple.com/ct/log_list/current_log_list.json). The set of valid CT logs doesn’t change that often (last change 15 Jan). If you have the latest list and have rebooted recently, then you may have a different issue (like the certificate is not actually present in the CT logs — you can check this — or is valid for too long).

@rsc, @ianlancetaylor: Based on the above, I believe this issue can be closed - or perhaps moved to a Discussion with an accepted answer. I believe that someone should file a report with Apple’s bug reporting tool Feedback Assistant to resolve this behavior with macOS if it hasn’t been. This issue may already be fixed and released, but as we describe above, users who encounter the issue are likely to be on out of date machines that have not updated or even restarted their machine recently.

For anyone struggling with this issue. I had the same problem:

package main

import (
	"log"
	"net/http"
	"os"
)

func main() {
	res, err := http.Get("https://charts.bitnami.com/bitnami/index.yaml")
	if err != nil {
		log.Fatal(err)
	}
	res.Write(os.Stdout)
}

go run <file-above>.go with version >= 1.18 lead to the mentioned certificate is not standards compliant error. It worked fine with 1.17.

Upgrading macOS Monterrey from 12.5 to 12.6.2 solved the issue for me!! It also seems to work with macOS 13 Ventura. I can now run the above program just fine with go version 1.19

Amazon Elastic Cache Redis is affected as well. x509: “*.testredis.3myxt2.use1.cache.amazonaws.com” certificate is not standards compliant

@rs We just implemented the change that @isnotnick described across all public ACME servers powered by Sectigo (which includes acme.zerossl.com).

Yup, that makes sense, just wanted to provide a viable workaround for people who do control their impl. I think the basic 1.20 fix will make this transparently disappear for most people (I will open a new issue for this, since it’ll require API changes.)