slsa-github-generator: [bug] "Generate builder" and "Run sigstore/cosign-installer" steps failing with `error updating to TUF remote mirror: invalid key`

The “Generate builder” and “Run sigstore/cosign-installer” steps have started failing for my workflows. This used to work fine, not sure if it is just an intermittent error or something more fundamental:

Here’s a build that worked 18 hours ago but is failing now (i.e. without any code changes): https://github.com/jkreileder/cf-ips-to-hcloud-fw/actions/runs/8339143012 (corresponding workflow)

  1. Generate builder error:
Verifying artifact slsa-generator-container-linux-amd64: FAILED: error retrieving Rekor public keys: updating local metadata and targets: error updating to TUF remote mirror: invalid key
remote status:{
	"mirror": "https://tuf-repo-cdn.sigstore.dev/",
	"metadata": {
		"root.json": {
			"version": 9,
			"len": 6766,
			"expiration": "12 Sep 24 06:53 UTC",
			"error": ""
		},
		"snapshot.json": {
			"version": 132,
			"len": 2302,
			"expiration": "09 Apr 24 16:16 UTC",
			"error": ""
		},
		"targets.json": {
			"version": 9,
			"len": 5478,
			"expiration": "12 Sep 24 06:13 UTC",
			"error": ""
		},
		"timestamp.json": {
			"version": 169,
			"len": 723,
			"expiration": "26 Mar 24 16:16 UTC",
			"error": ""
		}
	}
}

FAILED: SLSA verification failed: error retrieving Rekor public keys: updating local metadata and targets: error updating to TUF remote mirror: invalid key
remote status:{
	"mirror": "https://tuf-repo-cdn.sigstore.dev/",
	"metadata": {
		"root.json": {
			"version": 9,
			"len": 6766,
			"expiration": "12 Sep 24 06:53 UTC",
			"error": ""
		},
		"snapshot.json": {
			"version": 132,
			"len": 2302,
			"expiration": "09 Apr 24 16:16 UTC",
			"error": ""
		},
		"targets.json": {
			"version": 9,
			"len": 5478,
			"expiration": "12 Sep 24 06:13 UTC",
			"error": ""
		},
		"timestamp.json": {
			"version": 169,
			"len": 723,
			"expiration": "26 Mar 24 16:16 UTC",
			"error": ""
		}
	}
}
Error: Process completed with exit code 6.
  1. Run sigstore/cosign-installer@6e04d228eb30da1757ee4e1dd75a0ec73a653e06 error:
Error: getting ctlog public keys: updating local metadata and targets: error updating to TUF remote mirror: invalid key
remote status:{
	"mirror": "https://tuf-repo-cdn.sigstore.dev/",
	"metadata": {
		"root.json": {
			"version": 9,
			"len": 6766,
			"expiration": "12 Sep 24 06:53 UTC",
			"error": ""
		},
		"snapshot.json": {
			"version": 132,
			"len": 2302,
			"expiration": "09 Apr 24 16:16 UTC",
			"error": ""
		},
		"targets.json": {
			"version": 9,
			"len": 5478,
			"expiration": "12 Sep 24 06:13 UTC",
			"error": ""
		},
		"timestamp.json": {
			"version": 169,
			"len": 723,
			"expiration": "26 Mar 24 16:16 UTC",
			"error": ""
		}
	}
}
main.go:74: error during command execution: getting ctlog public keys: updating local metadata and targets: error updating to TUF remote mirror: invalid key
remote status:{
	"mirror": "https://tuf-repo-cdn.sigstore.dev/",
	"metadata": {
		"root.json": {
			"version": 9,
			"len": 6766,
			"expiration": "12 Sep 24 06:53 UTC",
			"error": ""
		},
		"snapshot.json": {
			"version": 132,
			"len": 2302,
			"expiration": "09 Apr 24 16:16 UTC",
			"error": ""
		},
		"targets.json": {
			"version": 9,
			"len": 5478,
			"expiration": "12 Sep 24 06:13 UTC",
			"error": ""
		},
		"timestamp.json": {
			"version": 169,
			"len": 723,
			"expiration": "26 Mar 24 16:16 UTC",
			"error": ""
		}
	}
}
Error: Process completed with exit code 1.

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 5
  • Comments: 23 (15 by maintainers)

Commits related to this issue

Most upvoted comments

Breaking changes are not something that is going to happen frequently and if they do, there will be a significant period of time to have clients migrate over.

@ddworken The latest version of slsa-verifier is still is able to verify old provenances. Why do you need to use an old version of slsa-verifier?

@haydentherapper @bobcallaway Probably unrelated to this issue but what is the Sigstore/Cosign solution for revoked root keys and in general how does Sigstore provide backward compatibility with old versions of certificates, and therefore verifying existing legitimate provenances that were signed with revoked keys?

We don’t expect this to happen. @haydentherapper can comment on the specifics of cosign.

Probably unrelated to this issue but what is the Sigstore/Cosign solution for revoked root keys and in general how does Sigstore provide backward compatibility with old versions of certificates, and therefore verifying existing legitimate provenances that were signed with revoked keys?

@behnazh-w Supporting validity windows was the motivation behind the design of the trust root spec. For each root of trust, a validity time period is specified. If a compromise were to occur, rather than revoking the root material entirely and assuming we can determine the window when compromise occurred, the root material would simply be specified as valid only up to the compromise.

All Sigstore clients (eg sigstore-python, js, etc) but Cosign support ingesting the roots of trust in this format currently. Cosign will be updated to read it soon.

Works again for me with the un-finalized v1.10.0 pre-release.

Thank you @laurentsimon for being so responsive on this issue; I understand the commit auto-closed the bug. It’s helpful to have something to follow. Good luck and thank you for your hard work securing software supply chains.

Generator at main won’t pass verification. We need to cut the release. @kpk47 is working on it

Hi, yes let’s do that. I think we need to revert a few breaking PRs we made recently in the slsa-github-generator. slsa-verifier should be ready to be released. @ramonpetgrave64 could you send PRs to temporarily revert the few breaking PRs we made recently?

@laurentsimon @ianlewis I’m sorry to bother you, but could you release a new version of slsa-github-generator to solve this issue? https://github.com/slsa-framework/slsa-github-generator/issues/3350#issuecomment-2008731217