kyverno: [Bug] Attestation verification fails if signatures differ
Kyverno Version
1.8.0-rc6
Kubernetes Version
1.24.x
Kubernetes Platform
K3d
Kyverno Rule Type
verifyImages
Description
Kyverno assumes that the subject in the (keyless) signature of the container image is the same subject as in any/all attestations. This causes any attestation verification to fail when an verifyImages rule is written which checks an attestation for such an image. The behavior may have been introduced in PR #4786 by @praddy26.
For example, in GitHub Actions, if the (keyless) image signature and all attestations are produced by the same workflow as exemplified here, then there is no problem. However, if any other workflows are responsible for attesting to a predicate as exemplified here, then it exposes this problem. Note that in the former workflow, the signature and all attestations are performed at the same time by the same workflow and, therefore, the subjects are the same. Now note how, in the latter, the signature, SBOM, and initial vulnerability scans are attested by the parent workflow, but the SLSA provenance attestation and vulnerability scan replacement are done by different workflows.
Steps to reproduce
- Apply this policy
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: check-keyless-sig
spec:
validationFailureAction: enforce
webhookTimeoutSeconds: 30
rules:
- name: check-zulu-sig
match:
any:
- resources:
kinds:
- Pod
verifyImages:
- imageReferences:
- "ghcr.io/chipzoller/zulu:*"
attestors:
- entries:
- keyless:
subject: "https://github.com/chipzoller/zulu/.github/workflows/slsa-generic-keyless.yaml@refs/tags/v*"
issuer: "https://token.actions.githubusercontent.com"
- Create a Pod using the
ghcr.io/chipzoller/zulu:v0.0.14
image and see this succeeds (correct). - Delete the policy in step number one and create this policy which only adds an attestation check for SLSA provenance.
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: check-slsa-attestations
spec:
validationFailureAction: enforce
webhookTimeoutSeconds: 30
rules:
- name: check-builder-id-keyless
match:
any:
- resources:
kinds:
- Pod
verifyImages:
- imageReferences:
- "ghcr.io/chipzoller/zulu:*"
attestors:
- entries:
- keyless:
subject: "https://github.com/chipzoller/zulu/.github/workflows/slsa-generic-keyless.yaml@refs/tags/v*"
issuer: "https://token.actions.githubusercontent.com"
attestations:
- predicateType: https://slsa.dev/provenance/v0.2
conditions:
- all:
# This expression uses a regex pattern to ensure the builder.id in the attestation is equal to the official
# SLSA provenance generator workflow and uses a tagged release in semver format. If using a specific SLSA
# provenance generation workflow, you may need to adjust the first input as necessary.
- key: "{{ regex_match('^https://github.com/slsa-framework/slsa-github-generator/.github/workflows/generator_container_slsa3.yml@refs/heads/main','{{ builder.id}}') }}"
operator: Equals
value: true
- Attempt to create another Pod using the same image. See that this fails (incorrect) with the following message:
Error from server: admission webhook "mutate.kyverno.svc-fail" denied the request:
policy Pod/default/zulu for resource violation:
check-slsa-attestations:
check-builder-id-keyless: 'failed to verify image ghcr.io/chipzoller/zulu:v0.0.14:
.attestors[0].entries[0].keyless: subject mismatch: expected https://github.com/chipzoller/zulu/.github/workflows/slsa-generic-keyless.yaml@refs/tags/v*,
received https://github.com/chipzoller/zulu/.github/workflows/vulnerability-scan.yaml@refs/heads/main'
- Notice how the message cites a subject mismatch due to Kyverno confusing the image signature with the attestation signature. The subject
https://github.com/chipzoller/zulu/.github/workflows/vulnerability-scan.yaml@refs/heads/main
is indeed a valid subject but for a different attestation and not the one targeted by the policy in step three.
Expected behavior
The Pod is allowed to be created because the image signature is valid and the attestation is valid according to the listed expression. This may be confirmed as shown below:
$ COSIGN_EXPERIMENTAL=1 cosign verify-attestation --type https://slsa.dev/provenance/v0.2 ghcr.io/chipzoller/zulu:v0.0.14 | jq -r .payload | base64 --decode | jq .predicate.builder.id
tuf: warning using deprecated ecdsa hex-encoded keys
<snip>
Verification for ghcr.io/chipzoller/zulu:v0.0.14 --
The following checks were performed on each of these signatures:
- The cosign claims were validated
- Existence of the claims in the transparency log was verified offline
- Any certificates were verified against the Fulcio roots.
Certificate subject: https://github.com/slsa-framework/slsa-github-generator/.github/workflows/generator_container_slsa3.yml@refs/heads/main
Certificate issuer URL: https://token.actions.githubusercontent.com
Certificate extension GitHub Workflow Trigger: push
Certificate extension GitHub Workflow SHA: d6a992eb8656487aba90ca4854023d15b185f61e
Certificate extension GitHub Workflow Name: slsa-generic-keyless
Certificate extension GitHub Workflow Trigger chipzoller/zulu
Certificate extension GitHub Workflow Ref: refs/tags/v0.0.14
"https://github.com/slsa-framework/slsa-github-generator/.github/workflows/generator_container_slsa3.yml@refs/heads/main"
Screenshots
No response
Kyverno logs
No response
Slack discussion
No response
Troubleshooting
- I have read and followed the documentation AND the troubleshooting guide.
- I have searched other issues in this repository and mine is not recorded.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 18 (18 by maintainers)
Got clarification from Jim, @praddy26 you are right, we will need something like this as you proposed https://github.com/kyverno/kyverno/issues/4847#issuecomment-1312492037.
Let’s meet and discuss next steps to unblock 1.8.2.
Seeing as how we continue to warn that verifyImages is a beta feature and explicitly call out “there may be breaking changes”, I don’t think it’s a huge deal right now. Ref.: https://main.kyverno.io/docs/writing-policies/verify-images/