serving: mutation failed: cannot decode incoming new object: json: unknown field "subresource"

/area networking

What version of Knative?

0.23

Expected Behavior

Hello world works

Actual Behavior

Kservice stuck, logs are showing a bunch of errors like:

{"severity":"INFO","timestamp":"2021-06-01T18:52:01.271342634Z","logger":"webhook","caller":"webhook/admission.go:131","message":"remote admission controller audit annotations=map[string]string(nil)","commit":"35efb31","knative.dev/pod":"webhook-6f4d97c469-dvfg7","knative.dev/kind":"serving.knative.dev/v1, Kind=Route","knative.dev/namespace":"default","knative.dev/name":"hello","knative.dev/operation":"UPDATE","knative.dev/resource":"serving.knative.dev/v1, Resource=routes","knative.dev/subresource":"status","knative.dev/userinfo":"{system:serviceaccount:knative-serving:controller 515c0bce-588c-44d2-b9ee-cdade6b60f1d [system:serviceaccounts system:serviceaccounts:knative-serving system:authenticated] map[authentication.kubernetes.io/pod-name:[controller-7c697c6857-7r5zj] authentication.kubernetes.io/pod-uid:[aa02fc23-43e6-4519-9e28-002016b408e7]]}","admissionreview/uid":"daf45280-2ffd-441a-8bcd-2fe6ee9047cd","admissionreview/allowed":false,"admissionreview/result":"&Status{ListMeta:ListMeta{SelfLink:,ResourceVersion:,Continue:,RemainingItemCount:nil,},Status:Failure,Message:mutation failed: cannot decode incoming new object: json: unknown field \"subresource\",Reason:BadRequest,Details:nil,Code:400,}"}

Steps to Reproduce the Problem

kind create cluster
istioctl install -y
kubectl apply --filename https://github.com/knative/serving/releases/download/v0.23.0/serving-crds.yaml
kubectl wait --for=condition=Established --all crd
kubectl apply --filename https://github.com/knative/serving/releases/download/v0.23.0/serving-core.yaml
kubectl wait pod --timeout=-1s --for=condition=Ready -l '!job-name' -n knative-serving

kubectl apply --filename https://github.com/knative/net-istio/releases/download/v0.23.0/net-istio.yaml
kubectl replace --filename https://github.com/knative/serving/releases/download/v0.23.0/serving-default-domain.yaml


cat <<EOF | kubectl apply -f -
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: hello
spec:
  template:
    spec:
      containers:
        - image: gcr.io/knative-samples/helloworld-go
          ports:
            - containerPort: 8080
          env:
            - name: TARGET
              value: "Knative"
EOF

After manually deleting all the mutating and validating webhooks seems thing to start working