cilium: K8s labels not shown in agent

General Information

• Cilium version (run cilium version) Client: 1.8.4 17d623853 2020-09-30T17:31:44-07:00 go version go1.14 linux/amd64
• Kernel version (run uname -a) Linux tt-lab17.ko.iszn.cz 4.19.0-9-amd64 #1 SMP Debian 4.19.118-2+deb10u1 (2020-06-07) x86_64 GNU/Linux
• Orchestration system version in use (e.g. kubectl version, Mesos, …) none, compiled from source; k8s 1.18.8, etcd 3.4.3
• Link to relevant artifacts (policies, deployments scripts, …) —
• Generate and upload a system zip:

How to reproduce the issue

1. compile from scratch version 1.8.4
2. launch ./daemon/cilium-agent --enable-l7-proxy=false --disable-envoy-version-check=true --k8s-kubeconfig-path /www/adm/kubernetes/conf/kubeconfig.kubelet.yaml --kvstore etcd --kvstore-opt=etcd.config=/home/ashley/etcd/etcd.conf --enable-host-firewall --enable-remote-node-identity -D on one node
3. launch ./operator/cilium-operator --k8s-kubeconfig-path /www/adm/kubernetes/conf/kubeconfig.kubelet.yaml --kvstore etcd --kvstore-opt=etcd.config=/home/ashley/etcd/etcd.conf --synchronize-k8s-nodes --synchronize-k8s-services -D somewhere as well
4. kubectl label node tt-lab17.ko.iszn.cz node-access=ssh
5. observe

root@tt-lab17:~/cilium# ./cilium/cilium endpoint list
ENDPOINT   POLICY (ingress)   POLICY (egress)   IDENTITY   LABELS (source:key[=value])   IPv6                 IPv4           STATUS   
           ENFORCEMENT        ENFORCEMENT                                                                                    
2034       Disabled           Disabled          1          reserved:host                                                     ready   
2428       Disabled           Disabled          4          reserved:health               f00d::a11:0:0:e90d   10.17.150.32   ready   

no k8s:node-access=ssh is listed there

but when creating the example ccnp via https://docs.cilium.io/en/v1.8/gettingstarted/host-firewall/ I do get

...
...
  status:
    nodes:
      tt-lab17.ko.iszn.cz:
        enforcing: true
        lastUpdated: "2020-10-08T10:04:58.238949465+02:00"
        localPolicyRevision: 2
        ok: true

I can definitely see the data in etcd, even the agent itself prints it out:

level=debug msg="Received key update via kvstore [value {\"Name\":\"tt-lab17.ko.iszn.cz\",\"Cluster\":\"default\",\"IPAddresses\":[{\"Type\":\"InternalIP\",\"IP\":\"10.248.15.17\"},{\"Type\":\"InternalIP\",\"IP\":\"fc00::10ca:1\"},{\"Type\":\"CiliumInternalIP\",\"IP\":\"10.17.193.171\"},{\"Type\":\"CiliumInternalIP\",\"IP\":\"f00d::a11:0:0:74d1\"}],\"IPv4AllocCIDR\":{\"IP\":\"10.17.0.0\",\"Mask\":\"//8AAA==\"},\"IPv6AllocCIDR\":{\"IP\":\"f00d::a11:0:0:0\",\"Mask\":\"////////////////AAAAAA==\"},\"IPv4HealthIP\":\"10.17.150.32\",\"IPv6HealthIP\":\"f00d::a11:0:0:e90d\",\"ClusterID\":0,\"Source\":\"local\",\"EncryptionKey\":0,\"Labels\":{\"beta.kubernetes.io/arch\":\"amd64\",\"beta.kubernetes.io/os\":\"linux\",\"kubernetes.io/arch\":\"amd64\",\"kubernetes.io/hostname\":\"tt-lab17.ko.iszn.cz\",\"kubernetes.io/os\":\"linux\",\"node-access\":\"ssh\",\"node-access2\":\"ssh\",\"node.kubernetes.io/cpu-avx\":\"true\",\"node.kubernetes.io/cpu-avx2\":\"true\",\"node.kubernetes.io/cpu-avx512cd\":\"true\",\"node.kubernetes.io/cpu-avx512f\":\"true\",\"node.kubernetes.io/switch\":\"tor-lab-5\"}}]" eventType=modify key=cilium/state/nodes/v1/default/tt-lab17.ko.iszn.cz storeName=store-cilium/state/nodes/v1 subsys=shared-store

Ashley