cilium: NetworkPolicy failures due to missing identity

Bug report

General Information

• Cilium version (run cilium version)
  • 1.10.3 installed via Helm in ENI mode + replacing kube-proxy
• Kernel version (run uname -a)
  • 5.4.129-63.229.amzn2.x86_64 (latest EKS-optimised AMI)
• Orchestration system version in use (e.g. kubectl version, …)
  • v1.21.2-eks-0389ca3
  • Ingress NGINX 0.49 w/AWS Network Load Balancer (nginx is terminating TLS)
• Link to relevant artifacts (policies, deployments scripts, …)
• Generate and upload a system zip:

curl -sLO https://git.io/cilium-sysdump-latest.zip && python cilium-sysdump-latest.zip

I unfortunately can’t provide too much information, however I’ll give what I can.

How to reproduce the issue

I’m running Cilium on EKS in ENI mode to provide NetworkPolicy capability not provided by the aws-vpc-cni (note: I have had this problem in both Cilium+ENI and Cilium chaining aws-vpc-cni). Cilium is installed and working as expected (the connectivity checks all pass). However when attempting to write some NetworkPolicies I’ve run into a strange issue.

I’ve created a NetworkPolicy to allow access from the ingress-nginx-controller pods in the ingress-nginx namespace, however when viewing flows via Hubble the traffic is always shown as below:

Source Pod	Source Service	Destination Service	Verdict
ingress-nginx-controller-some-id	No app name	some-service	dropped
ingress-nginx-controller-some-id	No app name	some-service	dropped
ingress-nginx-controller-some-id	ingress-nginx	some-service	forwarded
ingress-nginx-controller-some-id	No app name	some-service	dropped

The source is always the same, yet sometimes the “Source Service” is present and other times it is not. Because of this, the NetworkPolicy usually doesn’t allow traffic leading to 504 errors.

Running kubectl get ciliumid returns identities for all pods except for the Ingress NGINX pods. Is there any reason why it would be missing an identity? The only difference I can see between NGINX and the other pods is that it uses a LoadBalancer service. If that is a problem, how can I ensure it gets looped into Cilium’s control?

A standard k8s NetworkPolicy that has caused the issue:

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: allow-from-ingress
  namespace: foobar
spec:
  podSelector:
    matchLabels: {}
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: ingress-nginx

A more relaxed NetworkPolicy which should allow traffic from all namespaces:

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: allow-from-all
  namespace: foobar
spec:
  podSelector:
    matchLabels: {}
  ingress:
  - from:
    - namespaceSelector: {}

This policy also doesn’t allow traffic since I presume the lack of identity makes Cilium presume the traffic is coming from outside the cluster. I have also tried the CiliumNetworkPolicy equivalent of the above generated with the editor.cilium.io service.

Any help would be much appreciated.