calico: RKE: NetworkPolicy based on NamespaceSelector doesn't work
Expected Behavior
Pods in another namespace allow traffic in from nginx-ingress namespace with namepsaceSelector.
Current Behavior
- Calico has been deploy when launching rke. (https://rancher.com/docs/rke/latest/en/config-options/add-ons/network-plugins/)
- In RKE, nginx-ingress deployed with DaemonSet by RKE.
# ingress controller
ingress:
provider: nginx
options:
use-forwarded-headers: "true"
# CNI
network:
plugin: calico
We have a service running with containerPort: 80. We create a network policy to nginx-ingress that can connect to our service., but it not working with namespaceSelector but it work if I use node worker IPs.
Possible Solution
Steps to Reproduce (for bugs)
- When we use namespaceSeletor to create network policy, I can not access our website.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
generation: 12
name: marketing-landing-page-policy
namespace: marketing-portal
spec:
ingress:
- from:
- namespaceSelector:
matchLabels:
service: ingress-nginx
ports:
- port: 80
protocol: TCP
podSelector:
matchLabels:
app.kubernetes.io/name: marketing-landing-page
policyTypes:
- Ingress
- Egress
- I try to create a pod in nginx-ingress namespace and run nc, it’s connected.
kubectl run curl-test --labels="app=test-curl" --image=infoblox/dnstools:latest --namespace ingress-nginx -i --tty --rm
If you don't see a command prompt, try pressing enter.
dnstools# nc -v 172.22.1.153 80
172.22.1.153 (172.22.1.153:80) open
- However, nc from nginx-ingress-controller pod => time out
bash-5.0$ ./nc -v 172.22.1.153 80
Ncat: Version 6.49BETA1 ( http://nmap.org/ncat )
Ncat: Operation timed out.
- When we use IP worker node (projectcalico.org/IPv4IPIPTunnelAddr) nginx-ingress can connect to our service => but it’s not a good solution.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
generation: 12
name: marketing-landing-page-policy
namespace: marketing-portal
spec:
ingress:
- ipBlock:
cidr: 172.22.1.120/32
- ipBlock:
cidr: 172.22.1.121/32
- ipBlock:
cidr: 172.22.1.122/32
- ipBlock:
cidr: 172.22.1.123/32
ports:
- port: 80
protocol: TCP
podSelector:
matchLabels:
app.kubernetes.io/name: marketing-landing-page
policyTypes:
- Ingress
- Egress
Context
Could you please help me with a solution to use NetworkPolicy base on NamespaceSelector with nginx-ingress?
Your Environment
- Calico version: v3.13.4
- Orchestrator version (e.g. kubernetes, mesos, rkt): Kubernetes v1.16.13 (RKE)
- Operating System and version: Ubuntu 20.04 LTS
- Link to your project (optional):
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 15 (8 by maintainers)
@mgleung the RKE instance was updated yesterday and I tried today. Now it really works. Ingress in new RKE versions are not host networked anymore and network policies can be correctly set. So I think maybe this issue can be closed with the hint, that the ingress controller must not be run with hostNetwork: true Thank you.
Thanks for the answer. I think maybe the ingress controller used is at fault here… We’re using the ingress controller installed by RKE, if I see that correctly (
image: rancher/nginx-ingress-controller:nginx-0.43.0-rancher1), which deploys a DaemonSet withhostNetwork: true- and the hostNetwork seems to be the problem. The ingress controller you’re using is structured qute differently. According to https://rancher.com/docs/rke/latest/en/config-options/add-ons/ingress-controllers/#configuring-network-options newer RKE ingress installations do not use hostNetwork anymore (and before, it could have been configured not to use hostNetwork). Maybe I’ll ask our admins to upgrade the test cluster to see, if the issue will be solved then.EDIT: Forgot to mention that I started your deployment 1:1 (only did not install ingress-controller) and can still not access the nginx pod (timeout).
Hello. I want to confirm the behaviour initially described by @ducminhle. Namespace selectors per se are working, but whitelisting ingress-nginx namespace does not work. Or rather, it works, but the ingress-nginx controllers do not seem to be catched by the rule (also deployed a simple pod in ingress-nginx namespace to confirm that this pod CAN reach my “target” namespace). Some search indicated, that hostNetwork may be the problem:
Tried it with three different RKE/K8s/Calco versions (upgraded to newest stable RKE 1.2.7 yesterday from a few versions back - old RKE had k8s 1.17.7 - I don’t know RKE or Calio version, though. I think Calico was < 3.16 before)
Current versons:
Is this a known limitation? How can I isolate my namespace to only allow ingress from a few other pods/namespaces AND ingress-nginx? Using the internal IPs of the ingress-nginx-controllers would be very static…
What I did:
EDIT: Since deployments should be reachable from the outside, anyway, I decided to generally allow access to the http port from anywhere.