calico: egress traffic tutorial fails
Expected Behavior
Step 5 to allow egress traffic should work: https://docs.projectcalico.org/v3.1/getting-started/kubernetes/tutorials/advanced-policy
Current Behavior
nslookup is still denied until all network policies with egress are removed
kubectl get netpol --all-namespaces
kubectl delete netpol -n advanced-policy-demo default-deny-egress
kubectl delete netpol -n advanced-policy-demo allow-dns-access
Possible Solution
Steps to Reproduce (for bugs)
Follow the tutorial
Context
I need to set up egress and it isn’t working.
Your Environment
My environment is documented here
- Calico version: v3.1
- Orchestrator version (e.g. kubernetes, mesos, rkt): kubernetes v1.10.0
- Operating System and version: OSX latest stable
- Link to your project (optional):
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 16 (9 by maintainers)
You can probably close this. Unfortunately, I’m having trouble setting up a minikube cluster with
--vmdriver none. I mentioned that in https://github.com/projectcalico/calico/issues/1456Yes, looks like specifying
--vmdriver nonefixes the adv. tutorial. @ctaggart, please give that a try.@ctaggart - I reproduced the minikube v0.27.0 issue that you describe.
After applying the four policies - deny-all-ingress, allow-nginx-ingress, deny-all-egress and allow-dns-egress - I see substantial differences in the iptables / ipsets rules that calico creates on minikube versus kubeadm. See minikube-iptables versus kubeadm-iptables. Ipsets on kubeadm have the kube-dns ip address (for a particular iptable rule), whereas the minikube ipsets are empty.
calicoctl node diagsalso hangs on minikube - looks like our iptable programming container (felix) times out when dumping its status.I noticed another interesting difference which points to differences in how minikube and kubeadm handle cluster networking: on kubeadm, intra-pod traffic is visible from the node (e.g.
sudo tcpdump -i any host kube-dns-ip-addr and udp and port 53shows busybox DNS requests on kubeadm), however intra-pod traffic doesn’t seem to be directly visible from the node with minikube.Although the ingress rules appear to be working on minikube, it looks like calico doesn’t have a complete view of the cluster and is not necessarily creating correct iptables/ipsets. My quick take is that minikube + calico looks untrustworthy at this point.