calico: Outgoing packets does not follow configuration on Source IP

@MalikaML Maybe the docs is outdated, but you can try it by the following guide, Or Can you give a simple policy or gateway yamls @lou-lan?

Sure, here is example:

Install

If you want to try it out, you can install the latest version.

wget https://github.com/spidernet-io/egressgateway/blob/github_pages/charts/egressgateway-0.3.0-rc1.tgz
helm install --values.yaml egressgateway-0.3.0-rc1.tgz

Create EgressGateway CR

kubectl get nodes
NAME           STATUS   ROLES           AGE    VERSION
workstation1   Ready    control-plane   116d   v1.27.3
workstation2   Ready    <none>          116d   v1.27.3
workstation3   Ready    <none>          116d   v1.27.3

Choose a node as the Egress Node, this node will implement the Egress IP.

kubectl label node workstation3  egress=true

apiVersion: egressgateway.spidernet.io/v1beta1
kind: EgressGateway
metadata:
  name: egw1
spec:
  clusterDefault: true
  ippools:
    ipv4:
    - 10.6.1.55
    - 10.6.1.56               # Egress IP
    ipv6:
    - fd00::55
    - fd00::56                # Egress IP
  nodeSelector:
    selector:
      matchLabels:
        egress: "true"      # This means that nodes with egress=true will serve as traffic egress nodes.

Create an EgressPolicy to match the Pods that require Egress

Create test pod

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mock-app
  namespace: default
  labels:
    app: mock-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: mock-app
  template:
    metadata:
      labels:
        app: mock-app
    spec:
      nodeName: workstation2
      terminationGracePeriodSeconds: 1
      containers:
        - args:
            - "86400"
          command:
            - sleep
          image: registry.cn-shanghai.aliyuncs.com/loulan-public/tools:alpine
          imagePullPolicy: IfNotPresent
          name: sleep-container
          resources: {}

Create EgressPolicy

apiVersion: egressgateway.spidernet.io/v1beta1
kind: EgressPolicy
metadata:
  name: policy1
  namespace: default            # namesapce level
spec:
  appliedTo:
    podSelector:
      matchLabels:
        app: mock-app
  egressGatewayName: egw1
  egressIP:
    allocatorPolicy: default         # Auto assign an Egress IP, you can also specify it manually.

Test

~ kubectl exec -it mock-app-kgslh sh
/ # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
3: eth0@if32: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1430 qdisc noqueue state UP qlen 1000
    link/ether 4a:e2:c6:30:32:a1 brd ff:ff:ff:ff:ff:ff
    inet 10.21.52.94/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fd00:21::d0bb:cef5:64f2:295e/128 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::48e2:c6ff:fe30:32a1/64 scope link
       valid_lft forever preferred_lft forever

~ kubectl exec -it mock-app-kgslh sh
/ # curl 10.6.1.92:8080
Remote IP: 10.6.1.55:33082

You can see that your EgressIP has changed to the set IP.

@lou-lan Also, do I need to create it beforehand, and could you guide me on how to configure it? I’ve been searching online, but I haven’t found a clear answer yet. Thank you once again

@MalikaML

part 1 - think

EgressGateway can assign an Egress IP to a specific namespace/pod (or understood as, encapsulating the internet access traffic from a Pod, or all Pods under a namespace, as a specific Source IP).

Let’s take a step-by-step look at whether EgressGateway can solve your actual scenario.

First, let’s look at the principle of converting namespace/pod internet access traffic to an Egress IP: When EgressPolicy.spec.destSubnet is not filled in, the default behavior of EgressPolicy is to SNAT all traffic (except for accessing within the cluster) to the Egress IP. EgressGateway will automatically discover the CIDR within the cluster and can easily identify external access traffic. Of course, you can also manually specify that traffic accessing destSubnet is converted to Egress IP.

apiVersion: egressgateway.spidernet.io/v1beta1
kind: EgressPolicy
metadata:
  namespace: "default"
  name: "policy-test"
spec:
  ...
  destSubnet:
  - "1.1.1.1/32"

At this point, it seems that EgressGateway can solve the problem in your scenario. However, it’s important to note whether you need EgressGateway to record access requests? EgressGateway currently does not record access requests. If you are managing on a network device (such as a Cisco Switch), it’s easy to see this part of the traffic characteristics.

part 2 - put into effect

1. Install EgressGateway
2. Create EgressGateway CR
3. Maybe you can use this feature: Namespace Default EgressGateway
4. Create EgressPolicy to match Pods

part 3 - introspect

You should use EgressGateway when you have the above requirements. If you simply want to restrict internet traffic without identifying it, you should use k8s NetworkPolicy.