WSL: Missing `xt_recent` kernel module resulting in inability to run Kubernetes with Session Affinity

Windows Build Number

Microsoft Windows [Version 10.0.19042.1055]

WSL Version

  • WSL 2
  • WSL 1

Kernel Version

No response

Distro Version

Ubuntu 20.04

Other Software

Kind, v0.11.1 Kubernetes, v1.20.7 Docker Desktop (Windows), version 3.4.0

Repro Steps

  1. Install Docker Desktop (link)
  2. Get Kind (link)
    go get sigs.k8s.io/kind@v0.11.1
    
  3. Create a Kubernetes cluster
    kind create cluster
    
  4. Create a Service in Kubernetes with sessionAffinity: ClientIP (kubectl apply -f service.yaml).
    # service.yaml
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: session-affinity-test
    spec:
      selector:
        matchLabels:
          app: session-affinity-test
      template:
        metadata:
          labels:
            app: session-affinity-test
        spec:
          containers:
          - name: session-affinity-test
            image: mendhak/http-https-echo
            resources:
              limits:
                memory: "128Mi"
                cpu: "500m"
            ports:
            - containerPort: 8080
              name: web
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: session-affinity-service
      namespace: default
    spec:
      ports:
      - name: web
        port: 9093
        targetPort: web
      selector:
        app: session-affinity-test
      sessionAffinity: ClientIP    
    
  5. Try to access the service, or view Kube Proxy logs

Expected Behavior

Kube Proxy should have successfully applied iptables rules and Kubernetes service is accessible.

Some discussion about this issue: https://github.com/kubernetes-sigs/kind/issues/1740.

Yes, it looks like the current WSL2 Kernel is built without xt_recent, needed by iptables -m recent … which kube-proxy uses to implement sessionAffinity: ClientIP. Custom Kernel built with CONFIG_NETFILTER_XT_MATCH_RECENT=y fixed it for me. Submitted https://github.com/microsoft/WSL2-Linux-Kernel/pull/198 (4.19.y) and https://github.com/microsoft/WSL2-Linux-Kernel/pull/199 (5.4.y)

Actual Behavior

Service is inaccessible and any other service created afterwards is inaccessible. Which is caused by inability to apply iptables rules.

The following log is present in Kube Proxy:

E0720 14:29:10.934607       1 proxier.go:1507] Failed to execute iptables-restore: exit status 2 (iptables-restore v1.8.3 (legacy): Couldn't load match `recent':No such file or directory

Error occurred at line: 96
Try `iptables-restore -h' or 'iptables-restore --help' for more information.
)
I0720 14:29:10.934636       1 proxier.go:779] Sync failed; retrying in 30s

Diagnostic Logs

No response

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Reactions: 19
  • Comments: 18 (1 by maintainers)

Most upvoted comments

this issue has caused me nightmares since the whole cluster wasnt able to communicate anymore just because one container has sessionAffinity: ClientIp set.

@craigloewen-msft Any news on this? It’s been half a year. PRs that addressed this issue were opened over at microsoft/WSL2-Linux-Kernel and closed without a good reason.

@jiayali-ms you closed both PRs to the kernel trying to implement this fix (https://github.com/microsoft/WSL2-Linux-Kernel/pull/199 and https://github.com/microsoft/WSL2-Linux-Kernel/pull/198). On your closing comment, you said:

While we understand that it is frustrating to have a PR closed, it is never our intention to ignore feedback or stifle community discussions. We are taking this opportunity to increase transparency on how we work with the WSL community and ensure all members are routed to the appropriate resources.

If you would like contribute to, discuss, or report an issue relating to WSL, or the WSL 2 Linux kernel configuration, please do so at the WSL GitHub. The team is actively monitoring and responding to user feedback there: https://github.com/microsoft/WSL.

We are here, reporting the issue that causes so many k8s environments to be broken in WSL 2 and yet we didn’t hear anything back.

So, what’s up? Is this going to be fixed?

Recompiling the WSL 2 kernel only for this reason is an awful solution.

@therealkenc do you happen to know anything about this issue?

Oh, men, I am trying to provision an RKE2 worker node using Calico on Win11 WSL2 with a bridged network. And the kube-proxy pod suffers from this no xt_recent iptables failure. I am already using a custom kernel (built using native Ubuntu 22.04 and a modified ms kconfig) with many xt options enabled to support wireguard. I wonder why MS is so hostile to the Linux net stack… All they are trying to do is to limit the role of wsl a client behind NAT… besides, I still have several machines not compatible with win11, and need to figure out how to overcome the hyper-v switch nat mode…

MS team - without this fix there’s a whole class of WSL2 use-cases (developer local K8s on WSL2) that are invalidated. Please add this feature in an upcoming build - WSL2 has great potential as a development environment, but needs to be able to run local K8s without issue.

Any news on this? Having to recompile the WSL2 kernel to have a good working environment for k8s is annoying and consumes a good amount of time unless you have a powerful machine. It would be great to include this in the default build.

For future reference:

The following doc article about running Kind on WSL2 explains how to build your kernel to support session affinity: https://kind.sigs.k8s.io/docs/user/using-wsl2/#kubernetes-service-with-session-affinity