kubernetes: kube-proxy modes ipvs and userspace do not serve requests to localhost, while mode iptables does

What happened?

I create

apiVersion: v1
kind: Service
spec:
  type: NodePort
  selector:
    app: karana
  ports:
    - protocol: TCP
      port: 3890
      nodePort: 123

My node has IP 2.3.4.5. When I run kube-proxy in mode usermode or ipvs, requests for 2.3.4.5:123 are answered, while requests for localhost:123 are not answered.

When I run kube-proxy in mode:iptables, requests for 2.3.4.5:123 and localhost:123 are answered.

When I use ipvs-mode, the difference of ipvsadm -L before and after the service is utilized, is:

@@ -1,6 +1,12 @@
 IP Virtual Server version 1.2.1 (size=4096)
 Prot LocalAddress:Port Scheduler Flags
   -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
+TCP  d:32389 rr
+  -> 10.85.0.12:ndsconnect        Masq    1      0          1         
+TCP  d:32389 rr
+  -> 10.85.0.12:ndsconnect        Masq    1      0          0         
+TCP  d:32389 rr
+  -> 10.85.0.12:ndsconnect        Masq    1      0          0         
 TCP  d:https rr
   -> d:sun-sr-https               Masq    1      2          0         
 TCP  d:domain rr
@@ -9,6 +15,8 @@
 TCP  d:9153 rr
   -> 10.85.0.2:9153               Masq    1      0          0         
   -> 10.85.0.3:9153               Masq    1      0          0         
+TCP  d:ndsconnect rr
+  -> 10.85.0.12:ndsconnect        Masq    1      0          0         
 UDP  d:domain rr
   -> 10.85.0.2:domain             Masq    1      0          0         
   -> 10.85.0.3:domain             Masq    1      0          0         

Where d is the content of /etc/hostname.

For the record, creating a Pod with spec.containers[0].ports[0].hostPort: 3890 does answer requests for 2.3.4.5:3890, but not for localhost:3890, see https://github.com/containernetworking/plugins/issues/763 .

• When Pod.spec.containers.ports.hostPort or Service.spec.ports[0].nodePort is used, Kubernetes shall ensure that requests for localhost:port are served, in the same way as requests for nodeIP:port are answered. This works currenlty only for Service.NodePort, when kube-proxy is in iptables mode, not usermode, not ipvs.

What did you expect to happen?

See above.

How can we reproduce it (as minimally and precisely as possible)?

See above.

Anything else we need to know?

No response

Kubernetes version

$ kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"24+", GitVersion:"v1.24.4-rc.0.14+3dc15e1cdde539", GitCommit:"3dc15e1cdde539e62c75342d516eacb2a9042ce1", GitTreeState:"clean", BuildDate:"2022-08-05T06:17:42Z", GoVersion:"go1.18.4", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.3", GitCommit:"aef86a93758dc3cb2c658dd9657ab4ad4afc21cb", GitTreeState:"clean", BuildDate:"2022-07-13T14:23:26Z", GoVersion:"go1.18.3", Compiler:"gc", Platform:"linux/amd64"}

Cloud provider

OS version

$ cat /etc/os-release
NAME="Fedora Linux"
VERSION="36 (Workstation Edition)"
ID=fedora
VERSION_ID=36
VERSION_CODENAME=""
PLATFORM_ID="platform:f36"
PRETTY_NAME="Fedora Linux 36 (Workstation Edition)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:36"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f36/system-administrators-guide/"
SUPPORT_URL="https://ask.fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=36
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=36
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
VARIANT="Workstation Edition"
VARIANT_ID=workstation

$ uname -a
Linux d 5.18.16-200.fc36.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Aug 3 15:44:49 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Install tools

Container runtime (CRI) and version (if applicable)

$ crio --version
crio version 1.25.0
Version:        1.25.0
GitCommit:      822b4847d3722172e3095014e60c9dcf94655d2c
GitCommitDate:  2022-08-05T01:17:42Z
GitTreeState:   dirty
BuildDate:      2022-08-05T11:17:23Z
GoVersion:      go1.18.4
Compiler:       gc
Platform:       linux/amd64
Linkmode:       dynamic
BuildTags:      
  containers_image_ostree_stub
  exclude_graphdriver_btrfs
  btrfs_noversion
  containers_image_openpgp
  seccomp
  selinux
LDFlags:          -s -w -X github.com/cri-o/cri-o/internal/pkg/criocli.DefaultsPath="" -X github.com/cri-o/cri-o/internal/version.buildDate=2022-08-05T11:17:23Z 
SeccompEnabled:   true
AppArmorEnabled:  false
Dependencies:     

Related plugins (CNI, CSI, …) and versions (if applicable)

No response