calico: Windows Calico: denied traffic from other namespaces, but calls to NodePort service are not blocked

Steps to reproduce the issue:

Using latest install-calico-windows.ps1 script: calico-windows-v3.16.1.zip Windows Node is 1909 Server Core Cluster created with eksctl, one linux node and one Windows 1909 Server Core node

Services in green namespace:

PS U:\cont\customer> kubectl.exe get svc -n green
NAME        TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                       AGE
ae          ClusterIP   None             <none>        80/TCP                        108m
ae-tomcat   ClusterIP   None             <none>        80/TCP,8085/TCP,6009/TCP      6h32m
nginx       NodePort    172.20.247.197   <none>        80:30070/TCP,8085:30071/TCP   6h32m
slm         ClusterIP   None             <none>        27000/TCP                     6h32m
tomcat-1    ClusterIP   None             <none>        8085/TCP,6009/TCP             108m
tomcat-2    ClusterIP   None             <none>        8085/TCP                      108m

Network policy:

PS U:\cont\customer> kubectl.exe get networkpolicy -n green -o yaml
apiVersion: v1
items:
- apiVersion: networking.k8s.io/v1
  kind: NetworkPolicy
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"networking.k8s.io/v1","kind":"NetworkPolicy","metadata":{"annotations":{},"name":"deny-traffic-other-namespaces","namespace":"green"},"spec":{"ingress":[{"from":[{"podSelector":{}}]}],"podSelector":{"matchLabels":null}}}
    creationTimestamp: "2020-10-09T18:01:45Z"
    generation: 1
    name: deny-traffic-other-namespaces
    namespace: green
    resourceVersion: "41221"
    selfLink: /apis/networking.k8s.io/v1/namespaces/green/networkpolicies/deny-traffic-other-namespaces
    uid: bda673b0-3d59-4145-85ed-db07f1077fd3
  spec:
    ingress:
    - from:
      - podSelector: {}
    podSelector: {}
    policyTypes:
    - Ingress
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

Exec into green2 nginx container, and tried to reach nginx and ae containers in green namespace. The call to nginx works, but I expected this to be blocked like the call to the ae container.

PS C:\nginx\nginx-1.14.0> ipconfig.exe

Windows IP Configuration


Ethernet adapter vEthernet (cid-f242b750afde6e02008f7ec569855cdd3e3df03124a9d4d123da1130e1af21ba):

   Connection-specific DNS Suffix  . : green2.svc.cluster.local
   Link-local IPv6 Address . . . . . : fe80::d1df:6a63:e76a:3674%36
   IPv4 Address. . . . . . . . . . . : 10.12.40.199
   Subnet Mask . . . . . . . . . . . : 255.255.255.128
   Default Gateway . . . . . . . . . : 10.12.40.129
PS C:\nginx\nginx-1.14.0> Resolve-DnsName ae.green.svc.cluster.local

Name                                           Type   TTL   Section    IPAddress
----                                           ----   ---   -------    ---------
ae.green.svc.cluster.local                     A      5     Answer     10.12.40.115


PS C:\nginx\nginx-1.14.0> Resolve-DnsName nginx.green.svc.cluster.local

Name                                           Type   TTL   Section    IPAddress
----                                           ----   ---   -------    ---------
nginx.green.svc.cluster.local                  A      5     Answer     172.20.247.197


PS C:\nginx\nginx-1.14.0> curl nginx.green.svc.cluster.local -UseBasicParsing


StatusCode        : 200
StatusDescription : OK
Content           : <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
                    <html xmlns="http://www.w3.org/1999/xhtml">
                    <head>
                    <meta http-equiv="Content-Type" cont...
RawContent        : HTTP/1.1 200 OK
                    Connection: close
                    Strict-Transport-Security: max-age=31536000; includeSubDomains
                    X-Frame-Options: SAMEORIGIN
                    X-Content-Type-Options: nosniff
                    X-XSS-Protection: 1; mode=block
                    Host:...
Forms             :
Headers           : {[Connection, close], [Strict-Transport-Security, max-age=31536000; includeSubDomains], [X-Frame-Options, SAMEORIGIN], [X-Content-Type-Options, nosniff]...}
Images            : {@{outerHTML=<img src="iisstart.png" alt="IIS" width="960" height="600" />; tagName=IMG; src=iisstart.png; alt=IIS; width=960; height=600}}
InputFields       : {}
Links             : {@{outerHTML=<a href="http://go.microsoft.com/fwlink/?linkid=66138&amp;clcid=0x409"><img src="iisstart.png" alt="IIS" width="960" height="600" /></a>; tagName=A;
                    href=http://go.microsoft.com/fwlink/?linkid=66138&amp;clcid=0x409}}
ParsedHtml        :
RawContentLength  : 731



PS C:\nginx\nginx-1.14.0> curl ae.green.svc.cluster.local -UseBasicParsing -TimeoutSec 15
curl : The operation has timed out.
At line:1 char:1
+ curl ae.green.svc.cluster.local -UseBasicParsing -TimeoutSec 15
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

Is my policy set up correctly? Why does the call to the nginx service not get blocked?

Policy file:

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  namespace: green
  name: deny-traffic-other-namespaces
spec:
  podSelector:
    matchLabels:
  ingress:
  - from:
    - podSelector: {}

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 23 (11 by maintainers)

Most upvoted comments

@song-jiang Everything is working well for me now. I wanted to conclude with suggestions for others using EKS with Calico on Windows Nodes.

I followed this page for setup instructions on EKS: https://docs.projectcalico.org/getting-started/windows-calico/quickstart

Recommend adding more to this page to reflect the additional steps needed. If some of these steps are moved to the install-calico-windows.ps1 script, then of course they would not need to be called out:

Steps to enable the WinDSR feature on kube-proxy.
Possibly replace domain suffix of $env:NODENAME with internal domain name in C:\CalicoWindows\config.ps1 (see https://github.com/projectcalico/calico/issues/4040

Here’s the bootstrap script that I am running on each Windows node for Calico. If anyone uses this, please replace the version of kubectl that you are using, and replace ServiceCidr, DNSServerIPs, $domain_suffix and $node_suffix as needed. Note that the node suffix replacement may or may not be needed for everyone:

$path = Get-WmiObject -Query 'select * from win32_service where name="kube-proxy"' | Select -ExpandProperty pathname
$updatedPath = $path + " --enable-dsr=true --feature-gates=WinDSR=true"
Get-WmiObject win32_service -filter 'Name="kube-proxy"' | Invoke-WmiMethod -Name Change -ArgumentList @($null,$null,$null,$null,$null,$updatedPath)
Restart-Service -name kube-proxy
mkdir c:\k;cd c:\k
curl -o kubectl.exe https://amazon-eks.s3.us-west-2.amazonaws.com/1.18.9/2020-11-02/bin/windows/amd64/kubectl.exe
Invoke-WebRequest https://docs.projectcalico.org/scripts/install-calico-windows.ps1 -OutFile c:\install-calico-windows.ps1
c:\install-calico-windows.ps1 -ServiceCidr 172.20.0.0/16 -DNSServerIPs 172.20.0.10
& C:\CalicoWindows\stop-calico.ps1
$config_file = "C:\CalicoWindows\config.ps1"
$domain_suffix = "usw23.saas.int"
$node_suffix = "us-west-2.compute.internal"
(Get-Content $config_file) -replace $domain_suffix, $node_suffix | Set-Content $config_file -Force
& C:\CalicoWindows\start-calico.ps1

dave-meier on Jan 14, 2021