gluetun: Bug: Kubernetes services cannot be resolved due to DNS overrides and routing conflicts

TLDR: Kubernetes services cannot be resolved anymore because the DNS configuration is being overwritten

1. Is this urgent?

   • Yes
   • No

2. What VPN service provider are you using?

   • PIA
   • Mullvad
   • Windscribe
   • Surfshark
   • Cyberghost

3. What’s the version of the program?

Running version latest built on 2020-07-09T11:57:17Z (commit dc1c7ea)

1. What are you using to run the container?

   • Docker run
   • Docker Compose
   • Kubernetes
   • Docker stack
   • Docker swarm
   • Podman
   • Other:

2. Extra information

Logs:

DNS over TLS settings:
 |--DNS over TLS provider:
  |--cloudflare
 |--Caching: disabled
 |--Block malicious: disabled
 |--Block surveillance: disabled
 |--Block ads: disabled

...

2020-07-09T13:22:42.163Z        INFO    firewall configurator: accepting any input traffic on port 8888
2020-07-09T13:22:42.163Z        INFO    http server: listening on 0.0.0.0:8000
2020-07-09T13:22:42.163Z        INFO    dns configurator: using DNS address 1.1.1.1 internally
2020-07-09T13:22:42.163Z        INFO    dns configurator: using DNS address 1.1.1.1 system wide
2020-07-09T13:22:42.163Z        INFO    openvpn configurator: writing auth file /etc/openvpn/auth.conf
2020-07-09T13:22:42.164Z        INFO    openvpn configurator: starting openvpn
2020-07-09T13:22:42.166Z        INFO    openvpn: OpenVPN 2.4.9 x86_64-alpine-linux-musl [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD] built on Apr 20 2020
2020-07-09T13:22:42.166Z        INFO    openvpn: library versions: OpenSSL 1.1.1g  21 Apr 2020, LZO 2.10
2020-07-09T13:22:42.167Z        INFO    tinyproxy configurator: starting tinyproxy server
2020-07-09T13:22:42.168Z        INFO    openvpn: WARNING: you are using user/group/chroot/setcon without persist-tun -- this may cause restarts to fail
2020-07-09T13:22:42.170Z        INFO    openvpn: TCP/UDP: Preserving recently used remote address: [AF_INET]81.19.209.124:1194
2020-07-09T13:22:42.170Z        INFO    openvpn: UDP link local: (not bound)
2020-07-09T13:22:42.170Z        INFO    openvpn: UDP link remote: [AF_INET]81.19.209.124:1194
2020-07-09T13:22:42.170Z        INFO    openvpn: NOTE: UID/GID downgrade will be delayed because of --client, --pull, or --up-delay
2020-07-09T13:22:42.182Z        INFO    openvpn: WARNING: 'link-mtu' is used inconsistently, local='link-mtu 1633', remote='link-mtu 1581'
2020-07-09T13:22:42.182Z        INFO    openvpn: WARNING: 'cipher' is used inconsistently, local='cipher AES-256-CBC', remote='cipher AES-256-GCM'
2020-07-09T13:22:42.182Z        INFO    openvpn: WARNING: 'auth' is used inconsistently, local='auth SHA512', remote='auth [null-digest]'
2020-07-09T13:22:42.182Z        INFO    openvpn: [nl-ams-v024.prod.surfshark.com] Peer Connection Initiated with [AF_INET]81.19.209.124:1194

Configuration file:

apiVersion: v1                                
kind: Pod                                     
metadata:                                     
  name: vpn-test
  namespace: default                          
spec:                                         
  containers:                                 
  - name: shell
    image: ubuntu
    command: ['bash']                         
    stdin: true                               
    tty: true                                 
  - name: proxy                               
    env:                                      
      - name: USER                            
        value: ...
      - name: PASSWORD                        
        value: ...
      - name: VPNSP                           
        value: 'surfshark'                    
      - name: FIREWALL                        
        value: 'off'                           
      - name: EXTRA_SUBNETS                   
        value: '10.192.0.0/9'
      - name: SHADOWSOCKS                     
        value: 'on'                          
      - name: TINYPROXY                       
        value: 'on'                           
      - name: DOT                             
        value: 'on'                           
      - name: DOT_CACHING                     
        value: 'off'                          
      - name: BLOCK_MALICIOUS                 
        value: 'off'                          
      - name: DNS_UPDATE_PERIOD               
        value: '0'                            
    image: qmcgaw/private-internet-access     
    imagePullPolicy: Always                   
    ports:                                    
      - containerPort: 8888                   
      - containerPort: 8388                   
      - containerPort: 8388                   
        protocol: UDP                         
      - containerPort: 8000                   
    securityContext:                          
      privileged: true                        
      capabilities:                           
        add:                                  
          - NET_ADMIN                         

Host OS: DigitalOcean Kubernetes cluster

I believe that svc.cluster.local should be added to the search parameter in /etc/resolv.conf and that unbound needs to use the internal k8s dns server to resolve those local domainnames.

Running in a normal pod:

root@shell:/# nslookup kube-dns.kube-system
Server:         10.245.0.10
Address:        10.245.0.10#53

Name:   kube-dns.kube-system.svc.cluster.local
Address: 10.245.0.10

Running with the VPN sidecar:

root@shell:/# nslookup kube-dns.kube-system
;; Got SERVFAIL reply from 127.0.0.1, trying next server
;; Got SERVFAIL reply from 127.0.0.1, trying next server
Server:         127.0.0.1
Address:        127.0.0.1#53

** server can't find kube-dns.kube-system.default.svc.cluster.local: SERVFAIL

Running from the VPN sidecar:

/ # nslookup kube-dns.kube-system
Server:         127.0.0.1
Address:        127.0.0.1:53

** server can't find kube-dns.kube-system: SERVFAIL

** server can't find kube-dns.kube-system: SERVFAIL