nebula: Nodes not able to ping each other

I have a lighthouse in a cloud VM with public IP address; laptop in a home VLAN and workstation in another VLAN.

lighthouse 10.13.1.1 workstation 10.13.1.3 (LAN IP 10.12.1.6) laptop 10.13.1.4 (LAN IP 10.12.2.221)

All machines are running the latest version of nebula and flavours of Linux. My configurations are minimal and are as follows:

My lighthouse config:

pki:
  ca: /opt/nebula/ca.crt
  cert: /opt/nebula/lighthouse.crt
  key: /opt/nebula/lighthouse.key
static_host_map:
  "10.13.1.1": ["vm_public_IP:4242"]
lighthouse:
  am_lighthouse: true
  interval: 60
listen:
  host: "[::]"
  port: 4242
punchy:
  punch: true
  respond: true
cipher: aes
tun:
  disabled: false
  dev: nebula1
  drop_local_broadcast: false
  drop_multicast: false
  tx_queue: 500
  mtu: 1300
  routes:
  unsafe_routes:
logging:
  level: info
  format: text
firewall:
  conntrack:
    tcp_timeout: 12m
    udp_timeout: 3m
    default_timeout: 10m
    max_connections: 100000
  outbound:
    - port: any
      proto: any
      host: any
  inbound:
    - port: any
      proto: any
      host: any

My workstation and laptop config: (the only different part is cert and key files)

pki:
  ca: /home/ewon/nebula/ca.crt
  cert: /home/ewon/nebula/workstation.crt
  key: /home/ewon/nebula/workstation.key
static_host_map:
  "10.13.1.1": ["vm_public_IP:4242"]
lighthouse:
  am_lighthouse: false
  interval: 60
  hosts:
    - "10.13.1.1"
listen:
  host: "[::]"
  port: 0
punchy:
  punch: true
  respond: true
cipher: aes
tun:
  disabled: false
  dev: nebula1
  drop_local_broadcast: false
  drop_multicast: false
  tx_queue: 500
  mtu: 1300
  routes:
  unsafe_routes:
logging:
  level: info
  format: text
firewall:
  conntrack:
    tcp_timeout: 12m
    udp_timeout: 3m
    default_timeout: 10m
    max_connections: 100000
  outbound:
    - port: any
      proto: any
      host: any
  inbound:
    - port: any
      proto: any
      host: any

I can ping from nodes to lighthouse and vice versa. However, nodes cannot ping each other. If I ping from laptop to workstation, I get the following error messages on laptop:

...
ERRO[0018] Prevented a pending handshake race            certName=workstation fingerprint=5c0f3921e4fc49fc06b34fd2cc58a3242bfb69bde35728ef0219d466fcf0bb2c handshake="map[stage:1 style:ix_psk0]" initiatorIndex=526444663 issuer=b197e555563b8a4b5370b16b18dcc3ff4068caf3ec19818e86ff017ea1260845 remoteIndex=0 responderIndex=3029730967 udpAddr="10.12.1.6:45286" vpnIp=10.13.1.3
INFO[0019] Handshake timed out                           durationNs=8720123570 handshake="map[stage:1 style:ix_psk0]" initiatorIndex=250451344 remoteIndex=0 udpAddrs="[home_public_IP:32907 10.12.1.6:45286]" vpnIp=10.13.1.3
INFO[0020] Handshake message received                    certName=workstation fingerprint=5c0f3921e4fc49fc06b34fd2cc58a3242bfb69bde35728ef0219d466fcf0bb2c handshake="map[stage:1 style:ix_psk0]" initiatorIndex=526444663 issuer=b197e555563b8a4b5370b16b18dcc3ff4068caf3ec19818e86ff017ea1260845 remoteIndex=0 responderIndex=0 udpAddr="10.12.1.6:45286" vpnIp=10.13.1.3
INFO[0020] Handshake message sent                        certName=workstation fingerprint=5c0f3921e4fc49fc06b34fd2cc58a3242bfb69bde35728ef0219d466fcf0bb2c handshake="map[stage:2 style:ix_psk0]" initiatorIndex=526444663 issuer=b197e555563b8a4b5370b16b18dcc3ff4068caf3ec19818e86ff017ea1260845 remoteIndex=0 responderIndex=286921894 sentCachedPackets=0 udpAddr="10.12.1.6:45286" vpnIp=10.13.1.3
...                                                              

and on my workstation I have the following “info” messages:

...
INFO[0018] Handshake message sent                        handshake="map[stage:1 style:ix_psk0]" initiatorIndex=526444663 udpAddrs="[home_public_IP:12583 10.12.2.210:59517 10.12.2.211:59517 192.168.122.1:59517]" vpnIp=10.13.1.4
INFO[0020] Handshake timed out                           durationNs=10182400905 handshake="map[stage:1 style:ix_psk0]" initiatorIndex=526444663 remoteIndex=0 udpAddrs="[home_public_IP:12583 10.12.2.210:59517 10.12.2.211:59517 192.168.122.1:59517]" vpnIp=10.13.1.4
...

The logs on lighthouse is only the following:

root@vm:/opt/nebula# ./nebula -config lighthouse.config.yml
INFO[0000] Firewall rule added                           firewallRule="map[caName: caSha: direction:outgoing endPort:0 groups:[] host:any ip: proto:0 startPort:0]"
INFO[0000] Firewall rule added                           firewallRule="map[caName: caSha: direction:incoming endPort:0 groups:[] host:any ip: proto:0 startPort:0]"
INFO[0000] Firewall started                              firewallHash=21716b47a7a140e448077fe66c31b4b42f232e996818d7dd1c6c4991e066dbdb
INFO[0000] Main HostMap created                          network=10.13.1.1/24 preferredRanges="[]"
INFO[0000] UDP hole punching enabled
INFO[0000] Nebula interface is active                    build=1.5.2 interface=nebula1 network=10.13.1.1/24 udpAddr="[::]:4242"
INFO[0006] Handshake message received                    certName=laptop fingerprint=ea008f0243fbeb44254732ec24fd35fa729f1da67920f5c705ef73dced83a5b8 handshake="map[stage:1 style:ix_psk0]" initiatorIndex=392725562 issuer=b197e555563b8a4b5370b16b18dcc3ff4068caf3ec19818e86ff017ea1260845 remoteIndex=0 responderIndex=0 udpAddr="vm_public_IP:12583" vpnIp=10.13.1.4
INFO[0006] Handshake message sent                        certName=laptop fingerprint=ea008f0243fbeb44254732ec24fd35fa729f1da67920f5c705ef73dced83a5b8 handshake="map[stage:2 style:ix_psk0]" initiatorIndex=392725562 issuer=b197e555563b8a4b5370b16b18dcc3ff4068caf3ec19818e86ff017ea1260845 remoteIndex=0 responderIndex=2000406661 sentCachedPackets=0 udpAddr="vm_public_IP:12583" vpnIp=10.13.1.4
INFO[0011] Handshake message received                    certName=workstation fingerprint=5c0f3921e4fc49fc06b34fd2cc58a3242bfb69bde35728ef0219d466fcf0bb2c handshake="map[stage:1 style:ix_psk0]" initiatorIndex=792326294 issuer=b197e555563b8a4b5370b16b18dcc3ff4068caf3ec19818e86ff017ea1260845 remoteIndex=0 responderIndex=0 udpAddr="vm_public_IP:32907" vpnIp=10.13.1.3
INFO[0011] Handshake message sent                        certName=workstation fingerprint=5c0f3921e4fc49fc06b34fd2cc58a3242bfb69bde35728ef0219d466fcf0bb2c handshake="map[stage:2 style:ix_psk0]" initiatorIndex=792326294 issuer=b197e555563b8a4b5370b16b18dcc3ff4068caf3ec19818e86ff017ea1260845 remoteIndex=0 responderIndex=154643726 sentCachedPackets=0 udpAddr="vm_public_IP:32907" vpnIp=10.13.1.3

So it seems nodes can receive the requests from each other, but why ping (and ssh) won’t work? I made sure that workstation and laptop has no firewall rules in place. There’s no YAML syntax error either.