kubernetes: unidirectional UDP traffic to a NodePort service is not handled properly

@freehan - why did you close if it wasn’t fixed?

I just faced this issue and what worked for me was running sudo iptables -P FORWARD ACCEPT on all cluster nodes.

An unrelated comment: Kubeproxy and how it works by default which is using iptables is very problematic and this was the third time that I hit a problem related to kubeproxy and iptables and all of these 3 times it was a nightmare to pinpoint the real problem and even harder to find it on K8s github issues because of all of these stupid bots that are bombarding and closing issues without resolving them. This is exactly like sweeping the trash/problems under the carpet, please stop closing the issues without solving them.

We close issues if there is no response for 1 month.

open if needed

FWIW, we’re seeing the same issue when sending traces to Jaeger.

@avarf - I am sorry you are having a bad time with it. I want to help, but I’m not sure which issue you’re hitting at this point.

Sadly, there are a lot of different things that COULD be going wrong. I just ran my own attempt at a repro of this and I failed to repro the reported problem. It seems that either ctstate NEW covers “UNREPLIED” or ctstate ESTABLISHED covers it. Either way, I do not see this problem being reproduced.

Let’s take it apart a bit.

if you have no network-policy agent (e.g. calico) { 
  if you have default policy DROP {
    you need a rule or rules somewhere that allows services to work (e.g. `-p udp -j ALLOW`)
  } else /* default policy ALLOW */ {
    you don't need anything
  }
} else /* network policy enabled, I'll assume calico for this */ {
  if you have a network policy defined to allow this flow {
    policy should allow NEW connections and ESTABLISHED
  } else /* no policy defined */ {
    if you have default policy DROP {
      you need a rule or rules somewhere that allows services to work (e.g. `-p udp -j ALLOW`)
    } else /* default policy ALLOW */ {
      you don't need anything
    }
  }
}

My (admittedly manual) testing agrees with this. Here’s what I did.

• Create a UDP server and watch it’s output (literally nc -l -u -p <port>)
• Clear UDP conntrack (conntrack -D -p udp)
• Remove the default allow (iptables -t filter -D FORWARD -p udp -j ACCEPT)
• Wipe iptables -t filter -F cali-FORWARD
• UDP client sends data
• Confirm no packets no conntrack
• Add an iptables rule to allow NEW (iptables -t filter -A FORWARD -p udp -m conntrack --ctstate NEW -j ACCEPT)
• Re-run client
• Confirm packets flow, even over the course of tens of seconds
• Confirm a conntrack record which is “UNREPLIED”

So please tell me which problem we are facing for you? Even better, let’s open a NEW issue, without the baggage of this one. I need to see a full iptables-save as well as complete details about what’s happening.

@skbly7: This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed by commenting with the /remove-help command.

/assign @skbly7

The docs for NEW say “NEW The packet has started a new connection or otherwise associated with a connection which has not seen packets in both directions.” Sounds like that should match those packets. Has anyone tried adding it to the list of accepted states?