ovn: ICMP6 time exceeded in-transit response not sent from OVN router when source is external

We’ve noticed that ICMP6 time exceeded in-transit response packets are not sent from the intermediate OVN logical router hop when an MTR trace is started from an external source address targeted at an IP inside the OVN logical network (all NAT disabled).

If the source address is changed to an IP inside the same subnet as the OVN router’s external port address then the OVN router responds with the expected ICMP6 time exceeded in-transit response.

Example setup:

Container connected to a bridge called “lxdbr1” with an IP of fd42:cfe0:8030:65c4:216:3eff:fe07:96a3/64.

An OVN network behind an OVN logical router, with a provider port connected to another bridge called “lxdbr0”. The OVN logical router port’s IP address on the “lxdbr0” subnet is fd42:6610:36a8:1234:216:3eff:fe72:513b (a different /64 than lxdbr1).

The OVN router also has a static route added to it to route fd0b:bc39:4820:d4f8::/64 to a container connected to a logical switch port. This container has the IP setup of fd0b:bc39:4820:d4f8::1/64.

On the Linux host running the two bridges, there is also a static route of fd0b:bc39:4820:d4f8::1/64 configured to route to the OVN’s external IP of fd42:6610:36a8:1234:216:3eff:fe72:513b.

If I run an MTR trace from the container connected to lxdbr1, with a source address of fd42:cfe0:8030:65c4:216:3eff:fe07:96a3/64 to the OVN container’s address of fd0b:bc39:4820:d4f8::1/64, then we see this result:

                            My traceroute  [v0.93]
c1 (fd42:cfe0:8030:65c4:216:3eff:fe07:96a3)           2020-12-10T12:32:21+0000
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                      Packets               Pings
 Host                               Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. fd42:cfe0:8030:65c4::1           0.0%    11    0.2   0.2   0.1   0.2   0.0
 2. (waiting for reply)
 3. fd0b:bc39:4820:d4f8::1           0.0%    10    0.2   0.3   0.2   1.1   0.3

The OVN router in the intermediate hop is not responding with the ICMP6 time exceeded in-transit response. A tcpdump trace shows two ICMP6 echo requests, one for the OVN router and one of the target IP, but only 1 reply packet coming from the target IP, and no response from the OVN router.

sudo tcpdump -i lxdbr0 -nn
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lxdbr0, link-type EN10MB (Ethernet), capture size 262144 bytes
12:34:16.659989 IP6 fd42:cfe0:8030:65c4:216:3eff:fe07:96a3 > fd0b:bc39:4820:d4f8::1: ICMP6, echo request, seq 33020, length 24
12:34:16.994012 IP6 fd42:cfe0:8030:65c4:216:3eff:fe07:96a3 > fd0b:bc39:4820:d4f8::1: ICMP6, echo request, seq 33021, length 24
12:34:16.994779 IP6 fd0b:bc39:4820:d4f8::1 > fd42:cfe0:8030:65c4:216:3eff:fe07:96a3: ICMP6, echo reply, seq 33021, length 24
12:34:17.662156 IP6 fd42:cfe0:8030:65c4:216:3eff:fe07:96a3 > fd0b:bc39:4820:d4f8::1: ICMP6, echo request, seq 33023, length 24
12:34:17.996195 IP6 fd42:cfe0:8030:65c4:216:3eff:fe07:96a3 > fd0b:bc39:4820:d4f8::1: ICMP6, echo request, seq 33024, length 24
12:34:17.996258 IP6 fd0b:bc39:4820:d4f8::1 > fd42:cfe0:8030:65c4:216:3eff:fe07:96a3: ICMP6, echo reply, seq 33024, length 24
12:34:18.664205 IP6 fd42:cfe0:8030:65c4:216:3eff:fe07:96a3 > fd0b:bc39:4820:d4f8::1: ICMP6, echo request, seq 33026, length 24
12:34:18.998078 IP6 fd42:cfe0:8030:65c4:216:3eff:fe07:96a3 > fd0b:bc39:4820:d4f8::1: ICMP6, echo request, seq 33027, length 24
12:34:18.998118 IP6 fd0b:bc39:4820:d4f8::1 > fd42:cfe0:8030:65c4:216:3eff:fe07:96a3: ICMP6, echo reply, seq 33027, length 24

However if I then setup an IPv6 NAT rule on lxdbr1 to translate the source to the host’s IP on lxdbr0 (fd42:6610:36a8:1234::1) then we start seeing the expected responses:

                                     My traceroute  [v0.93]
c1 (fd42:cfe0:8030:65c4:216:3eff:fe07:96a3)                               2020-12-10T12:35:34+0000
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                                          Packets               Pings
 Host                                                   Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. fd42:cfe0:8030:65c4::1                               0.0%    17    0.1   0.2   0.1   0.2   0.0
 2. fd42:6610:36a8:1234:216:3eff:fe72:513b               0.0%    17    1.1   1.0   0.4   1.5   0.3
 3. fd0b:bc39:4820:d4f8::1                               0.0%    16    0.2   0.2   0.1   1.1   0.2

And a tcpdump shows the expected replies:

sudo tcpdump -i lxdbr0 -nn
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lxdbr0, link-type EN10MB (Ethernet), capture size 262144 bytes
12:35:52.782198 IP6 fd42:6610:36a8:1234::1 > fd0b:bc39:4820:d4f8::1: ICMP6, echo request, seq 33104, length 24
12:35:52.783417 IP6 fd42:6610:36a8:1234:216:3eff:fe72:513b > fd42:6610:36a8:1234::1: ICMP6, time exceeded in-transit for fd0b:bc39:4820:d4f8::1, length 72
12:35:53.116093 IP6 fd42:6610:36a8:1234::1 > fd0b:bc39:4820:d4f8::1: ICMP6, echo request, seq 33105, length 24
12:35:53.116787 IP6 fd0b:bc39:4820:d4f8::1 > fd42:6610:36a8:1234::1: ICMP6, echo reply, seq 33105, length 24
12:35:53.784050 IP6 fd42:6610:36a8:1234::1 > fd0b:bc39:4820:d4f8::1: ICMP6, echo request, seq 33107, length 24
12:35:53.785148 IP6 fd42:6610:36a8:1234:216:3eff:fe72:513b > fd42:6610:36a8:1234::1: ICMP6, time exceeded in-transit for fd0b:bc39:4820:d4f8::1, length 72
12:35:54.117792 IP6 fd42:6610:36a8:1234::1 > fd0b:bc39:4820:d4f8::1: ICMP6, echo request, seq 33108, length 24
12:35:54.117833 IP6 fd0b:bc39:4820:d4f8::1 > fd42:6610:36a8:1234::1: ICMP6, echo reply, seq 33108, length 24

So we were wondering is this a bug, or is this expected behaviour?

About this issue

Original URL
State: open
Created 3 years ago
Comments: 15 (6 by maintainers)

Commits related to this issue

ovn-nb: Document LRP gateway_mtu option The Logical Router Port gateway_mtu option is currently only documented together with implementation details in the ovn-northd documentation. As this is CMS A... — committed to ovsrobot/ovn by fnordahl 3 years ago
ovn-nb: Document LRP gateway_mtu option The Logical Router Port gateway_mtu option is currently only documented together with implementation details in the ovn-northd documentation. As this is CMS A... — committed to ovsrobot/ovn by fnordahl 3 years ago
ovn-nb: Document LRP gateway_mtu option The Logical Router Port gateway_mtu option is currently only documented together with implementation details in the ovn-northd documentation. As this is CMS A... — committed to ovn-org/ovn by fnordahl 3 years ago

Most upvoted comments

We have had other parties trip over the missing PMTU from external network as noted in https://bugs.launchpad.net/ubuntu/+source/ovn/+bug/1949120, so an SRU is surely something for the OVN team in Canonical to consider. It is non-trivial though as some code re-organization took place leading up to 21.09, but we’ll have a look.

fnordahl on Nov 9, 2021

@tomponline that’s actually a question for @fnordahl as he and his team maintain the focal packages.

In my case, I maintain OVS/OVN backports in my production PPA for my own clusters so I’ll make sure I get on 21.09 there soon. The snap itself currently doesn’t ship 21.09 as it wouldn’t build for some reason, but I’ll sort that out soon. It’s also not really relevant to the snap as we only use the OVN client tools from there whereas we really need the server side updated in this case.

stgraber on Nov 9, 2021

Ack, thanks for checking @tomponline. At least we got the PMTU part of the problem out of the way! I’ll have a look and see what could be done about the other part.

fnordahl on Nov 9, 2021

I would indeed consider this as CMS API and as such I think it should be mentioned in the ovn-nb documentation.

fnordahl on Nov 9, 2021