msquic: Cannot connect via IPv4 when listener binds on IPv6

Describe the bug

Discovered when investigating https://github.com/dotnet/runtime/issues/67442.

For purposes of an HTTP3 servers, we want to listen to all connections from any IP, which makes the server listen on an IPv6 “AnyAddress” and configure the socket such that it also receives IPv4 traffic (via mapped IPs).

Debugging shows that while MsQuic receives UDP datagrams sent using IPv4 addresses, the datagram will not get matched to any existing listener and the connection will get immediately closed with a TLS alert corresponding to a failed ALPN negotiation.

Affected OS

All
Windows Server 2022
Windows 11
Windows Insider Preview (specify affected build below)
Ubuntu
Debian
Other (specify below)

Additional OS information

Reproduced on Linux, haven’t tried reproducing on Windows.

MsQuic version

main

Steps taken to reproduce bug

Create a server that listens on an “IPv6 Any” address or IPv6 loopback address
Try to connect to the server using its IPv4 Address

The code I used to reproduce can be found at https://github.com/dotnet/runtime/tree/main/src/libraries/System.Net.Http/tests/StressTests/HttpStress, but the setup may be a bit complicated:

Build the .NET runtime (e.g. by running build.sh -s clr+libs+libs.tests -c Release in the repo root)
either install MsQuic globally or put built binaries in $repoRoot/artifacts/bin/testhost/net7.0-Linux-Release-x64/shared/Microsoft.NETCore.App/7.0.0/
run build-local.sh Release Release from the directory above
run both sides of the application:
- server: $repoRoot/artifacts/bin/testhost/net7.0-Linux-Release-x64/dotnet ./bin/Release/net7.0/HttpStress.dll -runMode server -http 3.0 -serverUri https://+:5001
- client: $repoRoot/artifacts/bin/testhost/net7.0-Linux-Release-x64/dotnet ./bin/Release/net7.0/HttpStress.dll -runMode server -http 3.0 -serverUri https://127.0.0.1:5001

Expected behavior

Connection is established without problems

Actual outcome

Connection is closed by the server with a TLS alert 120 (failed ALPN negotiation)

Additional details

No response

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 24 (19 by maintainers)

Most upvoted comments

int port = 5004;

TcpListener listener = new TcpListener(new IPEndPoint(IPAddress.IPv6Any, port));
listener.Start();
Console.WriteLine($"Listening on IPv6Any:{port}");

using TcpClient clientIpv4 = new TcpClient();
var ipv4connect = clientIpv4.ConnectAsync(new IPEndPoint(IPAddress.Loopback, port)).ContinueWith(x =>
{
    Console.WriteLine("Connected via Ipv4 Loopback");
});

using TcpClient clientIpv6 = new TcpClient();
var ipv6connect = clientIpv6.ConnectAsync(new IPEndPoint(IPAddress.IPv6Loopback, port)).ContinueWith(x =>
{
    Console.WriteLine("Connected via Ipv6 Loopback");
});

await Task.WhenAll(ipv4connect, ipv6connect);

This code swallows the Task Exceptions.

If you change it to:

using System.Net;
using System.Net.Sockets;

int port = 5004;

TcpListener listener = new TcpListener(new IPEndPoint(IPAddress.IPv6Any, port));
listener.Start();
Console.WriteLine($"Listening on IPv6Any:{port}");

Console.WriteLine("Connecting via IPv6 Loopback:");
using TcpClient clientIpv6 = new TcpClient();
await clientIpv6.ConnectAsync(new IPEndPoint(IPAddress.IPv6Loopback, port));
Console.WriteLine("Connected via IPv6 Loopback");

Console.WriteLine("Connecting via IPv4 Loopback:");
using TcpClient clientIpv4 = new TcpClient();
await clientIpv4.ConnectAsync(new IPEndPoint(IPAddress.Loopback, port));
Console.WriteLine("Connected via IPv4 Loopback");

On Linux this gives:

Listening on IPv6Any:5004
Connecting via IPv6 Loopback:
Connected via IPv6 Loopback
Connecting via IPv4 Loopback:
Unhandled exception. System.Net.Sockets.SocketException (111): Connection refused
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
   at System.Threading.Tasks.ValueTask.ValueTaskSourceAsTask.<>c.<.cctor>b__4_0(Object state)
--- End of stack trace from previous location ---
   at System.Net.Sockets.TcpClient.CompleteConnectAsync(Task task)
   at Program.<Main>$(String[] args) in /tmp/console/Program.cs:line 17
   at Program.<Main>(String[] args)

tmds on May 19, 2022

when I listen on IPv6Any, I can connect both using IPv4 and IPv6 loopback addresses with TCP

Did you verify this on Windows as well?

cc @maolson-msft would you mind weighing in here, at least in respect to Windows TCP behavior. How does Windows TCP handle dual-stack listeners?

But regardless, the current behavior is by design for MsQuic, and it would be a breaking change to modify it, which will not unless there is an extremely strong reason to do so. Matching a TCP behavior isn’t enough.

nibanks on May 11, 2022