quic-go: GSO fails on some platforms / interfaces

I’m seeing a lot of INTERNAL_ERROR (local): write udp [::]:443->[REDACTED]:42830: sendmsg: invalid argument on the server side, causing the connection to be disconnected. They only happen when the clients are aarch64 (raspberry pi).

Could this be related to the GSO change?

About this issue

Original URL
State: closed
Created a year ago
Comments: 71 (55 by maintainers)

Commits related to this issue

chore: update quic-go to 0.36.0 — committed to MetaCubeX/mihomo by wwqgtxx a year ago

Most upvoted comments

I’ve created https://github.com/quic-go/quic-go/pull/3934, which flips the environment variable around. With this PR, GSO would (for now) be disabled by default, until we can figure out a good solution for this issue. We could release this as v0.36.1. Any thoughts on that?

marten-seemann on Jun 30, 2023

@marten-seemann do you just need SSH access to a raspberry pi? Send me an email (address on profile) or we can chat over Signal to coordinate

zllovesuki on Jul 30, 2023

Thanks for the ping. We’ll hold the Caddy 2.7 stable release until a patch is ready. We’re fine with either a fix or disabling it.

mholt on Jun 23, 2023

GSO would have been my first guess as well. I suspect that for some reason we manage to enable GSO, but don’t correctly recognize that it was successfully enabled. This then causes the failure you’re seeing because once enabled, the kernel expects the segment length message for the sendmsg to be set: https://github.com/quic-go/quic-go/blob/21388c86bb2be0de9b5468d8b77f37281b64fcf0/sys_conn_df_linux.go#L70-L83

Can you try manually disabling GSO via the QUIC_GO_DISABLE_GSO environment variable? https://github.com/quic-go/quic-go/blob/21388c86bb2be0de9b5468d8b77f37281b64fcf0/sys_conn_df_linux.go#L46-L50 This should make the failure go away.

If it’s actually due to GSO, can you confirm if the syscall to enable it is successful in https://github.com/quic-go/quic-go/blob/21388c86bb2be0de9b5468d8b77f37281b64fcf0/sys_conn_df_linux.go#L52-L57

If my suspicion is correct, this would mean that we can’t rely on the return value of the syscall that enables GSO. Maybe we need to instead read the value back using GetSockoptInt in maybeSetGSO?

marten-seemann on Jun 23, 2023