utp: Lots of duplicate packets are being send

I am trying to hunt down an issue I have with this library where I notice that the uTP transfer does not back-off when there is TCP background traffic.

Tracing this issue I noticed that an extensive number of packets are being resend. I noticed this was primarily due to this call to resend which is caused by this call to ackSkipped. Using wireshark I noticed that lots of ACK packets do have the Selective ACKs extension enabled but do not have an extension bitmask:

screen shot 2017-05-30 at 15 14 07

According to the specification the length should be:

Note that the len field of extensions refer to bytes, which in this extension must be at least 4, and in multiples of 4.

There are also ACK packets that do have this set correctly:

screen shot 2017-05-30 at 15 13 23

Now as a quick hack I’ve tried not calling ackSkipped when there are no bytes in the extension:

		switch ext.Type {
		case extensionTypeSelectiveAck:
			if len(ext.Bytes) > 0 {
				c.ackSkipped(h.AckNr + 1)
				bitmask := selectiveAckBitmask{ext.Bytes}
				for i := 0; i < bitmask.NumBits(); i++ {
					if bitmask.BitIsSet(i) {
						nr := h.AckNr + 2 + uint16(i)
						c.ack(nr)
					} else {
						c.ackSkipped(h.AckNr + 2 + uint16(i))
					}
				}
			}
              }

and I’ve also changed ackSkipped to not resend the packets that often:

	switch send.acksSkipped {
	case 60, 120: // was 3, 60
		if logLevel >= 1 {
			log.Printf("acksSkipped = %d", send.acksSkipped)
		}
		ackSkippedResends.Add(1)
		send.resend()
		send.resendTimer.Reset(c.resendTimeout() * time.Duration(send.numResends))
	default:
	}

For me this is dramatically decreasing the number of duplicates that are being send.

However it does not fix my issue with uTP traffic throttling back when there is TCP background traffic.

Does this make sense?

About this issue

  • Original URL
  • State: open
  • Created 7 years ago
  • Comments: 16 (4 by maintainers)

Commits related to this issue

Most upvoted comments

Bare in mind, if you require cross compiling this makes things very difficult due to cgo.

Fixing the extension bytes shouldn’t affect resends at all. It should only potentially improve compatibility out there with other utp implementations. But it was helpful you spotted this, and I’ve fixed it.

The amount of resending should be directly affected by the timeouts and allowed unacked sends. However decreasing the allowed number of unacked sends will decrease throughput when there is high latency. I believe the spec limits unacked send packets by adjusting its window size. This implementation doesn’t do that.

It might be very effective to block writes, and thereby further unacked sends when there are sends that haven’t been acked in a timely fashion, as we track the usual latency.

I would guess that the issues you’re describing are all resulting from your OS being overwhelmed by the number of sends, and failing huge batches of consecutive sends, which results in “resend storms”. What OS/distribution/arch are on you on? What Go version? What kind of data rates are you achieving?

Thanks, this is very detailed. I did not know Wireshark was aware of uTP that’s helpful. If this implementation backs off in the presence of TCP in its current form, that would be coincidental, I’ve put no effort into ensuring that. I’ll take a more detailed look at your findings tomorrow.