nsq: failed ID not in flight
protocol error - E_FIN_FAILED FIN 09e3cdf1b2197000 failed ID not in flight
msg 09e3729b4fd97000 attempted 12 times, giving up
Thing what causes it, the data is lost yet?
- Requeued : 1,142,034
- Timed Out : 1,142,019
- Messages : 20,517,188
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Comments: 25 (10 by maintainers)
Yes, to extend the timeout, you can TOUCH before the timeout, repeatedly, up to the max-msg-timeout
What’s causing this are those “timed out” messages - NSQ isn’t hearing back from your consumer and it’s delivering the message to another consumer.
Configure nsqd to give it more time, or process the message faster, or just ignore it.
This isn’t necessarily a problem, if it happens rarely - it is a feature that catches the occasional “stuck” consumer. The message will be re-tried (probably by a different consumer, if there are multiple).
If this happens most of the time, then you have to fix something, but it’s not one specific thing. Maybe your max-in-flight is way too high, causing messages to wait to be processed for a long while after being delivered. Maybe you just need the consumer to request a few more minutes. Maybe the way you are using nsq just does not work out well, and you need to do something else.
@ephetic it is returned as part of the IDENTIFY response
Your nsq consumer library should have a “msgTimeout” option - e.g. in go-nsq it’s in the Config struct: https://godoc.org/github.com/nsqio/go-nsq#Config
That consumer-configured msgTimeout is honored by nsqd only up to nsqd’s
--max-msg-timeout(default 15 minutes).