node-redis: retry_strategy doesn't reset after flushing the error

  • Version: node_redis 2.6.5, redis 3.2.7
  • Platform: Node.js 7.5.0 on Mac OS 10.12.3
  • Description:

I’m using the following code

const redis = require('redis');

const client = redis.createClient({
  url: 'redis://localhost:6379',
  retry_strategy: options => new Error('Fail')
});

setInterval(() => client.incr('some-key', console.log), 5000);

running against a local redis (docker run --rm -p 6379:6379 redis)

After I see the first output, I kill the redis to simulate disconnect. When I’ve seen the error come in I restart the redis. The connection doesn’t come up again. I would expect that after flushing the error to the offline handlers, the client would try to reconnect on the next command. instead it stays in closed state. Also it’s returning with AbortError instead of new Error('Fail').

About this issue

  • Original URL
  • State: open
  • Created 7 years ago
  • Comments: 21 (3 by maintainers)

Most upvoted comments

Quick update here. I added the enable_offline_queue = false to my createClient options, and it seems to have done what I wanted it to, which is: “Continue to attempt to re-connect to my Redis server based on the retry_strategy function, but immediately throw errors in response to attempts to use the client in the meantime. Then, if able to reconnect, just start working again”.

FYI: Thows the following error on a get call: AbortError: SET can't be processed. Stream not writeable.

For my usage as a straight layer between the DB and my app, this should be fine. Would be nice to know if there’s a more “robust” solution possible where:

  • Client continues to attempt to reconnect based on retry_strategy
  • Calls/consumers/users of the Client don’t hang while it’s reconnecting
  • Commands can still be queued (conditionally?) to be executed if/when a connection is re-established…in case I’m incrementing things, etc.

Probably a lot of tough scenarios to consider here…

@BridgeAR It’s pretty much that right now, if redis connection breaks for some reason, all my commands just hang. Suppose I have a fallback for the operation that redis is doing then that fallback also hangs. I’d rather want redis to return an error after a few retries, which I can catch and employ the fallback. That doesn’t mean I don’t want the client to stop trying to connect. Redis might come back up in a minute.

I just don’t see the point of the current behavior. You return an error in the retry handler and then your client just dies forever. Who needs that sort of behavior unless you create a client for every command?

Or I might not see the use case for that ofcourse 😄

the flow I’m looking for is: on disconnect, retry connecting 3 times with 1 second interval. If at the third time it’s still disconnect, it should return all outstanding commands with an error. I don’t want it to stop trying to reconnect.

Thank you @newhouse, I’m not sure I would have ever found this.

IMO this should be the default behavior, or at least explicitly called out in the docs around retry_strategy. Let me know if you think the latter is reasonable, and I will open a PR to add the docs.

+1 for a solution to this problem.

I’m using Express, and while my retry_strategy is still returning integers to try and connect again in the future, the commands (and web requests) continue to pile/back-up instead of throwing something so they can get on with their lives…and hopefully a connection is eventually re-established before the strategy says to give up or whatever.

Can this be accomplished? @Janpot have you found anything currently available to solve this situation?

Maybe something with the enable_offline_queue and/or retry_unfulfilled_commands options can accomplish this?

Thanks to all for your hard work and help!

One gotcha with using enable_offline_queue = false is that commands sent right after calling createClient, but before the connection is actually opened will fail as well.

@jkirkwood Sounds like you’ll want to delay trying to send your first commands until the ready event (or similar appropriate event) is emitted?

@newhouse,

Thanks for mentioning this. We wanted the same behavior for our application and your solution worked for immediately returning an error to the caller, but allow us to retry to connect indefinitely.

Via the documentation for retry_strategy, it appears that node_redis will only attempt to reconnect if the function returns a number.

If you return a number from this function, the retry will happen exactly after that time in milliseconds.

In your case you are returning a non number, which does not satisfy the requirements for attempting to reconnect.

If you return a non-number, no further retry will happen and all offline commands are flushed with errors.