go-redis: connection pool timeout

I am using Redis as a caching layer for long running web services. I initialize the connection like so:

var (
    Queues  *redis.Client
    Tracker *redis.Client
)

func Connect(url string) {
    // cut away redis://
    url = url[8:]

    // connect to db #0
    Queues = redis.NewClient(&redis.Options{
        Addr:     url,
        Password: "",
        DB:       0,
    })

    _, err := Queues.Ping().Result()
    if err != nil {
        panic(err)
    }

    // connect to db #1
    Tracker = redis.NewClient(&redis.Options{
        Addr:     url,
        Password: "",
        DB:       1,
    })

    _, err = Tracker.Ping().Result()
    if err != nil {
        panic(err)
    }
}

Abeit in an upcoming patch (sysadmin is deploying a Redis cluster) it will be like so:

var (
    Cluster *redis.ClusterClient
)

func ConnectCluster(cluster, password string) {
    addresses := strings.Split(cluster, ",")
    Cluster = redis.NewClusterClient(&redis.ClusterOptions{
        Addrs: addresses,
        // Password: password,
    })

    _, err := Cluster.Ping().Result()
    if err != nil {
        panic(err)
    }
}

The above code gets run once when service boots up in main.go and the *redis.ClusterClient is being used for the lifetime of the process.

I realize there is an iherent problem with this approach, which is manifesting itself in connections timing out after a few days, and crashing the application: redis: connection pool timeout See logs here

Please advise, what would be a proper approach to use go-redis in this situation?

About this issue

  • Original URL
  • State: closed
  • Created 9 years ago
  • Comments: 17 (4 by maintainers)

Commits related to this issue

Most upvoted comments

That error can mean 2 things:

  • Redis is busy doing some expensive work (unlikely)
  • you use PubSub or Multi and don’t close it correctly (multi.Close() when multi is not needed any more) so connection is not returned to the pool

So yes, the default pool size is rather small. Based on your snippet, you use one connection per http request, which means your pool size must match the number of your concurrent http connections. I think, 1000 seems a little high, but it’s really only the maximum cap. It shouldn’t matter, as the client opens new connections on demand.

I will submit a PR to allow applications to instrument the pool size, this seems to be quite relevant. On 27 Nov 2015 7:10 am, “Netherdrake” notifications@github.com wrote:

I have bumped the options and deployed, lets see if this works:

Cluster = redis.NewClusterClient(&redis.ClusterOptions{
    Addrs:        addresses,
    PoolSize:     1000,
    PoolTimeout:  2 * time.Minute,
    IdleTimeout:  10 * time.Minute,
    ReadTimeout:  2 * time.Minute,
    WriteTimeout: 1 * time.Minute,
    // Password: password,
})

— Reply to this email directly or view it on GitHub https://github.com/go-redis/redis/issues/195#issuecomment-160056781.

Also, is there any way to disable panics on redis: connection pool timeout. I would much rather handle this in application rather than have the whole server blow up.