go-redis: high memory usage + solution

Hi,

I noticed that the memory usage was very high in my project. I did a memory profiling with inuse_space, and 90% of my memory is used by go-redis in WriteBuffer. If I understand correctly, each connection in the pool has its own WriteBuffer.

My projects runs 80 goroutines (on 8 CPUs) and each goroutine SET Redis keys. My Redis keys are large: several MB. (less than 100 MB) So, it’s very easy to understand why the memory usage is very high.

I think I have a solution, but it requires changes in go-redis internals. We could use a global sync.Pool of WriteBuffer instead.

WDYT ?

About this issue

Original URL
State: closed
Created 6 years ago
Reactions: 4
Comments: 24 (10 by maintainers)

Most upvoted comments

Here is a quick POC:

diff --git a/vendor/github.com/go-redis/redis/command.go b/vendor/github.com/go-redis/redis/command.go
index 992e614..4509a34 100644
--- a/vendor/github.com/go-redis/redis/command.go
+++ b/vendor/github.com/go-redis/redis/command.go
@@ -46,14 +46,15 @@ func firstCmdsErr(cmds []Cmder) error {
 }
 
 func writeCmd(cn *pool.Conn, cmds ...Cmder) error {
-	cn.Wb.Reset()
+	wb := proto.GetWriteBufferPool()
+	defer proto.PutWriteBufferPool(wb)
 	for _, cmd := range cmds {
-		if err := cn.Wb.Append(cmd.Args()); err != nil {
+		if err := wb.Append(cmd.Args()); err != nil {
 			return err
 		}
 	}
 
-	_, err := cn.Write(cn.Wb.Bytes())
+	_, err := cn.Write(wb.Bytes())
 	return err
 }
 
diff --git a/vendor/github.com/go-redis/redis/internal/pool/conn.go b/vendor/github.com/go-redis/redis/internal/pool/conn.go
index acaf366..13ef043 100644
--- a/vendor/github.com/go-redis/redis/internal/pool/conn.go
+++ b/vendor/github.com/go-redis/redis/internal/pool/conn.go
@@ -14,7 +14,6 @@ type Conn struct {
 	netConn net.Conn
 
 	Rd *proto.Reader
-	Wb *proto.WriteBuffer
 
 	Inited bool
 	usedAt atomic.Value
@@ -23,7 +22,6 @@ type Conn struct {
 func NewConn(netConn net.Conn) *Conn {
 	cn := &Conn{
 		netConn: netConn,
-		Wb:      proto.NewWriteBuffer(),
 	}
 	cn.Rd = proto.NewReader(cn.netConn)
 	cn.SetUsedAt(time.Now())
diff --git a/vendor/github.com/go-redis/redis/internal/proto/write_buffer.go b/vendor/github.com/go-redis/redis/internal/proto/write_buffer.go
index 664f4c3..a7ebbce 100644
--- a/vendor/github.com/go-redis/redis/internal/proto/write_buffer.go
+++ b/vendor/github.com/go-redis/redis/internal/proto/write_buffer.go
@@ -4,6 +4,7 @@ import (
 	"encoding"
 	"fmt"
 	"strconv"
+	"sync"
 )
 
 type WriteBuffer struct {
@@ -111,3 +112,19 @@ func formatUint(u uint64) string {
 func formatFloat(f float64) string {
 	return strconv.FormatFloat(f, 'f', -1, 64)
 }
+
+var writeBufferPool = &sync.Pool{
+	New: func() interface{} {
+		return NewWriteBuffer()
+	},
+}
+
+func GetWriteBufferPool() *WriteBuffer {
+	w := writeBufferPool.Get().(*WriteBuffer)
+	w.Reset()
+	return w
+}
+
+func PutWriteBufferPool(w *WriteBuffer) {
+	writeBufferPool.Put(w)
+}

pierrre on Aug 3, 2018