iris: Poor performance of session.UpdateExpiration on 200 thousands+ keys with new radix lib

Hello Makis!

I have more 200 thousands key in redis on prod server and found that after upgrading iris on v11 from redigo to radix lib that I have problems with perfomance when calling session.UpdateExpiration method (it`s executing minimum 10 seconds on a powerful prod server and more than 30 seconds on usual local).

While debugging I found place where freezes, it`s here:

func (r *Service) UpdateTTLMany(prefix string, newSecondsLifeTime int64) error {
	keys, err := r.getKeys(prefix)
	if err != nil {
		return err
	}
...
}

and inside getKeys here:

scanner := radix.NewScanner(r.pool, radix.ScanOpts{
		Command: "SCAN",
		Pattern: r.Config.Prefix + prefix + r.Config.Delim + "*", // get all of this session except its root sid.
		//	Count: 9999999999,
	})

	var key string
	for scanner.Next(&key) {
		keys = append(keys, key)
	}

Just has watched old v10 iris and old method getKeysConn did not cause such delay …

func (r *Service) getKeysConn(c redis.Conn, prefix string) ([]string, error) {
	if err := c.Send("SCAN", 0, "MATCH", r.Config.Prefix+prefix+"*", "COUNT", 9999999999); err != nil {
		return nil, err
	}

	if err := c.Flush(); err != nil {
		return nil, err
	}

	reply, err := c.Receive()
	if err != nil || reply == nil {
		return nil, err
	}

Maybe reason in difference between old and new scan commands? Could you help?

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 18 (10 by maintainers)

Commits related to this issue

Most upvoted comments

@r3eg that’s nice!

I think I found the issue on radix, it is calling a new scan command based on the returned cursor from the SCAN redis command, which is the cursor. This is the problem: by default the COUNT is 10, we didn’t provide a COUNT option on the radix implementation, while in the redigo one we have a COUNT of 9999999999 so it did fired one command to fetch your keys, and radix did 20000 commands to fetch 200 thousand keys. This is fixed now.

yes it is true . Bravo. Good point Thank You @kataras .

@kataras Hello Makis! Today I pushed changes to prod server, all works fine now, no problem with perfomance at all! Thank you again!