xmemcached: Problem when a memcached server changes IP address
We have to configure our client with failureMode=true, and we are getting errors like this when a memcached server gets down and comes back with a different IP:
net.rubyeye.xmemcached.exception.MemcachedException: Session(192.168.1.41:11211) has been closed
at net.rubyeye.xmemcached.impl.MemcachedConnector.send(MemcachedConnector.java:512)
at net.rubyeye.xmemcached.XMemcachedClient.sendCommand(XMemcachedClient.java:317)
at net.rubyeye.xmemcached.XMemcachedClient.fetch0(XMemcachedClient.java:644)
at net.rubyeye.xmemcached.XMemcachedClient.get0(XMemcachedClient.java:1085)
at net.rubyeye.xmemcached.XMemcachedClient.get(XMemcachedClient.java:1043)
at net.rubyeye.xmemcached.XMemcachedClient.get(XMemcachedClient.java:1054)
at net.rubyeye.xmemcached.XMemcachedClient.get(XMemcachedClient.java:1076)
The old IP was 192.168.1.41 and the new one is 192.168.1.44. When the server recovers with the new IP we can see logs like:
com.google.code.yanf4j.core.impl.AbstractController:? Add a session: 192.168.1.44:11211
However, the client is still using sessions with the old IP that are closed.
I have been debugging a bit and found that in the class net.rubyeye.xmemcached.impl.MemcachedConnector there is an attribute called sessionMap that contains sessions with both the old and new IP, because the new sessions do not override the old ones, and then all those sessions are passed to the session locator in the method updateSessions(). I think the session locator should receive only the sessions with the new IP.
Please could you take a look?
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Comments: 15 (14 by maintainers)
Commits related to this issue
- (fix) old sessions were kept when memcached server changed IP address with failure mode, close #68 — committed to killme2008/xmemcached by killme2008 6 years ago
- fix https://github.com/killme2008/xmemcached/issues/68 which already appeared somewhere in april 2020? when allowing disabling address resolution — committed to Raiv/xmemcached by Raiv 3 years ago
- Merge pull request #129 from Raiv/master fix https://github.com/killme2008/xmemcached/issues/68 — committed to killme2008/xmemcached by killme2008 3 years ago
I’ve fixed this issue, it happens in failure mode, the new release will be delivered ASAP.
I will look into it, thanks.