lettuce-core: Responses getting out of sync with requests
Current Behavior
We’ve recently updated lettuce to 6.1.1.RELEASE and after running it in our production environment for a week we ran into a troubling issue. At some point it looks like lettuce worked itself into a state where commands were receiving responses to different requests. This manifested as lots of deserialization errors (e.g. java.lang.UnsupportedOperationException: io.lettuce.core.output.ValueOutput does not support set(long)
), but also more seriously as incorrect data being returned to the caller.
Restarting the service brought things back to a working state. Prior to this happening the only other thing we observed were some RedisCommandTimeoutException
being thrown, but not at an anomalous level.
For the meantime we’ve added some more aggressive checking for this happening, so if it does reoccur I’d be happy to try and gather more information, but not sure what would be helpful?
Environment
- Lettuce version(s): 6.1.1.RELEASE
- Redis version: 3.2.10
About this issue
- Original URL
- State: open
- Created 2 years ago
- Comments: 16 (4 by maintainers)
We have got the same problem. And I reproduced the problem by using below codes:
My environment is:
I found if my business codes throw errors, it will be catched by the lettuce pool thread.While this will finally lead to
java.lang.UnsupportedOperationException: io.lettuce.core.output.ValueOutput does not support set(long)
exception. And If I indicate a thread pool to my business codes by publishOn, the error won’t be caught by lettuce thread.This can void the exception happening.Just uncomment ‘publishOn’. ‘subscribeOn’ is testing code.