rqlite: CAS conflict
Every few days the leader in my 3 node cluster ends up with this error:
Jun 20 08:17:37 web5 rqlited[4026513]: [raft] 2024/06/20 08:17:37 [ERROR] failed to take snapshot: error="failed to start snapshot: CAS conflict"
Jun 20 08:17:40 web5 rqlited[4026513]: [raft] 2024/06/20 08:17:40 [ERROR] failed to take snapshot: error="failed to start snapshot: CAS conflict"
Jun 20 08:17:40 web5 rqlited[4026513]: [store] 2024/06/20 08:17:40 failed to snapshot due to WAL threshold: failed to start snapshot: CAS conflict
I have a hard time figuring out what this means or how to resolve it. (I currently delete the database and have it restore from a different node as a work-around).
The WAL is fairly large when this happens:
total 3.1G
-rw-r--r-- 1 rqlite rqlite 902M Jun 19 14:25 db.sqlite
-rw-r--r-- 1 rqlite rqlite 3.7M Jun 20 08:21 db.sqlite-shm
-rw-r--r-- 1 rqlite rqlite 1.9G Jun 20 08:21 db.sqlite-wal
drwxr-xr-x 1 rqlite rqlite 0 Jun 17 09:01 raft
-rw------- 1 rqlite rqlite 321M Jun 20 08:21 raft.db
drwxr-xr-x 1 rqlite rqlite 106 Jun 19 14:25 rsnapshots
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 32 (20 by maintainers)
You could so other things too, like checking that what you get is an actual SQLite file.