rqlite: CAS conflict

Every few days the leader in my 3 node cluster ends up with this error:

Jun 20 08:17:37 web5 rqlited[4026513]: [raft] 2024/06/20 08:17:37 [ERROR] failed to take snapshot: error="failed to start snapshot: CAS conflict"
Jun 20 08:17:40 web5 rqlited[4026513]: [raft] 2024/06/20 08:17:40 [ERROR] failed to take snapshot: error="failed to start snapshot: CAS conflict"
Jun 20 08:17:40 web5 rqlited[4026513]: [store] 2024/06/20 08:17:40 failed to snapshot due to WAL threshold: failed to start snapshot: CAS conflict

I have a hard time figuring out what this means or how to resolve it. (I currently delete the database and have it restore from a different node as a work-around).

The WAL is fairly large when this happens:

total 3.1G
-rw-r--r-- 1 rqlite rqlite 902M Jun 19 14:25 db.sqlite
-rw-r--r-- 1 rqlite rqlite 3.7M Jun 20 08:21 db.sqlite-shm
-rw-r--r-- 1 rqlite rqlite 1.9G Jun 20 08:21 db.sqlite-wal
drwxr-xr-x 1 rqlite rqlite    0 Jun 17 09:01 raft
-rw------- 1 rqlite rqlite 321M Jun 20 08:21 raft.db
drwxr-xr-x 1 rqlite rqlite  106 Jun 19 14:25 rsnapshots

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 32 (20 by maintainers)

Most upvoted comments

You could so other things too, like checking that what you get is an actual SQLite file.