slatedb: Notify clients when there is a failure
Client puts and deletes don’t have the opportunity to receive SlateDBError::Fenced errors. How do clients detect when they’ve been fenced?
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 17 (5 by maintainers)
Commits related to this issue
- Replace panics in background tasks with errors Fixes: #111 — committed to palash25/slatedb by palash25 2 years ago
- Replace panics in background tasks with errors Fixes: #111 — committed to palash25/slatedb by palash25 2 years ago
- Replace panics in background tasks with errors Fixes: #111 — committed to palash25/slatedb by palash25 2 years ago
- Unify write commit pipe (#290) This PR migrates all put/delete calls to use write batches (and thus, the commit pipeline). For single row operations, a batch with one row is created under the hoo... — committed to slatedb/slatedb by criccomini 2 years ago
- fix(flush): Handle background flush errors. (#321) When a flush operation fails in the background, an error is populated in the COWDbState structure that handles the state of the database. Foreground... — committed to slatedb/slatedb by calavera 2 years ago
- Unify write commit pipe (#290) This PR migrates all put/delete calls to use write batches (and thus, the commit pipeline). For single row operations, a batch with one row is created under the hoo... — committed to flaneur2020/slatedb by criccomini 2 years ago
- fix(flush): Handle background flush errors. (#321) When a flush operation fails in the background, an error is populated in the COWDbState structure that handles the state of the database. Foreground... — committed to flaneur2020/slatedb by calavera 2 years ago
Sorry something came up this week. I will try to finish this up tomorrow or day after if I can’t make a PR by then we can have someone else try this
On Sat, Oct 5, 2024, 4:56 AM Chris Riccomini @.***> wrote:
starting work on this
sure, this seems interesting.
@dyedgreen you should also join the SlateDB Discord Server if you haven’t already. It would be great to have you there 🙂
Those two look good as well; happy to get started with #77 (from reading the issue that one seems a bit easier, so might be a better one to get started with? will also comment on that issue to prevent further races 🫣)
There is more discussion about this issue on Discord. The general idea is that we have several background threads: WAL flusher, memtable flusher, and compaction. If errors occur on these threads, they die, but the user is never notified of the failure. They continue put’ing and get’ing and weird behavior can happen.
Per-the discussion, I think the easiest path forward is:
error: Option<SlateDbError>inDbState.panic!()'s by setting theDbStateerror.DbStateerror.