badger: Invalid value pointer offset: 1659 greater than current offset: 0
I’m getting an error while iterating my data: Invalid value pointer offset: 1659 greater than current offset: 0
Further Tests:
db.Backup()leads toInvalid value pointer offset: 1659 greater than current offset: 746db.PurgeOlderVersions()runs but does not help
Feels like the value log is corrupted or something. Is there any preferred way to recover from this?
Here is my Database (only 16MB): https://files.lobaro.com/index.php/s/xWXphXDnNYRF0BN
My Code:
func iterateData(db *badger.DB, cb func(*model.Data) error) error {
err := db.View(func(txn *badger.Txn) error {
it := txn.NewIterator(badger.DefaultIteratorOptions)
defer it.Close()
prefix := []byte(PREFIX_DATA)
for it.Seek(prefix); it.ValidForPrefix(prefix); it.Next() {
item := it.Item()
k := item.Key()
v, err := item.Value()
if err != nil {
return err // Error appears here
}
entity := &model.Data{}
err = json.Unmarshal(v, entity)
if err != nil {
logrus.WithError(err).WithField("key", string(k)).Warn("Failed to unmarshal channel")
} else {
err := cb(entity)
if err != nil {
return err
}
}
}
return nil
})
return err
}
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 19 (13 by maintainers)
Badger writes the new SST files out, then syncs them, before it swaps them out for the old ones. So, if any crash occurs during this process, these new SST files would just not be used. And after they are synced up, the chances of them getting corrupted is extremely low (again, disk level corruption has to happen).
The process is similar for MANIFEST, IIRC.
Feel free to file an issue to add that option, or better still, send a PR. You can look at the recent PR which added ReadOnly option.
@manishrjain Maybe the user could be notified of the corrupted database, and left up to him (e.g., through a flag in
Open) whether or not to truncate what could be 1GB of (some) recoverable data.