tikv: How to recover from tikv failure

Question

I have 2 TiKV’s and 1 PD with a replication of 1. I lose all the data in one of the tikv, because of node failure (aws instance store). Now, how do I recover from this state?

I have tried pd-ctl store delete <id>, but its still in offline state. Can I force it to be in tombstone state because I have no replicas?

When I try to restart tikv, I get this error: duplicated store address:

Thanks for you help.

PD Version:

Release Version: v3.0.7
Git Commit Hash: 7a5909ed3bae74d0c6c728ca931f240233aca03a
Git Branch: HEAD
UTC Build Time:  2019-12-04 10:06:16

TiKV Version

TiKV Control (tikv-ctl)
Release Version:   3.0.7
Git Commit Hash:   ac6f02648a8c6ccb7ccafca20287e1b27007e4a0
Git Commit Branch: HEAD
UTC Build Time:    2019-12-04 10:06:20
Rust Version:      rustc 1.37.0-nightly (0e4a56b4b 2019-06-13)

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 16 (8 by maintainers)

Most upvoted comments

Would you like to try to run the command to remove the tombstone stores:

curl -XDELETE http://your-pd-address/pd/api/v1/stores/remove-tombstone