badgerhold: Transaction Conflict. Please retry error

Most of the time I get this error, Is this normal? if it is normal how can I try again in code Error:

panic: Transaction Conflict. Please retry

goroutine 10 [running]:
gitlab.com/nanovy/rent/server/database.(*BadgerDB).Test.func1(0xc0000240e0, 0x2, 0xc00000e640)
        /home/eren/Desktop/Nanovy/Rent/Server/database/connect.go:69 +0xf6
created by gitlab.com/nanovy/rent/server/database.(*BadgerDB).Test
        /home/eren/Desktop/Nanovy/Rent/Server/database/connect.go:62 +0x106
exit status 2

Code:


type Item struct {
	Id        uint64 `badgerhold:"key"`
	Name string
	Cost    int
}

func (bdb *BadgerDB) Test() {
	db := bdb.Store
	db.Badger().DropAll()

	var wg sync.WaitGroup

	for i := 0; i < 100; i++ {
		wg.Add(1)
		i := i
		go func() {
			defer wg.Done()
			item := Item{
				Name: "Glass",
				Cost: i,
			}
			if err := db.Insert(badgerhold.NextSequence(), &item); err != nil {
				panic(err)
			}
		}()
	}

	wg.Wait()

	var items []Item
	if err := db.Find(&items, badgerhold.Where("Cost").Ge(98)); err != nil {
		panic(err)
	}
	spew.Dump(items)
}

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 16 (7 by maintainers)

Most upvoted comments

Yep, that puts it firmly as an issue in Badgerhold and not Badger itself.

I’ll see if I can replicate this, and figure out how manage this. My guess is it’s fundamentally and issue with their serializable snapshot isolation.

Basically across multiple transactions an index could’ve been updated more than once. This may take a bit to come up with a workaround.

timshannon on Feb 17, 2021

I gladly welcome PRs. I’m crazy busy with work at the moment. I don’t have confidence that the WriteBatch API will help though, at least not without separating the writes to the index into a different transaction from the main data write. If you do that, then you’ll run into the issue of one transaction failing while the other commits, which would leave you with dirty data.

If this were SQL server or some other database, we’d solve this by running it in a serializable transaction.

My first thought at a solution is to simply retry if a transaction conflict is hit. The index will be up to date and correct (as long as the current index data is re-pulled), but it could be incredibly slow.

Like I said, I welcome PRs, otherwise I get to it as soon as I can.

On Wed, Feb 24, 2021 at 3:38 AM James notifications@github.com wrote:

I’ve had this problem too. I fixed it by sending writes to a goroutine running the code below, so that only one write occurs at a time.

func (storeDatum *badgerholdStoreDatum) BatchManager() { for { func() { defer storeDatum.mutex.RUnlock() storeDatum.mutex.RLock()
  	batchDatum := <-storeDatum.BatchChan

  	var err error

  	if storeDatum.IsStoreClosed {
  		err = ErrBadgerholdStoreClosed
  	} else {
  		switch batchDatum.Event {
  		case badgerholdBatchEventInsert:
  			err = storeDatum.Store.Insert(batchDatum.Key, batchDatum.Data)
  		case badgerholdBatchEventTxInsert:
  			err = storeDatum.Store.TxInsert(batchDatum.Tx, batchDatum.Key, batchDatum.Data)
  		case badgerholdBatchEventUpsert:
  			err = storeDatum.Store.Upsert(batchDatum.Key, batchDatum.Data)
  		case badgerholdBatchEventUpdate:
  			err = storeDatum.Store.Update(batchDatum.Key, batchDatum.Data)
  		}
  	}

  	if batchDatum.ErrChan != nil {
  		batchDatum.ErrChan <- err
  	}
  }()
} }

I’d love the issue to be fixed properly though (using Badger’s own WriteBatch API, as @aakarim https://github.com/aakarim says), because Badgerhold is an awesome little database! I’m really impressed with how effective and easy it is to use.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/timshannon/badgerhold/issues/46#issuecomment-784944518, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKE2NPDXGEAN56MCM5RLV3TATCJTANCNFSM4TXQYBJQ .

– Tim Shannon www.townsourced.com

timshannon on Feb 24, 2021

I’m going to merge this soon. Everything looks good to me, but with so many people interested in this fix I assume I’d have at least one other party to take a look at it.

timshannon on Aug 3, 2021