prisma: WriteConflict error with MongoDB

Hey folks,

With the lack of reports on write conflict errors ever since we’ve implemented retries, we will close this issue now. Thanks for the feedback and for bearing with us until we got it implemented y’all 🙏

Hey folks,

Starting in Prisma 4.9.0, all queries except interactive transactions are now automatically retried like MongoDB drivers do.

To be clear, what won’t be retried automatically are queries such as this one:

await prisma.$transaction(async tx => {
  await prisma.user.create();
  await prisma.post.create();
});

We hope that this will already solve most of your problems.

Thanks 🙏

if anybody needs it for nestjs prisma

import { Prisma } from '@prisma/client';
import * as retry from 'retry';

const IGNORE_ACTIONS = [
  'findUnique',
  'findMany',
  'findFirst',
  'aggregate',
  'count',
  'findRaw',
] as const;

export function retryMiddleware(): Prisma.Middleware {
  return async (params, next) => {
    const operation = retry.operation({
      retries: 4, // 1st time is not counted
      minTimeout: 100, // 100ms
      maxTimeout: 2000, // 2 seconds
      randomize: true,
      factor: 1.97, // https://www.wolframalpha.com/input?i2d=true&i=Sum%5B100*Power%5Bx%2Ck%5D%2C%7Bk%2C0%2C4%7D%5D+%3D+3+*+1000
    });

    if (~IGNORE_ACTIONS.indexOf(params.action as any)) {
      return await next(params);
    }

    return await new Promise((resolve, reject) => {
      operation.attempt(async (a) => {
        let result: any;
        let error: any = null;
        try {
          error = null;
          result = await next(params);
        } catch (e) {
          // Only handle WriteConflict issues
          if (
            e instanceof Prisma.PrismaClientKnownRequestError &&
            e.code === 'P2034'
          ) {
            error = e;
          } else {
            // This is another kind of errors, we stop retrying and reject the promise
            operation.stop();
            reject(e);
          }
        }

        // If error is null, this will be false and we can continue the execution
        if (operation.retry(error)) {
          return;
        }

        if (error) {
          reject(operation.mainError());
        } else {
          resolve(result);
        }
      });
    });
  };
}

Credit to @aladinflux

import { Prisma, PrismaClient } from '@prisma/client'
import retry from 'retry'

const IGNORE_ACTIONS = ['findUnique', 'findMany', 'findFirst', 'aggregate', 'count', 'findRaw'] as const

prisma.$use(async (params, next)=> {
  const operation = retry.operation({
    retries: 4, // 1st time is not counted
    minTimeout: 100, // 100ms
    maxTimeout: 2000, // 2 seconds
    randomize: true,
    factor: 1.97, // https://www.wolframalpha.com/input?i2d=true&i=Sum%5B100*Power%5Bx%2Ck%5D%2C%7Bk%2C0%2C4%7D%5D+%3D+3+*+1000
  })
  
  if(~IGNORE_ACTIONS.indexOf(params.action as any)) {
    return await next(params)
  }

  return await new Promise((resolve, reject) => {
    operation.attempt(async (a) => {
      let result: any
      let error: any = null
      try {
        error = null
        result = await next(params)
      } catch(e) {
        // Only handle WriteConflict issues
        if(e instanceof Prisma.PrismaClientKnownRequestError && e.code === 'P2034') {
          error = e
        } else {
          // This is another kind of errors, we stop retrying and reject the promise
          operation.stop()
          reject(e)
        }
      }

      // If error is null, this will be false and we can continue the execution
      if(operation.retry(error)) {
        return
      }

      if(error) {
        reject(operation.mainError())
      } else {
        resolve(result)
      }

    })
  })
})

Hey folks, thanks for keeping this thread alive. We’re currently working on this and would like your feedback on something.

While “regular” queries are safe to be retried automatically, we fear that automatic retries for interactive transactions might lead to undesirable behaviors.

See, while it’s not recommended to perform side-effects in a transaction callback, we know many users are doing it.

The consequence of performing silent, automatic retries on non-idempotent side-effects might be bad and surprising for many.

Now, after reviewing what most MongoDB drivers do, we’ve noticed that they all seem to be performing those automatic retries.

Here are two questions for you:

• Have you ever been bitten by those automatic retries while using Mongoose or any Mongo drivers?
• Also, have you ever needed to disable those automatic retries?

There are the few options we have (keep in mind we’re specifically talking about ITXs):

• Automatic retries enabled by default (put behind a preview feature flag until we GA it)
• Automatic retries disabled by default but configurable per ITXs

Based on your experience, let us know what you’d prefer.

(Please keep in mind that those API design decisions are always full of trade-offs and that we may respectfully have to choose a different direction than what y’all would like. Your feedback remains very valuable nonetheless)

Thanks for your help, cheers 🤓

Thanks for the continued feedback here.

As some of you already figured out, we were not MongoDB experts when we started our implementation and seem to have gotten some things not perfectly right - especially compared to other MongoDB drivers and libraries. As someone also pointed out, we come from a different direction in how we treat schema, data and relations, so this decision made sense to use at the time.

We are now investigating what exactly we are doing why, and what and how we can change this to minimize the impact of this error and optimally just get rid of it.

PS: We indeed use the official MongoDB Rust driver as @codeinearts mentioned.

We’re facing the same issue but solving it with a queue would be an overkill. I’m thinking of replacing prisma with another ORM. Has anyone tried another ORM after facing this issue? And did that fix the issue? Even very simple writes fail and cause sporadic issues.

Switched to Mongoose, because of this error and haven’t looked back. It works like a charm and is a simple technology compared to Prisma (Not as many moving parts). Albeit it does not have as many fancy features as Prisma.

But in my experience it is plenty sufficient.

I can reproduce the problem with 3.12 but with 4.10 we throw the following error when generating the client:

But the original problem is still reproducible even after fixing the types:

So we’ve already implemented that strategy as I mentioned above but with high throughput documents will still eventually trigger it - MongoDB’s non-transaction behaviour (Put updates in a queue) would be ideal for these situations as in our situation we are just using atomic updates to increment / decrement a user’s credits.

In these cases we had to re architecture our code to have a queue to process these in a way to avoid conflicts all together. So some way to say to the client “hey we actually want this up date to be ran outside of a mongo db transaction” would be ideal.

We haven’t seen the WriteConflict error since upgrading to 4.9.0 to. Good work 🙌

The way I solved this issue is by using a middleware that catches WriteConflict issues thrown by prisma and retries the write using the retry npm package to keep retrying the transaction for 5 times with different intervals until it succeeds and I’m happy to say that we got rid of all WriteConflict issues we were having.

You can try disconnecting prisma instance every time after the write operation is done. It is working for me.

From this

  const prisma = new PrismaClient();
  for (let i = 0; i < 100; i++) {
    await prisma.coll.create({
      data: {
           //...
      },
    });
  }
  //let prisma instrance destroy itself after some period

To this.

  for (let i = 0; i < 100; i++) {
     const prisma = new PrismaClient();
     await prisma.coll.create({
      data: {
          //...
      },
    });
    //disconnect immediately after creating
    prisma.$disconnect();
  }

Notice that it could affect writing performance. But I did not benchmark it.

We can recreate this error with atomic updates which are supposed to be to avoid WriteConflicts and race conditions.

Given the following API call:

await prisma.wallet.update({
            where: { id: "1" },
            data: {
                credits: {
                    increment: 1
                }
            },
        });

We will get a WriteConflict from MongoDB if two update calls are triggered at the same time. The expected behaviour is that the credits field would be incremented by MongoDB for each call without any WriteConflicts but unfortunately it throws an error.

I believe this is because Prisma is internally is using Transactions with MongoDB which fails whenever two clients touch the same document no matter if it’s using atomic updates or not.

TL;DR; We migrated all our db operations to Mongoose as a solution and did not receive a single WriteConflict. Also as a welcome side effect, db operations started taking a lot shorter.

I couldn’t find a solution to this with Prisma and unfortunately it was a mess when we went live and users started coming. Although prisma says that they support Mongo, the level of support is far from what anyone would need in production. RDBMS support could be better, but I don’t know I haven’t tried it.

The main issue here is, I think, that Prisma uses their own driver instead of using mongo’s own driver. (I stand corrected by @codeinearts, prisma uses official mongo drive for Rust, which in contrast with official node.js driver, uses transactions for everything. See his actual comment here: https://github.com/prisma/prisma/issues/12814#issuecomment-1339376488 ) And it’s a mistake on our side to choose Prisma over other options, as they didn’t claim to have 100% feature parity with official node.js mongo driver.

I have gotten this error in one endpoint of an express application. Happened for a simeple prisma.<model>.update() call.

Error: 
Invalid `prisma.classlists.update()` invocation:
Transaction failed due to a write conflict or a deadlock. Please retry your transaction
    at RequestHandler.handleRequestError (/home/pacs/mak03/users/freigeistapp/freigeist/freigeist_server/node_modules/@prisma/client/runtime/index.js:34843:13)
    at RequestHandler.handleAndLogRequestError (/home/pacs/mak03/users/freigeistapp/freigeist/freigeist_server/node_modules/@prisma/client/runtime/index.js:34815:12)
    at RequestHandler.request (/home/pacs/mak03/users/freigeistapp/freigeist/freigeist_server/node_modules/@prisma/client/runtime/index.js:34810:12)
    at async PrismaClient._request (/home/pacs/mak03/users/freigeistapp/freigeist/freigeist_server/node_modules/@prisma/client/runtime/index.js:35900:16)
    at async file:///home/pacs/mak03/users/freigeistapp/freigeist/freigeist_server/src/controllers/classes.controller.js:99:17

The express handler basically looks like this

export const updateClass = asyncHandler(async (req, res) => {
  const userId = getUserIdFromRequest(req);
  const schoolyear = Number(req.body.schoolyear);

  const classlist = await prisma.classlists.update({
    where: { user_id: userId, id: req.body._id },
    data: { title: req.body.title },
  });

  res.json({
    success: true,
    message: "Classes fetched successfully",
    classlist,
  });
});

so I can’t imagine the conflict coming from within this handler. Does this mean another request handler must be interfering in an asynchronous way? Does anyone know why prisma enforces transactions for simple updates?

From what I understand, and to answer your first question, yes, that should mean another request handler was at that moment interfering. This might or might not be desired:

a) if two users edit a document title simultaneously, you might NOT WANT to reflect the latest update if the last one didn’t finished

b) if two users add one like to a document, you might not care of the order as long as the sum is correct

In the case ‘b’ a writeConflict would be very prone to happen if not handled atomically and correctly (see https://www.prisma.io/docs/reference/api-reference/prisma-client-reference#atomic-number-operations)

As per your second question, is made like that by design and I have no idea, tho I think is kind of a rule of thumb to avoid certain scenarios like case ‘a’ mentioned above. Also, from a control point of view, It also does make sense to have visibility of the actual ‘underlying error’ instead of having an ‘automagically’ retry logic doing the work for us - although, highly desired IMHO -

As per the root cause, the reason is mongodb driver automatically handles retries for us in most cases, except when transactions are involved. As @Jolg42 shared above, this video explains the behaviour of write conflicts at mongodb very well https://www.youtube.com/watch?v=3x_Pf9rQGCo&t=723s

Since prisma uses transactions by default on everything, we get this ‘writeConflict’ error bubble up to end up slapping our faces

In your particular case, I’d not worry too much. However, you could try https://www.prisma.io/docs/concepts/components/prisma-client/transactions if needed. I’m not sure how it is implemented, but it might enclose the underlying error with some retry logic or similar - Although, overkill IMHO -.

about prisma engine

Prisma uses something called Tokio runtime, which is basically a Rust package made to efficiently handle async I/O (kind of JS V8 on steroids lol). That uses then Rust’s mongodb driver behind the scenes to make the actual connection to the database, which apparently neither does implement auto retries for transactions (nodejs mongodb driver implement that using session.withTransaction)

This behaviour could led to other possible potential issues when dealing for example with serverless databases like the one mongodb offers at atlas, as it depends on its retry behaviour in case of sockets close

More on mongodb retry behaviour can be read here https://www.mongodb.com/docs/manual/core/retryable-writes

@jcampbell05 MongoDB recommends retrying those transactions until they succeed. Their MongDB driver on NodeJS does this automatically using session.withTransaction. If Prisma copies this logic and retries the transactions, it could solve the problem. https://www.mongodb.com/docs/manual/core/retryable-writes/

This is still a huge issue for us, and it’s very consistent; updating the same model multiple times in a short period of time causes this error Transaction failed due to a write conflict or a deadlock. Please retry your transaction we are stuck with this issue for now, is there a workaround or an option with work with Prisma without enabling replica set?

@Weakky Amazing we will give a go 😀 I think the only thing left would be smart queuing of updates which only do atomic updates.

In this situation for high volume workloads a retry we just cause requests to back up. We had to re-architected our application to avoid this by queuing updates based on which document is being updated but that it overkill for simple applications.

Since MongoDB already queues updates outside of a transaction maybe it may be worth allowing us to update a document outside of a transaction. Only if we are using atomic updates.

Mongo’s current behaviour with Prisma currently results in a conflict error if two atomic updates touch the same document within a transaction. This is to stop a document being left in an inconsistent state but the whole purpose of atomic updates is that they can touch the same document and it will be updated correctly without leaving it in an inconsistent state.

Prisma could implement retries by default as this is just adding extra code to write if using Prisma 😗

That is expected for upserts and can be handled by retrying the query: https://www.prisma.io/docs/reference/api-reference/prisma-client-reference#remarks-8 This is a different problem from what this issue here is about though @luluhoc

We can confirm this issue. We sometimes receive the below error. In our case we think that happens when the same object is searched and updated for at least twice in a very short amount of time. It could also be related to stopping of the current (Prisma update) request and leaving it in a unfinished (bad?) state through a 2nd request (new one) from the same client.

Invalid `this.prismaService.user.update()` invocation in
/Users/pgm/node/src/modules/user/users.repository.ts:49:51

  46 // UPDATE
  47 
  48 async updateToken(userID: string, token: string): Promise<UserEntity> {
→ 49   const newUser = await this.prismaService.user.update(
  Error occurred during query execution:
ConnectorError(ConnectorError { user_facing_error: None, kind: RawDatabaseError { code: "unknown", message: "Command failed (WriteConflict): WriteConflict error: this operation conflicted with another operation. Please retry your operation or multi-document transaction.)" } })

This WriteConflict issue is really hitting us hard. We went to production with Prisma and we keep getting these issues and it’s extremely difficult to debug or find out about what caused the WriteConflict.

We have a simple code:

const tokenResult = await verifyToken(
  code,
)

if (tokenResult.err) {
  return new Err(INVALID_TOKEN_PROBLEM)
}

const userUpdate: Prisma.UserUpdateInput = {
  emailVerified: true,
  email,
}

userUpdate.temporary = false
userUpdate.temporaryEmail = null
const userGroups = new Set(tokenResult.val.user!.groups)
userGroups.add(GROUPS.MEMBER)
userUpdate.groups = [...userGroups]

const user = await prisma.user.update({
  where: {
    id: tokenResult.val.user!.id
  },
  data: userUpdate
})

// delete the previous token
await clearToken(tokenResult.val.id)

It simply fetches the token, then updates the user and clears the token. Prisma fails at the user update with a WriteConflict error.

If only prisma provided more context about the issue or would internally retry a failed transaction a couple of times before failing.

• Have you ever been bitten by those automatic retries while using Mongoose or any Mongo drivers? No we didn’t have any issues, we’re using MongDB in production (driveflux.com) and we didn’t face any issues. The solution I posted above worked for us and all WriteConflict issues disappeared. We didn’t notice any bad side effects.

• Also, have you ever needed to disable those automatic retries? Not at all

• Automatic retries enabled by default (put behind a preview feature flag until we GA it) Yes

Thank you @Weakky

@luluhoc you gotta credit mate.

@janpio , can we implement a similar solution that’s more optimised in prisma? This one has a cooldown time, it works well, but not the fastest.

Yeah, sorry I thought I responded with quote, I’ll edit the comment.

witched to Mongoose, because of this error and haven’t looked back. It works like a charm and is a simple technology compared to Prisma (Not as many moving parts). Albeit it does not have as many fancy features as Prisma. But in my experience it is plenty sufficient.

Interesting are you using Typescript ?

I struggled to get Mongoose working in our Typescript Application as it’s quite a big change unfortunately. So one option I’ve been thinking about is if there is a way I could build my own generator to spit out the mongoose code from the Prisma Schema.

#5764

Yes, this is how we use Typescript with Mongoose. It is quite effective:

import mongoose, {InferSchemaType} from 'mongoose';

const userSchema = new mongoose.Schema({
  name: { type: String, required: true }
});

type User = InferSchemaType<typeof userSchema>;
export default mongoose.model<User>('Category', userSchema);

It makes any operation with the model conform to the proper types.

I believe that Prisma must retry the transaction a couple of times before failing. That’s what the documentation in MongoDB says.

A workaround is to use a try catch and retry the query, however, it’ll be horrible to wrap every write with a try catch throughout the code.

Alright dpetrick, so after switching from Promise.all to a for loop, I didn’t get the WriteConflict issue. But this is a bug in my opinion. Primsa should be able to handle this situation.

iam facing this issue, i did a simple for loop for up to 1000 promises, and do a promise.all , i get deadlock error. i did not use any transaction and this is a mongodb driver. using “@prisma/client”: “^5.0.0”,

saveLinks.push((async () => {
          const existing = await prisma().crawler.findFirst({
            where: {
              siteId,
              moduleId: crawler.moduleId,
              moduleSourceId: crawler.moduleSourceId,
              url: {
                in: [completeURL.replace('https://', 'http://'), completeURL.replace('http://', 'https://')],
              }
            }
          })
          if (existing !== null) {
            return null
          }

          const res = await prisma().crawler.create({
            data: {
              siteId,
              articleId: crawler.articleId,
              moduleId: crawler.moduleId,
              moduleSourceId: crawler.moduleSourceId,
              url: completeURL,
              depth: crawler.depth + 1,
              baseUrl,
              status: 0,
              content: '',
              parentId: crawler.id,
            }
          })
          return res
        })())

@janpio hard to say as our mitigations are in place. But we haven’t seen it in a while.

Most of our remaining issues are random issues such as unexpected hangups and SSL issues with the hosted Mongo which I’m not sure are an issue with Prisma or just the random issues you get at scale.

• Have you ever been bitten by those automatic retries while using Mongoose or any Mongo drivers? Never when using mongoose

• Also, have you ever needed to disable those automatic retries? no

• Automatic retries enabled by default (put behind a preview feature flag until we GA it) Yes

Thanks @Weakky

• Have you ever been bitten by those automatic retries while using Mongoose or any Mongo drivers?

Never

• have you ever needed to disable those automatic retries?

No

• Automatic retries enabled by default (put behind a preview feature flag until we GA it)

Yes thank you

@emrah-b Not a prisma.io evangelist here, but some of what you state is wrong: prisma actually uses Rust’s official mongodb’s native driver, and that can be seen here https://github.com/prisma/prisma-engines/blob/main/query-engine/connectors/mongodb-query-connector/Cargo.toml

The problem is that it is using transactions for everything, which end up giving writeConflicts for some dead simple operations, which is the correct behaviour of mongodb’s native driver when using transactions.

Those “dead simple operations” would benefit a lot of the regular behaviour we all mongo db users native driver are used to: mongoclient to automatically retrying the queries for us.

Also, and I’ll agree in that’s sadly something true, is that in order to keep consistency with their api among all other dbms, it forces mongodb to use aggregate pipelines - kind of joins on mongo lol - and transactions, which could be the reason you’re experiencing a difference on performance compared to mongoose - after all, on most of the writes with mongoose you’re probably not having the overhead of aggregation pipelines -.

I have gotten this error in one endpoint of an express application. Happened for a simeple prisma.<model>.update() call.

Error: 
Invalid `prisma.classlists.update()` invocation:
Transaction failed due to a write conflict or a deadlock. Please retry your transaction
    at RequestHandler.handleRequestError (/home/pacs/mak03/users/freigeistapp/freigeist/freigeist_server/node_modules/@prisma/client/runtime/index.js:34843:13)
    at RequestHandler.handleAndLogRequestError (/home/pacs/mak03/users/freigeistapp/freigeist/freigeist_server/node_modules/@prisma/client/runtime/index.js:34815:12)
    at RequestHandler.request (/home/pacs/mak03/users/freigeistapp/freigeist/freigeist_server/node_modules/@prisma/client/runtime/index.js:34810:12)
    at async PrismaClient._request (/home/pacs/mak03/users/freigeistapp/freigeist/freigeist_server/node_modules/@prisma/client/runtime/index.js:35900:16)
    at async file:///home/pacs/mak03/users/freigeistapp/freigeist/freigeist_server/src/controllers/classes.controller.js:99:17

The express handler basically looks like this

export const updateClass = asyncHandler(async (req, res) => {
  const userId = getUserIdFromRequest(req);
  const schoolyear = Number(req.body.schoolyear);

  const classlist = await prisma.classlists.update({
    where: { user_id: userId, id: req.body._id },
    data: { title: req.body.title },
  });

  res.json({
    success: true,
    message: "Classes fetched successfully",
    classlist,
  });
});

so I can’t imagine the conflict coming from within this handler. Does this mean another request handler must be interfering in an asynchronous way? Does anyone know why prisma enforces transactions for simple updates?

@Kondamon James (@jcampbell05) is also a user of Prisma and MongoDB, who implemented that themselves. This is currently not included in Prisma.

@jcampbell05 Thanks! I need to organize some thoughts first about what this is about exactly. So we have a better understanding of it, from looking at all the comments it’s very much needed!

Early thought (work in progress): from a “first look”, it looks like it’s about how MongoDB works with transactions/writes, and the behavior is different depending on if you use writes and writes inside a transaction. (Prisma always uses a transaction)

https://www.mongodb.com/docs/manual/core/transactions-production-consideration/#in-progress-transactions-and-write-conflicts

In-progress Transactions and Write Conflicts If a transaction is in progress and a write outside the transaction modifies a document that an operation in the transaction later tries to modify, the transaction aborts because of a write conflict.

If a transaction is in progress and has taken a lock to modify a document, when a write outside the transaction tries to modify the same document, the write waits until the transaction ends.

This video explains it very well https://www.youtube.com/watch?v=3x_Pf9rQGCo&t=723s

@Jolg42 let me know if there is anything I can do to help you with this issue

apparently this still not solved … switching to mongoose

This error occurred when I ran a script to migrate data from my old database to MongoDB. Writing data + Promise.all will 100% cause it. The version I use is 4.1.1.

import { Database } from "$lib/nodejs/replit-db-esm"
import { PrismaClient } from '@prisma/client'

const db = new PrismaClient().proxyServer

const oldDb = new Database()

const proxyServers = await oldDb.get("PROXY-SERVERS")
const chunkSize = 2
for (let i = 0; i < proxyServers.length; i += chunkSize) {
    const serverChunk = proxyServers.slice(i, i + chunkSize)
    const promises = serverChunk.map(proxyServer => db.upsert({
        where: {
            ip: proxyServer.ip,
        },
        create: {
            ip: proxyServer.ip,
            port: Number(proxyServer.port),
            protocol: proxyServer.type,
            country: proxyServer.country,
            username: proxyServer.userPass.split(':')[0],
            password: proxyServer.userPass.split(':')[1],
            usages: {
                connectOrCreate: (proxyServer.usedFor || []).length
                    ? proxyServer.usedFor.map(el => ({ where: { name: el }, create: { name: el } }))
                    : [{ where: { name: "none" }, create: { name: "none" } }]
            }
        },
        update: {}
    }))
    await Promise.all(promises)
}

P/s: I replaced Promise.all with prisma.$transaction then the script is working like a charm now.

For me, it’s also happening, but differently and I’m trying everything to work around it.

I have a seeder function that populates the db with fake data, here’s how we’re doing it:

const sleep = () => new Promise((resolve) => setTimeout(() => resolve(true), 100))
async function* start () {
  
  // @ts-expect-error
  for(const model of prisma._dmmf.datamodel.models) {
    if(camelCase(model.name) in prisma) {
      // @ts-expect-error
      await prisma[camelCase(model.name)].deleteMany({})
    }
  }
  yield 5 //5%
  await sleep()

  // Make 5 random bananas
  await Promise.all(arr(5, () => seedBanana()))
  yield 10 //10%
  await sleep()

  // Make 10 random apples
  await Promise.all(arr(10, () => seedApple()))
  yield 25 //25%
  await sleep()

  /// ....etc.
  yield 100 //100%
}

const seedBanana = async () => {
  return await prisma.banana.create({
    data: {
      id: randomID(),
      name: faker.word.noun(),
      // this is to show that sometimes during seed, even if there's a seedApple function,
      // this seedBanana function creates apples as well, if this may help debugging.
      friendApple: { 
        create: {
          id: randomID(),
          name: faker.word.noun(),
        }
      },
      tree: {
        create: {
          id: randomID(),
          type: 'bananaTree'
          /// ...etc.
        }
      }
    }
  })
}

After creating multiple seeders, when we run this script on terminal, we always have a WriteConflict issue somewhere during the seeding. Not particularly at a certain point, but randomly during any seeding function. So sometimes, at 5%, sometimes all the way to 90% and it fails.

As you can see, we tried to add a sleep function to allow some time before the next seeding, but it doesn’t work.

I have fixed this issue as well, the solution was to use MongoDB to create the collections before seeding.

The problem is, when the database is empty and prisma has to seed data and create the collections at the same time, it will result in WriteConflict error, so here’s my solution:

async function* start () {
  
  const client = new MongoClient(process.env.DATABASE_URL!)
  await client.connect()
  const collections = await client.db().listCollections().toArray()
  // @ts-expect-error
  const models = prisma._dmmf.datamodel.models
  for(let m of models) {
    if(m.dbName) {
      if(!collections.find((c) => c.name === m.dbName)) {
        await client.db().createCollection(m.dbName)
      }
      
      await client.db().collection(m.dbName).deleteMany({})
    }
  }
  await client.close()

  // Rest of seeders code
}

If you can help us to get to a reproduction with that new knowledge, we will certainly take a look.

I start to suspect it’s Promise.all. The modified mitt package does a Promise.all(handlers) which is not running handlers serially and maybe 2 queries fire at the same time.

@chrisschaub Do you have a somewhat reliable reproduction, or maybe some load parameters to make it likely to run into this problem?

I have a DateTime field in my schema. If I comment out updating this field, the write errors stop. I’ve tried passing an ISO date string, and a JS date/time object. The errors are intermittent until I comment out this field. I’m just inserting 100 records at a time. I’ve tried each insert as a separate transaction, and not.

Also, @janpio , if I remove the deleteMany part, then it works unreliably. After multiple times re-running the seeder, it worked. So for every ~20 times running the command, one will work.

I expect we will face this issue with the migration of our data (not schema migration) as well. We’re going to migrate from our old database to a new DB with a new structure soon, and the migration script will run in a similar fashion.

So “creating multiple seeders” just means that I added more “seedXXX” functions.

Here’s a reproduction repo:

https://github.com/aladinflux/prisma-conflit-issue

The seeder code is inside prisma folder.

Just run npx prisma db seed after setting the DATABASE_URL env.

Thanks for the additional information. Can you share the connected schema so we have an easier time to run this ourselves? What exactly do you mean by “After creating multiple seeders, when we run this script on terminal, we …” - what are the exact steps for us to reproduce what you observe?