Thanks for the extra info @BrianJM. I’ll take another look into this tomorrow.

Hey @iamacup!

@DanRibbens i have created a ‘minimal’ repo and shared it with you directly and am happy for you to share inside Payload - iamacup/payload-transactions - please read the readme.

I pulled your configuration into a new npx create-payload-app blank template. Connected to a mongodb Atlas instance to reproduce the errors. With #5156, I’ve packed payload on main and copied it over to the test repo and the error is no longer occuring when I save new

You can read the description, but what was happening was a race condition in the access function where the more fields you have, the more likely it was to error.

There are still improvements to be made on transactions overall, as I am looking at feedback posted here: https://github.com/payloadcms/payload/issues/4078#issuecomment-1960861769

Thanks everyone contributing to the conversation! This is high priority for us to get right.

How do you disable transactions altogether? I can’t seem to find it in the docs.

This is not yet an option, but there is a PR (#4467) in the works.

I patched @payloadcms/db-mongodb/dist/index.js with the following:

- const _beginTransaction = require("./transactions/beginTransaction");
- beginTransaction: _beginTransaction.beginTransaction,
+ beginTransaction: undefined,

Hi @DanRibbens

Thank you very much for your quick answer.

Sorry but I don’t get you. I’m using a map to get my seed data (ex. getFishersData) but it’s synchronous and don’t write to Payload. And my Payload writing function createCollectionItems is using a for loop rather than a map, and is awaiting each Promise to resolve before the next create operation. So I don’t think that the transaction errors are related to this SO thread.

Did I miss something ? 🤔

Hi @DanRibbens

You still have access to my repo if you want to reproduce easily. The seed errors are definitely not generated by poorly handled async map. All the payload.create are awaited properly.

Thanks @DanRibbens - your efforts are much appreciated, I know how nebulous this one is!

A support rep at MongoDB Atlas pointed me to this related issue with the MongoDB Node.js driver. The rep said it was actively being looked into.

@BrianJM

Do you want me to open a new issue for the duplicate network requests?

Yes that is a good call!

I’ve gone and pruned out every single field apart from blocks and i am still getting the error reliably and instantly starting with no data - by creating a single record - my structure as you have seen is a bunch of nested blocks - i am certain somehow its getting messy down in the code when you get to a big nesting.

https://github.com/iamacup/payload-transactions/pull/1

I can remove the error entirely if i just chop the block structure off half way down.

edit: what i have said there is clearly a lie there is an array as well - brb removing

edit2: removing the array did not fix it - it is now just blocks in the project

@sandwit, You can disable all transactions by setting transactionOptions: false in the db adapter settings which will resolve your issues for now.

@iamacup What’s in allProductDataRequirementBlocks?

I SUSPECT it is to do with the media collection and uploads but i will do more today to see if i can get this more minimal. EDIT: i have since removed the media collection entirely and the error is still there, in the example there are no relationship fields now.

I have shared it directly with you now @BrianJM. it ain’t pretty though 😃

@DanRibbens i have created a ‘minimal’ repo and shared it with you directly and am happy for you to share inside Payload - https://github.com/iamacup/payload-transactions/ - please read the readme.

I am using the word ‘minimal’ because while it is simple in terms of collections and payload config (no plugins etc.) it’s got a bunch of blocks and the error is coming from there.

video here: https://drive.google.com/file/d/1ziwb8tCrkRKgqIk0Zuj-fsQe6AztKNdR/view?usp=sharing

it’s causing both:

• Attempted illegal state transition from [TRANSACTION_ABORTED] to [TRANSACTION_ABORTED]
• Given transaction number n does not match any in-progress transactions. The active transaction number is n-1

reliably

I SUSPECT it is to do with the media collection and uploads but i will do more today to see if i can get this more minimal.

EDIT: i have since removed the media collection entirely and the error is still there, in the example there are no relationship fields now.

const db = mongooseAdapter({
  url: process.env.DATABASE_URI,
});

export default buildConfig({
  admin: {
    user: Users.slug,
    bundler: webpackBundler(),
  },
  editor: lexicalEditor({}),
  collections: [Users, BrokenCollection],
  localization: {
    locales: [
      { label: "en", code: "en" },
      { label: "fr", code: "fr" },
    ],
    defaultLocale: "en",
    fallback: true,
  },
  typescript: {
    outputFile: path.resolve(__dirname, "payload-types.ts"),
    declare: false,
  },
  graphQL: {
    schemaOutputFile: path.resolve(__dirname, "generated-schema.graphql"),
  },
  db,
});

import { CollectionConfig } from "payload/types";
import { allProductDataRequirementBlocks } from "../../../blocks/dataRequirements/productData/all";

const BrokenCollection: CollectionConfig = {
  slug: "brokencollection",
  access: {
    read: () => true,
  },
  fields: [
    {
      name: "data",
      type: "blocks",
      minRows: 1,
      required: true,
      maxRows: 1,
      blocks: allProductDataRequirementBlocks,
    },
  ],
};

export default BrokenCollection;

@iamacup @denolfe @DanRibbens I am seeing the exact same issue, and have confirmed these recommendations.

I regularly receive these errors (latest payload) interacting with the UI, which causes the app to crash within minutes of use. These application crashed and errors occur on collections that have relationship fields, afterRead hooks (i.e., virtual fields), and/or filterOptions.

MongoServerError: Transaction with { txnNumber: 8 } has been aborted

MongoRuntimeError: Attempted illegal state transition from [TRANSACTION_ABORTED] to [TRANSACTION_ABORTED]

MongoServerError: Given transaction number 13 does not match any in-progress transactions. The active transaction number is 12

This seems like a race condition of sorts, as collections with more relationship fields and afterRead hooks consistently and more regularly cause application crashes.

The issues started when switching from a local mongodb instance to Atlas (free tier), during the last round of testing before launching an MVP.

@denolfe @DanRibbens i was previously getting these errors on project startup for 2.11.0, moving to 2.11.1 has fixed the startup errors but i still am able to produce them when loading a collection in the admin panel.

I have a video here: drive.google.com/file/d/1kR3DK3kdb0YyUcPwKxqppjBfpVS8iQAa/view?usp=sharing - do you have any guidance on how it might be possible to debug? Like the OP this is quite deep in the project logic - for me however this collection has no hooks or any logic.

I also have no filterOptions in the project at all.

(please note in the video, these errors are only when running with a replicaset, they do not happen if i just run mongo basic)

We have been using that flag since it’s available on mana.wiki for about a month. Nothing is broken yet!

The main problem would be we won’t unearth transaction race condition issues in larger production apps for PayloadCMS.

thanks @xHomu we have done this and its working, sorry to ask but do you know what the implications of doing this are in production?

@iamacup Not ideal, but we disabled transactions altogether with transactionOptions: false docs, so we can focus on building out our app.

@DanRibbens I just sent you an invitation to a repository, so you can reproduce. Just do npm run seed https://github.com/franckmartin/payload-mongo-transaction

I don’t know about others situations/projects, but it would be pretty difficult for me to create a reproduction. I think this only happens with a certain level of complexity/relationships/hooks that would be very difficult to simulate. I know that isn’t helpful, but it’s not for lack of trying! I also haven’t been able to reproduce in reasonably simple sandboxes, but I’m certain there’s an issue 😆

I think you’re instinct here @DanRibbens is correct re an error/abort during transaction. I’m hesitant to speak to this because I have very cursory understanding of node process management / workers / tasks (or even the payload hook system) but I think the single biggest development difficulty in Payload is lack of transparency in errors. Again I have no idea if that is fundamental to node async tasks so not necessarily a knock on Payload, but there does seem like there’s a huge swatch of errors like this that just get swallowed and if they were easily surfaced could significantly improve debugging.

I need to switch gears here, but I have been able to reliably reproduce it and more importantly reliably avoid it. I can’t say what I’ve found is super promising or helpful, but here it is.

I commented out data generators for my posts seeds and one by one uncommented and re-run seeds. What I determined was when I generated data for a relationship field I got the transaction error. If I commented out that generator, the error went away (I repeated this many times to make sure this was in fact triggering it). It’s also worth noting I experimented a bit with some other relationship fields and some of them always work and some of them never work. I can’t find any pattern, some relationships to upload collections work, others do not, some relationships to other non-upload collections work, others do not. I was even able to reproduce it in admin and this is where it gets even weirder:

I kept clicking Save over and over and the transaction id was always off by 1 and I kept getting that error, and then on the 9th time it saved successfully! It only happens on new posts, if I select that seed file on any other existing post I never get an error on save…