prisma: After upgrading to Prisma 3.10 from 3.9, remix site crashes on connection (Codesigning issue on M1 Macs)

I experienced something similar though with a Next.js application - I wasn’t able to put together a reproduction as my clean repo didn’t crash the process, so I decided to try deleting all .prisma generated directories before running prisma generate again, and that worked!

I have a monorepo, so I located all .prisma directories with find . -name .prisma, then deleted each one.

So at least for me, the issue manifested when I already had a generated client from 3.9, and cleaning up the old version before running generate again appears to have fixed it.

@pantharshit00 do you have an M1 Mac? From my understanding this specifically triggers on darwin-arm64.

The steps to reproduce that work for me, if useful:

1. Create a package.json with the following:

   {
     "dependencies": {
       "@prisma/client": "3.9.x",
       "prisma": "3.9.x"
     }
   }
   

2. Run yarn install && yarn prisma generate
3. Try using Prisma and notice that it works
4. Modify the package.json to the following:

   {
     "dependencies": {
       "@prisma/client": "3.10.x",
       "prisma": "3.10.x"
     }
   }
   

5. Run yarn install && yarn prisma generate
6. Try using Prisma and notice that the kernel immediately kills the node process

Might be superfluous since the cause seems to have been found by @jacobwgillespie above, but can confirm the issue also happens with SvelteKit (so not only Remix) after updating Prisma to 3.10.

Attempting to connect after starting a dev server ends the process with SIGKILL.

I am also on M1 and I did run prisma generate after update. Removing ./node-modules/.prisma and rerunning prisma generate solved the issue.

I still have a copy of my test files, here’s some zips:

• Installing 3.9.x first, then 3.10.x: https://drive.google.com/file/d/1lAQbSArGgQAiGdHD5QXRqo9aGZCSoalX/view?usp=sharing
• Only installing 3.10.x: https://drive.google.com/file/d/1WOoxtjxdQrYcNjLKRsj6VtwjmX7LjJiG/view?usp=sharing

You can see the leftover esm directory in the first zip…

One of the things that lead to the finding above was that if I made a copy of the project the copy worked. As in, if I upgraded from 3.9 to 3.10, the project was broken, but if I cp -r node_modules to another directory it worked. That seemed to imply that it wasn’t actually the contents of the files, but something about their metadata instead, which lead to investigating the kernel error messages.

@janpio fascinating, this is an interesting one, I was able to reproduce! For extra context, I did run prisma generate after upgrading to 3.10, which did not fix the issue until I deleted the .prisma/client directory.

So the issue is macOS code signing on M1 Macs!

Specifically, code signing breaks when replacing a binary with the contents of a new binary, for example you can reproduce this via the following:

$ cp /usr/bin/false ./example
$ ./example # works
$ cp /usr/bin/true ./example
$ ./example # killed by the kernel

By creating the example binary as a copy of the first binary, the binary is considered safe, but when replacing it with the copy of a different binary, it’s considered unsafe.

I believe that’s what is happening here, when upgrading from 3.9 to 3.10, the file node_modules/.prisma/client/libquery_engine-darwin-arm64.dylib.node is replaced with the 3.10 version, but since its path / inode is the same as before, it is considered no longer code signed by the macOS sandbox.

This manifests as the following log lines in Console:

default	22:54:01.120624+0000	kernel	CODE SIGNING: cs_invalid_page(0x118000000): p=70474[node] final status 0x23010204, denying page sending SIGKILL
default	22:54:01.120635+0000	kernel	CODE SIGNING: process 70474[node]: rejecting invalid page at address 0x118000000 from offset 0x0 in file "/Users/jacobwgillespie/Code/jacobwgillespie/primsa-3-10-issue/3-9-then-3-10/node_modules/.prisma/client/libquery_engine-darwin-arm64.dylib.node" (cs_mtime:1645742717.771967254 != mtime:1645742737.976117428) (signed:1 validated:1 tainted:1 nx:0 wpmapped:0 dirty:0 depth:0)
default	22:54:01.120702+0000	kernel	node[70474] Corpse allowed 1 of 5
default	22:54:01.169387+0000	ReportCrash	Formulating fatal 309 report for corpse[70474] node
default	22:54:01.171430+0000	ReportCrash	no MetricKit for process node type 309 bundleId (null)
default	22:54:01.171521+0000	ReportCrash	Sending event: com.apple.stability.crash {"exceptionCodes":"0x0000000000000032, 0x0000000118000000(\n    50,\n    4697620480\n)EXC_BAD_ACCESSSIGKILL (Code Signature Invalid)UNKNOWN_0x32 at 0x0000000118000000","incidentID":"15D796F0-C20F-41A4-92A4-69DA30481984","logwritten":0,"process":"node","responsibleApp":"iTerm2","terminationReasonExceptionCode":"0x2","terminationReasonNamespace":"CODESIGNING"}

Note the cs_mtime:1645742717.771967254 != mtime:1645742737.976117428 on the rejection, the file mtime at initial creation (v3.9) no longer matches the current mtime after upgrade (v3.10)

I think the potential fixes would be:

1. Delete the .dylib.node file in prisma generate before generating the new client (or delete the whole .prisma/client for that matter)
2. Put the Prisma version in the filename, so that on version upgrade, you’re guaranteed to get a new inode for the binary (libquery_...3.10.0.dylib.node, etc)

Another benefit to (1) is it would clean up old files, I also discovered that after upgrading from 3.9 to 3.10 a leftover .prisma/client/runtime/esm directory was still present from 3.9, but that turned out to be a red herring. But if Prisma cleaned its client directory before generate, that would potentially also clean up leftovers like esm.

Credit to https://openradar.appspot.com/FB8914243 for the simple reproduction of the issue above.

Got the same issue when upgrading Prisma from version 3.14 to 3.15. I’m on a MacBook Pro (14-inch, 2021) with an M1 Pro chip.

Remix builds the site, and starts an express server. The Prisma connection is managed through this code

export { PrismaClient as DbClient } from "@prisma/client";

let db: DbClient;

declare global {
    // eslint-disable-next-line no-var
    var __db: DbClient | undefined;
}

// this is needed because in development we don't want to restart
// the server with every change, but we want to make sure we don't
// create a new connection to the DB with every change either.
if (process.env.NODE_ENV === "production") {
    db = new DbClient();
    db.$connect();
} else {
    if (!global.__db) {
        global.__db = new DbClient();
        global.__db.$connect();
    }
    db = global.__db;
}

export { db };

which is form the Remix documentation (https://remix.run/docs/en/v1/tutorials/jokes#connect-to-the-database). As far as I can tell, it deadlocks when trying to connect to the database.

I have unfortunately not had the time to attempt a reproducing example, but I hope to do so. I wanted to create a bug before I had a reproduction with enough description that others could find it if they encounter the same issue.