prisma: Suspected memory leak in Lambda function after upgrading from 4.10.1
Bug description
We recently came across VT13001: [BUG] in scatter query whilst trying to run the db:push and other similar commands. Based on this ticket, we decided to upgrade our prisma version. The issue disappeared after upgrade but we started noticing our lambda functions crashing. From the lambda function logs:
Error: Runtime exited with error: signal: killed Runtime.ExitError
REPORT RequestId: xx Duration: 6174.19 ms Billed Duration: 6175 ms Memory Size: 850 MB Max Memory Used: 850 MB
The lambda function has always been configured to use 512 MB so we decided to upgrade to 850 MB in case the upgrade required more memory. But that didn’t help.
Pls see below for more info. We’re kind of in a limbo now; we cannot make any more schema changes due to the VT13001 bug above and we cannot upgrade to v5 due the lambda crashes. We do not need to upgrade to v5. If we have a fix for the VT13001 bug, we’re happy to stay on v4.10.1 for now.
Thank you.
How to reproduce
Upgrade to 4.15.0, 5.0.0 or 5.1.1 and create tables similar to the ones provided below.
Expected behavior
We expect it to work like it did in v4.10.1.
Prisma information
generator client {
provider = "prisma-client-js"
previewFeatures = ["filteredRelationCount"]
binaryTargets = ["rhel-openssl-1.0.x", "native"]
}
datasource db {
provider = "mysql"
url = env("DATABASE_URL")
relationMode = "prisma"
}
Not all queries/tables cause it to crash. Some queries are ok but others just crash. An example of such table:
model Foo {
id Int @id @default(autoincrement())
uid String @unique @db.Char(26)
name String @db.VarChar(100)
createdDate DateTime @default(now())
deleted Boolean @default(false)
deletedDate DateTime?
tableAMapping TableAMapping[]
tableBMapping TableBMapping[]
owner User @relation(fields: [ownerId], references: [id], onDelete: Cascade)
ownerId Int
fooType FooType @relation(fields: [fooTypeId], references: [id])
fooTypeId Int
anotherTableCredentials anotherTableCredentials? @relation(fields: [anotherTableCredentialsId], references: [id])
anotherTableCredentialsId Int?
## MORE FIELDS: Another 19 fields similar to anotherTableCredentials and anotherTableCredentialsId
@@index([name])
@@index([ownerId])
@@index([fooTypeId])
@@index([anotherTableCredentialsId])
## MORE INDEX: corresponding to the 19 fields mentioned above
@@index([uid])
}
A query such as:
await prisma.foo.findMany({
where: {
fooType: {
uid: request.fooTypeUid,
},
ownerId: user.id,
deleted: false,
name,
},
}),
or even:
await prisma.foo.count({
where: {
ownerId: user.id,
deleted: false,
},
}),
would fail.
Some other queries that use findMany seem to be ok.
Environment & setup
- OS: Amazon Linux 2 x86_64
- Database: MySQL
- Node.js version: 16.18
Prisma Version
It affects multiple versions 4.15.0, 5.0.0 & 5.1.1.
prisma : 5.1.1
@prisma/client : 5.1.1
Current platform : darwin-arm64
Query Engine (Node-API) : libquery-engine 6a3747c37ff169c90047725a05a6ef02e32ac97e (at node_modules/@prisma/engines/libquery_engine-darwin-arm64.dylib.node)
Schema Engine : schema-engine-cli 6a3747c37ff169c90047725a05a6ef02e32ac97e (at node_modules/@prisma/engines/schema-engine-darwin-arm64)
Schema Wasm : @prisma/prisma-schema-wasm 5.1.1-1.6a3747c37ff169c90047725a05a6ef02e32ac97e
Default Engines Hash : 6a3747c37ff169c90047725a05a6ef02e32ac97e
Studio : 0.492.0
About this issue
- Original URL
- State: closed
- Created 10 months ago
- Comments: 17 (9 by maintainers)
Commits related to this issue
- qe: fix prisma/20799 take 2 The quaint AST lets users keep track of unique index definitions in tables in order to generate `MERGE` statements in SQL Server. The SQL query connector diligently adds ... — committed to prisma/prisma-engines by tomhoule 10 months ago
- qe: fix pathological behaviour with many @uniques The quaint AST lets users keep track of unique index definitions in tables in order to generate `MERGE` statements in SQL Server. The SQL query conn... — committed to prisma/prisma-engines by tomhoule 10 months ago
- qe: fix pathological behaviour with many @uniques The quaint AST lets users keep track of unique index definitions in tables in order to generate `MERGE` statements in SQL Server. The SQL query conn... — committed to prisma/prisma-engines by tomhoule 10 months ago
- qe: fix pathological behaviour with many @uniques The quaint AST lets users keep track of unique index definitions in tables in order to generate `MERGE` statements in SQL Server. The SQL query conn... — committed to prisma/prisma-engines by tomhoule 10 months ago
- qe: fix pathological behaviour with many @uniques (#4179) The quaint AST lets users keep track of unique index definitions in tables in order to generate `MERGE` statements in SQL Server. The SQL... — committed to prisma/prisma-engines by tomhoule 10 months ago
Hi @janpio - I no longer see the memory leak in this new version. Thanks for your help. We look forward to the release 👍
This is now available as
prisma@5.3.0-dev.32and@prisma/client@5.3.0-dev.32, and can be tested locally. Optimally you (and @mattkindy) can confirm that this solves the problem, and then this will be shipped as part of 5.3.0 on September 12th.Thanks @janpio - we will test this today or latest Monday and update you.
Unfortunately, we are not yet on 5.x, so might be difficult for us to verify at this time.
Internal note: Reproduction is attempted via https://github.com/prisma/20799 + email conversation.Update: See below.
Thanks @janpio - I will drop you a test email then share the rest of the details there.
We have an ongoing scheduled process that runs a query (every minute) against a different table. Even though it also uses
findMany, it doesn’t cause any crashes. Normally, this scheduled process would have run once before I attempt to test the queries that fail.I just noticed that some of the debug message didn’t appear earlier but I see them now (might have misconfigured earlier):
It is very strange indeed. I’m going to do the migrate on a test database and I’ll let you know how it goes.