prisma: Suspected memory leak in Lambda function after upgrading from 4.10.1

Bug description

We recently came across VT13001: [BUG] in scatter query whilst trying to run the db:push and other similar commands. Based on this ticket, we decided to upgrade our prisma version. The issue disappeared after upgrade but we started noticing our lambda functions crashing. From the lambda function logs:

Error: Runtime exited with error: signal: killed Runtime.ExitError
REPORT RequestId: xx Duration: 6174.19 ms Billed Duration: 6175 ms Memory Size: 850 MB Max Memory Used: 850 MB 

The lambda function has always been configured to use 512 MB so we decided to upgrade to 850 MB in case the upgrade required more memory. But that didn’t help.

Pls see below for more info. We’re kind of in a limbo now; we cannot make any more schema changes due to the VT13001 bug above and we cannot upgrade to v5 due the lambda crashes. We do not need to upgrade to v5. If we have a fix for the VT13001 bug, we’re happy to stay on v4.10.1 for now.

Thank you.

How to reproduce

Upgrade to 4.15.0, 5.0.0 or 5.1.1 and create tables similar to the ones provided below.

Expected behavior

We expect it to work like it did in v4.10.1.

Prisma information

generator client {
    provider        = "prisma-client-js"
    previewFeatures = ["filteredRelationCount"]
    binaryTargets   = ["rhel-openssl-1.0.x", "native"]
}

datasource db {
    provider     = "mysql"
    url          = env("DATABASE_URL")
    relationMode = "prisma"
}

Not all queries/tables cause it to crash. Some queries are ok but others just crash. An example of such table:

model Foo {
    id                                   Int                                 @id @default(autoincrement())
    uid                                  String                              @unique @db.Char(26)
    name                                 String                              @db.VarChar(100)
    createdDate                          DateTime                            @default(now())
    deleted                              Boolean                             @default(false)
    deletedDate                          DateTime?
    tableAMapping                        TableAMapping[]
    tableBMapping                        TableBMapping[]
    owner                                User                                @relation(fields: [ownerId], references: [id], onDelete: Cascade)
    ownerId                              Int
    fooType                              FooType                             @relation(fields: [fooTypeId], references: [id])
    fooTypeId                            Int
    anotherTableCredentials              anotherTableCredentials?            @relation(fields: [anotherTableCredentialsId], references: [id])
    anotherTableCredentialsId            Int?
    ## MORE FIELDS: Another 19 fields similar to anotherTableCredentials and anotherTableCredentialsId

    @@index([name])
    @@index([ownerId])
    @@index([fooTypeId])
    @@index([anotherTableCredentialsId])
    ## MORE INDEX: corresponding to the 19 fields mentioned above
    
    @@index([uid])
}

A query such as:

await prisma.foo.findMany({
    where: {
        fooType: {
            uid: request.fooTypeUid,
        },
        ownerId: user.id,
        deleted: false,
        name,
    },
}),

or even:

await prisma.foo.count({
    where: {
        ownerId: user.id,
        deleted: false,
    },
}),

would fail.

Some other queries that use findMany seem to be ok.

Environment & setup

  • OS: Amazon Linux 2 x86_64
  • Database: MySQL
  • Node.js version: 16.18

Prisma Version

It affects multiple versions 4.15.0, 5.0.0 & 5.1.1.

prisma                  : 5.1.1
@prisma/client          : 5.1.1
Current platform        : darwin-arm64
Query Engine (Node-API) : libquery-engine 6a3747c37ff169c90047725a05a6ef02e32ac97e (at node_modules/@prisma/engines/libquery_engine-darwin-arm64.dylib.node)
Schema Engine           : schema-engine-cli 6a3747c37ff169c90047725a05a6ef02e32ac97e (at node_modules/@prisma/engines/schema-engine-darwin-arm64)
Schema Wasm             : @prisma/prisma-schema-wasm 5.1.1-1.6a3747c37ff169c90047725a05a6ef02e32ac97e
Default Engines Hash    : 6a3747c37ff169c90047725a05a6ef02e32ac97e
Studio                  : 0.492.0

About this issue

  • Original URL
  • State: closed
  • Created 10 months ago
  • Comments: 17 (9 by maintainers)

Commits related to this issue

Most upvoted comments

Hi @janpio - I no longer see the memory leak in this new version. Thanks for your help. We look forward to the release 👍

This is now available as prisma@5.3.0-dev.32 and @prisma/client@5.3.0-dev.32, and can be tested locally. Optimally you (and @mattkindy) can confirm that this solves the problem, and then this will be shipped as part of 5.3.0 on September 12th.

Thanks @janpio - we will test this today or latest Monday and update you.

This is now available as prisma@5.3.0-dev.32 and @prisma/client@5.3.0-dev.32, and can be tested locally. Optimally you (and @mattkindy) can confirm that this solves the problem, and then this will be shipped as part of 5.3.0 on September 12th.

Unfortunately, we are not yet on 5.x, so might be difficult for us to verify at this time.

Internal note: Reproduction is attempted via https://github.com/prisma/20799 + email conversation.

Update: See below.

Thanks @janpio - I will drop you a test email then share the rest of the details there.

We have an ongoing scheduled process that runs a query (every minute) against a different table. Even though it also uses findMany, it doesn’t cause any crashes. Normally, this scheduled process would have run once before I attempt to test the queries that fail.

I just noticed that some of the debug message didn’t appear earlier but I see them now (might have misconfigured earlier):

2023-08-23T08:52:37.889Z prisma:client:libraryEngine library started

<First query to get the user is run successfully>

<After the first query, I log the process.memoryUsage(): rss 91.8 MB, heapTotal 37.95 MB, heapUsed 20.25 MB, external 3.55 MB & arrayBuffers 0.05 MB >

2023-08-23T08:52:37.930Z prisma:client Prisma Client call:
2023-08-23T08:52:37.930Z prisma:client Generated request:
2023 -08 - 23T08: 52: 37.930Z prisma:client {
    "modelName": "Foo",
    "action": "findMany",
    "query": {
    "arguments": {
    "where": {
    "fooType": {
    "uid": "01H8GRXMMS0XB2QJRFR3QDYZ7G"
    },
    "ownerId": 1,
    "deleted": false,
    "name": "SomeAwesomeName"
    }
    },
    "selection": {
    "id": true
    }
    }
}
2023-08-23T08:52:37.930Z prisma:client:libraryEngine sending request, this.libraryStarted: true

It is very strange indeed. I’m going to do the migrate on a test database and I’ll let you know how it goes.