prisma: Error in connector: Error creating a database connection. (Operation timed out (mobc timeout))

I can reproduce this with a heroku database with connection limit of 10 and this simple script.

All timed out calls doesn’t recover. Connections in the heroku reaches max of 11 then it starts to give timeouts.

Reproduction: https://github.com/harshit-test-org/prisma-issue-4280

I am using the latest version of prisma (2.15.0). It’s throwing the new error (https://github.com/prisma/prisma-engines/pull/1515). But the error message doesn’t make sense. I have 150 connections as limit, 10 connections are used but it still fires an error:

Timed out fetching a new connection from the pool. Please consider reducing the number of requests or increasing the `connection_limit` parameter (https://www.prisma.io/docs/concepts/components/prisma-client/connection-management#connection-pool). Current limit: 150, amount of connections currently in use: 10.

The connection limit of my DB (postgres) is 180

Same issue here.

{
  "data": {
    "message": "Error: Error creating a database connection. (Operation timed out (mobc timeout))",
    "reason": {
      "clientVersion": "2.11.0",
      "message": "Error creating a database connection. (Operation timed out (mobc timeout))",
      "stack": "Error: Error creating a database connection. (Operation timed out (mobc timeout))\n    at LineStream.<anonymous> (/var/task/node_modules/@prisma/client/runtime/index.js:27430:31)\n    at LineStream.emit (events.js:315:20)\n    at addChunk (_stream_readable.js:295:12)\n    at readableAddChunk (_stream_readable.js:271:9)\n    at LineStream.Readable.push (_stream_readable.js:212:10)\n    at LineStream.Transform.push (_stream_transform.js:152:32)\n    at LineStream._pushBuffer (/var/task/node_modules/@prisma/client/runtime/index.js:24491:19)\n    at LineStream._transform (/var/task/node_modules/@prisma/client/runtime/index.js:24485:10)\n    at LineStream.Transform._read (_stream_transform.js:191:10)\n    at LineStream.Transform._write (_stream_transform.js:179:12)"
    },
    "stack": "Runtime.UnhandledPromiseRejection: Error: Error creating a database connection. (Operation timed out (mobc timeout))\n    at process.<anonymous> (/var/runtime/index.js:35:15)\n    at process.emit (events.js:327:22)\n    at processPromiseRejections (internal/process/promises.js:209:33)\n    at processTicksAndRejections (internal/process/task_queues.js:98:32)"
  },
  "level": "error",
  "message": "Unhandled Promise Rejection \t",
  "timestamp": "2020-11-20T07:03:55.934Z"
}

BTW I’m using Blitz.js + Vercel + DigitalOcean. More info about this stack at https://blitzjs.com/docs/deploy-vercel.

@mjknight50 so it looks like you have a current connection limit of 3, you can comfortably have 3 concurrent prisma queries sent at once.

Or if you’re running the Prisma Client inside a node http server, each handler runs concurrently, you can have 3 simultaneous visitors each sending 1 query.

If you’re above this, these requests will get queued and processed in FIFO order. If the requests spend too long in the queue, they will start to timeout.

This is the problem you’re running into and the solution is to either reducing the number of requests or increase the connection_limit parameter.

Just a note that increasing the connection_limit has tradeoffs and you may want to upgrade your DB to handle more connections. This answer may help if you’re running on RDS.

We found a limitation in the current version that even with pool_timeout=0 (which should disable the timeout) runs into a timeout at ~30 seconds after you start. Does this match your observations when you see the error?

I am also using the latest version of @prisma/client and @prisma/cli of 2.15.0 and am seeing that I’m getting the same error above, except it mentions that there are 0 connections in use.

I am deploying to Firebase cloud functions, running on Node 12.

Error: Timed out fetching a new connection from the pool. Please consider reducing the number of requests or increasing the `connection_limit` parameter (https://www.prisma.io/docs/concepts/components/prisma-client/connection-management#connection-pool). Current limit: 10, amount of connections currently in use: 0.

This is how I instantiate my PrismaClient

const Prisma = require('@prisma/client')

let prisma

if (process.env.NODE_ENV === 'production') {
    prisma = new Prisma.PrismaClient()
} else {
    if (!global.prisma) {
        global.prisma = new Prisma.PrismaClient()
    }

    prisma = global.prisma
}

module.exports = prisma

Thanks, everyone for your comments! We are looking into it right now and discuss solutions internally. We will definitely make sure to give you a solution for this.

@pantharshit00 I can reproduce that also for the Can't reach database server error, when the docker DB container is not started.

I found it as a regression because some time ago (a few weeks or months ago) it worked like a charm - I could start the db when the server was running and it could reconnect itself when e.g. the findMany was called (implicit connect).

@janpio thank you for your response!

I’m migrating data from an old environment to a new one. To improve speed, I spin up some child process (16) which runs some queries. All the child processes are using the same Prisma instance. However, it also occurs if I use a new PrismaClient per child process.

So I think the reason that Prisma triggers this is because of a lot of queries running in parallel as you mentioned.

Adding the &pool_timeout=0 doesn’t seem to fix the issue 😦 . You mean to add it like this : ...m:5432/postgres?pool_timeout=0 , right?

I am having the same issue.

@timsuchanek I have tried previously that with the same result. When I’m running a big amount of operations sooner than later I ended with the same error.

Even, In the ticket description, I’m running it with 10.

I think this problem is with the query engine https://www.prisma.io/docs/concepts/components/prisma-client/query-engine#the-query-engine-at-runtime we are facing a lot of stability problems with the query engine: timeouts, OOM…

This error shows with or without pgBouncer or cloudsql-proxy.

We face this also at my work and we are using pgBouncer. What would be really useful is a stacktrace when possible because it’s very hard to find out where is the problem.

Im having the same issue

error PrismaClientUnknownRequestError2:
Invalid `prisma.item.create()` invocation:


  Error in connector: Error creating a database connection. (Operation timed out (mobc timeout))
    at PrismaClientFetcher.request (C:\Users\MayerLench\Documents\VSCode\my-project\node_modules\@prisma\client\runtime\index.js:78786:15)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:93:5) {
  clientVersion: '2.11.0'
}

UPDATE: I had to serialize my requests being made to the database in order to workaround this issue. Although not sure why prisma doesn’t handle this

The library i used was throat js which allows you to add a concurrency on an array of factory functions that returns promises and runs them in sequence