prisma: Error opening a TLS connection - Race condition
Bug description
During some deployment (we have never experienced it locally) we receive the following error
Error: Error opening a TLS connection: error:24067044:random number generator:rand_pool_add:internal error:crypto/rand/rand_lib.c:733: (self signed certificate in certificate chain)
at startFn (/app/node_modules/@prisma/client/core/runtime/library.js:100:2563)
at Proxy.onModuleInit (/app/dist/apps/aurora/webpack:/libs/data-access/server/prisma-core/src/lib/base-core-prisma.service.ts:34:9)
at async Promise.all (index 1)
at callModuleInitHook (/app/node_modules/@nestjs/core/hooks/on-module-init.hook.js:43:5)
at NestApplication.callInitHook (/app/node_modules/@nestjs/core/nest-application-context.js:210:13)
at NestApplication.init (/app/node_modules/@nestjs/core/nest-application.js:97:9)
at NestApplication.listen (/app/node_modules/@nestjs/core/nest-application.js:155:33)
// base-core-prisma.service.ts
@Injectable()
export abstract class BaseCorePrismaService<T extends Prisma.PrismaClientOptions = Prisma.PrismaClientOptions>
extends PrismaClient
implements OnModuleInit, OnModuleDestroy
{
protected constructor(
protected readonly config: BaseCorePrismaConfig,
protected readonly optionsArg?: Prisma.Subset<T, Prisma.PrismaClientOptions>
) {
super(optionsArg);
}
async onModuleInit(): Promise<void> {
await this.$connect(); // Row 34
if (!this.config.readReplica) {
// Enable soft deletes
this.$use(SoftDeleteMiddleware);
}
}
async onModuleDestroy(): Promise<void> {
await this.$disconnect();
}
async enableShutdownHooks(app: INestApplication): Promise<void> {
this.$on('beforeExit', async () => {
await app.close();
});
}
}
This is an abstract class in our NestJS APIs that our db model wrapper extend so we have shared configuration between the class that connects to our Postgres DB and the class that connects to our MongoDB (thanks to prisma having a unified setup this makes it really easy! thanks!).
This error only happens sometimes (pretty rarely actually, as I thought it was fixed a month ago since we have not seen it since).
Our connection strings are using &TrustServerCertificate=true&sslmode=require
and the server is deployed in a Docker container based on FROM node:18.12.1-bullseye
to Elastic beanstalk on t4g.micro instances (Arm-based AWS Graviton2 processors), this does not seem to happen on the x86 elastic beanstalk environments (at least not that we have scene yet). the binary targets we have are binaryTargets = ["native", "linux-musl-openssl-3.0.x", "linux-arm64-openssl-1.1.x", "debian-openssl-1.1.x"]
How to reproduce
Reproduction is VERY tricky as it only happens rarely, only on ARM instances, and only when deployed to Elastic beanstalk. Open to ideas here I would love to be able to repro it.
Expected behavior
No issues with self signed certs and connecting.
Prisma information
>3000 lines, if a prisma engineer needs specifics feel free to let me know and I can provide what you need.
Removed some stuff for clarity, the Mongo service is using the base service in a similar way to the CorePrismaService
@Injectable()
export abstract class BaseCorePrismaService<T extends Prisma.PrismaClientOptions = Prisma.PrismaClientOptions>
extends PrismaClient
implements OnModuleInit, OnModuleDestroy
{
protected constructor(
protected readonly config: BaseCorePrismaConfig,
protected readonly optionsArg?: Prisma.Subset<T, Prisma.PrismaClientOptions>
) {
super(optionsArg);
}
async onModuleInit(): Promise<void> {
await this.$connect();
if (!this.config.readReplica) {
// Enable soft deletes
this.$use(SoftDeleteMiddleware);
}
}
async onModuleDestroy(): Promise<void> {
await this.$disconnect();
}
async enableShutdownHooks(app: INestApplication): Promise<void> {
this.$on('beforeExit', async () => {
await app.close();
});
}
}
@Injectable()
export class CorePrismaService extends BaseCorePrismaService {
constructor(@Inject(CORE_PRISMA_CONFIGURATION) configuration: CorePrismaConfiguration) {
super(
{},
{
log: configuration.debug ? ['query'] : undefined,
datasources: {
db: configuration.primary,
},
}
);
}
}
Environment & setup
AWS Linux t4g.micro Arm-based AWS Graviton2 processors Docker node:18.12.1-bullseye AWS RDS Postgres & MongoDB Atlas
Prisma Version
4.10.0
About this issue
- Original URL
- State: closed
- Created a year ago
- Reactions: 3
- Comments: 26 (7 by maintainers)
@darvelo that sounds very plausible, I’m 90% sure it indeed has the same root cause. It might be challenging to try to reproduce this exact error on my side so it would be very helpful if someone who encountered it could test
5.3.0-integration-arm-openssl-5-2-0.1
and see if it fixes the issue.@aqrln Could this issue possibly also have the same root cause as #18510? They both seem to be OpenSSL issues on arm64.
I switched my CI and container setup all over to x86 and haven’t seen this issue again.
I will see what I can do.
Yes, it also happened in 4.9.x, I upgraded to 4.10.0 in case that helped.