prisma: When generating the client, `prisma:get-platform` step infers incorrect libssl version in github action under ubuntu-latest

Bug description

Updating from 4.12 to 5.2, github actions began to fail when building the project (npx prisma migrate reset --force) with this error (from DEBUG=prisma:*):

PrismaClientInitializationError: Unable to require(`/home/runner/work/webapp/webapp/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node`).
Prisma cannot find the required `libssl` system library in your system. Please install openssl-1.1.x and try again.

Details: libssl.so.1.1: cannot open shared object file: No such file or directory

It seems like prisma:get-platform thinks the native target should be 1.1:

2023-08-30T22:19:03.575Z prisma:getGenerators  neededVersions {
  "2804dc98259d2ea960602aca6b8e7fdc03c1758f": {
    "engines": [
      "libqueryEngine"
    ],
    "binaryTargets": [
      {
        "fromEnvVar": null,
        "value": "debian-openssl-1.1.x",
        "native": true
      },
      {
        "fromEnvVar": null,
        "value": "debian-openssl-3.0.x"
      }
    ]
  }
}
2023-08-30T22:19:03.582Z prisma:get-platform  Found distro info:
{
  "targetDistro": "debian",
  "familyDistro": "debian",
  "originalDistro": "ubuntu"
}
2023-08-30T22:19:03.582Z prisma:get-platform  Trying platform-specific paths for "debian" (and "ubuntu")
2023-08-30T22:19:03.583Z prisma:get-platform  Found libssl.so file using platform-specific paths: libssl.so.1.1
2023-08-30T22:19:03.583Z prisma:get-platform  The parsed libssl version is: 1.1.x
2023-08-30T22:19:03.872Z prisma:getGenerators  {
  generatorBinaryPaths: {
    libqueryEngine: {
      'debian-openssl-1.1.x': '/home/runner/work/webapp/webapp/node_modules/prisma/libquery_engine-debian-openssl-1.1.x.so.node',
      'debian-openssl-3.0.x': '/home/runner/work/webapp/webapp/node_modules/prisma/libquery_engine-debian-openssl-3.0.x.so.node'
    }
  }
}

I originally thought this was a problem with generating the client, but in the action’s logs that seems to be done successfully:

Running generate... - Prisma Client
2023-08-30T22:19:04.093Z prisma:GeneratorProcess  2023-08-30T22:19:04.093Z prisma:loadEnv  project root found at /home/runner/work/webapp/webapp/package.json
2023-08-30T22:19:04.557Z prisma:GeneratorProcess  2023-08-30T22:19:04.557Z prisma:loadEnv  project root found at /home/runner/work/webapp/webapp/package.json
Running generate... - Prisma Json Types Generator
2023-08-30T22:19:04.678Z prisma:GeneratorProcess  child exited with code null on signal SIGTERM
✔ Generated Prisma Client (v5.2.0) to ./node_modules/@prisma/client in 755ms
✔ Generated Prisma Json Types Generator (3.0.0-beta.4) to ./prisma in 2.90s

This does not happen when developing locally (on darwin-arm64); the native binaryTarget is doing its job as expected.

How to reproduce

Full disclosure: I haven’t minimized this. But in our case, the only change was upgrading the prisma version from 4.12.0 to 5.2.0. Previously our builds were humming along happily!

Expected behavior

I expect the OpenSSL version to be inferred as 3.0.x on a system that only has OpenSSL 3 available.

Prisma information

generator client {
  provider        = "prisma-client-js"
  binaryTargets   = ["native", "debian-openssl-3.0.x"]
  previewFeatures = ["postgresqlExtensions"]
}

Environment & setup

OS: ubuntu-latest, which I think is 22.04
Database: postgresql 15.3
Node.js version: 18.16.1

Because of the node version, I wonder if it’s related to https://github.com/prisma/prisma/issues/19124… but we’ve been on this node version for a while! However, we’re not doing anything in parallel (afaik!), and the build doesn’t progress to tests, so I’ve seen no evidence of segfaulting yet.

Prisma Version

2023-08-30T23:01:42.891Z prisma:tryLoadEnv  Environment variables loaded from /home/runner/work/webapp/webapp/.env
Current platform        : debian-openssl-1.1.x
2023-08-[30](https://github.com/Kept-Inc/webapp/actions/runs/6031096057/job/16364150658#step:11:31)T23:01:42.895Z prisma:get-platform  Found distro info:
Query Engine (Node-API) : E_CANNOT_RESOLVE_VERSION (at node_modules/@prisma/engines/libquery_engine-debian-openssl-1.1.x.so.node)
{
Schema Engine           : schema-engine-cli 2804dc98259d2ea960602aca6b8e7fdc03c1758f (at node_modules/@prisma/engines/schema-engine-debian-openssl-1.1.x)
  "targetDistro": "debian",
Schema Wasm             : @prisma/prisma-schema-wasm 5.2.0-25.2804dc98259d2ea960602aca6b8e7fdc03c1758f
  "familyDistro": "debian",
Default Engines Hash    : 2804dc98259d2ea960602aca6b8e7fdc03c1758f
  "originalDistro": "ubuntu"
Studio                  : 0.494.0
}
Preview Features        : postgresqlExtensions

About this issue

Original URL
State: open
Created 10 months ago
Reactions: 1
Comments: 36 (15 by maintainers)

Commits related to this issue

repro https://github.com/prisma/prisma/issues/20905 — committed to prisma/ecosystem-tests by janpio 10 months ago
repro https://github.com/prisma/prisma/issues/20905 #2 — committed to prisma/ecosystem-tests by janpio 10 months ago

Most upvoted comments

Personally, I think it’s still worth keeping something open on the Prisma side, since there was a way to infer the working version of OpenSSL (ldconfig), and Prisma was doing that until after 5.0.0. Even if it was as naive as having Prisma try the next .so.* file, I probably wouldn’t have had to go on this journey (since after .so.1.1 is my desired .so.3).

It’s also kind of a catch-22 to suggest native for local development, but have native force OpenSSL 1.1.x despite Prisma’s documentation suggesting that 22.04 would reach for 3 (at least, that’s how I read the table a month ago 😉)! It seems like native actually means “the lowest available OpenSSL version”. Maybe the real bug was in the documentation all along…?

I think it would be more complicated, but it’d be be nice to have a more specifically informative error message for folks in my situation; something like:

We tried to use OpenSSL 1.1, but that didn’t work. There is another libssl.so.3 file available, but see <the documentation> for why Prisma will only attempt use 1.1 because the native target is specified, no matter what other binary targets are listed.

I will report back! At least, when the native target is not included in the binaryTargets list, and you remember to include --skip-generate to Prisma CLI commands after prisma generate, we have a green build!

haaksmash on Oct 14, 2023

I understand, but that does not exist right now.

Haha! Sorry, I’m sure it feels like I’m repeating earlier comments in this thread. Just consider it voting with my issue comments for Prisma’s future roadmap!

It seems very likely that the combination of prisma generate (to populate the .prisma/ directory) and the PRISMA_QUERY_ENGINE_LIBRARY variable (to point into .prisma/) will allow me to upgrade beyond 5.0.0, which is what I care about in the short term.

Thank you very much for coming on this nix-shell journey with me, @janpio ❤️

haaksmash on Oct 14, 2023

You use binaryTargets to make prisma generate download all the ones you want and need. (And the equivalent CLI env var if you also need the schema engine for migrations). Then you use PRISMA_QUERY_ENGINE_LIBRARY (and equivalent) to point to the correct one.

Your required engine for Prisma Client should actually be in the built Prisma Client, which is probably in node_modules/.prisma/client. node_modules/@prisma/engines then contains the schema engine for the CLI. The CLI directory under node_modules/prisma should actually not be needed I think. (Yes this is all a bit messy - it was not meant for your use case after all)

Either way, it feels very odd to point to anything in node_modules! I’d much prefer a environment variable that means “don’t infer your own version of OpenSSL, use THIS VERSION”, but it looks like setting LD_LIBRARY_PATH didn’t work out…

I understand, but that does not exist right now.

janpio on Oct 14, 2023

the problem is that Prisma stops looking for .so.* files after finding .so.1.1, even though my schema file requests debian-openssl-3.0.x. In my case, I’d expect Prisma to continue looking for libssl.so.3! Why else would I have 3.0.x as my binary target, after all?

I think Prisma thinks different here: binaryTargets only defines which platforms / binary targets should be made available to Prisma Client, not which one Prisma Client should use. It figures that out via its own detection logic and then expects the matching files to be present. (The main use case for binaryTargets is so that the same project can be run on different platforms or configurations, which require different setups and hence engine files.)

By the way: You can probably route around this whole detection logic by using the environment variables that directly provide the path to the engine files to Prisma. Maybe that is a workaround for you?

janpio on Oct 13, 2023

@janpio confusion is totally understandable! The .so.* files are there because of the base image (ubuntu), but the point of the nix-shell is to limit what executables and libraries are available to a minimal set necessary for running the project. It’s behaving kind of like package.json in a node project, but at the shell level… and more extreme, because it hides globally-installed “packages” unless they’ve been added to the project dependencies list.

This is why the build passes in your fork without the nix shell; released from the restrictions imposed by nix, the prisma client is able to access whatever is in the ubuntu image.

haaksmash on Oct 12, 2023

@OzTK i’m not sure what your risk tolerance is, but it seems like patch-package (at least SUPERFICIALLY) can get the build to pass successfully (and find SSL 3!), by removing that troublesome trailing .: https://github.com/haaksmash/prisma-20905/pull/3/commits/3d507169089df89038b73e7804dea639d2af60db

I’m going to hem and haw for a little longer before deciding to use that patch in my own production system, but it does seem pretty innocuous…

haaksmash on Oct 12, 2023

Thanks for the repro @haaksmash it was super useful to figure out a work-around! This means I’m now stuck on Prisma 5.0.0 until this is resolved though. A good solution for me would be to be able to provide the openssl path, as that would allow it to be fully compatible with a nix setup

OzTK on Oct 9, 2023

Thank you! This is not relevant to debugging, but ironic in an Alanis Morisette way: https://github.com/prisma/prisma/blob/a04e3499742d7246d505cb04c342d188579404d1/packages/get-platform/src/getPlatform.ts#L148

haaksmash on Sep 19, 2023

@janpio Reproduced the failure by adding an (essentially empty) seed script: https://github.com/haaksmash/prisma-20905/actions/runs/6043616043/job/16400924179

haaksmash on Aug 31, 2023

@janpio this public repo’s action is demonstrating the “incorrect” openSSL resolution: https://github.com/haaksmash/prisma-20905/actions/runs/6043463892/job/16400508990#step:7:118

but… it’s not crashing? somehow it’s finding openssl-1.1.x.so.node despite being on ubuntu-latest…???

haaksmash on Aug 31, 2023

Let me see if I can create a public repo that reproduces the issue! It’s in my best interest if my favorite libraries play well with my favorite operating system/package managers 😃

haaksmash on Aug 31, 2023

@janpio there must be something unique about our setup or I’d expect there to be a deluge of reports for this!

Our project uses Nix to manage most of its dependencies (in our action, this is installed via https://github.com/cachix/install-nix-action); we have the nixpkgs SHA pinned to 2de8efefb6ce7f5e4e75bdf57376a96555986841. We run our commands via nix develop, so our action is actually running nix develop --command npx prisma migrate reset --force; perhaps the nix middle-man is presenting an incorrect OpenSSL version…?

haaksmash on Aug 31, 2023