prisma: Timescale does not work with `migrate dev` and `migrate reset`: `extension "timescaledb" has already been loaded with another version`

Discussed in https://github.com/prisma/prisma/discussions/6963

<div type='discussions-op-text'>

Originally posted by pfried May 6, 2021 This is a question on how to setup timescale with prisma. Thanks to the post from @janpio I tried to create a similar setup. So this is related to #https://github.com/prisma/prisma/issues/3228

My goal is to ingest IoT data into timescaledb with the help of prisma since the whole raw sql stuff is just bad (I know some stuff will stay raw sql stuff for now, especially timescaledb functionality).

My desired setup

  1. Timescale 2.2.0 running on Timescale Forge
  2. Prisma / Prisma Client 2.22.0 used by AWS Lambda (will do a lambda layer / package as suggested by the docs).

Following the steps

The first step would be to create a migration which will install the timescale extension since the shadow database will run the migrations too (it is obiously already installed on Laravel forge). So the migration.sql file will look like:

CREATE EXTENSION IF NOT EXISTS timescaledb VERSION "2.2.0";

Case 1: Forge Database without shadow database

Try to run the npx prisma migrate dev command on the timescale forge database will result in an error since the command tries to create a table:

Error: Database error
Error querying the database: db error: ERROR: tsdb_admin: database prisma_migrate_shadow_db_ab8fe7fa-8d81-4493-bafe-bf5645e6d2cc is not an allowed database name
HINT: Contact your administrator to configure the "tsdb_admin.allowed_databases"

Following the suggestions in #4571 and here: https://www.prisma.io/docs/concepts/components/prisma-migrate#shadow-database the guess would be that it is not possible to use the timescale forge service for the shadow db. Which leads to:

Case 2: Forge database with shadow database on localhost

My localhost is a postgres 13 instance on windows with the timescale extension available.

Adding a shadow database in the environment and running the npx prisma migrate dev will result in an error telling that the timescaledb extension is already loaded (both instances run 2.2.0):

Error: P3006

Migration `20210505152706_` failed to apply cleanly to the shadow database. 
Error:
Database error
Error querying the database: db error: FEHLER: extension "timescaledb" has already been loaded with another version
DETAIL: The loaded version is "2.2.0".

If the timescale forge database has already has data a reset is suggested and if executed will result in:

Error: P3016

The fallback method for database resets failed, meaning Migrate could not clean up the database entirely. Original error: 
Database error
Error querying the database: db error: ERROR: must be owner of view pg_stat_statements
   0: sql_migration_connector::best_effort_reset

Case 3 Local Timescale database

If no shadow database is explicitly given (meaning the migrate command will create one) everything works on the local machine (different to Case 1).

Case 4 Local Timescale database with explicit shadow database on the same server

If I explicitly set a shadow database on my local machine I will end up in the same error as Case 2:

Error: P3006

Migration `20210505152706_` failed to apply cleanly to the shadow database. 
Error:
Database error
Error querying the database: db error: FEHLER: extension "timescaledb" has already been loaded with another version
DETAIL: The loaded version is "2.2.0".
</div>

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Comments: 18 (7 by maintainers)

Most upvoted comments

I opened an issue with Timescale asking for information: https://github.com/timescale/timescaledb/issues/5884

With the new views and extensions feature I created a showcase here: https://github.com/Nantis-GmbH/cdk-prisma-timescale-demo

It shows how to use TimescaleDB with Prisma on AWS using the latest features from prisma and timescale

The issue here still persists tough. Not sure whether to blame timescale or prisma. This is literally the last blocker (afaik) to having great usability of Prisma + Timescale

Funnily enough if you run prisma migrate reset twice quickly after each other (maybe connection reuse?) it works. Sry for not really having an idea what the root cause of all of this is, but I can only share observations

Is there a chance anyone here has found a reasonable workaround since July 2023? As a recent convert to Timescale and an advocate of Prisma, I find myself in this situation.

Before I get to the extension and reset problem, I confirmed along the way that the shadow database problem (that it can not be created automatically on Timescale Cloud) is handled properly today: https://github.com/prisma/prisma/issues/20266#issuecomment-1638466980

...
Datasource "db": PostgreSQL database "tsdb", schema "public" at "opj14wxfkv.kn5o1onhbq.tsdb.cloud.timescale.com:30943"

Error: P3014

Prisma Migrate could not create the shadow database. Please make sure the database user has permission to create databases. Read more about the shadow database (and workarounds) at https://pris.ly/d/migrate-shadow

Original error: 
db error: ERROR: tsdb_admin: database prisma_migrate_shadow_db_1dc9385e-8a23-4b7d-8f30-1c037906c5ef is not an allowed database name
HINT: Contact your administrator to configure the "tsdb_admin.allowed_databases"
   0: schema_core::state::DevDiagnostic
             at schema-engine/core/src/state.rs:266

Sure there is a free trial you can use at: https://console.cloud.timescale.com/signup

I created an MRE at https://github.com/Nantis-GmbH/prisma-timescale

@janpio sure, that recognizes the extension. thats all fine, but the “double” reset issue persists