coolify: [Bug]: Supabase containers keep restarting due to authentication-related error

Description

When attempting to deploy Supabase using Coolify v4.0.0-beta.306, the process fails, and the logs of the containers indicate an authentication-related error.

Do note that it works fine on v4.0.0-beta.297 except for that fact that:

  1. Minio Createbucket container fails to run and exits.
  2. Supabase Rest and Realtime Dev shows running (unhealthy).

Minimal Reproduction (if possible, example repository)

  1. Upgrade to Coolify v4.0.0-beta.306.
  2. Attempt to deploy Supabase.
  3. Observe the failure in the deployment process. Several containers would keep restarting.
  4. Check logs of the failed containers.

Exception or Error

No response

Version

v4.0.0-beta.306

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 13
  • Comments: 43 (8 by maintainers)

Commits related to this issue

Most upvoted comments

I’ve figured out the issue, can replicate and mitigate.

Coolify is setting the POSTGRES_HOST parameter to the POSTGRES_HOST environment parameter even though it has a hard coded value set.

You can resolve the issue by setting POSTGRES_HOST to some other value like POSTGRES_HOSTNAME, change all instances of POSTGRES_HOST parameter inside the docker-compose and then delete the POSTGRES_HOST after saving.

Issue: Postgres runs the init scripts before the network connection is ready by connecting directly to the socket, which is why for the POSTGRES_HOST=/var/run/postgresql env variable on supabase-db it’s set to a path.

When the env is incorrectly overridden the value is set to supabase-db which resolves to the docker network which isn’t initialised and is also not able to use the local access root.

I might still be on to win my bet. 😂

Refs:

https://github.com/docker-library/postgres/issues/941 https://raw.githubusercontent.com/docker-library/postgres/master/15/bullseye/docker-entrypoint.sh https://github.com/supabase/postgres/blob/develop/migrations/db/migrate.sh

It can also be replicated with a simpler flow: Create a service with this docker-compose:

  test:
    image: hashicorp/http-echo
    environment:
      - DB_HOST=value/two

Create an env called DB_HOST with some other value. Save the docker-compose, IE: “I just made a change” Observe that DB_HOST is overridden with some other value.

This will be fixed in the upcoming version.

This worked perfectly for me. For future reference, here are the steps I did to resolve :

  1. first deploy the stack via coolify
  2. wait for the deployment to fail
  3. stop all containers
  4. in the environment variable panel, or directly in the .env file in the server : a. replace the POSTGRES_HOSTvariable with POSTGRES_HOSTNAME
  5. in the service stack panel, click on the edit compose file or directly in the docker-compose.yml file in the server a. replace all use of POSTGRES_HOST variable with POSTGRES_HOSTNAME
  6. on the server, use docker compose down --volumes to remove the old db config
  7. deploy the stack again
  8. it should work

@deozza Try stopping the stack, removing the associated _supabase-db-data volume and restart the stack.

You can find the volume by running docker volume ls look for the one which has <the_random_stack_string>_supabase-db-data and then running docker volume rm <name>.

For example my random stack string that is before my url etc is rwkg84s my volume is rwkg84s_supabase-db-data so I would run docker volume rm rwkg84s_supabase-db-data

Once you’ve done that you should be able to start the service again and hopefully the migrations will run correctly.

I’ve figured out the issue, can replicate and mitigate.

@Mortalife seems you are right. I was guessing maybe other services are loading sooner than DB but I believe it should be an environment conflict as you mentioned. I hope supabase make it’s deployment process more robust in future. it’s a little tricky now.

How would they sell their cloud services if self hosting was that easy? It’s just marketing and it has to be somehow possible. But they don’t want the mass to self host supabase…

@diegofino15 and @gBusato, I can confirm that @Mortalife (here) and @deozza (here) instructions worked for me (thank you both).

To reiterate, make sure to:

  1. stop the services first
  2. delete the <uuid>_supabase-db-data volume as per @Mortalife 's instructions
  3. in Coolify’s Environment Variables section, rename the POSTGRES_HOST variable to POSTGRES_HOSTNAME
  4. ⚠️ in Coolify’s Service Stack > Edit Compose File, rename all instances of the POSTGRES_HOST variable to POSTGRES_HOSTNAME and save (this might make step 3 redundant but I didn’t test it)
  5. restart the service

I had missed Step 4 and the stack just recreated that POSTGRES_HOST Environment Variable.

The fix works for me aswell, but “Minio Createbucket” does not start. Did that work for you with this fix, @Torwent & @agalev ?

I’m pretty sure that’s not meant to be running. Runs once the very first time you start things up to create the minIO bucket and never runs again AFAIK

If I were a betting man, I’d say it was this commit: 1266810

No, it already did’nt work on Monday

Fair, I read through it in more detail and if I were a betting man, I’d have lost money! Haha Double checked the env’s passed to the containers and it’s correct. So my hypothesis was incorrect.