supabase: [Self Hosting] dependency failed to start: container supabase-vector is unhealthy

Bug report

  • I confirm this is a bug with Supabase, not with my own application.
  • I confirm I have searched the Docs, GitHub Discussions, and Discord.

Describe the bug

Self Hosting fail to start when docker compose is used.

To Reproduce

Steps to reproduce the behavior, please provide code snippets or a repository:

  1. Go to Gidpot machine’…’
  2. Choice the repository of supabase
  3. Open the project of supabase
  4. Navigate to docker directory
  5. Copy the .env.example to .env
  6. Run docker compose pull
  7. Run docker compose up

Expected behavior

Self Hosting start without problems

Screenshots

gitpod /workspace/supabase/docker (master) $ docker compose up
WARN[0000] The "MFA_ENABLED" variable is not set. Defaulting to a blank string. 
[+] Running 13/12
 ✔ Network docker_default                    Created                                                                                                                                                                                                                              0.9s 
 ✔ Container supabase-vector                 Created                                                                                                                                                                                                                              0.9s 
 ✔ Container supabase-imgproxy               Created                                                                                                                                                                                                                              0.9s 
 ✔ Container supabase-db                     Created                                                                                                                                                                                                                              0.0s 
 ✔ Container supabase-analytics              Created                                                                                                                                                                                                                              0.1s 
 ✔ Container supabase-meta                   Created                                                                                                                                                                                                                              0.1s 
 ✔ Container supabase-auth                   Created                                                                                                                                                                                                                              0.1s 
 ✔ Container realtime-dev.supabase-realtime  Created                                                                                                                                                                                                                              0.1s 
 ✔ Container supabase-kong                   Created                                                                                                                                                                                                                              0.1s 
 ✔ Container supabase-rest                   Created                                                                                                                                                                                                                              0.1s 
 ✔ Container supabase-edge-functions         Created                                                                                                                                                                                                                              0.1s 
 ✔ Container supabase-studio                 Created                                                                                                                                                                                                                              0.1s 
 ✔ Container supabase-storage                Created                                                                                                                                                                                                                              0.0s 
Attaching to realtime-dev.supabase-realtime, supabase-analytics, supabase-auth, supabase-db, supabase-edge-functions, supabase-imgproxy, supabase-kong, supabase-meta, supabase-rest, supabase-storage, supabase-studio, supabase-vector
supabase-imgproxy               | WARNING [2023-08-24T12:11:46Z] No keys defined, so signature checking is disabled 
supabase-imgproxy               | WARNING [2023-08-24T12:11:46Z] No salts defined, so signature checking is disabled 
supabase-imgproxy               | WARNING [2023-08-24T12:11:46Z] Exposing root via IMGPROXY_LOCAL_FILESYSTEM_ROOT is unsafe 
supabase-imgproxy               | INFO    [2023-08-24T12:11:46Z] Starting server at :5001 
supabase-vector                 | 2023-08-24T12:11:47.154332Z  INFO vector::app: Internal log rate limit configured. internal_log_rate_secs=10
supabase-vector                 | 2023-08-24T12:11:47.154623Z  INFO vector::app: Log level is enabled. level="vector=info,codec=info,vrl=info,file_source=info,tower_limit=trace,rdkafka=info,buffers=info,lapin=info,kube=info"
supabase-vector                 | 2023-08-24T12:11:47.154745Z  INFO vector::app: Loading configs. paths=["etc/vector/vector.yml"]
supabase-vector                 | 2023-08-24T12:11:47.250136Z  WARN vector::config::loading: Transform "kong_err" has no consumers
supabase-vector                 | 2023-08-24T12:11:47.250181Z  WARN vector::config::loading: Transform "router._unmatched" has no consumers
supabase-vector                 | 2023-08-24T12:11:47.762198Z  INFO vector::topology::running: Running healthchecks.
supabase-vector                 | 2023-08-24T12:11:47.762384Z  INFO vector::topology::builder: Healthcheck passed.
supabase-vector                 | 2023-08-24T12:11:47.762414Z  INFO vector::topology::builder: Healthcheck passed.
supabase-vector                 | 2023-08-24T12:11:47.762434Z  INFO vector::topology::builder: Healthcheck passed.
supabase-vector                 | 2023-08-24T12:11:47.762443Z  INFO vector::topology::builder: Healthcheck passed.
supabase-vector                 | 2023-08-24T12:11:47.762449Z  INFO vector::topology::builder: Healthcheck passed.
supabase-vector                 | 2023-08-24T12:11:47.762460Z  INFO vector::topology::builder: Healthcheck passed.
supabase-vector                 | 2023-08-24T12:11:47.762467Z  INFO vector::topology::builder: Healthcheck passed.
supabase-vector                 | 2023-08-24T12:11:47.762696Z  INFO vector: Vector has started. debug="false" version="0.28.1" arch="x86_64" revision="ff15924 2023-03-06"
supabase-vector                 | 2023-08-24T12:11:47.764148Z  INFO source{component_kind="source" component_id=docker_syslog component_type=syslog component_name=docker_syslog}: vector::sources::util::net::tcp: Listening. addr=0.0.0.0:9000
supabase-vector                 | 2023-08-24T12:11:47.777645Z  INFO vector::internal_events::api: API server running. address=0.0.0.0:9001 playground=http://0.0.0.0:9001/playground
supabase-imgproxy               | INFO    [2023-08-24T12:11:51Z] Started /health  request_id=d4BpW0mW6y5ZsiibDq93x method=GET client_ip=127.0.0.1
supabase-imgproxy               | INFO    [2023-08-24T12:11:51Z] Completed in 73.41µs /health  request_id=d4BpW0mW6y5ZsiibDq93x method=GET status=200 client_ip=127.0.0.1
supabase-imgproxy               | INFO    [2023-08-24T12:11:57Z] Started /health  request_id=uO8grDaxmu5EYDm5u2BTW method=GET client_ip=127.0.0.1
supabase-imgproxy               | INFO    [2023-08-24T12:11:57Z] Completed in 243.63µs /health  request_id=uO8grDaxmu5EYDm5u2BTW method=GET status=200 client_ip=127.0.0.1
supabase-imgproxy               | INFO    [2023-08-24T12:12:03Z] Started /health  request_id=g051Npsc73BC4yCeaBtMo method=GET client_ip=127.0.0.1
supabase-imgproxy               | INFO    [2023-08-24T12:12:03Z] Completed in 95.28µs /health  request_id=g051Npsc73BC4yCeaBtMo method=GET status=200 client_ip=127.0.0.1
dependency failed to start: container supabase-vector is unhealthy
gitpod /workspace/supabase/docker (master) $ 

System information

  • OS: Linux supabase-supabase-s84498vcpig 6.1.44-060144-generic #202308111259 SMP PREEMPT_DYNAMIC Fri Aug 11 14:20:00 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
  • Docker: Docker version 24.0.5, build ced0996

Additional context

The instance is new and is default configuration

About this issue

  • Original URL
  • State: closed
  • Created 10 months ago
  • Reactions: 5
  • Comments: 21 (1 by maintainers)

Most upvoted comments

For me, doing a sudo docker compose restart solved the issue.

After checking out changes from #17122 , I was able to get it running by adding VECTOR_API_PORT=9001 to my .env file

image

same error

Solution

Linux/MacOS:

sudo docker compose restart

Windows:

run cmd as administrator (via right click)

For me, doing a sudo docker compose restart solved the issue.

THIS! Got the same error and doing a compose restart was the only solution that worked.

Solution

Linux/MacOS:

sudo docker compose restart

Windows:

run cmd as administrator (via right click)

I don’t think it’s a solution, but a hotfix.

Maybe this ticket should be reopened again

I am facing the same issue for a self hosted supabase. Some more backgrund info:

  1. System: Ubuntu 22.04
  2. Docker-containers using rootless mode
  3. Using default docker and .env file
  4. Image: 20230921-d657f29

When starting with docker compose up startup fails:

[+] Running 13/13
 ✔ Network docker_default                    Created                                                                                                                                                                                                                     0.2s 
 ✔ Container supabase-imgproxy               Created                                                                                                                                                                                                                     0.9s 
 ✔ Container supabase-vector                 Created                                                                                                                                                                                                                     0.9s 
 ✔ Container supabase-db                     Created                                                                                                                                                                                                                     0.6s 
 ✔ Container supabase-analytics              Created                                                                                                                                                                                                                     0.7s 
 ✔ Container realtime-dev.supabase-realtime  Created                                                                                                                                                                                                                     0.8s 
 ✔ Container supabase-rest                   Created                                                                                                                                                                                                                     0.8s 
 ✔ Container supabase-kong                   Created                                                                                                                                                                                                                     0.8s 
 ✔ Container supabase-meta                   Created                                                                                                                                                                                                                     0.8s 
 ✔ Container supabase-auth                   Created                                                                                                                                                                                                                     0.8s 
 ✔ Container supabase-studio                 Created                                                                                                                                                                                                                     0.8s 
 ✔ Container supabase-edge-functions         Created                                                                                                                                                                                                                     0.8s 
 ✔ Container supabase-storage                Created                                                                                                                                                                                                                     0.6s 
Attaching to realtime-dev.supabase-realtime, supabase-analytics, supabase-auth, supabase-db, supabase-edge-functions, supabase-imgproxy, supabase-kong, supabase-meta, supabase-rest, supabase-storage, supabase-studio, supabase-vector
supabase-imgproxy               | WARNING [2023-10-17T16:01:58Z] No keys defined, so signature checking is disabled 
supabase-imgproxy               | WARNING [2023-10-17T16:01:58Z] No salts defined, so signature checking is disabled 
supabase-imgproxy               | WARNING [2023-10-17T16:01:58Z] Exposing root via IMGPROXY_LOCAL_FILESYSTEM_ROOT is unsafe 
supabase-imgproxy               | INFO    [2023-10-17T16:01:58Z] Starting server at :5001 
supabase-vector                 | 2023-10-17T16:01:59.059627Z  INFO vector::app: Internal log rate limit configured. internal_log_rate_secs=10
supabase-vector                 | 2023-10-17T16:01:59.059834Z  INFO vector::app: Log level is enabled. level="vector=info,codec=info,vrl=info,file_source=info,tower_limit=trace,rdkafka=info,buffers=info,lapin=info,kube=info"
supabase-vector                 | 2023-10-17T16:01:59.059907Z  INFO vector::app: Loading configs. paths=["etc/vector/vector.yml"]
supabase-vector                 | 2023-10-17T16:01:59.102856Z  WARN vector::config::loading: Transform "router._unmatched" has no consumers
supabase-vector                 | 2023-10-17T16:01:59.103118Z  INFO source{component_kind="source" component_id=docker_host component_type=docker_logs component_name=docker_host}: vector::sources::docker_logs: Capturing logs from now on. now=2023-10-17T16:01:59.103070781+00:00
supabase-vector                 | 2023-10-17T16:01:59.103191Z  INFO source{component_kind="source" component_id=docker_host component_type=docker_logs component_name=docker_host}: vector::sources::docker_logs: Listening to docker log events.
supabase-vector                 | 2023-10-17T16:01:59.242255Z  INFO vector::topology::running: Running healthchecks.
supabase-vector                 | 2023-10-17T16:01:59.242316Z  INFO vector::topology::builder: Healthcheck passed.
supabase-vector                 | 2023-10-17T16:01:59.242336Z  INFO vector::topology::builder: Healthcheck passed.
supabase-vector                 | 2023-10-17T16:01:59.242347Z  INFO vector::topology::builder: Healthcheck passed.
supabase-vector                 | 2023-10-17T16:01:59.242355Z  INFO vector::topology::builder: Healthcheck passed.
supabase-vector                 | 2023-10-17T16:01:59.242362Z  INFO vector::topology::builder: Healthcheck passed.
supabase-vector                 | 2023-10-17T16:01:59.242376Z  INFO vector::topology::builder: Healthcheck passed.
supabase-vector                 | 2023-10-17T16:01:59.242384Z  INFO vector::topology::builder: Healthcheck passed.
supabase-vector                 | 2023-10-17T16:01:59.242545Z  INFO vector: Vector has started. debug="false" version="0.28.1" arch="x86_64" revision="ff15924 2023-03-06"
supabase-vector                 | 2023-10-17T16:01:59.242592Z ERROR source{component_kind="source" component_id=docker_host component_type=docker_logs component_name=docker_host}: vector::sources::docker_logs: Listing currently running containers failed. error=error trying to connect: Connection refused (os error 111)
supabase-vector                 | 2023-10-17T16:01:59.250513Z  INFO vector::internal_events::api: API server running. address=0.0.0.0:9001 playground=http://0.0.0.0:9001/playground
supabase-vector                 | 2023-10-17T16:01:59.250532Z  INFO vector::app: All sources have finished.
supabase-vector                 | 2023-10-17T16:01:59.250535Z  INFO vector: Vector has stopped.
supabase-vector                 | 2023-10-17T16:01:59.250585Z  INFO vector::topology::running: Shutting down... Waiting on running components. remaining_components="kong_err, db_logs, logflare_db, logflare_functions, router, logflare_kong, rest_logs, logflare_rest, auth_logs, storage_logs, logflare_auth, logflare_storage, realtime_logs, logflare_realtime, project_logs, kong_logs" time_remaining="59 seconds left"
dependency failed to start: container supabase-vector exited (0)

Despite this, supabase is running well except for logflare (which depends on supabase-vector, so this makes sense).

As for the others above, running docker compose restart seems to work, as there is no error shown:


[+] Restarting 12/12
 ✔ Container supabase-storage                Started                                                                                                                                                                                                                     2.1s 
 ✔ Container supabase-studio                 Started                                                                                                                                                                                                                     9.7s 
 ✔ Container supabase-db                     Started                                                                                                                                                                                                                     7.2s 
 ✔ Container supabase-meta                   Started                                                                                                                                                                                                                     4.0s 
 ✔ Container supabase-analytics              Started                                                                                                                                                                                                                    10.0s 
 ✔ Container realtime-dev.supabase-realtime  Started                                                                                                                                                                                                                     2.9s 
 ✔ Container supabase-rest                   Started                                                                                                                                                                                                                     2.4s 
 ✔ Container supabase-edge-functions         Started                                                                                                                                                                                                                     6.0s 
 ✔ Container supabase-kong                   Started                                                                                                                                                                                                                     3.8s 
 ✔ Container supabase-auth                   Started                                                                                                                                                                                                                     4.7s 
 ✔ Container supabase-imgproxy               Started                                                                                                                                                                                                                    11.3s 
 ✔ Container supabase-vector                 Started                                                                                                                                                                                                                     6.9s

But actually when running docker ps it becomse obvious that supabase-vector is still not running.:

CONTAINER ID   IMAGE                              COMMAND                   CREATED         STATUS                    PORTS                                                                                                      NAMES
7da4155c4629   supabase/storage-api:v0.40.4       "docker-entrypoint.s…"    2 minutes ago   Up 49 seconds (healthy)   5000/tcp                                                                                                   supabase-storage
25c3560ecd8b   supabase/gotrue:v2.99.0            "gotrue"                  2 minutes ago   Up 49 seconds (healthy)                                                                                                              supabase-auth
ead938e4844d   supabase/studio:20230921-d657f29   "docker-entrypoint.s…"    2 minutes ago   Up 53 seconds (healthy)   3000/tcp                                                                                                   supabase-studio
548a6ec46112   kong:2.8.1                         "bash -c 'eval \"echo…"   2 minutes ago   Up 58 seconds (healthy)   0.0.0.0:8000->8000/tcp, :::8000->8000/tcp, 8001/tcp, 0.0.0.0:8443->8443/tcp, :::8443->8443/tcp, 8444/tcp   supabase-kong
f16c60c00b24   supabase/postgres-meta:v0.68.0     "docker-entrypoint.s…"    2 minutes ago   Up 58 seconds (healthy)   8080/tcp                                                                                                   supabase-meta
2bc27c96d91d   supabase/realtime:v2.10.1          "/usr/bin/tini -s -g…"    2 minutes ago   Up 59 seconds (healthy)                                                                                                              realtime-dev.supabase-realtime
bbd93453a9b3   supabase/edge-runtime:v1.18.1      "edge-runtime start …"    2 minutes ago   Up 57 seconds                                                                                                                        supabase-edge-functions
9b6b80efe239   postgrest/postgrest:v11.2.0        "postgrest"               2 minutes ago   Up About a minute         3000/tcp                                                                                                   supabase-rest
4f2b4e4aabf9   supabase/logflare:1.4.0            "sh -c '`cat <<EOF >…"    2 minutes ago   Up 52 seconds (healthy)   0.0.0.0:4000->4000/tcp, :::4000->4000/tcp                                                                  supabase-analytics
5e2c81933f45   supabase/postgres:15.1.0.117       "docker-entrypoint.s…"    2 minutes ago   Up 55 seconds (healthy)   0.0.0.0:5432->5432/tcp, :::5432->5432/tcp                                                                  supabase-db
fb5421fbe1ae   darthsim/imgproxy:v3.8.0           "imgproxy"                3 minutes ago   Up 51 seconds (healthy)   8080/tcp                                                                                                   supabase-imgproxy

Does anybody have the idea, where the problem is coming from?

I run into this issue, too.

  • I tried to import docker-compose.yml + .env (after customizing it) into portainer.
  • I added both files and I deployed it in portainer.
  • Portainer pulled all images but reported a vector problem.
  • In command line, I issued docker compose down.
  • I issued docker compose pull and later up.
  • It seemed running, but nothing listened on port 3000.
  • I added a missing ports section to the docker-composer.yml

Kijelölés_521

  • Now I have a working environment (I got studio, I was able to add extensions etc.)

I am not sure, what was about the vector, but I did nothing special, just the typical microsoft method: switch off and on and off and on.

For me, doing a sudo docker compose restart solved the issue.

+1

For me, doing a sudo docker compose restart solved the issue.

THIS! Got the same error and doing a compose restart was the only solution that worked.

sudo docker compose restart

This docker compose restart works for me as well for the following error

root@base:~/supabase/docker# docker compose up -d
...
...
...
 ⠿ kong Pulled                                                                                                              38.7s
   ⠿ 213ec9aee27d Pull complete                                                                                             15.3s
   ⠿ a70653f7a2d5 Pull complete                                                                                             15.3s
   ⠿ 531e3bd93090 Pull complete                                                                                             36.8s
   ⠿ 814dd06d26c7 Pull complete                                                                                             36.8s
 ⠿ rest Pulled                                                                                                              10.2s
   ⠿ e20ce8189632 Pull complete                                                                                              8.1s
WARN[0000] The "MFA_ENABLED" variable is not set. Defaulting to a blank string.
[+] Running 12/13
 ⠿ Network docker_default                    Created                                                                         0.2s
 ⠿ Container supabase-vector                 Waiting                                                                         1.6s
 ⠿ Container supabase-imgproxy               Started                                                                         1.0s
 ⠿ Container supabase-db                     Created                                                                         0.0s
 ⠿ Container supabase-analytics              Created                                                                         0.0s
 ⠿ Container supabase-edge-functions         Created                                                                         0.1s
 ⠿ Container supabase-rest                   Created                                                                         0.1s
 ⠿ Container supabase-meta                   Created                                                                         0.1s
 ⠿ Container realtime-dev.supabase-realtime  Created                                                                         0.1s
 ⠿ Container supabase-kong                   Created                                                                         0.1s
 ⠿ Container supabase-studio                 Created                                                                         0.1s
 ⠿ Container supabase-auth                   Created                                                                         0.1s
 ⠿ Container supabase-storage                Created                                                                         0.0s
container for service "vector" is unhealthy

root@base:~/supabase/docker# docker compose ps
WARN[0000] The "MFA_ENABLED" variable is not set. Defaulting to a blank string.
NAME                             COMMAND                   SERVICE             STATUS              PORTS
realtime-dev.supabase-realtime   "/usr/bin/tini -s -g…"    realtime            created
supabase-analytics               "sh -c '`cat <<EOF >…"    analytics           created
supabase-auth                    "gotrue"                  auth                created
supabase-db                      "docker-entrypoint.s…"    db                  created
supabase-edge-functions          "edge-runtime start …"    functions           created
supabase-imgproxy                "imgproxy"                imgproxy            running (healthy)   8080/tcp
supabase-kong                    "bash -c 'eval \"echo…"   kong                created
supabase-meta                    "docker-entrypoint.s…"    meta                created
supabase-rest                    "postgrest"               rest                created
supabase-storage                 "docker-entrypoint.s…"    storage             created
supabase-studio                  "docker-entrypoint.s…"    studio              created
supabase-vector                  "/usr/local/bin/vect…"    vector              exited (78)

But i realized that vector container is not working and showing docker psafter restart. This is really headache when someone is touching first supabase. Creating several droplets to try different variants. Resizing but eventually docker psis not responding. Url freezing, sorry its not good welcoming… Going away to alternatives, best for all