superduperdb: [BUG]: Unable to start backend service

System Information

{
  "cfg": {
    "apis": {
      "providers": {},
      "retry": {
        "stop_after_attempt": 2,
        "wait_max": 10.0,
        "wait_min": 4.0,
        "wait_multiplier": 1.0
      }
    },
    "cdc": false,
    "dask": {
      "password": "",
      "port": 8786,
      "username": "",
      "ip": "localhost",
      "deserializers": [],
      "serializers": [],
      "local": true
    },
    "data_layers": {
      "artifact": {
        "cls": "mongodb",
        "connection": "pymongo",
        "kwargs": {
          "password": "",
          "port": 27017,
          "username": "",
          "host": "localhost"
        },
        "name": "_filesystem:test_db"
      },
      "data_backend": {
        "cls": "mongodb",
        "connection": "pymongo",
        "kwargs": {
          "password": "",
          "port": 27017,
          "username": "",
          "host": "localhost"
        },
        "name": "test_db"
      },
      "metadata": {
        "cls": "mongodb",
        "connection": "pymongo",
        "kwargs": {
          "password": "",
          "port": 27017,
          "username": "",
          "host": "localhost"
        },
        "name": "test_db"
      }
    },
    "distributed": false,
    "logging": {
      "level": "INFO",
      "type": "STDERR",
      "kwargs": {}
    },
    "model_server": {
      "password": "",
      "port": 5001,
      "username": "",
      "host": "127.0.0.1"
    },
    "notebook": {
      "ip": "0.0.0.0",
      "password": "",
      "port": 8888,
      "token": ""
    },
    "server": {
      "host": "127.0.0.1",
      "port": 3223,
      "protocol": "http"
    },
    "vector_search": {
      "host": "localhost",
      "password": "",
      "port": 19530,
      "type": {
        "backfill_batch_size": 100,
        "inmemory": true
      },
      "backfill_batch_size": 100,
      "username": ""
    },
    "verbose": false,
    "downloads": {
      "hybrid": false,
      "root": "data/downloads"
    }
  },
  "cwd": "/Users/nicole/Documents/School Stuff/Third Year/Second Semester/Computer Graphics/Homeworks/superduperdb",
  "git": {
    "branch": "('branch', '--show-current') failed with [Errno 2] No such file or directory: 'branch'",
    "commit": "('show', '-s', '--format=\"%h: %s\"') failed with [Errno 2] No such file or directory: 'show'"
  },
  "hostname": "Nicoles-MBP-2",
  "os_uname": [
    "Darwin",
    "Nicoles-MBP-2",
    "18.7.0",
    "Darwin Kernel Version 18.7.0: Tue Aug 20 16:57:14 PDT 2019; root:xnu-4903.271.2~2/RELEASE_X86_64",
    "x86_64"
  ],
  "package_versions": {},
  "platform": {
    "platform": "macOS-10.14.6-x86_64-i386-64bit",
    "python_version": "3.11.4"
  },
  "startup_time": "2023-08-21 13:13:52.727953",
  "superduper_db_root": "/Users/nicole/Documents/School Stuff/Third Year/Second Semester/Computer Graphics/Homeworks/superduperdb",
  "sys": {
    "argv": [
      "/Users/nicole/Documents/School Stuff/Third Year/Second Semester/Computer Graphics/Homeworks/superduperdb/superduperdb/__main__.py",
      "info"
    ],
    "path": [
      "/Users/nicole/Documents/School Stuff/Third Year/Second Semester/Computer Graphics/Homeworks/superduperdb",
      "/Library/Frameworks/Python.framework/Versions/3.11/lib/python311.zip",
      "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11",
      "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/lib-dynload",
      "/Users/nicole/Documents/School Stuff/Third Year/Second Semester/Computer Graphics/Homeworks/superduperdb/venv/lib/python3.11/site-packages"
    ]
  }
}

What happened?

Cannot start the backend service after installation of required packages, keep receiving an error of a module not found even though the package is installed. Things I’ve tried:

1. Attempted to pin the versions of the packages
2. Create a virtualenv in python 3.8 (which is the minimum version of python required by some of the packages)
3. Upgrade lancedb
4. uninstall and install lance & lancedb
5. Remove cached folder and retry
6. Tried running it in a Conda environment
7. Installed packages individually (not with requirements.in)
8. Tried to install according to contribution guidelines (install -e .[dev with the pyproject.toml)
9. Updating my IDE 😅

Steps to reproduce

1. Create virtualenv in python 3.11 & activate it: a. virtualenv venv b. source venv/bin/activate
2. install relevant requirements (requirements.in) in the virtualenv : pip install -r backend/requirements.in - make sure you’re in the question-the-docs path
3. attempt to startup the service with uvicorn backend.main:app --host 0.0.0.0 --port 8000 --reload
4. Encounter error 😃

Relevant log output

ERROR:    Traceback (most recent call last):
  File "/Users/nicole/IntellijProjects/superduperdb/venv/lib/python3.11/site-packages/starlette/routing.py", line 677, in lifespan
    async with self.lifespan_context(app) as maybe_state:
  File "/Users/nicole/IntellijProjects/superduperdb/venv/lib/python3.11/site-packages/starlette/routing.py", line 566, in __aenter__
    await self._router.startup()
  File "/Users/nicole/IntellijProjects/superduperdb/venv/lib/python3.11/site-packages/starlette/routing.py", line 656, in startup
    handler()
  File "/Users/nicole/IntellijProjects/superduperdb/apps/question-the-docs/backend/app.py", line 38, in startup_db_client
    _app.superduperdb = superduper(_app.mongodb)
                        ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nicole/IntellijProjects/superduperdb/superduperdb/misc/superduper.py", line 14, in run
    return dts[0].create(item, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nicole/IntellijProjects/superduperdb/superduperdb/misc/superduper.py", line 45, in create
    from superduperdb.db.base.build import build_vector_database
  File "/Users/nicole/IntellijProjects/superduperdb/superduperdb/db/base/build.py", line 4, in <module>
    from superduperdb.db.base.backends import artifact_stores
  File "/Users/nicole/IntellijProjects/superduperdb/superduperdb/db/base/backends.py", line 8, in <module>
    from superduperdb.vector_search.lancedb_client import LanceVectorIndex
  File "/Users/nicole/IntellijProjects/superduperdb/superduperdb/vector_search/lancedb_client.py", line 5, in <module>
    import lancedb
  File "/Users/nicole/IntellijProjects/superduperdb/venv/lib/python3.11/site-packages/lancedb/__init__.py", line 14, in <module>
    from .db import LanceDBConnection, URI
  File "/Users/nicole/IntellijProjects/superduperdb/venv/lib/python3.11/site-packages/lancedb/db.py", line 20, in <module>
    from .table import LanceTable
  File "/Users/nicole/IntellijProjects/superduperdb/venv/lib/python3.11/site-packages/lancedb/table.py", line 24, in <module>
    from lance.vector import vec_to_table
ModuleNotFoundError: No module named 'lance.vector'