backstage: Cannot find module 'better-sqlite3' when running in Docker Container

Expected Behavior

Backstage should start up

Actual Behavior

When trying to start up Backstage in a Docker container we get: Cannot find module ‘better-sqlite3’

The fix is to move better-sqlite3": "^7.5.0 from the devDependencies section to the dependencies section in the \packages\backend\packages.json file.

Steps to Reproduce

Ran npx @backstage/create-app to create a instances of Backstage, my version in the backstage.json is 1.2.0
Then I added a Dockerfile to the root and used this, pretty much the same as from the multi-stage docker docks on backstage.io:

# Stage 1 - Create yarn install skeleton layer
FROM node:16-bullseye-slim AS packages

WORKDIR /app
COPY package.json yarn.lock ./

COPY packages packages

# Comment this out if you don't have any internal plugins
COPY plugins plugins

RUN find packages \! -name "package.json" -mindepth 2 -maxdepth 2 -exec rm -rf {} \+

# Stage 2 - Install dependencies and build packages
FROM node:16-bullseye-slim AS build

WORKDIR /app
COPY --from=packages /app .

# install sqlite3 dependencies
RUN apt-get update && \
    apt-get install -y --no-install-recommends libsqlite3-dev python3 build-essential && \
    yarn config set python /usr/bin/python3

RUN yarn install --frozen-lockfile --network-timeout 600000 && rm -rf "$(yarn cache dir)"

COPY . .

RUN yarn tsc
RUN yarn --cwd packages/backend build

# Stage 3 - Build the actual backend image and install production dependencies
FROM node:16-bullseye-slim

WORKDIR /app

# install sqlite3 and TechDocs dependencies
RUN apt-get update && \
    apt-get install -y --no-install-recommends libsqlite3-dev python3 build-essential && \
    rm -rf /var/lib/apt/lists/* && \
    yarn config set python /usr/bin/python3

# Copy the install dependencies from the build stage and context
COPY --from=build /app/yarn.lock /app/package.json /app/packages/backend/dist/skeleton.tar.gz ./
RUN tar xzf skeleton.tar.gz && rm skeleton.tar.gz

RUN yarn install --frozen-lockfile --production --network-timeout 600000 && rm -rf "$(yarn cache dir)"

# Copy the built packages from the build stage
COPY --from=build /app/packages/backend/dist/bundle.tar.gz .
RUN tar xzf bundle.tar.gz && rm bundle.tar.gz

# Copy any other files that we need at runtime
COPY app-config.yaml ./

CMD ["node", "packages/backend", "--config", "app-config.yaml"]

Then make sure your .dockerignore looks like this:

.git
*.local.yaml
node_modules
packages/*/dist
packages/*/node_modules
plugins/*/dist
plugins/*/node_modules

From the root of your project run this to build the docker container: docker image build -t backstage .
Now run the container: docker run -it -p 7007:7007 backstage

Notice the container does not start up and you get this error: Cannot find module ‘better-sqlite3’

Context

This prevents Backstage from running in a docker container based on the documentation on Backstage.io

Your Environment

Google Chrome Version 101.0.4951.64 (Official Build) (arm64)

OS:   Darwin 21.5.0 - darwin/x64
node: v14.18.1
yarn: 1.22.18
cli:  0.17.1 (installed)

Dependencies:
  @backstage/app-defaults                          1.0.2
  @backstage/backend-common                        0.13.5
  @backstage/backend-tasks                         0.3.1
  @backstage/catalog-client                        1.0.2
  @backstage/catalog-model                         1.0.2
  @backstage/cli-common                            0.1.9
  @backstage/cli                                   0.17.1
  @backstage/config-loader                         1.1.1
  @backstage/config                                1.0.1
  @backstage/core-app-api                          1.0.2
  @backstage/core-components                       0.9.4
  @backstage/core-plugin-api                       1.0.2
  @backstage/errors                                1.0.0
  @backstage/integration-react                     1.1.0
  @backstage/integration                           1.2.0
  @backstage/plugin-api-docs                       0.8.5
  @backstage/plugin-app-backend                    0.3.32
  @backstage/plugin-auth-backend                   0.14.0
  @backstage/plugin-auth-node                      0.2.1
  @backstage/plugin-catalog-backend                1.1.2
  @backstage/plugin-catalog-common                 1.0.2
  @backstage/plugin-catalog-graph                  0.2.17
  @backstage/plugin-catalog-import                 0.8.8
  @backstage/plugin-catalog-react                  1.1.0
  @backstage/plugin-catalog                        1.2.0
  @backstage/plugin-github-actions                 0.5.5
  @backstage/plugin-org                            0.5.5
  @backstage/plugin-permission-common              0.6.1
  @backstage/plugin-permission-node                0.6.1
  @backstage/plugin-permission-react               0.4.1
  @backstage/plugin-proxy-backend                  0.2.26
  @backstage/plugin-scaffolder-backend             1.2.0
  @backstage/plugin-scaffolder-common              1.1.0
  @backstage/plugin-scaffolder                     1.2.0
  @backstage/plugin-search-backend-module-pg       0.3.3
  @backstage/plugin-search-backend-node            0.6.1
  @backstage/plugin-search-backend                 0.5.2
  @backstage/plugin-search-common                  0.3.4
  @backstage/plugin-search-react                   0.2.0
  @backstage/plugin-search                         0.8.1
  @backstage/plugin-tech-radar                     0.5.12
  @backstage/plugin-techdocs-backend               1.1.1
  @backstage/plugin-techdocs-module-addons-contrib 1.0.0
  @backstage/plugin-techdocs-node                  1.1.1
  @backstage/plugin-techdocs-react                 1.0.0
  @backstage/plugin-techdocs                       1.1.1
  @backstage/plugin-user-settings                  0.4.4
  @backstage/release-manifests                     0.0.3
  @backstage/search-common                         0.3.4
  @backstage/test-utils                            1.1.0
  @backstage/theme                                 0.2.15
  @backstage/types                                 1.0.0
  @backstage/version-bridge                        1.0.1
✨  Done in 3.01s.

About this issue

Original URL
State: closed
Created 2 years ago
Reactions: 5
Comments: 18 (4 by maintainers)

Commits related to this issue

🐛 fix: hotfix sqilte dep issue See more details: https://github.com/backstage/backstage/issues/11651 — committed to webup/backstage-app by webup 2 years ago

Most upvoted comments

As a relatively new user to Backstage, this added a sizeable “uhhh maybe this isn’t ready for prime time” sentiment. If I follow the documentation about how to deploy Backstage, I expect it to work as documented. In this case, it does not.

jdotw on May 25, 2022

This is intended as of the v1.2 release. We removed SQLite from the Docker image as we now consider that to be the production deployment, which uses PostgreSQL. Having the SQLite dependency present in production means we both need to install additional dependencies from APT, as well as SQLite itself, both of which bloat the image and build process.

It’s something you can ofc change in your own app though, either by adding back better-sqlite as a prod dependency, installing it separately, or perhaps having a separate development image that runs a full yarn install.

Happy to receive feedback on this change though, if you think there’s something we can do that clarifies/simplifies things.

Rugvip on May 24, 2022

Ok, that makes sense to me. I really just logged this as I ran into this when I was trying to help someone on Discord and this was the fastest way to get something up and running. I’m wondering if just adding a note about this on the Docker documentation might be helpful? I’m thinking for those doing a Backstage POC and just want something simple to deploy?

awanlin on May 24, 2022

@derjust some times us as maintainers get a little lost amongst the weeds to be honest, and it’s hard for us to pretend to be people that are potentially first time readers, so we would really appreciate some contributions on where you think we could improve the docs or whatever to make it easier to get setup and stop leading you down rabbit holes. 🤗

Like I said previously, we shuffled some docs around and tried to improve some bits, so it could be that we’ve missed some things, or they don’t make sense anymore.

benjdlambert on Jun 8, 2022

@lenichols The fix is to move better-sqlite3": "^7.5.0 from the devDependencies section to the dependencies section in the \packages\backend\packages.json file.

@markdavidmc0 sorry to hear this has been an issue for you, the above fix does not resolve the problem for you? Could you provide more details then and I can try and help you out?

awanlin on Aug 8, 2022

I can’t talk for @jdotw nor @awanlin - but i think we all fell into the same trap:

While https://backstage.io/docs/deployment/docker mentions

Something that goes for all of these docker deployment strategies is that they are stateless, so for a production deployment you will want to set up and connect to an external PostgreSQL instance where the backend plugins can store their state, rather than using SQLite.

(Which makes sense as a statement but I think - in the spirit of *-as-code -there are valid production use cases that will not need/have a PostgreSQL - thus sticking with SQlite)

Further down in the document the multi-stage build installs SQlite dependencies

# install sqlite3 dependencies

and

# install sqlite3 dependencies, you can skip this if you don't use sqlite3 in the image

thus leaving the impression that the Dockerfile example there is ‘ready-to-go’ for SQlite deployments (and for PostgreSQL one can remove those two lines from the Dockerfile)

Not trying to complain here, just trying to help out as I think Backstage is a great tool though with some rough edges like this one 😃

derjust on Jun 8, 2022