turborepo: [turborepo] Using turbo prune via Dockerfile doesn't generate turbo out folder

What version of Turborepo are you using?

1.9.3

What package manager are you using / does the bug impact?

pnpm

What operating system are you using?

Mac

Describe the Bug

Hello,

I am in the process of migrating from Yarn to PNPM and have written a generic Dockerfile to reuse for Next.js applications. I am facing an issue when running the project via Docker, where the turbo out folder is not being generated. Based on my understanding, this should happen when we run the following command:

RUN turbo prune ${APP_NAME} --docker
ARG NODE=bitnami/node:20.9.0

FROM ${NODE} as builder
WORKDIR /app
ARG APP_NAME
COPY . .
RUN npm install -g pnpm
RUN npm install -g turbo
RUN echo $(turbo --version)
RUN turbo prune ${APP_NAME} --docker

FROM builder as installer
ARG CACHE_CLEAN=true
ARG APP_NAME
WORKDIR /app/apps/${APP_NAME}
COPY --from=builder /app/out/json/ .
COPY --from=builder /app/out/pnpm-lock.yaml ./pnpm-lock.yaml
RUN pnpm install && \
    if [ "$CACHE_CLEAN" = "true" ]; then pnpm store prune; fi
ENV PATH /app/apps/${APP_NAME}/node_modules/.bin:$PATH



# Build the project and its dependencies
COPY --from=builder /app/out/full/ .
COPY turbo.json turbo.json
ENV NEXT_TELEMETRY_DISABLED 1
RUN turbo run build --filter=${APP_NAME}...

FROM installer as production
ARG APP_NAME
ARG PORT
ENV NEXT_TELEMETRY_DISABLED 1
WORKDIR /app
ENV NODE_ENV=production

RUN addgroup --system --gid 1001 nodejs && \
    adduser --system --uid 1001 nextjs

COPY --from=installer /app/apps/${APP_NAME}/ ./
USER nextjs
EXPOSE ${PORT}
ENV PORT ${PORT}

CMD node --max-old-space-size=200 --optimize_for_size --gc_interval=100 server.js

For local development, I run it via docker-compose and pass the APP_NAME variable like this:

app-1:
    container_name: app1
    build:
      context: .
      dockerfile: ./tools/docker/nextjs/Dockerfile
      target: installer
      args:
        APP_NAME: app1
        PORT: 3000
        CACHE_CLEAN: "false"
    working_dir: /app
    restart: always
    environment:
      - NODE_ENV=development
    volumes:
      - .:/app
    ports:
      - "3000:3000"
    command: pnpm --filter app1 run dev

Here is my turbo.json configuration:

{
  "$schema": "https://turbo.build/schema.json",
  "pipeline": {
    "build": {
      "outputs": ["dist/**", ".next/**", "!.next/cache/**", "public/dist/**"],
      "dependsOn": ["^build"]
    },
    "test": {
      "outputs": ["coverage/**"],
      "dependsOn": []
    },
    "lint": {
      "dependsOn": ["^build"]
    },
    "dev": {
      "cache": false,
      "persistent": true
    },
    "clean": {
      "cache": false
    }
  }
}

Any help or guidance on resolving this issue would be greatly appreciated.

Expected Behavior

out folder should be generated on the root of monorepo

To Reproduce

Build the docker image and run the container.

Reproduction Repo

No response

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 23 (3 by maintainers)

Most upvoted comments

Thankfully, because of this thread, I now have a working Dockerfile for a Next app in Turborepo using PNPM.

I ran into a few issues on the way:

  1. Missing packages. First time running pnpm install I got a bunch of errors, this was due to some node dependencies depending on Python.
- RUN apk add --no-cache libc6-compat
+ RUN apk add --no-cache libc6-compat python3 py3-pip make g++
  1. Not copying over pnpm-workspace.yaml
COPY --from=builder /app/out/pnpm-lock.yaml ./pnpm-lock.yaml
+ COPY --from=builder /app/out/pnpm-workspace.yaml ./pnpm-workspace.yaml
  1. Move the install below /out/full copy. I got an error that I was missing a dependecy from an internal package my web app is using.
- RUN pnpm install
-
- COPY --from=builder /app/out/full/ .

+ COPY --from=builder /app/out/full/ .
+ 
+ RUN pnpm install
  1. Next.config configuration Make sure your next.config includes the standalone output:
// filename: next.config.mjs

/** @type {import('next').NextConfig} */
import path from 'path';
import { fileURLToPath } from 'url';

const __dirname = path.dirname(fileURLToPath(import.meta.url));

const nextConfig = {
+  output: 'standalone',
+  experimental: {
+   outputFileTracingRoot: path.join(__dirname, '../../'),
+  },
};

export default nextConfig;

Working Dockerfile

FROM node:20-alpine AS base

RUN corepack enable && corepack prepare pnpm@latest --activate
ENV PNPM_HOME="/pnpm"
ENV PATH="$PNPM_HOME:$PATH"
RUN pnpm add -g turbo@^2

FROM base AS builder
WORKDIR /app
COPY . .

# Generate a partial monorepo with a pruned lockfile for a target workspace.
RUN turbo prune customer --docker

FROM base AS installer
RUN apk add --no-cache libc6-compat python3 py3-pip make g++
RUN apk update

WORKDIR /app

COPY --from=builder /app/out/json/ .
COPY --from=builder /app/out/pnpm-lock.yaml ./pnpm-lock.yaml
COPY --from=builder /app/out/pnpm-workspace.yaml ./pnpm-workspace.yaml
COPY --from=builder /app/out/full/ .

RUN pnpm install
RUN pnpm turbo run build --filter=customer...

FROM base AS runner
WORKDIR /app

# Don't run production as root
RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs
USER nextjs

# NOTE: I'm using `.mjs`
COPY --from=installer /app/apps/customer/next.config.mjs .
COPY --from=installer /app/apps/customer/package.json .

# Automatically leverage output traces to reduce image size
# https://nextjs.org/docs/advanced-features/output-file-tracing
COPY --from=installer --chown=nextjs:nodejs /app/apps/customer/.next/standalone ./
COPY --from=installer --chown=nextjs:nodejs /app/apps/customer/.next/static ./apps/customer/.next/static
COPY --from=installer --chown=nextjs:nodejs /app/apps/customer/public ./apps/customer/public

EXPOSE 3000

ENV PORT 3000
CMD HOSTNAME="0.0.0.0" node apps/customer/server.js
docker build -f apps/customer/Dockerfile -t customer .
docker run -it -p 3000:3000 customer

I manage to make it run adding prop-deps

+FROM installer AS prod-deps
+WORKDIR /app
+RUN --mount=type=cache,id=pnpm,target=/pnpm/store pnpm install --prod --frozen-lockfile

Then

-FROM base AS runner
+FROM base AS prod-runner

+COPY --from=prod-deps --chown=nextjs:nodejs /app/node_modules ./node_modules
+COPY --from=prod-deps --chown=nextjs:nodejs /app/apps/web/package.json ./apps/web/package.json

So, my version is

“turbo”: “^1.9.3”, “next”: “13.1.5”,

For now, i decided to use this Dockerfile which works perfectly fine for now, and not utilizing turbo for building the app

# This is a generic Dockerfile for all apps which uses variable names and based on the execution context it takes the context of the apps
# APP_NAME and PORT locally are taken from docker-compose.yaml and in pipeline it's passed via the yaml file 
# of the repository host/DevSecOps like Giltab with .gitlab-ci.yaml
ARG NODE=bitnami/node:20.9.0

# BUILDER STAGE
FROM ${NODE} as installer
WORKDIR /app
ARG APP_NAME
COPY apps/${APP_NAME}/ ./
RUN npm install -g pnpm && pnpm install

# INSTALL DEPENDENCIES STAGE
FROM installer as builder
WORKDIR /app
RUN pnpm run build

# PRODUCTION DEPLOYMENT STAGE
FROM ${NODE} as runner_production
WORKDIR /app
ARG PORT
ENV NODE_ENV=production

RUN addgroup --system --gid 1001 nodejs && \
    adduser --system --uid 1001 nextjs

COPY --from=builder /app/next.config.js ./
COPY --from=builder /app/package.json ./package.json
# https://nextjs.org/docs/advanced-features/output-file-tracing
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static

EXPOSE ${PORT}

ENV PORT ${PORT}


USER nextjs
# When using sidecar/istio-proxy you need to tweak --max-old-space-size and --gc_interval for optimal memory management, usually at the moment
# one istio-proxy is taking 150Mi, when cloudops allow us to create sidecar objects we need to define namespaces from whom we use service mesh,
# that will reduce the memory footprint of istio-proxy, since currently one pod app starts with 20-25Mi while one istio-proxy pod starts with 100-150Mi
# That 100-150Mi should be taken in consideration when defining K8 resource limits.
# Usually in node.js you define 70-80% of what resource memory limit you define in Kubernetes, 
# but that's not the golden rule (tweak lower or higher values for optimal management), also substract 150Mi (istio-proxy memory) from that 70-80%. 
CMD node --max-old-space-size=200 --optimize_for_size --gc_interval=100 server.js

I am having a similar issues when I try to install using yarn 4.0.1. But I am missing next, and you next-server. Not sure if it is related.

2023-11-07 12:52:00 node:internal/modules/cjs/loader:1051
2023-11-07 12:52:00   throw err;
2023-11-07 12:52:00   ^
2023-11-07 12:52:00 
2023-11-07 12:52:00 Error: Cannot find module 'next'
2023-11-07 12:52:00 Require stack:
2023-11-07 12:52:00 - /app/apps/pio-app/server.js
2023-11-07 12:52:00     at Module._resolveFilename (node:internal/modules/cjs/loader:1048:15)
2023-11-07 12:52:00     at Module._load (node:internal/modules/cjs/loader:901:27)
2023-11-07 12:52:00     at Module.require (node:internal/modules/cjs/loader:1115:19)
2023-11-07 12:52:00     at require (node:internal/modules/helpers:130:18)
2023-11-07 12:52:00     at Object.<anonymous> (/app/apps/pio-app/server.js:23:1)
2023-11-07 12:52:00     at Module._compile (node:internal/modules/cjs/loader:1241:14)
2023-11-07 12:52:00     at Module._extensions..js (node:internal/modules/cjs/loader:1295:10)
2023-11-07 12:52:00     at Module.load (node:internal/modules/cjs/loader:1091:32)
2023-11-07 12:52:00     at Module._load (node:internal/modules/cjs/loader:938:12)
2023-11-07 12:52:00     at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:83:12) {
2023-11-07 12:52:00   code: 'MODULE_NOT_FOUND',
2023-11-07 12:52:00   requireStack: [ '/app/apps/pio-app/server.js' ]
2023-11-07 12:52:00 }
2023-11-07 12:52:00 
2023-11-07 12:52:00 Node.js v20.9.0

I have gotten it to work with pnpm, this might help you:

FROM node:20-alpine AS base

RUN npm install -g pnpm
RUN npm install -g turbo

FROM base AS builder
RUN apk add --no-cache libc6-compat
RUN apk update

# Set working directory
WORKDIR /app
COPY . .
RUN turbo prune @pio/app --docker

FROM base AS installer
RUN apk add --no-cache libc6-compat
RUN apk update
WORKDIR /app

COPY turbo.json turbo.json
COPY --from=builder /app/out/json/ .
COPY --from=builder /app/out/pnpm-lock.yaml ./pnpm-lock.yaml
RUN pnpm install

COPY --from=builder /app/out/full/ .
RUN turbo run build --filter=@pio/app...

FROM base AS runner
WORKDIR /app

ENV NODE_ENV production

RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs
USER nextjs

COPY --from=installer /app/apps/pio-app/next.config.js .
COPY --from=installer /app/apps/pio-app/package.json .

COPY --from=installer --chown=nextjs:nodejs /app/apps/pio-app/.next/standalone ./
COPY --from=installer --chown=nextjs:nodejs /app/apps/pio-app/.next/static ./apps/pio-app/.next/static
COPY --from=installer --chown=nextjs:nodejs /app/apps/pio-app/public ./apps/pio-app/public

EXPOSE 3000

ENV NEXT_TELEMETRY_DISABLED 1

CMD ["node", "apps/pio-app/server.js"]

Feels like turbo is not made for yarn berry,