next.js: Missing dependencies when using standalone output with pnpm 8

Verify canary release

  • I verified that the issue exists in the latest Next.js canary release

Provide environment information

Operating System:
      Platform: darwin
      Arch: arm64
      Version: Darwin Kernel Version 22.3.0: Mon Jan 30 20:38:37 PST 2023; root:xnu-8792.81.3~2/RELEASE_ARM64_T6000
    Binaries:
      Node: 16.19.1
      npm: 8.19.3
      Yarn: 1.22.19
      pnpm: 8.1.0
    Relevant packages:
      next: 13.2.5-canary.32
      eslint-config-next: N/A
      react: 18.2.0
      react-dom: 18.2.0

Which area(s) of Next.js are affected? (leave empty if unsure)

Standalone mode (output: “standalone”)

Link to the code that reproduces this issue

https://github.com/Bubbleinpit/nextjs13-standalone-with-pnpm8

To Reproduce

You can follow this README of the repo above to reproduce the issue. For convenience, I will also describe it here.

  1. execute next build and copy all necessary files to output dir
pnpm build

[ -d "build" ] && rm -r build
mkdir build

cp -r ./public ./build
cp -r ./.next/standalone/* ./build
cp -r ./.next/standalone/.next ./build
cp -r ./.next/static ./build/.next
  1. move output dir to another place and execute node server.js in it
mv ./build ~/build
cd ~/build
node ./server.js

Then you will see:

node:internal/modules/cjs/loader:1024
  throw err;
  ^

Error: Cannot find module 'styled-jsx'
Require stack:
- /Users/username/build/node_modules/next/dist/build/webpack/require-hook.js
- /Users/username/build/node_modules/next/dist/server/initialize-require-hook.js
- /Users/username/build/node_modules/next/dist/server/next-server.js
- /Users/username/build/server.js
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:1021:15)
    at Function.resolve (node:internal/modules/cjs/helpers:114:19)
    at Object.loadRequireHook (/Users/username/build/node_modules/next/dist/build/webpack/require-hook.js:38:21)
    at Object.<anonymous> (/Users/username/build/node_modules/next/dist/server/initialize-require-hook.js:3:19)
    at Module._compile (node:internal/modules/cjs/loader:1191:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1245:10)
    at Module.load (node:internal/modules/cjs/loader:1069:32)
    at Function.Module._load (node:internal/modules/cjs/loader:904:12)
    at Module.require (node:internal/modules/cjs/loader:1093:19)
    at require (node:internal/modules/cjs/helpers:108:18) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/Users/username/build/node_modules/next/dist/build/webpack/require-hook.js',
    '/Users/username/build/node_modules/next/dist/server/initialize-require-hook.js',
    '/Users/username/build/node_modules/next/dist/server/next-server.js',
    '/Users/username/build/server.js'
  ]
}

Describe the Bug

I’m using pnpm 8 and nextjs 13. And I turn on the appDir. I set the output mode as standalone to build the code on my dev computer and to deploy it on a prod server.

When I try to execute the server.js, it always says Error: Cannot find module 'xxx'.

node:internal/modules/cjs/loader:1024
  throw err;
  ^

Error: Cannot find module 'styled-jsx'
Require stack:
- /Users/username/build/node_modules/next/dist/build/webpack/require-hook.js
- /Users/username/build/node_modules/next/dist/server/initialize-require-hook.js
- /Users/username/build/node_modules/next/dist/server/next-server.js
- /Users/username/build/server.js
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:1021:15)
    at Function.resolve (node:internal/modules/cjs/helpers:114:19)
    at Object.loadRequireHook (/Users/username/build/node_modules/next/dist/build/webpack/require-hook.js:38:21)
    at Object.<anonymous> (/Users/username/build/node_modules/next/dist/server/initialize-require-hook.js:3:19)
    at Module._compile (node:internal/modules/cjs/loader:1191:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1245:10)
    at Module.load (node:internal/modules/cjs/loader:1069:32)
    at Function.Module._load (node:internal/modules/cjs/loader:904:12)
    at Module.require (node:internal/modules/cjs/loader:1093:19)
    at require (node:internal/modules/cjs/helpers:108:18) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/Users/username/build/node_modules/next/dist/build/webpack/require-hook.js',
    '/Users/username/build/node_modules/next/dist/server/initialize-require-hook.js',
    '/Users/username/build/node_modules/next/dist/server/next-server.js',
    '/Users/username/build/server.js'
  ]
}

Now I have to reinstall the dependencies in the output dir on my prod server to solve this issue. This is too much of a waste of time.

And I am sure that if I switch to use npm as the package manager, the issue will go away. I verified that.

Expected Behavior

I can start the server successfully with the steps to reproduce.

Which browser are you using? (if relevant)

No response

How are you deploying your application? (if relevant)

No response

About this issue

  • Original URL
  • State: open
  • Created a year ago
  • Reactions: 1
  • Comments: 15 (5 by maintainers)

Most upvoted comments

I think this is related to the architecture design of pnpm, which may use some symbolic links and hard links to reuse npm packages. When I searched for past issues of Next.js, I found that Next.js had also done some handling for this situation, such as copying the original files instead of just copying symbolic links. However, it seems that this handling is not comprehensive enough. I am not sure if this issue exists in Next.js 12, maybe we can try Next.js 12 again to figure out if the problem was introduced in version 13.

+3
Bubbleinpit on Apr 7, 2023

It seems that symlink problems can be avoided by installing the package with node-linker=hoisted in the pnpm configuration before standalone output.

ref. https://pnpm.io/ja/npmrc#node-linker

+2
ahomu on Aug 31, 2023

We hit this as well, when trying to build with pnpm but also when trying to build using Bazel and rules_js (which emulates the sort of directory structure pnpm creates, with symlinks everywhere, but doesn’t actually directly use pnpm). The issue is that the node_modules dir in standalone/node_modules just contains symlinks to the actual module files elsewhere on the filesystem, so if you copy just the standalone folder (as I would expect to work and does with yarn) you end up with broken symlinks that don’t go anywhere. If you copy the entire source .pnpm folder that the symlinks point to then you end up with a load of other dependencies for unrelated things also being worked on on that machine.

Ideally IMO all directories and ‘leaf’ files that are being copied into standalone by Next.js as part of the build process should be checked, and if they’re symlinks they should be dereferenced (potentially multiple times) and the target ‘real’ file copied.

As it is at the moment I’m just bypassing all of our pnpm and bazel code by loading the source tree into a build container and using yarn in there, then copying the resulting standalone folder into the output container. But that contaminates the rest of our build pipeline. I can’t have anything else use this component except as a black-box container, and I have to manually build within the container several dependencies that have already been built by bazel, and lose all caching of output artifacts etc. My CTO isn’t very happy about it, but it works…

+1
dancmeyers on May 30, 2023

I figured it out. Since the way i was doing Dockerfile was build outside of container, then copy the results from my local machine to container. I was having the impression that the “standalone” build is a bundled version of the node server with all the dependencies like the way i would bundle a express server for instance. And all i have to do is copy over the bundled server files.

But since it does depend on node_modules, this would not work. It seems like the only way to do this is to install dependencies and build all inside containers like the example Dockerfile.

+1
chentselee on Apr 24, 2023

@DuCanhGH Thank you for your help. Perhaps I should deploy using Docker, which may solve the problem. Maybe the behavior of the COPY command in the Dockerfile is different from the cp command in Linux. Thanks again, I will give it a try.

+1
Bubbleinpit on Apr 7, 2023

@Bubbleinpit Looking back, there’s this part in my Dockerfile, which originated from Next’s examples/with-docker:

# Production image
FROM node:18-alpine AS runner
WORKDIR /app

ENV NODE_ENV production

# Uncomment the following line in case you want to disable telemetry during runtime.
# ENV NEXT_TELEMETRY_DISABLED 1

RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs

COPY --from=installer --chown=nextjs:nodejs /app/apps/web/.next/standalone ./
COPY --from=installer --chown=nextjs:nodejs /app/apps/web/.next/static ./apps/web/.next/static
COPY --from=installer --chown=nextjs:nodejs /app/apps/web/public ./apps/web/public

USER nextjs

CMD node apps/web/server.js

And I’m using pnpm, which makes this problem look somewhat weird - I think Next is handling this correctly. I will try to look into it further later, but perhaps you will need help from the maintainers…

+1
DuCanhGH on Apr 7, 2023
Read more comments on GitHub
← next.js: Misleading documentation: variables in .env and .env.test are not being loaded in test environment
next.js: Missing node_modules in standalone output →