What if I want to build a dockerfile for the Backend? I don’t want all installs of my package.json in the docker image, so I think bundling the nest.js’ dependencies should be possible.

FROM node:10 as builder

ENV WORK_DIR /usr/src/app

RUN mkdir ${WORK_DIR}
WORKDIR ${WORK_DIR}

COPY package.json package-lock.json ${WORK_DIR}/

RUN [ "npm", "install" ]

COPY . ${WORK_DIR}

RUN [ "npm", "run", "build", "--" , "--project=backend", "--prod" ]
# RUN [ "npm", "run", "build", "--" , "--project=frontend", "--prod" ]

FROM node:10

ENV WORK_DIR /usr/src/app
WORKDIR ${WORK_DIR}

COPY --from=builder ${WORK_DIR}/dist/apps/backend ${WORK_DIR}
# COPY --from=builder ${WORK_DIR}/dist/apps/frontend ${WORK_DIR}/assets
EXPOSE 3333
CMD [ "node", "main.js" ]

Starting the resulting container will result in this output:

docker run -it -p 8080:3333 test-container
internal/modules/cjs/loader.js:584
    throw err;
    ^

Error: Cannot find module 'tslib'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:582:15)
    at Function.Module._load (internal/modules/cjs/loader.js:508:25)
    at Module.require (internal/modules/cjs/loader.js:637:17)
    at require (internal/modules/cjs/helpers.js:22:18)
    at Object.<anonymous> (/usr/src/app/main.js:91:18)
    at __webpack_require__ (/usr/src/app/main.js:20:30)
    at Module.<anonymous> (/usr/src/app/main.js:231:63)
    at __webpack_require__ (/usr/src/app/main.js:20:30)
    at Object.<anonymous> (/usr/src/app/main.js:222:18)
    at __webpack_require__ (/usr/src/app/main.js:20:30)

“externalDependencies”: “none” seems like the option I need, but if fails the build. Anybody got an idea how to handle this?

So basically I have dig into this issue a little bit and I have some takeaways to share. Firstly, this issue is not caused by 6.0.0 release - it wasn’t working properly in 5.x either. However, in 5.x you were able to compile your prod app unless you used ValidationPipe or CacheModule - then, it apparently stopped to work. This has changed though, because we are no longer using expressions (variables) inside require() function, but we’re rather passing a loader function that has dependency defined statically.

Hence, this for instance:

const cacheManager = loadPackage('cache-manager', 'CacheModule', () =>
   require('cache-manager'),
);

leads to compilation errors, because webpack traverse modules graph and tries to resolve cache-manager dependency even though(!) CacheModule might not be used (tree-shaking is being performed later on, once bundle is created). Consequently, it breaks Nest idea of lazy require() calls, because webpack tries to load them all during the bundling process.

How to fix this? The easiest solution would be to use webpack-node-externals package to avoid bundling node_modules (bundle & tree-shake only user code). Usage is pretty simple:

externals: [nodeExternals()], 

And it should be fine in 99% cases. However, sometimes you may prefer to bundle entire application (one file, smaller size, somewhat lower memory consumption, slightly faster bootstrap time [serverless]) - in this case, you could use webpack.IgnorePlugin plugin.

 plugins: [
    ...options.plugins,
    new webpack.IgnorePlugin({
      checkResource(resource) {
        const lazyImports = ['@nestjs/microservices', '@nestjs/platform-express', 'cache-manager', 'class-validator', 'class-transformer'];
        if (!lazyImports.includes(resource)) {
          return false;
        }
        try {
          require.resolve(resource);
        } catch (err) {
          return true;
        }
        return false;
      },
    }),
  ],

This workaround would ignore whitelisted packages IF they aren’t present (require.resolve() throws an error). Otherwise, it would bundle them as well. This example would work, but the problem here is that you basically have to hardcode these whitelisted packages in the actual builder (Nx node builder) in this case. Perhaps, there is a way to just conditionally provide additional plugin to determine webpack behavior determine on the framework which is being used. CC @FrozenPandaz

I’m just a humble member of the Nest community. Though, I do think we need to sympathize with the complexity of integrating many 3rd party systems together (such as Nest does), and how there can be problems that may not have eloquent solutions. Bundling and tree shaking code has been something the industry has been struggling with since the inception of modern Javascript (ES6+). Webpack and Rollup seem to be the current front running technologies of the industry. Historically, there was no exiting Javascript framework that managed to solve the bundling/tree shaking problem, due to the fact there were various competing Javascript module systems (Commonjs vs AMD vs ES6). Furthermore, there are framework specific module systems, such as Google’s Angular module system, which is what Nest’s own module system is inspired after. The Angular team took on the challenge to design a module system that would allow for the construction of a deterministic abstract syntax tree (AST). It contains the information needed to appropriately tree shake out unused code and determine what should actually be bundled. The industry has come a long way in a short amount of time, but there are still long standing problems to be resolved.

This of course leads us to the issue of this thread. The problem lies deeper than just at the framework level (Nest), or at the bundling level (Webpack). Therefore, we should be sympathetic to the fact that we may need some explicit configuration for the time being. The industry as a whole needs more time to mature. Until then, all of us require some proficiency with configuring bundling systems like Webpack.

got the same issue… and working with Nx workspace to manage multiples nestjs application… Any way to bundle all deps in dist folder of each nestjs application? Or a way to build correct package.json files with only required deps of each nestjs application??

@FrozenPandaz

I feel like (not sure) dependencies/ tree-shaking isn’t what is expected. If the user does not use ‘@nestjs/microservices’, it shouldn’t be in the bundle at all. In Angular I think the major facilitator of this is NgModule imports. If you don’t import HttpClientModule, no Http code will be brought into the compilation. Can you think of where in the source code those imports are brought in and how it can be avoided? Of course, it may be a breaking change so it’s too late for v6 😦.

See my response here, especially this part:

Webpack traverses modules graph and tries to resolve cache-manager dependency even though(!) CacheModule might not be used (tree-shaking is being performed later on, once bundle is created). Consequently, it breaks Nest idea of lazy require() calls, because webpack tries to load them all during the bundling process.

Basically, Nest in some cases is using require() calls lazily. For instance, if you don’t use either RedisClient or RedisServer, the require('redis') expression won’t be evaluated = the package won’t have to be installed. This allows us to avoid creating 10 packages for every existing transport strategy just to put 2 classes in there. Also, if someone doesn’t use redis strategy, he won’t be forced to install redis.

Another, more important example is ValidationPipe which dynamically loads both class-validator and class-transformer. Validation is considered as a crucial feature and is being using among lots of applications. However, we cannot put these 2 libraries into internal dependencies because they also expose decorators to developers that have to be used from outside Nest internal scope. Hence, we have to keep them as peer dependencies. In this case, we would have to again extract ValidationPipe (1 class) to a separate package.

The solution that I have included in my post (with webpack.IgnorePlugin()) generally allow you to still bundle + perform tree-shaking for everything (including lazy require() calls). It would obviously consume error messages about missing Nest-lazy-loaded packages (if they ARE actually missing), but it shouldn’t be a big deal since this build is being used in prod mode only(correct?). Also, during the tree-shaking step, webpack would get rid of this code that is not being used anyway so it wouldn’t affect the final bundle either.

So reassuming, if we still want to have tree-shaking + bundling, the only thing is to put this webpack.IgnorePlugin() solution (described here) as a custom webpack configuration (available in Nx 7.7, correct?) only for Nest apps.

I would like to deploy a standalone bundle without any node_modules.

If I understand correctly, the intended way of doing so would be to add "externalDependencies": "none". But it leads me to the same errors that ZenSoftware described. (I am using nest v6.2.4 in a Nx project).

Can someone confirm my following understanding of the work around suggested above:

• exclude problematic dependencies https://github.com/nestjs/nest/issues/1706#issuecomment-478145526
• then provide a minimalist package.json to install requiered node_modules

Did I miss something or is there any plan to correct the "externalDependencies": "none" error?

I created a PR to revert the change for now. I still suggest people look into bundling in their dependencies but I guess it’s hard to provide something that works for NestJS out of the box.

@xmlking Yes, that is correct! I recommend the following setting instead:

"externalDependencies": [
     "@nestjs/microservices",
     "@nestjs/platform-express",
     "class-transformer",
     "class-validator",
     "cache-manager"
]

@kamilmysliwiec Maybe the webpack team can help suggest a better way to handle such module references.

I tried with externalDependencies but I’ve got this error:

class NestApplication extends nest_application_context_1.NestApplicationContext {
TypeError: Class extends value undefined is not a constructor or null
    at Module../node_modules/@nestjs/core/nest-application.js

"externalDependencies": [
     "@nestjs/microservices",
     "@nestjs/platform-express",
     "class-transformer",
     "class-validator",
     "cache-manager"
]

i’m trying to bundle my apps with webpack to reduce package size as i’m using aws lambda functions and i don’t manage to make it work using the suggested solution …
i’m adding the @nest/graphql module to the app and there are quite lot warning and errors when building the app :

WARNING in ./node_modules/@nestjs/core/helpers/load-adapter.js 8:39-63
Critical dependency: the request of a dependency is an expression
 @ ./node_modules/@nestjs/core/nest-factory.js
 @ ./node_modules/@nestjs/core/index.js
 @ ./src/sls.ts

WARNING in ./node_modules/@nestjs/common/utils/load-package.util.js 8:39-59
Critical dependency: the request of a dependency is an expression
 @ ./node_modules/@nestjs/core/nest-application.js
 @ ./node_modules/@nestjs/core/index.js
 @ ./src/sls.ts

WARNING in ./node_modules/optional/optional.js 6:11-26
Critical dependency: the request of a dependency is an expression
 @ ./node_modules/@nestjs/core/nest-application.js
 @ ./node_modules/@nestjs/core/index.js
 @ ./src/sls.ts

WARNING in ./node_modules/express/lib/view.js 81:13-25
Critical dependency: the request of a dependency is an expression
 @ ./node_modules/express/lib/application.js
 @ ./node_modules/express/lib/express.js
 @ ./node_modules/express/index.js
 @ ./src/sls.ts

WARNING in ./node_modules/type-graphql/dist/helpers/loadResolversFromGlob.js 10:46-63
Critical dependency: the request of a dependency is an expression
 @ ./node_modules/type-graphql/dist/utils/buildSchema.js
 @ ./node_modules/type-graphql/dist/utils/index.js
 @ ./node_modules/type-graphql/dist/index.js
 @ ./node_modules/@nestjs/graphql/dist/graphql-schema-builder.js
 @ ./node_modules/@nestjs/graphql/dist/graphql.factory.js
 @ ./node_modules/@nestjs/graphql/dist/index.js
 @ ./node_modules/@nestjs/graphql/index.js
 @ ./src/app.module.ts
 @ ./src/sls.ts

WARNING in ./node_modules/merge-graphql-schemas/dist/index.esm.js 3782:21-31
Critical dependency: the request of a dependency is an expression
 @ ./node_modules/@nestjs/graphql/dist/graphql-types.loader.js
 @ ./node_modules/@nestjs/graphql/dist/index.js
 @ ./node_modules/@nestjs/graphql/index.js
 @ ./src/app.module.ts
 @ ./src/sls.ts

WARNING in ./node_modules/ts-morph/node_modules/typescript/lib/typescript.js 94814:19-45
Critical dependency: the request of a dependency is an expression
 @ ./node_modules/ts-morph/dist/typescript/public.js
 @ ./node_modules/ts-morph/dist/main.js
 @ ./node_modules/@nestjs/graphql/dist/graphql-ast.explorer.js
 @ ./node_modules/@nestjs/graphql/dist/index.js
 @ ./node_modules/@nestjs/graphql/index.js
 @ ./src/app.module.ts
 @ ./src/sls.ts

WARNING in ./node_modules/subscriptions-transport-ws/node_modules/ws/lib/buffer-util.js
Module not found: Error: Can't resolve 'bufferutil' in '/Users/odifisIH/Sites/cashmetrics/packages/core/node_modules/subscriptions-transport-ws/node_modules/ws/lib'
 @ ./node_modules/subscriptions-transport-ws/node_modules/ws/lib/buffer-util.js
 @ ./node_modules/subscriptions-transport-ws/node_modules/ws/lib/receiver.js
 @ ./node_modules/subscriptions-transport-ws/node_modules/ws/index.js
 @ ./node_modules/subscriptions-transport-ws/dist/server.js
 @ ./node_modules/subscriptions-transport-ws/dist/index.js
 @ ./node_modules/apollo-server-core/dist/ApolloServer.js
 @ ./node_modules/apollo-server-core/dist/index.js
 @ ./node_modules/apollo-server-express/dist/index.js
 @ ./node_modules/@nestjs/graphql/dist/graphql.factory.js
 @ ./node_modules/@nestjs/graphql/dist/index.js
 @ ./node_modules/@nestjs/graphql/index.js
 @ ./src/app.module.ts
 @ ./src/sls.ts

WARNING in ./node_modules/subscriptions-transport-ws/node_modules/ws/lib/validation.js
Module not found: Error: Can't resolve 'utf-8-validate' in '/Users/odifisIH/Sites/cashmetrics/packages/core/node_modules/subscriptions-transport-ws/node_modules/ws/lib'
 @ ./node_modules/subscriptions-transport-ws/node_modules/ws/lib/validation.js
 @ ./node_modules/subscriptions-transport-ws/node_modules/ws/lib/receiver.js
 @ ./node_modules/subscriptions-transport-ws/node_modules/ws/index.js
 @ ./node_modules/subscriptions-transport-ws/dist/server.js
 @ ./node_modules/subscriptions-transport-ws/dist/index.js
 @ ./node_modules/apollo-server-core/dist/ApolloServer.js
 @ ./node_modules/apollo-server-core/dist/index.js
 @ ./node_modules/apollo-server-express/dist/index.js
 @ ./node_modules/@nestjs/graphql/dist/graphql.factory.js
 @ ./node_modules/@nestjs/graphql/dist/index.js
 @ ./node_modules/@nestjs/graphql/index.js
 @ ./src/app.module.ts
 @ ./src/sls.ts

ERROR in ./node_modules/chokidar/node_modules/fsevents/fsevents.node 1:0
Module parse failed: Unexpected character '�' (1:0)
You may need an appropriate loader to handle this file type.
(Source code omitted for this binary file)
 @ ./node_modules/chokidar/node_modules/fsevents/fsevents.js 13:15-41
 @ ./node_modules/chokidar/lib/fsevents-handler.js
 @ ./node_modules/chokidar/index.js
 @ ./node_modules/@nestjs/graphql/dist/graphql-definitions.factory.js
 @ ./node_modules/@nestjs/graphql/dist/index.js
 @ ./node_modules/@nestjs/graphql/index.js
 @ ./src/app.module.ts
 @ ./src/sls.ts

this is my webpack config :

const webpack = require('webpack');
const path = require('path');
const nodeExternals = require('webpack-node-externals');



module.exports = {
  mode: 'production',
  entry: './src/sls.ts',
  target: 'node',
  module: {
    rules: [
      {
        test: /\.tsx?$/,
        use: 'ts-loader',
        exclude: /node_modules/
      }
    ]
  },
  resolve: {
    extensions: [ '.mjs','.tsx', '.ts', '.js' ]
  },
  // externals: [nodeExternals()],
  plugins: [
    new webpack.IgnorePlugin({
      checkResource(resource) {
        const lazyImports = ['@nestjs/microservices', '@nestjs/platform-express','@nestjs/grahpql', 'cache-manager', 'class-validator', 'class-transformer', 'graphql'];
        if (!lazyImports.includes(resource)) {
          return false;
        }
        try {
          require.resolve(resource);
        } catch (err) {
          return true;
        }
        return false;
      },
    }),
  ],
  output: {
    libraryTarget: 'commonjs',
    filename: 'build.js',
    path: path.resolve(__dirname, 'dist')
  }
};

im just using the graphql example from nestJS and is worth mentioning that without adding ‘.mjs’ to the extensions list webpack will also complain: https://github.com/graphql/graphql-js/issues/1272

thanks !

testing with nx webpackConfig option, https://github.com/xmlking/ngx-starter-kit/blob/develop/angular.json#L255

var IgnorePlugin = require('webpack').IgnorePlugin;

module.exports = (config, context) => {
  // extend, mutate, create a new config (whatever you want)

  config.plugins = [
    ...config.plugins,
    new IgnorePlugin({
      checkResource(resource) {
        const lazyImports = [
          '@nestjs/microservices',
          '@nestjs/platform-express',
          'cache-manager',
          'class-validator',
          'class-transformer',
        ];
        if (!lazyImports.includes(resource)) {
          return false;
        }
        try {
          require.resolve(resource);
        } catch (err) {
          return true;
        }
        return false;
      },
    }),
  ];

  return config;
};

@FrozenPandaz is this correct way to customize webpack ?

Edit: The following is a messy solution. Simply setting "externalDependencies": "all" in the angular.json file does the same thing. Nice find @xmlking! ^_^

---

Thanks @kamilmysliwiec for all the hard work. The community loves you for it ^_^

My build process is a bit more convoluted than I was hoping for, but I ended up using @kamilmysliwiec recommendation to use webpack-node-externals.

• Install @angular-builders/custom-webpack via npm i -D @angular-builders/custom-webpack
• Install webpack-node-externals via npm i -D webpack-node-externals
• Creat a webpack.partial.js at the root of your project with the following:

var nodeExternals = require('webpack-node-externals');

module.exports = {
  externals: [nodeExternals()]
};

• Create a build configuration in angular.json file. (Note that this is presuming your Nest app in Nx is named “api”)

...

"architect": {
  "custom": {
    "builder": "@angular-builders/custom-webpack:server",
    "options": {
      "customWebpackConfig": {
        "path": "./webpack.partial.js"
      },
      "outputPath": "dist/apps/api",
      "main": "apps/api/src/main.ts",
      "tsConfig": "apps/api/tsconfig.app.json"
    },
    "configurations": {
      "production": {
        "optimization": true,
        "fileReplacements": [
          {
            "replace": "apps/api/src/environments/environment.ts",
            "with": "apps/api/src/environments/environment.prod.ts"
          }
        ]
      }
    }
  }
}

...

• Create a small npm script: ng run api:custom --configuration=production to build the Nest app without node_modules
• Change your Dockerfile to install npm dependencies (if you are using Docker that is)

COPY package*.json ./
RUN npm ci --production

I have just published a repo which demonstrates how one can bundle Nest and containerize it for Docker. It attempts to address the issues that most people are having here. You can find it here, cheers!

💥 Nest 🔰 Webpack 🔰 Docker 💥

looks like nx already support excluding some or all node modules from bundling externalDependencies: 'all' | 'none' | string[]; here

for my case "externalDependencies": "all" worked.

@FrozenPandaz can I somehow help with this issue? It seems that Nx builder is trying to bundle all dependencies, even lazy loaded ones. For instance, this require (cache.providers.ts file in common):

const cacheManager = loadPackage('cache-manager', 'CacheModule', () =>
   require('cache-manager'),
);

shouldn’t be bundled if CacheModule is not being used.