build: Automatic installation of plugins makes build much slower

When a plugin is used in netlify.toml but is not inside package.json, we automatically install it during the build. This creates two performance issues:

  1. When user first install a plugin, they just look at their total build time and see that it increased. They then remove that plugin due to that increase, without trying another build. For caching plugin for netlify-plugin-gatsby-cache, they think caching is not working properly since it increases build time.
  2. Since we cache node_modules and we only install plugins when calling require() fails, the next build does not have that additional install time. However each time the user changes their dependencies, devDependencies or lock file, the node_modules get re-generated, and the additional automatic install time gets added to the build.

Another issue suggested by @fool on Slack was that npm gets outages from time to time, and that would make builds fail.

Note that this does not mean we should not automatically install plugins, but more likely that we should have a better caching strategy that’s both fast and resilient to npm outages.

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 21 (17 by maintainers)

Most upvoted comments

Another issue with automatic installations related to https://github.com/netlify/build/issues/1137#issuecomment-620060028 is that always installing the latest version prevents reproducible builds.

Related discussion of trying to cache NPM installs internal to our build infra: https://github.com/netlify/infra/issues/55

One small thing to consider if going the route of adding to an existing package.json to install plugins: not all repos will have a package.json, so the process would have to account for creating one (or just running the npm install xyz command) if one doesn’t already exist.