nx: Pruned lock file creation failed. The following package was not found in the root lock file when using local packages

Current Behavior

nx/src/lock-file/project-graph-pruning.js throws an error if a service requires a package from our monorepo which can not be in the root’s package-lock.json. Since we install the required package from a registry, it will never be in the package-lock.

Expected Behavior

Can we use the version written in the required package’s package.json?

GitHub Repo

difficult setup

Steps to Reproduce

difficult setup

Nx Report

Node : 19.7.0
   OS   : darwin arm64
   npm  : 9.5.0
   
   nx                      : 15.7.1
   @nrwl/jest              : 15.7.1
   @nrwl/linter            : 15.7.1
   @nrwl/workspace         : 15.7.1
   @nrwl/cli               : 15.7.1
   @nrwl/devkit            : 15.9.4
   @nrwl/esbuild           : 15.7.1
   @nrwl/eslint-plugin-nx  : 15.7.1
   @nrwl/js                : 15.7.1
   @nrwl/node              : 15.7.1
   @nrwl/tao               : 15.7.1
   @nrwl/vite              : 15.7.1
   @nrwl/webpack           : 15.7.1
   typescript              : 4.8.4
   ---------------------------------------
   Community plugins:
   @jscutlery/semver : 2.30.1

Failure Logs

Done compiling TypeScript files for project “engine-service”. /Users/reventwork/Code/revent/backend/node_modules/nx/src/lock-file/project-graph-pruning.js:40 throw new Error(Pruned lock file creation failed. The following package was not found in the root lock file: ${packageName}@${versionRange}); ^

Error: Pruned lock file creation failed. The following package was not found in the root lock file: @revent/configuration@1.0.7 at /Users/reventwork/Code/revent/backend/node_modules/nx/src/lock-file/project-graph-pruning.js:40:19 at Array.forEach (<anonymous>) at normalizeDependencies (/Users/reventwork/Code/revent/backend/node_modules/nx/src/lock-file/project-graph-pruning.js:26:42) at pruneProjectGraph (/Users/reventwork/Code/revent/backend/node_modules/nx/src/lock-file/project-graph-pruning.js:13:34) at createLockFile (/Users/reventwork/Code/revent/backend/node_modules/nx/src/lock-file/lock-file.js:122:75) at updatePackageJson (/Users/reventwork/Code/revent/backend/node_modules/@nrwl/js/src/utils/package-json/update-package-json.js:41:57) at /Users/reventwork/Code/revent/backend/node_modules/@nrwl/js/src/executors/tsc/tsc.impl.js:86:57 at Generator.next (<anonymous>) at fulfilled (/Users/reventwork/Code/revent/backend/node_modules/tslib/tslib.js:164:62) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Package Manager Version

No response

Operating System

  • macOS
  • Linux
  • Windows
  • Other (Please specify)

Additional Information

No response

About this issue

  • Original URL
  • State: open
  • Created 10 months ago
  • Reactions: 1
  • Comments: 16 (3 by maintainers)

Most upvoted comments

This is still happening in 17.0.2

@AgentEnder these seem to originate from the last refactoring. Can you please look into it?

I may be getting something similar

The error I see is:

 >  NX   An error occured while creating pruned lockfile

   Please open an issue at `https://github.com/nrwl/nx/issues/new?template=1-bug.yml` and provide a reproduction.
   To prevent the build from breaking we are returning the root lock file.
   If you run `npm install --package-lock-only` in your output folder it will regenerate the correct pruned lockfile.
   
Original error: Target project does not exist: npm:whatwg-url@5.0.0


   Error: Target project does not exist: npm:whatwg-url@5.0.0
    at validateCommonDependencyRules (/Users/gbanis/dev/safetymanager/node_modules/nx/src/project-graph/project-graph-builder.js:288:15)
    at validateDependency (/Users/gbanis/dev/safetymanager/node_modules/nx/src/project-graph/project-graph-builder.js:278:5)
    at ProjectGraphBuilder.addDependency (/Users/gbanis/dev/safetymanager/node_modules/nx/src/project-graph/project-graph-builder.js:165:9)
    at ProjectGraphBuilder.addStaticDependency (/Users/gbanis/dev/safetymanager/node_modules/nx/src/project-graph/project-graph-builder.js:82:14)
    at /Users/gbanis/dev/safetymanager/node_modules/nx/src/plugins/js/lock-file/project-graph-pruning.js:81:17
    at Array.forEach (<anonymous>)
    at traverseNode (/Users/gbanis/dev/safetymanager/node_modules/nx/src/plugins/js/lock-file/project-graph-pruning.js:78:36)
    at /Users/gbanis/dev/safetymanager/node_modules/nx/src/plugins/js/lock-file/project-graph-pruning.js:80:9
    at Array.forEach (<anonymous>)
    at traverseNode (/Users/gbanis/dev/safetymanager/node_modules/nx/src/plugins/js/lock-file/project-graph-pruning.js:78:36)

NX Report:

   Node   : 19.1.0
   OS     : darwin-x64
   npm    : 8.19.3
   
   nx (global)             : 16.8.1
   nx                      : 16.8.1
   @nx/js                  : 16.8.1
   @nx/jest                : 16.8.1
   @nx/linter              : 16.8.1
   @nx/workspace           : 16.8.1
   @nx/cypress             : 16.8.1
   @nx/devkit              : 16.8.1
   @nrwl/eslint-plugin-nx  : 16.8.1
   @nx/nest                : 16.8.1
   @nx/node                : 16.8.1
   @nx/react               : 16.8.1
   @nrwl/tao               : 16.8.1
   @nx/web                 : 16.8.1
   @nx/webpack             : 16.8.1
   typescript              : 5.1.6

whatwg-url@5.0.0 is found in the following deps in package-lock.json:

    "node_modules/node-fetch": {
      "version": "2.6.12",
      "resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-2.6.12.tgz",
      "integrity": "sha512-C/fGU2E8ToujUivIO0H+tpQ6HWo4eEmchoPIoXtxCrVghxdKq+QOHqEZW7tuP3KlV3bC8FRMO5nMCC7Zm1VP6g==",
      "dependencies": {
        "whatwg-url": "^5.0.0"
      },
      "engines": {
        "node": "4.x || >=6.0.0"
      },
      "peerDependencies": {
        "encoding": "^0.1.0"
      },
      "peerDependenciesMeta": {
        "encoding": {
          "optional": true
        }
      }
    },

    "node_modules/node-fetch": {
      "version": "2.6.12",
      "resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-2.6.12.tgz",
      "integrity": "sha512-C/fGU2E8ToujUivIO0H+tpQ6HWo4eEmchoPIoXtxCrVghxdKq+QOHqEZW7tuP3KlV3bC8FRMO5nMCC7Zm1VP6g==",
      "dependencies": {
        "whatwg-url": "^5.0.0"
      },
      "engines": {
        "node": "4.x || >=6.0.0"
      },
      "peerDependencies": {
        "encoding": "^0.1.0"
      },
      "peerDependenciesMeta": {
        "encoding": {
          "optional": true
        }
      }
    },

OS: macOS

Try this setting project.json:

excludeLibsInPackageJson: true

@meeroslav Thank you for your answer!

I did as you suggested and deleted the lock file and ./node_moduls but that doesn’t resolve the issue.

I also tried installing whatwg-url@5.0.0 and then a different dep fails.

Original error: Target project does not exist: npm:webidl-conversions@3.0.1

In fact, I tried doing the same multiple times but more deps have issues being pruned.

Hi all,

Based on your reported issues this is not something we can solve. The purpose of lock file pruning is to prune the existing lock file and isolate just the packages your project needs. We can’t come up with snapshots for packages that do not exist in the root lock file as that requires a round trip to the registry which beats the purpose of lock file pruning vs. built-in package manager install commands.

@paul-reventwork is there a reason your lock file does not contain @revent/configuration? How do you run tests against something that is missing its dependencies? It seems that you are doing something very custom in which case I would suggest skipping built-in lock file pruning and using install directly to generate a lock file based on your needs.

@gbanis your lock file mentions whatwg-url but does not contain it - that package is never installed. If it was your lock file would have a key like node_modules\whatwg-url. Since that dependency is not marked as optional, I would suspect that your lock file is broken. Try removing the lock file and node_modules and reinstalling them again.

@KeithGillette your packages are out of sync as nx report suggests. This could easily be the reason why things don’t work. Ensure all your packages are synchronized by running nx migrate 16.8.1 and check if the error is still there.

@trekinbami same as @gbanis, check if the package is actually installed in your node_modules.

If you checked all the steps above:

  • ensuring that all nx packages have the same version
  • cleared lock file and node_modules and reinstalled the dependencies
  • checked that the package is actually specified in your lock file and installed in node_modules and the error still persists, please paste all the snapshot parts of your lock file that mentions this package and package.json if defined there and we can use this to debug it.

Also getting a similar error on rebuilds following file changes after some recent dependency updates:

 >  NX   An error occured while creating pruned lockfile

   Please open an issue at `https://github.com/nrwl/nx/issues/new?template=1-bug.yml` and provide a reproduction.
   To prevent the build from breaking we are returning the root lock file.
   If you run `npm install --package-lock-only` in your output folder it will regenerate the correct pruned lockfile.
   
Original error: Target project does not exist: npm:bl@2.2.1


   Error: Target project does not exist: npm:bl@2.2.1
    at validateCommonDependencyRules (/Path/To/node_modules/nx/src/project-graph/project-graph-builder.js:288:15)
    at validateDependency (/Path/To/node_modules/nx/src/project-graph/project-graph-builder.js:278:5)
    at ProjectGraphBuilder.addDependency (/Path/To/node_modules/nx/src/project-graph/project-graph-builder.js:165:9)
    at ProjectGraphBuilder.addStaticDependency (/Path/To/node_modules/nx/src/project-graph/project-graph-builder.js:82:14)
    at /Path/To/node_modules/nx/src/plugins/js/lock-file/project-graph-pruning.js:81:17
    at Array.forEach (<anonymous>)
    at traverseNode (/Path/To/node_modules/nx/src/plugins/js/lock-file/project-graph-pruning.js:78:36)
    at /Path/To/node_modules/nx/src/plugins/js/lock-file/project-graph-pruning.js:80:9
    at Array.forEach (<anonymous>)
    at traverseNode (/Path/To/node_modules/nx/src/plugins/js/lock-file/project-graph-pruning.js:78:36)

bl-2-2-1 appears 4 times in our monorepository’s package-lock.json.

nx report

   Node   : 18.16.1
   OS     : darwin-arm64
   npm    : 9.5.1
   
   nx             : 16.8.1
   @nx/js         : 16.8.1
   @nx/jest       : 16.8.1
   @nx/linter     : 16.8.1
   @nx/workspace  : 16.8.1
   @nx/angular    : 16.8.1
   @nx/cypress    : 16.8.1
   @nx/devkit     : 16.5.1
   @nx/node       : 16.8.1
   @nrwl/tao      : 16.8.1
   @nx/webpack    : 16.8.1
   nx-cloud       : 16.4.0
   typescript     : 5.1.6
   ---------------------------------------
   Community plugins:
   apollo-angular : 5.0.1
   ---------------------------------------
   The following packages should match the installed version of nx
     - @nx/devkit@16.5.1
     - @nrwl/devkit@16.5.1
   
   To fix this, run `nx migrate nx@16.8.1`