pnpm: Unexpected transitive dep update

Update 2022-07-06: I’ve rewritten the description of this issue with a proper repro since the original was confusing.

Here’s one minimal repro:

Repro initial state (commit)

pnpm-workspace.yaml

packages:
  - 'packages/app-should-not-change'
  - 'packages/app-with-changes'

packages/app-should-not-change/package.json

{
  "name": "app-should-not-change",
  "dependencies": { "@pg1/copper-cable": "1.0.0"  }
}

packages/app-with-changes/package.json

{
  "name": "app-with-changes",
  "dependencies": {} // <-- this will change in the next step
}

pnpm-lock.yaml

lockfileVersion: 5.4

importers:

  packages/app-should-not-change:
    specifiers:
      '@pg1/copper-cable': 1.0.0
    dependencies:
      '@pg1/copper-cable': 1.0.0

  packages/app-with-changes:
    specifiers: {}

packages:

  /@pg1/copper-cable/1.0.0:
    resolution: {integrity: sha512-hxmgSZAYpqLvwipPIWVyUMTetMOnSiVIH8WPMFX42M8Ucu3nKWYYYW4olorF8XwO3CPdeUoQ2ZEWfgeza3z9Fw==}
    dependencies:
      '@pg1/copper-plate': 1.0.0
    dev: false

  /@pg1/copper-ore/1.0.0:
    resolution: {integrity: sha512-nD1Ni4OLv51dfUEyxGg2feRdNrTShtVtj+LMBMW5UOqcn94kVVED2T9+Gr66LjNEfcREEpvhgpP0Cu9OQE8GYw==}
    dev: false

  /@pg1/copper-plate/1.0.0:
    resolution: {integrity: sha512-kcTsuGBhSeZRXuqxAw5DahCO4p/ykPFq46rtJEAATy/gH1jqWqlPd6NwHXQgPum3HAV7vOW0p4hWMC9YNDt27g==}
    dependencies:
      '@pg1/copper-ore': 1.0.0
    dev: false

Problematic change (commit)

packages/app-with-changes/package.json

 {
   "name": "app-with-changes",
   "dependencies": {
+    "@pg1/copper-ore": "1.1.0",
+    "@pg1/copper-cable": "1.0.0"
   }
 }

pnpm-lock.yaml

Then we perform a pnpm --filter app-with-change install and this is the lockfile diff:

@@ -9,7 +9,12 @@ importers:
       '@pg1/copper-cable': 1.0.0
 
   packages/app-with-changes:
-    specifiers: {}
+    specifiers:
+      '@pg1/copper-cable': 1.0.0
+      '@pg1/copper-ore': 1.1.0
+    dependencies:
+      '@pg1/copper-cable': 1.0.0
+      '@pg1/copper-ore': 1.1.0
 
 packages:
 
@@ -19,12 +24,12 @@ packages:
       '@pg1/copper-plate': 1.0.0
     dev: false
 
-  /@pg1/copper-ore/1.0.0:
-    resolution: {integrity: sha512-nD1Ni4OLv51dfUEyxGg2feRdNrTShtVtj+LMBMW5UOqcn94kVVED2T9+Gr66LjNEfcREEpvhgpP0Cu9OQE8GYw==}
+  /@pg1/copper-ore/1.1.0:
+    resolution: {integrity: sha512-m58Vuv2360b6CEz/LkNHT2BmN/b2oElHVUDaa7WPYp+NraZ624V3MNPszaxjm4VQPUs662TDvAu4UMS0tz7HGw==}
     dev: false
 
   /@pg1/copper-plate/1.0.0:
     resolution: {integrity: sha512-kcTsuGBhSeZRXuqxAw5DahCO4p/ykPFq46rtJEAATy/gH1jqWqlPd6NwHXQgPum3HAV7vOW0p4hWMC9YNDt27g==}
     dependencies:
-      '@pg1/copper-ore': 1.0.0
+      '@pg1/copper-ore': 1.1.0
     dev: false

You can see that @pg1/copper-ore of 1.0.0 was removed from the lockfile. Ideally we want to keep two versions and ensure that app-should-not-change doesn’t get modified.

Hacky workaround demonstrating this is possible

One way to get pnpm to behave as expected is to add this to the root package.json:

+  "pnpm": {
+    "overrides": {
+      "@pg1/copper-plate@1.0.0>@pg1/copper-ore": "1.0.0"
+    }
+  }

This is the resulting lockfile diff after an install:

@@ -1,7 +1,13 @@
 lockfileVersion: 5.4
 
+overrides:
+  '@pg1/copper-plate@1.0.0>@pg1/copper-ore': 1.0.0
+
 importers:
 
+  .:
+    specifiers: {}
+
   packages/app-should-not-change:
     specifiers:
       '@pg1/copper-cable': 1.0.0
@@ -9,7 +15,12 @@ importers:
       '@pg1/copper-cable': 1.0.0
 
   packages/app-with-changes:
-    specifiers: {}
+    specifiers:
+      '@pg1/copper-cable': 1.0.0
+      '@pg1/copper-ore': 1.1.0
+    dependencies:
+      '@pg1/copper-cable': 1.0.0
+      '@pg1/copper-ore': 1.1.0
 
 packages:
 
@@ -23,6 +34,10 @@ packages:
     resolution: {integrity: sha512-nD1Ni4OLv51dfUEyxGg2feRdNrTShtVtj+LMBMW5UOqcn94kVVED2T9+Gr66LjNEfcREEpvhgpP0Cu9OQE8GYw==}
     dev: false
 
+  /@pg1/copper-ore/1.1.0:
+    resolution: {integrity: sha512-m58Vuv2360b6CEz/LkNHT2BmN/b2oElHVUDaa7WPYp+NraZ624V3MNPszaxjm4VQPUs662TDvAu4UMS0tz7HGw==}
+    dev: false
+
   /@pg1/copper-plate/1.0.0:
     resolution: {integrity: sha512-kcTsuGBhSeZRXuqxAw5DahCO4p/ykPFq46rtJEAATy/gH1jqWqlPd6NwHXQgPum3HAV7vOW0p4hWMC9YNDt27g==}
     dependencies:

Pnpm happily installs two copies of @pg1/copper-ore:

❯ jq .version < packages/app-with-changes/node_modules/@pg1/copper-ore/package.json
"1.1.0"
❯ jq .version < node_modules/.pnpm/@pg1+copper-plate@1.0.0/node_modules/@pg1/copper-ore/package.json
"1.0.0"

Context

At Stripe we want to ensure (for CI caching and application stability reasons) that transitive deps don’t unintentionally change underneath workspace apps. In practice we’re seeing a number of cases where pnpm consolidates transitive deps causing the dependency graphs for many projects to update at once. This is a similar-in-nature problem to the “docker monorepo” issues discussed in #3114.

I know there’s been a lot of work in the past on similar issues – there used to be a resolution-strategy config. This kind of feels like a problem that wouldn’t have existed with the old fast strategy which seemed to favor existing lockfile versions.

Ideally dependency consolidation for us should be an explicit operation typically performed by an infrastructure team since it potentially affects many projects. We don’t want our individual app owning teams to worry about their PRs unintentionally shifting dependencies for other workspace projects.

I fully realize this is a complicated area. Thank you for your time looking at this!

SEO: pocket-restrain

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Reactions: 1
  • Comments: 22 (17 by maintainers)

Most upvoted comments

Not that I’m aware of. We’ve continued to operate with split lockfiles and have had to build significant tooling around pnpm to achieve our goals. For example we have some fairly “interesting” postinstall scripts that muck around with the symlinks to help make sure we use only 1 copy of React, etc, across projects.

The more I’ve thought about this problem the more I’m convinced that a conceptual single lockfile is, with a single virtual store, the best solution. However, pnpm isn’t architected to make safe dependency updates without unintentionally affecting multiple projects.

In a large monorepo there are really at least two categories of users: 1) day to day developers who don’t want to accidentally update-the-world, and 2) monorepo operators who are concerned with normalizing the dep graph across all projects and doing broad swath changes. Ideally the tooling would cater to both types of users but AFAIK there’s no tooling that behaves like that.