pnpm: Unexpected transitive dep update
Update 2022-07-06: I’ve rewritten the description of this issue with a proper repro since the original was confusing.
Here’s one minimal repro:
Repro initial state (commit)
pnpm-workspace.yaml
packages:
- 'packages/app-should-not-change'
- 'packages/app-with-changes'
packages/app-should-not-change/package.json
{
"name": "app-should-not-change",
"dependencies": { "@pg1/copper-cable": "1.0.0" }
}
packages/app-with-changes/package.json
{
"name": "app-with-changes",
"dependencies": {} // <-- this will change in the next step
}
pnpm-lock.yaml
lockfileVersion: 5.4
importers:
packages/app-should-not-change:
specifiers:
'@pg1/copper-cable': 1.0.0
dependencies:
'@pg1/copper-cable': 1.0.0
packages/app-with-changes:
specifiers: {}
packages:
/@pg1/copper-cable/1.0.0:
resolution: {integrity: sha512-hxmgSZAYpqLvwipPIWVyUMTetMOnSiVIH8WPMFX42M8Ucu3nKWYYYW4olorF8XwO3CPdeUoQ2ZEWfgeza3z9Fw==}
dependencies:
'@pg1/copper-plate': 1.0.0
dev: false
/@pg1/copper-ore/1.0.0:
resolution: {integrity: sha512-nD1Ni4OLv51dfUEyxGg2feRdNrTShtVtj+LMBMW5UOqcn94kVVED2T9+Gr66LjNEfcREEpvhgpP0Cu9OQE8GYw==}
dev: false
/@pg1/copper-plate/1.0.0:
resolution: {integrity: sha512-kcTsuGBhSeZRXuqxAw5DahCO4p/ykPFq46rtJEAATy/gH1jqWqlPd6NwHXQgPum3HAV7vOW0p4hWMC9YNDt27g==}
dependencies:
'@pg1/copper-ore': 1.0.0
dev: false
Problematic change (commit)
packages/app-with-changes/package.json
{
"name": "app-with-changes",
"dependencies": {
+ "@pg1/copper-ore": "1.1.0",
+ "@pg1/copper-cable": "1.0.0"
}
}
pnpm-lock.yaml
Then we perform a pnpm --filter app-with-change install
and this is the lockfile diff:
@@ -9,7 +9,12 @@ importers:
'@pg1/copper-cable': 1.0.0
packages/app-with-changes:
- specifiers: {}
+ specifiers:
+ '@pg1/copper-cable': 1.0.0
+ '@pg1/copper-ore': 1.1.0
+ dependencies:
+ '@pg1/copper-cable': 1.0.0
+ '@pg1/copper-ore': 1.1.0
packages:
@@ -19,12 +24,12 @@ packages:
'@pg1/copper-plate': 1.0.0
dev: false
- /@pg1/copper-ore/1.0.0:
- resolution: {integrity: sha512-nD1Ni4OLv51dfUEyxGg2feRdNrTShtVtj+LMBMW5UOqcn94kVVED2T9+Gr66LjNEfcREEpvhgpP0Cu9OQE8GYw==}
+ /@pg1/copper-ore/1.1.0:
+ resolution: {integrity: sha512-m58Vuv2360b6CEz/LkNHT2BmN/b2oElHVUDaa7WPYp+NraZ624V3MNPszaxjm4VQPUs662TDvAu4UMS0tz7HGw==}
dev: false
/@pg1/copper-plate/1.0.0:
resolution: {integrity: sha512-kcTsuGBhSeZRXuqxAw5DahCO4p/ykPFq46rtJEAATy/gH1jqWqlPd6NwHXQgPum3HAV7vOW0p4hWMC9YNDt27g==}
dependencies:
- '@pg1/copper-ore': 1.0.0
+ '@pg1/copper-ore': 1.1.0
dev: false
You can see that @pg1/copper-ore
of 1.0.0
was removed from the lockfile. Ideally we want to keep two versions and ensure that app-should-not-change
doesn’t get modified.
Hacky workaround demonstrating this is possible
One way to get pnpm to behave as expected is to add this to the root package.json
:
+ "pnpm": {
+ "overrides": {
+ "@pg1/copper-plate@1.0.0>@pg1/copper-ore": "1.0.0"
+ }
+ }
This is the resulting lockfile diff after an install:
@@ -1,7 +1,13 @@
lockfileVersion: 5.4
+overrides:
+ '@pg1/copper-plate@1.0.0>@pg1/copper-ore': 1.0.0
+
importers:
+ .:
+ specifiers: {}
+
packages/app-should-not-change:
specifiers:
'@pg1/copper-cable': 1.0.0
@@ -9,7 +15,12 @@ importers:
'@pg1/copper-cable': 1.0.0
packages/app-with-changes:
- specifiers: {}
+ specifiers:
+ '@pg1/copper-cable': 1.0.0
+ '@pg1/copper-ore': 1.1.0
+ dependencies:
+ '@pg1/copper-cable': 1.0.0
+ '@pg1/copper-ore': 1.1.0
packages:
@@ -23,6 +34,10 @@ packages:
resolution: {integrity: sha512-nD1Ni4OLv51dfUEyxGg2feRdNrTShtVtj+LMBMW5UOqcn94kVVED2T9+Gr66LjNEfcREEpvhgpP0Cu9OQE8GYw==}
dev: false
+ /@pg1/copper-ore/1.1.0:
+ resolution: {integrity: sha512-m58Vuv2360b6CEz/LkNHT2BmN/b2oElHVUDaa7WPYp+NraZ624V3MNPszaxjm4VQPUs662TDvAu4UMS0tz7HGw==}
+ dev: false
+
/@pg1/copper-plate/1.0.0:
resolution: {integrity: sha512-kcTsuGBhSeZRXuqxAw5DahCO4p/ykPFq46rtJEAATy/gH1jqWqlPd6NwHXQgPum3HAV7vOW0p4hWMC9YNDt27g==}
dependencies:
Pnpm happily installs two copies of @pg1/copper-ore
:
❯ jq .version < packages/app-with-changes/node_modules/@pg1/copper-ore/package.json
"1.1.0"
❯ jq .version < node_modules/.pnpm/@pg1+copper-plate@1.0.0/node_modules/@pg1/copper-ore/package.json
"1.0.0"
Context
At Stripe we want to ensure (for CI caching and application stability reasons) that transitive deps don’t unintentionally change underneath workspace apps. In practice we’re seeing a number of cases where pnpm consolidates transitive deps causing the dependency graphs for many projects to update at once. This is a similar-in-nature problem to the “docker monorepo” issues discussed in #3114.
I know there’s been a lot of work in the past on similar issues – there used to be a resolution-strategy
config. This kind of feels like a problem that wouldn’t have existed with the old fast
strategy which seemed to favor existing lockfile versions.
Ideally dependency consolidation for us should be an explicit operation typically performed by an infrastructure team since it potentially affects many projects. We don’t want our individual app owning teams to worry about their PRs unintentionally shifting dependencies for other workspace projects.
I fully realize this is a complicated area. Thank you for your time looking at this!
SEO: pocket-restrain
About this issue
- Original URL
- State: open
- Created 2 years ago
- Reactions: 1
- Comments: 22 (17 by maintainers)
Not that I’m aware of. We’ve continued to operate with split lockfiles and have had to build significant tooling around pnpm to achieve our goals. For example we have some fairly “interesting”
postinstall
scripts that muck around with the symlinks to help make sure we use only 1 copy of React, etc, across projects.The more I’ve thought about this problem the more I’m convinced that a conceptual single lockfile is, with a single virtual store, the best solution. However, pnpm isn’t architected to make safe dependency updates without unintentionally affecting multiple projects.
In a large monorepo there are really at least two categories of users: 1) day to day developers who don’t want to accidentally update-the-world, and 2) monorepo operators who are concerned with normalizing the dep graph across all projects and doing broad swath changes. Ideally the tooling would cater to both types of users but AFAIK there’s no tooling that behaves like that.