turbo: [turborepo] [yarn 3] pruned lockfile missing resolution entries

What version of Turborepo are you using?

1.6.3

What package manager are you using / does the bug impact?

Yarn v2/v3 (node_modules linker only)

What operating system are you using?

Mac

Describe the Bug

To optimize our Docker builds we are attempting to use turbo prune to create a lighter weight image. After running prune we copy our pruned lockfile and .json files into a directory and then attempt to run an immutable install.

We have observed that when the root package.json includes yarn resolutions we are unable to run yarn --immutable successfully in the pruned monorepo.

For example

In this reproduction ajv is a dependency of eslint so it is a necessary dependency in the pruned monorepo.

ericjacobson@Erics-MacBook-Air pruned % yarn why ajv
├─ @eslint/eslintrc@npm:0.4.3
│  └─ ajv@npm:6.12.6 (via npm:^6.12.4)
│
├─ eslint@npm:7.32.0
│  └─ ajv@npm:6.12.6 (via npm:^6.10.0)
│
└─ table@npm:6.8.1
   └─ ajv@npm:8.11.2 (via npm:^8.0.1)

When the resolution is not included in the root package.json the generated out/yarn.lock is correct, and includes an ajv entry.

When the resolution is included in the root package.json then generated out/yarn.lock is incorrect, and does not include any ajv entries.

Expected Behavior

The pruned lockfile should include dependencies that have a corresponding yarn resolution.

To Reproduce

See the README.md in the reproduction repo for more details.

Reproduction Repo

https://github.com/erj826/turbo-prune-bug-repro-11-21

About this issue

Original URL
State: open
Created 2 years ago
Reactions: 4
Comments: 20 (9 by maintainers)

Commits related to this issue

feat: Use Rust Berry lockfile impl (#4684) ### Description Built on top of #4629 Overview of changes: - ~~Change `subgraph` so we calculate workspace's closure in parallel in a similar manner... — committed to vercel/turbo by chris-olszewski a year ago
fix: allow for workplace descriptors without a protocol (#4755) ### Description Fixes the issue found by @quinnturner in [#2891](https://github.com/vercel/turbo/issues/2791#issuecomment-152647712... — committed to vercel/turbo by chris-olszewski a year ago
fix: berry prune including all builtin patch descriptors (#4770) ### Description Addresses the issue @erj826 found in [#2791](https://github.com/vercel/turbo/issues/2791#issuecomment-1527616578) ... — committed to vercel/turbo by chris-olszewski a year ago
chore: vercel/turbo#2791 로 인하여 --immutable 옵션 삭제 — committed to syi0808/liblibrary by syi0808 9 months ago

Most upvoted comments

Any update on this? I don’t see much activity happening on this issue but I have the exact same problem in our app and this is quite blocking.

+13

alexbchr on Mar 21, 2023

I have the same problem with alex-dixon in version 1.10.12. After I ran turbo prune --docker and tried to yarn install --immutable, this error occurred:

➤ YN0000: ┌ Resolution step
➤ YN0000: └ Completed in 0s 423ms

➤ YN0000: ┌ Post-resolution validation
➤ YN0000: │ @@ -13,19 +13,8 @@
➤ YN0000: │    checksum: 09b7a97a05c80540db6c9e4ddf8c5d2ebb06cae5caf3a87e33c33f27f8c4d49d9c67a2d72f1570e796045288fad569f98a26ceba0c4f5fad2af84b6ad855c4fb
➤ YN0000: │    languageName: node
➤ YN0000: │    linkType: hard
➤ YN0000: │
➤ YN0028: │ -"@babel/types@npm:^7.8.3":
➤ YN0028: │ -  version: 7.20.7
➤ YN0028: │ -  resolution: "@babel/types@npm:7.20.7"
➤ YN0028: │ -  dependencies:
➤ YN0028: │ -    "@babel/helper-string-parser": ^7.19.4
➤ YN0028: │ -    "@babel/helper-validator-identifier": ^7.19.1
➤ YN0028: │ -    to-fast-properties: ^2.0.0
➤ YN0028: │ -  checksum: b39af241f0b72bba67fd6d0d23914f6faec8c0eba8015c181cbd5ea92e59fc91a52a1ab490d3520c7dbd19ddb9ebb76c476308f6388764f16d8201e37fae6811
➤ YN0028: │ -  languageName: node
➤ YN0028: │ -  linkType: hard
➤ YN0028: │ -
➤ YN0000: │  "@eslint/eslintrc@npm:^1.4.1":
➤ YN0000: │    version: 1.4.1
➤ YN0000: │    resolution: "@eslint/eslintrc@npm:1.4.1"
➤ YN0000: │    dependencies:
➤ YN0000: │ @@ -224,8 +213,22 @@
➤ YN0000: │    checksum: e009a2bfb50e90ca9b7c6e8f648f8464067271fd99116f881073fa6fa76dc8d0133181dd65e6614d5fb1220d671d67b0124aef7d97dc02d7e342ab143a47779d
➤ YN0000: │    languageName: node
➤ YN0000: │    linkType: hard
➤ YN0000: │
➤ YN0028: │ +"@types/node@npm:*":
➤ YN0028: │ +  version: 20.5.7
➤ YN0028: │ +  resolution: "@types/node@npm:20.5.7"
➤ YN0028: │ +  languageName: node
➤ YN0028: │ +  linkType: hard
➤ YN0028: │ +
➤ YN0028: │ +"@types/responselike@npm:^1.0.0":
➤ YN0028: │ +  version: 1.0.0
➤ YN0028: │ +  resolution: "@types/responselike@npm:1.0.0"
➤ YN0028: │ +  dependencies:
➤ YN0028: │ +    "@types/node": "*"
➤ YN0028: │ +  languageName: node
➤ YN0028: │ +  linkType: hard
➤ YN0028: │ +
➤ YN0000: │  "@types/semver@npm:^7.3.12":
➤ YN0000: │    version: 7.3.13
➤ YN0000: │    resolution: "@types/semver@npm:7.3.13"
➤ YN0000: │    checksum: 00c0724d54757c2f4bc60b5032fe91cda6410e48689633d5f35ece8a0a66445e3e57fa1d6e07eb780f792e82ac542948ec4d0b76eb3484297b79bd18b8cf1cb0
➤ YN0000: │
➤ YN0028: │ The lockfile would have been modified by this install, which is explicitly forbidden.
➤ YN0000: └ Completed
➤ YN0000: Failed with errors in 0s 442ms

ydhn on Aug 28, 2023

Not sure if this helps but I ran into this as well on 1.10.0:

 > [stage-0  8/11] RUN --mount=type=secret,mode=0644,id=yarnrc,target=/home/node/.yarnrc.yml yarn install --immutable:                                                                                             
#0 1.005 ➤ YN0000: ┌ Resolution step                                                                                                                                                                               
#0 1.170 ➤ YN0000: └ Completed                                                                                                                                                                                     
#0 1.191                                                                                                                                                                                                           
#0 1.191 ➤ YN0000: ┌ Post-resolution validation                                                                                                                                                                    
#0 1.191 ➤ YN0000: │ @@ -4,19 +4,8 @@
#0 1.191 ➤ YN0000: │  __metadata:
#0 1.191 ➤ YN0000: │    version: 6
#0 1.191 ➤ YN0000: │    cacheKey: 8
#0 1.191 ➤ YN0000: │  
#0 1.191 ➤ YN0028: │ -"@babel/types@npm:^7.8.3":
#0 1.192 ➤ YN0028: │ -  version: 7.22.4
#0 1.192 ➤ YN0028: │ -  resolution: "@babel/types@npm:7.22.4"
#0 1.192 ➤ YN0028: │ -  dependencies:
#0 1.192 ➤ YN0028: │ -    "@babel/helper-string-parser": ^7.21.5
#0 1.193 ➤ YN0028: │ -    "@babel/helper-validator-identifier": ^7.19.1
#0 1.193 ➤ YN0028: │ -    to-fast-properties: ^2.0.0
#0 1.193 ➤ YN0028: │ -  checksum: ffe36bb4f4a99ad13c426a98c3b508d70736036cae4e471d9c862e3a579847ed4f480686af0fce2633f6f7c0f0d3bf02da73da36e7edd3fde0b2061951dcba9a
#0 1.193 ➤ YN0028: │ -  languageName: node
#0 1.193 ➤ YN0028: │ -  linkType: hard
#0 1.193 ➤ YN0028: │ -
#0 1.193 ➤ YN0000: │  "@cspotcode/source-map-support@npm:^0.8.0":
#0 1.193 ➤ YN0000: │    version: 0.8.1
#0 1.193 ➤ YN0000: │    resolution: "@cspotcode/source-map-support@npm:0.8.1"
#0 1.193 ➤ YN0000: │    dependencies:
#0 1.193 ➤ YN0000: │ 
#0 1.194 ➤ YN0028: │ The lockfile would have been modified by this install, which is explicitly forbidden.
#0 1.194 ➤ YN0000: └ Completed
#0 1.194 ➤ YN0000: Failed with errors in 0s 192ms
------

When I remove it the install succeeds.

alex-dixon on Jun 11, 2023

@chris-olszewski thank you for the quick turnaround! Prune appears to be working without any issues for our monorepo using 1.9.4-canary.7!

erj826 on May 3, 2023

Only downgrading from 1.9.8 to 1.9.3 worked for me

cimabe on May 23, 2023

Hi, i’m not sure if the following is related to this same issue but seems to me that there are some similarities.

I have to patch a package in my project and i’m using yarn patch to do that (yarn v3), when i do turbo prune the package.json file in the /out folder doesn’t have the resolution with the patch. e.g. "package-to-patch@x.x.x": "patch:package-to-patch@x.x.x#.yarn/patches/package-to-patch@x.x.x-0608291d42.patch", it does have other resolutions but not the one that has the patch. The same happens with the .lock file, locally i see the package with the patch in the lock file but then in the /out i’m not seeing that.

is this the same issue or a different one? happy to provide more information if needed or any help to troubleshoot this.

JuanJo4 on May 17, 2023

Thanks for testing this out!

@quinnturner

In my large monorepo, there is, unfortunately, still some drift

Are there entire entries missing or is it primarily drift between the keys?

One edge case

I see the issue here, should have a fix up soon.

@erj826 If that’s the only difference, then that points to our inference of the compatibility patches being incorrect. Will be taking a look at this today.

chris-olszewski on Apr 28, 2023

This was first mentioned in my ticket in #2049. I will close my version as this ticket is more scoped. CC: @chris-olszewski, since you’re assigned to that ticket, perhaps you intend to assign yourself to this ticket?

quinnturner on Dec 21, 2022