berry: [Bug]: Yarn patches are not applied for deps using `resolutions`

Self-service

  • I’d be willing to implement a fix

Describe the bug

Short: Yarn patches (.yarn/patches) are not applied when node modules are restored from a cache (ignores the fetch step).

Long: In CI, we cache the .yarn/cache and .yarn/install-state.gz files based on the checksum of yarn.lock. This works pretty great and reduces the “fetch step” during yarn install down to 1 second. Overall, the entirety of yarn install takes less than 2 minutes (most of it is linking).

However, we’ve noticed the new patches do not get applied when we use this approach, while patches work locally. This is very apparent because builds crash in CI that were fixed by this patch, and the line numbers in the stack trace are pointing to the original pre-patch lines.

Since we have no way to apply patches manually, I’m not sure how to work around this. Busting the cache didn’t work. And also caching .yarn/patches didn’t work.

To reproduce

Our problem is unique to Buildkite CI but it can maybe be replicated with GitHub actions: https://github.com/actions/setup-node

Environment

System:
    OS: macOS 12.3
    CPU: (16) x64 Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
  Binaries:
    Node: 16.13.1 - /private/var/folders/93/p1hnsld57s79d4b503vnmq2w0000gn/T/xfs-b98bb0af/node
    Yarn: 3.1.0 - /private/var/folders/93/p1hnsld57s79d4b503vnmq2w0000gn/T/xfs-b98bb0af/yarn
    npm: 8.1.2 - ~/.nvm/versions/node/v16.13.1/bin/npm
  npmPackages:
    jest: ^27.4.7 => 27.5.1

Additional context

We’re using Buildkite with the cache plugin:

steps:
  - plugins:
    - <cache plugin>:
        key: yarn-node-modules-{{ checksum "yarn.lock" }}-v1
        paths:
          - ./.yarn/cache
          - ./.yarn/install-state.gz

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 2
  • Comments: 30 (6 by maintainers)

Commits related to this issue

Most upvoted comments

This particular problem is known (and fixed, once #4218 is merged). The resolutions syntax expects an exact match on the range compared to what the dependency truly is, but yarn patch-commit incorrectly sets a different range that generally doesn’t match what the dependency actually is. For example in your example repo, @react-native-community/cli is depended upon by react-native which lists it with the range "^6.0.0" (not "6.4.0").

To fixes that, just replace:

"resolutions": {
  "@react-native-community/cli@6.4.0": "patch:..."
}

By:

"resolutions": {
  "@react-native-community/cli@^6.0.0": "patch:..."
}

Another option is to remove the range altogether, in which case all instances of @react-native-community/cli will be patched (in your last message you said it doesn’t work, but I suspect you made a mistaken copy-paste and forgot the range in your snippets, as I just tested it and it definitely works):

"resolutions": {
  "@react-native-community/cli": "patch:..."
}

Yes this is is still very annoying. Anytime we use patches, they don’t work immediately, and we constantly have to tinker with the resolution version to get them working. A fix in a release would be much appreciated.

I’m on yarn 3.2.3 and was having this issue where the patches would only be applied on first install, but would not work on subsequent installs.

I fixed it by changing the resolutions added by yarn automatically from this:

"@expo/cli@0.4.9": "patch:@expo/cli@npm%3A0.6.2#./.yarn/patches/@expo-cli-npm-0.6.2-c9460e7c0e.patch",
"@expo/cli@^0.6.2": "patch:@expo/cli@npm%3A0.6.2#./.yarn/patches/@expo-cli-npm-0.6.2-c9460e7c0e.patch",
"@expo/cli@0.6.2": "patch:@expo/cli@npm%3A0.6.2#./.yarn/patches/@expo-cli-npm-0.6.2-c9460e7c0e.patch"

To this:

"@expo/cli": "patch:@expo/cli@npm:^0.6.2#./.yarn/patches/@expo-cli-npm-0.6.2-c9460e7c0e.patch",

Notice the unescaping of the colon after npm and the inclusion of the ^ before the version number.

Now the patch gets correctly applied every time!

It’s a bug fix so why it’s not coming in 3x?

When will this fix be released on the stable branch?

The fix has been released in v3.2.3

I also faced this issue today, can confirm that the ^ worked. It’s quite a hard error to debug when using pnp, so I really hope it will get fixed soon, as I think many people will just give up before they come in here and realize how simple the fix is.

I’m still experiencing issues with my patch resolution on 4.0.0-rc.14.

These were the resolutions generated by patch-commit on 3.x:

"@rewardopl/react-native-toast@1.2.0": "patch:@rewardopl/react-native-toast@npm:1.2.0::__archiveUrl=https%3A%2F%2Fnpm.pkg.github.com%2Fdownload%2F%40rewardopl%2Freact-native-toast%2F1.2.0%2F30fa3fff8cce2eaaaff35fbc218041161631fd35cf06375f76834d890cffc2bd#.yarn/patches/@rewardopl-react-native-toast-npm-1.2.0-723ae2f371",
"@rewardopl/react-native-ui@1.7.0": "patch:@rewardopl/react-native-ui@npm:1.7.0::__archiveUrl=https%3A%2F%2Fnpm.pkg.github.com%2Fdownload%2F%40rewardopl%2Freact-native-ui%2F1.7.0%2Fe3c4f7ab023dac67d7ad2157fb6502a8cf46ac5f7f7b035f59f2c28c7a824264#.yarn/patches/@rewardopl-react-native-ui-npm-1.7.0-7602a18b2b",
  • Changing key to @rewardopl/react-native-toast continues to throw an error. Note: unscoped packages didn’t have this problem even in 3.x.
  • Changing key to @rewardopl/react-native-toast@^1.2.0 does not throw Error: Patch locators must explicitly define their source anymore during install, but still doesn’t apply the patch. No error or warning is visible during install.

Edit: As per https://github.com/yarnpkg/berry/issues/4711#issue-1329337417, this worked:

"@rewardopl/react-native-toast@^1.2.0": "patch:@rewardopl/react-native-toast@npm:^1.2.0#.yarn/patches/@rewardopl-react-native-toast-npm-1.2.0-723ae2f371",
"@rewardopl/react-native-ui@^1.7.0": "patch:@rewardopl/react-native-ui@npm:^1.7.0#.yarn/patches/@rewardopl-react-native-ui-npm-1.7.0-7602a18b2b",

This particular problem is known (and fixed, once #4218 is merged). The resolutions syntax expects an exact match on the range compared to what the dependency truly is, but yarn patch-commit incorrectly sets a different range that generally doesn’t match what the dependency actually is. For example in your example repo, @react-native-community/cli is depended upon by react-native which lists it with the range "^6.0.0" (not "6.4.0").

To fixes that, just replace:

"resolutions": {
  "@react-native-community/cli@6.4.0": "patch:..."
}

By:

"resolutions": {
  "@react-native-community/cli@^6.0.0": "patch:..."
}

Another option is to remove the range altogether, in which case all instances of @react-native-community/cli will be patched (in your last message you said it doesn’t work, but I suspect you made a mistaken copy-paste and forgot the range in your snippets, as I just tested it and it definitely works):

"resolutions": {
  "@react-native-community/cli": "patch:..."
}

I’m still experiencing this issue and adding ^ manually works.

That… makes sense in retrospect. Will give that a try again on Monday.

Although the PR is improving patches, it would be nice if Yarn would warn if there’s a patch that doesn’t get applied for some reason.

I made the above change and found similar resolution higher up that was problematic in my case. Although for me the problem was it not applying all the patches, just some of them. Anyway, hope this helps…

"resolutions": {
    "@types/react": "17.0.2",
    "@types/react-dom": "17.0.2",
    "@aries-framework/core": "0.3.3",
  ...
    "@aries-framework/core@0.3.3": "patch:@aries-framework/core@npm:0.3.3#./.yarn/patches/@aries-framework-core-npm-0.3.3-dd6486de3d.patch",
  }

I had to remove "@aries-framework/core": "0.3.3", for mine to work.

When will this fix be released on the stable branch?

The fix has been released in v3.2.3

I am using yarn v3.3.1 and it didn’t work for me.

@arcanis if mine is a duplicate of this, and this is closed, and mine happened today on the latest release, how can this be fixed/closed?

Tested and works 👍

Thanks for the info and reading my rambling.

@arcanis @merceyz Tried another repo, same problem. Had teammates test patches and they report it also not patching, even locally.

So I created a simple repo as a test case and the patch still is not applied, even locally: https://github.com/milesj/yarn-patch-transient-deps We tried Yarn 3.2 and 3.1 and both failed.

This looks to be a legitimate bug in Yarn.


EDIT: Confirmed patching works for explicit deps (patch: in the version range), but not for transient deps OR explicit deps using resolutions:

Works:

  "devDependencies": {
    "@react-native-community/cli": "patch:@react-native-community/cli@npm:^6.4.0#../../.yarn/patches/@react-native-community-cli-npm-6.4.0-f99ee73e09",
  }

Does not work:

  "devDependencies": {
    "@react-native-community/cli": "^6.4.0"
  },
  "resolutions": {
    "@react-native-community/cli": "patch:@react-native-community/cli@npm:^6.4.0#../../.yarn/patches/@react-native-community-cli-npm-6.4.0-f99ee73e09",
  }

Also does not work (transitive dep):

  "resolutions": {
    "@react-native-community/cli": "patch:@react-native-community/cli@npm:^6.4.0#../../.yarn/patches/@react-native-community-cli-npm-6.4.0-f99ee73e09",
  }

So in CI I went ahead and cat the file that should be patched after yarn install, and it definitely does not have the patch applied. Just going to switch to patch-package.