pnpm: EPERM: operation not permitted, rename

pnpm version: 1.30.2

Code to reproduce the issue:

Install a very large number of dependencies (like 2,084).

Expected behavior:

Everything is always installed and unpacked.

Actual behavior:

Fairly often we get an error like:

EPERM: operation not permitted, rename 'Z:\sp-client\2\common\temp\pnpm-store\2\onedrive.pkgs.visualstudio.com\highcharts\5.0.14_stage' -> 'Z:\sp-client\2\common\temp\pnpm-store\2\onedrive.pkgs.visualstudio.com\highcharts\5.0.14\node_modules\highcharts'

This usually happens when the store is empty to start with. It seems like it may be an issue with file locking (possibly related to the Windows Defender holding locks on newly created files). This also occurs with different packages each time. It seems to happen more often with larger packages. My naive assumption is that the larger packages just got extracted to disk and we immediately try to rename them while MsMpEng.exe is also trying to scan them.

Rush runs pnpm in a loop, attempting installations up to 5 times, which helps mitigate the issue, but it definitely slows down installs (and confuses people when they see errors).

Suggestion is to run the rename operation in a retry loop, with an exponential backoff.

Additional information:

  • node -v prints: 8.9.4
  • Windows, OS X, or Linux?: Windows

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 27 (19 by maintainers)

Commits related to this issue

Most upvoted comments

Yes, we are good. We’ve done over 1,000 installations since then, and so far nobody has reported any problems. 😃 Thanks again!

as far as I understand, the fix worked. With pnpm@1.32.1 I optimized the fetchers. I removed the redundant file operations that were added during the “fast fix”

Rush has similar code:

https://github.com/Microsoft/web-build-tools/blob/7930ea56efe8f8d9a91aebf359025432aed44d94/apps/rush-lib/src/utilities/Utilities.ts#L110

FYI I encountered this issue mostly when trying to delete a file/folder and immediately recreate a new file/folder with the same name. Whereas if you reuse and existing folder (or overwrite an existing file), the problem did not seem to occur. This is better because it avoids the delay entirely.

I wonder if there is an officially documented best practice somewhere for coexisting with virus scanners.

Cool, I’ll have to optimize the code though because I added some redundant renamings. It was the fastest way to fix it

Thanks a ton for the quick fix! 😃 I just tested 1.31.6 and all operations appear to be working fine. We’ll have some other people do more in depth testing on Monday, but so far it looks good! I really appreciate all your help investigating these weird edge cases.

This should work now in pnpm@1.31.6

also, I reverted the initial fix in rename-overwrite, which did not help

Okay, I’ll keep testing out to see if I can find any more info.