pnpm: EPERM: operation not permitted, rename
pnpm version: 1.30.2
Code to reproduce the issue:
Install a very large number of dependencies (like 2,084).
Expected behavior:
Everything is always installed and unpacked.
Actual behavior:
Fairly often we get an error like:
EPERM: operation not permitted, rename 'Z:\sp-client\2\common\temp\pnpm-store\2\onedrive.pkgs.visualstudio.com\highcharts\5.0.14_stage' -> 'Z:\sp-client\2\common\temp\pnpm-store\2\onedrive.pkgs.visualstudio.com\highcharts\5.0.14\node_modules\highcharts'
This usually happens when the store is empty to start with. It seems like it may be an issue with file locking (possibly related to the Windows Defender holding locks on newly created files). This also occurs with different packages each time. It seems to happen more often with larger packages. My naive assumption is that the larger packages just got extracted to disk and we immediately try to rename them while MsMpEng.exe is also trying to scan them.
Rush runs pnpm in a loop, attempting installations up to 5 times, which helps mitigate the issue, but it definitely slows down installs (and confuses people when they see errors).
Suggestion is to run the rename operation in a retry loop, with an exponential backoff.
Additional information:
node -v
prints: 8.9.4- Windows, OS X, or Linux?: Windows
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 27 (19 by maintainers)
Commits related to this issue
- fix: retry more on EPERM ref pnpm/pnpm#1015 — committed to zkochan/rename-overwrite by zkochan 6 years ago
- fix: don't unpack to the same folder twice ref pnpm/pnpm#1015 — committed to pnpm/tarball-fetcher by zkochan 6 years ago
- perf: update package-store to v0.16 Getting rid of redundant renamings ref #1015 — committed to pnpm/pnpm by zkochan 6 years ago
Yes, we are good. We’ve done over 1,000 installations since then, and so far nobody has reported any problems. 😃 Thanks again!
as far as I understand, the fix worked. With
pnpm@1.32.1
I optimized the fetchers. I removed the redundant file operations that were added during the “fast fix”Rush has similar code:
https://github.com/Microsoft/web-build-tools/blob/7930ea56efe8f8d9a91aebf359025432aed44d94/apps/rush-lib/src/utilities/Utilities.ts#L110
FYI I encountered this issue mostly when trying to delete a file/folder and immediately recreate a new file/folder with the same name. Whereas if you reuse and existing folder (or overwrite an existing file), the problem did not seem to occur. This is better because it avoids the delay entirely.
I wonder if there is an officially documented best practice somewhere for coexisting with virus scanners.
Cool, I’ll have to optimize the code though because I added some redundant renamings. It was the fastest way to fix it
Thanks a ton for the quick fix! 😃 I just tested 1.31.6 and all operations appear to be working fine. We’ll have some other people do more in depth testing on Monday, but so far it looks good! I really appreciate all your help investigating these weird edge cases.
This should work now in
pnpm@1.31.6
also, I reverted the initial fix in
rename-overwrite
, which did not helpOkay, I’ll keep testing out to see if I can find any more info.