Bug report

What is the current behavior?

We have a large app in a monorepo. Turning on filesystem caching for the client bundle is fine and shows great speed improvement. With SSR, we have ~100 entrypoints - the first run appears to cache without error, but on warm runs (with no code changes) we’ve observed errors thrown from a number of different places (seeming to depend on the size and state of master, and perhaps other settings/package bumps). I would guess however that all are the same root cause having to do with the caching going bad somewhere.

Example errors - in each case I’m just posting the first of thousands that spam the console:

<t> [webpack.cache.PackFileCacheStrategy] restore cache content 1 (78.5 MiB): 1139.633175 ms
    [webpack.cache.PackFileCacheStrategy] starting to restore cache content 2 (2.03 GiB) because of request to: Compilation/codeGeneration|/PATH/TO/PROJECT/node_modules/babel-loader/lib/index.js??ruleSet[1].rules[0]!/PATH/TO/PROJECT/packages/react-component/src/components/Component/index.js|ssr-0
<w> [webpack.cache.PackFileCacheStrategy] Restoring failed for Compilation/codeGeneration|/PATH/TO/PROJECT/node_modules/babel-loader/lib/index.js??ruleSet[1].rules[0]!/PATH/TO/PROJECT/packages/react-component/src/components/Component/index.js|ssr-0 from pack: TypeError: Cannot read property 'map' of undefined
    [webpack.cache.PackFileCacheStrategy] TypeError: Cannot read property 'map' of undefined
        at /PATH/TO/PROJECT/node_modules/webpack5/lib/cache/PackFileCacheStrategy.js:787:22

<t> [webpack.cache.PackFileCacheStrategy] restore cache content 3 (112 MiB): 450.47154 ms
    [webpack.cache.PackFileCacheStrategy] starting to restore cache content 4 (6.01 GiB) because of request to: Compilation/codeGeneration|/PATH/TO/PROJECT/node_modules/cache-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[0]!/PATH/TO/PROJECT/node_modules/css-loader-for-wp5/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[1]!/PATH/TO/PROJECT/node_modules/postcss-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[2]!/PATH/TO/PROJECT/node_modules/resolve-url-loader/index.js??ruleSet[1].rules[3].oneOf[2].use[3]!/PATH/TO/PROJECT/node_modules/sass-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[4]!/PATH/TO/PROJECT/node_modules/LIB/lib/FILE.scss|ssr-0
<w> [webpack.cache.PackFileCacheStrategy] Restoring failed for Compilation/codeGeneration|/PATH/TO/PROJECT/node_modules/cache-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[0]!/PATH/TO/PROJECT/node_modules/css-loader-for-wp5/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[1]!/PATH/TO/PROJECT/node_modules/postcss-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[2]!/PATH/TO/PROJECT/node_modules/resolve-url-loader/index.js??ruleSet[1].rules[3].oneOf[2].use[3]!/PATH/TO/PROJECT/node_modules/sass-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[4]!/PATH/TO/PROJECT/node_modules/LIB/lib/FILE.scss|ssr-0 from pack: Error: Unexpected end of stream
    [webpack.cache.PackFileCacheStrategy] Error: Unexpected end of stream
        at ensureBuffer (/PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/BinaryMiddleware.js:588:11)
        at read (/PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/BinaryMiddleware.js:601:4)
        at Array.<anonymous> (/PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/BinaryMiddleware.js:909:21)
        at BinaryMiddleware._deserialize (/PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/BinaryMiddleware.js:936:26)
        at /PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/BinaryMiddleware.js:557:9
        at /PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/SerializerMiddleware.js:123:27

<t> [webpack.cache.PackFileCacheStrategy] restore cache content 3 (109 MiB): 445.1846 ms
    [webpack.cache.PackFileCacheStrategy] starting to restore cache content 4 (5.94 GiB) because of request to: Compilation/codeGeneration|/PATH/TO/PROJECT/node_modules/cache-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[0]!/PATH/TO/PROJECT/node_modules/css-loader-for-wp5/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[1]!/PATH/TO/PROJECT/node_modules/postcss-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[2]!/PATH/TO/PROJECT/node_modules/resolve-url-loader/index.js??ruleSet[1].rules[3].oneOf[2].use[3]!/PATH/TO/PROJECT/node_modules/sass-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[4]!/PATH/TO/PROJECT/packages/react-component/src/components/Icon/Icon.scss|ssr-1
<w> [webpack.cache.PackFileCacheStrategy] Restoring failed for Compilation/codeGeneration|/PATH/TO/PROJECT/node_modules/cache-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[0]!/PATH/TO/PROJECT/node_modules/css-loader-for-wp5/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[1]!/PATH/TO/PROJECT/node_modules/postcss-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[2]!/PATH/TO/PROJECT/node_modules/resolve-url-loader/index.js??ruleSet[1].rules[3].oneOf[2].use[3]!/PATH/TO/PROJECT/node_modules/sass-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[4]!/PATH/TO/PROJECT/packages/react-component/src/components/Icon/Icon.scss|ssr-1 from pack: Error: Unexpected end of object at position 1621788
    [webpack.cache.PackFileCacheStrategy] Error: Unexpected end of object at position 1621788
        at decodeValue (/PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/ObjectMiddleware.js:608:12)
        at ObjectMiddleware.deserialize (/PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/ObjectMiddleware.js:720:17)
        at /PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/ObjectMiddleware.js:711:19
        at /PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/SerializerMiddleware.js:123:27

<w> [webpack.cache.PackFileCacheStrategy] Restoring failed for RealContentHashPlugin|generate|page-title.HASH.INCOMPLETE.js from pack: Error: Unexpected header byte 0x1c
    [webpack.cache.PackFileCacheStrategy] Error: Unexpected header byte 0x1c
        at Array.<anonymous> (/PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/BinaryMiddleware.js:917:14)
        at BinaryMiddleware._deserialize (/PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/BinaryMiddleware.js:936:26)
        at /PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/BinaryMiddleware.js:557:9

Believe we also got a TypeError: Cannot read property 'deserialize' of undefined at some point.

I tried bisecting our bundles to see if there was a bad one. As far I can tell there is not, but this did reveal that when we build a “small” enough number of bundles (25-40 out of the 100 depending on size) everything seems to be fine. Finally I tracked down what I think is a big clue; if one of the pack files is > 2GB, that’s when we get a problem.

webpack_saved_good_cache/default-production$ ls -s
total 1930004
  67008 0.pack   101596 2.pack     7752 4.pack    35160 index.pack
  98896 1.pack  1571280 3.pack    13168 5.pack    35144 index.pack.old

webpack_saved_bad_cache/default-production$ ls -s
total 6592188
  53128 0.pack    25004 2.pack  6302896 4.pack
  49540 1.pack   114892 3.pack    46712 index.pack

I did experiment with the number of entrypoints to make a packfile just over or under 2GB (the examples above are more extreme) and that seems to be the key factor. Note also in console errors that the preceding starting to restore cache content 2 (2.03 GiB) line is always > 2GB. This all seems like it could be closely related to https://github.com/webpack/webpack/issues/12126 and https://github.com/webpack/webpack/pull/12191. However it is certainly possible this is a coincidence or red herring.

If the current behavior is a bug, please provide the steps to reproduce.

Given the complexity/size of our app and config I’m not sure I’ll be able to provide a repro but I’ll give what I can if you need it.

What is the expected behavior? Persistent caching on a large app should work with no or minimal errors

Other relevant information: webpack version: 5.62.1 (also tried out 5.64.4) Node.js version: 14.9.0 Operating System: MacOS

_Originally posted by @jpm4 in https://github.com/webpack/webpack/discussions/14906_