zlib-ng: Custom allocators can break alignment that's required for SIMD

I’ve been using zlib-ng with compat as my system zlib in a customized FreeBSD fork with no issues so far, until right now an innocent npm install run just crashed:

(lldb) bt
* thread #8, name = 'node', stop reason = signal SIGBUS: hardware error
  * frame #0: 0x000026e7e7271704 libz.so.6`crc32_fold_load(fold=0x000026e80c7450c8, fold0=0x0000000000e08d60, fold1=0x0000000000e08d50, fold2=0x0000000000e08d40, fold3=0x0000000000e08d30) at crc32_fold_pclmulqdq.c:221:14
    frame #1: 0x000026e7e7270e49 libz.so.6`crc32_fold_copy_pclmulqdq(crc=0x000026e80c7450c8, dst="", src="{\"name\":\"ui\",\"version\":\"0.1.0\",\"requires\":{\"@emotion/react\":\"^11.1.5\",\"@emotion/styled\":\"^11.1.5\",\"@fontsource/roboto\":\"^4.2.1\",\"@mui/material\":\"^5.0.1\",\"@mui/icons-material\":\"^5.0.1\",\"@mui/lab\":\"^5.0.0-alpha.48\",\"@mui/styles\":\"^5.0.1\",\"@react-hook/resize-observer\":\"^1.2.0\",\"@types/jest\":\"^26.0.20\",\"@types/node\":\"^16.3.1\",\"@types/react\":\"^17.0.0\",\"@types/react-dom\":\"^17.0.0\",\"date-fns\":\"^2.18.0\",\"date-fns-tz\":\"^1.1.3\",\"gzipper\":\"^5.0.0\",\"react\":\"^17.0.1\",\"react-dom\":\"^17.0.1\",\"react-scripts\":\"^4.0.3\",\"typescript\":\"^4.3.5\",\"@testing-library/dom\":\"^8.1.0\",\"@testing-library/jest-dom\":\"^5.11.4\",\"@testing-library/react\":\"^11.2.7\",\"@testing-library/user-event\":\"^12.8.3\",\"http-proxy-middleware\":\"^2.0.1\",\"msw\":\"^0.26.2\",\"prettier\":\"^2.2.1\"},\"dependencies\":{\"@babel/code-frame\":{\"version\":\"7.14.5\",\"integrity\":\"sha512-9pzDqyc6OLDaqe+zbACgFkb6fKMNG6CObKpnYXChRsvYGyEdc7CA2BaqeOM+vOtCS5ndmJicPJhKAwYRI6UfFw==\",\"requires\":{\"@babel/highlight\":\"^7.14.5\"}},\"@babel/compat-data\":{\"version\":\"7.14.7\",\"integrity\":\"sha512-nS6dZaISCXJ"..., len=65536) at crc32_fold_pclmulqdq.c:251:5
    frame #2: 0x000026e7e7282f90 libz.so.6`crc32_fold_copy_stub(crc=0x000026e80c7450c8, dst="", src="{\"name\":\"ui\",\"version\":\"0.1.0\",\"requires\":{\"@emotion/react\":\"^11.1.5\",\"@emotion/styled\":\"^11.1.5\",\"@fontsource/roboto\":\"^4.2.1\",\"@mui/material\":\"^5.0.1\",\"@mui/icons-material\":\"^5.0.1\",\"@mui/lab\":\"^5.0.0-alpha.48\",\"@mui/styles\":\"^5.0.1\",\"@react-hook/resize-observer\":\"^1.2.0\",\"@types/jest\":\"^26.0.20\",\"@types/node\":\"^16.3.1\",\"@types/react\":\"^17.0.0\",\"@types/react-dom\":\"^17.0.0\",\"date-fns\":\"^2.18.0\",\"date-fns-tz\":\"^1.1.3\",\"gzipper\":\"^5.0.0\",\"react\":\"^17.0.1\",\"react-dom\":\"^17.0.1\",\"react-scripts\":\"^4.0.3\",\"typescript\":\"^4.3.5\",\"@testing-library/dom\":\"^8.1.0\",\"@testing-library/jest-dom\":\"^5.11.4\",\"@testing-library/react\":\"^11.2.7\",\"@testing-library/user-event\":\"^12.8.3\",\"http-proxy-middleware\":\"^2.0.1\",\"msw\":\"^0.26.2\",\"prettier\":\"^2.2.1\"},\"dependencies\":{\"@babel/code-frame\":{\"version\":\"7.14.5\",\"integrity\":\"sha512-9pzDqyc6OLDaqe+zbACgFkb6fKMNG6CObKpnYXChRsvYGyEdc7CA2BaqeOM+vOtCS5ndmJicPJhKAwYRI6UfFw==\",\"requires\":{\"@babel/highlight\":\"^7.14.5\"}},\"@babel/compat-data\":{\"version\":\"7.14.7\",\"integrity\":\"sha512-nS6dZaISCXJ"..., len=65536) at functable.c:335:5
    frame #3: 0x000026e7e727d358 libz.so.6`read_buf(strm=0x000026e7f64da840, buf="", size=65536) at deflate.c:1084:9
    frame #4: 0x000026e7e727d04a libz.so.6`fill_window(s=0x000026e80c745008) at deflate.c:1199:13
    frame #5: 0x000026e7e727f8b2 libz.so.6`deflate_medium(s=0x000026e80c745008, flush=0) at deflate_medium.c:183:13
    frame #6: 0x000026e7e727e55c libz.so.6`deflate(strm=0x000026e7f64da840, flush=0) at deflate.c:922:18
    frame #7: 0x000026dfe47e24d7 node`___lldb_unnamed_symbol43331 + 279

What the f… 0x000026e80c7450c8, that’s not aligned to 16. Turns out:

(lldb) p *strm
(z_stream) $0 = {
…
  zalloc = 0x000028a3695e3140 (node`___lldb_unnamed_symbol43351)
  zfree = 0x000028a3695e3190 (node`___lldb_unnamed_symbol43352)
…
}

Oops, the allocator comes from node! Which of course does not know of zlib-ng’s alignment requirements!

Patching zlib-ng to blatantly disrespect the applications’ provided zalloc and zfree does of course help. But I’m surprised such a fundamental problem has not been documented on this issue tracker yet…

Something has to be done, maybe any potentially-SIMDed memory should use the aligned zalloc directly or something?

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 22 (21 by maintainers)

Most upvoted comments

Fix was merged into develop.

So this works and preserves the usage of aligned loads for the data (how much faster are they even?):

For modern (post Nehalem) implementations of the Intel ISA? Not much. But anything predating that, it’s a pretty substantial impact.

Yes, that works.

You’ve had a 0x16 there, not decimal 16 😃