scraper: Memory corruption (probably since 0.21.0)
We have a service at my company that uses scraper for extracting data from HTML that recently started crashing.
I’ve tracked down the issue that appears after bumping scraper from 0.19.0 to the latest version.
The crash happens once in a blue moon with the system allocator, however with Jemalloc on linux, it happens almost immediately.
I have extracted some of our logic that consistently reproduces the issue and uploaded it here: https://github.com/tamasfe/scraper-segfault .
Simply run cargo run and observe the segmentation fault message.
I did not yet dive into the causes and attempts to fix it as tracking it down took me several hours already, I hope the repro is minimal enough for people who are familiar with the project internals.
About this issue
- Original URL
- State: open
- Created 2 years ago
- Reactions: 3
- Comments: 26 (21 by maintainers)
I used this crate v0.20.0 with wasm-bindgen, which used to compile and run just fine. But since v0.21.0, there seems to be some issue with allocator. With console error hook, I identified that the dlmalloc code panics.
I suspect that this issue is related to my wasm issue, and might be related to https://github.com/rustwasm/wasm-pack/issues/1389. (I’m on cargo 1.86.0 btw)
Running the initial reproduction but using
parse_documentrepeatedly, I sometimes get just a segfault and sometimes this panic with a follow-up segfault. As written above, this really smells of stack-based memory corruption which of course makes the backtraces and such dubious.I get a panic in our tree sink implementation, i.e.
html5everwants to know the name of an element which does not exist. And then Rust’s panic handler segfaults trying to create a backtrace which suggests stack-based memory corruption (which jives with Valgrind’s report).I tried Miri, but stacked borrows fails somewhere inside
tendriland tree borrows does not handle the pointer-to-integer casts instring-cache.