quick-xml: `Reader` unnecessarily panics on 32-bit targets when the document size exceeds 4 GiB
quick_xml::reader::Reader<R: BufRead> allows reading arbitrarily large files without allocating much memory. I am using this in combination with flate2::read::GzDecoder in order to decompress and simultaneously parse very large XML files (several gigabytes per document) without having to store the decompressed file (either in memory or on disk).
Unfortunately, because offsets within the document are computed as usize internally, this causes quick-xml to panic when the document size exceeds 4 GiB on 32-bit targets.
Notably, buffer_position() returns usize, which is not necessarily sufficient for representing the correct position if the input document size exceeds 4 GiB.
Rust’s built-in std::io types typically use u64 for file sizes (and i64 for relative offsets within files). Hence, I believe it would be appropriate to use u64 for internal computations (and to return it from buffer_position() etc.) as well.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 15 (3 by maintainers)
I also just released 0.34.0 with this fix.
Thank you @Mingun and @pronebird for your work on this!