RoaringBitmap: RunOptimization Bug for specific BitSet range in RoaringBitmap
Describe the bug
When using runOptimize() on a very specific sequence of continuous bits (around Integer.MAX_VALUE) and then attempt to iterate over the bitmap, this will go into an infinite loop. The sequence of “continuous” bits needs to be set in the range (lower_bound, Integer.MAX_VALUE], where, lower_bound <= Integer.MAX_VALUE - 3.
In other words, whenever more than 4 bits are set below Integer.MAX_VALUE (included), running runOptimize will mess the state such that future iteration over the bitmap loops infinitely.
To Reproduce
public void testRoaringBitmap() {
Roaring64NavigableMap bitmap = new Roaring64NavigableMap();
final long PRE_ADDRESSES = 4L; // (does not fail with 3L, 2L or 1L)
final long POST_ADDRESSES = 120L; // Does not matter how many post max_value
final long INT_MAX_VALUE = 2147483647L;
for (long i = 0; i < PRE_ADDRESSES; i++) {
bitmap.addLong(INT_MAX_VALUE - i);
}
for (long index = 1; index < POST_ADDRESSES; index++) {
bitmap.addLong(INT_MAX_VALUE + index);
}
System.out.println("Size :: " + bitmap.getLongCardinality());
// Serialize
ByteBuf buffer = Unpooled.buffer();
// Improve compression
bitmap.runOptimize(); // This is the line causing the issue
try (ByteBufOutputStream outputStream = new ByteBufOutputStream(buffer);
DataOutputStream dataOutputStream = new DataOutputStream(outputStream)){
bitmap.serialize(dataOutputStream);
} catch (IOException ioe) {
throw new SerializerException("Unexpected error while serializing to a byte array");
}
// Deserialize
Roaring64NavigableMap map = new Roaring64NavigableMap();
try (ByteBufInputStream inputStream = new ByteBufInputStream(buffer)) {
map.deserialize(inputStream);
System.out.println("Deserialize size :: " + map.getLongCardinality());
System.out.println();
} catch (IOException ioe) {
throw new SerializerException("Exception when attempting to " +
"deserialize stream address space.", ioe);
}
SortedSet<Long> addresses = new TreeSet<>(Collections.reverseOrder());
map.forEach(e -> {
System.out.println("Adding :: " + e);
addresses.add(e);
});
}
RoaringBitmap version:
Tested on latest version of the library as well (0.9.3)
Java version:
Java 8
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 16 (16 by maintainers)
@blacelle Should be good now. It was a silly bug.
Nevermind. Looks obvious…
Can you spot the bug?