fastutil: NPE while putting into Object2IntOpenHashMap

@tenghuanhe This really seems like an issue on your side. Possible culprits might be actual hardware problems, faulty JVM / memory management on the OS level, or reflection being used (or some other things which bypass visibility / type safety). These objects cannot be in there simply by type safety guarantees of the JVM. The only way how references get into the map is through the add method and this is typed as String.

Are you maybe using JNI / native calls or JVM agents? These can also mess with memory I think.

Since the maps are so large, it could be a memory issue on OS / JVM level. Which JVM are you using?

Well, you definitely have to look for concurrent access. There no way the insertion in that piece of code can put in keys of the wrong type. So upgrading will not solve your problem.

It’s hard to reproduce this issue manually, but it occurs every several days

this smells a lot like concurrency issues, but the code really seems to be fine in that regard.

The objects you put in it are immutable (strings / ints), you never put nulls in there, I can’t really see what is going wrong. For reference, the 8.1.0 rehash code is:

protected void rehash(final int newN) {
	final K key[] = this.key;
	final int value[] = this.value;
	final int mask = newN - 1;
	final K newKey[] = (K[]) new Object[newN + 1];
	final int newValue[] = new int[newN + 1];
	int i = n, pos;

	for (int j = realSize(); j-- != 0;) {
		while (((key[--i]) == null));

		// NPE happens in this line
		if (!((newKey[pos = (it.unimi.dsi.fastutil.HashCommon.mix((key[i]).hashCode())) & mask]) == null))
			while (!((newKey[pos = (pos + 1) & mask]) == null));

		newKey[pos] = key[i];
		newValue[pos] = value[i];
	}

	newValue[newN] = value[n];
	n = newN;
	this.mask = mask;
	maxFill = maxFill(n, f);
	this.key = newKey;
	this.value = newValue;
}

and is effectively the same as the 8.5.6 code.

I can’t even see how a NPE can possibly occur there except if there were concurrent access (key[i] and newKey have to be non-null at that line except if key is modified concurrently). But the reference to the map (valueToId) is not being leaked and always guarded by a lock.