RoaringBitmap: Getting java.lang.ArrayIndexOutOfBoundsException: 255

I am trying to read a Roaring64BitMap stored on HDFS and getting the following exception

val hdfs = FileSystem.get(new Configuration())
val filePath = "/roaringBitMap/data/31455978"
val bitmap = new Roaring64Bitmap
val input = new DataInputStream(hdfs.open(new Path(filePath) ))
try{
      bitmap.deserialize(input)
    }catch{
      case e: Exception => e.printStackTrace()
    }finally {
      input.close()
}

java.lang.ArrayIndexOutOfBoundsException: 0 at org.roaringbitmap.art.Node48.replaceChildren(Node48.java:279) at org.roaringbitmap.art.Art.deserialize(Art.java:359) at org.roaringbitmap.art.Art.deserialize(Art.java:356) at org.roaringbitmap.art.Art.deserializeArt(Art.java:286) at org.roaringbitmap.longlong.HighLowContainer.deserialize(HighLowContainer.java:193) at org.roaringbitmap.longlong.Roaring64Bitmap.deserialize(Roaring64Bitmap.java:598) at $line34.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.liftedTree1$1(<pastie>:66) at $line34.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.<init>(<pastie>:65) at $line34.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.<init>(<pastie>:80) at $line34.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.<init>(<pastie>:82) at $line34.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.<init>(<pastie>:84) at $line34.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.<init>(<pastie>:86) at $line34.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.<init>(<pastie>:88) at $line34.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.<init>(<pastie>:90) at $line34.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.<init>(<pastie>:92) at $line34.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.<init>(<pastie>:94) at $line34.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw.<init>(<pastie>:96) at $line34.$read$$iw$$iw$$iw$$iw$$iw$$iw.<init>(<pastie>:98) at $line34.$read$$iw$$iw$$iw$$iw$$iw.<init>(<pastie>:100) at $line34.$read$$iw$$iw$$iw$$iw.<init>(<pastie>:102) at $line34.$read$$iw$$iw$$iw.<init>(<pastie>:104) at $line34.$read$$iw$$iw.<init>(<pastie>:106) at $line34.$read$$iw.<init>(<pastie>:108) at $line34.$read.<init>(<pastie>:110) at $line34.$read$.<init>(<pastie>:114) at $line34.$read$.<clinit>(<pastie>) at $line34.$eval$.$print$lzycompute(<pastie>:7) at $line34.$eval$.$print(<pastie>:6) at $line34.$eval.$print(<pastie>) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:793) at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1054) at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:645) at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:644) at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31) at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19) at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:644) at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:576) at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:572) at scala.tools.nsc.interpreter.ILoop$$anonfun$20.apply(ILoop.scala:762) at scala.tools.nsc.interpreter.ILoop$$anonfun$20.apply(ILoop.scala:762) at scala.tools.nsc.interpreter.IMain.withLabel(IMain.scala:116) at scala.tools.nsc.interpreter.ILoop.interpretCode$1(ILoop.scala:762) at scala.tools.nsc.interpreter.ILoop.pasteCommand(ILoop.scala:776) at scala.tools.nsc.interpreter.ILoop$$anonfun$standardCommands$9.apply(ILoop.scala:217) at scala.tools.nsc.interpreter.ILoop$$anonfun$standardCommands$9.apply(ILoop.scala:217) at scala.tools.nsc.interpreter.LoopCommands$LineCmd.apply(LoopCommands.scala:62) at scala.tools.nsc.interpreter.ILoop.colonCommand(ILoop.scala:698) at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:689) at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:404) at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:425) at org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply$mcZ$sp(SparkILoop.scala:285) at org.apache.spark.repl.SparkILoop.runClosure(SparkILoop.scala:159) at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:182) at org.apache.spark.repl.Main$.doMain(Main.scala:78) at org.apache.spark.repl.Main$.main(Main.scala:58) at org.apache.spark.repl.Main.main(Main.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:845) at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:161) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:920) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:929) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 25 (22 by maintainers)

Most upvoted comments

Shouldn’t have the deserialization process failed in case of an incomplete input?