IndexOutOfBoundsException on deserialization after updating to 1.6.1

[Bruno Aranda <ba...@apache.org>]

Hi,

We are trying to update from 1.3.2 to 1.6.1, but one of our jobs keeps
throwing an exception during deserialization:

java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
    at java.util.ArrayList.rangeCheck(ArrayList.java:657)
    at java.util.ArrayList.get(ArrayList.java:433)
    at
com.esotericsoftware.kryo.util.MapReferenceResolver.getReadObject(MapReferenceResolver.java:42)
    at com.esotericsoftware.kryo.Kryo.readReferenceOrNull(Kryo.java:805)
    at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:655)
    at
org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.copy(KryoSerializer.java:231)
    at
org.apache.flink.api.scala.typeutils.OptionSerializer.copy(OptionSerializer.scala:51)
    at
org.apache.flink.api.scala.typeutils.OptionSerializer.copy(OptionSerializer.scala:29)
    at
org.apache.flink.api.scala.typeutils.CaseClassSerializer.copy(CaseClassSerializer.scala:101)
    at
org.apache.flink.api.scala.typeutils.CaseClassSerializer.copy(CaseClassSerializer.scala:32)
    at
org.apache.flink.api.scala.typeutils.TraversableSerializer$$anonfun$copy$1.apply(TraversableSerializer.scala:69)
    at
org.apache.flink.api.scala.typeutils.TraversableSerializer$$anonfun$copy$1.apply(TraversableSerializer.scala:69)
    at scala.collection.immutable.List.foreach(List.scala:392)
    at
org.apache.flink.api.scala.typeutils.TraversableSerializer.copy(TraversableSerializer.scala:69)
    at
org.apache.flink.api.scala.typeutils.TraversableSerializer.copy(TraversableSerializer.scala:33)
    at
org.apache.flink.api.scala.typeutils.CaseClassSerializer.copy(CaseClassSerializer.scala:101)
    at
org.apache.flink.api.scala.typeutils.CaseClassSerializer.copy(CaseClassSerializer.scala:32)
    at
org.apache.flink.runtime.state.heap.CopyOnWriteStateTable.get(CopyOnWriteStateTable.java:287)
    at
org.apache.flink.runtime.state.heap.CopyOnWriteStateTable.get(CopyOnWriteStateTable.java:311)
    at
org.apache.flink.runtime.state.heap.HeapValueState.value(HeapValueState.java:73)
    at
org.apache.flink.streaming.api.scala.function.StatefulFunction$class.applyWithState(StatefulFunction.scala:41)
...

We haven't changed the code for the job (apart from updating to 1.6.1), so
we are not sure what may have changed. This is caused in a RichMapFunction
that extends a StatefulFunction. The state is a case class, and we create
its state serializer with the following code:

override protected lazy val stateSerializer: TypeSerializer[ClipinState] =
  api.scala.createTypeInformation[MyCaseClass].createSerializer(getRuntimeContext.getExecutionConfig)

Any clues on what may be going on or where to look further? This was not an
issue on 1.3.2...

Thanks!

Bruno

Re: IndexOutOfBoundsException on deserialization after updating to 1.6.1

[Bruno Aranda <br...@gmail.com>]

Hi,

Thanks for your reply. We are still trying to isolate it, because this job
was using a more complex state. I think it is caused by a case class that
has an Option[MyOtherClass], and MyOtherClass is an enumerator, implemented
using the enumeratum library. I have changed that option to be just a
Option[Boolean], and the failure seems not to happen anymore.

We may continue with the Boolean for now, I guess.... though this was not a
problem in an earlier Flink version, possible Kryo change?

Cheers,

Bruno

On Wed, 17 Oct 2018 at 15:40 aitozi <gj...@gmail.com> wrote:

> Hi,Bruno Aranda
>
> Could you provide an complete example to reproduce the exception?
>
> Thanks,
> Aitozi
>
>
>
> --
> Sent from:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>

Re: IndexOutOfBoundsException on deserialization after updating to 1.6.1

[aitozi <gj...@gmail.com>]

Hi,Bruno Aranda

Could you provide an complete example to reproduce the exception?

Thanks,
Aitozi



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/