You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Sudan S <su...@cred.club> on 2020/10/26 18:43:54 UTC
Getting UnsupportedException in Kyro for proto maps
Hi,
I have a proto which has a map field in it. When i try to use it in Flink
using Kyro serializer, im getting following error when pushing the output
to Kafka
Caused by: java.lang.UnsupportedOperationException
at java.util.Collections$UnmodifiableCollection.add(Collections.java:1057)
at
com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:134)
at
com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:40)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at
com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.read(DefaultArraySerializers.java:382)
at
com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.read(DefaultArraySerializers.java:303)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at
com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.read(DefaultArraySerializers.java:382)
at
com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.read(DefaultArraySerializers.java:303)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at
com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.read(DefaultArraySerializers.java:382)
at
com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.read(DefaultArraySerializers.java:303)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
Can u plz help on what is wrong?
--
*"The information contained in this e-mail and any accompanying documents
may contain information that is confidential or otherwise protected from
disclosure. If you are not the intended recipient of this message, or if
this message has been addressed to you in error, please immediately alert
the sender by replying to this e-mail and then delete this message,
including any attachments. Any dissemination, distribution or other use of
the contents of this message by anyone other than the intended recipient is
strictly prohibited. All messages sent to and from this e-mail address may
be monitored as permitted by applicable law and regulations to ensure
compliance with our internal policies and to protect our business."*
Re: Getting UnsupportedException in Kyro for proto maps
Posted by "Jaffe, Julian" <Ju...@activision.com>.
Are you registering the protobuf serializer with Kryo? (See https://flink.apache.org/news/2020/04/15/flink-serialization-tuning-vol-1.html#protobuf-via-kryo)
From: Sudan S <su...@cred.club>
Date: Monday, October 26, 2020 at 11:44 AM
To: User-Flink <us...@flink.apache.org>
Subject: Getting UnsupportedException in Kyro for proto maps
Hi,
I have a proto which has a map field in it. When i try to use it in Flink using Kyro serializer, im getting following error when pushing the output to Kafka
Caused by: java.lang.UnsupportedOperationException
at java.util.Collections$UnmodifiableCollection.add(Collections.java:1057)
at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:134)
at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:40)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.read(DefaultArraySerializers.java:382)
at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.read(DefaultArraySerializers.java:303)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.read(DefaultArraySerializers.java:382)
at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.read(DefaultArraySerializers.java:303)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.read(DefaultArraySerializers.java:382)
at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.read(DefaultArraySerializers.java:303)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782)
Can u plz help on what is wrong?
________________________________
"The information contained in this e-mail and any accompanying documents may contain information that is confidential or otherwise protected from disclosure. If you are not the intended recipient of this message, or if this message has been addressed to you in error, please immediately alert the sender by replying to this e-mail and then delete this message, including any attachments. Any dissemination, distribution or other use of the contents of this message by anyone other than the intended recipient is strictly prohibited. All messages sent to and from this e-mail address may be monitored as permitted by applicable law and regulations to ensure compliance with our internal policies and to protect our business."
________________________________