You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by shashank agarwal <sh...@gmail.com> on 2018/01/17 09:25:59 UTC
Which collection to use in Scala case class
Hello,
A quick question which scala collection should I use in my scala case class
which won't go through generic serializer.
I was using java.utill.List in my scala case class before, Will this create
the problem in savepoint and restore. cause, my restore is not working so i
am trying to replace that collection with other scala collection.
Even PatternStream in CEP using Range which goes through generic serializer.
when I put "env.getConfig.disableGenericTypes" in my program CEP gives
error. After analyzing I found Range was used in PatternStream so it's
throwing an error.
--
Thanks Regards
SHASHANK AGARWAL
--- Trying to mobilize the things....
Re: Which collection to use in Scala case class
Posted by Timo Walther <tw...@apache.org>.
I filed a more specific issue for this:
https://issues.apache.org/jira/browse/FLINK-8451
Am 1/18/18 um 10:47 AM schrieb shashank agarwal:
>
> @Chesnay , @Timo, yes it's simple case class which i am using with
> java.util.List and one case class with Option and Seq. With CEP.
>
> I have filed Jira bugs also for that. I have put logs also there.
>
> https://issues.apache.org/jira/browse/FLINK-7760
>
> I have the issue with Rocksdb checkpointing also I have filed this bug
> also:
>
> https://issues.apache.org/jira/browse/FLINK-7756
>
>
>
>
>
>
>
> On Wed, Jan 17, 2018 at 2:55 PM, shashank agarwal
> <shashank734@gmail.com <ma...@gmail.com>> wrote:
>
> Hello,
>
> A quick question which scala collection should I use in my scala
> case class which won't go through generic serializer.
>
> I was using java.utill.List in my scala case class before, Will
> this create the problem in savepoint and restore. cause, my
> restore is not working so i am trying to replace that collection
> with other scala collection.
>
> Even PatternStream in CEP using Range which goes through generic
> serializer.
>
> when I put "env.getConfig.disableGenericTypes" in my program CEP
> gives error. After analyzing I found Range was used in
> PatternStream so it's throwing an error.
>
> --
> Thanks Regards
>
> SHASHANK AGARWAL
> --- Trying to mobilize the things....
>
>
>
>
>
>
>
>
>
> --
> Thanks Regards
>
> SHASHANK AGARWAL
> --- Trying to mobilize the things....
>
Re: Which collection to use in Scala case class
Posted by shashank agarwal <sh...@gmail.com>.
@Chesnay , @Timo, yes it's simple case class which i am using with
java.util.List
and one case class with Option and Seq. With CEP.
I have filed Jira bugs also for that. I have put logs also there.
https://issues.apache.org/jira/browse/FLINK-7760
I have the issue with Rocksdb checkpointing also I have filed this bug also:
https://issues.apache.org/jira/browse/FLINK-7756
On Wed, Jan 17, 2018 at 2:55 PM, shashank agarwal <sh...@gmail.com>
wrote:
> Hello,
>
> A quick question which scala collection should I use in my scala case
> class which won't go through generic serializer.
>
> I was using java.utill.List in my scala case class before, Will this
> create the problem in savepoint and restore. cause, my restore is not
> working so i am trying to replace that collection with other scala
> collection.
>
> Even PatternStream in CEP using Range which goes through generic
> serializer.
>
> when I put "env.getConfig.disableGenericTypes" in my program CEP gives
> error. After analyzing I found Range was used in PatternStream so it's
> throwing an error.
>
> --
> Thanks Regards
>
> SHASHANK AGARWAL
> --- Trying to mobilize the things....
>
>
>
>
>
>
--
Thanks Regards
SHASHANK AGARWAL
--- Trying to mobilize the things....
Re: Which collection to use in Scala case class
Posted by Timo Walther <tw...@apache.org>.
The issue of Scala case classes that can not be restored is a known
issue in Flink 1.4. We need to investigate if it can be fixed easily.
@Shashank: Could you give us a little reproducible example? Just a case
class with a java.util.List in it?
@Gordon: Is there a Jira issue for this? I might look into this in the
next days.
Regards,
Timo
Am 1/17/18 um 4:19 PM schrieb Chesnay Schepler:
> Can you show us the exception you get when restoring?
>
> Looping in Time who knows more about how types are
> analyzed/serializers are constructed.
>
> On 17.01.2018 10:25, shashank agarwal wrote:
>> Hello,
>>
>> A quick question which scala collection should I use in my scala case
>> class which won't go through generic serializer.
>>
>> I was using java.utill.List in my scala case class before, Will this
>> create the problem in savepoint and restore. cause, my restore is not
>> working so i am trying to replace that collection with other scala
>> collection.
>>
>> Even PatternStream in CEP using Range which goes through generic
>> serializer.
>>
>> when I put "env.getConfig.disableGenericTypes" in my program CEP
>> gives error. After analyzing I found Range was used in PatternStream
>> so it's throwing an error.
>>
>> --
>> Thanks Regards
>>
>> SHASHANK AGARWAL
>> --- Trying to mobilize the things....
>>
>>
>>
>>
>>
>
>
Re: Which collection to use in Scala case class
Posted by Chesnay Schepler <ch...@apache.org>.
Can you show us the exception you get when restoring?
Looping in Time who knows more about how types are analyzed/serializers
are constructed.
On 17.01.2018 10:25, shashank agarwal wrote:
> Hello,
>
> A quick question which scala collection should I use in my scala case
> class which won't go through generic serializer.
>
> I was using java.utill.List in my scala case class before, Will this
> create the problem in savepoint and restore. cause, my restore is not
> working so i am trying to replace that collection with other scala
> collection.
>
> Even PatternStream in CEP using Range which goes through generic
> serializer.
>
> when I put "env.getConfig.disableGenericTypes" in my program CEP
> gives error. After analyzing I found Range was used in PatternStream
> so it's throwing an error.
>
> --
> Thanks Regards
>
> SHASHANK AGARWAL
> --- Trying to mobilize the things....
>
>
>
>
>