You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Sucheth S <su...@gmail.com> on 2022/10/11 07:29:03 UTC
Flink falls back on to kryo serializer for GenericTypes
Hello,
How to avoid flink's kryo serializer for GenericTypes ? Kryo is having some
performance issues.
Tried below but no luck.
env.getConfig().disableForceKryo();
env.getConfig().enableForceAvro();
Tried this - env.getConfig().disableGenericTypes();
getting - Generic types have been disabled in the ExecutionConfig and
type org.apache.avro.generic.GenericRecord is treated as a generic
type
Regards,
Sucheth Shivakumar
website : https://sucheths.com
mobile : +1(650)-576-8050
San Mateo, United States
Re: Flink falls back on to kryo serializer for GenericTypes
Posted by Hangxiang Yu <ma...@gmail.com>.
Hi Sucheth.
It's related to how you defined your GenericTypes.
You may still need to give some hints to flink if you are using complicated
generic types so what you tried may not be enough.
Could you share your generic type object ?
BTW, Maybe you could refer to [1] which I think it's similar to your
problem.
[1] https://issues.apache.org/jira/browse/FLINK-28653
On Tue, Oct 11, 2022 at 3:31 PM Sucheth S <su...@gmail.com> wrote:
> Hello,
>
> How to avoid flink's kryo serializer for GenericTypes ? Kryo is having
> some performance issues.
>
> Tried below but no luck.
>
> env.getConfig().disableForceKryo();
> env.getConfig().enableForceAvro();
>
> Tried this - env.getConfig().disableGenericTypes();
>
> getting - Generic types have been disabled in the ExecutionConfig and type org.apache.avro.generic.GenericRecord is treated as a generic type
>
>
>
> Regards,
> Sucheth Shivakumar
> website : https://sucheths.com
> mobile : +1(650)-576-8050
> San Mateo, United States
>
--
Best,
Hangxiang.
Re: Flink falls back on to kryo serializer for GenericTypes
Posted by Chesnay Schepler <ch...@apache.org>.
There's no alternative to Kryo for generic types, apart from
implementing your Flink serializer (but technically at that point the
type is no longer treated as a generic type).
enableForAvro only forces Avro to be used for POJO types.
On 11/10/2022 09:29, Sucheth S wrote:
> Hello,
>
> How to avoid flink's kryo serializer for GenericTypes ? Kryo is having
> some performance issues.
>
> Tried below but no luck.
> env.getConfig().disableForceKryo();
> env.getConfig().enableForceAvro();
> Tried this - env.getConfig().disableGenericTypes();
> getting - Generic types have been disabled in the ExecutionConfig and type org.apache.avro.generic.GenericRecord is treated as a generic type
>
>
> Regards,
> Sucheth Shivakumar
> website: https://sucheths.com
> mobile : +1(650)-576-8050
> San Mateo, United States
Re: Flink falls back on to kryo serializer for GenericTypes
Posted by Chesnay Schepler <ch...@apache.org>.
There's no alternative to Kryo for generic types, apart from
implementing your Flink serializer (but technically at that point the
type is no longer treated as a generic type).
enableForAvro only forces Avro to be used for POJO types.
On 11/10/2022 09:29, Sucheth S wrote:
> Hello,
>
> How to avoid flink's kryo serializer for GenericTypes ? Kryo is having
> some performance issues.
>
> Tried below but no luck.
> env.getConfig().disableForceKryo();
> env.getConfig().enableForceAvro();
> Tried this - env.getConfig().disableGenericTypes();
> getting - Generic types have been disabled in the ExecutionConfig and type org.apache.avro.generic.GenericRecord is treated as a generic type
>
>
> Regards,
> Sucheth Shivakumar
> website: https://sucheths.com
> mobile : +1(650)-576-8050
> San Mateo, United States