You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by kb <Ke...@comcast.com> on 2019/04/09 16:11:35 UTC
Details of the downsides of “falling back” to Kyro rather than using Flink’s built in serde
Hi all,
I was looking at
https://ci.apache.org/projects/flink/flink-docs-stable/dev/types_serialization.html,
and https://www.youtube.com/watch?v=euFMWFDThiE to improve our state
management and back pressure.
Both of these resources mention ensuring objects used in the flow are valid
POJOs to avoid “falling back” to Kyro.
Are the specific downsides listed anywhere?
Thanks
kb
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: Details of the downsides of “falling back” to Kyro rather than using Flink’s built in serde
Posted by kb <Ke...@comcast.com>.
Hi Konstantin,
Thank you! This is exactly what I was looking for.
Thanks
Kevin
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: Details of the downsides of “falling back” to Kyro rather than using Flink’s built in serde
Posted by Konstantin Knauf <ko...@ververica.com>.
Hi Kevin,
there are no performance downsides to using Flink POJOs. You are just
limited in the types you can use (e.g. you can not use Collections).
In Flink 1.7 you might want to use Avro (SpecificRecord) for your state
objects to benefit from Flink's built-in state schema evolution
capabilities. As of Flink 1.8, state schema evolution also works
out-of-the-box with Flink POJOs. So, this potential downside does not exist
going forward.
[1] contains a very basic benchmark comparing different serializers in
Flink. The results are not representative and the absolute values don't
have any meaning, but it should give you an idea of the potential
differences.
Hope this helps,
Konstantin
[1]
https://github.com/ververica/flink-training-troubleshooting/blob/master/slides/2019-04-01%20_%20OPERATIONS%20_%206%20SERIALIZATION.pdf
On Tue, Apr 9, 2019 at 6:11 PM kb <Ke...@comcast.com> wrote:
> Hi all,
>
> I was looking at
>
> https://ci.apache.org/projects/flink/flink-docs-stable/dev/types_serialization.html
> ,
> and https://www.youtube.com/watch?v=euFMWFDThiE to improve our state
> management and back pressure.
>
> Both of these resources mention ensuring objects used in the flow are valid
> POJOs to avoid “falling back” to Kyro.
>
> Are the specific downsides listed anywhere?
>
> Thanks
> kb
>
>
>
>
> --
> Sent from:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>
--
Konstantin Knauf | Solutions Architect
+49 160 91394525
<https://www.ververica.com/>
Follow us @VervericaData
--
Join Flink Forward <https://flink-forward.org/> - The Apache Flink
Conference
Stream Processing | Event Driven | Real Time
--
Data Artisans GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
--
Data Artisans GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen