Objects with fields that are not serializable

[Dominik Wosiński <wo...@gmail.com>]

Hey,
I have a question about using classes with fields that are not serializable
in DataStream. Basically, I would like to use the  Java's Optional in
DataStream. So Say I have a class *Data *that has several optional fields
and I would like to have *DataStream<Data>*. I don't think this should
cause any issues, but I thought it can be good to ask whether this can
cause any issues with Flink Jobs.

Thanks,
Best,
Dom.

Re: Objects with fields that are not serializable

[Timo Walther <tw...@apache.org>]

Hi Dominik,

Flink does not use Java serialization logic for network communication. 
So objects must not implement `Serializable` for usage during runtime 
(DataStream<Data>).

Only if those classes are member variables of a Function like 
MapFunction, they need to serializable to ship the function code to the 
cluster.

You can check which serializer and nested serializers are used by calling:

dataStream.getType().createTypeSerializer(env.getExecutionConfig)

Regards,
Timo



On 14.04.20 12:55, Dominik Wosiński wrote:
> Hey,
> I have a question about using classes with fields that are not 
> serializable in DataStream. Basically, I would like to use the  Java's 
> Optional in DataStream. So Say I have a class /Data /that has several 
> optional fields and I would like to have /DataStream<Data>/. I don't 
> think this should cause any issues, but I thought it can be good to ask 
> whether this can cause any issues with Flink Jobs.
> 
> Thanks,
> Best,
> Dom.