You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by OskarM <pe...@gmail.com> on 2020/03/02 14:14:40 UTC

java.time.LocalDateTime in POJO type

Hi all,

I wanted to use LocalDateTime field in my POJO class used in Flink's
pipeline. However when I run the job I can see in the logs following
statements:

/class java.time.LocalDateTime does not contain a getter for field date
class java.time.LocalDateTime does not contain a setter for field date
Class class java.time.LocalDateTime cannot be used as a POJO type because
not all fields are valid POJO fields, and must be processed as GenericType.
Please read the Flink documentation on "Data Types & Serialization" for
details of the effect on performance./

I don't see any mention about my POJO type. Does it mean that my class is
still being handled by Flink's internal serializer instead of Kryo / other
fallback mechanism? 
Should I be concerned with those logs mentioned above?

Flink's version I use is: Apache Flink 1.10.0 for Scala 2.12, and I can see
TypeInformation's dedicated to java.time api in the
org.apache.flink.api.common.typeinfo.Types in the library itself.

Best regards,
Oskar



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: java.time.LocalDateTime in POJO type

Posted by Guowei Ma <gu...@gmail.com>.

Hi KristoffSC,

As far as I know, there is no simple API to let you directly use the
LocalTimeTypeInfo for LocalDataTime in your POJO class. (maybe other guys
know)

If the serializer/deserializer of LocalDataTime is very critical for you
there might be two methods.

1. Using the StreamExecutionEnvironment::registerTypeWithKryoSerializer to
register your own serializer/deserializer for the LocalDataTime.class
2. Register a new TypeInfoFactory for your Pojo. [1] This could reuse
the LocalTimeTypeInfo.

[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.10/dev/types_serialization.html#defining-type-information-using-a-factory

Best,
Guowei


KristoffSC <kr...@gmail.com> 于2020年3月5日周四 下午10:58写道：

> Thanks,
> do you have any example how I could use it?
>
> Basically I have a POJO class that has LocalDateTime filed in it.
>
>
>
> --
> Sent from:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>

Re: java.time.LocalDateTime in POJO type

Posted by KristoffSC <kr...@gmail.com>.

Thanks,
do you have any example how I could use it?

Basically I have a POJO class that has LocalDateTime filed in it.



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: java.time.LocalDateTime in POJO type

Posted by JingsongLee <lz...@aliyun.com>.

Hi,

I'v introduced LocalDateTime type information to flink-core.
But for compatibility reason, I revert the modification in TypeExtractor.
It seems that at present you can only use Types.LOCAL_DATE_TIME explicitly.

[1] http://jira.apache.org/jira/browse/FLINK-12850

Best,
Jingsong Lee


------------------------------------------------------------------
From:KristoffSC <kr...@gmail.com>
Send Time:2020年3月3日(星期二) 03:47
To:user <us...@flink.apache.org>
Subject:Re: java.time.LocalDateTime in POJO type

Hi Tzu-Li,
I think you misunderstood Oskar's question. 
The question was if there are there any plans to support Java's
LocalDateTime in Flink's "native" de/serialization mechanism. As we can read
in [1], for basic types, Flink supports all Java primitives and their boxed
form, plus void, String, Date, BigDecimal, and BigInteger.

So we have Java Date, the question is, will there be a support for
LocalDateTime? 

Thanks,
Krzysztof

[1]
https://ci.apache.org/projects/flink/flink-docs-stable/dev/types_serialization.html#flinks-typeinformation-class



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: java.time.LocalDateTime in POJO type

Posted by KristoffSC <kr...@gmail.com>.

Hi Tzu-Li,
I think you misunderstood Oskar's question. 
The question was if there are there any plans to support Java's
LocalDateTime in Flink's "native" de/serialization mechanism. As we can read
in [1], for basic types, Flink supports all Java primitives and their boxed
form, plus void, String, Date, BigDecimal, and BigInteger.

So we have Java Date, the question is, will there be a support for
LocalDateTime? 

Thanks,
Krzysztof

[1]
https://ci.apache.org/projects/flink/flink-docs-stable/dev/types_serialization.html#flinks-typeinformation-class



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: java.time.LocalDateTime in POJO type

Posted by "Tzu-Li (Gordon) Tai" <tz...@apache.org>.

Hi,

What that LOG means (i.e. "must be processed as a Generic Type") is that
Flink will have to fallback to using Kryo for the serialization for that
type.

You should be concerned about that if:
1) That type is being used for some persisted state in snapshots. That would
be the case if you've registered state of that type, or is used as the input
for some built-in operator that persists input records in state (e.g. window
operators). Kryo generally does not have a friendly schema evolution story,
so you would want to avoid that going into production.
2) Kryo itself is not the fastest compared to Flink's POJO serializer, so
that would be something to consider as well even if the type is only used
for transient, on-wire data.

I think in your case, since your POJO contains an inner field that cannot be
recognized as a POJO (i.e. the LocalDateTime), then your outer class is also
not recognized as a POJO.

BR,
Gordon



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/