[jira] [Comment Edited] (FLINK-28367) OffsetDateTime does not work with keyBy

["Lim Qing Wei (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-28367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17561943#comment-17561943 ] 

Lim Qing Wei edited comment on FLINK-28367 at 7/3/22 9:30 PM:
--------------------------------------------------------------

I had a look, I think part of it is that we are using a very old version of Kryo, this issue is fixed in the newer version.

 

One way to fix it easily is to add new kryo version to your deps, as documented here: [https://github.com/EsotericSoftware/kryo#with-maven]

If you're using sbt, just add the following to your deps should suffice, it fixes the issue for me (using your example, thanks for that)
{code:java}
"com.esotericsoftware" % "kryo" % "5.3.0"{code}
 

I think it might be a good idea to bump Flink's Kryo, but I dont know what it entails, is it going to be a pain to upgrade because we depends on many obselete API? Might be good to get contributor's input here, and have it tracked in dedicated ticket


was (Author: JIRAUSER289353):
I had a look, I think part of it is that we are using a very old version of Kryo, this issue is likely fixed in the newer version.

 

One way to fix it easily is to add new kryo version to your deps, as documented here: [https://github.com/EsotericSoftware/kryo#with-maven]

 

I think it might be a good idea to bump Flink's Kryo, but I dont know what it entails, as in is it going to be a pain to upgrade because we depends on a lot of obselete API? Might be good to get contributor's input here

> OffsetDateTime does not work with keyBy
> ---------------------------------------
>
>                 Key: FLINK-28367
>                 URL: https://issues.apache.org/jira/browse/FLINK-28367
>             Project: Flink
>          Issue Type: Bug
>          Components: API / DataStream, API / Type Serialization System
>    Affects Versions: 1.15.0
>         Environment: * Java 1.8 (openjdk 1.8.0_322)
>  * Scala 2.12.15
>  * Flink 1.15.0
>            Reporter: Henrik Laxhuber
>            Priority: Minor
>
> Using keyBy incorrectly (de-)serializes java.time.OffsetDateTime types - the offset gets lost and becomes null.
> Here's a minimal non-working example:
>  
> {code:java}
> import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment
> import java.time.OffsetDateTime
> object MWE {
>   def main(args: Array[String]): Unit = {
>     val env = StreamExecutionEnvironment.getExecutionEnvironment
>     env
>       .fromElements("2022-07-03T15:35:48.142Z", "2022-07-03T15:35:48.438Z")
>       .map(OffsetDateTime.parse(_))
>       .keyBy((t: OffsetDateTime) => t)
>       .print()
>     env.execute()
>   }
> } {code}
>  
> Expected Output:
> {code:java}
> 2022-07-03T15:35:48.438Z
> 2022-07-03T15:35:48.142Z{code}
> Actual Output:
> {code:java}
> 2022-07-03T15:35:48.438null
> 2022-07-03T15:35:48.142null{code}
> The issue arises whenever keyBy and OffsetDateTime are involved; I believe it could have something to do with the way that flink serializes the state.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)