[jira] [Closed] (FLINK-7002) Partitioning broken if enum is used in compound key specified using field expression

["Nico Kruber (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-7002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Nico Kruber closed FLINK-7002.
------------------------------
    Resolution: Won't Fix

Actually, this is not a Flink issue, but an issue of enums in Java and their implementation of {{hashCode}} which relies on the enum instance's memory address and therefore may be different in each JVM.

You could instead use the enum's ordinal or its name in the key selector implementation.

Please also refer to this for some more info:
https://stackoverflow.com/questions/49140654/flink-error-key-group-is-not-in-keygrouprange

> Partitioning broken if enum is used in compound key specified using field expression
> ------------------------------------------------------------------------------------
>
>                 Key: FLINK-7002
>                 URL: https://issues.apache.org/jira/browse/FLINK-7002
>             Project: Flink
>          Issue Type: Bug
>          Components: API / Type Serialization System
>    Affects Versions: 1.2.0, 1.3.1
>            Reporter: Sebastian Klemke
>            Priority: Major
>         Attachments: TestJob.java, WorkingTestJob.java, testdata.avro
>
>
> When groupBy() or keyBy() is used with multiple field expressions, at least one of them being an enum type serialized using EnumTypeInfo, partitioning seems random, resulting in incorrectly grouped/keyed output datasets/datastreams.
> The attached Flink DataSet API jobs and the test dataset detail the issue: Both jobs count (id, type) occurrences, TestJob uses field expressions to group, WorkingTestJob uses a KeySelector function.
> Expected output for both is 6 records, with frequency value 100_000 each. If you run in LocalEnvironment, results are in fact equivalent. But when run on a cluster with 5 TaskManagers, only KeySelector function with String key produces correct results whereas field expressions produce random, non-repeatable, wrong results.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)