[jira] [Updated] (FLINK-40796) State processor API no longer rejects key types that cannot be hashed reliably

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-40796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated FLINK-40796:
-----------------------------------
    Labels: pull-request-available  (was: )

> State processor API no longer rejects key types that cannot be hashed reliably
> ------------------------------------------------------------------------------
>
>                 Key: FLINK-40796
>                 URL: https://issues.apache.org/jira/browse/FLINK-40796
>             Project: Flink
>          Issue Type: Bug
>          Components: API / State Processor
>    Affects Versions: 2.4.0
>            Reporter: sepuri sai krishna
>            Priority: Major
>              Labels: pull-request-available
>
> FLINK-40751 replaced `stream.keyBy(keySelector)` in `StateBootstrapTransformation` with a `PartitionTransformation` built directly, so the key type validation that `KeyedStream` performs is no longer reached. `OneInputStateTransformation#keyBy` only extracts the key type, so nothing validates it now.
> Before, bootstrapping keyed state with an array key failed at build time:
> {code}
> InvalidProgramException: Type byte[] cannot be used as key. Contained UNSUPPORTED key types: byte[]. Look at the keyBy() documentation for the conditions a type has to satisfy in order to be eligible for a key.
> {code}
> On master the same job builds, runs and writes a savepoint. The same applies to an enum key and to a POJO that does not override `hashCode`.
> The check comes from FLINK-5874, which restricted key types in the DataStream API because such a key cannot be hashed reliably. Without it the state is not partitioned consistently: two equal `byte[] {1, 2, 3}` keys can be assigned to different key groups (48 and 39 in a local run with `maxParallelism` 128). A job restoring the savepoint would reject the key type in its own `keyBy`, so the effect is that an up-front error is replaced by a savepoint that cannot be used.
> Reported by Gyula Fora on https://github.com/apache/flink/pull/29268 as worth fixing with a test.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)