[jira] [Created] (FLINK-40796) State processor API no longer rejects key types that cannot be hashed reliably

["sepuri sai krishna (Jira)" <ji...@apache.org>]

sepuri sai krishna created FLINK-40796:
------------------------------------------

             Summary: State processor API no longer rejects key types that cannot be hashed reliably
                 Key: FLINK-40796
                 URL: https://issues.apache.org/jira/browse/FLINK-40796
             Project: Flink
          Issue Type: Bug
          Components: API / State Processor
    Affects Versions: 2.4.0
            Reporter: sepuri sai krishna


FLINK-40751 replaced `stream.keyBy(keySelector)` in `StateBootstrapTransformation` with a `PartitionTransformation` built directly, so the key type validation that `KeyedStream` performs is no longer reached. `OneInputStateTransformation#keyBy` only extracts the key type, so nothing validates it now.

Before, bootstrapping keyed state with an array key failed at build time:

{code}
InvalidProgramException: Type byte[] cannot be used as key. Contained UNSUPPORTED key types: byte[]. Look at the keyBy() documentation for the conditions a type has to satisfy in order to be eligible for a key.
{code}

On master the same job builds, runs and writes a savepoint. The same applies to an enum key and to a POJO that does not override `hashCode`.

The check comes from FLINK-5874, which restricted key types in the DataStream API because such a key cannot be hashed reliably. Without it the state is not partitioned consistently: two equal `byte[] {1, 2, 3}` keys can be assigned to different key groups (48 and 39 in a local run with `maxParallelism` 128). A job restoring the savepoint would reject the key type in its own `keyBy`, so the effect is that an up-front error is replaced by a savepoint that cannot be used.

Reported by Gyula Fora on https://github.com/apache/flink/pull/29268 as worth fixing with a test.




--
This message was sent by Atlassian Jira
(v8.20.10#820010)