You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Xintong Song (Jira)" <ji...@apache.org> on 2023/03/23 08:52:20 UTC
[jira] [Updated] (FLINK-20036) Join Has NoUniqueKey when using mini-batch
[ https://issues.apache.org/jira/browse/FLINK-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xintong Song updated FLINK-20036:
---------------------------------
Fix Version/s: 1.18.0
(was: 1.17.0)
> Join Has NoUniqueKey when using mini-batch
> ------------------------------------------
>
> Key: FLINK-20036
> URL: https://issues.apache.org/jira/browse/FLINK-20036
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner
> Affects Versions: 1.11.2
> Reporter: Rex Remind
> Priority: Minor
> Labels: auto-deprioritized-major
> Fix For: 1.18.0
>
>
> Hello,
>
> We tried out mini-batch mode and our Join suddenly had NoUniqueKey.
> Join:
> {code:java}
> Table membershipsTable = tableEnv.from(SOURCE_MEMBERSHIPS)
> .renameColumns($("id").as("membership_id"))
> .select($("*")).join(usersTable, $("user_id").isEqual($("id")));
> {code}
> Mini-batch config:
> {code:java}
> configuration.setString("table.exec.mini-batch.enabled", "true"); // enable mini-batch optimization
> configuration.setString("table.exec.mini-batch.allow-latency", "5 s"); // use 5 seconds to buffer input records
> configuration.setString("table.exec.mini-batch.size", "5000"); // the maximum number of records can be buffered by each aggregate operator task
> {code}
>
> Join with mini-batch:
> {code:java}
> Join(joinType=[InnerJoin], where=[(user_id = id0)], select=[id,
> group_id, user_id, uuid, owner, id0, deleted_at],
> leftInputSpec=[NoUniqueKey], rightInputSpec=[NoUniqueKey])
> {code}
> Join without mini-batch:
> {code:java}
> Join(joinType=[InnerJoin], where=[(user_id = id0)], select=[id, group_id, user_id, uuid, owner, id0, deleted_at], leftInputSpec=[HasUniqueKey], rightInputSpec=[JoinKeyContainsUniqueKey])
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)