You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Aiden (Jira)" <ji...@apache.org> on 2022/05/07 09:17:00 UTC
[jira] [Updated] (FLINK-27541) Support distribute by For FlinkSql
[ https://issues.apache.org/jira/browse/FLINK-27541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aiden updated FLINK-27541:
--------------------------
Description:
Now we cann't add a shuffle-operation in a sql-job.
Sometimes , for example, I have a kafka-source(three partitions) with parallelism three. And then I have a lookup-join function, I want process the data distribute by id so that the data can split into thre parallelism evenly (The source maybe slant seriously).
In DataStream API i can do it with keyby(), but it's so sad that i can do nothing when i use a sql;
Maybe we can do it like 'select id, f1,f2 from sourceTable distribute by id' like we do it in SparkSql.
was:
Now we cann't add a shuffle-operation in a sql-job.
Sometimes , for example, I have a kafka-source(three partitions) with parallelism three. And then I have a lookup-join function, I want process the data distribute by id so that the data can split into thre parallelism evenly (The source maybe slant seriously). In DataStream API i can do it with keyby(), but it's so sad that i can do nothing when i use a sql;
> Support distribute by For FlinkSql
> ----------------------------------
>
> Key: FLINK-27541
> URL: https://issues.apache.org/jira/browse/FLINK-27541
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner
> Reporter: Aiden
> Priority: Minor
>
> Now we cann't add a shuffle-operation in a sql-job.
> Sometimes , for example, I have a kafka-source(three partitions) with parallelism three. And then I have a lookup-join function, I want process the data distribute by id so that the data can split into thre parallelism evenly (The source maybe slant seriously).
> In DataStream API i can do it with keyby(), but it's so sad that i can do nothing when i use a sql;
> Maybe we can do it like 'select id, f1,f2 from sourceTable distribute by id' like we do it in SparkSql.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)