You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Aiden (Jira)" <ji...@apache.org> on 2022/05/07 09:17:00 UTC

[jira] [Updated] (FLINK-27541) Support distribute by For FlinkSql

     [ https://issues.apache.org/jira/browse/FLINK-27541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Aiden updated FLINK-27541:
--------------------------
    Description: 
Now we cann't add a shuffle-operation in a sql-job.
Sometimes , for example,  I have a kafka-source(three partitions) with parallelism three. And then I have a lookup-join function, I want process the data distribute by id so that the data can split into thre parallelism evenly (The source maybe slant seriously).  
In DataStream API i can do it with keyby(),  but it's so sad that i can  do nothing when i use a sql;
Maybe we can do it like 'select id, f1,f2 from sourceTable distribute by id' like we do it in SparkSql.




  was:
Now we cann't add a shuffle-operation in a sql-job.
Sometimes , for example,  I have a kafka-source(three partitions) with parallelism three. And then I have a lookup-join function, I want process the data distribute by id so that the data can split into thre parallelism evenly (The source maybe slant seriously).  In DataStream API i can do it with keyby(),  but it's so sad that i can  do nothing when i use a sql;


> Support distribute by For FlinkSql
> ----------------------------------
>
>                 Key: FLINK-27541
>                 URL: https://issues.apache.org/jira/browse/FLINK-27541
>             Project: Flink
>          Issue Type: Improvement
>          Components: Table SQL / Planner
>            Reporter: Aiden
>            Priority: Minor
>
> Now we cann't add a shuffle-operation in a sql-job.
> Sometimes , for example,  I have a kafka-source(three partitions) with parallelism three. And then I have a lookup-join function, I want process the data distribute by id so that the data can split into thre parallelism evenly (The source maybe slant seriously).  
> In DataStream API i can do it with keyby(),  but it's so sad that i can  do nothing when i use a sql;
> Maybe we can do it like 'select id, f1,f2 from sourceTable distribute by id' like we do it in SparkSql.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)