You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by forideal <fs...@163.com> on 2020/03/17 16:50:26 UTC
Flink SQL, How can i set parallelism in clause of group by ?
Hello everyone
I'm a Flink SQL user. Now i have a question.How can i set parallelism in clause of group by.
For example
SELECT
T.user_id,
D.user_name
(SELECT
user_id,
MIN(processtime)
from my_table
group by user_id,TUMBLE(processtime, INTERVAL'15'SECOND)) AS T
LEFTJOIN my_dim_table FOR SYSTEM_TIME ASOF T.proc_time AS D ON T.user_id = D.user_id
If the partition of the topic is 3, then,the parallelism of the job is 3.
In this example,there are three operator
0ne is Source operator, parallelism is 3
Two is GroupWindowAggregate operator,parallelism is 3
Three is LookupJoin operator,parallelism is 3
I want to change the parallelism of GroupWindowAggregate,but i can't.
Best wishes
forideal
Re: Flink SQL, How can i set parallelism in clause of group by ?
Posted by Benchao Li <li...@gmail.com>.
Hi forideal,
Currently, there is no way to change an operator's parallelism for SQL. All
operators use global parallelism as their parallelism, except for some
'global operator' which can only has parallelism of 1.
BTW, does changing the parallelism of all operators meets your need?
forideal <fs...@163.com> 于2020年3月18日周三 上午12:50写道:
> Hello everyone
>
> I'm a Flink SQL user. Now i have a question.How can i set parallelism
> in clause of group by.
> For example
> SELECT
> T.user_id,
> D.user_name
> (SELECT
> user_id,
> MIN(processtime)
> from my_table
> group by user_id,TUMBLE(processtime, INTERVAL '15' SECOND)) AS T
> LEFT JOIN my_dim_table FOR SYSTEM_TIME AS OF T.proc_time AS D ON
> T.user_id = D.user_id
>
> If the partition of the topic is 3, then,the parallelism of the job
> is 3.
> In this example,there are three operator
> 0ne is Source operator, parallelism is 3
> Two is GroupWindowAggregate operator,parallelism is 3
> Three is LookupJoin operator,parallelism is 3
>
> I want to change the parallelism of GroupWindowAggregate,but i
> can't.
>
> Best wishes
> forideal
>
>
>
>
--
Benchao Li
School of Electronics Engineering and Computer Science, Peking University
Tel:+86-15650713730
Email: libenchao@gmail.com; libenchao@pku.edu.cn