You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by forideal <fs...@163.com> on 2020/03/17 16:50:26 UTC

Flink SQL, How can i set parallelism in clause of group by ?

Hello everyone


     I'm a Flink SQL user. Now i have a question.How can i set parallelism in clause of group by.
     For example
     SELECT 
            T.user_id,
            D.user_name
     (SELECT 
             user_id,
             MIN(processtime)
     from my_table 
    group by user_id,TUMBLE(processtime, INTERVAL'15'SECOND)) AS T
     LEFTJOIN my_dim_table FOR SYSTEM_TIME ASOF T.proc_time AS D ON T.user_id = D.user_id
     
     If the partition of the topic is 3, then,the parallelism of the job is 3.
     In this example,there are three operator
         0ne is Source operator, parallelism is 3
         Two is GroupWindowAggregate operator,parallelism is 3
         Three is LookupJoin operator,parallelism is 3


      I want to change the parallelism of GroupWindowAggregate,but i can't.


Best wishes
forideal

Re: Flink SQL, How can i set parallelism in clause of group by ?

Posted by Benchao Li <li...@gmail.com>.
Hi forideal,

Currently, there is no way to change an operator's parallelism for SQL. All
operators use global parallelism as their parallelism, except for some
'global operator' which can only has parallelism of 1.

BTW, does changing the parallelism of all operators meets your need?

forideal <fs...@163.com> 于2020年3月18日周三 上午12:50写道:

> Hello everyone
>
>      I'm a Flink SQL user. Now i have a question.How can i set parallelism
> in clause of group by.
>      For example
>      SELECT
>             T.user_id,
>             D.user_name
>      (SELECT
>              user_id,
>              MIN(processtime)
>      from my_table
>     group by user_id,TUMBLE(processtime, INTERVAL '15' SECOND)) AS T
>      LEFT JOIN my_dim_table FOR SYSTEM_TIME AS OF T.proc_time AS D ON
> T.user_id = D.user_id
>
>      If the partition of the topic is 3, then,the parallelism of the job
> is 3.
>      In this example,there are three operator
>          0ne is Source operator, parallelism is 3
>          Two is GroupWindowAggregate operator,parallelism is 3
>          Three is LookupJoin operator,parallelism is 3
>
>       I want to change the parallelism of GroupWindowAggregate,but i
> can't.
>
> Best wishes
> forideal
>
>
>
>


-- 

Benchao Li
School of Electronics Engineering and Computer Science, Peking University
Tel:+86-15650713730
Email: libenchao@gmail.com; libenchao@pku.edu.cn