You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Qihua Yang <ya...@gmail.com> on 2022/01/19 06:51:24 UTC
JDBC table reading
Hi,
I plan to use the database as flink source by using flink JDBC. I know
setNumPartitions can be used for parallelism in table reading.
If the num of task managers is less than numPartitions, what is the
behavior?
For example:
I config setNumPartitions(20), but flink application only has 10 task
managers. How flink assign tasks? Or that is totally wrong configuration?
num of tasks has to be larger than numPartitions?
Thanks,
Qihua
Re: JDBC table reading
Posted by Qihua Yang <ya...@gmail.com>.
Hi Caizhi,
Got it! Thanks for your clarification.
On Tue, Jan 18, 2022 at 11:45 PM Caizhi Weng <ts...@gmail.com> wrote:
> Hi!
>
> "But flink application only has 10 task managers". I assume that you're
> talking about task slots instead of task managers.
>
> If there are more partitions than task slots, each task slot may be
> assigned more than one partition. By default this is a first come first
> served assignment. After a task slot finishes its current partition, it
> will ask from the job manager for the next partition unless all partitions
> have been distributed.
>
> Qihua Yang <ya...@gmail.com> 于2022年1月19日周三 14:51写道：
>
>> Hi,
>>
>> I plan to use the database as flink source by using flink JDBC. I know
>> setNumPartitions can be used for parallelism in table reading.
>> If the num of task managers is less than numPartitions, what is the
>> behavior?
>> For example:
>> I config setNumPartitions(20), but flink application only has 10 task
>> managers. How flink assign tasks? Or that is totally wrong configuration?
>> num of tasks has to be larger than numPartitions?
>>
>> Thanks,
>> Qihua
>>
>
Re: JDBC table reading
Posted by Caizhi Weng <ts...@gmail.com>.
Hi!
"But flink application only has 10 task managers". I assume that you're
talking about task slots instead of task managers.
If there are more partitions than task slots, each task slot may be
assigned more than one partition. By default this is a first come first
served assignment. After a task slot finishes its current partition, it
will ask from the job manager for the next partition unless all partitions
have been distributed.
Qihua Yang <ya...@gmail.com> 于2022年1月19日周三 14:51写道：
> Hi,
>
> I plan to use the database as flink source by using flink JDBC. I know
> setNumPartitions can be used for parallelism in table reading.
> If the num of task managers is less than numPartitions, what is the
> behavior?
> For example:
> I config setNumPartitions(20), but flink application only has 10 task
> managers. How flink assign tasks? Or that is totally wrong configuration?
> num of tasks has to be larger than numPartitions?
>
> Thanks,
> Qihua
>