You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by srikanth flink <fl...@gmail.com> on 2019/11/06 14:03:25 UTC
Job Distribution Strategy On Cluster.
Hi there,
I'm running Flink with 3 node cluster.
While running my jobs(both SQL client and jar submission), the jobs are
being assigned to single machine instead of distribution among the cluster.
How could I achieve the job distribution to make use of the computation
power?
Thanks
Srikanth
Re: Job Distribution Strategy On Cluster.
Posted by srikanth flink <fl...@gmail.com>.
Great, thanks for the update.
On Tue, Nov 12, 2019 at 8:51 AM Zhu Zhu <re...@gmail.com> wrote:
> There is no plan for release 1.9.2 yet.
> Flink 1.10.0 is planned to be released in early January.
>
> Thanks,
> Zhu Zhu
>
> srikanth flink <fl...@gmail.com> 于2019年11月11日周一 下午9:53写道：
>
>> Zhu Zhu,
>>
>> That's awesome and is what I'm looking for.
>> Any update on when would be the next release date?
>>
>> Thanks
>> Srikanth
>>
>> On Mon, Nov 11, 2019 at 3:40 PM Zhu Zhu <re...@gmail.com> wrote:
>>
>>> Hi Srikanth,
>>>
>>> Is this issue what you encounter? FLINK-12122: a job would tend to fill
>>> one TM before using another.
>>> If it is, you may need to wait for the release 1.9.2 or 1.10, since it
>>> is just fixed.
>>>
>>> Thanks,
>>> Zhu Zhu
>>>
>>> vino yang <ya...@gmail.com> 于2019年11月11日周一 下午5:48写道：
>>>
>>>> Hi srikanth,
>>>>
>>>> What's your job's parallelism?
>>>>
>>>> In some scenes, many operators are chained with each other. if it's
>>>> parallelism is 1, it would just use a single slot.
>>>>
>>>> Best,
>>>> Vino
>>>>
>>>> srikanth flink <fl...@gmail.com> 于2019年11月6日周三 下午10:03写道：
>>>>
>>>>> Hi there,
>>>>>
>>>>> I'm running Flink with 3 node cluster.
>>>>> While running my jobs(both SQL client and jar submission), the jobs
>>>>> are being assigned to single machine instead of distribution among the
>>>>> cluster. How could I achieve the job distribution to make use of the
>>>>> computation power?
>>>>>
>>>>> Thanks
>>>>> Srikanth
>>>>>
>>>>
Re: Job Distribution Strategy On Cluster.
Posted by Zhu Zhu <re...@gmail.com>.
There is no plan for release 1.9.2 yet.
Flink 1.10.0 is planned to be released in early January.
Thanks,
Zhu Zhu
srikanth flink <fl...@gmail.com> 于2019年11月11日周一 下午9:53写道：
> Zhu Zhu,
>
> That's awesome and is what I'm looking for.
> Any update on when would be the next release date?
>
> Thanks
> Srikanth
>
> On Mon, Nov 11, 2019 at 3:40 PM Zhu Zhu <re...@gmail.com> wrote:
>
>> Hi Srikanth,
>>
>> Is this issue what you encounter? FLINK-12122: a job would tend to fill
>> one TM before using another.
>> If it is, you may need to wait for the release 1.9.2 or 1.10, since it is
>> just fixed.
>>
>> Thanks,
>> Zhu Zhu
>>
>> vino yang <ya...@gmail.com> 于2019年11月11日周一 下午5:48写道：
>>
>>> Hi srikanth,
>>>
>>> What's your job's parallelism?
>>>
>>> In some scenes, many operators are chained with each other. if it's
>>> parallelism is 1, it would just use a single slot.
>>>
>>> Best,
>>> Vino
>>>
>>> srikanth flink <fl...@gmail.com> 于2019年11月6日周三 下午10:03写道：
>>>
>>>> Hi there,
>>>>
>>>> I'm running Flink with 3 node cluster.
>>>> While running my jobs(both SQL client and jar submission), the jobs are
>>>> being assigned to single machine instead of distribution among the cluster.
>>>> How could I achieve the job distribution to make use of the computation
>>>> power?
>>>>
>>>> Thanks
>>>> Srikanth
>>>>
>>>
Re: Job Distribution Strategy On Cluster.
Posted by srikanth flink <fl...@gmail.com>.
Zhu Zhu,
That's awesome and is what I'm looking for.
Any update on when would be the next release date?
Thanks
Srikanth
On Mon, Nov 11, 2019 at 3:40 PM Zhu Zhu <re...@gmail.com> wrote:
> Hi Srikanth,
>
> Is this issue what you encounter? FLINK-12122: a job would tend to fill
> one TM before using another.
> If it is, you may need to wait for the release 1.9.2 or 1.10, since it is
> just fixed.
>
> Thanks,
> Zhu Zhu
>
> vino yang <ya...@gmail.com> 于2019年11月11日周一 下午5:48写道：
>
>> Hi srikanth,
>>
>> What's your job's parallelism?
>>
>> In some scenes, many operators are chained with each other. if it's
>> parallelism is 1, it would just use a single slot.
>>
>> Best,
>> Vino
>>
>> srikanth flink <fl...@gmail.com> 于2019年11月6日周三 下午10:03写道：
>>
>>> Hi there,
>>>
>>> I'm running Flink with 3 node cluster.
>>> While running my jobs(both SQL client and jar submission), the jobs are
>>> being assigned to single machine instead of distribution among the cluster.
>>> How could I achieve the job distribution to make use of the computation
>>> power?
>>>
>>> Thanks
>>> Srikanth
>>>
>>
Re: Job Distribution Strategy On Cluster.
Posted by Zhu Zhu <re...@gmail.com>.
Hi Srikanth,
Is this issue what you encounter? FLINK-12122: a job would tend to fill one
TM before using another.
If it is, you may need to wait for the release 1.9.2 or 1.10, since it is
just fixed.
Thanks,
Zhu Zhu
vino yang <ya...@gmail.com> 于2019年11月11日周一 下午5:48写道：
> Hi srikanth,
>
> What's your job's parallelism?
>
> In some scenes, many operators are chained with each other. if it's
> parallelism is 1, it would just use a single slot.
>
> Best,
> Vino
>
> srikanth flink <fl...@gmail.com> 于2019年11月6日周三 下午10:03写道：
>
>> Hi there,
>>
>> I'm running Flink with 3 node cluster.
>> While running my jobs(both SQL client and jar submission), the jobs are
>> being assigned to single machine instead of distribution among the cluster.
>> How could I achieve the job distribution to make use of the computation
>> power?
>>
>> Thanks
>> Srikanth
>>
>
Re: Job Distribution Strategy On Cluster.
Posted by srikanth flink <fl...@gmail.com>.
Vina,
I've set parallelism as 6 while max parallelism is 128.
Thanks
Srikanth
On Mon, Nov 11, 2019 at 3:18 PM vino yang <ya...@gmail.com> wrote:
> Hi srikanth,
>
> What's your job's parallelism?
>
> In some scenes, many operators are chained with each other. if it's
> parallelism is 1, it would just use a single slot.
>
> Best,
> Vino
>
> srikanth flink <fl...@gmail.com> 于2019年11月6日周三 下午10:03写道：
>
>> Hi there,
>>
>> I'm running Flink with 3 node cluster.
>> While running my jobs(both SQL client and jar submission), the jobs are
>> being assigned to single machine instead of distribution among the cluster.
>> How could I achieve the job distribution to make use of the computation
>> power?
>>
>> Thanks
>> Srikanth
>>
>
Re: Job Distribution Strategy On Cluster.
Posted by vino yang <ya...@gmail.com>.
Hi srikanth,
What's your job's parallelism?
In some scenes, many operators are chained with each other. if it's
parallelism is 1, it would just use a single slot.
Best,
Vino
srikanth flink <fl...@gmail.com> 于2019年11月6日周三 下午10:03写道：
> Hi there,
>
> I'm running Flink with 3 node cluster.
> While running my jobs(both SQL client and jar submission), the jobs are
> being assigned to single machine instead of distribution among the cluster.
> How could I achieve the job distribution to make use of the computation
> power?
>
> Thanks
> Srikanth
>