You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by 范超 <fa...@mgtv.com> on 2020/08/18 02:24:04 UTC
答复: How to specify the number of TaskManagers in Yarn Cluster using Per-Job Mode
Thanks Yangze
All 3 machines NodeManager is started.
I just don't know why not three machines each running a Flink TaskManager and how to achieve this
-----邮件原件-----
发件人: Yangze Guo [mailto:karmagyz@gmail.com]
发送时间: 2020年8月18日 星期二 10:10
收件人: 范超 <fa...@mgtv.com>
抄送: user (user@flink.apache.org) <us...@flink.apache.org>
主题: Re: How to specify the number of TaskManagers in Yarn Cluster using Per-Job Mode
Hi,
Do you start the NodeManager in all the three machines? If so, could you check all the NMs correctly connect to the ResourceManager?
Best,
Yangze Guo
On Tue, Aug 18, 2020 at 10:01 AM 范超 <fa...@mgtv.com> wrote:
>
> Hi, Dev and Users
> I’ve 3 machines each one is 8 cores and 16GB memory.
> Following it’s my Resource Manager screenshot the cluster have 36GB total.
> I specify the paralism to 3 or even up to 12, But the task manager is always running on two nodes not all three machine, the third node does not start the task manager.
> I tried set the –p –tm –jm parameters, but it always the same, only different is more container on the two maching but not all three machine start the task manager.
> My question is how to set the cli parameter to start all of my three
> machine (all task manager start on 3 machines)
>
> Thanks a lot
> [cid:image001.png@01D67546.62291B70]
>
>
> Chao fan
>
答复: How to specify the number of TaskManagers in Yarn Cluster using Per-Job Mode
Posted by 范超 <fa...@mgtv.com>.
Thanks Yangze for providing these links I'll try it !
-----邮件原件-----
发件人: Yangze Guo [mailto:karmagyz@gmail.com]
发送时间: 2020年8月18日 星期二 12:57
收件人: 范超 <fa...@mgtv.com>
抄送: user (user@flink.apache.org) <us...@flink.apache.org>
主题: Re: How to specify the number of TaskManagers in Yarn Cluster using Per-Job Mode
The number of TM mainly depends on the parallelism and job graph.
Flink now allows you to set the maximum slots number (slotmanager-number-of-slots-max[1]). There is also a plan to support setting the minimum number of slots[2].
[1] https://ci.apache.org/projects/flink/flink-docs-master/ops/config.html#slotmanager-number-of-slots-max
[2] https://issues.apache.org/jira/browse/FLINK-15959
Best,
Yangze Guo
On Tue, Aug 18, 2020 at 12:21 PM 范超 <fa...@mgtv.com> wrote:
>
> Thanks Yangze
>
> 1. Do you meet any problem when deploying on Yarn or running Flink job?
> My job works well
>
> 2. Why do you need to start the TMs on all the three machines?
> From cluster perspective, I wonder if the process pressure can be balance to 3 machines.
>
> 3. Flink can control how many TM to start, but where to start the TMs depends on Yarn.
> Yes, the job where to start the TM is depend on Yarn.
> Could you please tell me parameter controls how many TM to start, the
> yn parameter is delete from 1.10 as the 1.9 doc sample list[1] below
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-release-1.9/ops/cli.ht
> ml
>
> Run example program using a per-job YARN cluster with 2 TaskManagers:
>
> ./bin/flink run -m yarn-cluster -yn 2 \
> ./examples/batch/WordCount.jar \
> --input hdfs:///user/hamlet.txt --output
> hdfs:///user/wordcount_out
>
> -----邮件原件-----
> 发件人: Yangze Guo [mailto:karmagyz@gmail.com]
> 发送时间: 2020年8月18日 星期二 11:31
> 收件人: 范超 <fa...@mgtv.com>
> 抄送: user (user@flink.apache.org) <us...@flink.apache.org>
> 主题: Re: How to specify the number of TaskManagers in Yarn Cluster
> using Per-Job Mode
>
> Hi,
>
> Flink can control how many TM to start, but where to start the TMs depends on Yarn.
>
> Do you meet any problem when deploying on Yarn or running Flink job?
> Why do you need to start the TMs on all the three machines?
>
> Best,
> Yangze Guo
>
> On Tue, Aug 18, 2020 at 11:25 AM 范超 <fa...@mgtv.com> wrote:
> >
> > Thanks Yangze
> > The reason why I don’t deploying a standalone cluster, it's because there kafka, kudu, hadoop, zookeeper on these machines, maybe currently using the yarn to manage resources is the best choice for me.
> > If Flink can not control how many tm to start , could anyone
> > providing me some best practice for deploying on yarn please? I read
> > the [1] and still don't very clear
> >
> > [1]
> > https://www.ververica.com/blog/how-to-size-your-apache-flink-cluster
> > -g
> > eneral-guidelines
> >
> > -----邮件原件-----
> > 发件人: Yangze Guo [mailto:karmagyz@gmail.com]
> > 发送时间: 2020年8月18日 星期二 10:50
> > 收件人: 范超 <fa...@mgtv.com>
> > 抄送: user (user@flink.apache.org) <us...@flink.apache.org>
> > 主题: Re: How to specify the number of TaskManagers in Yarn Cluster
> > using Per-Job Mode
> >
> > Hi,
> >
> > I think that is only related to the Yarn scheduling strategy. AFAIK, Flink could not control it. You could check the RM log to figure out why it did not schedule the containers to all the three machines. BTW, if you have specific requirements to start with all the three machines, how about deploying a standalone cluster instead?
> >
> > Best,
> > Yangze Guo
> >
> > On Tue, Aug 18, 2020 at 10:24 AM 范超 <fa...@mgtv.com> wrote:
> > >
> > > Thanks Yangze
> > >
> > > All 3 machines NodeManager is started.
> > >
> > > I just don't know why not three machines each running a Flink
> > > TaskManager and how to achieve this
> > >
> > > -----邮件原件-----
> > > 发件人: Yangze Guo [mailto:karmagyz@gmail.com]
> > > 发送时间: 2020年8月18日 星期二 10:10
> > > 收件人: 范超 <fa...@mgtv.com>
> > > 抄送: user (user@flink.apache.org) <us...@flink.apache.org>
> > > 主题: Re: How to specify the number of TaskManagers in Yarn Cluster
> > > using Per-Job Mode
> > >
> > > Hi,
> > >
> > > Do you start the NodeManager in all the three machines? If so, could you check all the NMs correctly connect to the ResourceManager?
> > >
> > > Best,
> > > Yangze Guo
> > >
> > > On Tue, Aug 18, 2020 at 10:01 AM 范超 <fa...@mgtv.com> wrote:
> > > >
> > > > Hi, Dev and Users
> > > > I’ve 3 machines each one is 8 cores and 16GB memory.
> > > > Following it’s my Resource Manager screenshot the cluster have 36GB total.
> > > > I specify the paralism to 3 or even up to 12, But the task manager is always running on two nodes not all three machine, the third node does not start the task manager.
> > > > I tried set the –p –tm –jm parameters, but it always the same, only different is more container on the two maching but not all three machine start the task manager.
> > > > My question is how to set the cli parameter to start all of my
> > > > three machine (all task manager start on 3 machines)
> > > >
> > > > Thanks a lot
> > > > [cid:image001.png@01D67546.62291B70]
> > > >
> > > >
> > > > Chao fan
> > > >
Re: How to specify the number of TaskManagers in Yarn Cluster using
Per-Job Mode
Posted by Yangze Guo <ka...@gmail.com>.
The number of TM mainly depends on the parallelism and job graph.
Flink now allows you to set the maximum slots number
(slotmanager-number-of-slots-max[1]). There is also a plan to support
setting the minimum number of slots[2].
[1] https://ci.apache.org/projects/flink/flink-docs-master/ops/config.html#slotmanager-number-of-slots-max
[2] https://issues.apache.org/jira/browse/FLINK-15959
Best,
Yangze Guo
On Tue, Aug 18, 2020 at 12:21 PM 范超 <fa...@mgtv.com> wrote:
>
> Thanks Yangze
>
> 1. Do you meet any problem when deploying on Yarn or running Flink job?
> My job works well
>
> 2. Why do you need to start the TMs on all the three machines?
> From cluster perspective, I wonder if the process pressure can be balance to 3 machines.
>
> 3. Flink can control how many TM to start, but where to start the TMs depends on Yarn.
> Yes, the job where to start the TM is depend on Yarn.
> Could you please tell me parameter controls how many TM to start, the yn parameter is delete from 1.10 as the 1.9 doc sample list[1] below
>
> [1] https://ci.apache.org/projects/flink/flink-docs-release-1.9/ops/cli.html
>
> Run example program using a per-job YARN cluster with 2 TaskManagers:
>
> ./bin/flink run -m yarn-cluster -yn 2 \
> ./examples/batch/WordCount.jar \
> --input hdfs:///user/hamlet.txt --output hdfs:///user/wordcount_out
>
> -----邮件原件-----
> 发件人: Yangze Guo [mailto:karmagyz@gmail.com]
> 发送时间: 2020年8月18日 星期二 11:31
> 收件人: 范超 <fa...@mgtv.com>
> 抄送: user (user@flink.apache.org) <us...@flink.apache.org>
> 主题: Re: How to specify the number of TaskManagers in Yarn Cluster using Per-Job Mode
>
> Hi,
>
> Flink can control how many TM to start, but where to start the TMs depends on Yarn.
>
> Do you meet any problem when deploying on Yarn or running Flink job?
> Why do you need to start the TMs on all the three machines?
>
> Best,
> Yangze Guo
>
> On Tue, Aug 18, 2020 at 11:25 AM 范超 <fa...@mgtv.com> wrote:
> >
> > Thanks Yangze
> > The reason why I don’t deploying a standalone cluster, it's because there kafka, kudu, hadoop, zookeeper on these machines, maybe currently using the yarn to manage resources is the best choice for me.
> > If Flink can not control how many tm to start , could anyone providing
> > me some best practice for deploying on yarn please? I read the [1] and
> > still don't very clear
> >
> > [1]
> > https://www.ververica.com/blog/how-to-size-your-apache-flink-cluster-g
> > eneral-guidelines
> >
> > -----邮件原件-----
> > 发件人: Yangze Guo [mailto:karmagyz@gmail.com]
> > 发送时间: 2020年8月18日 星期二 10:50
> > 收件人: 范超 <fa...@mgtv.com>
> > 抄送: user (user@flink.apache.org) <us...@flink.apache.org>
> > 主题: Re: How to specify the number of TaskManagers in Yarn Cluster
> > using Per-Job Mode
> >
> > Hi,
> >
> > I think that is only related to the Yarn scheduling strategy. AFAIK, Flink could not control it. You could check the RM log to figure out why it did not schedule the containers to all the three machines. BTW, if you have specific requirements to start with all the three machines, how about deploying a standalone cluster instead?
> >
> > Best,
> > Yangze Guo
> >
> > On Tue, Aug 18, 2020 at 10:24 AM 范超 <fa...@mgtv.com> wrote:
> > >
> > > Thanks Yangze
> > >
> > > All 3 machines NodeManager is started.
> > >
> > > I just don't know why not three machines each running a Flink
> > > TaskManager and how to achieve this
> > >
> > > -----邮件原件-----
> > > 发件人: Yangze Guo [mailto:karmagyz@gmail.com]
> > > 发送时间: 2020年8月18日 星期二 10:10
> > > 收件人: 范超 <fa...@mgtv.com>
> > > 抄送: user (user@flink.apache.org) <us...@flink.apache.org>
> > > 主题: Re: How to specify the number of TaskManagers in Yarn Cluster
> > > using Per-Job Mode
> > >
> > > Hi,
> > >
> > > Do you start the NodeManager in all the three machines? If so, could you check all the NMs correctly connect to the ResourceManager?
> > >
> > > Best,
> > > Yangze Guo
> > >
> > > On Tue, Aug 18, 2020 at 10:01 AM 范超 <fa...@mgtv.com> wrote:
> > > >
> > > > Hi, Dev and Users
> > > > I’ve 3 machines each one is 8 cores and 16GB memory.
> > > > Following it’s my Resource Manager screenshot the cluster have 36GB total.
> > > > I specify the paralism to 3 or even up to 12, But the task manager is always running on two nodes not all three machine, the third node does not start the task manager.
> > > > I tried set the –p –tm –jm parameters, but it always the same, only different is more container on the two maching but not all three machine start the task manager.
> > > > My question is how to set the cli parameter to start all of my
> > > > three machine (all task manager start on 3 machines)
> > > >
> > > > Thanks a lot
> > > > [cid:image001.png@01D67546.62291B70]
> > > >
> > > >
> > > > Chao fan
> > > >
答复: How to specify the number of TaskManagers in Yarn Cluster using Per-Job Mode
Posted by 范超 <fa...@mgtv.com>.
Thanks Yangze
1. Do you meet any problem when deploying on Yarn or running Flink job?
My job works well
2. Why do you need to start the TMs on all the three machines?
From cluster perspective, I wonder if the process pressure can be balance to 3 machines.
3. Flink can control how many TM to start, but where to start the TMs depends on Yarn.
Yes, the job where to start the TM is depend on Yarn.
Could you please tell me parameter controls how many TM to start, the yn parameter is delete from 1.10 as the 1.9 doc sample list[1] below
[1] https://ci.apache.org/projects/flink/flink-docs-release-1.9/ops/cli.html
Run example program using a per-job YARN cluster with 2 TaskManagers:
./bin/flink run -m yarn-cluster -yn 2 \
./examples/batch/WordCount.jar \
--input hdfs:///user/hamlet.txt --output hdfs:///user/wordcount_out
-----邮件原件-----
发件人: Yangze Guo [mailto:karmagyz@gmail.com]
发送时间: 2020年8月18日 星期二 11:31
收件人: 范超 <fa...@mgtv.com>
抄送: user (user@flink.apache.org) <us...@flink.apache.org>
主题: Re: How to specify the number of TaskManagers in Yarn Cluster using Per-Job Mode
Hi,
Flink can control how many TM to start, but where to start the TMs depends on Yarn.
Do you meet any problem when deploying on Yarn or running Flink job?
Why do you need to start the TMs on all the three machines?
Best,
Yangze Guo
On Tue, Aug 18, 2020 at 11:25 AM 范超 <fa...@mgtv.com> wrote:
>
> Thanks Yangze
> The reason why I don’t deploying a standalone cluster, it's because there kafka, kudu, hadoop, zookeeper on these machines, maybe currently using the yarn to manage resources is the best choice for me.
> If Flink can not control how many tm to start , could anyone providing
> me some best practice for deploying on yarn please? I read the [1] and
> still don't very clear
>
> [1]
> https://www.ververica.com/blog/how-to-size-your-apache-flink-cluster-g
> eneral-guidelines
>
> -----邮件原件-----
> 发件人: Yangze Guo [mailto:karmagyz@gmail.com]
> 发送时间: 2020年8月18日 星期二 10:50
> 收件人: 范超 <fa...@mgtv.com>
> 抄送: user (user@flink.apache.org) <us...@flink.apache.org>
> 主题: Re: How to specify the number of TaskManagers in Yarn Cluster
> using Per-Job Mode
>
> Hi,
>
> I think that is only related to the Yarn scheduling strategy. AFAIK, Flink could not control it. You could check the RM log to figure out why it did not schedule the containers to all the three machines. BTW, if you have specific requirements to start with all the three machines, how about deploying a standalone cluster instead?
>
> Best,
> Yangze Guo
>
> On Tue, Aug 18, 2020 at 10:24 AM 范超 <fa...@mgtv.com> wrote:
> >
> > Thanks Yangze
> >
> > All 3 machines NodeManager is started.
> >
> > I just don't know why not three machines each running a Flink
> > TaskManager and how to achieve this
> >
> > -----邮件原件-----
> > 发件人: Yangze Guo [mailto:karmagyz@gmail.com]
> > 发送时间: 2020年8月18日 星期二 10:10
> > 收件人: 范超 <fa...@mgtv.com>
> > 抄送: user (user@flink.apache.org) <us...@flink.apache.org>
> > 主题: Re: How to specify the number of TaskManagers in Yarn Cluster
> > using Per-Job Mode
> >
> > Hi,
> >
> > Do you start the NodeManager in all the three machines? If so, could you check all the NMs correctly connect to the ResourceManager?
> >
> > Best,
> > Yangze Guo
> >
> > On Tue, Aug 18, 2020 at 10:01 AM 范超 <fa...@mgtv.com> wrote:
> > >
> > > Hi, Dev and Users
> > > I’ve 3 machines each one is 8 cores and 16GB memory.
> > > Following it’s my Resource Manager screenshot the cluster have 36GB total.
> > > I specify the paralism to 3 or even up to 12, But the task manager is always running on two nodes not all three machine, the third node does not start the task manager.
> > > I tried set the –p –tm –jm parameters, but it always the same, only different is more container on the two maching but not all three machine start the task manager.
> > > My question is how to set the cli parameter to start all of my
> > > three machine (all task manager start on 3 machines)
> > >
> > > Thanks a lot
> > > [cid:image001.png@01D67546.62291B70]
> > >
> > >
> > > Chao fan
> > >
Re: How to specify the number of TaskManagers in Yarn Cluster using
Per-Job Mode
Posted by Yangze Guo <ka...@gmail.com>.
Hi,
Flink can control how many TM to start, but where to start the TMs
depends on Yarn.
Do you meet any problem when deploying on Yarn or running Flink job?
Why do you need to start the TMs on all the three machines?
Best,
Yangze Guo
On Tue, Aug 18, 2020 at 11:25 AM 范超 <fa...@mgtv.com> wrote:
>
> Thanks Yangze
> The reason why I don’t deploying a standalone cluster, it's because there kafka, kudu, hadoop, zookeeper on these machines, maybe currently using the yarn to manage resources is the best choice for me.
> If Flink can not control how many tm to start , could anyone providing me some best practice for deploying on yarn please? I read the [1] and still don't very clear
>
> [1] https://www.ververica.com/blog/how-to-size-your-apache-flink-cluster-general-guidelines
>
> -----邮件原件-----
> 发件人: Yangze Guo [mailto:karmagyz@gmail.com]
> 发送时间: 2020年8月18日 星期二 10:50
> 收件人: 范超 <fa...@mgtv.com>
> 抄送: user (user@flink.apache.org) <us...@flink.apache.org>
> 主题: Re: How to specify the number of TaskManagers in Yarn Cluster using Per-Job Mode
>
> Hi,
>
> I think that is only related to the Yarn scheduling strategy. AFAIK, Flink could not control it. You could check the RM log to figure out why it did not schedule the containers to all the three machines. BTW, if you have specific requirements to start with all the three machines, how about deploying a standalone cluster instead?
>
> Best,
> Yangze Guo
>
> On Tue, Aug 18, 2020 at 10:24 AM 范超 <fa...@mgtv.com> wrote:
> >
> > Thanks Yangze
> >
> > All 3 machines NodeManager is started.
> >
> > I just don't know why not three machines each running a Flink
> > TaskManager and how to achieve this
> >
> > -----邮件原件-----
> > 发件人: Yangze Guo [mailto:karmagyz@gmail.com]
> > 发送时间: 2020年8月18日 星期二 10:10
> > 收件人: 范超 <fa...@mgtv.com>
> > 抄送: user (user@flink.apache.org) <us...@flink.apache.org>
> > 主题: Re: How to specify the number of TaskManagers in Yarn Cluster
> > using Per-Job Mode
> >
> > Hi,
> >
> > Do you start the NodeManager in all the three machines? If so, could you check all the NMs correctly connect to the ResourceManager?
> >
> > Best,
> > Yangze Guo
> >
> > On Tue, Aug 18, 2020 at 10:01 AM 范超 <fa...@mgtv.com> wrote:
> > >
> > > Hi, Dev and Users
> > > I’ve 3 machines each one is 8 cores and 16GB memory.
> > > Following it’s my Resource Manager screenshot the cluster have 36GB total.
> > > I specify the paralism to 3 or even up to 12, But the task manager is always running on two nodes not all three machine, the third node does not start the task manager.
> > > I tried set the –p –tm –jm parameters, but it always the same, only different is more container on the two maching but not all three machine start the task manager.
> > > My question is how to set the cli parameter to start all of my three
> > > machine (all task manager start on 3 machines)
> > >
> > > Thanks a lot
> > > [cid:image001.png@01D67546.62291B70]
> > >
> > >
> > > Chao fan
> > >
答复: How to specify the number of TaskManagers in Yarn Cluster using Per-Job Mode
Posted by 范超 <fa...@mgtv.com>.
Thanks Yangze
The reason why I don’t deploying a standalone cluster, it's because there kafka, kudu, hadoop, zookeeper on these machines, maybe currently using the yarn to manage resources is the best choice for me.
If Flink can not control how many tm to start , could anyone providing me some best practice for deploying on yarn please? I read the [1] and still don't very clear
[1] https://www.ververica.com/blog/how-to-size-your-apache-flink-cluster-general-guidelines
-----邮件原件-----
发件人: Yangze Guo [mailto:karmagyz@gmail.com]
发送时间: 2020年8月18日 星期二 10:50
收件人: 范超 <fa...@mgtv.com>
抄送: user (user@flink.apache.org) <us...@flink.apache.org>
主题: Re: How to specify the number of TaskManagers in Yarn Cluster using Per-Job Mode
Hi,
I think that is only related to the Yarn scheduling strategy. AFAIK, Flink could not control it. You could check the RM log to figure out why it did not schedule the containers to all the three machines. BTW, if you have specific requirements to start with all the three machines, how about deploying a standalone cluster instead?
Best,
Yangze Guo
On Tue, Aug 18, 2020 at 10:24 AM 范超 <fa...@mgtv.com> wrote:
>
> Thanks Yangze
>
> All 3 machines NodeManager is started.
>
> I just don't know why not three machines each running a Flink
> TaskManager and how to achieve this
>
> -----邮件原件-----
> 发件人: Yangze Guo [mailto:karmagyz@gmail.com]
> 发送时间: 2020年8月18日 星期二 10:10
> 收件人: 范超 <fa...@mgtv.com>
> 抄送: user (user@flink.apache.org) <us...@flink.apache.org>
> 主题: Re: How to specify the number of TaskManagers in Yarn Cluster
> using Per-Job Mode
>
> Hi,
>
> Do you start the NodeManager in all the three machines? If so, could you check all the NMs correctly connect to the ResourceManager?
>
> Best,
> Yangze Guo
>
> On Tue, Aug 18, 2020 at 10:01 AM 范超 <fa...@mgtv.com> wrote:
> >
> > Hi, Dev and Users
> > I’ve 3 machines each one is 8 cores and 16GB memory.
> > Following it’s my Resource Manager screenshot the cluster have 36GB total.
> > I specify the paralism to 3 or even up to 12, But the task manager is always running on two nodes not all three machine, the third node does not start the task manager.
> > I tried set the –p –tm –jm parameters, but it always the same, only different is more container on the two maching but not all three machine start the task manager.
> > My question is how to set the cli parameter to start all of my three
> > machine (all task manager start on 3 machines)
> >
> > Thanks a lot
> > [cid:image001.png@01D67546.62291B70]
> >
> >
> > Chao fan
> >
Re: How to specify the number of TaskManagers in Yarn Cluster using
Per-Job Mode
Posted by Yangze Guo <ka...@gmail.com>.
Hi,
I think that is only related to the Yarn scheduling strategy. AFAIK,
Flink could not control it. You could check the RM log to figure out
why it did not schedule the containers to all the three machines. BTW,
if you have specific requirements to start with all the three
machines, how about deploying a standalone cluster instead?
Best,
Yangze Guo
On Tue, Aug 18, 2020 at 10:24 AM 范超 <fa...@mgtv.com> wrote:
>
> Thanks Yangze
>
> All 3 machines NodeManager is started.
>
> I just don't know why not three machines each running a Flink TaskManager and how to achieve this
>
> -----邮件原件-----
> 发件人: Yangze Guo [mailto:karmagyz@gmail.com]
> 发送时间: 2020年8月18日 星期二 10:10
> 收件人: 范超 <fa...@mgtv.com>
> 抄送: user (user@flink.apache.org) <us...@flink.apache.org>
> 主题: Re: How to specify the number of TaskManagers in Yarn Cluster using Per-Job Mode
>
> Hi,
>
> Do you start the NodeManager in all the three machines? If so, could you check all the NMs correctly connect to the ResourceManager?
>
> Best,
> Yangze Guo
>
> On Tue, Aug 18, 2020 at 10:01 AM 范超 <fa...@mgtv.com> wrote:
> >
> > Hi, Dev and Users
> > I’ve 3 machines each one is 8 cores and 16GB memory.
> > Following it’s my Resource Manager screenshot the cluster have 36GB total.
> > I specify the paralism to 3 or even up to 12, But the task manager is always running on two nodes not all three machine, the third node does not start the task manager.
> > I tried set the –p –tm –jm parameters, but it always the same, only different is more container on the two maching but not all three machine start the task manager.
> > My question is how to set the cli parameter to start all of my three
> > machine (all task manager start on 3 machines)
> >
> > Thanks a lot
> > [cid:image001.png@01D67546.62291B70]
> >
> >
> > Chao fan
> >