You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Saiph Kappa <sa...@gmail.com> on 2016/02/24 01:43:05 UTC
How to use all available task managers
Hi,
I am running a flink stream application on a cluster with 6 slaves/task
managers. I have set in flink-conf.yaml of every machine
"parallelization.degree.default: 6". However, when I run my application it
just uses one task slot and not all of them. Am I missing something?
Thanks.
Re: How to use all available task managers
Posted by "Matthias J. Sax" <mj...@apache.org>.
Could it be, that you would need to edit client local flink-conf.yaml
instead of the TaskManager config files? (In case, you do not want to
specify parallelism via env.setParallelism(int);)
-Matthias
On 02/24/2016 04:19 PM, Saiph Kappa wrote:
> Thanks! It worked now :-)
>
> On Wed, Feb 24, 2016 at 2:48 PM, Ufuk Celebi <uce@apache.org
> <ma...@apache.org>> wrote:
>
> You can use the environment to set it the job parallelism to 6 e.g.
> env.setParallelism(6).
>
> Setting this will override the default behaviour. Maybe that's why the
> default parallelism is not working... you might have it set to 1
> already?
>
> On Wed, Feb 24, 2016 at 3:41 PM, Saiph Kappa <saiph.kappa@gmail.com
> <ma...@gmail.com>> wrote:
> > I set "parallelism.default: 6" on flink-conf.yaml of all 6
> machines, and
> > still, my job only uses 1 task manager. Why?
> >
> >
> > On Wed, Feb 24, 2016 at 8:31 AM, Till Rohrmann
> <trohrmann@apache.org <ma...@apache.org>> wrote:
> >>
> >> Hi Saiph,
> >>
> >> I think the configuration value should be parallelism.default: 6.
> That
> >> will execute jobs which have not parallelism defined with a DOP of 6.
> >>
> >> Cheers,
> >> Till
> >>
> >>
> >> On Wed, Feb 24, 2016 at 1:43 AM, Saiph Kappa
> <saiph.kappa@gmail.com <ma...@gmail.com>>
> >> wrote:
> >>>
> >>> Hi,
> >>>
> >>> I am running a flink stream application on a cluster with 6
> slaves/task
> >>> managers. I have set in flink-conf.yaml of every machine
> >>> "parallelization.degree.default: 6". However, when I run my
> application it
> >>> just uses one task slot and not all of them. Am I missing something?
> >>>
> >>> Thanks.
> >>
> >>
> >
>
>
Re: How to use all available task managers
Posted by Saiph Kappa <sa...@gmail.com>.
Thanks! It worked now :-)
On Wed, Feb 24, 2016 at 2:48 PM, Ufuk Celebi <uc...@apache.org> wrote:
> You can use the environment to set it the job parallelism to 6 e.g.
> env.setParallelism(6).
>
> Setting this will override the default behaviour. Maybe that's why the
> default parallelism is not working... you might have it set to 1
> already?
>
> On Wed, Feb 24, 2016 at 3:41 PM, Saiph Kappa <sa...@gmail.com>
> wrote:
> > I set "parallelism.default: 6" on flink-conf.yaml of all 6 machines, and
> > still, my job only uses 1 task manager. Why?
> >
> >
> > On Wed, Feb 24, 2016 at 8:31 AM, Till Rohrmann <tr...@apache.org>
> wrote:
> >>
> >> Hi Saiph,
> >>
> >> I think the configuration value should be parallelism.default: 6. That
> >> will execute jobs which have not parallelism defined with a DOP of 6.
> >>
> >> Cheers,
> >> Till
> >>
> >>
> >> On Wed, Feb 24, 2016 at 1:43 AM, Saiph Kappa <sa...@gmail.com>
> >> wrote:
> >>>
> >>> Hi,
> >>>
> >>> I am running a flink stream application on a cluster with 6
> slaves/task
> >>> managers. I have set in flink-conf.yaml of every machine
> >>> "parallelization.degree.default: 6". However, when I run my
> application it
> >>> just uses one task slot and not all of them. Am I missing something?
> >>>
> >>> Thanks.
> >>
> >>
> >
>
Re: How to use all available task managers
Posted by Ufuk Celebi <uc...@apache.org>.
You can use the environment to set it the job parallelism to 6 e.g.
env.setParallelism(6).
Setting this will override the default behaviour. Maybe that's why the
default parallelism is not working... you might have it set to 1
already?
On Wed, Feb 24, 2016 at 3:41 PM, Saiph Kappa <sa...@gmail.com> wrote:
> I set "parallelism.default: 6" on flink-conf.yaml of all 6 machines, and
> still, my job only uses 1 task manager. Why?
>
>
> On Wed, Feb 24, 2016 at 8:31 AM, Till Rohrmann <tr...@apache.org> wrote:
>>
>> Hi Saiph,
>>
>> I think the configuration value should be parallelism.default: 6. That
>> will execute jobs which have not parallelism defined with a DOP of 6.
>>
>> Cheers,
>> Till
>>
>>
>> On Wed, Feb 24, 2016 at 1:43 AM, Saiph Kappa <sa...@gmail.com>
>> wrote:
>>>
>>> Hi,
>>>
>>> I am running a flink stream application on a cluster with 6 slaves/task
>>> managers. I have set in flink-conf.yaml of every machine
>>> "parallelization.degree.default: 6". However, when I run my application it
>>> just uses one task slot and not all of them. Am I missing something?
>>>
>>> Thanks.
>>
>>
>
Re: How to use all available task managers
Posted by Saiph Kappa <sa...@gmail.com>.
I set "parallelism.default: 6" on flink-conf.yaml of all 6 machines, and
still, my job only uses 1 task manager. Why?
On Wed, Feb 24, 2016 at 8:31 AM, Till Rohrmann <tr...@apache.org> wrote:
> Hi Saiph,
>
> I think the configuration value should be parallelism.default: 6. That
> will execute jobs which have not parallelism defined with a DOP of 6.
>
> Cheers,
> Till
>
>
> On Wed, Feb 24, 2016 at 1:43 AM, Saiph Kappa <sa...@gmail.com>
> wrote:
>
>> Hi,
>>
>> I am running a flink stream application on a cluster with 6 slaves/task
>> managers. I have set in flink-conf.yaml of every machine
>> "parallelization.degree.default: 6". However, when I run my application it
>> just uses one task slot and not all of them. Am I missing something?
>>
>> Thanks.
>>
>
>
Re: How to use all available task managers
Posted by Till Rohrmann <tr...@apache.org>.
Hi Saiph,
I think the configuration value should be parallelism.default: 6. That will
execute jobs which have not parallelism defined with a DOP of 6.
Cheers,
Till
On Wed, Feb 24, 2016 at 1:43 AM, Saiph Kappa <sa...@gmail.com> wrote:
> Hi,
>
> I am running a flink stream application on a cluster with 6 slaves/task
> managers. I have set in flink-conf.yaml of every machine
> "parallelization.degree.default: 6". However, when I run my application it
> just uses one task slot and not all of them. Am I missing something?
>
> Thanks.
>