You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by Merto Mertek <ma...@gmail.com> on 2012/03/13 18:49:44 UTC

Fairscheduler - disable "default" pool

I know that by design all unmarked jobs goes to that pool, however I am
doing some testing and I am interested if is possible to disable it..

Thanks

Re: Fairscheduler - disable "default" pool

Posted by Merto Mertek <ma...@gmail.com>.

Hm,
I would like to achieve that if I call poolManager.getPools() I get just
pools that I defined in my allocation file.. Harsh I tried your option too
but it still shows on the servlet page which means it is present and still
persist in poolManager..

Meanwhile I have found that probably I will have to change some code in
PoolManager (comment line 144 which is supposed to create the default
pool). Thanks both of you for your effort..

On 13 March 2012 21:42, Harsh J <ha...@cloudera.com> wrote:

> Merto,
>
> Are you looking to disable all undefined pools completely, preventing
> submissions to it?
> For that, "mapred.fairscheduler.allow.undeclared.pools" needs to be
> set to false.
>
> On Wed, Mar 14, 2012 at 1:36 AM, Merto Mertek <ma...@gmail.com> wrote:
> > Thanks for your workaround, but I think that with this you just put a
> > constraint on the pool that it will not accept any job. I am doing some
> > calculation with weights and do not want the default pool weight to be
> > included in the computation..
> >
> >
> > On 13 March 2012 18:52, Jean-Daniel Cryans <jd...@apache.org> wrote:
> >
> >> We do it here by setting this:
> >>
> >> <poolMaxJobsDefault>0</poolMaxJobsDefault>
> >>
> >> So that you _must_ have a pool (that's configured with a different
> >> maxRunningJobs) in order to run jobs.
> >>
> >> Hope this helps,
> >>
> >> J-D
> >>
> >> On Tue, Mar 13, 2012 at 10:49 AM, Merto Mertek <ma...@gmail.com>
> >> wrote:
> >> > I know that by design all unmarked jobs goes to that pool, however I
> am
> >> > doing some testing and I am interested if is possible to disable it..
> >> >
> >> > Thanks
> >>
>
>
>
> --
> Harsh J
>

Re: Fairscheduler - disable "default" pool

Posted by Harsh J <ha...@cloudera.com>.

Merto,

Are you looking to disable all undefined pools completely, preventing
submissions to it?
For that, "mapred.fairscheduler.allow.undeclared.pools" needs to be
set to false.

On Wed, Mar 14, 2012 at 1:36 AM, Merto Mertek <ma...@gmail.com> wrote:
> Thanks for your workaround, but I think that with this you just put a
> constraint on the pool that it will not accept any job. I am doing some
> calculation with weights and do not want the default pool weight to be
> included in the computation..
>
>
> On 13 March 2012 18:52, Jean-Daniel Cryans <jd...@apache.org> wrote:
>
>> We do it here by setting this:
>>
>> <poolMaxJobsDefault>0</poolMaxJobsDefault>
>>
>> So that you _must_ have a pool (that's configured with a different
>> maxRunningJobs) in order to run jobs.
>>
>> Hope this helps,
>>
>> J-D
>>
>> On Tue, Mar 13, 2012 at 10:49 AM, Merto Mertek <ma...@gmail.com>
>> wrote:
>> > I know that by design all unmarked jobs goes to that pool, however I am
>> > doing some testing and I am interested if is possible to disable it..
>> >
>> > Thanks
>>



-- 
Harsh J

Re: Fairscheduler - disable "default" pool

Posted by Merto Mertek <ma...@gmail.com>.

Thanks for your workaround, but I think that with this you just put a
constraint on the pool that it will not accept any job. I am doing some
calculation with weights and do not want the default pool weight to be
included in the computation..

On 13 March 2012 18:52, Jean-Daniel Cryans <jd...@apache.org> wrote:

> We do it here by setting this:
>
> <poolMaxJobsDefault>0</poolMaxJobsDefault>
>
> So that you _must_ have a pool (that's configured with a different
> maxRunningJobs) in order to run jobs.
>
> Hope this helps,
>
> J-D
>
> On Tue, Mar 13, 2012 at 10:49 AM, Merto Mertek <ma...@gmail.com>
> wrote:
> > I know that by design all unmarked jobs goes to that pool, however I am
> > doing some testing and I am interested if is possible to disable it..
> >
> > Thanks
>

Re: Fairscheduler - disable "default" pool

Posted by Jean-Daniel Cryans <jd...@apache.org>.

We do it here by setting this:

<poolMaxJobsDefault>0</poolMaxJobsDefault>

So that you _must_ have a pool (that's configured with a different
maxRunningJobs) in order to run jobs.

Hope this helps,

J-D

On Tue, Mar 13, 2012 at 10:49 AM, Merto Mertek <ma...@gmail.com> wrote:
> I know that by design all unmarked jobs goes to that pool, however I am
> doing some testing and I am interested if is possible to disable it..
>
> Thanks