You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Chetna C <ch...@gmail.com> on 2015/09/17 11:54:37 UTC
Configuring capacity scheduler?
Hi All,
Is there any way to configure capacity scheduler capacities in terms
of resources, i.e x mb, y cores, z disks? I could see, this feature is
available for fair scheduler.
If anyone has configured scheduler like this, please guide.
Thanks,
Chetna Chaudhari
Re: Configuring capacity scheduler?
Posted by Chetna C <ch...@gmail.com>.
Hi Laxman,
Thanks for the reply. It means in my case where I want to configure
queues in terms of absolute resources, I should go with fair scheduler only.
Thanks,
Chetna Chaudhari
On 17 September 2015 at 17:50, Laxman Ch <la...@gmail.com> wrote:
> Hi Chetna,
>
> All capacity scheduler queue configurations are in terms of percentage
> only and not absolute (as you asked). This is done to auto-scale the queues
> when new nodes are added to the cluster. Capacity scheduler enforces the
> following
>
> - Sumtotal of all allocations at any given level of queue tree must be 100.
> - Applications can be schedulerd only on leaf queues.
>
>
> On 17 September 2015 at 16:14, Chetna C <ch...@gmail.com> wrote:
>
>> Hi Yimen,
>> I read the documentation, but I couldn't find a way to configure
>> queue's in terms of resources. Documentation stated only way to configure
>> in terms of %. I would like some thing like
>> queue1 -> resource allocation -> 81920 mb, 20 vcores
>> instead of
>> queue1 -> resource allocation -> 25%.
>>
>>
>>
>> Thanks,
>> Chetna Chaudhari
>>
>> On 17 September 2015 at 16:00, YIMEN GAEL <yi...@gmail.com> wrote:
>>
>>> Hello Chetna,
>>>
>>> Please Kindly read the official documentation
>>>
>>> http://hadoop.apache.org/docs/r1.2.1/capacity_scheduler.html
>>>
>>> Regards
>>> Le jeu. 17 sept. 2015 à 11:54, Chetna C <ch...@gmail.com> a écrit :
>>>
>>>> Hi All,
>>>> Is there any way to configure capacity scheduler capacities in
>>>> terms of resources, i.e x mb, y cores, z disks? I could see, this feature
>>>> is available for fair scheduler.
>>>> If anyone has configured scheduler like this, please guide.
>>>>
>>>> Thanks,
>>>> Chetna Chaudhari
>>>>
>>>
>>
>
>
> --
> Thanks,
> Laxman
>
Re: Configuring capacity scheduler?
Posted by Chetna C <ch...@gmail.com>.
Hi Laxman,
Thanks for the reply. It means in my case where I want to configure
queues in terms of absolute resources, I should go with fair scheduler only.
Thanks,
Chetna Chaudhari
On 17 September 2015 at 17:50, Laxman Ch <la...@gmail.com> wrote:
> Hi Chetna,
>
> All capacity scheduler queue configurations are in terms of percentage
> only and not absolute (as you asked). This is done to auto-scale the queues
> when new nodes are added to the cluster. Capacity scheduler enforces the
> following
>
> - Sumtotal of all allocations at any given level of queue tree must be 100.
> - Applications can be schedulerd only on leaf queues.
>
>
> On 17 September 2015 at 16:14, Chetna C <ch...@gmail.com> wrote:
>
>> Hi Yimen,
>> I read the documentation, but I couldn't find a way to configure
>> queue's in terms of resources. Documentation stated only way to configure
>> in terms of %. I would like some thing like
>> queue1 -> resource allocation -> 81920 mb, 20 vcores
>> instead of
>> queue1 -> resource allocation -> 25%.
>>
>>
>>
>> Thanks,
>> Chetna Chaudhari
>>
>> On 17 September 2015 at 16:00, YIMEN GAEL <yi...@gmail.com> wrote:
>>
>>> Hello Chetna,
>>>
>>> Please Kindly read the official documentation
>>>
>>> http://hadoop.apache.org/docs/r1.2.1/capacity_scheduler.html
>>>
>>> Regards
>>> Le jeu. 17 sept. 2015 à 11:54, Chetna C <ch...@gmail.com> a écrit :
>>>
>>>> Hi All,
>>>> Is there any way to configure capacity scheduler capacities in
>>>> terms of resources, i.e x mb, y cores, z disks? I could see, this feature
>>>> is available for fair scheduler.
>>>> If anyone has configured scheduler like this, please guide.
>>>>
>>>> Thanks,
>>>> Chetna Chaudhari
>>>>
>>>
>>
>
>
> --
> Thanks,
> Laxman
>
Re: Configuring capacity scheduler?
Posted by Chetna C <ch...@gmail.com>.
Hi Laxman,
Thanks for the reply. It means in my case where I want to configure
queues in terms of absolute resources, I should go with fair scheduler only.
Thanks,
Chetna Chaudhari
On 17 September 2015 at 17:50, Laxman Ch <la...@gmail.com> wrote:
> Hi Chetna,
>
> All capacity scheduler queue configurations are in terms of percentage
> only and not absolute (as you asked). This is done to auto-scale the queues
> when new nodes are added to the cluster. Capacity scheduler enforces the
> following
>
> - Sumtotal of all allocations at any given level of queue tree must be 100.
> - Applications can be schedulerd only on leaf queues.
>
>
> On 17 September 2015 at 16:14, Chetna C <ch...@gmail.com> wrote:
>
>> Hi Yimen,
>> I read the documentation, but I couldn't find a way to configure
>> queue's in terms of resources. Documentation stated only way to configure
>> in terms of %. I would like some thing like
>> queue1 -> resource allocation -> 81920 mb, 20 vcores
>> instead of
>> queue1 -> resource allocation -> 25%.
>>
>>
>>
>> Thanks,
>> Chetna Chaudhari
>>
>> On 17 September 2015 at 16:00, YIMEN GAEL <yi...@gmail.com> wrote:
>>
>>> Hello Chetna,
>>>
>>> Please Kindly read the official documentation
>>>
>>> http://hadoop.apache.org/docs/r1.2.1/capacity_scheduler.html
>>>
>>> Regards
>>> Le jeu. 17 sept. 2015 à 11:54, Chetna C <ch...@gmail.com> a écrit :
>>>
>>>> Hi All,
>>>> Is there any way to configure capacity scheduler capacities in
>>>> terms of resources, i.e x mb, y cores, z disks? I could see, this feature
>>>> is available for fair scheduler.
>>>> If anyone has configured scheduler like this, please guide.
>>>>
>>>> Thanks,
>>>> Chetna Chaudhari
>>>>
>>>
>>
>
>
> --
> Thanks,
> Laxman
>
Re: Configuring capacity scheduler?
Posted by Chetna C <ch...@gmail.com>.
Hi Laxman,
Thanks for the reply. It means in my case where I want to configure
queues in terms of absolute resources, I should go with fair scheduler only.
Thanks,
Chetna Chaudhari
On 17 September 2015 at 17:50, Laxman Ch <la...@gmail.com> wrote:
> Hi Chetna,
>
> All capacity scheduler queue configurations are in terms of percentage
> only and not absolute (as you asked). This is done to auto-scale the queues
> when new nodes are added to the cluster. Capacity scheduler enforces the
> following
>
> - Sumtotal of all allocations at any given level of queue tree must be 100.
> - Applications can be schedulerd only on leaf queues.
>
>
> On 17 September 2015 at 16:14, Chetna C <ch...@gmail.com> wrote:
>
>> Hi Yimen,
>> I read the documentation, but I couldn't find a way to configure
>> queue's in terms of resources. Documentation stated only way to configure
>> in terms of %. I would like some thing like
>> queue1 -> resource allocation -> 81920 mb, 20 vcores
>> instead of
>> queue1 -> resource allocation -> 25%.
>>
>>
>>
>> Thanks,
>> Chetna Chaudhari
>>
>> On 17 September 2015 at 16:00, YIMEN GAEL <yi...@gmail.com> wrote:
>>
>>> Hello Chetna,
>>>
>>> Please Kindly read the official documentation
>>>
>>> http://hadoop.apache.org/docs/r1.2.1/capacity_scheduler.html
>>>
>>> Regards
>>> Le jeu. 17 sept. 2015 à 11:54, Chetna C <ch...@gmail.com> a écrit :
>>>
>>>> Hi All,
>>>> Is there any way to configure capacity scheduler capacities in
>>>> terms of resources, i.e x mb, y cores, z disks? I could see, this feature
>>>> is available for fair scheduler.
>>>> If anyone has configured scheduler like this, please guide.
>>>>
>>>> Thanks,
>>>> Chetna Chaudhari
>>>>
>>>
>>
>
>
> --
> Thanks,
> Laxman
>
Re: Configuring capacity scheduler?
Posted by Laxman Ch <la...@gmail.com>.
Hi Chetna,
All capacity scheduler queue configurations are in terms of percentage only
and not absolute (as you asked). This is done to auto-scale the queues when
new nodes are added to the cluster. Capacity scheduler enforces the
following
- Sumtotal of all allocations at any given level of queue tree must be 100.
- Applications can be schedulerd only on leaf queues.
On 17 September 2015 at 16:14, Chetna C <ch...@gmail.com> wrote:
> Hi Yimen,
> I read the documentation, but I couldn't find a way to configure
> queue's in terms of resources. Documentation stated only way to configure
> in terms of %. I would like some thing like
> queue1 -> resource allocation -> 81920 mb, 20 vcores
> instead of
> queue1 -> resource allocation -> 25%.
>
>
>
> Thanks,
> Chetna Chaudhari
>
> On 17 September 2015 at 16:00, YIMEN GAEL <yi...@gmail.com> wrote:
>
>> Hello Chetna,
>>
>> Please Kindly read the official documentation
>>
>> http://hadoop.apache.org/docs/r1.2.1/capacity_scheduler.html
>>
>> Regards
>> Le jeu. 17 sept. 2015 à 11:54, Chetna C <ch...@gmail.com> a écrit :
>>
>>> Hi All,
>>> Is there any way to configure capacity scheduler capacities in
>>> terms of resources, i.e x mb, y cores, z disks? I could see, this feature
>>> is available for fair scheduler.
>>> If anyone has configured scheduler like this, please guide.
>>>
>>> Thanks,
>>> Chetna Chaudhari
>>>
>>
>
--
Thanks,
Laxman
Re: Configuring capacity scheduler?
Posted by Laxman Ch <la...@gmail.com>.
Hi Chetna,
All capacity scheduler queue configurations are in terms of percentage only
and not absolute (as you asked). This is done to auto-scale the queues when
new nodes are added to the cluster. Capacity scheduler enforces the
following
- Sumtotal of all allocations at any given level of queue tree must be 100.
- Applications can be schedulerd only on leaf queues.
On 17 September 2015 at 16:14, Chetna C <ch...@gmail.com> wrote:
> Hi Yimen,
> I read the documentation, but I couldn't find a way to configure
> queue's in terms of resources. Documentation stated only way to configure
> in terms of %. I would like some thing like
> queue1 -> resource allocation -> 81920 mb, 20 vcores
> instead of
> queue1 -> resource allocation -> 25%.
>
>
>
> Thanks,
> Chetna Chaudhari
>
> On 17 September 2015 at 16:00, YIMEN GAEL <yi...@gmail.com> wrote:
>
>> Hello Chetna,
>>
>> Please Kindly read the official documentation
>>
>> http://hadoop.apache.org/docs/r1.2.1/capacity_scheduler.html
>>
>> Regards
>> Le jeu. 17 sept. 2015 à 11:54, Chetna C <ch...@gmail.com> a écrit :
>>
>>> Hi All,
>>> Is there any way to configure capacity scheduler capacities in
>>> terms of resources, i.e x mb, y cores, z disks? I could see, this feature
>>> is available for fair scheduler.
>>> If anyone has configured scheduler like this, please guide.
>>>
>>> Thanks,
>>> Chetna Chaudhari
>>>
>>
>
--
Thanks,
Laxman
Re: Configuring capacity scheduler?
Posted by Laxman Ch <la...@gmail.com>.
Hi Chetna,
All capacity scheduler queue configurations are in terms of percentage only
and not absolute (as you asked). This is done to auto-scale the queues when
new nodes are added to the cluster. Capacity scheduler enforces the
following
- Sumtotal of all allocations at any given level of queue tree must be 100.
- Applications can be schedulerd only on leaf queues.
On 17 September 2015 at 16:14, Chetna C <ch...@gmail.com> wrote:
> Hi Yimen,
> I read the documentation, but I couldn't find a way to configure
> queue's in terms of resources. Documentation stated only way to configure
> in terms of %. I would like some thing like
> queue1 -> resource allocation -> 81920 mb, 20 vcores
> instead of
> queue1 -> resource allocation -> 25%.
>
>
>
> Thanks,
> Chetna Chaudhari
>
> On 17 September 2015 at 16:00, YIMEN GAEL <yi...@gmail.com> wrote:
>
>> Hello Chetna,
>>
>> Please Kindly read the official documentation
>>
>> http://hadoop.apache.org/docs/r1.2.1/capacity_scheduler.html
>>
>> Regards
>> Le jeu. 17 sept. 2015 à 11:54, Chetna C <ch...@gmail.com> a écrit :
>>
>>> Hi All,
>>> Is there any way to configure capacity scheduler capacities in
>>> terms of resources, i.e x mb, y cores, z disks? I could see, this feature
>>> is available for fair scheduler.
>>> If anyone has configured scheduler like this, please guide.
>>>
>>> Thanks,
>>> Chetna Chaudhari
>>>
>>
>
--
Thanks,
Laxman
Re: Configuring capacity scheduler?
Posted by Laxman Ch <la...@gmail.com>.
Hi Chetna,
All capacity scheduler queue configurations are in terms of percentage only
and not absolute (as you asked). This is done to auto-scale the queues when
new nodes are added to the cluster. Capacity scheduler enforces the
following
- Sumtotal of all allocations at any given level of queue tree must be 100.
- Applications can be schedulerd only on leaf queues.
On 17 September 2015 at 16:14, Chetna C <ch...@gmail.com> wrote:
> Hi Yimen,
> I read the documentation, but I couldn't find a way to configure
> queue's in terms of resources. Documentation stated only way to configure
> in terms of %. I would like some thing like
> queue1 -> resource allocation -> 81920 mb, 20 vcores
> instead of
> queue1 -> resource allocation -> 25%.
>
>
>
> Thanks,
> Chetna Chaudhari
>
> On 17 September 2015 at 16:00, YIMEN GAEL <yi...@gmail.com> wrote:
>
>> Hello Chetna,
>>
>> Please Kindly read the official documentation
>>
>> http://hadoop.apache.org/docs/r1.2.1/capacity_scheduler.html
>>
>> Regards
>> Le jeu. 17 sept. 2015 à 11:54, Chetna C <ch...@gmail.com> a écrit :
>>
>>> Hi All,
>>> Is there any way to configure capacity scheduler capacities in
>>> terms of resources, i.e x mb, y cores, z disks? I could see, this feature
>>> is available for fair scheduler.
>>> If anyone has configured scheduler like this, please guide.
>>>
>>> Thanks,
>>> Chetna Chaudhari
>>>
>>
>
--
Thanks,
Laxman
Re: Configuring capacity scheduler?
Posted by Chetna C <ch...@gmail.com>.
Hi Yimen,
I read the documentation, but I couldn't find a way to configure
queue's in terms of resources. Documentation stated only way to configure
in terms of %. I would like some thing like
queue1 -> resource allocation -> 81920 mb, 20 vcores
instead of
queue1 -> resource allocation -> 25%.
Thanks,
Chetna Chaudhari
On 17 September 2015 at 16:00, YIMEN GAEL <yi...@gmail.com> wrote:
> Hello Chetna,
>
> Please Kindly read the official documentation
>
> http://hadoop.apache.org/docs/r1.2.1/capacity_scheduler.html
>
> Regards
> Le jeu. 17 sept. 2015 à 11:54, Chetna C <ch...@gmail.com> a écrit :
>
>> Hi All,
>> Is there any way to configure capacity scheduler capacities in
>> terms of resources, i.e x mb, y cores, z disks? I could see, this feature
>> is available for fair scheduler.
>> If anyone has configured scheduler like this, please guide.
>>
>> Thanks,
>> Chetna Chaudhari
>>
>
Re: Configuring capacity scheduler?
Posted by Chetna C <ch...@gmail.com>.
Hi Yimen,
I read the documentation, but I couldn't find a way to configure
queue's in terms of resources. Documentation stated only way to configure
in terms of %. I would like some thing like
queue1 -> resource allocation -> 81920 mb, 20 vcores
instead of
queue1 -> resource allocation -> 25%.
Thanks,
Chetna Chaudhari
On 17 September 2015 at 16:00, YIMEN GAEL <yi...@gmail.com> wrote:
> Hello Chetna,
>
> Please Kindly read the official documentation
>
> http://hadoop.apache.org/docs/r1.2.1/capacity_scheduler.html
>
> Regards
> Le jeu. 17 sept. 2015 à 11:54, Chetna C <ch...@gmail.com> a écrit :
>
>> Hi All,
>> Is there any way to configure capacity scheduler capacities in
>> terms of resources, i.e x mb, y cores, z disks? I could see, this feature
>> is available for fair scheduler.
>> If anyone has configured scheduler like this, please guide.
>>
>> Thanks,
>> Chetna Chaudhari
>>
>
Re: Configuring capacity scheduler?
Posted by Chetna C <ch...@gmail.com>.
Hi Yimen,
I read the documentation, but I couldn't find a way to configure
queue's in terms of resources. Documentation stated only way to configure
in terms of %. I would like some thing like
queue1 -> resource allocation -> 81920 mb, 20 vcores
instead of
queue1 -> resource allocation -> 25%.
Thanks,
Chetna Chaudhari
On 17 September 2015 at 16:00, YIMEN GAEL <yi...@gmail.com> wrote:
> Hello Chetna,
>
> Please Kindly read the official documentation
>
> http://hadoop.apache.org/docs/r1.2.1/capacity_scheduler.html
>
> Regards
> Le jeu. 17 sept. 2015 à 11:54, Chetna C <ch...@gmail.com> a écrit :
>
>> Hi All,
>> Is there any way to configure capacity scheduler capacities in
>> terms of resources, i.e x mb, y cores, z disks? I could see, this feature
>> is available for fair scheduler.
>> If anyone has configured scheduler like this, please guide.
>>
>> Thanks,
>> Chetna Chaudhari
>>
>
Re: Configuring capacity scheduler?
Posted by Chetna C <ch...@gmail.com>.
Hi Yimen,
I read the documentation, but I couldn't find a way to configure
queue's in terms of resources. Documentation stated only way to configure
in terms of %. I would like some thing like
queue1 -> resource allocation -> 81920 mb, 20 vcores
instead of
queue1 -> resource allocation -> 25%.
Thanks,
Chetna Chaudhari
On 17 September 2015 at 16:00, YIMEN GAEL <yi...@gmail.com> wrote:
> Hello Chetna,
>
> Please Kindly read the official documentation
>
> http://hadoop.apache.org/docs/r1.2.1/capacity_scheduler.html
>
> Regards
> Le jeu. 17 sept. 2015 à 11:54, Chetna C <ch...@gmail.com> a écrit :
>
>> Hi All,
>> Is there any way to configure capacity scheduler capacities in
>> terms of resources, i.e x mb, y cores, z disks? I could see, this feature
>> is available for fair scheduler.
>> If anyone has configured scheduler like this, please guide.
>>
>> Thanks,
>> Chetna Chaudhari
>>
>
Re: Configuring capacity scheduler?
Posted by YIMEN GAEL <yi...@gmail.com>.
Hello Chetna,
Please Kindly read the official documentation
http://hadoop.apache.org/docs/r1.2.1/capacity_scheduler.html
Regards
Le jeu. 17 sept. 2015 à 11:54, Chetna C <ch...@gmail.com> a écrit :
> Hi All,
> Is there any way to configure capacity scheduler capacities in
> terms of resources, i.e x mb, y cores, z disks? I could see, this feature
> is available for fair scheduler.
> If anyone has configured scheduler like this, please guide.
>
> Thanks,
> Chetna Chaudhari
>
Re: Configuring capacity scheduler?
Posted by YIMEN GAEL <yi...@gmail.com>.
Hello Chetna,
Please Kindly read the official documentation
http://hadoop.apache.org/docs/r1.2.1/capacity_scheduler.html
Regards
Le jeu. 17 sept. 2015 à 11:54, Chetna C <ch...@gmail.com> a écrit :
> Hi All,
> Is there any way to configure capacity scheduler capacities in
> terms of resources, i.e x mb, y cores, z disks? I could see, this feature
> is available for fair scheduler.
> If anyone has configured scheduler like this, please guide.
>
> Thanks,
> Chetna Chaudhari
>
Re: Configuring capacity scheduler?
Posted by YIMEN GAEL <yi...@gmail.com>.
Hello Chetna,
Please Kindly read the official documentation
http://hadoop.apache.org/docs/r1.2.1/capacity_scheduler.html
Regards
Le jeu. 17 sept. 2015 à 11:54, Chetna C <ch...@gmail.com> a écrit :
> Hi All,
> Is there any way to configure capacity scheduler capacities in
> terms of resources, i.e x mb, y cores, z disks? I could see, this feature
> is available for fair scheduler.
> If anyone has configured scheduler like this, please guide.
>
> Thanks,
> Chetna Chaudhari
>
Re: Configuring capacity scheduler?
Posted by YIMEN GAEL <yi...@gmail.com>.
Hello Chetna,
Please Kindly read the official documentation
http://hadoop.apache.org/docs/r1.2.1/capacity_scheduler.html
Regards
Le jeu. 17 sept. 2015 à 11:54, Chetna C <ch...@gmail.com> a écrit :
> Hi All,
> Is there any way to configure capacity scheduler capacities in
> terms of resources, i.e x mb, y cores, z disks? I could see, this feature
> is available for fair scheduler.
> If anyone has configured scheduler like this, please guide.
>
> Thanks,
> Chetna Chaudhari
>