You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Akmal Abbasov <ak...@icloud.com> on 2015/05/04 17:51:14 UTC
Number of vcores for YARN
Hi,
when I execute
cat /proc/cpuinfo | grep ^processor | wc -l
I get 2, do I need to specify this value in yarn.nodemanager.resource.cpu-vcores
or there is some kind of ratio between pcore and vcore?
I found yarn.nodemanager.vcores-pcores-ratio but it seems that is is deprecated, since I cannot find it in hadoop 2.5.1.
Thank you.
Regards,
Akmal Abbasov
Re: Number of vcores for YARN
Posted by Akmal Abbasov <ak...@icloud.com>.
Hi Yves,
Now I got it.
Thank you.
Regards,
Akmal Abbasov
> On 05 May 2015, at 13:03, yves callaert <yv...@hotmail.com> wrote:
>
> Hi,
> From the hortonworks guide:
>
> In YARN, vcores (virtual cores) are used to normalize CPU resources across the cluster. The vcores value sets the number of CPU cores that can be allocated for containers.
> Set the number of vcores so that it matches the number of physical CPU cores on the NodeManager hosts. set the following property in the /etc/hadoop/conf/yarn-site.xml file on the ResourceManager and NodeManager hosts:
>
>
> With Regards,
> Yves
>
>
>
> From: akmal.abbasov@icloud.com
> Subject: Number of vcores for YARN
> Date: Mon, 4 May 2015 17:51:14 +0200
> To: user@hadoop.apache.org
>
> Hi,
> when I execute
> cat /proc/cpuinfo | grep ^processor | wc -l
> I get 2, do I need to specify this value in yarn.nodemanager.resource.cpu-vcores
> or there is some kind of ratio between pcore and vcore?
> I found yarn.nodemanager.vcores-pcores-ratio but it seems that is is deprecated, since I cannot find it in hadoop 2.5.1.
> Thank you.
>
> Regards,
> Akmal Abbasov
Re: Number of vcores for YARN
Posted by Akmal Abbasov <ak...@icloud.com>.
Hi Yves,
Now I got it.
Thank you.
Regards,
Akmal Abbasov
> On 05 May 2015, at 13:03, yves callaert <yv...@hotmail.com> wrote:
>
> Hi,
> From the hortonworks guide:
>
> In YARN, vcores (virtual cores) are used to normalize CPU resources across the cluster. The vcores value sets the number of CPU cores that can be allocated for containers.
> Set the number of vcores so that it matches the number of physical CPU cores on the NodeManager hosts. set the following property in the /etc/hadoop/conf/yarn-site.xml file on the ResourceManager and NodeManager hosts:
>
>
> With Regards,
> Yves
>
>
>
> From: akmal.abbasov@icloud.com
> Subject: Number of vcores for YARN
> Date: Mon, 4 May 2015 17:51:14 +0200
> To: user@hadoop.apache.org
>
> Hi,
> when I execute
> cat /proc/cpuinfo | grep ^processor | wc -l
> I get 2, do I need to specify this value in yarn.nodemanager.resource.cpu-vcores
> or there is some kind of ratio between pcore and vcore?
> I found yarn.nodemanager.vcores-pcores-ratio but it seems that is is deprecated, since I cannot find it in hadoop 2.5.1.
> Thank you.
>
> Regards,
> Akmal Abbasov
Re: Number of vcores for YARN
Posted by Akmal Abbasov <ak...@icloud.com>.
Hi Yves,
Now I got it.
Thank you.
Regards,
Akmal Abbasov
> On 05 May 2015, at 13:03, yves callaert <yv...@hotmail.com> wrote:
>
> Hi,
> From the hortonworks guide:
>
> In YARN, vcores (virtual cores) are used to normalize CPU resources across the cluster. The vcores value sets the number of CPU cores that can be allocated for containers.
> Set the number of vcores so that it matches the number of physical CPU cores on the NodeManager hosts. set the following property in the /etc/hadoop/conf/yarn-site.xml file on the ResourceManager and NodeManager hosts:
>
>
> With Regards,
> Yves
>
>
>
> From: akmal.abbasov@icloud.com
> Subject: Number of vcores for YARN
> Date: Mon, 4 May 2015 17:51:14 +0200
> To: user@hadoop.apache.org
>
> Hi,
> when I execute
> cat /proc/cpuinfo | grep ^processor | wc -l
> I get 2, do I need to specify this value in yarn.nodemanager.resource.cpu-vcores
> or there is some kind of ratio between pcore and vcore?
> I found yarn.nodemanager.vcores-pcores-ratio but it seems that is is deprecated, since I cannot find it in hadoop 2.5.1.
> Thank you.
>
> Regards,
> Akmal Abbasov
Re: Number of vcores for YARN
Posted by Akmal Abbasov <ak...@icloud.com>.
Hi Yves,
Now I got it.
Thank you.
Regards,
Akmal Abbasov
> On 05 May 2015, at 13:03, yves callaert <yv...@hotmail.com> wrote:
>
> Hi,
> From the hortonworks guide:
>
> In YARN, vcores (virtual cores) are used to normalize CPU resources across the cluster. The vcores value sets the number of CPU cores that can be allocated for containers.
> Set the number of vcores so that it matches the number of physical CPU cores on the NodeManager hosts. set the following property in the /etc/hadoop/conf/yarn-site.xml file on the ResourceManager and NodeManager hosts:
>
>
> With Regards,
> Yves
>
>
>
> From: akmal.abbasov@icloud.com
> Subject: Number of vcores for YARN
> Date: Mon, 4 May 2015 17:51:14 +0200
> To: user@hadoop.apache.org
>
> Hi,
> when I execute
> cat /proc/cpuinfo | grep ^processor | wc -l
> I get 2, do I need to specify this value in yarn.nodemanager.resource.cpu-vcores
> or there is some kind of ratio between pcore and vcore?
> I found yarn.nodemanager.vcores-pcores-ratio but it seems that is is deprecated, since I cannot find it in hadoop 2.5.1.
> Thank you.
>
> Regards,
> Akmal Abbasov
RE: Number of vcores for YARN
Posted by yves callaert <yv...@hotmail.com>.
Hi,
>From the hortonworks guide:
In YARN, vcores (virtual cores) are used to normalize CPU resources across the
cluster. The vcores value sets the number of CPU cores that can be allocated for
containers.
Set the number of vcores so that it matches the number of physical CPU cores on the
NodeManager hosts. set the following property in the
/etc/hadoop/conf/yarn-site.xml file on the ResourceManager and
NodeManager hosts:
With Regards,
Yves
From: akmal.abbasov@icloud.com
Subject: Number of vcores for YARN
Date: Mon, 4 May 2015 17:51:14 +0200
To: user@hadoop.apache.org
Hi,when I execute cat /proc/cpuinfo | grep ^processor | wc -l I get 2, do I need to specify this value in yarn.nodemanager.resource.cpu-vcoresor there is some kind of ratio between pcore and vcore?I found yarn.nodemanager.vcores-pcores-ratio but it seems that is is deprecated, since I cannot find it in hadoop 2.5.1.Thank you.
Regards,Akmal Abbasov
RE: Number of vcores for YARN
Posted by yves callaert <yv...@hotmail.com>.
Hi,
>From the hortonworks guide:
In YARN, vcores (virtual cores) are used to normalize CPU resources across the
cluster. The vcores value sets the number of CPU cores that can be allocated for
containers.
Set the number of vcores so that it matches the number of physical CPU cores on the
NodeManager hosts. set the following property in the
/etc/hadoop/conf/yarn-site.xml file on the ResourceManager and
NodeManager hosts:
With Regards,
Yves
From: akmal.abbasov@icloud.com
Subject: Number of vcores for YARN
Date: Mon, 4 May 2015 17:51:14 +0200
To: user@hadoop.apache.org
Hi,when I execute cat /proc/cpuinfo | grep ^processor | wc -l I get 2, do I need to specify this value in yarn.nodemanager.resource.cpu-vcoresor there is some kind of ratio between pcore and vcore?I found yarn.nodemanager.vcores-pcores-ratio but it seems that is is deprecated, since I cannot find it in hadoop 2.5.1.Thank you.
Regards,Akmal Abbasov
RE: Number of vcores for YARN
Posted by yves callaert <yv...@hotmail.com>.
Hi,
>From the hortonworks guide:
In YARN, vcores (virtual cores) are used to normalize CPU resources across the
cluster. The vcores value sets the number of CPU cores that can be allocated for
containers.
Set the number of vcores so that it matches the number of physical CPU cores on the
NodeManager hosts. set the following property in the
/etc/hadoop/conf/yarn-site.xml file on the ResourceManager and
NodeManager hosts:
With Regards,
Yves
From: akmal.abbasov@icloud.com
Subject: Number of vcores for YARN
Date: Mon, 4 May 2015 17:51:14 +0200
To: user@hadoop.apache.org
Hi,when I execute cat /proc/cpuinfo | grep ^processor | wc -l I get 2, do I need to specify this value in yarn.nodemanager.resource.cpu-vcoresor there is some kind of ratio between pcore and vcore?I found yarn.nodemanager.vcores-pcores-ratio but it seems that is is deprecated, since I cannot find it in hadoop 2.5.1.Thank you.
Regards,Akmal Abbasov
RE: Number of vcores for YARN
Posted by yves callaert <yv...@hotmail.com>.
Hi,
>From the hortonworks guide:
In YARN, vcores (virtual cores) are used to normalize CPU resources across the
cluster. The vcores value sets the number of CPU cores that can be allocated for
containers.
Set the number of vcores so that it matches the number of physical CPU cores on the
NodeManager hosts. set the following property in the
/etc/hadoop/conf/yarn-site.xml file on the ResourceManager and
NodeManager hosts:
With Regards,
Yves
From: akmal.abbasov@icloud.com
Subject: Number of vcores for YARN
Date: Mon, 4 May 2015 17:51:14 +0200
To: user@hadoop.apache.org
Hi,when I execute cat /proc/cpuinfo | grep ^processor | wc -l I get 2, do I need to specify this value in yarn.nodemanager.resource.cpu-vcoresor there is some kind of ratio between pcore and vcore?I found yarn.nodemanager.vcores-pcores-ratio but it seems that is is deprecated, since I cannot find it in hadoop 2.5.1.Thank you.
Regards,Akmal Abbasov