Regarding Yarn virtual core configuration.

[Sairam Kannan <sk...@hawk.iit.edu>]

Hi all,
                 I have a doubt in setting the Yarn's virtual core
configuration. I have an Intel CPU slaves with hyperthreading enabled. The
total number of physical core is 8. Logical core is twice the physical core
( 8 * 2 = 16 ). While setting the "yarn.nodemanager.resource.cpu-vcores"
property, which value should I give ( 8 or 16 ) ?. I am trying to maximize
the virtual core usage so that the job can be finished faster. Looking
forward to hear from you

Regards,
Sairam Kannan

Re: Regarding Yarn virtual core configuration.

[Sandeep Deshmukh <sa...@datatorrent.com>]

Set it to 16 -X.

X could be the processes that are long running on your machine[1].

yarn.nodemanager.resource.cpu-vcores


The number of cores that a node manager can allocate to containers is
controlled by the yarn.nodemanager.resource.cpu-vcores property. It should
be set to the total number of cores on the machine, minus a core for each
daemon process running on the machine (datanode, node manager, and any
other long-running processes).

[1] http://crazyadmins.com/tag/tuning-yarn-to-get-maximum-performance/

Regards,
Sandeep

On Sat, Dec 10, 2016 at 2:24 PM, Sairam Kannan <sk...@hawk.iit.edu>
wrote:

> Hi all,
>                  I have a doubt in setting the Yarn's virtual core
> configuration. I have an Intel CPU slaves with hyperthreading enabled. The
> total number of physical core is 8. Logical core is twice the physical core
> ( 8 * 2 = 16 ). While setting the "yarn.nodemanager.resource.cpu-vcores"
> property, which value should I give ( 8 or 16 ) ?. I am trying to maximize
> the virtual core usage so that the job can be finished faster. Looking
> forward to hear from you
>
> Regards,
> Sairam Kannan
>