You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by André Hacker <an...@gmail.com> on 2013/10/03 18:57:28 UTC
Non data-local scheduling
Hi,
I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity
scheduler (default settings for scheduler) and replication factor 3.
I have exclusive access to the cluster to run a benchmark job and I wonder
why there are so few data-local and so many rack-local maps.
The input format calculates 44 input splits and 44 map tasks, however, it
seems to be random how many of them are processed data locally. Here the
counters of my last tries:
data-local / rack-local:
Test 1: data-local:15 rack-local: 29
Test 2: data-local:18 rack-local: 26
I don't understand why there is not always 100% data local. This should not
be a problem since the blocks of my input file are distributed over all
nodes.
Maybe someone can give me a hint.
Thanks,
André Hacker, TU Berlin
Re: Non data-local scheduling
Posted by Chris Mawata <ch...@gmail.com>.
Try playing with the block size vs split size. If the blocks are very
large and the splits small then multiple splits correspond to the same
block and if there are more splits than replicas you get rack local
processing.
On 10/3/2013 12:57 PM, André Hacker wrote:
> Hi,
>
> I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity
> scheduler (default settings for scheduler) and replication factor 3.
>
> I have exclusive access to the cluster to run a benchmark job and I
> wonder why there are so few data-local and so many rack-local maps.
>
> The input format calculates 44 input splits and 44 map tasks, however,
> it seems to be random how many of them are processed data locally.
> Here the counters of my last tries:
>
> data-local / rack-local:
> Test 1: data-local:15 rack-local: 29
> Test 2: data-local:18 rack-local: 26
>
> I don't understand why there is not always 100% data local. This
> should not be a problem since the blocks of my input file are
> distributed over all nodes.
>
> Maybe someone can give me a hint.
>
> Thanks,
> André Hacker, TU Berlin
RE: Non data-local scheduling
Posted by John Lilley <jo...@redpoint.net>.
Is this option set on a per-application-instance basis or is it a cluster-wide setting (or both)?
Is this a MapReduce-specific issue, or a YARN issue?
I don't understand how the problem arises in the first place. For example, if I have an idle cluster with 10 nodes and each node has four containers available at the requested capacity, and my YARN application requests 20 containers with node affinity such that I'm desiring two containers per node , why wouldn't YARN give me exactly the task mapping that I requested?
Thanks,
John
From: Sandy Ryza [mailto:sandy.ryza@cloudera.com]
Sent: Thursday, October 03, 2013 12:32 PM
To: user@hadoop.apache.org
Subject: Re: Non data-local scheduling
Ah, I was going off the Fair Scheduler equivalent, didn't realize they were different. In that case you might try setting it to something like half the nodes in the cluster.
Nodes are constantly heartbeating to the Resource Manager. When a node heartbeats, the scheduler checks to see whether the node has any free space, and, if it does, offers it to an application. From the application's perspective, this offer is a "scheduling opportunity". Each application will pass up yarn.scheduler.capacity.node-locality-delay before placing a container on a non-local node.
-Sandy
On Thu, Oct 3, 2013 at 10:36 AM, André Hacker <an...@gmail.com>> wrote:
Thanks, but I can't set this to a fraction, it wants to see an integer.
My documentation is slightly different:
"Number of missed scheduling opportunities after which the CapacityScheduler attempts to schedule rack-local containers. Typically this should be set to number of racks in the cluster, this feature is disabled by default, set to -1."
I set it to 1 and now I had 33 data local and 11 rack local tasks, which is a better, but still not optimal.
Couldn't find a good description of what this feature means (what is a scheduling opportunity, how many are there?). It does not seem to be in the current documentation http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html
2013/10/3 Sandy Ryza <sa...@cloudera.com>>
Hi Andre,
Try setting yarn.scheduler.capacity.node-locality-delay to a number between 0 and 1. This will turn on delay scheduling - here's the doc on how this works:
For applications that request containers on particular nodes, the number of scheduling opportunities since the last container assignment to wait before accepting a placement on another node. Expressed as a float between 0 and 1, which, as a fraction of the cluster size, is the number of scheduling opportunities to pass up. The default value of -1.0 means don't pass up any scheduling opportunities.
-Sandy
On Thu, Oct 3, 2013 at 9:57 AM, André Hacker <an...@gmail.com>> wrote:
Hi,
I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity scheduler (default settings for scheduler) and replication factor 3.
I have exclusive access to the cluster to run a benchmark job and I wonder why there are so few data-local and so many rack-local maps.
The input format calculates 44 input splits and 44 map tasks, however, it seems to be random how many of them are processed data locally. Here the counters of my last tries:
data-local / rack-local:
Test 1: data-local:15 rack-local: 29
Test 2: data-local:18 rack-local: 26
I don't understand why there is not always 100% data local. This should not be a problem since the blocks of my input file are distributed over all nodes.
Maybe someone can give me a hint.
Thanks,
André Hacker, TU Berlin
RE: Non data-local scheduling
Posted by John Lilley <jo...@redpoint.net>.
Is this option set on a per-application-instance basis or is it a cluster-wide setting (or both)?
Is this a MapReduce-specific issue, or a YARN issue?
I don't understand how the problem arises in the first place. For example, if I have an idle cluster with 10 nodes and each node has four containers available at the requested capacity, and my YARN application requests 20 containers with node affinity such that I'm desiring two containers per node , why wouldn't YARN give me exactly the task mapping that I requested?
Thanks,
John
From: Sandy Ryza [mailto:sandy.ryza@cloudera.com]
Sent: Thursday, October 03, 2013 12:32 PM
To: user@hadoop.apache.org
Subject: Re: Non data-local scheduling
Ah, I was going off the Fair Scheduler equivalent, didn't realize they were different. In that case you might try setting it to something like half the nodes in the cluster.
Nodes are constantly heartbeating to the Resource Manager. When a node heartbeats, the scheduler checks to see whether the node has any free space, and, if it does, offers it to an application. From the application's perspective, this offer is a "scheduling opportunity". Each application will pass up yarn.scheduler.capacity.node-locality-delay before placing a container on a non-local node.
-Sandy
On Thu, Oct 3, 2013 at 10:36 AM, André Hacker <an...@gmail.com>> wrote:
Thanks, but I can't set this to a fraction, it wants to see an integer.
My documentation is slightly different:
"Number of missed scheduling opportunities after which the CapacityScheduler attempts to schedule rack-local containers. Typically this should be set to number of racks in the cluster, this feature is disabled by default, set to -1."
I set it to 1 and now I had 33 data local and 11 rack local tasks, which is a better, but still not optimal.
Couldn't find a good description of what this feature means (what is a scheduling opportunity, how many are there?). It does not seem to be in the current documentation http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html
2013/10/3 Sandy Ryza <sa...@cloudera.com>>
Hi Andre,
Try setting yarn.scheduler.capacity.node-locality-delay to a number between 0 and 1. This will turn on delay scheduling - here's the doc on how this works:
For applications that request containers on particular nodes, the number of scheduling opportunities since the last container assignment to wait before accepting a placement on another node. Expressed as a float between 0 and 1, which, as a fraction of the cluster size, is the number of scheduling opportunities to pass up. The default value of -1.0 means don't pass up any scheduling opportunities.
-Sandy
On Thu, Oct 3, 2013 at 9:57 AM, André Hacker <an...@gmail.com>> wrote:
Hi,
I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity scheduler (default settings for scheduler) and replication factor 3.
I have exclusive access to the cluster to run a benchmark job and I wonder why there are so few data-local and so many rack-local maps.
The input format calculates 44 input splits and 44 map tasks, however, it seems to be random how many of them are processed data locally. Here the counters of my last tries:
data-local / rack-local:
Test 1: data-local:15 rack-local: 29
Test 2: data-local:18 rack-local: 26
I don't understand why there is not always 100% data local. This should not be a problem since the blocks of my input file are distributed over all nodes.
Maybe someone can give me a hint.
Thanks,
André Hacker, TU Berlin
RE: Non data-local scheduling
Posted by John Lilley <jo...@redpoint.net>.
Is this option set on a per-application-instance basis or is it a cluster-wide setting (or both)?
Is this a MapReduce-specific issue, or a YARN issue?
I don't understand how the problem arises in the first place. For example, if I have an idle cluster with 10 nodes and each node has four containers available at the requested capacity, and my YARN application requests 20 containers with node affinity such that I'm desiring two containers per node , why wouldn't YARN give me exactly the task mapping that I requested?
Thanks,
John
From: Sandy Ryza [mailto:sandy.ryza@cloudera.com]
Sent: Thursday, October 03, 2013 12:32 PM
To: user@hadoop.apache.org
Subject: Re: Non data-local scheduling
Ah, I was going off the Fair Scheduler equivalent, didn't realize they were different. In that case you might try setting it to something like half the nodes in the cluster.
Nodes are constantly heartbeating to the Resource Manager. When a node heartbeats, the scheduler checks to see whether the node has any free space, and, if it does, offers it to an application. From the application's perspective, this offer is a "scheduling opportunity". Each application will pass up yarn.scheduler.capacity.node-locality-delay before placing a container on a non-local node.
-Sandy
On Thu, Oct 3, 2013 at 10:36 AM, André Hacker <an...@gmail.com>> wrote:
Thanks, but I can't set this to a fraction, it wants to see an integer.
My documentation is slightly different:
"Number of missed scheduling opportunities after which the CapacityScheduler attempts to schedule rack-local containers. Typically this should be set to number of racks in the cluster, this feature is disabled by default, set to -1."
I set it to 1 and now I had 33 data local and 11 rack local tasks, which is a better, but still not optimal.
Couldn't find a good description of what this feature means (what is a scheduling opportunity, how many are there?). It does not seem to be in the current documentation http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html
2013/10/3 Sandy Ryza <sa...@cloudera.com>>
Hi Andre,
Try setting yarn.scheduler.capacity.node-locality-delay to a number between 0 and 1. This will turn on delay scheduling - here's the doc on how this works:
For applications that request containers on particular nodes, the number of scheduling opportunities since the last container assignment to wait before accepting a placement on another node. Expressed as a float between 0 and 1, which, as a fraction of the cluster size, is the number of scheduling opportunities to pass up. The default value of -1.0 means don't pass up any scheduling opportunities.
-Sandy
On Thu, Oct 3, 2013 at 9:57 AM, André Hacker <an...@gmail.com>> wrote:
Hi,
I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity scheduler (default settings for scheduler) and replication factor 3.
I have exclusive access to the cluster to run a benchmark job and I wonder why there are so few data-local and so many rack-local maps.
The input format calculates 44 input splits and 44 map tasks, however, it seems to be random how many of them are processed data locally. Here the counters of my last tries:
data-local / rack-local:
Test 1: data-local:15 rack-local: 29
Test 2: data-local:18 rack-local: 26
I don't understand why there is not always 100% data local. This should not be a problem since the blocks of my input file are distributed over all nodes.
Maybe someone can give me a hint.
Thanks,
André Hacker, TU Berlin
RE: Non data-local scheduling
Posted by John Lilley <jo...@redpoint.net>.
Is this option set on a per-application-instance basis or is it a cluster-wide setting (or both)?
Is this a MapReduce-specific issue, or a YARN issue?
I don't understand how the problem arises in the first place. For example, if I have an idle cluster with 10 nodes and each node has four containers available at the requested capacity, and my YARN application requests 20 containers with node affinity such that I'm desiring two containers per node , why wouldn't YARN give me exactly the task mapping that I requested?
Thanks,
John
From: Sandy Ryza [mailto:sandy.ryza@cloudera.com]
Sent: Thursday, October 03, 2013 12:32 PM
To: user@hadoop.apache.org
Subject: Re: Non data-local scheduling
Ah, I was going off the Fair Scheduler equivalent, didn't realize they were different. In that case you might try setting it to something like half the nodes in the cluster.
Nodes are constantly heartbeating to the Resource Manager. When a node heartbeats, the scheduler checks to see whether the node has any free space, and, if it does, offers it to an application. From the application's perspective, this offer is a "scheduling opportunity". Each application will pass up yarn.scheduler.capacity.node-locality-delay before placing a container on a non-local node.
-Sandy
On Thu, Oct 3, 2013 at 10:36 AM, André Hacker <an...@gmail.com>> wrote:
Thanks, but I can't set this to a fraction, it wants to see an integer.
My documentation is slightly different:
"Number of missed scheduling opportunities after which the CapacityScheduler attempts to schedule rack-local containers. Typically this should be set to number of racks in the cluster, this feature is disabled by default, set to -1."
I set it to 1 and now I had 33 data local and 11 rack local tasks, which is a better, but still not optimal.
Couldn't find a good description of what this feature means (what is a scheduling opportunity, how many are there?). It does not seem to be in the current documentation http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html
2013/10/3 Sandy Ryza <sa...@cloudera.com>>
Hi Andre,
Try setting yarn.scheduler.capacity.node-locality-delay to a number between 0 and 1. This will turn on delay scheduling - here's the doc on how this works:
For applications that request containers on particular nodes, the number of scheduling opportunities since the last container assignment to wait before accepting a placement on another node. Expressed as a float between 0 and 1, which, as a fraction of the cluster size, is the number of scheduling opportunities to pass up. The default value of -1.0 means don't pass up any scheduling opportunities.
-Sandy
On Thu, Oct 3, 2013 at 9:57 AM, André Hacker <an...@gmail.com>> wrote:
Hi,
I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity scheduler (default settings for scheduler) and replication factor 3.
I have exclusive access to the cluster to run a benchmark job and I wonder why there are so few data-local and so many rack-local maps.
The input format calculates 44 input splits and 44 map tasks, however, it seems to be random how many of them are processed data locally. Here the counters of my last tries:
data-local / rack-local:
Test 1: data-local:15 rack-local: 29
Test 2: data-local:18 rack-local: 26
I don't understand why there is not always 100% data local. This should not be a problem since the blocks of my input file are distributed over all nodes.
Maybe someone can give me a hint.
Thanks,
André Hacker, TU Berlin
Re: Non data-local scheduling
Posted by Sandy Ryza <sa...@cloudera.com>.
Ah, I was going off the Fair Scheduler equivalent, didn't realize they were
different. In that case you might try setting it to something like half
the nodes in the cluster.
Nodes are constantly heartbeating to the Resource Manager. When a node
heartbeats, the scheduler checks to see whether the node has any free
space, and, if it does, offers it to an application. From the
application's perspective, this offer is a "scheduling opportunity". Each
application will pass up yarn.scheduler.capacity.node-locality-delay before
placing a container on a non-local node.
-Sandy
On Thu, Oct 3, 2013 at 10:36 AM, André Hacker <an...@gmail.com>wrote:
> Thanks, but I can't set this to a fraction, it wants to see an integer.
> My documentation is slightly different:
> "Number of missed scheduling opportunities after which the
> CapacityScheduler attempts to schedule rack-local containers. Typically
> this should be set to number of racks in the cluster, this feature is
> disabled by default, set to -1."
>
> I set it to 1 and now I had 33 data local and 11 rack local tasks, which
> is a better, but still not optimal.
>
> Couldn't find a good description of what this feature means (what is a
> scheduling opportunity, how many are there?). It does not seem to be in the
> current documentation
> http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html
>
>
> 2013/10/3 Sandy Ryza <sa...@cloudera.com>
>
>> Hi Andre,
>>
>> Try setting yarn.scheduler.capacity.node-locality-delay to a number
>> between 0 and 1. This will turn on delay scheduling - here's the doc on
>> how this works:
>>
>> For applications that request containers on particular nodes, the number
>> of scheduling opportunities since the last container assignment to wait
>> before accepting a placement on another node. Expressed as a float between
>> 0 and 1, which, as a fraction of the cluster size, is the number of
>> scheduling opportunities to pass up. The default value of -1.0 means don't
>> pass up any scheduling opportunities.
>>
>> -Sandy
>>
>>
>> On Thu, Oct 3, 2013 at 9:57 AM, André Hacker <an...@gmail.com>wrote:
>>
>>> Hi,
>>>
>>> I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity
>>> scheduler (default settings for scheduler) and replication factor 3.
>>>
>>> I have exclusive access to the cluster to run a benchmark job and I
>>> wonder why there are so few data-local and so many rack-local maps.
>>>
>>> The input format calculates 44 input splits and 44 map tasks, however,
>>> it seems to be random how many of them are processed data locally. Here the
>>> counters of my last tries:
>>>
>>> data-local / rack-local:
>>> Test 1: data-local:15 rack-local: 29
>>> Test 2: data-local:18 rack-local: 26
>>>
>>> I don't understand why there is not always 100% data local. This should
>>> not be a problem since the blocks of my input file are distributed over all
>>> nodes.
>>>
>>> Maybe someone can give me a hint.
>>>
>>> Thanks,
>>> André Hacker, TU Berlin
>>>
>>
>>
>
Re: Non data-local scheduling
Posted by Sandy Ryza <sa...@cloudera.com>.
Ah, I was going off the Fair Scheduler equivalent, didn't realize they were
different. In that case you might try setting it to something like half
the nodes in the cluster.
Nodes are constantly heartbeating to the Resource Manager. When a node
heartbeats, the scheduler checks to see whether the node has any free
space, and, if it does, offers it to an application. From the
application's perspective, this offer is a "scheduling opportunity". Each
application will pass up yarn.scheduler.capacity.node-locality-delay before
placing a container on a non-local node.
-Sandy
On Thu, Oct 3, 2013 at 10:36 AM, André Hacker <an...@gmail.com>wrote:
> Thanks, but I can't set this to a fraction, it wants to see an integer.
> My documentation is slightly different:
> "Number of missed scheduling opportunities after which the
> CapacityScheduler attempts to schedule rack-local containers. Typically
> this should be set to number of racks in the cluster, this feature is
> disabled by default, set to -1."
>
> I set it to 1 and now I had 33 data local and 11 rack local tasks, which
> is a better, but still not optimal.
>
> Couldn't find a good description of what this feature means (what is a
> scheduling opportunity, how many are there?). It does not seem to be in the
> current documentation
> http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html
>
>
> 2013/10/3 Sandy Ryza <sa...@cloudera.com>
>
>> Hi Andre,
>>
>> Try setting yarn.scheduler.capacity.node-locality-delay to a number
>> between 0 and 1. This will turn on delay scheduling - here's the doc on
>> how this works:
>>
>> For applications that request containers on particular nodes, the number
>> of scheduling opportunities since the last container assignment to wait
>> before accepting a placement on another node. Expressed as a float between
>> 0 and 1, which, as a fraction of the cluster size, is the number of
>> scheduling opportunities to pass up. The default value of -1.0 means don't
>> pass up any scheduling opportunities.
>>
>> -Sandy
>>
>>
>> On Thu, Oct 3, 2013 at 9:57 AM, André Hacker <an...@gmail.com>wrote:
>>
>>> Hi,
>>>
>>> I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity
>>> scheduler (default settings for scheduler) and replication factor 3.
>>>
>>> I have exclusive access to the cluster to run a benchmark job and I
>>> wonder why there are so few data-local and so many rack-local maps.
>>>
>>> The input format calculates 44 input splits and 44 map tasks, however,
>>> it seems to be random how many of them are processed data locally. Here the
>>> counters of my last tries:
>>>
>>> data-local / rack-local:
>>> Test 1: data-local:15 rack-local: 29
>>> Test 2: data-local:18 rack-local: 26
>>>
>>> I don't understand why there is not always 100% data local. This should
>>> not be a problem since the blocks of my input file are distributed over all
>>> nodes.
>>>
>>> Maybe someone can give me a hint.
>>>
>>> Thanks,
>>> André Hacker, TU Berlin
>>>
>>
>>
>
Re: Non data-local scheduling
Posted by Sandy Ryza <sa...@cloudera.com>.
Ah, I was going off the Fair Scheduler equivalent, didn't realize they were
different. In that case you might try setting it to something like half
the nodes in the cluster.
Nodes are constantly heartbeating to the Resource Manager. When a node
heartbeats, the scheduler checks to see whether the node has any free
space, and, if it does, offers it to an application. From the
application's perspective, this offer is a "scheduling opportunity". Each
application will pass up yarn.scheduler.capacity.node-locality-delay before
placing a container on a non-local node.
-Sandy
On Thu, Oct 3, 2013 at 10:36 AM, André Hacker <an...@gmail.com>wrote:
> Thanks, but I can't set this to a fraction, it wants to see an integer.
> My documentation is slightly different:
> "Number of missed scheduling opportunities after which the
> CapacityScheduler attempts to schedule rack-local containers. Typically
> this should be set to number of racks in the cluster, this feature is
> disabled by default, set to -1."
>
> I set it to 1 and now I had 33 data local and 11 rack local tasks, which
> is a better, but still not optimal.
>
> Couldn't find a good description of what this feature means (what is a
> scheduling opportunity, how many are there?). It does not seem to be in the
> current documentation
> http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html
>
>
> 2013/10/3 Sandy Ryza <sa...@cloudera.com>
>
>> Hi Andre,
>>
>> Try setting yarn.scheduler.capacity.node-locality-delay to a number
>> between 0 and 1. This will turn on delay scheduling - here's the doc on
>> how this works:
>>
>> For applications that request containers on particular nodes, the number
>> of scheduling opportunities since the last container assignment to wait
>> before accepting a placement on another node. Expressed as a float between
>> 0 and 1, which, as a fraction of the cluster size, is the number of
>> scheduling opportunities to pass up. The default value of -1.0 means don't
>> pass up any scheduling opportunities.
>>
>> -Sandy
>>
>>
>> On Thu, Oct 3, 2013 at 9:57 AM, André Hacker <an...@gmail.com>wrote:
>>
>>> Hi,
>>>
>>> I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity
>>> scheduler (default settings for scheduler) and replication factor 3.
>>>
>>> I have exclusive access to the cluster to run a benchmark job and I
>>> wonder why there are so few data-local and so many rack-local maps.
>>>
>>> The input format calculates 44 input splits and 44 map tasks, however,
>>> it seems to be random how many of them are processed data locally. Here the
>>> counters of my last tries:
>>>
>>> data-local / rack-local:
>>> Test 1: data-local:15 rack-local: 29
>>> Test 2: data-local:18 rack-local: 26
>>>
>>> I don't understand why there is not always 100% data local. This should
>>> not be a problem since the blocks of my input file are distributed over all
>>> nodes.
>>>
>>> Maybe someone can give me a hint.
>>>
>>> Thanks,
>>> André Hacker, TU Berlin
>>>
>>
>>
>
Re: Non data-local scheduling
Posted by Sandy Ryza <sa...@cloudera.com>.
Ah, I was going off the Fair Scheduler equivalent, didn't realize they were
different. In that case you might try setting it to something like half
the nodes in the cluster.
Nodes are constantly heartbeating to the Resource Manager. When a node
heartbeats, the scheduler checks to see whether the node has any free
space, and, if it does, offers it to an application. From the
application's perspective, this offer is a "scheduling opportunity". Each
application will pass up yarn.scheduler.capacity.node-locality-delay before
placing a container on a non-local node.
-Sandy
On Thu, Oct 3, 2013 at 10:36 AM, André Hacker <an...@gmail.com>wrote:
> Thanks, but I can't set this to a fraction, it wants to see an integer.
> My documentation is slightly different:
> "Number of missed scheduling opportunities after which the
> CapacityScheduler attempts to schedule rack-local containers. Typically
> this should be set to number of racks in the cluster, this feature is
> disabled by default, set to -1."
>
> I set it to 1 and now I had 33 data local and 11 rack local tasks, which
> is a better, but still not optimal.
>
> Couldn't find a good description of what this feature means (what is a
> scheduling opportunity, how many are there?). It does not seem to be in the
> current documentation
> http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html
>
>
> 2013/10/3 Sandy Ryza <sa...@cloudera.com>
>
>> Hi Andre,
>>
>> Try setting yarn.scheduler.capacity.node-locality-delay to a number
>> between 0 and 1. This will turn on delay scheduling - here's the doc on
>> how this works:
>>
>> For applications that request containers on particular nodes, the number
>> of scheduling opportunities since the last container assignment to wait
>> before accepting a placement on another node. Expressed as a float between
>> 0 and 1, which, as a fraction of the cluster size, is the number of
>> scheduling opportunities to pass up. The default value of -1.0 means don't
>> pass up any scheduling opportunities.
>>
>> -Sandy
>>
>>
>> On Thu, Oct 3, 2013 at 9:57 AM, André Hacker <an...@gmail.com>wrote:
>>
>>> Hi,
>>>
>>> I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity
>>> scheduler (default settings for scheduler) and replication factor 3.
>>>
>>> I have exclusive access to the cluster to run a benchmark job and I
>>> wonder why there are so few data-local and so many rack-local maps.
>>>
>>> The input format calculates 44 input splits and 44 map tasks, however,
>>> it seems to be random how many of them are processed data locally. Here the
>>> counters of my last tries:
>>>
>>> data-local / rack-local:
>>> Test 1: data-local:15 rack-local: 29
>>> Test 2: data-local:18 rack-local: 26
>>>
>>> I don't understand why there is not always 100% data local. This should
>>> not be a problem since the blocks of my input file are distributed over all
>>> nodes.
>>>
>>> Maybe someone can give me a hint.
>>>
>>> Thanks,
>>> André Hacker, TU Berlin
>>>
>>
>>
>
Re: Non data-local scheduling
Posted by André Hacker <an...@gmail.com>.
Thanks, but I can't set this to a fraction, it wants to see an integer.
My documentation is slightly different:
"Number of missed scheduling opportunities after which the
CapacityScheduler attempts to schedule rack-local containers. Typically
this should be set to number of racks in the cluster, this feature is
disabled by default, set to -1."
I set it to 1 and now I had 33 data local and 11 rack local tasks, which is
a better, but still not optimal.
Couldn't find a good description of what this feature means (what is a
scheduling opportunity, how many are there?). It does not seem to be in the
current documentation
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html
2013/10/3 Sandy Ryza <sa...@cloudera.com>
> Hi Andre,
>
> Try setting yarn.scheduler.capacity.node-locality-delay to a number
> between 0 and 1. This will turn on delay scheduling - here's the doc on
> how this works:
>
> For applications that request containers on particular nodes, the number
> of scheduling opportunities since the last container assignment to wait
> before accepting a placement on another node. Expressed as a float between
> 0 and 1, which, as a fraction of the cluster size, is the number of
> scheduling opportunities to pass up. The default value of -1.0 means don't
> pass up any scheduling opportunities.
>
> -Sandy
>
>
> On Thu, Oct 3, 2013 at 9:57 AM, André Hacker <an...@gmail.com>wrote:
>
>> Hi,
>>
>> I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity
>> scheduler (default settings for scheduler) and replication factor 3.
>>
>> I have exclusive access to the cluster to run a benchmark job and I
>> wonder why there are so few data-local and so many rack-local maps.
>>
>> The input format calculates 44 input splits and 44 map tasks, however, it
>> seems to be random how many of them are processed data locally. Here the
>> counters of my last tries:
>>
>> data-local / rack-local:
>> Test 1: data-local:15 rack-local: 29
>> Test 2: data-local:18 rack-local: 26
>>
>> I don't understand why there is not always 100% data local. This should
>> not be a problem since the blocks of my input file are distributed over all
>> nodes.
>>
>> Maybe someone can give me a hint.
>>
>> Thanks,
>> André Hacker, TU Berlin
>>
>
>
Re: Non data-local scheduling
Posted by André Hacker <an...@gmail.com>.
Thanks, but I can't set this to a fraction, it wants to see an integer.
My documentation is slightly different:
"Number of missed scheduling opportunities after which the
CapacityScheduler attempts to schedule rack-local containers. Typically
this should be set to number of racks in the cluster, this feature is
disabled by default, set to -1."
I set it to 1 and now I had 33 data local and 11 rack local tasks, which is
a better, but still not optimal.
Couldn't find a good description of what this feature means (what is a
scheduling opportunity, how many are there?). It does not seem to be in the
current documentation
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html
2013/10/3 Sandy Ryza <sa...@cloudera.com>
> Hi Andre,
>
> Try setting yarn.scheduler.capacity.node-locality-delay to a number
> between 0 and 1. This will turn on delay scheduling - here's the doc on
> how this works:
>
> For applications that request containers on particular nodes, the number
> of scheduling opportunities since the last container assignment to wait
> before accepting a placement on another node. Expressed as a float between
> 0 and 1, which, as a fraction of the cluster size, is the number of
> scheduling opportunities to pass up. The default value of -1.0 means don't
> pass up any scheduling opportunities.
>
> -Sandy
>
>
> On Thu, Oct 3, 2013 at 9:57 AM, André Hacker <an...@gmail.com>wrote:
>
>> Hi,
>>
>> I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity
>> scheduler (default settings for scheduler) and replication factor 3.
>>
>> I have exclusive access to the cluster to run a benchmark job and I
>> wonder why there are so few data-local and so many rack-local maps.
>>
>> The input format calculates 44 input splits and 44 map tasks, however, it
>> seems to be random how many of them are processed data locally. Here the
>> counters of my last tries:
>>
>> data-local / rack-local:
>> Test 1: data-local:15 rack-local: 29
>> Test 2: data-local:18 rack-local: 26
>>
>> I don't understand why there is not always 100% data local. This should
>> not be a problem since the blocks of my input file are distributed over all
>> nodes.
>>
>> Maybe someone can give me a hint.
>>
>> Thanks,
>> André Hacker, TU Berlin
>>
>
>
Re: Non data-local scheduling
Posted by André Hacker <an...@gmail.com>.
Thanks, but I can't set this to a fraction, it wants to see an integer.
My documentation is slightly different:
"Number of missed scheduling opportunities after which the
CapacityScheduler attempts to schedule rack-local containers. Typically
this should be set to number of racks in the cluster, this feature is
disabled by default, set to -1."
I set it to 1 and now I had 33 data local and 11 rack local tasks, which is
a better, but still not optimal.
Couldn't find a good description of what this feature means (what is a
scheduling opportunity, how many are there?). It does not seem to be in the
current documentation
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html
2013/10/3 Sandy Ryza <sa...@cloudera.com>
> Hi Andre,
>
> Try setting yarn.scheduler.capacity.node-locality-delay to a number
> between 0 and 1. This will turn on delay scheduling - here's the doc on
> how this works:
>
> For applications that request containers on particular nodes, the number
> of scheduling opportunities since the last container assignment to wait
> before accepting a placement on another node. Expressed as a float between
> 0 and 1, which, as a fraction of the cluster size, is the number of
> scheduling opportunities to pass up. The default value of -1.0 means don't
> pass up any scheduling opportunities.
>
> -Sandy
>
>
> On Thu, Oct 3, 2013 at 9:57 AM, André Hacker <an...@gmail.com>wrote:
>
>> Hi,
>>
>> I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity
>> scheduler (default settings for scheduler) and replication factor 3.
>>
>> I have exclusive access to the cluster to run a benchmark job and I
>> wonder why there are so few data-local and so many rack-local maps.
>>
>> The input format calculates 44 input splits and 44 map tasks, however, it
>> seems to be random how many of them are processed data locally. Here the
>> counters of my last tries:
>>
>> data-local / rack-local:
>> Test 1: data-local:15 rack-local: 29
>> Test 2: data-local:18 rack-local: 26
>>
>> I don't understand why there is not always 100% data local. This should
>> not be a problem since the blocks of my input file are distributed over all
>> nodes.
>>
>> Maybe someone can give me a hint.
>>
>> Thanks,
>> André Hacker, TU Berlin
>>
>
>
Re: Non data-local scheduling
Posted by André Hacker <an...@gmail.com>.
Thanks, but I can't set this to a fraction, it wants to see an integer.
My documentation is slightly different:
"Number of missed scheduling opportunities after which the
CapacityScheduler attempts to schedule rack-local containers. Typically
this should be set to number of racks in the cluster, this feature is
disabled by default, set to -1."
I set it to 1 and now I had 33 data local and 11 rack local tasks, which is
a better, but still not optimal.
Couldn't find a good description of what this feature means (what is a
scheduling opportunity, how many are there?). It does not seem to be in the
current documentation
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html
2013/10/3 Sandy Ryza <sa...@cloudera.com>
> Hi Andre,
>
> Try setting yarn.scheduler.capacity.node-locality-delay to a number
> between 0 and 1. This will turn on delay scheduling - here's the doc on
> how this works:
>
> For applications that request containers on particular nodes, the number
> of scheduling opportunities since the last container assignment to wait
> before accepting a placement on another node. Expressed as a float between
> 0 and 1, which, as a fraction of the cluster size, is the number of
> scheduling opportunities to pass up. The default value of -1.0 means don't
> pass up any scheduling opportunities.
>
> -Sandy
>
>
> On Thu, Oct 3, 2013 at 9:57 AM, André Hacker <an...@gmail.com>wrote:
>
>> Hi,
>>
>> I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity
>> scheduler (default settings for scheduler) and replication factor 3.
>>
>> I have exclusive access to the cluster to run a benchmark job and I
>> wonder why there are so few data-local and so many rack-local maps.
>>
>> The input format calculates 44 input splits and 44 map tasks, however, it
>> seems to be random how many of them are processed data locally. Here the
>> counters of my last tries:
>>
>> data-local / rack-local:
>> Test 1: data-local:15 rack-local: 29
>> Test 2: data-local:18 rack-local: 26
>>
>> I don't understand why there is not always 100% data local. This should
>> not be a problem since the blocks of my input file are distributed over all
>> nodes.
>>
>> Maybe someone can give me a hint.
>>
>> Thanks,
>> André Hacker, TU Berlin
>>
>
>
Re: Non data-local scheduling
Posted by Sandy Ryza <sa...@cloudera.com>.
Hi Andre,
Try setting yarn.scheduler.capacity.node-locality-delay to a number between
0 and 1. This will turn on delay scheduling - here's the doc on how this
works:
For applications that request containers on particular nodes, the number of
scheduling opportunities since the last container assignment to wait before
accepting a placement on another node. Expressed as a float between 0 and
1, which, as a fraction of the cluster size, is the number of scheduling
opportunities to pass up. The default value of -1.0 means don't pass up any
scheduling opportunities.
-Sandy
On Thu, Oct 3, 2013 at 9:57 AM, André Hacker <an...@gmail.com> wrote:
> Hi,
>
> I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity
> scheduler (default settings for scheduler) and replication factor 3.
>
> I have exclusive access to the cluster to run a benchmark job and I wonder
> why there are so few data-local and so many rack-local maps.
>
> The input format calculates 44 input splits and 44 map tasks, however, it
> seems to be random how many of them are processed data locally. Here the
> counters of my last tries:
>
> data-local / rack-local:
> Test 1: data-local:15 rack-local: 29
> Test 2: data-local:18 rack-local: 26
>
> I don't understand why there is not always 100% data local. This should
> not be a problem since the blocks of my input file are distributed over all
> nodes.
>
> Maybe someone can give me a hint.
>
> Thanks,
> André Hacker, TU Berlin
>
Re: Non data-local scheduling
Posted by Sandy Ryza <sa...@cloudera.com>.
Hi Andre,
Try setting yarn.scheduler.capacity.node-locality-delay to a number between
0 and 1. This will turn on delay scheduling - here's the doc on how this
works:
For applications that request containers on particular nodes, the number of
scheduling opportunities since the last container assignment to wait before
accepting a placement on another node. Expressed as a float between 0 and
1, which, as a fraction of the cluster size, is the number of scheduling
opportunities to pass up. The default value of -1.0 means don't pass up any
scheduling opportunities.
-Sandy
On Thu, Oct 3, 2013 at 9:57 AM, André Hacker <an...@gmail.com> wrote:
> Hi,
>
> I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity
> scheduler (default settings for scheduler) and replication factor 3.
>
> I have exclusive access to the cluster to run a benchmark job and I wonder
> why there are so few data-local and so many rack-local maps.
>
> The input format calculates 44 input splits and 44 map tasks, however, it
> seems to be random how many of them are processed data locally. Here the
> counters of my last tries:
>
> data-local / rack-local:
> Test 1: data-local:15 rack-local: 29
> Test 2: data-local:18 rack-local: 26
>
> I don't understand why there is not always 100% data local. This should
> not be a problem since the blocks of my input file are distributed over all
> nodes.
>
> Maybe someone can give me a hint.
>
> Thanks,
> André Hacker, TU Berlin
>
Re: Non data-local scheduling
Posted by Sandy Ryza <sa...@cloudera.com>.
Hi Andre,
Try setting yarn.scheduler.capacity.node-locality-delay to a number between
0 and 1. This will turn on delay scheduling - here's the doc on how this
works:
For applications that request containers on particular nodes, the number of
scheduling opportunities since the last container assignment to wait before
accepting a placement on another node. Expressed as a float between 0 and
1, which, as a fraction of the cluster size, is the number of scheduling
opportunities to pass up. The default value of -1.0 means don't pass up any
scheduling opportunities.
-Sandy
On Thu, Oct 3, 2013 at 9:57 AM, André Hacker <an...@gmail.com> wrote:
> Hi,
>
> I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity
> scheduler (default settings for scheduler) and replication factor 3.
>
> I have exclusive access to the cluster to run a benchmark job and I wonder
> why there are so few data-local and so many rack-local maps.
>
> The input format calculates 44 input splits and 44 map tasks, however, it
> seems to be random how many of them are processed data locally. Here the
> counters of my last tries:
>
> data-local / rack-local:
> Test 1: data-local:15 rack-local: 29
> Test 2: data-local:18 rack-local: 26
>
> I don't understand why there is not always 100% data local. This should
> not be a problem since the blocks of my input file are distributed over all
> nodes.
>
> Maybe someone can give me a hint.
>
> Thanks,
> André Hacker, TU Berlin
>
Re: Non data-local scheduling
Posted by Chris Mawata <ch...@gmail.com>.
Try playing with the block size vs split size. If the blocks are very
large and the splits small then multiple splits correspond to the same
block and if there are more splits than replicas you get rack local
processing.
On 10/3/2013 12:57 PM, André Hacker wrote:
> Hi,
>
> I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity
> scheduler (default settings for scheduler) and replication factor 3.
>
> I have exclusive access to the cluster to run a benchmark job and I
> wonder why there are so few data-local and so many rack-local maps.
>
> The input format calculates 44 input splits and 44 map tasks, however,
> it seems to be random how many of them are processed data locally.
> Here the counters of my last tries:
>
> data-local / rack-local:
> Test 1: data-local:15 rack-local: 29
> Test 2: data-local:18 rack-local: 26
>
> I don't understand why there is not always 100% data local. This
> should not be a problem since the blocks of my input file are
> distributed over all nodes.
>
> Maybe someone can give me a hint.
>
> Thanks,
> André Hacker, TU Berlin
Re: Non data-local scheduling
Posted by Arun C Murthy <ac...@hortonworks.com>.
It's cluster-wide setting and scheduler-specific.
For CS please set yarn.scheduler.capacity.node-locality-delay to #machines you have in your rack (typically 20 or 40).
Looks like the doc in capacity-scheduler.xml is broken, would you mind opening a jira to fix it and add it to the the http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html?
thanks!
Arun
On Oct 3, 2013, at 9:57 AM, André Hacker <an...@gmail.com> wrote:
> Hi,
>
> I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity scheduler (default settings for scheduler) and replication factor 3.
>
> I have exclusive access to the cluster to run a benchmark job and I wonder why there are so few data-local and so many rack-local maps.
>
> The input format calculates 44 input splits and 44 map tasks, however, it seems to be random how many of them are processed data locally. Here the counters of my last tries:
>
> data-local / rack-local:
> Test 1: data-local:15 rack-local: 29
> Test 2: data-local:18 rack-local: 26
>
> I don't understand why there is not always 100% data local. This should not be a problem since the blocks of my input file are distributed over all nodes.
>
> Maybe someone can give me a hint.
>
> Thanks,
> André Hacker, TU Berlin
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Non data-local scheduling
Posted by Chris Mawata <ch...@gmail.com>.
Try playing with the block size vs split size. If the blocks are very
large and the splits small then multiple splits correspond to the same
block and if there are more splits than replicas you get rack local
processing.
On 10/3/2013 12:57 PM, André Hacker wrote:
> Hi,
>
> I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity
> scheduler (default settings for scheduler) and replication factor 3.
>
> I have exclusive access to the cluster to run a benchmark job and I
> wonder why there are so few data-local and so many rack-local maps.
>
> The input format calculates 44 input splits and 44 map tasks, however,
> it seems to be random how many of them are processed data locally.
> Here the counters of my last tries:
>
> data-local / rack-local:
> Test 1: data-local:15 rack-local: 29
> Test 2: data-local:18 rack-local: 26
>
> I don't understand why there is not always 100% data local. This
> should not be a problem since the blocks of my input file are
> distributed over all nodes.
>
> Maybe someone can give me a hint.
>
> Thanks,
> André Hacker, TU Berlin
Re: Non data-local scheduling
Posted by Arun C Murthy <ac...@hortonworks.com>.
It's cluster-wide setting and scheduler-specific.
For CS please set yarn.scheduler.capacity.node-locality-delay to #machines you have in your rack (typically 20 or 40).
Looks like the doc in capacity-scheduler.xml is broken, would you mind opening a jira to fix it and add it to the the http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html?
thanks!
Arun
On Oct 3, 2013, at 9:57 AM, André Hacker <an...@gmail.com> wrote:
> Hi,
>
> I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity scheduler (default settings for scheduler) and replication factor 3.
>
> I have exclusive access to the cluster to run a benchmark job and I wonder why there are so few data-local and so many rack-local maps.
>
> The input format calculates 44 input splits and 44 map tasks, however, it seems to be random how many of them are processed data locally. Here the counters of my last tries:
>
> data-local / rack-local:
> Test 1: data-local:15 rack-local: 29
> Test 2: data-local:18 rack-local: 26
>
> I don't understand why there is not always 100% data local. This should not be a problem since the blocks of my input file are distributed over all nodes.
>
> Maybe someone can give me a hint.
>
> Thanks,
> André Hacker, TU Berlin
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Non data-local scheduling
Posted by Chris Mawata <ch...@gmail.com>.
Try playing with the block size vs split size. If the blocks are very
large and the splits small then multiple splits correspond to the same
block and if there are more splits than replicas you get rack local
processing.
On 10/3/2013 12:57 PM, André Hacker wrote:
> Hi,
>
> I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity
> scheduler (default settings for scheduler) and replication factor 3.
>
> I have exclusive access to the cluster to run a benchmark job and I
> wonder why there are so few data-local and so many rack-local maps.
>
> The input format calculates 44 input splits and 44 map tasks, however,
> it seems to be random how many of them are processed data locally.
> Here the counters of my last tries:
>
> data-local / rack-local:
> Test 1: data-local:15 rack-local: 29
> Test 2: data-local:18 rack-local: 26
>
> I don't understand why there is not always 100% data local. This
> should not be a problem since the blocks of my input file are
> distributed over all nodes.
>
> Maybe someone can give me a hint.
>
> Thanks,
> André Hacker, TU Berlin
Re: Non data-local scheduling
Posted by Sandy Ryza <sa...@cloudera.com>.
Hi Andre,
Try setting yarn.scheduler.capacity.node-locality-delay to a number between
0 and 1. This will turn on delay scheduling - here's the doc on how this
works:
For applications that request containers on particular nodes, the number of
scheduling opportunities since the last container assignment to wait before
accepting a placement on another node. Expressed as a float between 0 and
1, which, as a fraction of the cluster size, is the number of scheduling
opportunities to pass up. The default value of -1.0 means don't pass up any
scheduling opportunities.
-Sandy
On Thu, Oct 3, 2013 at 9:57 AM, André Hacker <an...@gmail.com> wrote:
> Hi,
>
> I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity
> scheduler (default settings for scheduler) and replication factor 3.
>
> I have exclusive access to the cluster to run a benchmark job and I wonder
> why there are so few data-local and so many rack-local maps.
>
> The input format calculates 44 input splits and 44 map tasks, however, it
> seems to be random how many of them are processed data locally. Here the
> counters of my last tries:
>
> data-local / rack-local:
> Test 1: data-local:15 rack-local: 29
> Test 2: data-local:18 rack-local: 26
>
> I don't understand why there is not always 100% data local. This should
> not be a problem since the blocks of my input file are distributed over all
> nodes.
>
> Maybe someone can give me a hint.
>
> Thanks,
> André Hacker, TU Berlin
>
Re: Non data-local scheduling
Posted by Arun C Murthy <ac...@hortonworks.com>.
It's cluster-wide setting and scheduler-specific.
For CS please set yarn.scheduler.capacity.node-locality-delay to #machines you have in your rack (typically 20 or 40).
Looks like the doc in capacity-scheduler.xml is broken, would you mind opening a jira to fix it and add it to the the http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html?
thanks!
Arun
On Oct 3, 2013, at 9:57 AM, André Hacker <an...@gmail.com> wrote:
> Hi,
>
> I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity scheduler (default settings for scheduler) and replication factor 3.
>
> I have exclusive access to the cluster to run a benchmark job and I wonder why there are so few data-local and so many rack-local maps.
>
> The input format calculates 44 input splits and 44 map tasks, however, it seems to be random how many of them are processed data locally. Here the counters of my last tries:
>
> data-local / rack-local:
> Test 1: data-local:15 rack-local: 29
> Test 2: data-local:18 rack-local: 26
>
> I don't understand why there is not always 100% data local. This should not be a problem since the blocks of my input file are distributed over all nodes.
>
> Maybe someone can give me a hint.
>
> Thanks,
> André Hacker, TU Berlin
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Non data-local scheduling
Posted by Arun C Murthy <ac...@hortonworks.com>.
It's cluster-wide setting and scheduler-specific.
For CS please set yarn.scheduler.capacity.node-locality-delay to #machines you have in your rack (typically 20 or 40).
Looks like the doc in capacity-scheduler.xml is broken, would you mind opening a jira to fix it and add it to the the http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html?
thanks!
Arun
On Oct 3, 2013, at 9:57 AM, André Hacker <an...@gmail.com> wrote:
> Hi,
>
> I have a 25 node cluster, running hadoop 2.1.0-beta, with capacity scheduler (default settings for scheduler) and replication factor 3.
>
> I have exclusive access to the cluster to run a benchmark job and I wonder why there are so few data-local and so many rack-local maps.
>
> The input format calculates 44 input splits and 44 map tasks, however, it seems to be random how many of them are processed data locally. Here the counters of my last tries:
>
> data-local / rack-local:
> Test 1: data-local:15 rack-local: 29
> Test 2: data-local:18 rack-local: 26
>
> I don't understand why there is not always 100% data local. This should not be a problem since the blocks of my input file are distributed over all nodes.
>
> Maybe someone can give me a hint.
>
> Thanks,
> André Hacker, TU Berlin
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.