You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by YouPeng Yang <yy...@gmail.com> on 2013/01/28 14:13:41 UTC
what does the yarn.nodemanager.aux-services stand for
Hi All
I am wondering what the yarn.nodemanager.aux-services stands for 。
there even is not description about it in the yarn-default.xml。So i‘ve just
ignored it when i tried the example,as a result ,it failed。
I have do some google，however，everyone just setup it in the
yarn-site.xml,but no one tells the description about it。
please,helps will be appreciated .
.
Re: what does the yarn.nodemanager.aux-services stand for
Posted by YouPeng Yang <yy...@gmail.com>.
Hi Harsh
it is great helpful ,thank you.
2013/1/28 Harsh J <ha...@cloudera.com>
> Hi,
>
> The function of yarn.nodemanager.aux-services is to run any custom
> Service [1] implementations from the user, in the YARN NodeManager's
> runtime. The "aux" stands for auxiliary.
>
> One such example of where this comes useful is the ShuffleHandler [2]
> service, which helps MR do its shuffle work, without needing the NM
> itself to provide a shuffle service unnecessarily (in case MR is not
> required). Therefore, a ShuffleHandler is necessary to be configured
> if you need to run MR2 apps (jobs), but not otherwise. A clear
> difference, in case you're confused between MR2 and YARN, can be found
> at [3].
>
> Do feel free to ask any further questions you have.
>
> [1] -
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/Service.java?view=markup
> [2] -
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java?view=markup
> [3] - http://www.cloudera.com/blog/2012/10/mr2-and-yarn-briefly-explained/
>
> On Mon, Jan 28, 2013 at 6:43 PM, YouPeng Yang <yy...@gmail.com>
> wrote:
> > Hi All
> > I am wondering what the yarn.nodemanager.aux-services stands for 。
> > there even is not description about it in the yarn-default.xml。So i‘ve
> just
> > ignored it when i tried the example,as a result ,it failed。
> > I have do some google，however，everyone just setup it in the
> > yarn-site.xml,but no one tells the description about it。
> > please,helps will be appreciated .
> >
> > .
> >
>
>
>
> --
> Harsh J
>
Re: what does the yarn.nodemanager.aux-services stand for
Posted by YouPeng Yang <yy...@gmail.com>.
Hi Harsh
it is great helpful ,thank you.
2013/1/28 Harsh J <ha...@cloudera.com>
> Hi,
>
> The function of yarn.nodemanager.aux-services is to run any custom
> Service [1] implementations from the user, in the YARN NodeManager's
> runtime. The "aux" stands for auxiliary.
>
> One such example of where this comes useful is the ShuffleHandler [2]
> service, which helps MR do its shuffle work, without needing the NM
> itself to provide a shuffle service unnecessarily (in case MR is not
> required). Therefore, a ShuffleHandler is necessary to be configured
> if you need to run MR2 apps (jobs), but not otherwise. A clear
> difference, in case you're confused between MR2 and YARN, can be found
> at [3].
>
> Do feel free to ask any further questions you have.
>
> [1] -
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/Service.java?view=markup
> [2] -
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java?view=markup
> [3] - http://www.cloudera.com/blog/2012/10/mr2-and-yarn-briefly-explained/
>
> On Mon, Jan 28, 2013 at 6:43 PM, YouPeng Yang <yy...@gmail.com>
> wrote:
> > Hi All
> > I am wondering what the yarn.nodemanager.aux-services stands for 。
> > there even is not description about it in the yarn-default.xml。So i‘ve
> just
> > ignored it when i tried the example,as a result ,it failed。
> > I have do some google，however，everyone just setup it in the
> > yarn-site.xml,but no one tells the description about it。
> > please,helps will be appreciated .
> >
> > .
> >
>
>
>
> --
> Harsh J
>
Re: what does the yarn.nodemanager.aux-services stand for
Posted by YouPeng Yang <yy...@gmail.com>.
Hi Harsh
it is great helpful ,thank you.
2013/1/28 Harsh J <ha...@cloudera.com>
> Hi,
>
> The function of yarn.nodemanager.aux-services is to run any custom
> Service [1] implementations from the user, in the YARN NodeManager's
> runtime. The "aux" stands for auxiliary.
>
> One such example of where this comes useful is the ShuffleHandler [2]
> service, which helps MR do its shuffle work, without needing the NM
> itself to provide a shuffle service unnecessarily (in case MR is not
> required). Therefore, a ShuffleHandler is necessary to be configured
> if you need to run MR2 apps (jobs), but not otherwise. A clear
> difference, in case you're confused between MR2 and YARN, can be found
> at [3].
>
> Do feel free to ask any further questions you have.
>
> [1] -
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/Service.java?view=markup
> [2] -
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java?view=markup
> [3] - http://www.cloudera.com/blog/2012/10/mr2-and-yarn-briefly-explained/
>
> On Mon, Jan 28, 2013 at 6:43 PM, YouPeng Yang <yy...@gmail.com>
> wrote:
> > Hi All
> > I am wondering what the yarn.nodemanager.aux-services stands for 。
> > there even is not description about it in the yarn-default.xml。So i‘ve
> just
> > ignored it when i tried the example,as a result ,it failed。
> > I have do some google，however，everyone just setup it in the
> > yarn-site.xml,but no one tells the description about it。
> > please,helps will be appreciated .
> >
> > .
> >
>
>
>
> --
> Harsh J
>
Re: what does the yarn.nodemanager.aux-services stand for
Posted by YouPeng Yang <yy...@gmail.com>.
Hi Harsh
it is great helpful ,thank you.
2013/1/28 Harsh J <ha...@cloudera.com>
> Hi,
>
> The function of yarn.nodemanager.aux-services is to run any custom
> Service [1] implementations from the user, in the YARN NodeManager's
> runtime. The "aux" stands for auxiliary.
>
> One such example of where this comes useful is the ShuffleHandler [2]
> service, which helps MR do its shuffle work, without needing the NM
> itself to provide a shuffle service unnecessarily (in case MR is not
> required). Therefore, a ShuffleHandler is necessary to be configured
> if you need to run MR2 apps (jobs), but not otherwise. A clear
> difference, in case you're confused between MR2 and YARN, can be found
> at [3].
>
> Do feel free to ask any further questions you have.
>
> [1] -
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/Service.java?view=markup
> [2] -
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java?view=markup
> [3] - http://www.cloudera.com/blog/2012/10/mr2-and-yarn-briefly-explained/
>
> On Mon, Jan 28, 2013 at 6:43 PM, YouPeng Yang <yy...@gmail.com>
> wrote:
> > Hi All
> > I am wondering what the yarn.nodemanager.aux-services stands for 。
> > there even is not description about it in the yarn-default.xml。So i‘ve
> just
> > ignored it when i tried the example,as a result ,it failed。
> > I have do some google，however，everyone just setup it in the
> > yarn-site.xml,but no one tells the description about it。
> > please,helps will be appreciated .
> >
> > .
> >
>
>
>
> --
> Harsh J
>
Re: what does the yarn.nodemanager.aux-services stand for
Posted by Harsh J <ha...@cloudera.com>.
Hi,
The function of yarn.nodemanager.aux-services is to run any custom
Service [1] implementations from the user, in the YARN NodeManager's
runtime. The "aux" stands for auxiliary.
One such example of where this comes useful is the ShuffleHandler [2]
service, which helps MR do its shuffle work, without needing the NM
itself to provide a shuffle service unnecessarily (in case MR is not
required). Therefore, a ShuffleHandler is necessary to be configured
if you need to run MR2 apps (jobs), but not otherwise. A clear
difference, in case you're confused between MR2 and YARN, can be found
at [3].
Do feel free to ask any further questions you have.
[1] - http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/Service.java?view=markup
[2] - http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java?view=markup
[3] - http://www.cloudera.com/blog/2012/10/mr2-and-yarn-briefly-explained/
On Mon, Jan 28, 2013 at 6:43 PM, YouPeng Yang <yy...@gmail.com> wrote:
> Hi All
> I am wondering what the yarn.nodemanager.aux-services stands for 。
> there even is not description about it in the yarn-default.xml。So i‘ve just
> ignored it when i tried the example,as a result ,it failed。
> I have do some google，however，everyone just setup it in the
> yarn-site.xml,but no one tells the description about it。
> please,helps will be appreciated .
>
> .
>
--
Harsh J
Re: what does the yarn.nodemanager.aux-services stand for
Posted by Harsh J <ha...@cloudera.com>.
Hi,
The function of yarn.nodemanager.aux-services is to run any custom
Service [1] implementations from the user, in the YARN NodeManager's
runtime. The "aux" stands for auxiliary.
One such example of where this comes useful is the ShuffleHandler [2]
service, which helps MR do its shuffle work, without needing the NM
itself to provide a shuffle service unnecessarily (in case MR is not
required). Therefore, a ShuffleHandler is necessary to be configured
if you need to run MR2 apps (jobs), but not otherwise. A clear
difference, in case you're confused between MR2 and YARN, can be found
at [3].
Do feel free to ask any further questions you have.
[1] - http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/Service.java?view=markup
[2] - http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java?view=markup
[3] - http://www.cloudera.com/blog/2012/10/mr2-and-yarn-briefly-explained/
On Mon, Jan 28, 2013 at 6:43 PM, YouPeng Yang <yy...@gmail.com> wrote:
> Hi All
> I am wondering what the yarn.nodemanager.aux-services stands for 。
> there even is not description about it in the yarn-default.xml。So i‘ve just
> ignored it when i tried the example,as a result ,it failed。
> I have do some google，however，everyone just setup it in the
> yarn-site.xml,but no one tells the description about it。
> please,helps will be appreciated .
>
> .
>
--
Harsh J
Re: what does the yarn.nodemanager.aux-services stand for
Posted by Harsh J <ha...@cloudera.com>.
Hi,
The function of yarn.nodemanager.aux-services is to run any custom
Service [1] implementations from the user, in the YARN NodeManager's
runtime. The "aux" stands for auxiliary.
One such example of where this comes useful is the ShuffleHandler [2]
service, which helps MR do its shuffle work, without needing the NM
itself to provide a shuffle service unnecessarily (in case MR is not
required). Therefore, a ShuffleHandler is necessary to be configured
if you need to run MR2 apps (jobs), but not otherwise. A clear
difference, in case you're confused between MR2 and YARN, can be found
at [3].
Do feel free to ask any further questions you have.
[1] - http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/Service.java?view=markup
[2] - http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java?view=markup
[3] - http://www.cloudera.com/blog/2012/10/mr2-and-yarn-briefly-explained/
On Mon, Jan 28, 2013 at 6:43 PM, YouPeng Yang <yy...@gmail.com> wrote:
> Hi All
> I am wondering what the yarn.nodemanager.aux-services stands for 。
> there even is not description about it in the yarn-default.xml。So i‘ve just
> ignored it when i tried the example,as a result ,it failed。
> I have do some google，however，everyone just setup it in the
> yarn-site.xml,but no one tells the description about it。
> please,helps will be appreciated .
>
> .
>
--
Harsh J
Re: what does the yarn.nodemanager.aux-services stand for
Posted by Harsh J <ha...@cloudera.com>.
Hi,
The function of yarn.nodemanager.aux-services is to run any custom
Service [1] implementations from the user, in the YARN NodeManager's
runtime. The "aux" stands for auxiliary.
One such example of where this comes useful is the ShuffleHandler [2]
service, which helps MR do its shuffle work, without needing the NM
itself to provide a shuffle service unnecessarily (in case MR is not
required). Therefore, a ShuffleHandler is necessary to be configured
if you need to run MR2 apps (jobs), but not otherwise. A clear
difference, in case you're confused between MR2 and YARN, can be found
at [3].
Do feel free to ask any further questions you have.
[1] - http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/service/Service.java?view=markup
[2] - http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java?view=markup
[3] - http://www.cloudera.com/blog/2012/10/mr2-and-yarn-briefly-explained/
On Mon, Jan 28, 2013 at 6:43 PM, YouPeng Yang <yy...@gmail.com> wrote:
> Hi All
> I am wondering what the yarn.nodemanager.aux-services stands for 。
> there even is not description about it in the yarn-default.xml。So i‘ve just
> ignored it when i tried the example,as a result ,it failed。
> I have do some google，however，everyone just setup it in the
> yarn-site.xml,but no one tells the description about it。
> please,helps will be appreciated .
>
> .
>
--
Harsh J