You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by mg <us...@gmail.com> on 2012/08/16 18:28:57 UTC
Relevance of mapreduce.* configuration properties for MR V2
Hi,
I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha)
cluster running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3
(Free Edition).
In CM, there are a number of options for setting mapreduce.*
configuration properties on the YARN client page.
Some of the properties and their explanations in the GUI still refer to
JobTracker and TaskTracker, e.g.,
- mapreduce.jobtracker.handler.count,
- mapreduce.tasktracker.map.tasks.maximum,
- mapreduce.tasktracker.reduce.tasks.maximum
I wonder whether these and a number of other mapreduce.* (e.g.,
mapreduce.job.reduces) properties are observed by the MR2
ApplicationMaster (and how), or not.
Can anyone clarify or point to respective documentation?
Thanks,
Martin
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by Alok Kumar <al...@gmail.com>.
hi,
check your namenode logs present in $HADOOP_HOME/logs dir.
It'll tell u exact reason why it's not coming up..
I believe some permission issue or hv u not formatted ur namenode ?
Thanks,
Alok
On Sat, Aug 18, 2012 at 8:59 PM, rahul p <ra...@gmail.com>wrote:
> hi Shashwat,
>
> Please let me know what needs to be changed.
>
> [image: Inline image 1]
>
>
> On Sat, Aug 18, 2012 at 11:27 PM, rahul p <ra...@gmail.com>wrote:
>
>> hi,
>> please find the etc/hosts file screenshot
>>
>>
>>
>> On Sat, Aug 18, 2012 at 10:50 PM, Shashwat Shriparv <
>> dwivedishashwat@gmail.com> wrote:
>>
>>> What is content of your /etc/hosts file?****
>>>
>>> ** **
>>>
>>> *From:* rahul p [mailto:rahulpoolanchalil@gmail.com]
>>> *Sent:* Saturday, August 18, 2012 4:49 PM
>>>
>>
>>
>>
>>> *To:* user@hadoop.apache.org
>>> *Subject:* Re: Relevance of mapreduce.* configuration properties for MR
>>> V2****
>>>
>>> ** **
>>>
>>> Hi,****
>>>
>>> When i give bin/start-fs all ****
>>>
>>> namenode s not coming up..****
>>>
>>> On Fri, Aug 17, 2012 at 12:28 AM, mg <us...@gmail.com>
>>> wrote:****
>>>
>>> Hi,
>>>
>>> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha)
>>> cluster running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3
>>> (Free Edition).
>>>
>>> In CM, there are a number of options for setting mapreduce.*
>>> configuration properties on the YARN client page.
>>>
>>> Some of the properties and their explanations in the GUI still refer to
>>> JobTracker and TaskTracker, e.g.,
>>> - mapreduce.jobtracker.handler.count,
>>> - mapreduce.tasktracker.map.tasks.maximum,
>>> - mapreduce.tasktracker.reduce.tasks.maximum
>>>
>>> I wonder whether these and a number of other mapreduce.* (e.g.,
>>> mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
>>> (and how), or not.
>>>
>>> Can anyone clarify or point to respective documentation?
>>>
>>> Thanks,
>>> Martin****
>>>
>>> ** **
>>>
>>
>>
>
--
Alok Kumar
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by Alok Kumar <al...@gmail.com>.
hi,
check your namenode logs present in $HADOOP_HOME/logs dir.
It'll tell u exact reason why it's not coming up..
I believe some permission issue or hv u not formatted ur namenode ?
Thanks,
Alok
On Sat, Aug 18, 2012 at 8:59 PM, rahul p <ra...@gmail.com>wrote:
> hi Shashwat,
>
> Please let me know what needs to be changed.
>
> [image: Inline image 1]
>
>
> On Sat, Aug 18, 2012 at 11:27 PM, rahul p <ra...@gmail.com>wrote:
>
>> hi,
>> please find the etc/hosts file screenshot
>>
>>
>>
>> On Sat, Aug 18, 2012 at 10:50 PM, Shashwat Shriparv <
>> dwivedishashwat@gmail.com> wrote:
>>
>>> What is content of your /etc/hosts file?****
>>>
>>> ** **
>>>
>>> *From:* rahul p [mailto:rahulpoolanchalil@gmail.com]
>>> *Sent:* Saturday, August 18, 2012 4:49 PM
>>>
>>
>>
>>
>>> *To:* user@hadoop.apache.org
>>> *Subject:* Re: Relevance of mapreduce.* configuration properties for MR
>>> V2****
>>>
>>> ** **
>>>
>>> Hi,****
>>>
>>> When i give bin/start-fs all ****
>>>
>>> namenode s not coming up..****
>>>
>>> On Fri, Aug 17, 2012 at 12:28 AM, mg <us...@gmail.com>
>>> wrote:****
>>>
>>> Hi,
>>>
>>> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha)
>>> cluster running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3
>>> (Free Edition).
>>>
>>> In CM, there are a number of options for setting mapreduce.*
>>> configuration properties on the YARN client page.
>>>
>>> Some of the properties and their explanations in the GUI still refer to
>>> JobTracker and TaskTracker, e.g.,
>>> - mapreduce.jobtracker.handler.count,
>>> - mapreduce.tasktracker.map.tasks.maximum,
>>> - mapreduce.tasktracker.reduce.tasks.maximum
>>>
>>> I wonder whether these and a number of other mapreduce.* (e.g.,
>>> mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
>>> (and how), or not.
>>>
>>> Can anyone clarify or point to respective documentation?
>>>
>>> Thanks,
>>> Martin****
>>>
>>> ** **
>>>
>>
>>
>
--
Alok Kumar
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by Alok Kumar <al...@gmail.com>.
hi,
check your namenode logs present in $HADOOP_HOME/logs dir.
It'll tell u exact reason why it's not coming up..
I believe some permission issue or hv u not formatted ur namenode ?
Thanks,
Alok
On Sat, Aug 18, 2012 at 8:59 PM, rahul p <ra...@gmail.com>wrote:
> hi Shashwat,
>
> Please let me know what needs to be changed.
>
> [image: Inline image 1]
>
>
> On Sat, Aug 18, 2012 at 11:27 PM, rahul p <ra...@gmail.com>wrote:
>
>> hi,
>> please find the etc/hosts file screenshot
>>
>>
>>
>> On Sat, Aug 18, 2012 at 10:50 PM, Shashwat Shriparv <
>> dwivedishashwat@gmail.com> wrote:
>>
>>> What is content of your /etc/hosts file?****
>>>
>>> ** **
>>>
>>> *From:* rahul p [mailto:rahulpoolanchalil@gmail.com]
>>> *Sent:* Saturday, August 18, 2012 4:49 PM
>>>
>>
>>
>>
>>> *To:* user@hadoop.apache.org
>>> *Subject:* Re: Relevance of mapreduce.* configuration properties for MR
>>> V2****
>>>
>>> ** **
>>>
>>> Hi,****
>>>
>>> When i give bin/start-fs all ****
>>>
>>> namenode s not coming up..****
>>>
>>> On Fri, Aug 17, 2012 at 12:28 AM, mg <us...@gmail.com>
>>> wrote:****
>>>
>>> Hi,
>>>
>>> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha)
>>> cluster running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3
>>> (Free Edition).
>>>
>>> In CM, there are a number of options for setting mapreduce.*
>>> configuration properties on the YARN client page.
>>>
>>> Some of the properties and their explanations in the GUI still refer to
>>> JobTracker and TaskTracker, e.g.,
>>> - mapreduce.jobtracker.handler.count,
>>> - mapreduce.tasktracker.map.tasks.maximum,
>>> - mapreduce.tasktracker.reduce.tasks.maximum
>>>
>>> I wonder whether these and a number of other mapreduce.* (e.g.,
>>> mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
>>> (and how), or not.
>>>
>>> Can anyone clarify or point to respective documentation?
>>>
>>> Thanks,
>>> Martin****
>>>
>>> ** **
>>>
>>
>>
>
--
Alok Kumar
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by Alok Kumar <al...@gmail.com>.
hi,
check your namenode logs present in $HADOOP_HOME/logs dir.
It'll tell u exact reason why it's not coming up..
I believe some permission issue or hv u not formatted ur namenode ?
Thanks,
Alok
On Sat, Aug 18, 2012 at 8:59 PM, rahul p <ra...@gmail.com>wrote:
> hi Shashwat,
>
> Please let me know what needs to be changed.
>
> [image: Inline image 1]
>
>
> On Sat, Aug 18, 2012 at 11:27 PM, rahul p <ra...@gmail.com>wrote:
>
>> hi,
>> please find the etc/hosts file screenshot
>>
>>
>>
>> On Sat, Aug 18, 2012 at 10:50 PM, Shashwat Shriparv <
>> dwivedishashwat@gmail.com> wrote:
>>
>>> What is content of your /etc/hosts file?****
>>>
>>> ** **
>>>
>>> *From:* rahul p [mailto:rahulpoolanchalil@gmail.com]
>>> *Sent:* Saturday, August 18, 2012 4:49 PM
>>>
>>
>>
>>
>>> *To:* user@hadoop.apache.org
>>> *Subject:* Re: Relevance of mapreduce.* configuration properties for MR
>>> V2****
>>>
>>> ** **
>>>
>>> Hi,****
>>>
>>> When i give bin/start-fs all ****
>>>
>>> namenode s not coming up..****
>>>
>>> On Fri, Aug 17, 2012 at 12:28 AM, mg <us...@gmail.com>
>>> wrote:****
>>>
>>> Hi,
>>>
>>> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha)
>>> cluster running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3
>>> (Free Edition).
>>>
>>> In CM, there are a number of options for setting mapreduce.*
>>> configuration properties on the YARN client page.
>>>
>>> Some of the properties and their explanations in the GUI still refer to
>>> JobTracker and TaskTracker, e.g.,
>>> - mapreduce.jobtracker.handler.count,
>>> - mapreduce.tasktracker.map.tasks.maximum,
>>> - mapreduce.tasktracker.reduce.tasks.maximum
>>>
>>> I wonder whether these and a number of other mapreduce.* (e.g.,
>>> mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
>>> (and how), or not.
>>>
>>> Can anyone clarify or point to respective documentation?
>>>
>>> Thanks,
>>> Martin****
>>>
>>> ** **
>>>
>>
>>
>
--
Alok Kumar
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by rahul p <ra...@gmail.com>.
hi Shashwat,
Please let me know what needs to be changed.
[image: Inline image 1]
On Sat, Aug 18, 2012 at 11:27 PM, rahul p <ra...@gmail.com>wrote:
> hi,
> please find the etc/hosts file screenshot
>
>
>
> On Sat, Aug 18, 2012 at 10:50 PM, Shashwat Shriparv <
> dwivedishashwat@gmail.com> wrote:
>
>> What is content of your /etc/hosts file?****
>>
>> ** **
>>
>> *From:* rahul p [mailto:rahulpoolanchalil@gmail.com]
>> *Sent:* Saturday, August 18, 2012 4:49 PM
>>
>
>
>
>> *To:* user@hadoop.apache.org
>> *Subject:* Re: Relevance of mapreduce.* configuration properties for MR
>> V2****
>>
>> ** **
>>
>> Hi,****
>>
>> When i give bin/start-fs all ****
>>
>> namenode s not coming up..****
>>
>> On Fri, Aug 17, 2012 at 12:28 AM, mg <us...@gmail.com>
>> wrote:****
>>
>> Hi,
>>
>> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha) cluster
>> running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3 (Free
>> Edition).
>>
>> In CM, there are a number of options for setting mapreduce.*
>> configuration properties on the YARN client page.
>>
>> Some of the properties and their explanations in the GUI still refer to
>> JobTracker and TaskTracker, e.g.,
>> - mapreduce.jobtracker.handler.count,
>> - mapreduce.tasktracker.map.tasks.maximum,
>> - mapreduce.tasktracker.reduce.tasks.maximum
>>
>> I wonder whether these and a number of other mapreduce.* (e.g.,
>> mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
>> (and how), or not.
>>
>> Can anyone clarify or point to respective documentation?
>>
>> Thanks,
>> Martin****
>>
>> ** **
>>
>
>
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by rahul p <ra...@gmail.com>.
hi Shashwat,
Please let me know what needs to be changed.
[image: Inline image 1]
On Sat, Aug 18, 2012 at 11:27 PM, rahul p <ra...@gmail.com>wrote:
> hi,
> please find the etc/hosts file screenshot
>
>
>
> On Sat, Aug 18, 2012 at 10:50 PM, Shashwat Shriparv <
> dwivedishashwat@gmail.com> wrote:
>
>> What is content of your /etc/hosts file?****
>>
>> ** **
>>
>> *From:* rahul p [mailto:rahulpoolanchalil@gmail.com]
>> *Sent:* Saturday, August 18, 2012 4:49 PM
>>
>
>
>
>> *To:* user@hadoop.apache.org
>> *Subject:* Re: Relevance of mapreduce.* configuration properties for MR
>> V2****
>>
>> ** **
>>
>> Hi,****
>>
>> When i give bin/start-fs all ****
>>
>> namenode s not coming up..****
>>
>> On Fri, Aug 17, 2012 at 12:28 AM, mg <us...@gmail.com>
>> wrote:****
>>
>> Hi,
>>
>> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha) cluster
>> running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3 (Free
>> Edition).
>>
>> In CM, there are a number of options for setting mapreduce.*
>> configuration properties on the YARN client page.
>>
>> Some of the properties and their explanations in the GUI still refer to
>> JobTracker and TaskTracker, e.g.,
>> - mapreduce.jobtracker.handler.count,
>> - mapreduce.tasktracker.map.tasks.maximum,
>> - mapreduce.tasktracker.reduce.tasks.maximum
>>
>> I wonder whether these and a number of other mapreduce.* (e.g.,
>> mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
>> (and how), or not.
>>
>> Can anyone clarify or point to respective documentation?
>>
>> Thanks,
>> Martin****
>>
>> ** **
>>
>
>
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by rahul p <ra...@gmail.com>.
hi Shashwat,
Please let me know what needs to be changed.
[image: Inline image 1]
On Sat, Aug 18, 2012 at 11:27 PM, rahul p <ra...@gmail.com>wrote:
> hi,
> please find the etc/hosts file screenshot
>
>
>
> On Sat, Aug 18, 2012 at 10:50 PM, Shashwat Shriparv <
> dwivedishashwat@gmail.com> wrote:
>
>> What is content of your /etc/hosts file?****
>>
>> ** **
>>
>> *From:* rahul p [mailto:rahulpoolanchalil@gmail.com]
>> *Sent:* Saturday, August 18, 2012 4:49 PM
>>
>
>
>
>> *To:* user@hadoop.apache.org
>> *Subject:* Re: Relevance of mapreduce.* configuration properties for MR
>> V2****
>>
>> ** **
>>
>> Hi,****
>>
>> When i give bin/start-fs all ****
>>
>> namenode s not coming up..****
>>
>> On Fri, Aug 17, 2012 at 12:28 AM, mg <us...@gmail.com>
>> wrote:****
>>
>> Hi,
>>
>> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha) cluster
>> running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3 (Free
>> Edition).
>>
>> In CM, there are a number of options for setting mapreduce.*
>> configuration properties on the YARN client page.
>>
>> Some of the properties and their explanations in the GUI still refer to
>> JobTracker and TaskTracker, e.g.,
>> - mapreduce.jobtracker.handler.count,
>> - mapreduce.tasktracker.map.tasks.maximum,
>> - mapreduce.tasktracker.reduce.tasks.maximum
>>
>> I wonder whether these and a number of other mapreduce.* (e.g.,
>> mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
>> (and how), or not.
>>
>> Can anyone clarify or point to respective documentation?
>>
>> Thanks,
>> Martin****
>>
>> ** **
>>
>
>
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by rahul p <ra...@gmail.com>.
hi Shashwat,
Please let me know what needs to be changed.
[image: Inline image 1]
On Sat, Aug 18, 2012 at 11:27 PM, rahul p <ra...@gmail.com>wrote:
> hi,
> please find the etc/hosts file screenshot
>
>
>
> On Sat, Aug 18, 2012 at 10:50 PM, Shashwat Shriparv <
> dwivedishashwat@gmail.com> wrote:
>
>> What is content of your /etc/hosts file?****
>>
>> ** **
>>
>> *From:* rahul p [mailto:rahulpoolanchalil@gmail.com]
>> *Sent:* Saturday, August 18, 2012 4:49 PM
>>
>
>
>
>> *To:* user@hadoop.apache.org
>> *Subject:* Re: Relevance of mapreduce.* configuration properties for MR
>> V2****
>>
>> ** **
>>
>> Hi,****
>>
>> When i give bin/start-fs all ****
>>
>> namenode s not coming up..****
>>
>> On Fri, Aug 17, 2012 at 12:28 AM, mg <us...@gmail.com>
>> wrote:****
>>
>> Hi,
>>
>> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha) cluster
>> running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3 (Free
>> Edition).
>>
>> In CM, there are a number of options for setting mapreduce.*
>> configuration properties on the YARN client page.
>>
>> Some of the properties and their explanations in the GUI still refer to
>> JobTracker and TaskTracker, e.g.,
>> - mapreduce.jobtracker.handler.count,
>> - mapreduce.tasktracker.map.tasks.maximum,
>> - mapreduce.tasktracker.reduce.tasks.maximum
>>
>> I wonder whether these and a number of other mapreduce.* (e.g.,
>> mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
>> (and how), or not.
>>
>> Can anyone clarify or point to respective documentation?
>>
>> Thanks,
>> Martin****
>>
>> ** **
>>
>
>
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by rahul p <ra...@gmail.com>.
hi,
please find the etc/hosts file screenshot
On Sat, Aug 18, 2012 at 10:50 PM, Shashwat Shriparv <
dwivedishashwat@gmail.com> wrote:
> What is content of your /etc/hosts file?****
>
> ** **
>
> *From:* rahul p [mailto:rahulpoolanchalil@gmail.com]
> *Sent:* Saturday, August 18, 2012 4:49 PM
>
> *To:* user@hadoop.apache.org
> *Subject:* Re: Relevance of mapreduce.* configuration properties for MR V2
> ****
>
> ** **
>
> Hi,****
>
> When i give bin/start-fs all ****
>
> namenode s not coming up..****
>
> On Fri, Aug 17, 2012 at 12:28 AM, mg <us...@gmail.com>
> wrote:****
>
> Hi,
>
> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha) cluster
> running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3 (Free
> Edition).
>
> In CM, there are a number of options for setting mapreduce.* configuration
> properties on the YARN client page.
>
> Some of the properties and their explanations in the GUI still refer to
> JobTracker and TaskTracker, e.g.,
> - mapreduce.jobtracker.handler.count,
> - mapreduce.tasktracker.map.tasks.maximum,
> - mapreduce.tasktracker.reduce.tasks.maximum
>
> I wonder whether these and a number of other mapreduce.* (e.g.,
> mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
> (and how), or not.
>
> Can anyone clarify or point to respective documentation?
>
> Thanks,
> Martin****
>
> ** **
>
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by rahul p <ra...@gmail.com>.
hi,
please find the etc/hosts file screenshot
On Sat, Aug 18, 2012 at 10:50 PM, Shashwat Shriparv <
dwivedishashwat@gmail.com> wrote:
> What is content of your /etc/hosts file?****
>
> ** **
>
> *From:* rahul p [mailto:rahulpoolanchalil@gmail.com]
> *Sent:* Saturday, August 18, 2012 4:49 PM
>
> *To:* user@hadoop.apache.org
> *Subject:* Re: Relevance of mapreduce.* configuration properties for MR V2
> ****
>
> ** **
>
> Hi,****
>
> When i give bin/start-fs all ****
>
> namenode s not coming up..****
>
> On Fri, Aug 17, 2012 at 12:28 AM, mg <us...@gmail.com>
> wrote:****
>
> Hi,
>
> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha) cluster
> running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3 (Free
> Edition).
>
> In CM, there are a number of options for setting mapreduce.* configuration
> properties on the YARN client page.
>
> Some of the properties and their explanations in the GUI still refer to
> JobTracker and TaskTracker, e.g.,
> - mapreduce.jobtracker.handler.count,
> - mapreduce.tasktracker.map.tasks.maximum,
> - mapreduce.tasktracker.reduce.tasks.maximum
>
> I wonder whether these and a number of other mapreduce.* (e.g.,
> mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
> (and how), or not.
>
> Can anyone clarify or point to respective documentation?
>
> Thanks,
> Martin****
>
> ** **
>
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by rahul p <ra...@gmail.com>.
hi,
please find the etc/hosts file screenshot
On Sat, Aug 18, 2012 at 10:50 PM, Shashwat Shriparv <
dwivedishashwat@gmail.com> wrote:
> What is content of your /etc/hosts file?****
>
> ** **
>
> *From:* rahul p [mailto:rahulpoolanchalil@gmail.com]
> *Sent:* Saturday, August 18, 2012 4:49 PM
>
> *To:* user@hadoop.apache.org
> *Subject:* Re: Relevance of mapreduce.* configuration properties for MR V2
> ****
>
> ** **
>
> Hi,****
>
> When i give bin/start-fs all ****
>
> namenode s not coming up..****
>
> On Fri, Aug 17, 2012 at 12:28 AM, mg <us...@gmail.com>
> wrote:****
>
> Hi,
>
> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha) cluster
> running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3 (Free
> Edition).
>
> In CM, there are a number of options for setting mapreduce.* configuration
> properties on the YARN client page.
>
> Some of the properties and their explanations in the GUI still refer to
> JobTracker and TaskTracker, e.g.,
> - mapreduce.jobtracker.handler.count,
> - mapreduce.tasktracker.map.tasks.maximum,
> - mapreduce.tasktracker.reduce.tasks.maximum
>
> I wonder whether these and a number of other mapreduce.* (e.g.,
> mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
> (and how), or not.
>
> Can anyone clarify or point to respective documentation?
>
> Thanks,
> Martin****
>
> ** **
>
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by rahul p <ra...@gmail.com>.
hi,
please find the etc/hosts file screenshot
On Sat, Aug 18, 2012 at 10:50 PM, Shashwat Shriparv <
dwivedishashwat@gmail.com> wrote:
> What is content of your /etc/hosts file?****
>
> ** **
>
> *From:* rahul p [mailto:rahulpoolanchalil@gmail.com]
> *Sent:* Saturday, August 18, 2012 4:49 PM
>
> *To:* user@hadoop.apache.org
> *Subject:* Re: Relevance of mapreduce.* configuration properties for MR V2
> ****
>
> ** **
>
> Hi,****
>
> When i give bin/start-fs all ****
>
> namenode s not coming up..****
>
> On Fri, Aug 17, 2012 at 12:28 AM, mg <us...@gmail.com>
> wrote:****
>
> Hi,
>
> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha) cluster
> running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3 (Free
> Edition).
>
> In CM, there are a number of options for setting mapreduce.* configuration
> properties on the YARN client page.
>
> Some of the properties and their explanations in the GUI still refer to
> JobTracker and TaskTracker, e.g.,
> - mapreduce.jobtracker.handler.count,
> - mapreduce.tasktracker.map.tasks.maximum,
> - mapreduce.tasktracker.reduce.tasks.maximum
>
> I wonder whether these and a number of other mapreduce.* (e.g.,
> mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
> (and how), or not.
>
> Can anyone clarify or point to respective documentation?
>
> Thanks,
> Martin****
>
> ** **
>
RE: Relevance of mapreduce.* configuration properties for MR V2
Posted by Shashwat Shriparv <dw...@gmail.com>.
What is content of your /etc/hosts file?
From: rahul p [mailto:rahulpoolanchalil@gmail.com]
Sent: Saturday, August 18, 2012 4:49 PM
To: user@hadoop.apache.org
Subject: Re: Relevance of mapreduce.* configuration properties for MR V2
Hi,
When i give bin/start-fs all
namenode s not coming up..
On Fri, Aug 17, 2012 at 12:28 AM, mg <userformailinglists@gmail.com
<ma...@gmail.com> > wrote:
Hi,
I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha) cluster
running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3 (Free
Edition).
In CM, there are a number of options for setting mapreduce.* configuration
properties on the YARN client page.
Some of the properties and their explanations in the GUI still refer to
JobTracker and TaskTracker, e.g.,
- mapreduce.jobtracker.handler.count,
- mapreduce.tasktracker.map.tasks.maximum,
- mapreduce.tasktracker.reduce.tasks.maximum
I wonder whether these and a number of other mapreduce.* (e.g.,
mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
(and how), or not.
Can anyone clarify or point to respective documentation?
Thanks,
Martin
RE: Relevance of mapreduce.* configuration properties for MR V2
Posted by Shashwat Shriparv <dw...@gmail.com>.
What is content of your /etc/hosts file?
From: rahul p [mailto:rahulpoolanchalil@gmail.com]
Sent: Saturday, August 18, 2012 4:49 PM
To: user@hadoop.apache.org
Subject: Re: Relevance of mapreduce.* configuration properties for MR V2
Hi,
When i give bin/start-fs all
namenode s not coming up..
On Fri, Aug 17, 2012 at 12:28 AM, mg <userformailinglists@gmail.com
<ma...@gmail.com> > wrote:
Hi,
I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha) cluster
running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3 (Free
Edition).
In CM, there are a number of options for setting mapreduce.* configuration
properties on the YARN client page.
Some of the properties and their explanations in the GUI still refer to
JobTracker and TaskTracker, e.g.,
- mapreduce.jobtracker.handler.count,
- mapreduce.tasktracker.map.tasks.maximum,
- mapreduce.tasktracker.reduce.tasks.maximum
I wonder whether these and a number of other mapreduce.* (e.g.,
mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
(and how), or not.
Can anyone clarify or point to respective documentation?
Thanks,
Martin
RE: Relevance of mapreduce.* configuration properties for MR V2
Posted by Shashwat Shriparv <dw...@gmail.com>.
What is content of your /etc/hosts file?
From: rahul p [mailto:rahulpoolanchalil@gmail.com]
Sent: Saturday, August 18, 2012 4:49 PM
To: user@hadoop.apache.org
Subject: Re: Relevance of mapreduce.* configuration properties for MR V2
Hi,
When i give bin/start-fs all
namenode s not coming up..
On Fri, Aug 17, 2012 at 12:28 AM, mg <userformailinglists@gmail.com
<ma...@gmail.com> > wrote:
Hi,
I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha) cluster
running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3 (Free
Edition).
In CM, there are a number of options for setting mapreduce.* configuration
properties on the YARN client page.
Some of the properties and their explanations in the GUI still refer to
JobTracker and TaskTracker, e.g.,
- mapreduce.jobtracker.handler.count,
- mapreduce.tasktracker.map.tasks.maximum,
- mapreduce.tasktracker.reduce.tasks.maximum
I wonder whether these and a number of other mapreduce.* (e.g.,
mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
(and how), or not.
Can anyone clarify or point to respective documentation?
Thanks,
Martin
RE: Relevance of mapreduce.* configuration properties for MR V2
Posted by Shashwat Shriparv <dw...@gmail.com>.
What is content of your /etc/hosts file?
From: rahul p [mailto:rahulpoolanchalil@gmail.com]
Sent: Saturday, August 18, 2012 4:49 PM
To: user@hadoop.apache.org
Subject: Re: Relevance of mapreduce.* configuration properties for MR V2
Hi,
When i give bin/start-fs all
namenode s not coming up..
On Fri, Aug 17, 2012 at 12:28 AM, mg <userformailinglists@gmail.com
<ma...@gmail.com> > wrote:
Hi,
I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha) cluster
running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3 (Free
Edition).
In CM, there are a number of options for setting mapreduce.* configuration
properties on the YARN client page.
Some of the properties and their explanations in the GUI still refer to
JobTracker and TaskTracker, e.g.,
- mapreduce.jobtracker.handler.count,
- mapreduce.tasktracker.map.tasks.maximum,
- mapreduce.tasktracker.reduce.tasks.maximum
I wonder whether these and a number of other mapreduce.* (e.g.,
mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
(and how), or not.
Can anyone clarify or point to respective documentation?
Thanks,
Martin
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by rahul p <ra...@gmail.com>.
Hi,
When i give bin/start-fs all
namenode s not coming up..
On Fri, Aug 17, 2012 at 12:28 AM, mg <us...@gmail.com> wrote:
> Hi,
>
> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha) cluster
> running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3 (Free
> Edition).
>
> In CM, there are a number of options for setting mapreduce.* configuration
> properties on the YARN client page.
>
> Some of the properties and their explanations in the GUI still refer to
> JobTracker and TaskTracker, e.g.,
> - mapreduce.jobtracker.handler.**count,
> - mapreduce.tasktracker.map.**tasks.maximum,
> - mapreduce.tasktracker.reduce.**tasks.maximum
>
> I wonder whether these and a number of other mapreduce.* (e.g.,
> mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
> (and how), or not.
>
> Can anyone clarify or point to respective documentation?
>
> Thanks,
> Martin
>
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by mg <us...@gmail.com>.
Thanks again, that helps a lot.
Additionally to the properties listed on that page, I found that
mapreduce.job.reduces limits the number of reduce tasks per job (as with
MR V1)
Cheers
Martin
On 16.08.2012 20:42, Arun C Murthy wrote:
> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html
>
> On Aug 16, 2012, at 10:37 AM, mg wrote:
>
>> Hi,
>>
>> thanks.
>>
>> But there are many more mapreduce.* properties. ...
>>
>> Does anyone have more information on those with respect to whether
>> they are relevant to YARN, and if so, what they exactly mean in the
>> YARN context?
>>
>> I am trying to optimize settings in order to use as much RAM and CPU
>> as possible. The cluster has 8 node managers each with 96GB RAM and 16
>> CPUs (cpuinfo: Intel(R) Xeon(R) CPU E @ 2.40GHz)
>>
>> Cheers,
>> Martin
>>
>> On 16.08.2012 18:40, anil gupta wrote:
>>> Hi,
>>>
>>> AFAIK, these properties are being ignored by YARN:
>>> - mapreduce.tasktracker.map.tasks.maximum,
>>> - mapreduce.tasktracker.reduce.tasks.maximum
>>>
>>> Thanks,
>>> Anil Gupta
>>>
>>>
>>> On Thu, Aug 16, 2012 at 9:28 AM, mg <userformailinglists@gmail.com
>>> <ma...@gmail.com>> wrote:
>>>
>>> Hi,
>>>
>>> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha)
>>> cluster running HDFS, YARN, and HBase managed by Cloudera Manager
>>> 4.0.3 (Free Edition).
>>>
>>>
>>> In CM, there are a number of options for setting mapreduce.*
>>> configuration properties on the YARN client page.
>>>
>>> Some of the properties and their explanations in the GUI still refer
>>> to JobTracker and TaskTracker, e.g.,
>>>
>>> - mapreduce.jobtracker.handler.__count,
>>> - mapreduce.tasktracker.map.__tasks.maximum,
>>> - mapreduce.tasktracker.reduce.__tasks.maximum
>>>
>>> I wonder whether these and a number of other mapreduce.* (e.g.,
>>> mapreduce.job.reduces) properties are observed by the MR2
>>> ApplicationMaster (and how), or not.
>>>
>>>
>>> Can anyone clarify or point to respective documentation?
>>>
>>> Thanks,
>>> Martin
>>>
>>>
>>>
>>>
>>> --
>>> Thanks & Regards,
>>> Anil Gupta
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by mg <us...@gmail.com>.
Thanks again, that helps a lot.
Additionally to the properties listed on that page, I found that
mapreduce.job.reduces limits the number of reduce tasks per job (as with
MR V1)
Cheers
Martin
On 16.08.2012 20:42, Arun C Murthy wrote:
> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html
>
> On Aug 16, 2012, at 10:37 AM, mg wrote:
>
>> Hi,
>>
>> thanks.
>>
>> But there are many more mapreduce.* properties. ...
>>
>> Does anyone have more information on those with respect to whether
>> they are relevant to YARN, and if so, what they exactly mean in the
>> YARN context?
>>
>> I am trying to optimize settings in order to use as much RAM and CPU
>> as possible. The cluster has 8 node managers each with 96GB RAM and 16
>> CPUs (cpuinfo: Intel(R) Xeon(R) CPU E @ 2.40GHz)
>>
>> Cheers,
>> Martin
>>
>> On 16.08.2012 18:40, anil gupta wrote:
>>> Hi,
>>>
>>> AFAIK, these properties are being ignored by YARN:
>>> - mapreduce.tasktracker.map.tasks.maximum,
>>> - mapreduce.tasktracker.reduce.tasks.maximum
>>>
>>> Thanks,
>>> Anil Gupta
>>>
>>>
>>> On Thu, Aug 16, 2012 at 9:28 AM, mg <userformailinglists@gmail.com
>>> <ma...@gmail.com>> wrote:
>>>
>>> Hi,
>>>
>>> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha)
>>> cluster running HDFS, YARN, and HBase managed by Cloudera Manager
>>> 4.0.3 (Free Edition).
>>>
>>>
>>> In CM, there are a number of options for setting mapreduce.*
>>> configuration properties on the YARN client page.
>>>
>>> Some of the properties and their explanations in the GUI still refer
>>> to JobTracker and TaskTracker, e.g.,
>>>
>>> - mapreduce.jobtracker.handler.__count,
>>> - mapreduce.tasktracker.map.__tasks.maximum,
>>> - mapreduce.tasktracker.reduce.__tasks.maximum
>>>
>>> I wonder whether these and a number of other mapreduce.* (e.g.,
>>> mapreduce.job.reduces) properties are observed by the MR2
>>> ApplicationMaster (and how), or not.
>>>
>>>
>>> Can anyone clarify or point to respective documentation?
>>>
>>> Thanks,
>>> Martin
>>>
>>>
>>>
>>>
>>> --
>>> Thanks & Regards,
>>> Anil Gupta
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by mg <us...@gmail.com>.
Thanks again, that helps a lot.
Additionally to the properties listed on that page, I found that
mapreduce.job.reduces limits the number of reduce tasks per job (as with
MR V1)
Cheers
Martin
On 16.08.2012 20:42, Arun C Murthy wrote:
> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html
>
> On Aug 16, 2012, at 10:37 AM, mg wrote:
>
>> Hi,
>>
>> thanks.
>>
>> But there are many more mapreduce.* properties. ...
>>
>> Does anyone have more information on those with respect to whether
>> they are relevant to YARN, and if so, what they exactly mean in the
>> YARN context?
>>
>> I am trying to optimize settings in order to use as much RAM and CPU
>> as possible. The cluster has 8 node managers each with 96GB RAM and 16
>> CPUs (cpuinfo: Intel(R) Xeon(R) CPU E @ 2.40GHz)
>>
>> Cheers,
>> Martin
>>
>> On 16.08.2012 18:40, anil gupta wrote:
>>> Hi,
>>>
>>> AFAIK, these properties are being ignored by YARN:
>>> - mapreduce.tasktracker.map.tasks.maximum,
>>> - mapreduce.tasktracker.reduce.tasks.maximum
>>>
>>> Thanks,
>>> Anil Gupta
>>>
>>>
>>> On Thu, Aug 16, 2012 at 9:28 AM, mg <userformailinglists@gmail.com
>>> <ma...@gmail.com>> wrote:
>>>
>>> Hi,
>>>
>>> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha)
>>> cluster running HDFS, YARN, and HBase managed by Cloudera Manager
>>> 4.0.3 (Free Edition).
>>>
>>>
>>> In CM, there are a number of options for setting mapreduce.*
>>> configuration properties on the YARN client page.
>>>
>>> Some of the properties and their explanations in the GUI still refer
>>> to JobTracker and TaskTracker, e.g.,
>>>
>>> - mapreduce.jobtracker.handler.__count,
>>> - mapreduce.tasktracker.map.__tasks.maximum,
>>> - mapreduce.tasktracker.reduce.__tasks.maximum
>>>
>>> I wonder whether these and a number of other mapreduce.* (e.g.,
>>> mapreduce.job.reduces) properties are observed by the MR2
>>> ApplicationMaster (and how), or not.
>>>
>>>
>>> Can anyone clarify or point to respective documentation?
>>>
>>> Thanks,
>>> Martin
>>>
>>>
>>>
>>>
>>> --
>>> Thanks & Regards,
>>> Anil Gupta
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by mg <us...@gmail.com>.
Thanks again, that helps a lot.
Additionally to the properties listed on that page, I found that
mapreduce.job.reduces limits the number of reduce tasks per job (as with
MR V1)
Cheers
Martin
On 16.08.2012 20:42, Arun C Murthy wrote:
> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html
>
> On Aug 16, 2012, at 10:37 AM, mg wrote:
>
>> Hi,
>>
>> thanks.
>>
>> But there are many more mapreduce.* properties. ...
>>
>> Does anyone have more information on those with respect to whether
>> they are relevant to YARN, and if so, what they exactly mean in the
>> YARN context?
>>
>> I am trying to optimize settings in order to use as much RAM and CPU
>> as possible. The cluster has 8 node managers each with 96GB RAM and 16
>> CPUs (cpuinfo: Intel(R) Xeon(R) CPU E @ 2.40GHz)
>>
>> Cheers,
>> Martin
>>
>> On 16.08.2012 18:40, anil gupta wrote:
>>> Hi,
>>>
>>> AFAIK, these properties are being ignored by YARN:
>>> - mapreduce.tasktracker.map.tasks.maximum,
>>> - mapreduce.tasktracker.reduce.tasks.maximum
>>>
>>> Thanks,
>>> Anil Gupta
>>>
>>>
>>> On Thu, Aug 16, 2012 at 9:28 AM, mg <userformailinglists@gmail.com
>>> <ma...@gmail.com>> wrote:
>>>
>>> Hi,
>>>
>>> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha)
>>> cluster running HDFS, YARN, and HBase managed by Cloudera Manager
>>> 4.0.3 (Free Edition).
>>>
>>>
>>> In CM, there are a number of options for setting mapreduce.*
>>> configuration properties on the YARN client page.
>>>
>>> Some of the properties and their explanations in the GUI still refer
>>> to JobTracker and TaskTracker, e.g.,
>>>
>>> - mapreduce.jobtracker.handler.__count,
>>> - mapreduce.tasktracker.map.__tasks.maximum,
>>> - mapreduce.tasktracker.reduce.__tasks.maximum
>>>
>>> I wonder whether these and a number of other mapreduce.* (e.g.,
>>> mapreduce.job.reduces) properties are observed by the MR2
>>> ApplicationMaster (and how), or not.
>>>
>>>
>>> Can anyone clarify or point to respective documentation?
>>>
>>> Thanks,
>>> Martin
>>>
>>>
>>>
>>>
>>> --
>>> Thanks & Regards,
>>> Anil Gupta
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by Arun C Murthy <ac...@hortonworks.com>.
http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html
On Aug 16, 2012, at 10:37 AM, mg wrote:
> Hi,
>
> thanks.
>
> But there are many more mapreduce.* properties. ...
>
> Does anyone have more information on those with respect to whether they are relevant to YARN, and if so, what they exactly mean in the YARN context?
>
> I am trying to optimize settings in order to use as much RAM and CPU as possible. The cluster has 8 node managers each with 96GB RAM and 16 CPUs (cpuinfo: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz)
>
> Cheers,
> Martin
>
> On 16.08.2012 18:40, anil gupta wrote:
>> Hi,
>>
>> AFAIK, these properties are being ignored by YARN:
>> - mapreduce.tasktracker.map.tasks.maximum,
>> - mapreduce.tasktracker.reduce.tasks.maximum
>>
>> Thanks,
>> Anil Gupta
>>
>>
>> On Thu, Aug 16, 2012 at 9:28 AM, mg <userformailinglists@gmail.com
>> <ma...@gmail.com>> wrote:
>>
>> Hi,
>>
>> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha)
>> cluster running HDFS, YARN, and HBase managed by Cloudera Manager
>> 4.0.3 (Free Edition).
>>
>>
>> In CM, there are a number of options for setting mapreduce.*
>> configuration properties on the YARN client page.
>>
>> Some of the properties and their explanations in the GUI still refer
>> to JobTracker and TaskTracker, e.g.,
>>
>> - mapreduce.jobtracker.handler.__count,
>> - mapreduce.tasktracker.map.__tasks.maximum,
>> - mapreduce.tasktracker.reduce.__tasks.maximum
>>
>> I wonder whether these and a number of other mapreduce.* (e.g.,
>> mapreduce.job.reduces) properties are observed by the MR2
>> ApplicationMaster (and how), or not.
>>
>>
>> Can anyone clarify or point to respective documentation?
>>
>> Thanks,
>> Martin
>>
>>
>>
>>
>> --
>> Thanks & Regards,
>> Anil Gupta
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by Arun C Murthy <ac...@hortonworks.com>.
http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html
On Aug 16, 2012, at 10:37 AM, mg wrote:
> Hi,
>
> thanks.
>
> But there are many more mapreduce.* properties. ...
>
> Does anyone have more information on those with respect to whether they are relevant to YARN, and if so, what they exactly mean in the YARN context?
>
> I am trying to optimize settings in order to use as much RAM and CPU as possible. The cluster has 8 node managers each with 96GB RAM and 16 CPUs (cpuinfo: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz)
>
> Cheers,
> Martin
>
> On 16.08.2012 18:40, anil gupta wrote:
>> Hi,
>>
>> AFAIK, these properties are being ignored by YARN:
>> - mapreduce.tasktracker.map.tasks.maximum,
>> - mapreduce.tasktracker.reduce.tasks.maximum
>>
>> Thanks,
>> Anil Gupta
>>
>>
>> On Thu, Aug 16, 2012 at 9:28 AM, mg <userformailinglists@gmail.com
>> <ma...@gmail.com>> wrote:
>>
>> Hi,
>>
>> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha)
>> cluster running HDFS, YARN, and HBase managed by Cloudera Manager
>> 4.0.3 (Free Edition).
>>
>>
>> In CM, there are a number of options for setting mapreduce.*
>> configuration properties on the YARN client page.
>>
>> Some of the properties and their explanations in the GUI still refer
>> to JobTracker and TaskTracker, e.g.,
>>
>> - mapreduce.jobtracker.handler.__count,
>> - mapreduce.tasktracker.map.__tasks.maximum,
>> - mapreduce.tasktracker.reduce.__tasks.maximum
>>
>> I wonder whether these and a number of other mapreduce.* (e.g.,
>> mapreduce.job.reduces) properties are observed by the MR2
>> ApplicationMaster (and how), or not.
>>
>>
>> Can anyone clarify or point to respective documentation?
>>
>> Thanks,
>> Martin
>>
>>
>>
>>
>> --
>> Thanks & Regards,
>> Anil Gupta
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by Arun C Murthy <ac...@hortonworks.com>.
http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html
On Aug 16, 2012, at 10:37 AM, mg wrote:
> Hi,
>
> thanks.
>
> But there are many more mapreduce.* properties. ...
>
> Does anyone have more information on those with respect to whether they are relevant to YARN, and if so, what they exactly mean in the YARN context?
>
> I am trying to optimize settings in order to use as much RAM and CPU as possible. The cluster has 8 node managers each with 96GB RAM and 16 CPUs (cpuinfo: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz)
>
> Cheers,
> Martin
>
> On 16.08.2012 18:40, anil gupta wrote:
>> Hi,
>>
>> AFAIK, these properties are being ignored by YARN:
>> - mapreduce.tasktracker.map.tasks.maximum,
>> - mapreduce.tasktracker.reduce.tasks.maximum
>>
>> Thanks,
>> Anil Gupta
>>
>>
>> On Thu, Aug 16, 2012 at 9:28 AM, mg <userformailinglists@gmail.com
>> <ma...@gmail.com>> wrote:
>>
>> Hi,
>>
>> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha)
>> cluster running HDFS, YARN, and HBase managed by Cloudera Manager
>> 4.0.3 (Free Edition).
>>
>>
>> In CM, there are a number of options for setting mapreduce.*
>> configuration properties on the YARN client page.
>>
>> Some of the properties and their explanations in the GUI still refer
>> to JobTracker and TaskTracker, e.g.,
>>
>> - mapreduce.jobtracker.handler.__count,
>> - mapreduce.tasktracker.map.__tasks.maximum,
>> - mapreduce.tasktracker.reduce.__tasks.maximum
>>
>> I wonder whether these and a number of other mapreduce.* (e.g.,
>> mapreduce.job.reduces) properties are observed by the MR2
>> ApplicationMaster (and how), or not.
>>
>>
>> Can anyone clarify or point to respective documentation?
>>
>> Thanks,
>> Martin
>>
>>
>>
>>
>> --
>> Thanks & Regards,
>> Anil Gupta
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by Arun C Murthy <ac...@hortonworks.com>.
http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html
On Aug 16, 2012, at 10:37 AM, mg wrote:
> Hi,
>
> thanks.
>
> But there are many more mapreduce.* properties. ...
>
> Does anyone have more information on those with respect to whether they are relevant to YARN, and if so, what they exactly mean in the YARN context?
>
> I am trying to optimize settings in order to use as much RAM and CPU as possible. The cluster has 8 node managers each with 96GB RAM and 16 CPUs (cpuinfo: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz)
>
> Cheers,
> Martin
>
> On 16.08.2012 18:40, anil gupta wrote:
>> Hi,
>>
>> AFAIK, these properties are being ignored by YARN:
>> - mapreduce.tasktracker.map.tasks.maximum,
>> - mapreduce.tasktracker.reduce.tasks.maximum
>>
>> Thanks,
>> Anil Gupta
>>
>>
>> On Thu, Aug 16, 2012 at 9:28 AM, mg <userformailinglists@gmail.com
>> <ma...@gmail.com>> wrote:
>>
>> Hi,
>>
>> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha)
>> cluster running HDFS, YARN, and HBase managed by Cloudera Manager
>> 4.0.3 (Free Edition).
>>
>>
>> In CM, there are a number of options for setting mapreduce.*
>> configuration properties on the YARN client page.
>>
>> Some of the properties and their explanations in the GUI still refer
>> to JobTracker and TaskTracker, e.g.,
>>
>> - mapreduce.jobtracker.handler.__count,
>> - mapreduce.tasktracker.map.__tasks.maximum,
>> - mapreduce.tasktracker.reduce.__tasks.maximum
>>
>> I wonder whether these and a number of other mapreduce.* (e.g.,
>> mapreduce.job.reduces) properties are observed by the MR2
>> ApplicationMaster (and how), or not.
>>
>>
>> Can anyone clarify or point to respective documentation?
>>
>> Thanks,
>> Martin
>>
>>
>>
>>
>> --
>> Thanks & Regards,
>> Anil Gupta
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by mg <us...@gmail.com>.
Hi,
thanks.
But there are many more mapreduce.* properties. ...
Does anyone have more information on those with respect to whether they
are relevant to YARN, and if so, what they exactly mean in the YARN context?
I am trying to optimize settings in order to use as much RAM and CPU as
possible. The cluster has 8 node managers each with 96GB RAM and 16 CPUs
(cpuinfo: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz)
Cheers,
Martin
On 16.08.2012 18:40, anil gupta wrote:
> Hi,
>
> AFAIK, these properties are being ignored by YARN:
> - mapreduce.tasktracker.map.tasks.maximum,
> - mapreduce.tasktracker.reduce.tasks.maximum
>
> Thanks,
> Anil Gupta
>
>
> On Thu, Aug 16, 2012 at 9:28 AM, mg <userformailinglists@gmail.com
> <ma...@gmail.com>> wrote:
>
> Hi,
>
> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha)
> cluster running HDFS, YARN, and HBase managed by Cloudera Manager
> 4.0.3 (Free Edition).
>
>
> In CM, there are a number of options for setting mapreduce.*
> configuration properties on the YARN client page.
>
> Some of the properties and their explanations in the GUI still refer
> to JobTracker and TaskTracker, e.g.,
>
> - mapreduce.jobtracker.handler.__count,
> - mapreduce.tasktracker.map.__tasks.maximum,
> - mapreduce.tasktracker.reduce.__tasks.maximum
>
> I wonder whether these and a number of other mapreduce.* (e.g.,
> mapreduce.job.reduces) properties are observed by the MR2
> ApplicationMaster (and how), or not.
>
>
> Can anyone clarify or point to respective documentation?
>
> Thanks,
> Martin
>
>
>
>
> --
> Thanks & Regards,
> Anil Gupta
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by mg <us...@gmail.com>.
Hi,
thanks.
But there are many more mapreduce.* properties. ...
Does anyone have more information on those with respect to whether they
are relevant to YARN, and if so, what they exactly mean in the YARN context?
I am trying to optimize settings in order to use as much RAM and CPU as
possible. The cluster has 8 node managers each with 96GB RAM and 16 CPUs
(cpuinfo: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz)
Cheers,
Martin
On 16.08.2012 18:40, anil gupta wrote:
> Hi,
>
> AFAIK, these properties are being ignored by YARN:
> - mapreduce.tasktracker.map.tasks.maximum,
> - mapreduce.tasktracker.reduce.tasks.maximum
>
> Thanks,
> Anil Gupta
>
>
> On Thu, Aug 16, 2012 at 9:28 AM, mg <userformailinglists@gmail.com
> <ma...@gmail.com>> wrote:
>
> Hi,
>
> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha)
> cluster running HDFS, YARN, and HBase managed by Cloudera Manager
> 4.0.3 (Free Edition).
>
>
> In CM, there are a number of options for setting mapreduce.*
> configuration properties on the YARN client page.
>
> Some of the properties and their explanations in the GUI still refer
> to JobTracker and TaskTracker, e.g.,
>
> - mapreduce.jobtracker.handler.__count,
> - mapreduce.tasktracker.map.__tasks.maximum,
> - mapreduce.tasktracker.reduce.__tasks.maximum
>
> I wonder whether these and a number of other mapreduce.* (e.g.,
> mapreduce.job.reduces) properties are observed by the MR2
> ApplicationMaster (and how), or not.
>
>
> Can anyone clarify or point to respective documentation?
>
> Thanks,
> Martin
>
>
>
>
> --
> Thanks & Regards,
> Anil Gupta
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by mg <us...@gmail.com>.
Hi,
thanks.
But there are many more mapreduce.* properties. ...
Does anyone have more information on those with respect to whether they
are relevant to YARN, and if so, what they exactly mean in the YARN context?
I am trying to optimize settings in order to use as much RAM and CPU as
possible. The cluster has 8 node managers each with 96GB RAM and 16 CPUs
(cpuinfo: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz)
Cheers,
Martin
On 16.08.2012 18:40, anil gupta wrote:
> Hi,
>
> AFAIK, these properties are being ignored by YARN:
> - mapreduce.tasktracker.map.tasks.maximum,
> - mapreduce.tasktracker.reduce.tasks.maximum
>
> Thanks,
> Anil Gupta
>
>
> On Thu, Aug 16, 2012 at 9:28 AM, mg <userformailinglists@gmail.com
> <ma...@gmail.com>> wrote:
>
> Hi,
>
> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha)
> cluster running HDFS, YARN, and HBase managed by Cloudera Manager
> 4.0.3 (Free Edition).
>
>
> In CM, there are a number of options for setting mapreduce.*
> configuration properties on the YARN client page.
>
> Some of the properties and their explanations in the GUI still refer
> to JobTracker and TaskTracker, e.g.,
>
> - mapreduce.jobtracker.handler.__count,
> - mapreduce.tasktracker.map.__tasks.maximum,
> - mapreduce.tasktracker.reduce.__tasks.maximum
>
> I wonder whether these and a number of other mapreduce.* (e.g.,
> mapreduce.job.reduces) properties are observed by the MR2
> ApplicationMaster (and how), or not.
>
>
> Can anyone clarify or point to respective documentation?
>
> Thanks,
> Martin
>
>
>
>
> --
> Thanks & Regards,
> Anil Gupta
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by mg <us...@gmail.com>.
Hi,
thanks.
But there are many more mapreduce.* properties. ...
Does anyone have more information on those with respect to whether they
are relevant to YARN, and if so, what they exactly mean in the YARN context?
I am trying to optimize settings in order to use as much RAM and CPU as
possible. The cluster has 8 node managers each with 96GB RAM and 16 CPUs
(cpuinfo: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz)
Cheers,
Martin
On 16.08.2012 18:40, anil gupta wrote:
> Hi,
>
> AFAIK, these properties are being ignored by YARN:
> - mapreduce.tasktracker.map.tasks.maximum,
> - mapreduce.tasktracker.reduce.tasks.maximum
>
> Thanks,
> Anil Gupta
>
>
> On Thu, Aug 16, 2012 at 9:28 AM, mg <userformailinglists@gmail.com
> <ma...@gmail.com>> wrote:
>
> Hi,
>
> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha)
> cluster running HDFS, YARN, and HBase managed by Cloudera Manager
> 4.0.3 (Free Edition).
>
>
> In CM, there are a number of options for setting mapreduce.*
> configuration properties on the YARN client page.
>
> Some of the properties and their explanations in the GUI still refer
> to JobTracker and TaskTracker, e.g.,
>
> - mapreduce.jobtracker.handler.__count,
> - mapreduce.tasktracker.map.__tasks.maximum,
> - mapreduce.tasktracker.reduce.__tasks.maximum
>
> I wonder whether these and a number of other mapreduce.* (e.g.,
> mapreduce.job.reduces) properties are observed by the MR2
> ApplicationMaster (and how), or not.
>
>
> Can anyone clarify or point to respective documentation?
>
> Thanks,
> Martin
>
>
>
>
> --
> Thanks & Regards,
> Anil Gupta
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by anil gupta <an...@gmail.com>.
Hi,
AFAIK, these properties are being ignored by YARN:
- mapreduce.tasktracker.map.tasks.maximum,
- mapreduce.tasktracker.reduce.tasks.maximum
Thanks,
Anil Gupta
On Thu, Aug 16, 2012 at 9:28 AM, mg <us...@gmail.com> wrote:
> Hi,
>
> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha) cluster
> running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3 (Free
> Edition).
>
>
> In CM, there are a number of options for setting mapreduce.* configuration
> properties on the YARN client page.
>
> Some of the properties and their explanations in the GUI still refer to
> JobTracker and TaskTracker, e.g.,
>
> - mapreduce.jobtracker.handler.**count,
> - mapreduce.tasktracker.map.**tasks.maximum,
> - mapreduce.tasktracker.reduce.**tasks.maximum
>
> I wonder whether these and a number of other mapreduce.* (e.g.,
> mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
> (and how), or not.
>
>
> Can anyone clarify or point to respective documentation?
>
> Thanks,
> Martin
>
--
Thanks & Regards,
Anil Gupta
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by anil gupta <an...@gmail.com>.
Hi,
AFAIK, these properties are being ignored by YARN:
- mapreduce.tasktracker.map.tasks.maximum,
- mapreduce.tasktracker.reduce.tasks.maximum
Thanks,
Anil Gupta
On Thu, Aug 16, 2012 at 9:28 AM, mg <us...@gmail.com> wrote:
> Hi,
>
> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha) cluster
> running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3 (Free
> Edition).
>
>
> In CM, there are a number of options for setting mapreduce.* configuration
> properties on the YARN client page.
>
> Some of the properties and their explanations in the GUI still refer to
> JobTracker and TaskTracker, e.g.,
>
> - mapreduce.jobtracker.handler.**count,
> - mapreduce.tasktracker.map.**tasks.maximum,
> - mapreduce.tasktracker.reduce.**tasks.maximum
>
> I wonder whether these and a number of other mapreduce.* (e.g.,
> mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
> (and how), or not.
>
>
> Can anyone clarify or point to respective documentation?
>
> Thanks,
> Martin
>
--
Thanks & Regards,
Anil Gupta
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by rahul p <ra...@gmail.com>.
Hi,
When i give bin/start-fs all
namenode s not coming up..
On Fri, Aug 17, 2012 at 12:28 AM, mg <us...@gmail.com> wrote:
> Hi,
>
> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha) cluster
> running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3 (Free
> Edition).
>
> In CM, there are a number of options for setting mapreduce.* configuration
> properties on the YARN client page.
>
> Some of the properties and their explanations in the GUI still refer to
> JobTracker and TaskTracker, e.g.,
> - mapreduce.jobtracker.handler.**count,
> - mapreduce.tasktracker.map.**tasks.maximum,
> - mapreduce.tasktracker.reduce.**tasks.maximum
>
> I wonder whether these and a number of other mapreduce.* (e.g.,
> mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
> (and how), or not.
>
> Can anyone clarify or point to respective documentation?
>
> Thanks,
> Martin
>
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by rahul p <ra...@gmail.com>.
Hi,
When i give bin/start-fs all
namenode s not coming up..
On Fri, Aug 17, 2012 at 12:28 AM, mg <us...@gmail.com> wrote:
> Hi,
>
> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha) cluster
> running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3 (Free
> Edition).
>
> In CM, there are a number of options for setting mapreduce.* configuration
> properties on the YARN client page.
>
> Some of the properties and their explanations in the GUI still refer to
> JobTracker and TaskTracker, e.g.,
> - mapreduce.jobtracker.handler.**count,
> - mapreduce.tasktracker.map.**tasks.maximum,
> - mapreduce.tasktracker.reduce.**tasks.maximum
>
> I wonder whether these and a number of other mapreduce.* (e.g.,
> mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
> (and how), or not.
>
> Can anyone clarify or point to respective documentation?
>
> Thanks,
> Martin
>
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by rahul p <ra...@gmail.com>.
Hi,
When i give bin/start-fs all
namenode s not coming up..
On Fri, Aug 17, 2012 at 12:28 AM, mg <us...@gmail.com> wrote:
> Hi,
>
> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha) cluster
> running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3 (Free
> Edition).
>
> In CM, there are a number of options for setting mapreduce.* configuration
> properties on the YARN client page.
>
> Some of the properties and their explanations in the GUI still refer to
> JobTracker and TaskTracker, e.g.,
> - mapreduce.jobtracker.handler.**count,
> - mapreduce.tasktracker.map.**tasks.maximum,
> - mapreduce.tasktracker.reduce.**tasks.maximum
>
> I wonder whether these and a number of other mapreduce.* (e.g.,
> mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
> (and how), or not.
>
> Can anyone clarify or point to respective documentation?
>
> Thanks,
> Martin
>
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by anil gupta <an...@gmail.com>.
Hi,
AFAIK, these properties are being ignored by YARN:
- mapreduce.tasktracker.map.tasks.maximum,
- mapreduce.tasktracker.reduce.tasks.maximum
Thanks,
Anil Gupta
On Thu, Aug 16, 2012 at 9:28 AM, mg <us...@gmail.com> wrote:
> Hi,
>
> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha) cluster
> running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3 (Free
> Edition).
>
>
> In CM, there are a number of options for setting mapreduce.* configuration
> properties on the YARN client page.
>
> Some of the properties and their explanations in the GUI still refer to
> JobTracker and TaskTracker, e.g.,
>
> - mapreduce.jobtracker.handler.**count,
> - mapreduce.tasktracker.map.**tasks.maximum,
> - mapreduce.tasktracker.reduce.**tasks.maximum
>
> I wonder whether these and a number of other mapreduce.* (e.g.,
> mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
> (and how), or not.
>
>
> Can anyone clarify or point to respective documentation?
>
> Thanks,
> Martin
>
--
Thanks & Regards,
Anil Gupta
Re: Relevance of mapreduce.* configuration properties for MR V2
Posted by anil gupta <an...@gmail.com>.
Hi,
AFAIK, these properties are being ignored by YARN:
- mapreduce.tasktracker.map.tasks.maximum,
- mapreduce.tasktracker.reduce.tasks.maximum
Thanks,
Anil Gupta
On Thu, Aug 16, 2012 at 9:28 AM, mg <us...@gmail.com> wrote:
> Hi,
>
> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha) cluster
> running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3 (Free
> Edition).
>
>
> In CM, there are a number of options for setting mapreduce.* configuration
> properties on the YARN client page.
>
> Some of the properties and their explanations in the GUI still refer to
> JobTracker and TaskTracker, e.g.,
>
> - mapreduce.jobtracker.handler.**count,
> - mapreduce.tasktracker.map.**tasks.maximum,
> - mapreduce.tasktracker.reduce.**tasks.maximum
>
> I wonder whether these and a number of other mapreduce.* (e.g.,
> mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
> (and how), or not.
>
>
> Can anyone clarify or point to respective documentation?
>
> Thanks,
> Martin
>
--
Thanks & Regards,
Anil Gupta