You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Bill Bruns <bi...@yahoo.com> on 2014/02/05 09:32:35 UTC

Where to find a list of most Hadoop2 config file parameters?

Hello Folks,

Is there a fairly complete list somewhere, giving most of the parameters from the configuration files for Apache Hadoop2 and MapReduce?
(Including Yarn)

I am especially interested in parameters that affect the activities of the ResouceManager and Application Masters, such as the parameters that are found in capacity-scheduler.xml
 
Bill 

________________________________
 From: VJ Shalish <vj...@gmail.com>
To: user@hadoop.apache.org; Raj Hadoop <ha...@yahoo.com> 
Cc: SCM Users <sc...@cloudera.org> 
Sent: Tuesday, February 4, 2014 10:10 PM
Subject: Re: A Hadoop Cluster @Home using CentOS 6.5, Cloudera Manager 4.8.1 and Cloudera Parcels
 



Hi Raj,
 
Thankyou.
I think you would be able to setup virtual machines in Mac as well.
 
Thanks
Shalish.

On Wed, Feb 5, 2014 at 2:48 AM, Raj Hadoop <ha...@yahoo.com> wrote:


>Hi Shalish -
>
>
>This is a really wonderful work. Let me go through it.
>
>
>And one more thing - Can we use this setup on a Mac computer. Is it OS dependent ? Please advise.
>
>
>Thanks,
>Raj
>
>
>
>
>On Tuesday, February 4, 2014 3:47 PM, VJ Shalish <vj...@gmail.com> wrote:
>
>Hi All,
> 
>        Based on my research and experience, I am starting a blog with detailed steps which would help the Bigdata beginners and freaks to set up their own 
>Hadoop Cluster in your laptop using the latest Cloudera Manager and Cloudera Parcels. I have completed 2 Episodes. Seeking your valuable feedback.
> 
>http://shalishvj.wordpress.com/2014/02/03/a-hadoop-cluster-home-creating-a-virtual-machine-with-centos-6-5-hadoop-cluster-installation-using-cloudera-manager-and-cloudera-parcels/
>
>http://shalishvj.wordpress.com/2014/02/04/a-hadoop-cluster-home-using-centos-6-5-cloudera-manager-4-8-1-and-cloudera-parcels-episode-2/
> 
>Thanks
>Shalish.
>
>

Re: Where to find a list of most Hadoop2 config file parameters?

Posted by Harsh J <ha...@cloudera.com>.
Hey Bill,

Most of the documented parameters can be found under the
yarn-default.xml and mapred-default.xml with their descriptions:
http://hadoop.apache.org/docs/stable2/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
and http://hadoop.apache.org/docs/stable2/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml.
Is this what you're looking for?

On Wed, Feb 5, 2014 at 2:02 PM, Bill Bruns <bi...@yahoo.com> wrote:
> Hello Folks,
>
> Is there a fairly complete list somewhere, giving most of the parameters
> from the configuration files for Apache Hadoop2 and MapReduce?
> (Including Yarn)
>
> I am especially interested in parameters that affect the activities of the
> ResouceManager and Application Masters, such as the parameters that are
> found in capacity-scheduler.xml
>
> Bill
> ________________________________
> From: VJ Shalish <vj...@gmail.com>
> To: user@hadoop.apache.org; Raj Hadoop <ha...@yahoo.com>
> Cc: SCM Users <sc...@cloudera.org>
> Sent: Tuesday, February 4, 2014 10:10 PM
> Subject: Re: A Hadoop Cluster @Home using CentOS 6.5, Cloudera Manager 4.8.1
> and Cloudera Parcels
>
>
> Hi Raj,
>
> Thankyou.
> I think you would be able to setup virtual machines in Mac as well.
>
> Thanks
> Shalish.
> On Wed, Feb 5, 2014 at 2:48 AM, Raj Hadoop <ha...@yahoo.com> wrote:
>
>
> Hi Shalish -
>
> This is a really wonderful work. Let me go through it.
>
> And one more thing - Can we use this setup on a Mac computer. Is it OS
> dependent ? Please advise.
>
> Thanks,
> Raj
>
>
> On Tuesday, February 4, 2014 3:47 PM, VJ Shalish <vj...@gmail.com>
> wrote:
> Hi All,
>
>         Based on my research and experience, I am starting a blog with
> detailed steps which would help the Bigdata beginners and freaks to set up
> their own
> Hadoop Cluster in your laptop using the latest Cloudera Manager and Cloudera
> Parcels. I have completed 2 Episodes. Seeking your valuable feedback.
>
> http://shalishvj.wordpress.com/2014/02/03/a-hadoop-cluster-home-creating-a-virtual-machine-with-centos-6-5-hadoop-cluster-installation-using-cloudera-manager-and-cloudera-parcels/
>
> http://shalishvj.wordpress.com/2014/02/04/a-hadoop-cluster-home-using-centos-6-5-cloudera-manager-4-8-1-and-cloudera-parcels-episode-2/
>
> Thanks
> Shalish.
>
>
>
>
>



-- 
Harsh J

Re: Where to find a list of most Hadoop2 config file parameters?

Posted by Harsh J <ha...@cloudera.com>.
Hey Bill,

Most of the documented parameters can be found under the
yarn-default.xml and mapred-default.xml with their descriptions:
http://hadoop.apache.org/docs/stable2/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
and http://hadoop.apache.org/docs/stable2/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml.
Is this what you're looking for?

On Wed, Feb 5, 2014 at 2:02 PM, Bill Bruns <bi...@yahoo.com> wrote:
> Hello Folks,
>
> Is there a fairly complete list somewhere, giving most of the parameters
> from the configuration files for Apache Hadoop2 and MapReduce?
> (Including Yarn)
>
> I am especially interested in parameters that affect the activities of the
> ResouceManager and Application Masters, such as the parameters that are
> found in capacity-scheduler.xml
>
> Bill
> ________________________________
> From: VJ Shalish <vj...@gmail.com>
> To: user@hadoop.apache.org; Raj Hadoop <ha...@yahoo.com>
> Cc: SCM Users <sc...@cloudera.org>
> Sent: Tuesday, February 4, 2014 10:10 PM
> Subject: Re: A Hadoop Cluster @Home using CentOS 6.5, Cloudera Manager 4.8.1
> and Cloudera Parcels
>
>
> Hi Raj,
>
> Thankyou.
> I think you would be able to setup virtual machines in Mac as well.
>
> Thanks
> Shalish.
> On Wed, Feb 5, 2014 at 2:48 AM, Raj Hadoop <ha...@yahoo.com> wrote:
>
>
> Hi Shalish -
>
> This is a really wonderful work. Let me go through it.
>
> And one more thing - Can we use this setup on a Mac computer. Is it OS
> dependent ? Please advise.
>
> Thanks,
> Raj
>
>
> On Tuesday, February 4, 2014 3:47 PM, VJ Shalish <vj...@gmail.com>
> wrote:
> Hi All,
>
>         Based on my research and experience, I am starting a blog with
> detailed steps which would help the Bigdata beginners and freaks to set up
> their own
> Hadoop Cluster in your laptop using the latest Cloudera Manager and Cloudera
> Parcels. I have completed 2 Episodes. Seeking your valuable feedback.
>
> http://shalishvj.wordpress.com/2014/02/03/a-hadoop-cluster-home-creating-a-virtual-machine-with-centos-6-5-hadoop-cluster-installation-using-cloudera-manager-and-cloudera-parcels/
>
> http://shalishvj.wordpress.com/2014/02/04/a-hadoop-cluster-home-using-centos-6-5-cloudera-manager-4-8-1-and-cloudera-parcels-episode-2/
>
> Thanks
> Shalish.
>
>
>
>
>



-- 
Harsh J

Re: Where to find a list of most Hadoop2 config file parameters?

Posted by Harsh J <ha...@cloudera.com>.
Hey Bill,

Most of the documented parameters can be found under the
yarn-default.xml and mapred-default.xml with their descriptions:
http://hadoop.apache.org/docs/stable2/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
and http://hadoop.apache.org/docs/stable2/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml.
Is this what you're looking for?

On Wed, Feb 5, 2014 at 2:02 PM, Bill Bruns <bi...@yahoo.com> wrote:
> Hello Folks,
>
> Is there a fairly complete list somewhere, giving most of the parameters
> from the configuration files for Apache Hadoop2 and MapReduce?
> (Including Yarn)
>
> I am especially interested in parameters that affect the activities of the
> ResouceManager and Application Masters, such as the parameters that are
> found in capacity-scheduler.xml
>
> Bill
> ________________________________
> From: VJ Shalish <vj...@gmail.com>
> To: user@hadoop.apache.org; Raj Hadoop <ha...@yahoo.com>
> Cc: SCM Users <sc...@cloudera.org>
> Sent: Tuesday, February 4, 2014 10:10 PM
> Subject: Re: A Hadoop Cluster @Home using CentOS 6.5, Cloudera Manager 4.8.1
> and Cloudera Parcels
>
>
> Hi Raj,
>
> Thankyou.
> I think you would be able to setup virtual machines in Mac as well.
>
> Thanks
> Shalish.
> On Wed, Feb 5, 2014 at 2:48 AM, Raj Hadoop <ha...@yahoo.com> wrote:
>
>
> Hi Shalish -
>
> This is a really wonderful work. Let me go through it.
>
> And one more thing - Can we use this setup on a Mac computer. Is it OS
> dependent ? Please advise.
>
> Thanks,
> Raj
>
>
> On Tuesday, February 4, 2014 3:47 PM, VJ Shalish <vj...@gmail.com>
> wrote:
> Hi All,
>
>         Based on my research and experience, I am starting a blog with
> detailed steps which would help the Bigdata beginners and freaks to set up
> their own
> Hadoop Cluster in your laptop using the latest Cloudera Manager and Cloudera
> Parcels. I have completed 2 Episodes. Seeking your valuable feedback.
>
> http://shalishvj.wordpress.com/2014/02/03/a-hadoop-cluster-home-creating-a-virtual-machine-with-centos-6-5-hadoop-cluster-installation-using-cloudera-manager-and-cloudera-parcels/
>
> http://shalishvj.wordpress.com/2014/02/04/a-hadoop-cluster-home-using-centos-6-5-cloudera-manager-4-8-1-and-cloudera-parcels-episode-2/
>
> Thanks
> Shalish.
>
>
>
>
>



-- 
Harsh J

Re: Where to find a list of most Hadoop2 config file parameters?

Posted by Harsh J <ha...@cloudera.com>.
Hey Bill,

Most of the documented parameters can be found under the
yarn-default.xml and mapred-default.xml with their descriptions:
http://hadoop.apache.org/docs/stable2/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
and http://hadoop.apache.org/docs/stable2/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml.
Is this what you're looking for?

On Wed, Feb 5, 2014 at 2:02 PM, Bill Bruns <bi...@yahoo.com> wrote:
> Hello Folks,
>
> Is there a fairly complete list somewhere, giving most of the parameters
> from the configuration files for Apache Hadoop2 and MapReduce?
> (Including Yarn)
>
> I am especially interested in parameters that affect the activities of the
> ResouceManager and Application Masters, such as the parameters that are
> found in capacity-scheduler.xml
>
> Bill
> ________________________________
> From: VJ Shalish <vj...@gmail.com>
> To: user@hadoop.apache.org; Raj Hadoop <ha...@yahoo.com>
> Cc: SCM Users <sc...@cloudera.org>
> Sent: Tuesday, February 4, 2014 10:10 PM
> Subject: Re: A Hadoop Cluster @Home using CentOS 6.5, Cloudera Manager 4.8.1
> and Cloudera Parcels
>
>
> Hi Raj,
>
> Thankyou.
> I think you would be able to setup virtual machines in Mac as well.
>
> Thanks
> Shalish.
> On Wed, Feb 5, 2014 at 2:48 AM, Raj Hadoop <ha...@yahoo.com> wrote:
>
>
> Hi Shalish -
>
> This is a really wonderful work. Let me go through it.
>
> And one more thing - Can we use this setup on a Mac computer. Is it OS
> dependent ? Please advise.
>
> Thanks,
> Raj
>
>
> On Tuesday, February 4, 2014 3:47 PM, VJ Shalish <vj...@gmail.com>
> wrote:
> Hi All,
>
>         Based on my research and experience, I am starting a blog with
> detailed steps which would help the Bigdata beginners and freaks to set up
> their own
> Hadoop Cluster in your laptop using the latest Cloudera Manager and Cloudera
> Parcels. I have completed 2 Episodes. Seeking your valuable feedback.
>
> http://shalishvj.wordpress.com/2014/02/03/a-hadoop-cluster-home-creating-a-virtual-machine-with-centos-6-5-hadoop-cluster-installation-using-cloudera-manager-and-cloudera-parcels/
>
> http://shalishvj.wordpress.com/2014/02/04/a-hadoop-cluster-home-using-centos-6-5-cloudera-manager-4-8-1-and-cloudera-parcels-episode-2/
>
> Thanks
> Shalish.
>
>
>
>
>



-- 
Harsh J