You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by kvorion <kv...@gmail.com> on 2009/11/12 18:02:58 UTC
About Hadoop pseudo distribution
Hi All,
I have been trying to set up a hadoop cluster on a number of machines, a few
of which are multicore machines. I have been wondering whether the hadoop
pseudo distribution is something that can help me take advantage of the
multiple cores on my machines. All the tutorials say that the pseudo
distribution mode lets you start each daemon in a separate java process. I
have the following configuration settings for hadoop-site.xml:
<property>
<name>fs.default.name</name>
<value>hdfs://athena:9000</value>
</property>
<property>
<name>mapred.job.tracker</name>
<value>athena:9001</value>
</property>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
I am not sure if this is really running in the pseudo-distribution mode. Are
there any indicators or outputs that confirm what mode you are running in?
--
View this message in context: http://old.nabble.com/About-Hadoop-pseudo-distribution-tp26322382p26322382.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.
Re: About Hadoop pseudo distribution
Posted by Doss_IPH <do...@intellipowerhive.com>.
Hi
Hadoop pseudo distribution runs five java processors which are given below
1. namenode,
2. secondarynamenode
3. datanode
4 jobtracker
5. tasktracker
As you know namenode, secondarynamenode, datanode processors are for HDFS
and jobtracker ,tastktracker are for MR (Map Reduce).
kvorion wrote:
>
> Hi All,
>
> I have been trying to set up a hadoop cluster on a number of machines, a
> few of which are multicore machines. I have been wondering whether the
> hadoop pseudo distribution is something that can help me take advantage of
> the multiple cores on my machines. All the tutorials say that the pseudo
> distribution mode lets you start each daemon in a separate java process. I
> have the following configuration settings for hadoop-site.xml:
>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://athena:9000</value>
> </property>
>
> <property>
> <name>mapred.job.tracker</name>
> <value>athena:9001</value>
> </property>
>
> <property>
> <name>dfs.replication</name>
> <value>2</value>
> </property>
>
> I am not sure if this is really running in the pseudo-distribution mode.
> Are there any indicators or outputs that confirm what mode you are running
> in?
>
>
>
--
View this message in context: http://old.nabble.com/About-Hadoop-pseudo-distribution-tp26322382p26605201.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.
Re: About Hadoop pseudo distribution
Posted by Steve Loughran <st...@apache.org>.
kvorion wrote:
> Hi All,
>
> I have been trying to set up a hadoop cluster on a number of machines, a few
> of which are multicore machines. I have been wondering whether the hadoop
> pseudo distribution is something that can help me take advantage of the
> multiple cores on my machines. All the tutorials say that the pseudo
> distribution mode lets you start each daemon in a separate java process. I
> have the following configuration settings for hadoop-site.xml:
You should just give every node with more cores more task tracker slots;
this will give it more work from the Job Tracker
Re: About Hadoop pseudo distribution
Posted by Raymond Jennings III <ra...@yahoo.com>.
If I understand you correctly you can run "jps" and see the java jvm's running on each machine - that should tell you if you are running in pseudo mode or not.
--- On Thu, 11/12/09, kvorion <kv...@gmail.com> wrote:
> From: kvorion <kv...@gmail.com>
> Subject: About Hadoop pseudo distribution
> To: core-user@hadoop.apache.org
> Date: Thursday, November 12, 2009, 12:02 PM
>
> Hi All,
>
> I have been trying to set up a hadoop cluster on a number
> of machines, a few
> of which are multicore machines. I have been wondering
> whether the hadoop
> pseudo distribution is something that can help me take
> advantage of the
> multiple cores on my machines. All the tutorials say that
> the pseudo
> distribution mode lets you start each daemon in a separate
> java process. I
> have the following configuration settings for
> hadoop-site.xml:
>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://athena:9000</value>
> </property>
>
> <property>
> <name>mapred.job.tracker</name>
> <value>athena:9001</value>
> </property>
>
> <property>
> <name>dfs.replication</name>
> <value>2</value>
> </property>
>
> I am not sure if this is really running in the
> pseudo-distribution mode. Are
> there any indicators or outputs that confirm what mode you
> are running in?
>
>
> --
> View this message in context: http://old.nabble.com/About-Hadoop-pseudo-distribution-tp26322382p26322382.html
> Sent from the Hadoop core-user mailing list archive at
> Nabble.com.
>
>