You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Blargy <zm...@hotmail.com> on 2010/06/24 06:11:07 UTC
Newb question on HDFS
Hey all. Im trying to start up a psuedo distributed setup following the
directions in Hadoop In Action however I am having problems. For some reason
my TaskTracker and JobTracker are not starting up but there is nothing in my
log files explaining why.
FYI My development machine is a Mac.
hdfs-site.xml
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:54310</value>
<description>
The name of the default file system. A URI whose scheme and authority
determine the FileSystem implementation.
</description>
</property>
<property>
<name>mapred.job.tracker</name>
<value>localhost.com:54311</value>
<description>
The host and port that the MapReduce job tracker runs at.
</description>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/Users/root/workspace/hadoop/datastore/pseudo/</value>
<description>
A base for other temporary directories.
</description>
</property>
<property>
<name>dfs.replication</name>
<value>1</value>
<description>
The actual number of replications can be specified when the file is
created.
</description>
</property>
</configuration>
./bin/start-all.sh output
starting namenode, logging to ...
localhost: starting datanode, logging to ...
localhost: starting secondarynamenode, logging to ...
starting jobtracker, logging to ...
localhost: starting tasktracker, logging to
JPS output
$ jps
4317 DataNode
4384 SecondaryNameNode
4250 NameNode
467 Main
4561 Jps
--
View this message in context: http://lucene.472066.n3.nabble.com/Newb-question-on-HDFS-tp918683p918683.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
Re: Newb question on HDFS
Posted by Jeff Zhang <zj...@gmail.com>.
change localhost.com to localhost
On Thu, Jun 24, 2010 at 12:11 PM, Blargy <zm...@hotmail.com> wrote:
>
> Hey all. Im trying to start up a psuedo distributed setup following the
> directions in Hadoop In Action however I am having problems. For some reason
> my TaskTracker and JobTracker are not starting up but there is nothing in my
> log files explaining why.
>
> FYI My development machine is a Mac.
>
> hdfs-site.xml
>
> <configuration>
>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:54310</value>
> <description>
> The name of the default file system. A URI whose scheme and authority
> determine the FileSystem implementation.
> </description>
> </property>
>
> <property>
> <name>mapred.job.tracker</name>
> <value>localhost.com:54311</value>
> <description>
> The host and port that the MapReduce job tracker runs at.
> </description>
> </property>
>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/Users/root/workspace/hadoop/datastore/pseudo/</value>
> <description>
> A base for other temporary directories.
> </description>
> </property>
>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> <description>
> The actual number of replications can be specified when the file is
> created.
> </description>
> </property>
>
> </configuration>
>
> ./bin/start-all.sh output
>
> starting namenode, logging to ...
> localhost: starting datanode, logging to ...
> localhost: starting secondarynamenode, logging to ...
> starting jobtracker, logging to ...
> localhost: starting tasktracker, logging to
>
> JPS output
>
> $ jps
> 4317 DataNode
> 4384 SecondaryNameNode
> 4250 NameNode
> 467 Main
> 4561 Jps
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Newb-question-on-HDFS-tp918683p918683.html
> Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
>
--
Best Regards
Jeff Zhang
Re: Newb question on HDFS
Posted by Blargy <zm...@hotmail.com>.
I think I figured it out. I had the above configurations in hdfs-site.xml not
hadoop-site.xml. Once I changed that it worked.
--
View this message in context: http://lucene.472066.n3.nabble.com/Newb-question-on-HDFS-tp918683p920090.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
Re: Newb question on HDFS
Posted by Blargy <zm...@hotmail.com>.
Thanks Chris however I've already tried that as well as changing around the
ports.
Ill try looking at tutorial though... thanks
--
View this message in context: http://lucene.472066.n3.nabble.com/Newb-question-on-HDFS-tp918683p919852.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
Re: Newb question on HDFS
Posted by Chris Collord <cc...@lanl.gov>.
Hello, from one newb to another!
You have this:
<property>
<name>mapred.job.tracker</name>
<value>localhost.com:54311</value>
<description>
The host and port that the MapReduce job tracker runs at.
</description>
</property>
You need to get rid of the ".com" on localhost. It should just say "localhost".
It's actually preferred that you use the FQDN instead, but if you're just trying this out on one machine, it's probably not necessary.
Here's an excellent tutorial to get you started:
http://www.michael-noll.com/wiki/Running_Hadoop_On_Ubuntu_Linux_%28Single-Node_Cluster%29
~Chris
On 06/24/2010 09:01 AM, Blargy wrote:
> Found this in the jobtracker logs...
>
> 2010-06-24 07:59:27,233 FATAL org.apache.hadoop.mapred.JobTracker:
> java.lang.RuntimeException: Not a host:port pair: local
>
>
> Any idea?
>
--
------------------------------
Chris Collord, ACS-PO 9/80 A
------------------------------
Re: Newb question on HDFS
Posted by Blargy <zm...@hotmail.com>.
Found this in the jobtracker logs...
2010-06-24 07:59:27,233 FATAL org.apache.hadoop.mapred.JobTracker:
java.lang.RuntimeException: Not a host:port pair: local
Any idea?
--
View this message in context: http://lucene.472066.n3.nabble.com/Newb-question-on-HDFS-tp918683p919699.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
Re: Newb question on HDFS
Posted by Edward Choi <mp...@gmail.com>.
I am not a hadoop expert but maybe your machine could already be using port 54310. Worth checking.
From mp2893's iPhone
2010. 6. 24. 오후 1:11 Blargy <zm...@hotmail.com> 작성:
>
> Hey all. Im trying to start up a psuedo distributed setup following the
> directions in Hadoop In Action however I am having problems. For some reason
> my TaskTracker and JobTracker are not starting up but there is nothing in my
> log files explaining why.
>
> FYI My development machine is a Mac.
>
> hdfs-site.xml
>
> <configuration>
>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:54310</value>
> <description>
> The name of the default file system. A URI whose scheme and authority
> determine the FileSystem implementation.
> </description>
> </property>
>
> <property>
> <name>mapred.job.tracker</name>
> <value>localhost.com:54311</value>
> <description>
> The host and port that the MapReduce job tracker runs at.
> </description>
> </property>
>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/Users/root/workspace/hadoop/datastore/pseudo/</value>
> <description>
> A base for other temporary directories.
> </description>
> </property>
>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> <description>
> The actual number of replications can be specified when the file is
> created.
> </description>
> </property>
>
> </configuration>
>
> ./bin/start-all.sh output
>
> starting namenode, logging to ...
> localhost: starting datanode, logging to ...
> localhost: starting secondarynamenode, logging to ...
> starting jobtracker, logging to ...
> localhost: starting tasktracker, logging to
>
> JPS output
>
> $ jps
> 4317 DataNode
> 4384 SecondaryNameNode
> 4250 NameNode
> 467 Main
> 4561 Jps
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Newb-question-on-HDFS-tp918683p918683.html
> Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
Re: Newb question on HDFS
Posted by Mafish Liu <ma...@gmail.com>.
2010/6/24 Blargy <zm...@hotmail.com>:
>
> Hey all. Im trying to start up a psuedo distributed setup following the
> directions in Hadoop In Action however I am having problems. For some reason
> my TaskTracker and JobTracker are not starting up but there is nothing in my
> log files explaining why.
>
> FYI My development machine is a Mac.
>
> hdfs-site.xml
>
> <configuration>
>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:54310</value>
> <description>
> The name of the default file system. A URI whose scheme and authority
> determine the FileSystem implementation.
> </description>
> </property>
>
> <property>
> <name>mapred.job.tracker</name>
> <value>localhost.com:54311</value>
Can you access localhost.com ?
> <description>
> The host and port that the MapReduce job tracker runs at.
> </description>
> </property>
>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/Users/root/workspace/hadoop/datastore/pseudo/</value>
> <description>
> A base for other temporary directories.
> </description>
> </property>
>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> <description>
> The actual number of replications can be specified when the file is
> created.
> </description>
> </property>
>
> </configuration>
>
> ./bin/start-all.sh output
>
> starting namenode, logging to ...
> localhost: starting datanode, logging to ...
> localhost: starting secondarynamenode, logging to ...
> starting jobtracker, logging to ...
> localhost: starting tasktracker, logging to
>
> JPS output
>
> $ jps
> 4317 DataNode
> 4384 SecondaryNameNode
> 4250 NameNode
> 467 Main
> 4561 Jps
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Newb-question-on-HDFS-tp918683p918683.html
> Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
>