Re: Running tasktrackers on datanodes

[Lucas Nazário dos Santos <na...@gmail.com>]

Very insightful your response.

The problem was with the SSH server. It wasn't working. Now everything seems
to be running ok.

Thanks Ollie,
Lucas


On 7/31/07, ojh06@doc.ic.ac.uk <oj...@doc.ic.ac.uk> wrote:
>
> Hi Lucas,
>
> Sounds strange, it should work. As long as you have all the names of
> the slave machines in the slaves file the start-mapred script should
> ssh into each one and start a tasktracker. Immediate thoughts are: Is
> your ssh working? Is hadoop installed on the same directory structure
> as on the machine you run the script from (ignore if running from NFS
> mount). Have you looked in the error logs on the slave machines, there
> might be some useful information there if the script tried to start a
> tasktracker and failed? What does the console print when you run
> hadoop-mapred.sh? It should say something like "machinex: starting
> tasktracker, logging to /somefile/" for each machine listed in your
> slaves file.
>
> Good luck,
> Ollie
>
> Quoting Lucas Nazário dos Santos <na...@gmail.com>:
>
> > Hello,
> >
> > I was reading Hadoop's getting started (
> > http://wiki.apache.org/lucene-hadoop/GettingStartedWithHadoop), and in
> the
> > section named "Starting up a larger cluster" I had a doubt about
> starting
> > tasktrackers on datanodes. The tutorial says that after running the
> > start-dfs.sh script on the node I want as the namenode, I need to
> execute
> > the script start-mapred.sh, and this will "bring up the Map/Reduce
> cluster
> > with Jobtracker running on the machine you ran the command on and
> > Tasktrackers running on machines listed in the slaves file".
> >
> > It actually brings up the Jobtracker on the machine I ran the command
> on,
> > but not the tasktrackers on slave machines. I need to enter slave by
> slave
> > starting the tasktracker manually, issuing the command "hadoop
> tasktracker"
> > what, according to Hadoop's getting started documentation, is something
> > unnecessary.
> >
> > Did I misunderstand something?
> >
> > Thanks in advance,
> > Lucas
> >
>
>
>
>

Re: Running tasktrackers on datanodes

[Lucas Nazário dos Santos <na...@gmail.com>]

Hello,

There really was a problem with SSH, but it's ok now. When I issue the
command start-all.sh from the master node (after formating the namenode), it
properly connects to the slave. The problem is that the datanode, as well as
the tasktracker are not initialized.

SSH is working (at least, it seems to be)
Hadoop is installed on the same directory structure
(c:\hadoop-install\hadoop) in both master and slave nodes
Nothing is logged within the slave log folder

The unique clue is the console. When I issue the command start-all.sh, it
prints... (192.168.1.149 is the slave; 192.168.0.244 is the master)

$ start-all.sh
starting namenode, logging to
/cygdrive/c/hadoop-install/hadoop/bin/../logs/hado
op-iscurriculum-namenode-luana.out
iscurriculum@192.168.1.149's password:
192.168.1.149: starting datanode, logging to
/cygdrive/c/hadoop-install/hadoop/b
.out./logs/hadoop-iscurriculum-datanode-floripa76
192.168.1.149: /cygdrive/c/hadoop-install/hadoop/bin/hadoop-daemon.sh: line
98:
/cygdrive/c/hadoop-install/hadoop/bin/../logs/hadoop-iscurriculum-datanode-flori
.out: No such file or directory
192.168.1.149: head: cannot open
`/cygdrive/c/hadoop-install/hadoop/bin/../logs/
hadoop-iscurriculum-datanode-floripa76\r.out' for reading: No such file or
direc
tory
iscurriculum@192.168.0.244's password:
192.168.0.244: starting secondarynamenode, logging to
/cygdrive/c/hadoop-install
.outoop/bin/../logs/hadoop-iscurriculum-secondarynamenode-luana
192.168.0.244: cat: cannot open file
/dev/fs/C/tmp/hadoop-iscurriculum-secondary
namenode.pid : No such file or directory
192.168.0.244: /cygdrive/c/hadoop-install/hadoop/bin/hadoop-daemon.sh: line
98:
/cygdrive/c/hadoop-install/hadoop/bin/../logs/hadoop-iscurriculum-secondarynamen
.out: No such file or directory
192.168.0.244: head: Cannot open file
'/dev/fs/C/cygdrive/c/hadoop-install/hadoo
.out': No such file or directorym-secondarynamenode-luana
starting jobtracker, logging to
/cygdrive/c/hadoop-install/hadoop/bin/../logs/ha
doop-iscurriculum-jobtracker-luana.out
iscurriculum@192.168.1.149's password:
192.168.1.149: starting tasktracker, logging to
/cygdrive/c/hadoop-install/hadoo
.outn/../logs/hadoop-iscurriculum-tasktracker-floripa76
192.168.1.149: /cygdrive/c/hadoop-install/hadoop/bin/hadoop-daemon.sh: line
98:
/cygdrive/c/hadoop-install/hadoop/bin/../logs/hadoop-iscurriculum-tasktracker-fl
.out: No such file or directory
192.168.1.149: head: cannot open
`/cygdrive/c/hadoop-install/hadoop/bin/../logs/
hadoop-iscurriculum-tasktracker-floripa76\r.out' for reading: No such file
or di
rectory


Thanks,
Lucas



On 8/2/07, Lucas Nazário dos Santos < nazario.lucas@gmail.com> wrote:
>
> Very insightful your response.
>
> The problem was with the SSH server. It wasn't working. Now everything
> seems to be running ok.
>
> Thanks Ollie,
> Lucas
>
>
> On 7/31/07, ojh06@doc.ic.ac.uk < ojh06@doc.ic.ac.uk> wrote:
> >
> > Hi Lucas,
> >
> > Sounds strange, it should work. As long as you have all the names of
> > the slave machines in the slaves file the start-mapred script should
> > ssh into each one and start a tasktracker. Immediate thoughts are: Is
> > your ssh working? Is hadoop installed on the same directory structure
> > as on the machine you run the script from (ignore if running from NFS
> > mount). Have you looked in the error logs on the slave machines, there
> > might be some useful information there if the script tried to start a
> > tasktracker and failed? What does the console print when you run
> > hadoop-mapred.sh? It should say something like "machinex: starting
> > tasktracker, logging to /somefile/" for each machine listed in your
> > slaves file.
> >
> > Good luck,
> > Ollie
> >
> > Quoting Lucas Nazário dos Santos <nazario.lucas@gmail.com >:
> >
> > > Hello,
> > >
> > > I was reading Hadoop's getting started (
> > > http://wiki.apache.org/lucene-hadoop/GettingStartedWithHadoop), and in
> > the
> > > section named "Starting up a larger cluster" I had a doubt about
> > starting
> > > tasktrackers on datanodes. The tutorial says that after running the
> > > start-dfs.sh script on the node I want as the namenode, I need to
> > execute
> > > the script start-mapred.sh, and this will "bring up the Map/Reduce
> > cluster
> > > with Jobtracker running on the machine you ran the command on and
> > > Tasktrackers running on machines listed in the slaves file".
> > >
> > > It actually brings up the Jobtracker on the machine I ran the command
> > on,
> > > but not the tasktrackers on slave machines. I need to enter slave by
> > slave
> > > starting the tasktracker manually, issuing the command "hadoop
> > tasktracker"
> > > what, according to Hadoop's getting started documentation, is
> > something
> > > unnecessary.
> > >
> > > Did I misunderstand something?
> > >
> > > Thanks in advance,
> > > Lucas
> > >
> >
> >
> >
> >
>