multiple hadoop instances on same cluster

[Gang Luo <lg...@yahoo.com.cn>]

Hello folks,
I am trying to run multiple hadoop instances on the same cluster. I find it hard 
to share. First I try two  instances, each of them run with the same master and 
slaves. Only one of them could work. I try to divide the cluster such that 
hadoop 1 use machine 0-9 and hadoop 2 uses machine 10-19. Still, only one of 
them could work. The HDFS of the second hadoop is working well, but 
start-mapred.sh will result in such exception "java.io.IOException: Connection 
reset by peer" in the log. 


Any ideas on this or suggestion on how to run multiple hadoop instance on one 
cluster? I can total divide up the cluster such that different instances run on 
different set of machines.

Thanks.

-Gang



      

Re: multiple hadoop instances on same cluster

[Konstantin Boudnik <co...@apache.org>]

Make sure the instances' ports aren't conflicting and all directories
(NN, JT, etc.) are unique. That should do it.
--
  Take care,
Konstantin (Cos) Boudnik

On Mon, Feb 21, 2011 at 20:09, Gang Luo <lg...@yahoo.com.cn> wrote:
> Hello folks,
> I am trying to run multiple hadoop instances on the same cluster. I find it hard
> to share. First I try two  instances, each of them run with the same master and
> slaves. Only one of them could work. I try to divide the cluster such that
> hadoop 1 use machine 0-9 and hadoop 2 uses machine 10-19. Still, only one of
> them could work. The HDFS of the second hadoop is working well, but
> start-mapred.sh will result in such exception "java.io.IOException: Connection
> reset by peer" in the log.
>
>
> Any ideas on this or suggestion on how to run multiple hadoop instance on one
> cluster? I can total divide up the cluster such that different instances run on
> different set of machines.
>
> Thanks.
>
> -Gang
>
>
>
>
>