ObjectWritable

[Weishung Chung <we...@gmail.com>]

What is the main use of org.apache.hadoop.io.ObjectWritable ? Thank you :)

Re: multiple hadoop instances on same cluster

[Konstantin Boudnik <co...@apache.org>]

Make sure the instances' ports aren't conflicting and all directories
(NN, JT, etc.) are unique. That should do it.
--
  Take care,
Konstantin (Cos) Boudnik

On Mon, Feb 21, 2011 at 20:09, Gang Luo <lg...@yahoo.com.cn> wrote:
> Hello folks,
> I am trying to run multiple hadoop instances on the same cluster. I find it hard
> to share. First I try two  instances, each of them run with the same master and
> slaves. Only one of them could work. I try to divide the cluster such that
> hadoop 1 use machine 0-9 and hadoop 2 uses machine 10-19. Still, only one of
> them could work. The HDFS of the second hadoop is working well, but
> start-mapred.sh will result in such exception "java.io.IOException: Connection
> reset by peer" in the log.
>
>
> Any ideas on this or suggestion on how to run multiple hadoop instance on one
> cluster? I can total divide up the cluster such that different instances run on
> different set of machines.
>
> Thanks.
>
> -Gang
>
>
>
>
>

multiple hadoop instances on same cluster

[Gang Luo <lg...@yahoo.com.cn>]

Hello folks,
I am trying to run multiple hadoop instances on the same cluster. I find it hard 
to share. First I try two  instances, each of them run with the same master and 
slaves. Only one of them could work. I try to divide the cluster such that 
hadoop 1 use machine 0-9 and hadoop 2 uses machine 10-19. Still, only one of 
them could work. The HDFS of the second hadoop is working well, but 
start-mapred.sh will result in such exception "java.io.IOException: Connection 
reset by peer" in the log. 


Any ideas on this or suggestion on how to run multiple hadoop instance on one 
cluster? I can total divide up the cluster such that different instances run on 
different set of machines.

Thanks.

-Gang



      

Re: ObjectWritable

[Weishung Chung <we...@gmail.com>]

Thank you for the explanation. Avro is a good serialization tool. I haven't
looked at the codes yet. I will probably dig into the codes very soon.

On Mon, Feb 21, 2011 at 10:20 AM, Harsh J <qw...@gmail.com> wrote:

> Hello,
>
> On Mon, Feb 21, 2011 at 9:33 PM, Weishung Chung <we...@gmail.com>
> wrote:
> > What is the main use of org.apache.hadoop.io.ObjectWritable ? Thank you
> :)
>
> To use any primitive Java object as a Writable without requiring it to
> be implementing that interface. It will write out a class name for
> every type of object you put into it along with the object itself,
> when serializing - to deserialize properly.
>
> Maybe not so offtopic: The more I see Writables being used, the more I
> feel like promoting the use of Apache's Avro instead.
>
> --
> Harsh J
> www.harshj.com
>

Re: ObjectWritable

[Harsh J <qw...@gmail.com>]

Hello,

On Mon, Feb 21, 2011 at 9:33 PM, Weishung Chung <we...@gmail.com> wrote:
> What is the main use of org.apache.hadoop.io.ObjectWritable ? Thank you :)

To use any primitive Java object as a Writable without requiring it to
be implementing that interface. It will write out a class name for
every type of object you put into it along with the object itself,
when serializing - to deserialize properly.

Maybe not so offtopic: The more I see Writables being used, the more I
feel like promoting the use of Apache's Avro instead.

-- 
Harsh J
www.harshj.com