You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by z3r0c001 <ce...@gmail.com> on 2009/10/15 03:58:02 UTC
help with Hadoop custom Writable types implementation
I'm trying to implement Writable interface. but not sure how to
serialize/write/read data from nested objects in
public class StorageClass implements Writable{
public String xStr;
public String yStr;
public List<SubStorage> sStor
//omitted ctors
@override
public void write(DataOutput out) throws IOException{
out.writeChars(xStr);
out.WriteChars(yStr);
//WHAT SHOULD I DO FOR List<SubStorage>
}
@override
public void readFields(DataInput in) throws IOException{
xStr = in.readLine();
yStr = in.readLine();
//WHAT SHOULD I DO FOR List<SubStorage>
}
}
public class SubStorage{
public String x;
public String y;
}
Re: help with Hadoop custom Writable types implementation
Posted by Jeff Zhang <zj...@gmail.com>.
Write:
first write the length of your list, and then write the item in list one by
one
Read:
read the length of your list, initialized your list and read the item one by
one, and append the item to list
I suggest you use array instead of list
On Thu, Oct 15, 2009 at 12:01 PM, z3r0c001 <ce...@gmail.com> wrote:
> I'm trying to implement Writable interface. but not sure how to
> serialize/write/read data from nested objects in
>
> public class StorageClass implements Writable{
>
> public String xStr;
> public String yStr;
>
> public List<SubStorage> sStor
>
> //omitted ctors
>
>
> @override
> public void write(DataOutput out) throws IOException{
> out.writeChars(xStr);
> out.WriteChars(yStr);
>
> //WHAT SHOULD I DO FOR List<SubStorage>
>
> }
>
> @override
> public void readFields(DataInput in) throws IOException{
> xStr = in.readLine();
> yStr = in.readLine();
>
> //WHAT SHOULD I DO FOR List<SubStorage>
> }
>
> }
>
> public class SubStorage{
> public String x;
> public String y;
> }
>
Re: help with Hadoop custom Writable types implementation
Posted by Amogh Vasekar <am...@yahoo-inc.com>.
Hi,
AFAIK readline is not recommended on DataInput types. Also, look into writableutils to see if something there may be used.
Hope this helps.
Amogh
On 10/15/09 9:31 AM, "z3r0c001" <ce...@gmail.com> wrote:
I'm trying to implement Writable interface. but not sure how to
serialize/write/read data from nested objects in
public class StorageClass implements Writable{
public String xStr;
public String yStr;
public List<SubStorage> sStor
//omitted ctors
@override
public void write(DataOutput out) throws IOException{
out.writeChars(xStr);
out.WriteChars(yStr);
//WHAT SHOULD I DO FOR List<SubStorage>
}
@override
public void readFields(DataInput in) throws IOException{
xStr = in.readLine();
yStr = in.readLine();
//WHAT SHOULD I DO FOR List<SubStorage>
}
}
public class SubStorage{
public String x;
public String y;
}
Fwd: help with Hadoop custom Writable types implementation
Posted by z3r0c001 <ce...@gmail.com>.
I'm trying to implement Writable interface. but not sure how to
serialize/write/read data from nested objects in
public class StorageClass implements Writable{
public String xStr;
public String yStr;
public List<SubStorage> sStor
//omitted ctors
@override
public void write(DataOutput out) throws IOException{
out.writeChars(xStr);
out.WriteChars(yStr);
//WHAT SHOULD I DO FOR List<SubStorage>
}
@override
public void readFields(DataInput in) throws IOException{
xStr = in.readLine();
yStr = in.readLine();
//WHAT SHOULD I DO FOR List<SubStorage>
}
}
public class SubStorage{
public String x;
public String y;
}
Re: help with Hadoop custom Writable types implementation
Posted by Jeff Zhang <zj...@gmail.com>.
agree, you'd better make all the fields implement Writable.
and then call their read write in your read write function.
On Thu, Oct 15, 2009 at 3:53 PM, Eason.Lee <le...@gmail.com> wrote:
> I think SubStorage should implements Writable as well~~
>
> Or
>
> You can try ObjectWritable
>
> 2009/10/15 z3r0c001 <ce...@gmail.com>
>
> > I'm trying to implement Writable interface. but not sure how to
> > serialize/write/read data from nested objects in
> >
> > public class StorageClass implements Writable{
> >
> > public String xStr;
> > public String yStr;
> >
> > public List<SubStorage> sStor
> >
> > //omitted ctors
> >
> >
> > @override
> > public void write(DataOutput out) throws IOException{
> > out.writeChars(xStr);
> > out.WriteChars(yStr);
> >
> > //WHAT SHOULD I DO FOR List<SubStorage>
> >
> > }
> >
> > @override
> > public void readFields(DataInput in) throws IOException{
> > xStr = in.readLine();
> > yStr = in.readLine();
> >
> > //WHAT SHOULD I DO FOR List<SubStorage>
> > }
> >
> > }
> >
> > public class SubStorage{
> > public String x;
> > public String y;
> > }
> >
>
Re: help with Hadoop custom Writable types implementation
Posted by "Eason.Lee" <le...@gmail.com>.
I think SubStorage should implements Writable as well~~
Or
You can try ObjectWritable
2009/10/15 z3r0c001 <ce...@gmail.com>
> I'm trying to implement Writable interface. but not sure how to
> serialize/write/read data from nested objects in
>
> public class StorageClass implements Writable{
>
> public String xStr;
> public String yStr;
>
> public List<SubStorage> sStor
>
> //omitted ctors
>
>
> @override
> public void write(DataOutput out) throws IOException{
> out.writeChars(xStr);
> out.WriteChars(yStr);
>
> //WHAT SHOULD I DO FOR List<SubStorage>
>
> }
>
> @override
> public void readFields(DataInput in) throws IOException{
> xStr = in.readLine();
> yStr = in.readLine();
>
> //WHAT SHOULD I DO FOR List<SubStorage>
> }
>
> }
>
> public class SubStorage{
> public String x;
> public String y;
> }
>