You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by akshaya iyengar <ak...@gmail.com> on 2009/10/06 07:54:59 UTC
Having multiple values in Value field
I am having issues having multiple values in my value field.My desired
result is
<key>,<float,int> or even <key>,<float,float>.
It seems easy in Python where I can pass a tuple as value.What is the best
way to do this in Java.
I have tried ArrayWritable but it looks like I need to write my own class
like IntArrayWritable.
Thanks,
Akshaya
Re: Having multiple values in Value field
Posted by akshaya iyengar <ak...@gmail.com>.
Thanks Tom. The link really was helpful. the CSVs were getting nasty to
handle.
On Tue, Oct 6, 2009 at 12:27 PM, Tom Chen <to...@gogii.net> wrote:
> Hi Akshaya,
>
> Take a look at the yahoo hadoop tutorial for custom data types.
>
> http://developer.yahoo.com/hadoop/tutorial/module5.html#types
>
> It's actually quite easy to create your own types and stream them. You can
> use the
>
> void readFields(DataInput in);
> void write(DataOutput out);
>
> methods to initialize your objects.
>
> Tom
>
>
> On Tue, Oct 6, 2009 at 12:52 AM, Amogh Vasekar <am...@yahoo-inc.com>
> wrote:
> >>> You can always pass them as comma delimited strings
> > Which would be pretty expensive per <k,v> right? Would avro be looking
> into solving such problems?
> >
> > Amogh
> >
> > -----Original Message-----
> > From: Jason Venner [mailto:jason.hadoop@gmail.com]
> > Sent: Tuesday, October 06, 2009 11:33 AM
> > To: common-user@hadoop.apache.org
> > Subject: Re: Having multiple values in Value field
> >
> > You can always pass them as comma delimited strings, which is what you
> are
> > already doing with your python streaming code, and then use Text as your
> > value.
> >
> > On Mon, Oct 5, 2009 at 10:54 PM, akshaya iyengar
> > <ak...@gmail.com>wrote:
> >
> >> I am having issues having multiple values in my value field.My desired
> >> result is
> >> <key>,<float,int> or even <key>,<float,float>.
> >>
> >> It seems easy in Python where I can pass a tuple as value.What is the
> best
> >> way to do this in Java.
> >> I have tried ArrayWritable but it looks like I need to write my own
> class
> >> like IntArrayWritable.
> >>
> >> Thanks,
> >> Akshaya
> >>
> >
> >
> >
> > --
> > Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> > http://www.amazon.com/dp/1430219424?tag=jewlerymall
> > www.prohadoopbook.com a community for Hadoop Professionals
> >
>
Re: Having multiple values in Value field
Posted by Tom Chen <to...@gogii.net>.
Hi Akshaya,
Take a look at the yahoo hadoop tutorial for custom data types.
http://developer.yahoo.com/hadoop/tutorial/module5.html#types
It's actually quite easy to create your own types and stream them. You can
use the
void readFields(DataInput in);
void write(DataOutput out);
methods to initialize your objects.
Tom
On Tue, Oct 6, 2009 at 12:52 AM, Amogh Vasekar <am...@yahoo-inc.com> wrote:
>>> You can always pass them as comma delimited strings
> Which would be pretty expensive per <k,v> right? Would avro be looking
into solving such problems?
>
> Amogh
>
> -----Original Message-----
> From: Jason Venner [mailto:jason.hadoop@gmail.com]
> Sent: Tuesday, October 06, 2009 11:33 AM
> To: common-user@hadoop.apache.org
> Subject: Re: Having multiple values in Value field
>
> You can always pass them as comma delimited strings, which is what you are
> already doing with your python streaming code, and then use Text as your
> value.
>
> On Mon, Oct 5, 2009 at 10:54 PM, akshaya iyengar
> <ak...@gmail.com>wrote:
>
>> I am having issues having multiple values in my value field.My desired
>> result is
>> <key>,<float,int> or even <key>,<float,float>.
>>
>> It seems easy in Python where I can pass a tuple as value.What is the
best
>> way to do this in Java.
>> I have tried ArrayWritable but it looks like I need to write my own class
>> like IntArrayWritable.
>>
>> Thanks,
>> Akshaya
>>
>
>
>
> --
> Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> http://www.amazon.com/dp/1430219424?tag=jewlerymall
> www.prohadoopbook.com a community for Hadoop Professionals
>
RE: Having multiple values in Value field
Posted by Amogh Vasekar <am...@yahoo-inc.com>.
>> You can always pass them as comma delimited strings
Which would be pretty expensive per <k,v> right? Would avro be looking into solving such problems?
Amogh
-----Original Message-----
From: Jason Venner [mailto:jason.hadoop@gmail.com]
Sent: Tuesday, October 06, 2009 11:33 AM
To: common-user@hadoop.apache.org
Subject: Re: Having multiple values in Value field
You can always pass them as comma delimited strings, which is what you are
already doing with your python streaming code, and then use Text as your
value.
On Mon, Oct 5, 2009 at 10:54 PM, akshaya iyengar
<ak...@gmail.com>wrote:
> I am having issues having multiple values in my value field.My desired
> result is
> <key>,<float,int> or even <key>,<float,float>.
>
> It seems easy in Python where I can pass a tuple as value.What is the best
> way to do this in Java.
> I have tried ArrayWritable but it looks like I need to write my own class
> like IntArrayWritable.
>
> Thanks,
> Akshaya
>
--
Pro Hadoop, a book to guide you from beginner to hadoop mastery,
http://www.amazon.com/dp/1430219424?tag=jewlerymall
www.prohadoopbook.com a community for Hadoop Professionals
Re: Having multiple values in Value field
Posted by Jason Venner <ja...@gmail.com>.
You can always pass them as comma delimited strings, which is what you are
already doing with your python streaming code, and then use Text as your
value.
On Mon, Oct 5, 2009 at 10:54 PM, akshaya iyengar
<ak...@gmail.com>wrote:
> I am having issues having multiple values in my value field.My desired
> result is
> <key>,<float,int> or even <key>,<float,float>.
>
> It seems easy in Python where I can pass a tuple as value.What is the best
> way to do this in Java.
> I have tried ArrayWritable but it looks like I need to write my own class
> like IntArrayWritable.
>
> Thanks,
> Akshaya
>
--
Pro Hadoop, a book to guide you from beginner to hadoop mastery,
http://www.amazon.com/dp/1430219424?tag=jewlerymall
www.prohadoopbook.com a community for Hadoop Professionals