You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by ri...@nokia.com on 2012/06/12 22:56:45 UTC
hadoop.io.DoubleWritable v/s hive.serde2.io.DoubleWritable
Hi Guys,
I am writing a UDF in hive to convert a double value to string, so the evaluate method of my UDF class looks like
import org.apache.hadoop.hive.ql.exec.UDF;
import org.apache.hadoop.io.Text;
//import org.apache.hadoop.io.DoubleWritable; - does not work
import org.apache.hadoop.hive.serde2.io.DoubleWritable;
public Text evaluate(DoubleWritable d){
}
When I looked at different UDF examples the Hadoop Writables seem to work fine in case of Text and IntWritable but for DoubleWritable I was getting an error. I figured out looking at couple of examples online that I should use the DoubleWritable from hive.serde2 package instead of hadoop.io package.
Can someone please explain why this special case for DoubleWritable?
Thanks,
Richin
Re: hadoop.io.DoubleWritable v/s hive.serde2.io.DoubleWritable
Posted by Edward Capriolo <ed...@gmail.com>.
Hive likely wishes to format the data differently then Hadoop does.
Hive re-uses what it can. I would diff the two .java files and find
out for yourself :)
On Tue, Jun 12, 2012 at 5:20 PM, <ri...@nokia.com> wrote:
> Hi Edward,
>
> Sorry, If I was not clear. My question is around difference between DoubleWritable in hadoop and hive, other writables from hadoop works fine in hive.
> Hive.serde types are limited to Double, Byte, Short and Timestamp.
>
> I am using hive 0.8
>
> Richin
>
> -----Original Message-----
> From: ext Edward Capriolo [mailto:edlinuxguru@gmail.com]
> Sent: Tuesday, June 12, 2012 5:12 PM
> To: user@hive.apache.org
> Subject: Re: hadoop.io.DoubleWritable v/s hive.serde2.io.DoubleWritable
>
> If you use Double or double hive will automatically convert. I would always recommend the hive.serde types.
>
> Edward
>
> On Tue, Jun 12, 2012 at 4:56 PM, <ri...@nokia.com> wrote:
>> Hi Guys,
>>
>>
>>
>> I am writing a UDF in hive to convert a double value to string, so the
>> evaluate method of my UDF class looks like
>>
>>
>>
>> import org.apache.hadoop.hive.ql.exec.UDF;
>>
>> import org.apache.hadoop.io.Text;
>>
>> //import org.apache.hadoop.io.DoubleWritable; - does not work
>>
>> import org.apache.hadoop.hive.serde2.io.DoubleWritable;
>>
>>
>>
>> public Text evaluate(DoubleWritable d){
>>
>>
>>
>> }
>>
>>
>>
>> When I looked at different UDF examples the Hadoop Writables seem to
>> work fine in case of Text and IntWritable but for DoubleWritable I was
>> getting an error. I figured out looking at couple of examples online
>> that I should use the DoubleWritable from hive.serde2 package instead of hadoop.io package.
>>
>>
>>
>> Can someone please explain why this special case for DoubleWritable?
>>
>>
>>
>> Thanks,
>>
>> Richin
RE: hadoop.io.DoubleWritable v/s hive.serde2.io.DoubleWritable
Posted by ri...@nokia.com.
Hi Edward,
Sorry, If I was not clear. My question is around difference between DoubleWritable in hadoop and hive, other writables from hadoop works fine in hive.
Hive.serde types are limited to Double, Byte, Short and Timestamp.
I am using hive 0.8
Richin
-----Original Message-----
From: ext Edward Capriolo [mailto:edlinuxguru@gmail.com]
Sent: Tuesday, June 12, 2012 5:12 PM
To: user@hive.apache.org
Subject: Re: hadoop.io.DoubleWritable v/s hive.serde2.io.DoubleWritable
If you use Double or double hive will automatically convert. I would always recommend the hive.serde types.
Edward
On Tue, Jun 12, 2012 at 4:56 PM, <ri...@nokia.com> wrote:
> Hi Guys,
>
>
>
> I am writing a UDF in hive to convert a double value to string, so the
> evaluate method of my UDF class looks like
>
>
>
> import org.apache.hadoop.hive.ql.exec.UDF;
>
> import org.apache.hadoop.io.Text;
>
> //import org.apache.hadoop.io.DoubleWritable; - does not work
>
> import org.apache.hadoop.hive.serde2.io.DoubleWritable;
>
>
>
> public Text evaluate(DoubleWritable d){
>
>
>
> }
>
>
>
> When I looked at different UDF examples the Hadoop Writables seem to
> work fine in case of Text and IntWritable but for DoubleWritable I was
> getting an error. I figured out looking at couple of examples online
> that I should use the DoubleWritable from hive.serde2 package instead of hadoop.io package.
>
>
>
> Can someone please explain why this special case for DoubleWritable?
>
>
>
> Thanks,
>
> Richin
Re: hadoop.io.DoubleWritable v/s hive.serde2.io.DoubleWritable
Posted by Edward Capriolo <ed...@gmail.com>.
If you use Double or double hive will automatically convert. I would
always recommend the hive.serde types.
Edward
On Tue, Jun 12, 2012 at 4:56 PM, <ri...@nokia.com> wrote:
> Hi Guys,
>
>
>
> I am writing a UDF in hive to convert a double value to string, so the
> evaluate method of my UDF class looks like
>
>
>
> import org.apache.hadoop.hive.ql.exec.UDF;
>
> import org.apache.hadoop.io.Text;
>
> //import org.apache.hadoop.io.DoubleWritable; - does not work
>
> import org.apache.hadoop.hive.serde2.io.DoubleWritable;
>
>
>
> public Text evaluate(DoubleWritable d){
>
>
>
> }
>
>
>
> When I looked at different UDF examples the Hadoop Writables seem to work
> fine in case of Text and IntWritable but for DoubleWritable I was getting an
> error. I figured out looking at couple of examples online that I should use
> the DoubleWritable from hive.serde2 package instead of hadoop.io package.
>
>
>
> Can someone please explain why this special case for DoubleWritable?
>
>
>
> Thanks,
>
> Richin