You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Eyal Allweil <ey...@yahoo.com.INVALID> on 2015/07/27 17:50:13 UTC
Hive UDF's vs. "native" Pig UDF's
Is there any reason not use Hive UDF's in Pig 0.15?
I'm thinking mostly about performance, but if there are any other reasons I'd be happy to hear them.
For example, we have a simple Java implementation of lpad that we use. Should we bother keeping it, or can we use the Hive version?
I posted a question about this on StackOverflow, but it didn't seem to get anyone's attention.
Thanks,Eyal
Using Hive UDF's in Pig
| |
| | | | | | | |
| Using Hive UDF's in PigIs there any reason not use Hive UDF's in Pig 0.15? I'm thinking mostly about performance, but if there are any other reasons I'd be happy to hear them. For example... |
| |
| View on stackoverflow.com | Preview by Yahoo |
| |
| |
Re: Hive UDF's vs. "native" Pig UDF's
Posted by Daniel Dai <da...@hortonworks.com>.
There are some moderate overhead for Hive UDF. My test shows around
10%-20% slow down than Pig native UDF. I will create a document Jira.
Thanks,
Daniel
On 9/9/15, 9:27 AM, "Rohini Palaniswamy" <ro...@gmail.com> wrote:
>Daniel,
> Not sure you saw this. We will have to document the performance
>implications of hive udfs. Does the wrapping/unwrapping cause significant
>overhead to impact performance or is it negligible?
>
>Regards,
>Rohini
>
>On Mon, Jul 27, 2015 at 8:50 AM, Eyal Allweil <
>eyal_allweil@yahoo.com.invalid> wrote:
>
>> Is there any reason not use Hive UDF's in Pig 0.15?
>> I'm thinking mostly about performance, but if there are any other
>>reasons
>> I'd be happy to hear them.
>> For example, we have a simple Java implementation of lpad that we use.
>> Should we bother keeping it, or can we use the Hive version?
>> I posted a question about this on StackOverflow, but it didn't seem to
>>get
>> anyone's attention.
>> Thanks,Eyal
>> Using Hive UDF's in Pig
>>
>> | |
>> | | | | | | | |
>> | Using Hive UDF's in PigIs there any reason not use Hive UDF's in Pig
>> 0.15? I'm thinking mostly about performance, but if there are any other
>> reasons I'd be happy to hear them. For example... |
>> | |
>> | View on stackoverflow.com | Preview by Yahoo |
>> | |
>> | |
>>
>>
>>
Re: Hive UDF's vs. "native" Pig UDF's
Posted by Daniel Dai <da...@hortonworks.com>.
There are some moderate overhead for Hive UDF. My test shows around
10%-20% slow down than Pig native UDF. I will create a document Jira.
Thanks,
Daniel
On 9/9/15, 9:27 AM, "Rohini Palaniswamy" <ro...@gmail.com> wrote:
>Daniel,
> Not sure you saw this. We will have to document the performance
>implications of hive udfs. Does the wrapping/unwrapping cause significant
>overhead to impact performance or is it negligible?
>
>Regards,
>Rohini
>
>On Mon, Jul 27, 2015 at 8:50 AM, Eyal Allweil <
>eyal_allweil@yahoo.com.invalid> wrote:
>
>> Is there any reason not use Hive UDF's in Pig 0.15?
>> I'm thinking mostly about performance, but if there are any other
>>reasons
>> I'd be happy to hear them.
>> For example, we have a simple Java implementation of lpad that we use.
>> Should we bother keeping it, or can we use the Hive version?
>> I posted a question about this on StackOverflow, but it didn't seem to
>>get
>> anyone's attention.
>> Thanks,Eyal
>> Using Hive UDF's in Pig
>>
>> | |
>> | | | | | | | |
>> | Using Hive UDF's in PigIs there any reason not use Hive UDF's in Pig
>> 0.15? I'm thinking mostly about performance, but if there are any other
>> reasons I'd be happy to hear them. For example... |
>> | |
>> | View on stackoverflow.com | Preview by Yahoo |
>> | |
>> | |
>>
>>
>>
Re: Hive UDF's vs. "native" Pig UDF's
Posted by Rohini Palaniswamy <ro...@gmail.com>.
Daniel,
Not sure you saw this. We will have to document the performance
implications of hive udfs. Does the wrapping/unwrapping cause significant
overhead to impact performance or is it negligible?
Regards,
Rohini
On Mon, Jul 27, 2015 at 8:50 AM, Eyal Allweil <
eyal_allweil@yahoo.com.invalid> wrote:
> Is there any reason not use Hive UDF's in Pig 0.15?
> I'm thinking mostly about performance, but if there are any other reasons
> I'd be happy to hear them.
> For example, we have a simple Java implementation of lpad that we use.
> Should we bother keeping it, or can we use the Hive version?
> I posted a question about this on StackOverflow, but it didn't seem to get
> anyone's attention.
> Thanks,Eyal
> Using Hive UDF's in Pig
>
> | |
> | | | | | | | |
> | Using Hive UDF's in PigIs there any reason not use Hive UDF's in Pig
> 0.15? I'm thinking mostly about performance, but if there are any other
> reasons I'd be happy to hear them. For example... |
> | |
> | View on stackoverflow.com | Preview by Yahoo |
> | |
> | |
>
>
>
Re: Hive UDF's vs. "native" Pig UDF's
Posted by Rohini Palaniswamy <ro...@gmail.com>.
Daniel,
Not sure you saw this. We will have to document the performance
implications of hive udfs. Does the wrapping/unwrapping cause significant
overhead to impact performance or is it negligible?
Regards,
Rohini
On Mon, Jul 27, 2015 at 8:50 AM, Eyal Allweil <
eyal_allweil@yahoo.com.invalid> wrote:
> Is there any reason not use Hive UDF's in Pig 0.15?
> I'm thinking mostly about performance, but if there are any other reasons
> I'd be happy to hear them.
> For example, we have a simple Java implementation of lpad that we use.
> Should we bother keeping it, or can we use the Hive version?
> I posted a question about this on StackOverflow, but it didn't seem to get
> anyone's attention.
> Thanks,Eyal
> Using Hive UDF's in Pig
>
> | |
> | | | | | | | |
> | Using Hive UDF's in PigIs there any reason not use Hive UDF's in Pig
> 0.15? I'm thinking mostly about performance, but if there are any other
> reasons I'd be happy to hear them. For example... |
> | |
> | View on stackoverflow.com | Preview by Yahoo |
> | |
> | |
>
>
>