You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Dexin Wang <wa...@gmail.com> on 2010/12/16 02:17:17 UTC
increment counters in Pig UDF
Is it possible to increment a counter in Pig UDF (in either Load/Eval/Store
Func).
Since we have access to counters using the
org.apache.hadoop.mapred.Reporter:
http://hadoop.apache.org/common/docs/r0.20.2/mapred_tutorial.html#Counters
the other way to ask this question is how do we get an instance of Reporter
in UDF? Thanks.
Dexin
Re: increment counters in Pig UDF
Posted by Dexin Wang <wa...@gmail.com>.
Perfect. Thanks.
On Dec 16, 2010, at 7:41 AM, Dmitriy Ryaboy <dv...@gmail.com> wrote:
> We can do better in 0.8:
>
>
> PigStatusReporter reporter = PigStatusReporter.getInstance();
>
> if (reporter != null) {
>
> reporter.getCounter(myCounterEnum).increment(123L);
>
> }
>
>
> getCounter works with both enums and Strings.
>
> -D
>
> On Wed, Dec 15, 2010 at 5:29 PM, Daniel Dai <ji...@yahoo-inc.com> wrote:
>
>> Yes, you can use EvalFunc.warn(Object o, String msg, Enum warningEnum).
>>
>> Daniel
>>
>>
>> Dexin Wang wrote:
>>
>>> Is it possible to increment a counter in Pig UDF (in either
>>> Load/Eval/Store
>>> Func).
>>>
>>> Since we have access to counters using the
>>> org.apache.hadoop.mapred.Reporter:
>>>
>>> http://hadoop.apache.org/common/docs/r0.20.2/mapred_tutorial.html#Counters
>>>
>>> the other way to ask this question is how do we get an instance of
>>> Reporter
>>> in UDF? Thanks.
>>>
>>> Dexin
>>>
>>>
>>
>>
Re: increment counters in Pig UDF
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
We can do better in 0.8:
PigStatusReporter reporter = PigStatusReporter.getInstance();
if (reporter != null) {
reporter.getCounter(myCounterEnum).increment(123L);
}
getCounter works with both enums and Strings.
-D
On Wed, Dec 15, 2010 at 5:29 PM, Daniel Dai <ji...@yahoo-inc.com> wrote:
> Yes, you can use EvalFunc.warn(Object o, String msg, Enum warningEnum).
>
> Daniel
>
>
> Dexin Wang wrote:
>
>> Is it possible to increment a counter in Pig UDF (in either
>> Load/Eval/Store
>> Func).
>>
>> Since we have access to counters using the
>> org.apache.hadoop.mapred.Reporter:
>>
>> http://hadoop.apache.org/common/docs/r0.20.2/mapred_tutorial.html#Counters
>>
>> the other way to ask this question is how do we get an instance of
>> Reporter
>> in UDF? Thanks.
>>
>> Dexin
>>
>>
>
>
Re: increment counters in Pig UDF
Posted by Daniel Dai <ji...@yahoo-inc.com>.
Yes, you can use EvalFunc.warn(Object o, String msg, Enum warningEnum).
Daniel
Dexin Wang wrote:
> Is it possible to increment a counter in Pig UDF (in either Load/Eval/Store
> Func).
>
> Since we have access to counters using the
> org.apache.hadoop.mapred.Reporter:
>
> http://hadoop.apache.org/common/docs/r0.20.2/mapred_tutorial.html#Counters
>
> the other way to ask this question is how do we get an instance of Reporter
> in UDF? Thanks.
>
> Dexin
>