You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Ahmed Sobhi <ah...@gmail.com> on 2012/03/30 10:59:19 UTC
Pig 0.9 UDF Counters
As I wanted to increment some counters in some UDFs I wrote, I came across
http://squarecog.wordpress.com/2010/12/24/incrementing-hadoop-counters-in-apache-pig/
as
THE answer
which basically says I should use something like
PigStatusReporter reporter = PigStatusReporter.getInstance();
if (reporter != null) {
reporter.getCounter(key).increment(incr);
}
My issue however lies in that reporter.getCounter(key) always returns null
Also, I'm not sure what to do for the case when the reporter itself is null
Any ideas?
--
Best Regards,
Ahmed Sobhi
http://about.me/humanzz/bio
Re: Pig 0.9 UDF Counters
Posted by Jonathan Coveney <jc...@gmail.com>.
In the JRuby patch, the class ElephantBird uses to make using counters
easier will be added to Pig as well. Until then, I'd use the EB class if
possible.
2012/3/30 Ahmed Sobhi <ah...@gmail.com>
> I was using Enums yeah.
>
> With ElephantBird it does work as it caches the counts till the
> report/counter become available
>
> I just hope that once they become available, they're available, otherwise,
> I might be losing many counts
>
> Thanks
>
> On Fri, Mar 30, 2012 at 11:41 AM, Gianmarco De Francisci Morales <
> gdfm@apache.org> wrote:
>
> > Hi Ahmed,
> >
> > What are you using as key?
> > The method should work with both Strings and Enums.
> > If you are using one, can you try with the other?
> >
> > If it is only a temporary null, you can probably work around it by using
> > ElephantBird as suggested in the blog post you linked.
> >
> > Cheers,
> > --
> > Gianmarco
> >
> >
> >
> > On Fri, Mar 30, 2012 at 10:59, Ahmed Sobhi <ah...@gmail.com>
> wrote:
> >
> > > As I wanted to increment some counters in some UDFs I wrote, I came
> > across
> > >
> > >
> >
> http://squarecog.wordpress.com/2010/12/24/incrementing-hadoop-counters-in-apache-pig/
> > > as
> > > THE answer
> > > which basically says I should use something like
> > >
> > > PigStatusReporter reporter = PigStatusReporter.getInstance();
> > > if (reporter != null) {
> > > reporter.getCounter(key).increment(incr);
> > > }
> > >
> > >
> > > My issue however lies in that reporter.getCounter(key) always returns
> > null
> > >
> > > Also, I'm not sure what to do for the case when the reporter itself is
> > null
> > >
> > > Any ideas?
> > >
> > > --
> > > Best Regards,
> > > Ahmed Sobhi
> > > http://about.me/humanzz/bio
> > >
> >
>
>
>
> --
> Best Regards,
> Ahmed Sobhi
> http://about.me/humanzz/bio
>
Re: Pig 0.9 UDF Counters
Posted by Ahmed Sobhi <ah...@gmail.com>.
I was using Enums yeah.
With ElephantBird it does work as it caches the counts till the
report/counter become available
I just hope that once they become available, they're available, otherwise,
I might be losing many counts
Thanks
On Fri, Mar 30, 2012 at 11:41 AM, Gianmarco De Francisci Morales <
gdfm@apache.org> wrote:
> Hi Ahmed,
>
> What are you using as key?
> The method should work with both Strings and Enums.
> If you are using one, can you try with the other?
>
> If it is only a temporary null, you can probably work around it by using
> ElephantBird as suggested in the blog post you linked.
>
> Cheers,
> --
> Gianmarco
>
>
>
> On Fri, Mar 30, 2012 at 10:59, Ahmed Sobhi <ah...@gmail.com> wrote:
>
> > As I wanted to increment some counters in some UDFs I wrote, I came
> across
> >
> >
> http://squarecog.wordpress.com/2010/12/24/incrementing-hadoop-counters-in-apache-pig/
> > as
> > THE answer
> > which basically says I should use something like
> >
> > PigStatusReporter reporter = PigStatusReporter.getInstance();
> > if (reporter != null) {
> > reporter.getCounter(key).increment(incr);
> > }
> >
> >
> > My issue however lies in that reporter.getCounter(key) always returns
> null
> >
> > Also, I'm not sure what to do for the case when the reporter itself is
> null
> >
> > Any ideas?
> >
> > --
> > Best Regards,
> > Ahmed Sobhi
> > http://about.me/humanzz/bio
> >
>
--
Best Regards,
Ahmed Sobhi
http://about.me/humanzz/bio
Re: Pig 0.9 UDF Counters
Posted by Gianmarco De Francisci Morales <gd...@apache.org>.
Hi Ahmed,
What are you using as key?
The method should work with both Strings and Enums.
If you are using one, can you try with the other?
If it is only a temporary null, you can probably work around it by using
ElephantBird as suggested in the blog post you linked.
Cheers,
--
Gianmarco
On Fri, Mar 30, 2012 at 10:59, Ahmed Sobhi <ah...@gmail.com> wrote:
> As I wanted to increment some counters in some UDFs I wrote, I came across
>
> http://squarecog.wordpress.com/2010/12/24/incrementing-hadoop-counters-in-apache-pig/
> as
> THE answer
> which basically says I should use something like
>
> PigStatusReporter reporter = PigStatusReporter.getInstance();
> if (reporter != null) {
> reporter.getCounter(key).increment(incr);
> }
>
>
> My issue however lies in that reporter.getCounter(key) always returns null
>
> Also, I'm not sure what to do for the case when the reporter itself is null
>
> Any ideas?
>
> --
> Best Regards,
> Ahmed Sobhi
> http://about.me/humanzz/bio
>