You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@oozie.apache.org by Siva Subramanian <si...@box.com> on 2012/12/18 21:21:25 UTC

Accessing MR Job Counters via Oozie Client

Hi There,

I need to get event count from all of the map tasks for a map reduce action
triggered through oozie. Is there a way to access the counters via Oozie
client API? If not, what would be the alternative? I dont want to
explicitly write the counters to a hdfs file & read them back.

We are using Oozie 2.3.2.

Thanks,
Siva
-- 
*Siva Subramanian*

Re: Accessing MR Job Counters via Oozie Client

Posted by Mona Chitnis <ch...@yahoo-inc.com>.

Hi Siva,

This feature is available Oozie 3.2 onwards. The latest released version
is 3.3 available for download at
http://www.apache.org/dyn/closer.cgi/oozie/3.3.0/

Relevant documentation
http://oozie.apache.org/docs/3.2.0-incubating/WorkflowFunctionalSpec.html#a
4.2.5_Hadoop_EL_Functions

--
Mona

On 12/18/12 1:15 PM, "Siva Subramanian" <si...@box.com> wrote:

>Thanks, Alejandro.
>
>What version of oozie has this feature?
>
>On Tue, Dec 18, 2012 at 1:00 PM, Alejandro Abdelnur
><tu...@cloudera.com>wrote:
>
>> Hi Siva,
>>
>> With Oozie 2.3.2 this is not possible.
>>
>> You could get the Hadoop job ids (external ID) from the client API and
>>then
>> for each job Id contact Hadoop and get the counters for the job.
>>
>> thx
>>
>>
>> On Tue, Dec 18, 2012 at 8:21 PM, Siva Subramanian <si...@box.com> wrote:
>>
>> > Hi There,
>> >
>> > I need to get event count from all of the map tasks for a map reduce
>> action
>> > triggered through oozie. Is there a way to access the counters via
>>Oozie
>> > client API? If not, what would be the alternative? I dont want to
>> > explicitly write the counters to a hdfs file & read them back.
>> >
>> > We are using Oozie 2.3.2.
>> >
>> > Thanks,
>> > Siva
>> > --
>> > *Siva Subramanian*
>> >
>>
>>
>>
>> --
>> Alejandro
>>

Re: Accessing MR Job Counters via Oozie Client

Posted by Siva Subramanian <si...@box.com>.

Thanks, Alejandro.

What version of oozie has this feature?

On Tue, Dec 18, 2012 at 1:00 PM, Alejandro Abdelnur <tu...@cloudera.com>wrote:

> Hi Siva,
>
> With Oozie 2.3.2 this is not possible.
>
> You could get the Hadoop job ids (external ID) from the client API and then
> for each job Id contact Hadoop and get the counters for the job.
>
> thx
>
>
> On Tue, Dec 18, 2012 at 8:21 PM, Siva Subramanian <si...@box.com> wrote:
>
> > Hi There,
> >
> > I need to get event count from all of the map tasks for a map reduce
> action
> > triggered through oozie. Is there a way to access the counters via Oozie
> > client API? If not, what would be the alternative? I dont want to
> > explicitly write the counters to a hdfs file & read them back.
> >
> > We are using Oozie 2.3.2.
> >
> > Thanks,
> > Siva
> > --
> > *Siva Subramanian*
> >
>
>
>
> --
> Alejandro
>

Re: Accessing MR Job Counters via Oozie Client

Posted by Alejandro Abdelnur <tu...@cloudera.com>.

Hi Siva,

With Oozie 2.3.2 this is not possible.

You could get the Hadoop job ids (external ID) from the client API and then
for each job Id contact Hadoop and get the counters for the job.

thx

On Tue, Dec 18, 2012 at 8:21 PM, Siva Subramanian <si...@box.com> wrote:

> Hi There,
>
> I need to get event count from all of the map tasks for a map reduce action
> triggered through oozie. Is there a way to access the counters via Oozie
> client API? If not, what would be the alternative? I dont want to
> explicitly write the counters to a hdfs file & read them back.
>
> We are using Oozie 2.3.2.
>
> Thanks,
> Siva
> --
> *Siva Subramanian*
>

-- 
Alejandro