output custom counter value

["Dandekar, Sheel" <Sh...@intuit.com>]

Hi Oozie users,

I'm triggering a streaming MapReduce action through Oozie and creating a custom counter to count the number of ignored records. At the end of my job, I want to extract the final value of that counter and log it in my application. I'm aware that Oozie allows me to perform subsequent actions based on the value of this counter. However, I simply wish to capture and log the value of this custom counter. I've so far not found a way to do that. My mapper, reducer and workflow.xml can be found at

https://gist.github.com/anonymous/b215daf99a3bd6b03b70

UPDATE: I've been told that when I launch an oozie job in this way, the wordcount action is a secondary job which is launched by the original Oozie job id that is returned.  I have no idea how to get this secondary job id - however, being able to access this may bring me one step closer to getting the counter value.

Thanks,
Sheel

Re: output custom counter value

["Dandekar, Sheel" <Sh...@intuit.com>]

Problem has been solved.  Was using an old jar
(oozie-sharelib-streaming-4.0.0-cdh5.1.3.jar).  When I replaced it with
CDH version 5.4.5 the counter is now available through external stats.

On 11/17/15, 2:45 PM, "Dandekar, Sheel" <Sh...@intuit.com> wrote:

>Hi Oozie users,
>
>I'm triggering a streaming MapReduce action through Oozie and creating a
>custom counter to count the number of ignored records. At the end of my
>job, I want to extract the final value of that counter and log it in my
>application. I'm aware that Oozie allows me to perform subsequent actions
>based on the value of this counter. However, I simply wish to capture and
>log the value of this custom counter. I've so far not found a way to do
>that. My mapper, reducer and workflow.xml can be found at
>
>https://gist.github.com/anonymous/b215daf99a3bd6b03b70
>
>UPDATE: I've been told that when I launch an oozie job in this way, the
>wordcount action is a secondary job which is launched by the original
>Oozie job id that is returned.  I have no idea how to get this secondary
>job id - however, being able to access this may bring me one step closer
>to getting the counter value.
>
>Thanks,
>Sheel
>
>