You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@crunch.apache.org by "Micah Whitacre (JIRA)" <ji...@apache.org> on 2014/04/14 01:19:15 UTC
[jira] [Assigned] (CRUNCH-272) Unable to correlate crunch jobs
within Oozie
[ https://issues.apache.org/jira/browse/CRUNCH-272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Micah Whitacre reassigned CRUNCH-272:
-------------------------------------
Assignee: Micah Whitacre
> Unable to correlate crunch jobs within Oozie
> --------------------------------------------
>
> Key: CRUNCH-272
> URL: https://issues.apache.org/jira/browse/CRUNCH-272
> Project: Crunch
> Issue Type: Improvement
> Reporter: Mike Zimmerman
> Assignee: Micah Whitacre
>
> I'm not really sure if this should be logged to Oozie or to Crunch, so please feel free to move as needed.
> I would like to request a way to decorate map/reduce jobs that are spawned by a Crunch pipeline so that I can programmatically determine their origin. The primary use case for this is integration with Oozie. Oozie launches a single map job to run a java action (in our case this java action runs a crunch job). Traceability from this original "launcher" job to the jobs created by the crunch job is impossible without trolling logs. This leaves a big black hole for the system operator to assess the performance/impact of these jobs. My initial thought was to provide a simple way to indicate a correlationId or similar on a map/reduce job and then make it accessible within Oozie to query for. Obviously, that request would have to come after the correlation feature was available within map/reduce.
--
This message was sent by Atlassian JIRA
(v6.2#6252)