You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@oozie.apache.org by "Alejandro Abdelnur (JIRA)" <ji...@apache.org> on 2011/08/29 02:39:38 UTC
[jira] [Commented] (OOZIE-30) Oozie to drill down to the hadoop job
id created by Pig
[ https://issues.apache.org/jira/browse/OOZIE-30?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13092584#comment-13092584 ]
Alejandro Abdelnur commented on OOZIE-30:
-----------------------------------------
Oozie Pig ActionExecutor implementation should be refactored to obtain the Hadoop Job IDs via Pig's new API.
Oozie should return those job IDs to the client via the HTTP REST and Oozie client API.
After that, the user can use those job IDs to query Hadoop directly.
> Oozie to drill down to the hadoop job id created by Pig
> --------------------------------------------------------
>
> Key: OOZIE-30
> URL: https://issues.apache.org/jira/browse/OOZIE-30
> Project: Apache Oozie (Incubating)
> Issue Type: Improvement
> Reporter: Mohammad Kamrul Islam
>
> At present, Oozie supports to drill down till to MR job and the Launcher Mapper job (for pig/Java action). However, Oozie doesn't have a drill down option of going to the hadoop jobs created by pig scripts.
> Users want to see/access to those job ids through WS API/CLI/UI.
> In existing release, Oozie collect those ids in a hacky way (after parsing the pig log file). In the latest Pig, pig allows to expose the stats that contains the pig-created hadoop jobs ids (most possibly). We could utilize this new approach to get the ids. Also , at present, we don't store that in databases that could be sent as part of query. In future release. we could add an entry to store these ids in DB table. In addition, we should allow to return those as part of job query and shows those in the UI as well.
> The Question is:
> Should we expose this as job ids only? Or we can expose those as additional external URLs that could be used through UI to jump to the corresponding JT page.
> Comments?
>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira