You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by Purshotam Shah <pu...@yahoo-inc.com> on 2013/12/30 19:22:39 UTC
Review Request 16489: [OOZIE-1658]Add bundle, coord,
wf and action related information to launched M/R jobs.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16489/
-----------------------------------------------------------
Review request for oozie.
Bugs: OOZIE-1658
https://issues.apache.org/jira/browse/OOZIE-1658
Repository: oozie
Description
-------
We should add workflow related information to launched job. This information can be used for analytics like how many oozie jobs are submitted for a particular period, what is the total number of failed pig jobs.. etc.
JobInfo will contain information of bundle, coordinator, workflow and actions. If enabled, Hadoop job will have
property(oozie.job.info) which value is multiple key/value pair separated by ";".
User can also add custom workflow property to jobinfo by adding property which prefix with "oozie.job.info."
Eg.
oozie.job.info="bundle.id=;bundle.name=;coord.name=;coord.nominal.time=;coord.name=;wf.id=;
wf.name=;action.name=;action.type=;launcher=true"
Jobinfo can enabled/disabled using "oozie.action.jobinfo.enable"
Diffs
-----
http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1551329
http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java PRE-CREATION
http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/command/bundle/BundleStartXCommand.java 1551329
http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionStartXCommand.java 1551329
http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/resources/oozie-default.xml 1551329
http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java PRE-CREATION
Diff: https://reviews.apache.org/r/16489/diff/
Testing
-------
Thanks,
Purshotam Shah
Re: Review Request 16489: [OOZIE-1658]Add bundle, coord,
wf and action related information to launched M/R jobs.
Posted by Rohini Palaniswamy <ro...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16489/#review31150
-----------------------------------------------------------
Just few minor comments
http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java
<https://reviews.apache.org/r/16489/#comment59520>
Use , as separator. Configuration class in hadoop has utility methods which support , as delimiter by default and most of the hadoop configurations follow that convention.
http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/resources/oozie-default.xml
<https://reviews.apache.org/r/16489/#comment59517>
, etc from mapreduce job history logs and configuration.
http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java
<https://reviews.apache.org/r/16489/#comment59518>
Redundant. can be removed
- Rohini Palaniswamy
On Dec. 30, 2013, 6:22 p.m., Purshotam Shah wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16489/
> -----------------------------------------------------------
>
> (Updated Dec. 30, 2013, 6:22 p.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-1658
> https://issues.apache.org/jira/browse/OOZIE-1658
>
>
> Repository: oozie
>
>
> Description
> -------
>
> We should add workflow related information to launched job. This information can be used for analytics like how many oozie jobs are submitted for a particular period, what is the total number of failed pig jobs.. etc.
>
> JobInfo will contain information of bundle, coordinator, workflow and actions. If enabled, Hadoop job will have
> property(oozie.job.info) which value is multiple key/value pair separated by ";".
>
> User can also add custom workflow property to jobinfo by adding property which prefix with "oozie.job.info."
>
> Eg.
> oozie.job.info="bundle.id=;bundle.name=;coord.name=;coord.nominal.time=;coord.name=;wf.id=;
> wf.name=;action.name=;action.type=;launcher=true"
>
>
> Jobinfo can enabled/disabled using "oozie.action.jobinfo.enable"
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1551329
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java PRE-CREATION
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/command/bundle/BundleStartXCommand.java 1551329
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionStartXCommand.java 1551329
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/resources/oozie-default.xml 1551329
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/16489/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Purshotam Shah
>
>
Re: Review Request 16489: [OOZIE-1658]Add bundle, coord,
wf and action related information to launched M/R jobs.
Posted by Rohini Palaniswamy <ro...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16489/#review31249
-----------------------------------------------------------
Ship it!
Ship It!
- Rohini Palaniswamy
On Jan. 5, 2014, 6:56 p.m., Purshotam Shah wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16489/
> -----------------------------------------------------------
>
> (Updated Jan. 5, 2014, 6:56 p.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-1658
> https://issues.apache.org/jira/browse/OOZIE-1658
>
>
> Repository: oozie
>
>
> Description
> -------
>
> We should add workflow related information to launched job. This information can be used for analytics like how many oozie jobs are submitted for a particular period, what is the total number of failed pig jobs.. etc.
>
> JobInfo will contain information of bundle, coordinator, workflow and actions. If enabled, Hadoop job will have
> property(oozie.job.info) which value is multiple key/value pair separated by ";".
>
> User can also add custom workflow property to jobinfo by adding property which prefix with "oozie.job.info."
>
> Eg.
> oozie.job.info="bundle.id=;bundle.name=;coord.name=;coord.nominal.time=;coord.name=;wf.id=;
> wf.name=;action.name=;action.type=;launcher=true"
>
>
> Jobinfo can enabled/disabled using "oozie.action.jobinfo.enable"
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1551329
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java PRE-CREATION
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/command/bundle/BundleStartXCommand.java 1551329
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionStartXCommand.java 1551329
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/resources/oozie-default.xml 1551329
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/16489/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Purshotam Shah
>
>
Re: Review Request 16489: [OOZIE-1658]Add bundle, coord,
wf and action related information to launched M/R jobs.
Posted by Purshotam Shah <pu...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16489/
-----------------------------------------------------------
(Updated Jan. 5, 2014, 6:56 p.m.)
Review request for oozie.
Changes
-------
review comment.
Bugs: OOZIE-1658
https://issues.apache.org/jira/browse/OOZIE-1658
Repository: oozie
Description
-------
We should add workflow related information to launched job. This information can be used for analytics like how many oozie jobs are submitted for a particular period, what is the total number of failed pig jobs.. etc.
JobInfo will contain information of bundle, coordinator, workflow and actions. If enabled, Hadoop job will have
property(oozie.job.info) which value is multiple key/value pair separated by ";".
User can also add custom workflow property to jobinfo by adding property which prefix with "oozie.job.info."
Eg.
oozie.job.info="bundle.id=;bundle.name=;coord.name=;coord.nominal.time=;coord.name=;wf.id=;
wf.name=;action.name=;action.type=;launcher=true"
Jobinfo can enabled/disabled using "oozie.action.jobinfo.enable"
Diffs (updated)
-----
http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1551329
http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java PRE-CREATION
http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/command/bundle/BundleStartXCommand.java 1551329
http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionStartXCommand.java 1551329
http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/resources/oozie-default.xml 1551329
http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java PRE-CREATION
Diff: https://reviews.apache.org/r/16489/diff/
Testing
-------
Thanks,
Purshotam Shah