You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by pa...@gmail.com on 2011/12/07 08:06:39 UTC
Review Request: OOZIE-588: Oozie to allow drill down to hadoop job's details
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3047/
-----------------------------------------------------------
Review request for oozie, Mohammad Islam and Angelo K. Huang.
Summary
-------
OOZIE-588: Oozie to allow drill down to hadoop job's details
Summary:
https://issues.apache.org/jira/browse/OOZIE-588
Features Implemented:
- Expose action stats information (hadoop counters and pig stats) through CLI -info -verbose option
- Expose external child IDs (eg: hadoop job IDs for the pig action) through CLI -info -verbose option
- Expose job counter info as EL Functions
- Expose external child IDs as EL Functions
- User level configuration to specify if stats should be collected or not (with a default setting)
- Oozie level configuration to specify the max allowable size for stats
This addresses bug OOZIE-588.
https://issues.apache.org/jira/browse/OOZIE-588
Diffs
-----
trunk/core/src/test/java/org/apache/oozie/client/rest/TestJsonWorkflowAction.java 1211308
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 1211308
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 1211308
trunk/core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java 1211308
trunk/core/src/test/java/org/apache/oozie/action/decision/TestDecisionActionExecutor.java 1211308
trunk/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java 1211308
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestHadoopELFunctions.java 1211308
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsRunningGetJPAExecutor.java 1211308
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowJobGetActionsJPAExecutor.java 1211308
trunk/core/src/main/java/org/apache/oozie/store/WorkflowStore.java 1211308
trunk/core/src/test/java/org/apache/oozie/TestActionBean.java 1211308
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionSubsetGetJPAExecutor.java 1211308
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsGetForJobJPAExecutor.java 1211308
trunk/core/src/main/java/org/apache/oozie/client/rest/JsonWorkflowAction.java 1211308
trunk/core/src/main/java/org/apache/oozie/command/wf/ActionCommand.java 1211308
trunk/core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 1211308
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionGetJPAExecutor.java 1211308
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 1211308
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MRStats.java PRE-CREATION
trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionType.java PRE-CREATION
trunk/core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1211308
trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionStats.java PRE-CREATION
trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1211308
trunk/core/src/main/java/org/apache/oozie/DagELFunctions.java 1211308
trunk/client/src/main/java/org/apache/oozie/client/rest/JsonTags.java 1211308
trunk/client/src/main/java/org/apache/oozie/client/rest/JsonToBean.java 1211308
trunk/client/src/main/java/org/apache/oozie/client/WorkflowAction.java 1211308
trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1211308
Diff: https://reviews.apache.org/r/3047/diff
Testing
-------
- Tested against existing test cases and added tests where required.
- Tests for the new classes MRStats and OoziePigStats have not been added. This is work in progress.
Thanks,
params
Re: Review Request: OOZIE-588: Oozie to allow drill down to hadoop job's
details
Posted by pa...@gmail.com.
> On 2011-12-08 20:22:40, Virag Kothari wrote:
> > trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java, line 156
> > <https://reviews.apache.org/r/3047/diff/2/?file=63091#file63091line156>
> >
> > Add a check for max. size of stats before setting the value to context
Added.
> On 2011-12-08 20:22:40, Virag Kothari wrote:
> > trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java, line 197
> > <https://reviews.apache.org/r/3047/diff/2/?file=63091#file63091line197>
> >
> > cant you try inlineConf.get(key) to avoid the for loop
Added. The get() method also helps specify a default value.
Used the get(key, defaultValue) signature.
> On 2011-12-08 20:22:40, Virag Kothari wrote:
> > trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java, line 75
> > <https://reviews.apache.org/r/3047/diff/2/?file=63105#file63105line75>
> >
> > Can you add test case for hadoop counters obtained through MRStats
Added two test cases in TestMapReduceActionExecutor.java to test setExecutionStats(). Also added asserts for setExternalChildIDs().
- params
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3047/#review3751
-----------------------------------------------------------
On 2011-12-07 23:53:52, params wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3047/
> -----------------------------------------------------------
>
> (Updated 2011-12-07 23:53:52)
>
>
> Review request for oozie, Mohammad Islam and Angelo K. Huang.
>
>
> Summary
> -------
>
> OOZIE-588: Oozie to allow drill down to hadoop job's details
>
> Summary:
> https://issues.apache.org/jira/browse/OOZIE-588
>
> Features Implemented:
> - Expose action stats information (hadoop counters and pig stats) through CLI -info -verbose option
> - Expose external child IDs (eg: hadoop job IDs for the pig action) through CLI -info -verbose option
> - Expose job counter info as EL Functions
> - Expose external child IDs as EL Functions
> - User level configuration to specify if stats should be collected or not (with a default setting)
> - Oozie level configuration to specify the max allowable size for stats
>
>
> This addresses bug OOZIE-588.
> https://issues.apache.org/jira/browse/OOZIE-588
>
>
> Diffs
> -----
>
> trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1211308
> trunk/client/src/main/java/org/apache/oozie/client/WorkflowAction.java 1211308
> trunk/client/src/main/java/org/apache/oozie/client/rest/JsonTags.java 1211308
> trunk/client/src/main/java/org/apache/oozie/client/rest/JsonToBean.java 1211308
> trunk/core/src/main/java/org/apache/oozie/DagELFunctions.java 1211308
> trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1211308
> trunk/core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionStats.java PRE-CREATION
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionType.java PRE-CREATION
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/MRStats.java PRE-CREATION
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/client/rest/JsonWorkflowAction.java 1211308
> trunk/core/src/main/java/org/apache/oozie/command/wf/ActionCommand.java 1211308
> trunk/core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionGetJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionSubsetGetJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsGetForJobJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsRunningGetJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowJobGetActionsJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/store/WorkflowStore.java 1211308
> trunk/core/src/test/java/org/apache/oozie/TestActionBean.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/decision/TestDecisionActionExecutor.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestHadoopELFunctions.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java 1211308
> trunk/core/src/test/java/org/apache/oozie/client/rest/TestJsonWorkflowAction.java 1211308
>
> Diff: https://reviews.apache.org/r/3047/diff
>
>
> Testing
> -------
>
> - Tested against existing test cases and added tests where required.
> - Tests for the new classes MRStats and OoziePigStats have not been added. This is work in progress.
>
>
> Thanks,
>
> params
>
>
Re: Review Request: OOZIE-588: Oozie to allow drill down to hadoop job's
details
Posted by Virag Kothari <vi...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3047/#review3751
-----------------------------------------------------------
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java
<https://reviews.apache.org/r/3047/#comment8476>
Add a check for max. size of stats before setting the value to context
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java
<https://reviews.apache.org/r/3047/#comment8477>
cant you try inlineConf.get(key) to avoid the for loop
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java
<https://reviews.apache.org/r/3047/#comment8478>
Can you add test case for hadoop counters obtained through MRStats
- Virag
On 2011-12-07 23:53:52, params wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3047/
> -----------------------------------------------------------
>
> (Updated 2011-12-07 23:53:52)
>
>
> Review request for oozie, Mohammad Islam and Angelo K. Huang.
>
>
> Summary
> -------
>
> OOZIE-588: Oozie to allow drill down to hadoop job's details
>
> Summary:
> https://issues.apache.org/jira/browse/OOZIE-588
>
> Features Implemented:
> - Expose action stats information (hadoop counters and pig stats) through CLI -info -verbose option
> - Expose external child IDs (eg: hadoop job IDs for the pig action) through CLI -info -verbose option
> - Expose job counter info as EL Functions
> - Expose external child IDs as EL Functions
> - User level configuration to specify if stats should be collected or not (with a default setting)
> - Oozie level configuration to specify the max allowable size for stats
>
>
> This addresses bug OOZIE-588.
> https://issues.apache.org/jira/browse/OOZIE-588
>
>
> Diffs
> -----
>
> trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1211308
> trunk/client/src/main/java/org/apache/oozie/client/WorkflowAction.java 1211308
> trunk/client/src/main/java/org/apache/oozie/client/rest/JsonTags.java 1211308
> trunk/client/src/main/java/org/apache/oozie/client/rest/JsonToBean.java 1211308
> trunk/core/src/main/java/org/apache/oozie/DagELFunctions.java 1211308
> trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1211308
> trunk/core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionStats.java PRE-CREATION
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionType.java PRE-CREATION
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/MRStats.java PRE-CREATION
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/client/rest/JsonWorkflowAction.java 1211308
> trunk/core/src/main/java/org/apache/oozie/command/wf/ActionCommand.java 1211308
> trunk/core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionGetJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionSubsetGetJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsGetForJobJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsRunningGetJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowJobGetActionsJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/store/WorkflowStore.java 1211308
> trunk/core/src/test/java/org/apache/oozie/TestActionBean.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/decision/TestDecisionActionExecutor.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestHadoopELFunctions.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java 1211308
> trunk/core/src/test/java/org/apache/oozie/client/rest/TestJsonWorkflowAction.java 1211308
>
> Diff: https://reviews.apache.org/r/3047/diff
>
>
> Testing
> -------
>
> - Tested against existing test cases and added tests where required.
> - Tests for the new classes MRStats and OoziePigStats have not been added. This is work in progress.
>
>
> Thanks,
>
> params
>
>
Re: Review Request: OOZIE-588: Oozie to allow drill down to hadoop job's
details
Posted by Prakhar Sharma <pr...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3047/#review3724
-----------------------------------------------------------
Ship it!
+1
- Prakhar
On 2011-12-07 23:53:52, params wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3047/
> -----------------------------------------------------------
>
> (Updated 2011-12-07 23:53:52)
>
>
> Review request for oozie, Mohammad Islam and Angelo K. Huang.
>
>
> Summary
> -------
>
> OOZIE-588: Oozie to allow drill down to hadoop job's details
>
> Summary:
> https://issues.apache.org/jira/browse/OOZIE-588
>
> Features Implemented:
> - Expose action stats information (hadoop counters and pig stats) through CLI -info -verbose option
> - Expose external child IDs (eg: hadoop job IDs for the pig action) through CLI -info -verbose option
> - Expose job counter info as EL Functions
> - Expose external child IDs as EL Functions
> - User level configuration to specify if stats should be collected or not (with a default setting)
> - Oozie level configuration to specify the max allowable size for stats
>
>
> This addresses bug OOZIE-588.
> https://issues.apache.org/jira/browse/OOZIE-588
>
>
> Diffs
> -----
>
> trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1211308
> trunk/client/src/main/java/org/apache/oozie/client/WorkflowAction.java 1211308
> trunk/client/src/main/java/org/apache/oozie/client/rest/JsonTags.java 1211308
> trunk/client/src/main/java/org/apache/oozie/client/rest/JsonToBean.java 1211308
> trunk/core/src/main/java/org/apache/oozie/DagELFunctions.java 1211308
> trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1211308
> trunk/core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionStats.java PRE-CREATION
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionType.java PRE-CREATION
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/MRStats.java PRE-CREATION
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/client/rest/JsonWorkflowAction.java 1211308
> trunk/core/src/main/java/org/apache/oozie/command/wf/ActionCommand.java 1211308
> trunk/core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionGetJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionSubsetGetJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsGetForJobJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsRunningGetJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowJobGetActionsJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/store/WorkflowStore.java 1211308
> trunk/core/src/test/java/org/apache/oozie/TestActionBean.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/decision/TestDecisionActionExecutor.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestHadoopELFunctions.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java 1211308
> trunk/core/src/test/java/org/apache/oozie/client/rest/TestJsonWorkflowAction.java 1211308
>
> Diff: https://reviews.apache.org/r/3047/diff
>
>
> Testing
> -------
>
> - Tested against existing test cases and added tests where required.
> - Tests for the new classes MRStats and OoziePigStats have not been added. This is work in progress.
>
>
> Thanks,
>
> params
>
>
Re: Review Request: OOZIE-588: Oozie to allow drill down to hadoop job's
details
Posted by "Angelo K. Huang" <an...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3047/#review3805
-----------------------------------------------------------
trunk/core/src/main/java/org/apache/oozie/DagELFunctions.java
<https://reviews.apache.org/r/3047/#comment8620>
MapReduceActionExecutor already has this. Could you merge to one place?
trunk/core/src/main/java/org/apache/oozie/DagELFunctions.java
<https://reviews.apache.org/r/3047/#comment8621>
it can be private
trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionType.java
<https://reviews.apache.org/r/3047/#comment8625>
Why this enum has its own class? Can it be in any existed class?
It can be part of ActionStats.
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java
<https://reviews.apache.org/r/3047/#comment8628>
private XLog log = XLog.getLog(getClass());
This already defined in the beginning of class.
- Angelo K.
On 2011-12-09 22:26:29, params wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3047/
> -----------------------------------------------------------
>
> (Updated 2011-12-09 22:26:29)
>
>
> Review request for oozie, Mohammad Islam and Angelo K. Huang.
>
>
> Summary
> -------
>
> OOZIE-588: Oozie to allow drill down to hadoop job's details
>
> Summary:
> https://issues.apache.org/jira/browse/OOZIE-588
>
> Features Implemented:
> - Expose action stats information (hadoop counters and pig stats) through CLI -info -verbose option
> - Expose external child IDs (eg: hadoop job IDs for the pig action) through CLI -info -verbose option
> - Expose job counter info as EL Functions
> - Expose external child IDs as EL Functions
> - User level configuration to specify if stats should be collected or not (with a default setting)
> - Oozie level configuration to specify the max allowable size for stats
>
>
> This addresses bug OOZIE-588.
> https://issues.apache.org/jira/browse/OOZIE-588
>
>
> Diffs
> -----
>
> trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1212662
> trunk/client/src/main/java/org/apache/oozie/client/WorkflowAction.java 1212662
> trunk/client/src/main/java/org/apache/oozie/client/rest/JsonTags.java 1212662
> trunk/client/src/main/java/org/apache/oozie/client/rest/JsonToBean.java 1212662
> trunk/core/src/main/java/org/apache/oozie/DagELFunctions.java 1212662
> trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1212662
> trunk/core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1212662
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionStats.java PRE-CREATION
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionType.java PRE-CREATION
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/MRStats.java PRE-CREATION
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 1212662
> trunk/core/src/main/java/org/apache/oozie/client/rest/JsonWorkflowAction.java 1212662
> trunk/core/src/main/java/org/apache/oozie/command/wf/ActionCommand.java 1212662
> trunk/core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 1212662
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionGetJPAExecutor.java 1212662
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionSubsetGetJPAExecutor.java 1212662
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsGetForJobJPAExecutor.java 1212662
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsRunningGetJPAExecutor.java 1212662
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowJobGetActionsJPAExecutor.java 1212662
> trunk/core/src/main/java/org/apache/oozie/store/WorkflowStore.java 1212662
> trunk/core/src/test/java/org/apache/oozie/TestActionBean.java 1212662
> trunk/core/src/test/java/org/apache/oozie/action/decision/TestDecisionActionExecutor.java 1212662
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java 1212662
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestHadoopELFunctions.java 1212662
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 1212662
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 1212662
> trunk/core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java 1212662
> trunk/core/src/test/java/org/apache/oozie/client/rest/TestJsonWorkflowAction.java 1212662
>
> Diff: https://reviews.apache.org/r/3047/diff
>
>
> Testing
> -------
>
> - Tested against existing test cases and added tests where required.
> - Tests for the new classes MRStats and OoziePigStats have not been added. This is work in progress.
>
>
> Thanks,
>
> params
>
>
Re: Review Request: OOZIE-588: Oozie to allow drill down to hadoop job's
details
Posted by pa...@gmail.com.
> On 2011-12-10 01:21:40, Angelo K. Huang wrote:
> > ship it after the fix
Fixed. Revised diff uploaded.
- params
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3047/#review3820
-----------------------------------------------------------
On 2011-12-10 01:14:25, params wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3047/
> -----------------------------------------------------------
>
> (Updated 2011-12-10 01:14:25)
>
>
> Review request for oozie, Mohammad Islam and Angelo K. Huang.
>
>
> Summary
> -------
>
> OOZIE-588: Oozie to allow drill down to hadoop job's details
>
> Summary:
> https://issues.apache.org/jira/browse/OOZIE-588
>
> Features Implemented:
> - Expose action stats information (hadoop counters and pig stats) through CLI -info -verbose option
> - Expose external child IDs (eg: hadoop job IDs for the pig action) through CLI -info -verbose option
> - Expose job counter info as EL Functions
> - Expose external child IDs as EL Functions
> - User level configuration to specify if stats should be collected or not (with a default setting)
> - Oozie level configuration to specify the max allowable size for stats
>
>
> This addresses bug OOZIE-588.
> https://issues.apache.org/jira/browse/OOZIE-588
>
>
> Diffs
> -----
>
> trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1212662
> trunk/client/src/main/java/org/apache/oozie/client/WorkflowAction.java 1212662
> trunk/client/src/main/java/org/apache/oozie/client/rest/JsonTags.java 1212662
> trunk/client/src/main/java/org/apache/oozie/client/rest/JsonToBean.java 1212662
> trunk/core/src/main/java/org/apache/oozie/DagELFunctions.java 1212662
> trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1212662
> trunk/core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1212662
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionStats.java PRE-CREATION
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/MRStats.java PRE-CREATION
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 1212662
> trunk/core/src/main/java/org/apache/oozie/client/rest/JsonWorkflowAction.java 1212662
> trunk/core/src/main/java/org/apache/oozie/command/wf/ActionCommand.java 1212662
> trunk/core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 1212662
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionGetJPAExecutor.java 1212662
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionSubsetGetJPAExecutor.java 1212662
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsGetForJobJPAExecutor.java 1212662
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsRunningGetJPAExecutor.java 1212662
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowJobGetActionsJPAExecutor.java 1212662
> trunk/core/src/main/java/org/apache/oozie/store/WorkflowStore.java 1212662
> trunk/core/src/test/java/org/apache/oozie/TestActionBean.java 1212662
> trunk/core/src/test/java/org/apache/oozie/action/decision/TestDecisionActionExecutor.java 1212662
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java 1212662
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestHadoopELFunctions.java 1212662
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 1212662
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 1212662
> trunk/core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java 1212662
> trunk/core/src/test/java/org/apache/oozie/client/rest/TestJsonWorkflowAction.java 1212662
>
> Diff: https://reviews.apache.org/r/3047/diff
>
>
> Testing
> -------
>
> - Tested against existing test cases and added tests where required.
> - Tests for the new classes MRStats and OoziePigStats have not been added. This is work in progress.
>
>
> Thanks,
>
> params
>
>
Re: Review Request: OOZIE-588: Oozie to allow drill down to hadoop job's
details
Posted by "Angelo K. Huang" <an...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3047/#review3820
-----------------------------------------------------------
Ship it!
ship it after the fix
trunk/core/src/main/java/org/apache/oozie/DagELFunctions.java
<https://reviews.apache.org/r/3047/#comment8648>
just reuse MapReduceActionExecutor.HADOOP_COUNTERS
- Angelo K.
On 2011-12-10 01:14:25, params wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3047/
> -----------------------------------------------------------
>
> (Updated 2011-12-10 01:14:25)
>
>
> Review request for oozie, Mohammad Islam and Angelo K. Huang.
>
>
> Summary
> -------
>
> OOZIE-588: Oozie to allow drill down to hadoop job's details
>
> Summary:
> https://issues.apache.org/jira/browse/OOZIE-588
>
> Features Implemented:
> - Expose action stats information (hadoop counters and pig stats) through CLI -info -verbose option
> - Expose external child IDs (eg: hadoop job IDs for the pig action) through CLI -info -verbose option
> - Expose job counter info as EL Functions
> - Expose external child IDs as EL Functions
> - User level configuration to specify if stats should be collected or not (with a default setting)
> - Oozie level configuration to specify the max allowable size for stats
>
>
> This addresses bug OOZIE-588.
> https://issues.apache.org/jira/browse/OOZIE-588
>
>
> Diffs
> -----
>
> trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1212662
> trunk/client/src/main/java/org/apache/oozie/client/WorkflowAction.java 1212662
> trunk/client/src/main/java/org/apache/oozie/client/rest/JsonTags.java 1212662
> trunk/client/src/main/java/org/apache/oozie/client/rest/JsonToBean.java 1212662
> trunk/core/src/main/java/org/apache/oozie/DagELFunctions.java 1212662
> trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1212662
> trunk/core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1212662
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionStats.java PRE-CREATION
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/MRStats.java PRE-CREATION
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 1212662
> trunk/core/src/main/java/org/apache/oozie/client/rest/JsonWorkflowAction.java 1212662
> trunk/core/src/main/java/org/apache/oozie/command/wf/ActionCommand.java 1212662
> trunk/core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 1212662
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionGetJPAExecutor.java 1212662
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionSubsetGetJPAExecutor.java 1212662
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsGetForJobJPAExecutor.java 1212662
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsRunningGetJPAExecutor.java 1212662
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowJobGetActionsJPAExecutor.java 1212662
> trunk/core/src/main/java/org/apache/oozie/store/WorkflowStore.java 1212662
> trunk/core/src/test/java/org/apache/oozie/TestActionBean.java 1212662
> trunk/core/src/test/java/org/apache/oozie/action/decision/TestDecisionActionExecutor.java 1212662
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java 1212662
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestHadoopELFunctions.java 1212662
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 1212662
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 1212662
> trunk/core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java 1212662
> trunk/core/src/test/java/org/apache/oozie/client/rest/TestJsonWorkflowAction.java 1212662
>
> Diff: https://reviews.apache.org/r/3047/diff
>
>
> Testing
> -------
>
> - Tested against existing test cases and added tests where required.
> - Tests for the new classes MRStats and OoziePigStats have not been added. This is work in progress.
>
>
> Thanks,
>
> params
>
>
Re: Review Request: OOZIE-588: Oozie to allow drill down to hadoop job's
details
Posted by pa...@gmail.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3047/
-----------------------------------------------------------
(Updated 2011-12-10 01:43:42.703298)
Review request for oozie, Mohammad Islam and Angelo K. Huang.
Changes
-------
Reverting previous change.
Summary
-------
OOZIE-588: Oozie to allow drill down to hadoop job's details
Summary:
https://issues.apache.org/jira/browse/OOZIE-588
Features Implemented:
- Expose action stats information (hadoop counters and pig stats) through CLI -info -verbose option
- Expose external child IDs (eg: hadoop job IDs for the pig action) through CLI -info -verbose option
- Expose job counter info as EL Functions
- Expose external child IDs as EL Functions
- User level configuration to specify if stats should be collected or not (with a default setting)
- Oozie level configuration to specify the max allowable size for stats
This addresses bug OOZIE-588.
https://issues.apache.org/jira/browse/OOZIE-588
Diffs (updated)
-----
trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1212662
trunk/client/src/main/java/org/apache/oozie/client/WorkflowAction.java 1212662
trunk/client/src/main/java/org/apache/oozie/client/rest/JsonTags.java 1212662
trunk/client/src/main/java/org/apache/oozie/client/rest/JsonToBean.java 1212662
trunk/core/src/main/java/org/apache/oozie/DagELFunctions.java 1212662
trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1212662
trunk/core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionStats.java PRE-CREATION
trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionType.java PRE-CREATION
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MRStats.java PRE-CREATION
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/client/rest/JsonWorkflowAction.java 1212662
trunk/core/src/main/java/org/apache/oozie/command/wf/ActionCommand.java 1212662
trunk/core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 1212662
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionGetJPAExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionSubsetGetJPAExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsGetForJobJPAExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsRunningGetJPAExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowJobGetActionsJPAExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/store/WorkflowStore.java 1212662
trunk/core/src/test/java/org/apache/oozie/TestActionBean.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/decision/TestDecisionActionExecutor.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestHadoopELFunctions.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java 1212662
trunk/core/src/test/java/org/apache/oozie/client/rest/TestJsonWorkflowAction.java 1212662
Diff: https://reviews.apache.org/r/3047/diff
Testing
-------
- Tested against existing test cases and added tests where required.
- Tests for the new classes MRStats and OoziePigStats have not been added. This is work in progress.
Thanks,
params
Re: Review Request: OOZIE-588: Oozie to allow drill down to hadoop job's
details
Posted by pa...@gmail.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3047/
-----------------------------------------------------------
(Updated 2011-12-10 01:23:31.128069)
Review request for oozie, Mohammad Islam and Angelo K. Huang.
Changes
-------
Updated diff based on previous comment.
Summary
-------
OOZIE-588: Oozie to allow drill down to hadoop job's details
Summary:
https://issues.apache.org/jira/browse/OOZIE-588
Features Implemented:
- Expose action stats information (hadoop counters and pig stats) through CLI -info -verbose option
- Expose external child IDs (eg: hadoop job IDs for the pig action) through CLI -info -verbose option
- Expose job counter info as EL Functions
- Expose external child IDs as EL Functions
- User level configuration to specify if stats should be collected or not (with a default setting)
- Oozie level configuration to specify the max allowable size for stats
This addresses bug OOZIE-588.
https://issues.apache.org/jira/browse/OOZIE-588
Diffs (updated)
-----
trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1212662
trunk/client/src/main/java/org/apache/oozie/client/WorkflowAction.java 1212662
trunk/client/src/main/java/org/apache/oozie/client/rest/JsonTags.java 1212662
trunk/client/src/main/java/org/apache/oozie/client/rest/JsonToBean.java 1212662
trunk/core/src/main/java/org/apache/oozie/DagELFunctions.java 1212662
trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1212662
trunk/core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionStats.java PRE-CREATION
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MRStats.java PRE-CREATION
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/client/rest/JsonWorkflowAction.java 1212662
trunk/core/src/main/java/org/apache/oozie/command/wf/ActionCommand.java 1212662
trunk/core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 1212662
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionGetJPAExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionSubsetGetJPAExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsGetForJobJPAExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsRunningGetJPAExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowJobGetActionsJPAExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/store/WorkflowStore.java 1212662
trunk/core/src/test/java/org/apache/oozie/TestActionBean.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/decision/TestDecisionActionExecutor.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestHadoopELFunctions.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java 1212662
trunk/core/src/test/java/org/apache/oozie/client/rest/TestJsonWorkflowAction.java 1212662
Diff: https://reviews.apache.org/r/3047/diff
Testing
-------
- Tested against existing test cases and added tests where required.
- Tests for the new classes MRStats and OoziePigStats have not been added. This is work in progress.
Thanks,
params
Re: Review Request: OOZIE-588: Oozie to allow drill down to hadoop job's
details
Posted by pa...@gmail.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3047/
-----------------------------------------------------------
(Updated 2011-12-10 01:14:25.958488)
Review request for oozie, Mohammad Islam and Angelo K. Huang.
Changes
-------
Incorporated review feedback.
Summary
-------
OOZIE-588: Oozie to allow drill down to hadoop job's details
Summary:
https://issues.apache.org/jira/browse/OOZIE-588
Features Implemented:
- Expose action stats information (hadoop counters and pig stats) through CLI -info -verbose option
- Expose external child IDs (eg: hadoop job IDs for the pig action) through CLI -info -verbose option
- Expose job counter info as EL Functions
- Expose external child IDs as EL Functions
- User level configuration to specify if stats should be collected or not (with a default setting)
- Oozie level configuration to specify the max allowable size for stats
This addresses bug OOZIE-588.
https://issues.apache.org/jira/browse/OOZIE-588
Diffs (updated)
-----
trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1212662
trunk/client/src/main/java/org/apache/oozie/client/WorkflowAction.java 1212662
trunk/client/src/main/java/org/apache/oozie/client/rest/JsonTags.java 1212662
trunk/client/src/main/java/org/apache/oozie/client/rest/JsonToBean.java 1212662
trunk/core/src/main/java/org/apache/oozie/DagELFunctions.java 1212662
trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1212662
trunk/core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionStats.java PRE-CREATION
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MRStats.java PRE-CREATION
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/client/rest/JsonWorkflowAction.java 1212662
trunk/core/src/main/java/org/apache/oozie/command/wf/ActionCommand.java 1212662
trunk/core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 1212662
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionGetJPAExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionSubsetGetJPAExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsGetForJobJPAExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsRunningGetJPAExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowJobGetActionsJPAExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/store/WorkflowStore.java 1212662
trunk/core/src/test/java/org/apache/oozie/TestActionBean.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/decision/TestDecisionActionExecutor.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestHadoopELFunctions.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java 1212662
trunk/core/src/test/java/org/apache/oozie/client/rest/TestJsonWorkflowAction.java 1212662
Diff: https://reviews.apache.org/r/3047/diff
Testing
-------
- Tested against existing test cases and added tests where required.
- Tests for the new classes MRStats and OoziePigStats have not been added. This is work in progress.
Thanks,
params
Re: Review Request: OOZIE-588: Oozie to allow drill down to hadoop job's
details
Posted by pa...@gmail.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3047/
-----------------------------------------------------------
(Updated 2011-12-09 22:26:29.374472)
Review request for oozie, Mohammad Islam and Angelo K. Huang.
Changes
-------
Changing MapReduceAE to support the recent change in JavaAE. Using getMaxExternalStatsSize() to get the max stats size.
Summary
-------
OOZIE-588: Oozie to allow drill down to hadoop job's details
Summary:
https://issues.apache.org/jira/browse/OOZIE-588
Features Implemented:
- Expose action stats information (hadoop counters and pig stats) through CLI -info -verbose option
- Expose external child IDs (eg: hadoop job IDs for the pig action) through CLI -info -verbose option
- Expose job counter info as EL Functions
- Expose external child IDs as EL Functions
- User level configuration to specify if stats should be collected or not (with a default setting)
- Oozie level configuration to specify the max allowable size for stats
This addresses bug OOZIE-588.
https://issues.apache.org/jira/browse/OOZIE-588
Diffs (updated)
-----
trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1212662
trunk/client/src/main/java/org/apache/oozie/client/WorkflowAction.java 1212662
trunk/client/src/main/java/org/apache/oozie/client/rest/JsonTags.java 1212662
trunk/client/src/main/java/org/apache/oozie/client/rest/JsonToBean.java 1212662
trunk/core/src/main/java/org/apache/oozie/DagELFunctions.java 1212662
trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1212662
trunk/core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionStats.java PRE-CREATION
trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionType.java PRE-CREATION
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MRStats.java PRE-CREATION
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/client/rest/JsonWorkflowAction.java 1212662
trunk/core/src/main/java/org/apache/oozie/command/wf/ActionCommand.java 1212662
trunk/core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 1212662
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionGetJPAExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionSubsetGetJPAExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsGetForJobJPAExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsRunningGetJPAExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowJobGetActionsJPAExecutor.java 1212662
trunk/core/src/main/java/org/apache/oozie/store/WorkflowStore.java 1212662
trunk/core/src/test/java/org/apache/oozie/TestActionBean.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/decision/TestDecisionActionExecutor.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestHadoopELFunctions.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 1212662
trunk/core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java 1212662
trunk/core/src/test/java/org/apache/oozie/client/rest/TestJsonWorkflowAction.java 1212662
Diff: https://reviews.apache.org/r/3047/diff
Testing
-------
- Tested against existing test cases and added tests where required.
- Tests for the new classes MRStats and OoziePigStats have not been added. This is work in progress.
Thanks,
params
Re: Review Request: OOZIE-588: Oozie to allow drill down to hadoop job's
details
Posted by pa...@gmail.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3047/
-----------------------------------------------------------
(Updated 2011-12-09 19:42:17.424687)
Review request for oozie, Mohammad Islam and Angelo K. Huang.
Changes
-------
Incorporated review comments. Added new testcases.
Summary
-------
OOZIE-588: Oozie to allow drill down to hadoop job's details
Summary:
https://issues.apache.org/jira/browse/OOZIE-588
Features Implemented:
- Expose action stats information (hadoop counters and pig stats) through CLI -info -verbose option
- Expose external child IDs (eg: hadoop job IDs for the pig action) through CLI -info -verbose option
- Expose job counter info as EL Functions
- Expose external child IDs as EL Functions
- User level configuration to specify if stats should be collected or not (with a default setting)
- Oozie level configuration to specify the max allowable size for stats
This addresses bug OOZIE-588.
https://issues.apache.org/jira/browse/OOZIE-588
Diffs (updated)
-----
trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1212593
trunk/client/src/main/java/org/apache/oozie/client/WorkflowAction.java 1212593
trunk/client/src/main/java/org/apache/oozie/client/rest/JsonTags.java 1212593
trunk/client/src/main/java/org/apache/oozie/client/rest/JsonToBean.java 1212593
trunk/core/src/main/java/org/apache/oozie/DagELFunctions.java 1212593
trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1212593
trunk/core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1212593
trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionStats.java PRE-CREATION
trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionType.java PRE-CREATION
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MRStats.java PRE-CREATION
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 1212593
trunk/core/src/main/java/org/apache/oozie/client/rest/JsonWorkflowAction.java 1212593
trunk/core/src/main/java/org/apache/oozie/command/wf/ActionCommand.java 1212593
trunk/core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 1212593
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionGetJPAExecutor.java 1212593
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionSubsetGetJPAExecutor.java 1212593
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsGetForJobJPAExecutor.java 1212593
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsRunningGetJPAExecutor.java 1212593
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowJobGetActionsJPAExecutor.java 1212593
trunk/core/src/main/java/org/apache/oozie/store/WorkflowStore.java 1212593
trunk/core/src/test/java/org/apache/oozie/TestActionBean.java 1212593
trunk/core/src/test/java/org/apache/oozie/action/decision/TestDecisionActionExecutor.java 1212593
trunk/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java 1212593
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestHadoopELFunctions.java 1212593
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 1212593
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 1212593
trunk/core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java 1212593
trunk/core/src/test/java/org/apache/oozie/client/rest/TestJsonWorkflowAction.java 1212593
Diff: https://reviews.apache.org/r/3047/diff
Testing
-------
- Tested against existing test cases and added tests where required.
- Tests for the new classes MRStats and OoziePigStats have not been added. This is work in progress.
Thanks,
params
Re: Review Request: OOZIE-588: Oozie to allow drill down to hadoop job's
details
Posted by pa...@gmail.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3047/
-----------------------------------------------------------
(Updated 2011-12-07 23:53:52.679179)
Review request for oozie, Mohammad Islam and Angelo K. Huang.
Changes
-------
Incorporated review comments.
Summary
-------
OOZIE-588: Oozie to allow drill down to hadoop job's details
Summary:
https://issues.apache.org/jira/browse/OOZIE-588
Features Implemented:
- Expose action stats information (hadoop counters and pig stats) through CLI -info -verbose option
- Expose external child IDs (eg: hadoop job IDs for the pig action) through CLI -info -verbose option
- Expose job counter info as EL Functions
- Expose external child IDs as EL Functions
- User level configuration to specify if stats should be collected or not (with a default setting)
- Oozie level configuration to specify the max allowable size for stats
This addresses bug OOZIE-588.
https://issues.apache.org/jira/browse/OOZIE-588
Diffs (updated)
-----
trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1211308
trunk/client/src/main/java/org/apache/oozie/client/WorkflowAction.java 1211308
trunk/client/src/main/java/org/apache/oozie/client/rest/JsonTags.java 1211308
trunk/client/src/main/java/org/apache/oozie/client/rest/JsonToBean.java 1211308
trunk/core/src/main/java/org/apache/oozie/DagELFunctions.java 1211308
trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1211308
trunk/core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1211308
trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionStats.java PRE-CREATION
trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionType.java PRE-CREATION
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MRStats.java PRE-CREATION
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 1211308
trunk/core/src/main/java/org/apache/oozie/client/rest/JsonWorkflowAction.java 1211308
trunk/core/src/main/java/org/apache/oozie/command/wf/ActionCommand.java 1211308
trunk/core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 1211308
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionGetJPAExecutor.java 1211308
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionSubsetGetJPAExecutor.java 1211308
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsGetForJobJPAExecutor.java 1211308
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsRunningGetJPAExecutor.java 1211308
trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowJobGetActionsJPAExecutor.java 1211308
trunk/core/src/main/java/org/apache/oozie/store/WorkflowStore.java 1211308
trunk/core/src/test/java/org/apache/oozie/TestActionBean.java 1211308
trunk/core/src/test/java/org/apache/oozie/action/decision/TestDecisionActionExecutor.java 1211308
trunk/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java 1211308
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestHadoopELFunctions.java 1211308
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 1211308
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 1211308
trunk/core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java 1211308
trunk/core/src/test/java/org/apache/oozie/client/rest/TestJsonWorkflowAction.java 1211308
Diff: https://reviews.apache.org/r/3047/diff
Testing
-------
- Tested against existing test cases and added tests where required.
- Tests for the new classes MRStats and OoziePigStats have not been added. This is work in progress.
Thanks,
params
Re: Review Request: OOZIE-588: Oozie to allow drill down to hadoop job's
details
Posted by pa...@gmail.com.
> On 2011-12-07 21:29:50, Prakhar Sharma wrote:
> > trunk/core/src/main/java/org/apache/oozie/client/rest/JsonWorkflowAction.java, line 325
> > <https://reviews.apache.org/r/3047/diff/1/?file=62775#file62775line325>
> >
> > [Cosmetic comment]
> > Place getStats() next to setStats()?
Moved.
> On 2011-12-07 21:29:50, Prakhar Sharma wrote:
> > trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java, line 200
> > <https://reviews.apache.org/r/3047/diff/1/?file=62774#file62774line200>
> >
> > [Cosmetic comment]
> > Compiler will most likely remove local variables 'name' and 'value'. You can also :)
Changed.
> On 2011-12-07 21:29:50, Prakhar Sharma wrote:
> > trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java, line 158
> > <https://reviews.apache.org/r/3047/diff/1/?file=62774#file62774line158>
> >
> > Remove <<Params>> :)
Removed.
> On 2011-12-07 21:29:50, Prakhar Sharma wrote:
> > trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java, line 152
> > <https://reviews.apache.org/r/3047/diff/1/?file=62774#file62774line152>
> >
> > Link to the comment above regarding ActionType parameter to MRStats ctor being redundant.
The first argument is no more required to be passed (ActionType is set implicitly inside the class based on the type of stats).
> On 2011-12-07 21:29:50, Prakhar Sharma wrote:
> > trunk/core/src/main/java/org/apache/oozie/action/hadoop/MRStats.java, line 32
> > <https://reviews.apache.org/r/3047/diff/1/?file=62773#file62773line32>
> >
> > Isn't ActionType parameter to the ctor redundant? Will this class ever create an object with value of ActionType parameter other than MAP_REDUCE? If yes, then you can ignore this comment.
Yes, it seems redundant. Removing it from here. Instead, it ll be set inside the ctor based on the type of action.
> On 2011-12-07 21:29:50, Prakhar Sharma wrote:
> > trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionStats.java, line 39
> > <https://reviews.apache.org/r/3047/diff/1/?file=62771#file62771line39>
> >
> > Parent class should not know the type of derived classes. setCurrentActionType is being called only from derived classes. This method can be just removed.
Removing the setter.
> On 2011-12-07 21:29:50, Prakhar Sharma wrote:
> > trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionStats.java, line 25
> > <https://reviews.apache.org/r/3047/diff/1/?file=62771#file62771line25>
> >
> > Make currentActionType protected and set it directly in derived classes and do away with the corresponding setter.
> >
> > Even better design will be not to have the currentActionType in parent class altogether and have an abstract method (like toJSON()) which returns the type. Motivation being: Parent should not care who is deriving from it.
Made currentActionType protected. Corresponding setter will be removed.
> On 2011-12-07 21:29:50, Prakhar Sharma wrote:
> > trunk/core/src/main/java/org/apache/oozie/DagELFunctions.java, line 116
> > <https://reviews.apache.org/r/3047/diff/1/?file=62768#file62768line116>
> >
> > Rational behind short circuit condition: action.getData() == null
> >
> > Inside the if block you are not accessing return value of action.getData(). Looks like you might end up missing some cases due to the check.
> >
> > If ensuring (action.getData() != null && action.getExternalChildIds() != null) not being true is the motivation, then may be consider throwing an exception?
Agreed. This might miss some cases.
Removing getData() check from the if condition check. Retaining just the required check alone (which is to test that externalChildIDs are not null).
> On 2011-12-07 21:29:50, Prakhar Sharma wrote:
> > trunk/client/src/main/java/org/apache/oozie/client/rest/JsonTags.java, line 69
> > <https://reviews.apache.org/r/3047/diff/1/?file=62766#file62766line69>
> >
> > Rename "externalchildids" to "externalChildIds".
> > Rest of the variable values appear to be following the later format.
Renamed.
> On 2011-12-07 21:29:50, Prakhar Sharma wrote:
> > trunk/core/src/main/java/org/apache/oozie/DagELFunctions.java, line 42
> > <https://reviews.apache.org/r/3047/diff/1/?file=62768#file62768line42>
> >
> > Specific reason for the trailing ':' in value of HADOOP_JOBS_PREFIX?
Hadoop Job IDs are stored in the WorkflowInstance as a property (hadoopJobs:job1,job2). Since, we are creating this property in the DAGELFunctions.java, we need to prefix "hadoopJobs:" to the data retrieved from getExternalChildIDs() function - ("job1,job2").
For the other variables stored in WorkflowInstance, this prefixing is not required as they are already retrieved as properties (A:a).
- params
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3047/#review3711
-----------------------------------------------------------
On 2011-12-07 23:53:52, params wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3047/
> -----------------------------------------------------------
>
> (Updated 2011-12-07 23:53:52)
>
>
> Review request for oozie, Mohammad Islam and Angelo K. Huang.
>
>
> Summary
> -------
>
> OOZIE-588: Oozie to allow drill down to hadoop job's details
>
> Summary:
> https://issues.apache.org/jira/browse/OOZIE-588
>
> Features Implemented:
> - Expose action stats information (hadoop counters and pig stats) through CLI -info -verbose option
> - Expose external child IDs (eg: hadoop job IDs for the pig action) through CLI -info -verbose option
> - Expose job counter info as EL Functions
> - Expose external child IDs as EL Functions
> - User level configuration to specify if stats should be collected or not (with a default setting)
> - Oozie level configuration to specify the max allowable size for stats
>
>
> This addresses bug OOZIE-588.
> https://issues.apache.org/jira/browse/OOZIE-588
>
>
> Diffs
> -----
>
> trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1211308
> trunk/client/src/main/java/org/apache/oozie/client/WorkflowAction.java 1211308
> trunk/client/src/main/java/org/apache/oozie/client/rest/JsonTags.java 1211308
> trunk/client/src/main/java/org/apache/oozie/client/rest/JsonToBean.java 1211308
> trunk/core/src/main/java/org/apache/oozie/DagELFunctions.java 1211308
> trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1211308
> trunk/core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionStats.java PRE-CREATION
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionType.java PRE-CREATION
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/MRStats.java PRE-CREATION
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/client/rest/JsonWorkflowAction.java 1211308
> trunk/core/src/main/java/org/apache/oozie/command/wf/ActionCommand.java 1211308
> trunk/core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionGetJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionSubsetGetJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsGetForJobJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsRunningGetJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowJobGetActionsJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/store/WorkflowStore.java 1211308
> trunk/core/src/test/java/org/apache/oozie/TestActionBean.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/decision/TestDecisionActionExecutor.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestHadoopELFunctions.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java 1211308
> trunk/core/src/test/java/org/apache/oozie/client/rest/TestJsonWorkflowAction.java 1211308
>
> Diff: https://reviews.apache.org/r/3047/diff
>
>
> Testing
> -------
>
> - Tested against existing test cases and added tests where required.
> - Tests for the new classes MRStats and OoziePigStats have not been added. This is work in progress.
>
>
> Thanks,
>
> params
>
>
Re: Review Request: OOZIE-588: Oozie to allow drill down to hadoop job's
details
Posted by Prakhar Sharma <pr...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3047/#review3711
-----------------------------------------------------------
trunk/client/src/main/java/org/apache/oozie/client/rest/JsonTags.java
<https://reviews.apache.org/r/3047/#comment8305>
Rename "externalchildids" to "externalChildIds".
Rest of the variable values appear to be following the later format.
trunk/core/src/main/java/org/apache/oozie/DagELFunctions.java
<https://reviews.apache.org/r/3047/#comment8307>
Specific reason for the trailing ':' in value of HADOOP_JOBS_PREFIX?
trunk/core/src/main/java/org/apache/oozie/DagELFunctions.java
<https://reviews.apache.org/r/3047/#comment8308>
Rational behind short circuit condition: action.getData() == null
Inside the if block you are not accessing return value of action.getData(). Looks like you might end up missing some cases due to the check.
If ensuring (action.getData() != null && action.getExternalChildIds() != null) not being true is the motivation, then may be consider throwing an exception?
trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionStats.java
<https://reviews.apache.org/r/3047/#comment8312>
Make currentActionType protected and set it directly in derived classes and do away with the corresponding setter.
Even better design will be not to have the currentActionType in parent class altogether and have an abstract method (like toJSON()) which returns the type. Motivation being: Parent should not care who is deriving from it.
trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionStats.java
<https://reviews.apache.org/r/3047/#comment8310>
Parent class should not know the type of derived classes. setCurrentActionType is being called only from derived classes. This method can be just removed.
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MRStats.java
<https://reviews.apache.org/r/3047/#comment8317>
Isn't ActionType parameter to the ctor redundant? Will this class ever create an object with value of ActionType parameter other than MAP_REDUCE? If yes, then you can ignore this comment.
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java
<https://reviews.apache.org/r/3047/#comment8321>
Link to the comment above regarding ActionType parameter to MRStats ctor being redundant.
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java
<https://reviews.apache.org/r/3047/#comment8322>
Remove <<Params>> :)
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java
<https://reviews.apache.org/r/3047/#comment8324>
[Styling related comment]
Alternative can be:
if (actionConf == null) {
return "";
}
// code right now inside if
// Less indentation -> More readability
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java
<https://reviews.apache.org/r/3047/#comment8325>
[Cosmetic comment]
Compiler will most likely remove local variables 'name' and 'value'. You can also :)
trunk/core/src/main/java/org/apache/oozie/client/rest/JsonWorkflowAction.java
<https://reviews.apache.org/r/3047/#comment8326>
[Cosmetic comment]
Place getStats() next to setStats()?
- Prakhar
On 2011-12-07 07:06:39, params wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3047/
> -----------------------------------------------------------
>
> (Updated 2011-12-07 07:06:39)
>
>
> Review request for oozie, Mohammad Islam and Angelo K. Huang.
>
>
> Summary
> -------
>
> OOZIE-588: Oozie to allow drill down to hadoop job's details
>
> Summary:
> https://issues.apache.org/jira/browse/OOZIE-588
>
> Features Implemented:
> - Expose action stats information (hadoop counters and pig stats) through CLI -info -verbose option
> - Expose external child IDs (eg: hadoop job IDs for the pig action) through CLI -info -verbose option
> - Expose job counter info as EL Functions
> - Expose external child IDs as EL Functions
> - User level configuration to specify if stats should be collected or not (with a default setting)
> - Oozie level configuration to specify the max allowable size for stats
>
>
> This addresses bug OOZIE-588.
> https://issues.apache.org/jira/browse/OOZIE-588
>
>
> Diffs
> -----
>
> trunk/core/src/test/java/org/apache/oozie/client/rest/TestJsonWorkflowAction.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/decision/TestDecisionActionExecutor.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java 1211308
> trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestHadoopELFunctions.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsRunningGetJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowJobGetActionsJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/store/WorkflowStore.java 1211308
> trunk/core/src/test/java/org/apache/oozie/TestActionBean.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionSubsetGetJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionsGetForJobJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/client/rest/JsonWorkflowAction.java 1211308
> trunk/core/src/main/java/org/apache/oozie/command/wf/ActionCommand.java 1211308
> trunk/core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 1211308
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionGetJPAExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/MRStats.java PRE-CREATION
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionType.java PRE-CREATION
> trunk/core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1211308
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionStats.java PRE-CREATION
> trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1211308
> trunk/core/src/main/java/org/apache/oozie/DagELFunctions.java 1211308
> trunk/client/src/main/java/org/apache/oozie/client/rest/JsonTags.java 1211308
> trunk/client/src/main/java/org/apache/oozie/client/rest/JsonToBean.java 1211308
> trunk/client/src/main/java/org/apache/oozie/client/WorkflowAction.java 1211308
> trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1211308
>
> Diff: https://reviews.apache.org/r/3047/diff
>
>
> Testing
> -------
>
> - Tested against existing test cases and added tests where required.
> - Tests for the new classes MRStats and OoziePigStats have not been added. This is work in progress.
>
>
> Thanks,
>
> params
>
>