You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by Purshotam Shah <pu...@yahoo-inc.com> on 2014/01/14 03:02:29 UTC
Review Request 16840: OOZIE-1668 Coord log streaming start and end time
should be of action list start and end time
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16840/
-----------------------------------------------------------
Review request for oozie.
Bugs: OOZIE-1668
https://issues.apache.org/jira/browse/OOZIE-1668
Repository: oozie-git
Description
-------
Coord log streaming start and end time should be of action list start and end time
Diffs
-----
core/src/main/java/org/apache/oozie/CoordinatorEngine.java 57587c2
core/src/test/java/org/apache/oozie/TestCoordinatorEngineStreamLog.java a29a07e
Diff: https://reviews.apache.org/r/16840/diff/
Testing
-------
Refactored test-cases.
Each test cases run a coord job which takes 5 min to complete.
Made changes so that coord job is executed only once and all test cases use that for validate.
Thanks,
Purshotam Shah
Re: Review Request 16840: OOZIE-1668 Coord log streaming start and end time
should be of action list start and end time
Posted by Mona Chitnis <mo...@yahoo.in>.
> On Jan. 23, 2014, 4 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionModifiedDateForRangeJPAExecutor.java, line 32
> > <https://reviews.apache.org/r/16840/diff/3/?file=433918#file433918line32>
> >
> > Please do not write new JPAExecutor classes. Add to the QueryExecutor class.
we discussed this yesterday. currently the QueryExecutor classes are meant to return beans - coord job bean, action bean etc. His queries here are returning a count and a maximum timestamp so cant fit in the query executor
- Mona
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16840/#review32594
-----------------------------------------------------------
On Jan. 22, 2014, 1:24 a.m., Purshotam Shah wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16840/
> -----------------------------------------------------------
>
> (Updated Jan. 22, 2014, 1:24 a.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-1668
> https://issues.apache.org/jira/browse/OOZIE-1668
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> Coord log streaming start and end time should be of action list start and end time
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/BundleEngine.java e8c8f31
> core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 5d8d6df
> core/src/main/java/org/apache/oozie/CoordinatorEngine.java 57587c2
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionModifiedDateForRangeJPAExecutor.java PRE-CREATION
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionRunningCountForRangeJPAExecutor.java PRE-CREATION
> core/src/main/java/org/apache/oozie/util/CoordActionsInDateRange.java 6746922
> core/src/test/java/org/apache/oozie/TestCoordinatorEngineStreamLog.java a29a07e
>
> Diff: https://reviews.apache.org/r/16840/diff/
>
>
> Testing
> -------
>
> Refactored test-cases.
> Each test cases run a coord job which takes 5 min to complete.
> Made changes so that coord job is executed only once and all test cases use that for validate.
>
>
> Thanks,
>
> Purshotam Shah
>
>
Re: Review Request 16840: OOZIE-1668 Coord log streaming start and end time
should be of action list start and end time
Posted by Purshotam Shah <pu...@yahoo-inc.com>.
> On Jan. 23, 2014, 4 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/CoordinatorEngine.java, line 376
> > <https://reviews.apache.org/r/16840/diff/1-3/?file=422275#file422275line376>
> >
> > Instead of making another query below to get endtime, here (new named query) get both id and endTime for a coord action and use a comparator on that to sort by endtime.
If list is huge, it need to get lastmodified date and status of each action and iterate see to status or find max lastmodified.
Making two more DB call is simple and efficient.
> On Jan. 23, 2014, 4 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/CoordinatorActionBean.java, line 162
> > <https://reviews.apache.org/r/16840/diff/3/?file=433916#file433916line162>
> >
> > There could be PREP or SUSPENDED workflows. Need to check for endtime = null instead of status
If action is PREP or SUSPENDED, last modified date will tell when it was last modified. So i believe it's ok.
- Purshotam
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16840/#review32594
-----------------------------------------------------------
On Jan. 23, 2014, 11:24 p.m., Purshotam Shah wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16840/
> -----------------------------------------------------------
>
> (Updated Jan. 23, 2014, 11:24 p.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-1668
> https://issues.apache.org/jira/browse/OOZIE-1668
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> Coord log streaming start and end time should be of action list start and end time
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/BundleEngine.java e8c8f31
> core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 5d8d6df
> core/src/main/java/org/apache/oozie/CoordinatorEngine.java 57587c2
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionModifiedDateForRangeJPAExecutor.java PRE-CREATION
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionRunningCountForRangeJPAExecutor.java PRE-CREATION
> core/src/main/java/org/apache/oozie/util/CoordActionsInDateRange.java 6746922
> core/src/test/java/org/apache/oozie/TestCoordinatorEngineStreamLog.java a29a07e
>
> Diff: https://reviews.apache.org/r/16840/diff/
>
>
> Testing
> -------
>
> Refactored test-cases.
> Each test cases run a coord job which takes 5 min to complete.
> Made changes so that coord job is executed only once and all test cases use that for validate.
>
>
> Thanks,
>
> Purshotam Shah
>
>
Re: Review Request 16840: OOZIE-1668 Coord log streaming start and end time
should be of action list start and end time
Posted by Rohini Palaniswamy <ro...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16840/#review32594
-----------------------------------------------------------
core/src/main/java/org/apache/oozie/CoordinatorEngine.java
<https://reviews.apache.org/r/16840/#comment61499>
If there is only one action (which is most of the time), you can get the details of starttime and endtime from getCoordAction(tempList.get(0)) itself instead of making a db query.
core/src/main/java/org/apache/oozie/CoordinatorEngine.java
<https://reviews.apache.org/r/16840/#comment61501>
Instead of making another query below to get endtime, here (new named query) get both id and endTime for a coord action and use a comparator on that to sort by endtime.
core/src/main/java/org/apache/oozie/CoordinatorActionBean.java
<https://reviews.apache.org/r/16840/#comment61500>
There could be PREP or SUSPENDED workflows. Need to check for endtime = null instead of status
core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionModifiedDateForRangeJPAExecutor.java
<https://reviews.apache.org/r/16840/#comment61498>
Please do not write new JPAExecutor classes. Add to the QueryExecutor class.
- Rohini Palaniswamy
On Jan. 22, 2014, 1:24 a.m., Purshotam Shah wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16840/
> -----------------------------------------------------------
>
> (Updated Jan. 22, 2014, 1:24 a.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-1668
> https://issues.apache.org/jira/browse/OOZIE-1668
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> Coord log streaming start and end time should be of action list start and end time
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/BundleEngine.java e8c8f31
> core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 5d8d6df
> core/src/main/java/org/apache/oozie/CoordinatorEngine.java 57587c2
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionModifiedDateForRangeJPAExecutor.java PRE-CREATION
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionRunningCountForRangeJPAExecutor.java PRE-CREATION
> core/src/main/java/org/apache/oozie/util/CoordActionsInDateRange.java 6746922
> core/src/test/java/org/apache/oozie/TestCoordinatorEngineStreamLog.java a29a07e
>
> Diff: https://reviews.apache.org/r/16840/diff/
>
>
> Testing
> -------
>
> Refactored test-cases.
> Each test cases run a coord job which takes 5 min to complete.
> Made changes so that coord job is executed only once and all test cases use that for validate.
>
>
> Thanks,
>
> Purshotam Shah
>
>
Re: Review Request 16840: OOZIE-1668 Coord log streaming start and end time
should be of action list start and end time
Posted by Rohini Palaniswamy <ro...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16840/#review32683
-----------------------------------------------------------
Ship it!
Ship It!
- Rohini Palaniswamy
On Jan. 23, 2014, 11:24 p.m., Purshotam Shah wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16840/
> -----------------------------------------------------------
>
> (Updated Jan. 23, 2014, 11:24 p.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-1668
> https://issues.apache.org/jira/browse/OOZIE-1668
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> Coord log streaming start and end time should be of action list start and end time
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/BundleEngine.java e8c8f31
> core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 5d8d6df
> core/src/main/java/org/apache/oozie/CoordinatorEngine.java 57587c2
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionModifiedDateForRangeJPAExecutor.java PRE-CREATION
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionRunningCountForRangeJPAExecutor.java PRE-CREATION
> core/src/main/java/org/apache/oozie/util/CoordActionsInDateRange.java 6746922
> core/src/test/java/org/apache/oozie/TestCoordinatorEngineStreamLog.java a29a07e
>
> Diff: https://reviews.apache.org/r/16840/diff/
>
>
> Testing
> -------
>
> Refactored test-cases.
> Each test cases run a coord job which takes 5 min to complete.
> Made changes so that coord job is executed only once and all test cases use that for validate.
>
>
> Thanks,
>
> Purshotam Shah
>
>
Re: Review Request 16840: OOZIE-1668 Coord log streaming start and end time
should be of action list start and end time
Posted by Rohini Palaniswamy <ro...@gmail.com>.
> On Jan. 24, 2014, 1:15 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/CoordinatorEngine.java, line 356
> > <https://reviews.apache.org/r/16840/diff/3-4/?file=433917#file433917line356>
> >
> > Do a endtime=null instead of RUNNING check. There are other states like PREP, WAITING, etc apart from RUNNING
Puru pointed out that there is no endtime for coord action. And it should be ok to stream till last modified time. Refresh would anyway show new log statements.
- Rohini
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16840/#review32681
-----------------------------------------------------------
On Jan. 23, 2014, 11:24 p.m., Purshotam Shah wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16840/
> -----------------------------------------------------------
>
> (Updated Jan. 23, 2014, 11:24 p.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-1668
> https://issues.apache.org/jira/browse/OOZIE-1668
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> Coord log streaming start and end time should be of action list start and end time
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/BundleEngine.java e8c8f31
> core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 5d8d6df
> core/src/main/java/org/apache/oozie/CoordinatorEngine.java 57587c2
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionModifiedDateForRangeJPAExecutor.java PRE-CREATION
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionRunningCountForRangeJPAExecutor.java PRE-CREATION
> core/src/main/java/org/apache/oozie/util/CoordActionsInDateRange.java 6746922
> core/src/test/java/org/apache/oozie/TestCoordinatorEngineStreamLog.java a29a07e
>
> Diff: https://reviews.apache.org/r/16840/diff/
>
>
> Testing
> -------
>
> Refactored test-cases.
> Each test cases run a coord job which takes 5 min to complete.
> Made changes so that coord job is executed only once and all test cases use that for validate.
>
>
> Thanks,
>
> Purshotam Shah
>
>
Re: Review Request 16840: OOZIE-1668 Coord log streaming start and end time
should be of action list start and end time
Posted by Rohini Palaniswamy <ro...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16840/#review32681
-----------------------------------------------------------
core/src/main/java/org/apache/oozie/CoordinatorEngine.java
<https://reviews.apache.org/r/16840/#comment61625>
Do a endtime=null instead of RUNNING check. There are other states like PREP, WAITING, etc apart from RUNNING
- Rohini Palaniswamy
On Jan. 23, 2014, 11:24 p.m., Purshotam Shah wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16840/
> -----------------------------------------------------------
>
> (Updated Jan. 23, 2014, 11:24 p.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-1668
> https://issues.apache.org/jira/browse/OOZIE-1668
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> Coord log streaming start and end time should be of action list start and end time
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/BundleEngine.java e8c8f31
> core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 5d8d6df
> core/src/main/java/org/apache/oozie/CoordinatorEngine.java 57587c2
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionModifiedDateForRangeJPAExecutor.java PRE-CREATION
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionRunningCountForRangeJPAExecutor.java PRE-CREATION
> core/src/main/java/org/apache/oozie/util/CoordActionsInDateRange.java 6746922
> core/src/test/java/org/apache/oozie/TestCoordinatorEngineStreamLog.java a29a07e
>
> Diff: https://reviews.apache.org/r/16840/diff/
>
>
> Testing
> -------
>
> Refactored test-cases.
> Each test cases run a coord job which takes 5 min to complete.
> Made changes so that coord job is executed only once and all test cases use that for validate.
>
>
> Thanks,
>
> Purshotam Shah
>
>
Re: Review Request 16840: OOZIE-1668 Coord log streaming start and end time
should be of action list start and end time
Posted by Purshotam Shah <pu...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16840/
-----------------------------------------------------------
(Updated Jan. 23, 2014, 11:24 p.m.)
Review request for oozie.
Changes
-------
review comment.
Bugs: OOZIE-1668
https://issues.apache.org/jira/browse/OOZIE-1668
Repository: oozie-git
Description
-------
Coord log streaming start and end time should be of action list start and end time
Diffs (updated)
-----
core/src/main/java/org/apache/oozie/BundleEngine.java e8c8f31
core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 5d8d6df
core/src/main/java/org/apache/oozie/CoordinatorEngine.java 57587c2
core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionModifiedDateForRangeJPAExecutor.java PRE-CREATION
core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionRunningCountForRangeJPAExecutor.java PRE-CREATION
core/src/main/java/org/apache/oozie/util/CoordActionsInDateRange.java 6746922
core/src/test/java/org/apache/oozie/TestCoordinatorEngineStreamLog.java a29a07e
Diff: https://reviews.apache.org/r/16840/diff/
Testing
-------
Refactored test-cases.
Each test cases run a coord job which takes 5 min to complete.
Made changes so that coord job is executed only once and all test cases use that for validate.
Thanks,
Purshotam Shah
Re: Review Request 16840: OOZIE-1668 Coord log streaming start and end time
should be of action list start and end time
Posted by Purshotam Shah <pu...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16840/
-----------------------------------------------------------
(Updated Jan. 22, 2014, 1:24 a.m.)
Review request for oozie.
Changes
-------
Review comment.
Bugs: OOZIE-1668
https://issues.apache.org/jira/browse/OOZIE-1668
Repository: oozie-git
Description
-------
Coord log streaming start and end time should be of action list start and end time
Diffs (updated)
-----
core/src/main/java/org/apache/oozie/BundleEngine.java e8c8f31
core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 5d8d6df
core/src/main/java/org/apache/oozie/CoordinatorEngine.java 57587c2
core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionModifiedDateForRangeJPAExecutor.java PRE-CREATION
core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionRunningCountForRangeJPAExecutor.java PRE-CREATION
core/src/main/java/org/apache/oozie/util/CoordActionsInDateRange.java 6746922
core/src/test/java/org/apache/oozie/TestCoordinatorEngineStreamLog.java a29a07e
Diff: https://reviews.apache.org/r/16840/diff/
Testing
-------
Refactored test-cases.
Each test cases run a coord job which takes 5 min to complete.
Made changes so that coord job is executed only once and all test cases use that for validate.
Thanks,
Purshotam Shah
Re: Review Request 16840: OOZIE-1668 Coord log streaming start and end time
should be of action list start and end time
Posted by Rohini Palaniswamy <ro...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16840/#review32240
-----------------------------------------------------------
core/src/main/java/org/apache/oozie/CoordinatorEngine.java
<https://reviews.apache.org/r/16840/#comment60969>
Can you fix BundleEngine.streamLog as well to take into account bundle endtime if it is not null.
- Rohini Palaniswamy
On Jan. 14, 2014, 2:02 a.m., Purshotam Shah wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16840/
> -----------------------------------------------------------
>
> (Updated Jan. 14, 2014, 2:02 a.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-1668
> https://issues.apache.org/jira/browse/OOZIE-1668
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> Coord log streaming start and end time should be of action list start and end time
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/CoordinatorEngine.java 57587c2
> core/src/test/java/org/apache/oozie/TestCoordinatorEngineStreamLog.java a29a07e
>
> Diff: https://reviews.apache.org/r/16840/diff/
>
>
> Testing
> -------
>
> Refactored test-cases.
> Each test cases run a coord job which takes 5 min to complete.
> Made changes so that coord job is executed only once and all test cases use that for validate.
>
>
> Thanks,
>
> Purshotam Shah
>
>
Re: Review Request 16840: OOZIE-1668 Coord log streaming start and end time
should be of action list start and end time
Posted by Purshotam Shah <pu...@yahoo-inc.com>.
> On Jan. 18, 2014, 12:49 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/CoordinatorEngine.java, line 363
> > <https://reviews.apache.org/r/16840/diff/2/?file=422334#file422334line363>
> >
> > You need to get max of endtime iterating through action list instead of just picking the last action's. And if one of them is still RUNNING better to do new Date().
> >
> > For eg: You can have action 3 ending after action 5 if data was available later or if it was rerun.
>
> Purshotam Shah wrote:
> Thanks for pointing out it.
>
> In place of making multiple DB call to get the status and last modified time for each action.
>
> We make two calls.
> 1. Get the running count for action list in range.
> If any of them is running than end date = current date.
>
> 2. If none of them are running, make a DB call get max of last modified date.
>
>
Thanks for pointing out it.
In place of making multiple DB call to get the status and last modified time for each action.
We make two calls.
1. Get the running count for action list in range.
If any of them is running than end date = current date.
2. If none of them are running, make a DB call get max of last modified date.
- Purshotam
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16840/#review32237
-----------------------------------------------------------
On Jan. 22, 2014, 1:24 a.m., Purshotam Shah wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16840/
> -----------------------------------------------------------
>
> (Updated Jan. 22, 2014, 1:24 a.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-1668
> https://issues.apache.org/jira/browse/OOZIE-1668
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> Coord log streaming start and end time should be of action list start and end time
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/BundleEngine.java e8c8f31
> core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 5d8d6df
> core/src/main/java/org/apache/oozie/CoordinatorEngine.java 57587c2
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionModifiedDateForRangeJPAExecutor.java PRE-CREATION
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionRunningCountForRangeJPAExecutor.java PRE-CREATION
> core/src/main/java/org/apache/oozie/util/CoordActionsInDateRange.java 6746922
> core/src/test/java/org/apache/oozie/TestCoordinatorEngineStreamLog.java a29a07e
>
> Diff: https://reviews.apache.org/r/16840/diff/
>
>
> Testing
> -------
>
> Refactored test-cases.
> Each test cases run a coord job which takes 5 min to complete.
> Made changes so that coord job is executed only once and all test cases use that for validate.
>
>
> Thanks,
>
> Purshotam Shah
>
>
Re: Review Request 16840: OOZIE-1668 Coord log streaming start and end time
should be of action list start and end time
Posted by Purshotam Shah <pu...@yahoo-inc.com>.
> On Jan. 18, 2014, 12:49 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/CoordinatorEngine.java, line 363
> > <https://reviews.apache.org/r/16840/diff/2/?file=422334#file422334line363>
> >
> > You need to get max of endtime iterating through action list instead of just picking the last action's. And if one of them is still RUNNING better to do new Date().
> >
> > For eg: You can have action 3 ending after action 5 if data was available later or if it was rerun.
Thanks for pointing out it.
In place of making multiple DB call to get the status and last modified time for each action.
We make two calls.
1. Get the running count for action list in range.
If any of them is running than end date = current date.
2. If none of them are running, make a DB call get max of last modified date.
- Purshotam
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16840/#review32237
-----------------------------------------------------------
On Jan. 22, 2014, 1:24 a.m., Purshotam Shah wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16840/
> -----------------------------------------------------------
>
> (Updated Jan. 22, 2014, 1:24 a.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-1668
> https://issues.apache.org/jira/browse/OOZIE-1668
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> Coord log streaming start and end time should be of action list start and end time
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/BundleEngine.java e8c8f31
> core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 5d8d6df
> core/src/main/java/org/apache/oozie/CoordinatorEngine.java 57587c2
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionModifiedDateForRangeJPAExecutor.java PRE-CREATION
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionRunningCountForRangeJPAExecutor.java PRE-CREATION
> core/src/main/java/org/apache/oozie/util/CoordActionsInDateRange.java 6746922
> core/src/test/java/org/apache/oozie/TestCoordinatorEngineStreamLog.java a29a07e
>
> Diff: https://reviews.apache.org/r/16840/diff/
>
>
> Testing
> -------
>
> Refactored test-cases.
> Each test cases run a coord job which takes 5 min to complete.
> Made changes so that coord job is executed only once and all test cases use that for validate.
>
>
> Thanks,
>
> Purshotam Shah
>
>
Re: Review Request 16840: OOZIE-1668 Coord log streaming start and end time
should be of action list start and end time
Posted by Rohini Palaniswamy <ro...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16840/#review32237
-----------------------------------------------------------
core/src/main/java/org/apache/oozie/CoordinatorEngine.java
<https://reviews.apache.org/r/16840/#comment60966>
You need to get max of endtime iterating through action list instead of just picking the last action's. And if one of them is still RUNNING better to do new Date().
For eg: You can have action 3 ending after action 5 if data was available later or if it was rerun.
- Rohini Palaniswamy
On Jan. 14, 2014, 2:02 a.m., Purshotam Shah wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16840/
> -----------------------------------------------------------
>
> (Updated Jan. 14, 2014, 2:02 a.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-1668
> https://issues.apache.org/jira/browse/OOZIE-1668
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> Coord log streaming start and end time should be of action list start and end time
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/CoordinatorEngine.java 57587c2
> core/src/test/java/org/apache/oozie/TestCoordinatorEngineStreamLog.java a29a07e
>
> Diff: https://reviews.apache.org/r/16840/diff/
>
>
> Testing
> -------
>
> Refactored test-cases.
> Each test cases run a coord job which takes 5 min to complete.
> Made changes so that coord job is executed only once and all test cases use that for validate.
>
>
> Thanks,
>
> Purshotam Shah
>
>
Re: Review Request 16840: OOZIE-1668 Coord log streaming start and end time
should be of action list start and end time
Posted by Ryota Egashira <eg...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16840/#review32189
-----------------------------------------------------------
core/src/test/java/org/apache/oozie/TestCoordinatorEngineStreamLog.java
<https://reviews.apache.org/r/16840/#comment60908>
just minor comment, good to add out-of-sequence action list, like '3-5, 1' to check sorting.
- Ryota Egashira
On Jan. 14, 2014, 2:02 a.m., Purshotam Shah wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16840/
> -----------------------------------------------------------
>
> (Updated Jan. 14, 2014, 2:02 a.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-1668
> https://issues.apache.org/jira/browse/OOZIE-1668
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> Coord log streaming start and end time should be of action list start and end time
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/CoordinatorEngine.java 57587c2
> core/src/test/java/org/apache/oozie/TestCoordinatorEngineStreamLog.java a29a07e
>
> Diff: https://reviews.apache.org/r/16840/diff/
>
>
> Testing
> -------
>
> Refactored test-cases.
> Each test cases run a coord job which takes 5 min to complete.
> Made changes so that coord job is executed only once and all test cases use that for validate.
>
>
> Thanks,
>
> Purshotam Shah
>
>
Re: Review Request 16840: OOZIE-1668 Coord log streaming start and end time
should be of action list start and end time
Posted by Purshotam Shah <pu...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16840/
-----------------------------------------------------------
(Updated Jan. 14, 2014, 2:02 a.m.)
Review request for oozie.
Bugs: OOZIE-1668
https://issues.apache.org/jira/browse/OOZIE-1668
Repository: oozie-git
Description
-------
Coord log streaming start and end time should be of action list start and end time
Diffs
-----
core/src/main/java/org/apache/oozie/CoordinatorEngine.java 57587c2
core/src/test/java/org/apache/oozie/TestCoordinatorEngineStreamLog.java a29a07e
Diff: https://reviews.apache.org/r/16840/diff/
Testing
-------
Refactored test-cases.
Each test cases run a coord job which takes 5 min to complete.
Made changes so that coord job is executed only once and all test cases use that for validate.
Thanks,
Purshotam Shah