You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by Robert Kanter <rk...@cloudera.com> on 2015/11/10 23:36:43 UTC
Review Request 40157: OOZIE-2397: LAST_ONLY and NONE don't properly
handle READY actions
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40157/
-----------------------------------------------------------
Review request for oozie.
Bugs: OOZIE-2397
https://issues.apache.org/jira/browse/OOZIE-2397
Repository: oozie-git
Description
-------
https://issues.apache.org/jira/browse/OOZIE-2397
Diffs
-----
core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 3ec2eaf
core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java 179dfbf
core/src/main/java/org/apache/oozie/command/coord/CoordActionReadyXCommand.java ad9cb12
core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java 131368a
core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetReadyActionsJPAExecutor.java af81091
core/src/main/java/org/apache/oozie/executor/jpa/CoordJobQueryExecutor.java 45d2fc9
core/src/test/java/org/apache/oozie/command/coord/TestCoordActionInputCheckXCommand.java f78dafd
core/src/test/java/org/apache/oozie/command/coord/TestCoordActionReadyXCommand.java PRE-CREATION
core/src/test/java/org/apache/oozie/command/coord/TestCoordCommandUtils.java 652d792
core/src/test/java/org/apache/oozie/command/coord/TestCoordMaterializeTransitionXCommand.java 77f5518
core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobGetReadyActionsJPAExecutor.java c666b87
core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobQueryExecutor.java ee7ad6a
core/src/test/java/org/apache/oozie/test/XDataTestCase.java e6a7d9c
docs/src/site/twiki/CoordinatorFunctionalSpec.twiki faff37f
Diff: https://reviews.apache.org/r/40157/diff/
Testing
-------
- Ran relevant unit tests
- Verified in a cluster that it now has the correct behavior. I had a Coordinator with a concurrency of 1 and Coord Actions that took longer than the frequency. This way, we'd have READY actions that should transition to SKIPPED once the RUNNING action finished.
Thanks,
Robert Kanter
Re: Review Request 40157: OOZIE-2397: LAST_ONLY and NONE don't
properly handle READY actions
Posted by Purshotam Shah <pu...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40157/#review107916
-----------------------------------------------------------
Ship it!
Ship It!
- Purshotam Shah
On Nov. 25, 2015, 12:47 a.m., Robert Kanter wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40157/
> -----------------------------------------------------------
>
> (Updated Nov. 25, 2015, 12:47 a.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-2397
> https://issues.apache.org/jira/browse/OOZIE-2397
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/OOZIE-2397
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/CoordinatorActionBean.java c3d4bb4
> core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 3ec2eaf
> core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java 179dfbf
> core/src/main/java/org/apache/oozie/command/coord/CoordActionReadyXCommand.java ad9cb12
> core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java 131368a
> core/src/main/java/org/apache/oozie/executor/jpa/CoordActionGetForInputCheckJPAExecutor.java e58646b
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetReadyActionsJPAExecutor.java af81091
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobQueryExecutor.java 45d2fc9
> core/src/test/java/org/apache/oozie/command/coord/TestCoordActionInputCheckXCommand.java f78dafd
> core/src/test/java/org/apache/oozie/command/coord/TestCoordActionReadyXCommand.java PRE-CREATION
> core/src/test/java/org/apache/oozie/command/coord/TestCoordCommandUtils.java 652d792
> core/src/test/java/org/apache/oozie/command/coord/TestCoordMaterializeTransitionXCommand.java 77f5518
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordActionGetForInputCheckJPAExecutor.java ca1d80a
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobGetReadyActionsJPAExecutor.java c666b87
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobQueryExecutor.java ee7ad6a
> core/src/test/java/org/apache/oozie/test/XDataTestCase.java e6a7d9c
> docs/src/site/twiki/CoordinatorFunctionalSpec.twiki faff37f
>
> Diff: https://reviews.apache.org/r/40157/diff/
>
>
> Testing
> -------
>
> - Ran relevant unit tests
> - Verified in a cluster that it now has the correct behavior. I had a Coordinator with a concurrency of 1 and Coord Actions that took longer than the frequency. This way, we'd have READY actions that should transition to SKIPPED once the RUNNING action finished.
>
>
> Thanks,
>
> Robert Kanter
>
>
Re: Review Request 40157: OOZIE-2397: LAST_ONLY and NONE don't
properly handle READY actions
Posted by Robert Kanter <rk...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40157/
-----------------------------------------------------------
(Updated Nov. 25, 2015, 12:47 a.m.)
Review request for oozie.
Changes
-------
Fixes for end_of_duration
Bugs: OOZIE-2397
https://issues.apache.org/jira/browse/OOZIE-2397
Repository: oozie-git
Description
-------
https://issues.apache.org/jira/browse/OOZIE-2397
Diffs (updated)
-----
core/src/main/java/org/apache/oozie/CoordinatorActionBean.java c3d4bb4
core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 3ec2eaf
core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java 179dfbf
core/src/main/java/org/apache/oozie/command/coord/CoordActionReadyXCommand.java ad9cb12
core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java 131368a
core/src/main/java/org/apache/oozie/executor/jpa/CoordActionGetForInputCheckJPAExecutor.java e58646b
core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetReadyActionsJPAExecutor.java af81091
core/src/main/java/org/apache/oozie/executor/jpa/CoordJobQueryExecutor.java 45d2fc9
core/src/test/java/org/apache/oozie/command/coord/TestCoordActionInputCheckXCommand.java f78dafd
core/src/test/java/org/apache/oozie/command/coord/TestCoordActionReadyXCommand.java PRE-CREATION
core/src/test/java/org/apache/oozie/command/coord/TestCoordCommandUtils.java 652d792
core/src/test/java/org/apache/oozie/command/coord/TestCoordMaterializeTransitionXCommand.java 77f5518
core/src/test/java/org/apache/oozie/executor/jpa/TestCoordActionGetForInputCheckJPAExecutor.java ca1d80a
core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobGetReadyActionsJPAExecutor.java c666b87
core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobQueryExecutor.java ee7ad6a
core/src/test/java/org/apache/oozie/test/XDataTestCase.java e6a7d9c
docs/src/site/twiki/CoordinatorFunctionalSpec.twiki faff37f
Diff: https://reviews.apache.org/r/40157/diff/
Testing
-------
- Ran relevant unit tests
- Verified in a cluster that it now has the correct behavior. I had a Coordinator with a concurrency of 1 and Coord Actions that took longer than the frequency. This way, we'd have READY actions that should transition to SKIPPED once the RUNNING action finished.
Thanks,
Robert Kanter
Re: Review Request 40157: OOZIE-2397: LAST_ONLY and NONE don't
properly handle READY actions
Posted by Robert Kanter <rk...@cloudera.com>.
> On Nov. 23, 2015, 8:15 p.m., Purshotam Shah wrote:
> > core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java, line 778
> > <https://reviews.apache.org/r/40157/diff/1-2/?file=1122526#file1122526line778>
> >
> > I guess we missing "end_of_duration". If end_of_duration is set we need to adjust start time.
I don't think I'm familiar with "end_of_duration". What does that do? How do I handle that?
- Robert
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40157/#review107636
-----------------------------------------------------------
On Nov. 19, 2015, 5:41 a.m., Robert Kanter wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40157/
> -----------------------------------------------------------
>
> (Updated Nov. 19, 2015, 5:41 a.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-2397
> https://issues.apache.org/jira/browse/OOZIE-2397
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/OOZIE-2397
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/CoordinatorActionBean.java c3d4bb4
> core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 3ec2eaf
> core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java 179dfbf
> core/src/main/java/org/apache/oozie/command/coord/CoordActionReadyXCommand.java ad9cb12
> core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java 131368a
> core/src/main/java/org/apache/oozie/executor/jpa/CoordActionGetForInputCheckJPAExecutor.java e58646b
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetReadyActionsJPAExecutor.java af81091
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobQueryExecutor.java 45d2fc9
> core/src/test/java/org/apache/oozie/command/coord/TestCoordActionInputCheckXCommand.java f78dafd
> core/src/test/java/org/apache/oozie/command/coord/TestCoordActionReadyXCommand.java PRE-CREATION
> core/src/test/java/org/apache/oozie/command/coord/TestCoordCommandUtils.java 652d792
> core/src/test/java/org/apache/oozie/command/coord/TestCoordMaterializeTransitionXCommand.java 77f5518
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordActionGetForInputCheckJPAExecutor.java ca1d80a
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobGetReadyActionsJPAExecutor.java c666b87
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobQueryExecutor.java ee7ad6a
> core/src/test/java/org/apache/oozie/test/XDataTestCase.java e6a7d9c
> docs/src/site/twiki/CoordinatorFunctionalSpec.twiki faff37f
>
> Diff: https://reviews.apache.org/r/40157/diff/
>
>
> Testing
> -------
>
> - Ran relevant unit tests
> - Verified in a cluster that it now has the correct behavior. I had a Coordinator with a concurrency of 1 and Coord Actions that took longer than the frequency. This way, we'd have READY actions that should transition to SKIPPED once the RUNNING action finished.
>
>
> Thanks,
>
> Robert Kanter
>
>
Re: Review Request 40157: OOZIE-2397: LAST_ONLY and NONE don't
properly handle READY actions
Posted by Robert Kanter <rk...@cloudera.com>.
> On Nov. 23, 2015, 8:15 p.m., Purshotam Shah wrote:
> > core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java, line 778
> > <https://reviews.apache.org/r/40157/diff/1-2/?file=1122526#file1122526line778>
> >
> > I guess we missing "end_of_duration". If end_of_duration is set we need to adjust start time.
>
> Robert Kanter wrote:
> I don't think I'm familiar with "end_of_duration". What does that do? How do I handle that?
>
> Purshotam Shah wrote:
> end_of_duration will be set if one use frquency as coord:endOfDays or coord:endOfMonths.
> Even if start Time=2013-10-12T14:00Z with frequency="${coord:endOfMonths(1)}", 1st nominal time will be 2013-11-01T00:00Z
>
> This can be handled by following code. You can also look at CoordMaterializeTransitionXCommand.
> TimeZone appTz = DateUtils.getTimeZone(coordJob.getTimeZone());
> TimeUnit freqTU = TimeUnit.valueOf(coordJob.getTimeUnitStr());
> TimeUnit endOfFlag = TimeUnit.valueOf(eJob.getAttributeValue("end_of_duration"));
> Calendar start = Calendar.getInstance(appTz);
> start.setTime(startTime);
> DateUtils.moveToEnd(start, endOfFlag);
It's unfortunate that to get the "end_of_duration" we have to get and parse the XML too :(
I'll post an update later today.
- Robert
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40157/#review107636
-----------------------------------------------------------
On Nov. 19, 2015, 5:41 a.m., Robert Kanter wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40157/
> -----------------------------------------------------------
>
> (Updated Nov. 19, 2015, 5:41 a.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-2397
> https://issues.apache.org/jira/browse/OOZIE-2397
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/OOZIE-2397
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/CoordinatorActionBean.java c3d4bb4
> core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 3ec2eaf
> core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java 179dfbf
> core/src/main/java/org/apache/oozie/command/coord/CoordActionReadyXCommand.java ad9cb12
> core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java 131368a
> core/src/main/java/org/apache/oozie/executor/jpa/CoordActionGetForInputCheckJPAExecutor.java e58646b
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetReadyActionsJPAExecutor.java af81091
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobQueryExecutor.java 45d2fc9
> core/src/test/java/org/apache/oozie/command/coord/TestCoordActionInputCheckXCommand.java f78dafd
> core/src/test/java/org/apache/oozie/command/coord/TestCoordActionReadyXCommand.java PRE-CREATION
> core/src/test/java/org/apache/oozie/command/coord/TestCoordCommandUtils.java 652d792
> core/src/test/java/org/apache/oozie/command/coord/TestCoordMaterializeTransitionXCommand.java 77f5518
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordActionGetForInputCheckJPAExecutor.java ca1d80a
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobGetReadyActionsJPAExecutor.java c666b87
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobQueryExecutor.java ee7ad6a
> core/src/test/java/org/apache/oozie/test/XDataTestCase.java e6a7d9c
> docs/src/site/twiki/CoordinatorFunctionalSpec.twiki faff37f
>
> Diff: https://reviews.apache.org/r/40157/diff/
>
>
> Testing
> -------
>
> - Ran relevant unit tests
> - Verified in a cluster that it now has the correct behavior. I had a Coordinator with a concurrency of 1 and Coord Actions that took longer than the frequency. This way, we'd have READY actions that should transition to SKIPPED once the RUNNING action finished.
>
>
> Thanks,
>
> Robert Kanter
>
>
Re: Review Request 40157: OOZIE-2397: LAST_ONLY and NONE don't
properly handle READY actions
Posted by Purshotam Shah <pu...@yahoo-inc.com>.
> On Nov. 23, 2015, 8:15 p.m., Purshotam Shah wrote:
> > core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java, line 778
> > <https://reviews.apache.org/r/40157/diff/1-2/?file=1122526#file1122526line778>
> >
> > I guess we missing "end_of_duration". If end_of_duration is set we need to adjust start time.
>
> Robert Kanter wrote:
> I don't think I'm familiar with "end_of_duration". What does that do? How do I handle that?
end_of_duration will be set if one use frquency as coord:endOfDays or coord:endOfMonths.
Even if start Time=2013-10-12T14:00Z with frequency="${coord:endOfMonths(1)}", 1st nominal time will be 2013-11-01T00:00Z
This can be handled by following code. You can also look at CoordMaterializeTransitionXCommand.
TimeZone appTz = DateUtils.getTimeZone(coordJob.getTimeZone());
TimeUnit freqTU = TimeUnit.valueOf(coordJob.getTimeUnitStr());
TimeUnit endOfFlag = TimeUnit.valueOf(eJob.getAttributeValue("end_of_duration"));
Calendar start = Calendar.getInstance(appTz);
start.setTime(startTime);
DateUtils.moveToEnd(start, endOfFlag);
- Purshotam
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40157/#review107636
-----------------------------------------------------------
On Nov. 19, 2015, 5:41 a.m., Robert Kanter wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40157/
> -----------------------------------------------------------
>
> (Updated Nov. 19, 2015, 5:41 a.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-2397
> https://issues.apache.org/jira/browse/OOZIE-2397
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/OOZIE-2397
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/CoordinatorActionBean.java c3d4bb4
> core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 3ec2eaf
> core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java 179dfbf
> core/src/main/java/org/apache/oozie/command/coord/CoordActionReadyXCommand.java ad9cb12
> core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java 131368a
> core/src/main/java/org/apache/oozie/executor/jpa/CoordActionGetForInputCheckJPAExecutor.java e58646b
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetReadyActionsJPAExecutor.java af81091
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobQueryExecutor.java 45d2fc9
> core/src/test/java/org/apache/oozie/command/coord/TestCoordActionInputCheckXCommand.java f78dafd
> core/src/test/java/org/apache/oozie/command/coord/TestCoordActionReadyXCommand.java PRE-CREATION
> core/src/test/java/org/apache/oozie/command/coord/TestCoordCommandUtils.java 652d792
> core/src/test/java/org/apache/oozie/command/coord/TestCoordMaterializeTransitionXCommand.java 77f5518
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordActionGetForInputCheckJPAExecutor.java ca1d80a
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobGetReadyActionsJPAExecutor.java c666b87
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobQueryExecutor.java ee7ad6a
> core/src/test/java/org/apache/oozie/test/XDataTestCase.java e6a7d9c
> docs/src/site/twiki/CoordinatorFunctionalSpec.twiki faff37f
>
> Diff: https://reviews.apache.org/r/40157/diff/
>
>
> Testing
> -------
>
> - Ran relevant unit tests
> - Verified in a cluster that it now has the correct behavior. I had a Coordinator with a concurrency of 1 and Coord Actions that took longer than the frequency. This way, we'd have READY actions that should transition to SKIPPED once the RUNNING action finished.
>
>
> Thanks,
>
> Robert Kanter
>
>
Re: Review Request 40157: OOZIE-2397: LAST_ONLY and NONE don't
properly handle READY actions
Posted by Purshotam Shah <pu...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40157/#review107636
-----------------------------------------------------------
core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java (line 778)
<https://reviews.apache.org/r/40157/#comment166865>
I guess we missing "end_of_duration". If end_of_duration is set we need to adjust start time.
- Purshotam Shah
On Nov. 19, 2015, 5:41 a.m., Robert Kanter wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40157/
> -----------------------------------------------------------
>
> (Updated Nov. 19, 2015, 5:41 a.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-2397
> https://issues.apache.org/jira/browse/OOZIE-2397
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/OOZIE-2397
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/CoordinatorActionBean.java c3d4bb4
> core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 3ec2eaf
> core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java 179dfbf
> core/src/main/java/org/apache/oozie/command/coord/CoordActionReadyXCommand.java ad9cb12
> core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java 131368a
> core/src/main/java/org/apache/oozie/executor/jpa/CoordActionGetForInputCheckJPAExecutor.java e58646b
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetReadyActionsJPAExecutor.java af81091
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobQueryExecutor.java 45d2fc9
> core/src/test/java/org/apache/oozie/command/coord/TestCoordActionInputCheckXCommand.java f78dafd
> core/src/test/java/org/apache/oozie/command/coord/TestCoordActionReadyXCommand.java PRE-CREATION
> core/src/test/java/org/apache/oozie/command/coord/TestCoordCommandUtils.java 652d792
> core/src/test/java/org/apache/oozie/command/coord/TestCoordMaterializeTransitionXCommand.java 77f5518
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordActionGetForInputCheckJPAExecutor.java ca1d80a
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobGetReadyActionsJPAExecutor.java c666b87
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobQueryExecutor.java ee7ad6a
> core/src/test/java/org/apache/oozie/test/XDataTestCase.java e6a7d9c
> docs/src/site/twiki/CoordinatorFunctionalSpec.twiki faff37f
>
> Diff: https://reviews.apache.org/r/40157/diff/
>
>
> Testing
> -------
>
> - Ran relevant unit tests
> - Verified in a cluster that it now has the correct behavior. I had a Coordinator with a concurrency of 1 and Coord Actions that took longer than the frequency. This way, we'd have READY actions that should transition to SKIPPED once the RUNNING action finished.
>
>
> Thanks,
>
> Robert Kanter
>
>
Re: Review Request 40157: OOZIE-2397: LAST_ONLY and NONE don't
properly handle READY actions
Posted by Robert Kanter <rk...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40157/
-----------------------------------------------------------
(Updated Nov. 19, 2015, 5:41 a.m.)
Review request for oozie.
Changes
-------
The 002 patch changes how the next action nominal time is computed as suggested by Puru. I also updated and cleaned up affected unit tests.
Bugs: OOZIE-2397
https://issues.apache.org/jira/browse/OOZIE-2397
Repository: oozie-git
Description
-------
https://issues.apache.org/jira/browse/OOZIE-2397
Diffs (updated)
-----
core/src/main/java/org/apache/oozie/CoordinatorActionBean.java c3d4bb4
core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 3ec2eaf
core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java 179dfbf
core/src/main/java/org/apache/oozie/command/coord/CoordActionReadyXCommand.java ad9cb12
core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java 131368a
core/src/main/java/org/apache/oozie/executor/jpa/CoordActionGetForInputCheckJPAExecutor.java e58646b
core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetReadyActionsJPAExecutor.java af81091
core/src/main/java/org/apache/oozie/executor/jpa/CoordJobQueryExecutor.java 45d2fc9
core/src/test/java/org/apache/oozie/command/coord/TestCoordActionInputCheckXCommand.java f78dafd
core/src/test/java/org/apache/oozie/command/coord/TestCoordActionReadyXCommand.java PRE-CREATION
core/src/test/java/org/apache/oozie/command/coord/TestCoordCommandUtils.java 652d792
core/src/test/java/org/apache/oozie/command/coord/TestCoordMaterializeTransitionXCommand.java 77f5518
core/src/test/java/org/apache/oozie/executor/jpa/TestCoordActionGetForInputCheckJPAExecutor.java ca1d80a
core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobGetReadyActionsJPAExecutor.java c666b87
core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobQueryExecutor.java ee7ad6a
core/src/test/java/org/apache/oozie/test/XDataTestCase.java e6a7d9c
docs/src/site/twiki/CoordinatorFunctionalSpec.twiki faff37f
Diff: https://reviews.apache.org/r/40157/diff/
Testing
-------
- Ran relevant unit tests
- Verified in a cluster that it now has the correct behavior. I had a Coordinator with a concurrency of 1 and Coord Actions that took longer than the frequency. This way, we'd have READY actions that should transition to SKIPPED once the RUNNING action finished.
Thanks,
Robert Kanter
Re: Review Request 40157: OOZIE-2397: LAST_ONLY and NONE don't
properly handle READY actions
Posted by Robert Kanter <rk...@cloudera.com>.
> On Nov. 17, 2015, 1:04 a.m., Purshotam Shah wrote:
> > core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java, line 778
> > <https://reviews.apache.org/r/40157/diff/1/?file=1122526#file1122526line778>
> >
> > I remember this doesn't work well with diffrent timezone.
> > You might need to compute it from start date
> >
> > Like from CoordMaterializeTransitionXCommand.java
> > .....
> > int intFrequency = Integer.parseInt(coordJob.getFrequency());
> > effStart = (Calendar) origStart.clone();
> > effStart.add(freqTU.getCalendarUnit(), actionNumber + 1 * intFrequency);
> > .....
The action number is indexed by 1, so it ends up just being
````
effStart.add(freqTU.getCalendarUnit(), actionNumber * intFrequency);
````
- Robert
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40157/#review106773
-----------------------------------------------------------
On Nov. 19, 2015, 5:41 a.m., Robert Kanter wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40157/
> -----------------------------------------------------------
>
> (Updated Nov. 19, 2015, 5:41 a.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-2397
> https://issues.apache.org/jira/browse/OOZIE-2397
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/OOZIE-2397
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/CoordinatorActionBean.java c3d4bb4
> core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 3ec2eaf
> core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java 179dfbf
> core/src/main/java/org/apache/oozie/command/coord/CoordActionReadyXCommand.java ad9cb12
> core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java 131368a
> core/src/main/java/org/apache/oozie/executor/jpa/CoordActionGetForInputCheckJPAExecutor.java e58646b
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetReadyActionsJPAExecutor.java af81091
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobQueryExecutor.java 45d2fc9
> core/src/test/java/org/apache/oozie/command/coord/TestCoordActionInputCheckXCommand.java f78dafd
> core/src/test/java/org/apache/oozie/command/coord/TestCoordActionReadyXCommand.java PRE-CREATION
> core/src/test/java/org/apache/oozie/command/coord/TestCoordCommandUtils.java 652d792
> core/src/test/java/org/apache/oozie/command/coord/TestCoordMaterializeTransitionXCommand.java 77f5518
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordActionGetForInputCheckJPAExecutor.java ca1d80a
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobGetReadyActionsJPAExecutor.java c666b87
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobQueryExecutor.java ee7ad6a
> core/src/test/java/org/apache/oozie/test/XDataTestCase.java e6a7d9c
> docs/src/site/twiki/CoordinatorFunctionalSpec.twiki faff37f
>
> Diff: https://reviews.apache.org/r/40157/diff/
>
>
> Testing
> -------
>
> - Ran relevant unit tests
> - Verified in a cluster that it now has the correct behavior. I had a Coordinator with a concurrency of 1 and Coord Actions that took longer than the frequency. This way, we'd have READY actions that should transition to SKIPPED once the RUNNING action finished.
>
>
> Thanks,
>
> Robert Kanter
>
>
Re: Review Request 40157: OOZIE-2397: LAST_ONLY and NONE don't
properly handle READY actions
Posted by Purshotam Shah <pu...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40157/#review106773
-----------------------------------------------------------
core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java (line 778)
<https://reviews.apache.org/r/40157/#comment165546>
I remember this doesn't work well with diffrent timezone.
You might need to compute it from start date
Like from CoordMaterializeTransitionXCommand.java
.....
int intFrequency = Integer.parseInt(coordJob.getFrequency());
effStart = (Calendar) origStart.clone();
effStart.add(freqTU.getCalendarUnit(), actionNumber + 1 * intFrequency);
.....
- Purshotam Shah
On Nov. 10, 2015, 10:36 p.m., Robert Kanter wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40157/
> -----------------------------------------------------------
>
> (Updated Nov. 10, 2015, 10:36 p.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-2397
> https://issues.apache.org/jira/browse/OOZIE-2397
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/OOZIE-2397
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 3ec2eaf
> core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java 179dfbf
> core/src/main/java/org/apache/oozie/command/coord/CoordActionReadyXCommand.java ad9cb12
> core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java 131368a
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetReadyActionsJPAExecutor.java af81091
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobQueryExecutor.java 45d2fc9
> core/src/test/java/org/apache/oozie/command/coord/TestCoordActionInputCheckXCommand.java f78dafd
> core/src/test/java/org/apache/oozie/command/coord/TestCoordActionReadyXCommand.java PRE-CREATION
> core/src/test/java/org/apache/oozie/command/coord/TestCoordCommandUtils.java 652d792
> core/src/test/java/org/apache/oozie/command/coord/TestCoordMaterializeTransitionXCommand.java 77f5518
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobGetReadyActionsJPAExecutor.java c666b87
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobQueryExecutor.java ee7ad6a
> core/src/test/java/org/apache/oozie/test/XDataTestCase.java e6a7d9c
> docs/src/site/twiki/CoordinatorFunctionalSpec.twiki faff37f
>
> Diff: https://reviews.apache.org/r/40157/diff/
>
>
> Testing
> -------
>
> - Ran relevant unit tests
> - Verified in a cluster that it now has the correct behavior. I had a Coordinator with a concurrency of 1 and Coord Actions that took longer than the frequency. This way, we'd have READY actions that should transition to SKIPPED once the RUNNING action finished.
>
>
> Thanks,
>
> Robert Kanter
>
>