You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by "Craig Peters (Created) (JIRA)" <ji...@apache.org> on 2011/11/23 20:15:40 UTC
[jira] [Created] (OOZIE-614) Oozie does not behave as expected when
using coordinator execution order LAST_ONLY
Oozie does not behave as expected when using coordinator execution order LAST_ONLY
----------------------------------------------------------------------------------
Key: OOZIE-614
URL: https://issues.apache.org/jira/browse/OOZIE-614
Project: Oozie
Issue Type: Bug
Reporter: Craig Peters
After executing the last coordinator action on a queue for a job, the prior coordinator actions will still be executed if a later coordinator action is not created. The behavior expected based upon the documentation for Coordinator is that the older coordinator actions are discarded. See: http://yahoo.github.com/oozie/releases/3.1.0/CoordinatorFunctionalSpec.html#a6.3._Synchronous_Coordinator_Application_Definition
When using the LAST_ONLY execution order the coordinator job needs to somehow discard the older coordinator actions as the documentation describes. This may result the desired behavior for many users when in steady state operation.
Using a simple approach of invalidating or killing prior coordinator actions there are likely to be cases when this functionality won't behave as expected because at any given time the latest coordinator on the queue in the READY state will not be the latest potential. This is because the newer actions haven't been created yet due to the batch creation of coordinator actions oldest first. Some discussion of the best way to address this challenge is warranted I believe.
Another consideration is that the user will not be able to easily discriminate between coordinator actions that were discarded (or skipped) due to the execution order and those that were killed for any other reason. Perhaps it makes sense to introduce a new state? It could be DISCARDED to match the current documentation or perhaps SKIPPED.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-614) Oozie does not behave as expected
when using coordinator execution order LAST_ONLY
Posted by "Maxime Petazzoni (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13236826#comment-13236826 ]
Maxime Petazzoni commented on OOZIE-614:
----------------------------------------
I think this is very confusing. What the documentation leads to believe, which is what I think would be the most obvious and simplest behavior, would be to have LAST_ONLY do the following: regardless of the start time defined, the first materialization of the coordinator happens whenever the defined frequency next hits. No other previous materializations should be created, even in a DISCARDED state or whatever.
> Oozie does not behave as expected when using coordinator execution order LAST_ONLY
> ----------------------------------------------------------------------------------
>
> Key: OOZIE-614
> URL: https://issues.apache.org/jira/browse/OOZIE-614
> Project: Oozie
> Issue Type: Bug
> Reporter: Craig Peters
>
> After executing the last coordinator action on a queue for a job, the prior coordinator actions will still be executed if a later coordinator action is not created. The behavior expected based upon the documentation for Coordinator is that the older coordinator actions are discarded. See: http://yahoo.github.com/oozie/releases/3.1.0/CoordinatorFunctionalSpec.html#a6.3._Synchronous_Coordinator_Application_Definition
> When using the LAST_ONLY execution order the coordinator job needs to somehow discard the older coordinator actions as the documentation describes. This may result the desired behavior for many users when in steady state operation.
> Using a simple approach of invalidating or killing prior coordinator actions there are likely to be cases when this functionality won't behave as expected because at any given time the latest coordinator on the queue in the READY state will not be the latest potential. This is because the newer actions haven't been created yet due to the batch creation of coordinator actions oldest first. Some discussion of the best way to address this challenge is warranted I believe.
> Another consideration is that the user will not be able to easily discriminate between coordinator actions that were discarded (or skipped) due to the execution order and those that were killed for any other reason. Perhaps it makes sense to introduce a new state? It could be DISCARDED to match the current documentation or perhaps SKIPPED.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (OOZIE-614) Oozie does not behave as expected
when using coordinator execution order LAST_ONLY
Posted by "Ryota Egashira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ryota Egashira reassigned OOZIE-614:
------------------------------------
Assignee: Ryota Egashira
> Oozie does not behave as expected when using coordinator execution order LAST_ONLY
> ----------------------------------------------------------------------------------
>
> Key: OOZIE-614
> URL: https://issues.apache.org/jira/browse/OOZIE-614
> Project: Oozie
> Issue Type: Bug
> Reporter: Craig Peters
> Assignee: Ryota Egashira
>
> After executing the last coordinator action on a queue for a job, the prior coordinator actions will still be executed if a later coordinator action is not created. The behavior expected based upon the documentation for Coordinator is that the older coordinator actions are discarded. See: http://yahoo.github.com/oozie/releases/3.1.0/CoordinatorFunctionalSpec.html#a6.3._Synchronous_Coordinator_Application_Definition
> When using the LAST_ONLY execution order the coordinator job needs to somehow discard the older coordinator actions as the documentation describes. This may result the desired behavior for many users when in steady state operation.
> Using a simple approach of invalidating or killing prior coordinator actions there are likely to be cases when this functionality won't behave as expected because at any given time the latest coordinator on the queue in the READY state will not be the latest potential. This is because the newer actions haven't been created yet due to the batch creation of coordinator actions oldest first. Some discussion of the best way to address this challenge is warranted I believe.
> Another consideration is that the user will not be able to easily discriminate between coordinator actions that were discarded (or skipped) due to the execution order and those that were killed for any other reason. Perhaps it makes sense to introduce a new state? It could be DISCARDED to match the current documentation or perhaps SKIPPED.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira