You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by "Alejandro Abdelnur (Created) (JIRA)" <ji...@apache.org> on 2011/11/02 20:05:32 UTC

[jira] [Created] (OOZIE-592) new WF library implementation (lite2)

new WF library implementation (lite2)
-------------------------------------

                 Key: OOZIE-592
                 URL: https://issues.apache.org/jira/browse/OOZIE-592
             Project: Oozie
          Issue Type: Improvement
            Reporter: Alejandro Abdelnur
            Assignee: Alejandro Abdelnur


Current WF library implementation (lite) has the following issues/limitations:

* Under certain conditions kill/failure all running actions are not being properly killed (leaving things in odd states and/or runaway actions)
* Exceptions are properly propagated handled
* start/end/kill/fork/join nodes do not  provide hooks to create entries in the DB in the WF actions table (useful for WF visualization)

In addition, the following new features would allow more flexible WF programming:

* support for a SUSPEND node, to programmatically suspend a WF in certain condition.
* support for SHARD/UNSHARD nodes, similar to FORK/JOIN but it is a single execution lane that gets multiplexed N times

These 2 new features would be exposed in a new schema.

The implementation approach for this would be broken down into a few JIRAs:

* WF lite2 implementation with testcases
* WF lite2 node handlers
* WF lite2 service




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (OOZIE-592) new WF library implementation (lite2)

Posted by "Alejandro Abdelnur (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/OOZIE-592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13142422#comment-13142422 ] 

Alejandro Abdelnur commented on OOZIE-592:
------------------------------------------

In addition, WF state would be serialized as JSON data in the DB instead of Writable
                
> new WF library implementation (lite2)
> -------------------------------------
>
>                 Key: OOZIE-592
>                 URL: https://issues.apache.org/jira/browse/OOZIE-592
>             Project: Oozie
>          Issue Type: Improvement
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>
> Current WF library implementation (lite) has the following issues/limitations:
> * Under certain conditions kill/failure all running actions are not being properly killed (leaving things in odd states and/or runaway actions)
> * Exceptions are properly propagated handled
> * start/end/kill/fork/join nodes do not  provide hooks to create entries in the DB in the WF actions table (useful for WF visualization)
> In addition, the following new features would allow more flexible WF programming:
> * support for a SUSPEND node, to programmatically suspend a WF in certain condition.
> * support for SHARD/UNSHARD nodes, similar to FORK/JOIN but it is a single execution lane that gets multiplexed N times
> These 2 new features would be exposed in a new schema.
> The implementation approach for this would be broken down into a few JIRAs:
> * WF lite2 implementation with testcases
> * WF lite2 node handlers
> * WF lite2 service

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (OOZIE-592) new WF library implementation (lite2)

Posted by "Mohammad Kamrul Islam (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/OOZIE-592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13142741#comment-13142741 ] 

Mohammad Kamrul Islam commented on OOZIE-592:
---------------------------------------------

As part of storing and displaying the pig stats and job counter(#OOZIE-588), we might need to modify the WFInstance.
Therefore, we want to know your plan of when the change might be available. We don't want to modify the same thing that would create possible  merging issue.


                
> new WF library implementation (lite2)
> -------------------------------------
>
>                 Key: OOZIE-592
>                 URL: https://issues.apache.org/jira/browse/OOZIE-592
>             Project: Oozie
>          Issue Type: Improvement
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>
> Current WF library implementation (lite) has the following issues/limitations:
> * Under certain conditions kill/failure all running actions are not being properly killed (leaving things in odd states and/or runaway actions)
> * Exceptions are properly propagated handled
> * start/end/kill/fork/join nodes do not  provide hooks to create entries in the DB in the WF actions table (useful for WF visualization)
> In addition, the following new features would allow more flexible WF programming:
> * support for a SUSPEND node, to programmatically suspend a WF in certain condition.
> * support for SHARD/UNSHARD nodes, similar to FORK/JOIN but it is a single execution lane that gets multiplexed N times
> These 2 new features would be exposed in a new schema.
> The implementation approach for this would be broken down into a few JIRAs:
> * WF lite2 implementation with testcases
> * WF lite2 node handlers
> * WF lite2 service

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (OOZIE-592) new WF library implementation (lite2)

Posted by "Mohammad Kamrul Islam (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/OOZIE-592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13142742#comment-13142742 ] 

Mohammad Kamrul Islam commented on OOZIE-592:
---------------------------------------------

I like this JSON object concept. Do we think we need to keep the current versioning concept to avoid both forward and backward compatibility?
 
                
> new WF library implementation (lite2)
> -------------------------------------
>
>                 Key: OOZIE-592
>                 URL: https://issues.apache.org/jira/browse/OOZIE-592
>             Project: Oozie
>          Issue Type: Improvement
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>
> Current WF library implementation (lite) has the following issues/limitations:
> * Under certain conditions kill/failure all running actions are not being properly killed (leaving things in odd states and/or runaway actions)
> * Exceptions are properly propagated handled
> * start/end/kill/fork/join nodes do not  provide hooks to create entries in the DB in the WF actions table (useful for WF visualization)
> In addition, the following new features would allow more flexible WF programming:
> * support for a SUSPEND node, to programmatically suspend a WF in certain condition.
> * support for SHARD/UNSHARD nodes, similar to FORK/JOIN but it is a single execution lane that gets multiplexed N times
> These 2 new features would be exposed in a new schema.
> The implementation approach for this would be broken down into a few JIRAs:
> * WF lite2 implementation with testcases
> * WF lite2 node handlers
> * WF lite2 service

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira