You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@oozie.apache.org by "Roman Shaposhnik (JIRA)" <ji...@apache.org> on 2011/09/10 05:37:10 UTC
[jira] [Closed] (OOZIE-89) GH-49: Supporting bundle in oozie
[ https://issues.apache.org/jira/browse/OOZIE-89?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Roman Shaposhnik closed OOZIE-89.
---------------------------------
Resolution: Fixed
> GH-49: Supporting bundle in oozie
> ---------------------------------
>
> Key: OOZIE-89
> URL: https://issues.apache.org/jira/browse/OOZIE-89
> Project: Oozie
> Issue Type: Bug
> Reporter: Hadoop QA
>
> Oozie currently has two level of abstractions:
> 1. Workflow that execute DAG of actions.
> 2. Coordinator that executes workflow periodically when the specified set of data directories are available.
> This issue proposes another abstraction called 'bundle' that will batch a set of coordinator applications. The user will be able to start/stop/suspend/resume/rerun in the bundle level.
> ******* The proposed high-level requirements to support bundle are enumerated below:
> 1. This feature will allow user to specify a list of coordinator applications in XML file format.
> 2. The name of the bundle xml file is not hard-coded. User can specify any name as bundle file.
> 3. User will submit a bundle by specifying the bundle application path in config file . An example command is: oozie job -run -config <bundle.properties>
> 4. Bundle application path is defined in config file as property "oozie.application.bundle.path" with a value of full path to bundle xml in the hdfs.
> 5. User can also submit a bundle job through WS API.
> 7. User will be able to define variables /parameters for each coordinator application.
> 8. All variables should be resolved during job submission. For any resolved variable, oozie will throw an Exception.
> 9. User will be able to submit a bundle with an user-defined external id to avoid duplicate submissions in case of Timeout in first submission.
> 10. Oozie will not support any explicit dependencies among the coordinator XML in bundle definition.
> 11. Oozie will not support any partial bundle submission.
> 12. When user will submit a bundle , it will get a bundle id to track. Oozie will put the bundle job into PREP state.
> 13. User will be able to start a bundle using bundle id. It will put the bundle job into RUNNING state.
> 14. User will be able to combine submit and start into run that will start the bundle immediately.
> 15. User will be able to optionally specify the kick-off time to determine when to start a bundle. The bundle will not run until kick-off time reached.
> 16. User will be able to query Oozie for its status through CLI and WS API.
> 17. User will be able to query Oozie for all coordinator jobs that it started through CLI and WS API.
> 18. User will be able to kill a bundle id that will kill all spawned coordinator jobs.
> 19. User will be able to suspend a bundle id that will suspend all spawned coordinator jobs.
> 20. User will be able to pause a bundle id with a future time that will pause all spawned coordinator jobs.
> 21. User will be able to resume a bundle id that will resume all spawned coordinator jobs.
> 22. Bundle rerun requirements TBD.
> This is a sample bundle XML :
> =========================
> <bundle-app name="MY_BUNDLE" xmlns="uri:oozie:bundle:0.1">
>
> <controls>
> <kick-off-time>2009-02-02T00:00Z</kick-off-time>
> </controls>
> <coordinator>
> <configuration>
> <property>
> <name>START_TIME</name>
> <value>2009-02-01T00:00Z</value>
> <property>
> .................
> ...............
> </configuration>
> <app-path>hdfs:${NAME_NODE}/tmp/bundle-apps/coordinator1.xml</app-path>
> <coordinator>
> <coordinator>
> <configuration>
> <property>
> <name>END_TIME</name>
> <value>2010-02-01T00:00Z</value>
> <property>
> .................
> ...............
> </configuration>
> <app-path>hdfs:${NAME_NODE}/tmp/bundle-apps/coordinator1.xml</app-path>
> <coordinator>
> </bundle-app>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira