You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@oozie.apache.org by rk...@apache.org on 2016/10/21 20:09:31 UTC
oozie git commit: OOZIE-2674 Improve oozie commads documentation
(abhishekbafna via rkanter)
Repository: oozie
Updated Branches:
refs/heads/master 64ea00fc3 -> 1bf91be7b
OOZIE-2674 Improve oozie commads documentation (abhishekbafna via rkanter)
Project: http://git-wip-us.apache.org/repos/asf/oozie/repo
Commit: http://git-wip-us.apache.org/repos/asf/oozie/commit/1bf91be7
Tree: http://git-wip-us.apache.org/repos/asf/oozie/tree/1bf91be7
Diff: http://git-wip-us.apache.org/repos/asf/oozie/diff/1bf91be7
Branch: refs/heads/master
Commit: 1bf91be7b3f824ca4a9fb3e705764574f5dae4da
Parents: 64ea00f
Author: Robert Kanter <rk...@cloudera.com>
Authored: Fri Oct 21 13:08:54 2016 -0700
Committer: Robert Kanter <rk...@cloudera.com>
Committed: Fri Oct 21 13:08:54 2016 -0700
----------------------------------------------------------------------
docs/src/site/twiki/DG_CommandLineTool.twiki | 353 ++++++++++---------
.../src/site/twiki/WorkflowFunctionalSpec.twiki | 2 +-
release-log.txt | 1 +
3 files changed, 198 insertions(+), 158 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/oozie/blob/1bf91be7/docs/src/site/twiki/DG_CommandLineTool.twiki
----------------------------------------------------------------------
diff --git a/docs/src/site/twiki/DG_CommandLineTool.twiki b/docs/src/site/twiki/DG_CommandLineTool.twiki
index 567e260..e7b165c 100644
--- a/docs/src/site/twiki/DG_CommandLineTool.twiki
+++ b/docs/src/site/twiki/DG_CommandLineTool.twiki
@@ -21,168 +21,207 @@ usage:
the env variable 'OOZIE_TIMEZONE' is used as default value for the '-timezone' option
the env variable 'OOZIE_AUTH' is used as default value for the '-auth' option
custom headers for Oozie web services can be specified using '-Dheader:NAME=VALUE'
+</verbatim>
- oozie help : display usage
-.
- oozie version : show client version
-.
- oozie job <OPTIONS> : job operations
- -action <arg> coordinator rerun/kill on action ids (requires -rerun/-kill);
- coordinator log retrieval on action ids (requires -log)
- -allruns Get workflow jobs corresponding to a coordinator action
- including all the reruns
- -auth <arg> select authentication type [SIMPLE|KERBEROS]
- -change <arg> change a coordinator or bundle job
- -config <arg> job configuration file '.xml' or '.properties'
- -configcontent <arg> job configuration
- -coordinator <arg> bundle rerun on coordinator names (requires -rerun)
- -D <property=value> set/override value for given property
- -date <arg> coordinator/bundle rerun on action dates (requires -rerun);
- coordinator log retrieval on action dates (requires -log)
- -debug Use debug mode to see debugging statements on stdout
- -definition <arg> job definition
- -diff <arg> Show diff of the new coord definition and properties with the
- existing one (default true)
- -doas <arg> doAs user, impersonates as the specified user
- -dryrun Dryrun a workflow (since 3.3.2) or coordinator (since 2.0)
- job without actually executing it
- -failed re-runs the failed workflow actions of the coordinator actions (requires -rerun)
- -filter <arg> <key><comparator><value>[;<key><comparator><value>]*
- (All Coordinator actions satisfying the filters will be retrieved).
- key: status or nominaltime
- comparator: =, !=, <, <=, >, >=. = is used as OR and others as AND
- status: values are valid status like SUCCEEDED, KILLED etc. Only = and != apply
- for status.
- nominaltime: time of format yyyy-MM-dd'T'HH:mm'Z'
- -ignore <arg> change status of a coordinator job or action to IGNORED
- (-action required to ignore coord actions)
- -info <arg> info of a job
- -interval <arg> polling interval in minutes (default is 5, requires -poll)
- -kill <arg> kill a job (coordinator can mention -action or -date)
- -len <arg> number of actions (default TOTAL ACTIONS, requires -info)
- -localtime use local time (same as passing your time zone to -timezone).
- Overrides -timezone option
- -log <arg> job log
- -errorlog <arg> job error log
- -auditlog <arg> job audit log
- -logfilter <arg> job log search parameter. Can be specified as -logfilter
- opt1=val1;opt2=val1;opt3=val1. Supported options are recent,
- start, end, loglevel, text, limit and debug
- -nocleanup do not clean up output-events of the coordinator rerun
- actions (requires -rerun)
- -offset <arg> job info offset of actions (default '1', requires -info)
- -oozie <arg> Oozie URL
- -order <arg> order to show coord actions (default ascending order, 'desc'
- for descending order, requires -info)
- -poll <arg> poll Oozie until a job reaches a terminal state or a timeout
- occurs
- -refresh re-materialize the coordinator rerun actions (requires
- -rerun)
- -rerun <arg> rerun a job (coordinator requires -action or -date, bundle
- requires -coordinator or -date)
- -resume <arg> resume a job
- -run run a job
- -start <arg> start a job
- -submit submit a job
- -suspend <arg> suspend a job
- -timeout <arg> timeout in minutes (default is 30, negative values indicate
- no timeout, requires -poll)
- -timezone <arg> use time zone with the specified ID (default GMT).
- See 'oozie info -timezones' for a list
- -update <arg> Update coord definition and properties
- -value <arg> new endtime/concurrency/pausetime value for changing a
- coordinator job
- -verbose verbose mode
- -sladisable disables sla alerts for the job and its children
- -slaenable enables sla alerts for the job and its children
- -slachange Update sla param for jobs, supported param are should-start, should-end and max-duration
-
- oozie jobs <OPTIONS> : jobs status
- -auth <arg> select authentication type [SIMPLE|KERBEROS]
- -doas <arg> doAs user, impersonates as the specified user.
- -filter <arg> user=<U>\;name=<N>\;group=<G>\;status=<S>\;frequency=<F>\;unit=<M>\;startcreatedtime=<SC>\;
- endcreatedtime=<EC>\;sortby=<SB>
- -jobtype <arg> job type ('Supported in Oozie-2.0 or later versions ONLY - coordinator' or 'wf' (default))
- -len <arg> number of jobs (default '100')
- -localtime use local time (same as passing your time zone to -timezone). Overrides -timezone option
- -offset <arg> jobs offset (default '1')
- -oozie <arg> Oozie URL
- -timezone <arg> use time zone with the specified ID (default GMT). See 'oozie info -timezones' for a list
- -kill kill all jobs that satisfy the filter, len, offset, or/and jobtype options. If it's used without
- other options, it will kill all the first 50 workflow jobs. Command will fail if one or more
- of the jobs is in wrong state.
- -suspend suspend all jobs that satisfy the filter, len, offset, or/and jobtype options. If it's used without
- other options, it will suspend all the first 50 workflow jobs. Command will fail if one or more
- of the jobs is in wrong state.
- -resume resume all jobs that satisfy the filter, len, offset, or/and jobtype options. If it's used without
- other options, it will resume all the first 50 workflow jobs. Command will fail if one or more
- of the jobs is in wrong state.
- -verbose verbose mode
-.
- oozie admin <OPTIONS> : admin operations
- -auth <arg> select authentication type [SIMPLE|KERBEROS]
- -configuration show Oozie system configuration
- -doas <arg> doAs user, impersonates as the specified user
- -instrumentation show Oozie system instrumentation
- -javasysprops show Oozie Java system properties
- -metrics show Oozie system metrics
- -oozie <arg> Oozie URL
- -osenv show Oozie system OS environment
- -queuedump show Oozie server queue elements
- -servers list available Oozie servers (more than one only if HA is enabled)
- -shareliblist List available sharelib that can be specified in a workflow action
- -sharelibupdate Update server to use a newer version of sharelib
- -status show the current system status
- -systemmode <arg> Supported in Oozie-2.0 or later versions ONLY. Change oozie
- system mode [NORMAL|NOWEBSERVICE|SAFEMODE]
- -version show Oozie server build version
-.
- oozie validate <OPTIONS> <ARGS> : validate a workflow, coordinator, bundle XML file
- -auth <arg> select authentication type [SIMPLE|KERBEROS]
- -oozie <arg> Oozie URL
-.
- oozie sla <OPTIONS> : sla operations (Deprecated as of Oozie 4.0)
- -auth <arg> select authentication type [SIMPLE|KERBEROS]
- -len <arg> number of results (default '100', max limited by oozie server setting which defaults to '1000')
- -offset <arg> start offset (default '0')
- -oozie <arg> Oozie URL
- -filter <arg> jobid=<JobID/ActionID>\;appname=<Application Name>
-.
- oozie pig <OPTIONS> -X <ARGS> : submit a pig job, everything after '-X' are pass-through parameters to pig, any '-D' arguments after '-X' are put in <configuration>
+---+++ Oozie basic commands
+<verbatim>
+
+oozie help : display usage
+
+oozie version : show client version
+
+</verbatim>
+
+---+++ Oozie job operation commands
+<verbatim>
+
+oozie job <OPTIONS> : job operations
+ -action <arg> coordinator rerun/kill on action ids (requires -rerun/-kill);
+ coordinator log retrieval on action ids (requires -log)
+ -allruns Get workflow jobs corresponding to a coordinator action
+ including all the reruns
+ -auth <arg> select authentication type [SIMPLE|KERBEROS]
+ -change <arg> change a coordinator or bundle job
+ -config <arg> job configuration file '.xml' or '.properties'
+ -configcontent <arg> job configuration
+ -coordinator <arg> bundle rerun on coordinator names (requires -rerun)
+ -D <property=value> set/override value for given property
+ -date <arg> coordinator/bundle rerun on action dates (requires -rerun);
+ coordinator log retrieval on action dates (requires -log)
+ -debug Use debug mode to see debugging statements on stdout
+ -definition <arg> job definition
+ -diff <arg> Show diff of the new coord definition and properties with the
+ existing one (default true)
+ -doas <arg> doAs user, impersonates as the specified user
+ -dryrun Dryrun a workflow (since 3.3.2) or coordinator (since 2.0)
+ job without actually executing it
+ -failed re-runs the failed workflow actions of the coordinator actions (requires -rerun)
+ -filter <arg> <key><comparator><value>[;<key><comparator><value>]*
+ (All Coordinator actions satisfying the filters will be retrieved).
+ key: status or nominaltime
+ comparator: =, !=, <, <=, >, >=. = is used as OR and others as AND
+ status: values are valid status like SUCCEEDED, KILLED etc. Only = and != apply
+ for status.
+ nominaltime: time of format yyyy-MM-dd'T'HH:mm'Z'
+ -ignore <arg> change status of a coordinator job or action to IGNORED
+ (-action required to ignore coord actions)
+ -info <arg> info of a job
+ -interval <arg> polling interval in minutes (default is 5, requires -poll)
+ -kill <arg> kill a job (coordinator can mention -action or -date)
+ -len <arg> number of actions (default TOTAL ACTIONS, requires -info)
+ -localtime use local time (same as passing your time zone to -timezone).
+ Overrides -timezone option
+ -log <arg> job log
+ -errorlog <arg> job error log
+ -auditlog <arg> job audit log
+ -logfilter <arg> job log search parameter. Can be specified as -logfilter
+ opt1=val1;opt2=val1;opt3=val1. Supported options are recent,
+ start, end, loglevel, text, limit and debug
+ -nocleanup do not clean up output-events of the coordinator rerun
+ actions (requires -rerun)
+ -offset <arg> job info offset of actions (default '1', requires -info)
+ -oozie <arg> Oozie URL
+ -order <arg> order to show coord actions (default ascending order, 'desc'
+ for descending order, requires -info)
+ -poll <arg> poll Oozie until a job reaches a terminal state or a timeout
+ occurs
+ -refresh re-materialize the coordinator rerun actions (requires
+ -rerun)
+ -rerun <arg> rerun a job (coordinator requires -action or -date, bundle
+ requires -coordinator or -date)
+ -resume <arg> resume a job
+ -run run a job
+ -start <arg> start a job
+ -submit submit a job
+ -suspend <arg> suspend a job
+ -timeout <arg> timeout in minutes (default is 30, negative values indicate
+ no timeout, requires -poll)
+ -timezone <arg> use time zone with the specified ID (default GMT).
+ See 'oozie info -timezones' for a list
+ -update <arg> Update coord definition and properties
+ -value <arg> new endtime/concurrency/pausetime value for changing a
+ coordinator job
+ -verbose verbose mode
+ -sladisable disables sla alerts for the job and its children
+ -slaenable enables sla alerts for the job and its children
+ -slachange Update sla param for jobs, supported param are should-start, should-end and max-duration
+
+</verbatim>
+
+---+++ Oozie jobs operation commands
+<verbatim>
+oozie jobs <OPTIONS> : jobs status
+ -auth <arg> select authentication type [SIMPLE|KERBEROS]
+ -doas <arg> doAs user, impersonates as the specified user.
+ -filter <arg> user=<U>\;name=<N>\;group=<G>\;status=<S>\;frequency=<F>\;unit=<M>\;startcreatedtime=<SC>\;
+ endcreatedtime=<EC>\;sortby=<SB>
+ -jobtype <arg> job type ('Supported in Oozie-2.0 or later versions ONLY - coordinator' or 'wf' (default))
+ -len <arg> number of jobs (default '100')
+ -localtime use local time (same as passing your time zone to -timezone). Overrides -timezone option
+ -offset <arg> jobs offset (default '1')
+ -oozie <arg> Oozie URL
+ -timezone <arg> use time zone with the specified ID (default GMT). See 'oozie info -timezones' for a list
+ -kill kill all jobs that satisfy the filter, len, offset, or/and jobtype options. If it's used without
+ other options, it will kill all the first 50 workflow jobs. Command will fail if one or more
+ of the jobs is in wrong state.
+ -suspend suspend all jobs that satisfy the filter, len, offset, or/and jobtype options. If it's used without
+ other options, it will suspend all the first 50 workflow jobs. Command will fail if one or more
+ of the jobs is in wrong state.
+ -resume resume all jobs that satisfy the filter, len, offset, or/and jobtype options. If it's used without
+ other options, it will resume all the first 50 workflow jobs. Command will fail if one or more
+ of the jobs is in wrong state.
+ -verbose verbose mode
+</verbatim>
+
+---+++ Oozie admin operation commands
+<verbatim>
+oozie admin <OPTIONS> : admin operations
+ -auth <arg> select authentication type [SIMPLE|KERBEROS]
+ -configuration show Oozie system configuration
+ -doas <arg> doAs user, impersonates as the specified user
+ -instrumentation show Oozie system instrumentation
+ -javasysprops show Oozie Java system properties
+ -metrics show Oozie system metrics
+ -oozie <arg> Oozie URL
+ -osenv show Oozie system OS environment
+ -queuedump show Oozie server queue elements
+ -servers list available Oozie servers (more than one only if HA is enabled)
+ -shareliblist List available sharelib that can be specified in a workflow action
+ -sharelibupdate Update server to use a newer version of sharelib
+ -status show the current system status
+ -systemmode <arg> Supported in Oozie-2.0 or later versions ONLY. Change oozie
+ system mode [NORMAL|NOWEBSERVICE|SAFEMODE]
+ -version show Oozie server build version
+</verbatim>
+
+---+++ Oozie validate command
+<verbatim>
+oozie validate <OPTIONS> <ARGS> : validate a workflow, coordinator, bundle XML file
+ -auth <arg> select authentication type [SIMPLE|KERBEROS]
+ -oozie <arg> Oozie URL
+</verbatim>
+
+---+++ Oozie SLA operation commands
+<verbatim>
+oozie sla <OPTIONS> : sla operations (Deprecated as of Oozie 4.0)
+ -auth <arg> select authentication type [SIMPLE|KERBEROS]
+ -len <arg> number of results (default '100', max limited by oozie server setting which defaults to '1000')
+ -offset <arg> start offset (default '0')
+ -oozie <arg> Oozie URL
+ -filter <arg> jobid=<JobID/ActionID>\;appname=<Application Name>
+</verbatim>
+
+---+++ Oozie Pig submit command
+<verbatim>
+oozie pig <OPTIONS> -X <ARGS> : submit a pig job, everything after '-X' are pass-through parameters to pig, any '-D' arguments
+ after '-X' are put in <configuration>
+ -auth <arg> select authentication type [SIMPLE|KERBEROS]
+ -doas <arg> doAs user, impersonates as the specified user.
+ -config <arg> job configuration file '.properties'
+ -D <property=value> set/override value for given property
+ -file <arg> Pig script
+ -oozie <arg> Oozie URL
+ -P <property=value> set parameters for script
+</verbatim>
+
+---+++ Oozie Hive submit command
+<verbatim>
+oozie hive <OPTIONS> -X<ARGS> : submit a hive job, everything after '-X' are pass-through parameters to hive, any '-D' arguments
+ after '-X' are put in <configuration>
+ -auth <arg> select authentication type [SIMPLE|KERBEROS]
+ -config <arg> job configuration file '.properties'
+ -D <property=value> set/override value for given property
+ -doas <arg> doAs user, impersonates as the specified user
+ -file <arg> hive script
+ -oozie <arg> Oozie URL
+ -P <property=value> set parameters for script
+</verbatim>
+
+---+++ Oozie Sqoop submit command
+<verbatim>
+oozie sqoop <OPTIONS> -X<ARGS> : submit a sqoop job, any '-D' arguments after '-X' are put in <configuration>
+ -auth <arg> select authentication type [SIMPLE|KERBEROS]
+ -config <arg> job configuration file '.properties'
+ -D <property=value> set/override value for given property
+ -doas <arg> doAs user, impersonates as the specified user
+ -command <arg> sqoop command
+ -oozie <arg> Oozie URL
+</verbatim>
+
+---+++ Oozie info command
+<verbatim>
+oozie info <OPTIONS> : get more detailed info about specific topics
+ -timezones display a list of available time zones
+</verbatim>
+
+---+++ Oozie MapReduce job command
+<verbatim>
+oozie mapreduce <OPTIONS> : submit a mapreduce job
-auth <arg> select authentication type [SIMPLE|KERBEROS]
- -doas <arg> doAs user, impersonates as the specified user.
-config <arg> job configuration file '.properties'
-D <property=value> set/override value for given property
- -file <arg> Pig script
+ -doas <arg> doAs user, impersonates as the specified user
-oozie <arg> Oozie URL
- -P <property=value> set parameters for script
-.
- oozie hive <OPTIONS> -X<ARGS> : submit a hive job, everything after '-X' are pass-through parameters to hive, any '-D' arguments after '-X' are put in <configuration>
- -auth <arg> select authentication type [SIMPLE|KERBEROS]
- -config <arg> job configuration file '.properties'
- -D <property=value> set/override value for given property
- -doas <arg> doAs user, impersonates as the specified user
- -file <arg> hive script
- -oozie <arg> Oozie URL
- -P <property=value> set parameters for script
-.
- oozie sqoop <OPTIONS> -X<ARGS> : submit a sqoop job, any '-D' arguments after '-X' are put in <configuration>
- -auth <arg> select authentication type [SIMPLE|KERBEROS]
- -config <arg> job configuration file '.properties'
- -D <property=value> set/override value for given property
- -doas <arg> doAs user, impersonates as the specified user
- -command <arg> sqoop command
- -oozie <arg> Oozie URL
-.
- oozie info <OPTIONS> : get more detailed info about specific topics
- -timezones display a list of available time zones
-.
- oozie mapreduce <OPTIONS> : submit a mapreduce job
- -auth <arg> select authentication type [SIMPLE|KERBEROS]
- -config <arg> job configuration file '.properties'
- -D <property=value> set/override value for given property
- -doas <arg> doAs user, impersonates as the specified user
- -oozie <arg> Oozie URL
</verbatim>
---++ Common CLI Options
http://git-wip-us.apache.org/repos/asf/oozie/blob/1bf91be7/docs/src/site/twiki/WorkflowFunctionalSpec.twiki
----------------------------------------------------------------------
diff --git a/docs/src/site/twiki/WorkflowFunctionalSpec.twiki b/docs/src/site/twiki/WorkflowFunctionalSpec.twiki
index 7474ea5..b7eaf5e 100644
--- a/docs/src/site/twiki/WorkflowFunctionalSpec.twiki
+++ b/docs/src/site/twiki/WorkflowFunctionalSpec.twiki
@@ -213,7 +213,7 @@ The workflow definition language is XML based and it is called hPDL (Hadoop Proc
Refer to the Appendix A for the[[WorkflowFunctionalSpec#OozieWFSchema][Oozie Workflow Definition XML Schema]]. Appendix
B has [[WorkflowFunctionalSpec#OozieWFExamples][Workflow Definition Examples]].
----++ 2.1 Cycles in Workflow Definitions
+---+++ 2.1 Cycles in Workflow Definitions
Oozie does not support cycles in workflow definitions, workflow definitions must be a strict DAG.
http://git-wip-us.apache.org/repos/asf/oozie/blob/1bf91be7/release-log.txt
----------------------------------------------------------------------
diff --git a/release-log.txt b/release-log.txt
index d4dbb74..77cc5ad 100644
--- a/release-log.txt
+++ b/release-log.txt
@@ -6,6 +6,7 @@ OOZIE-2634 Queue dump command message is confusing when the queue is empty (andr
-- Oozie 4.3.0 release
+OOZIE-2674 Improve oozie commads documentation (abhishekbafna via rkanter)
OOZIE-2710 Oozie HCatalog example workflow fails (abhishekbafna via shwethags)
OOZIE-2705 Oozie Spark action ignores spark.executor.extraJavaOptions and spark.driver.extraJavaOptions (gezapeti via rkanter)
OOZIE-2621 Use hive-exec-<version>-core instead of hive-exec in oozie-core (gezapeti via rkanter)