You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@uima.apache.org by ch...@apache.org on 2013/06/11 23:11:05 UTC
svn commit: r1491963 - in
/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli:
ducc-process-submit.tex ducc-service-cancel.tex ducc-service-submit.tex
ducc-submit.tex
Author: challngr
Date: Tue Jun 11 21:11:05 2013
New Revision: 1491963
URL: http://svn.apache.org/r1491963
Log:
UIMA-2682 Duccbook updates.
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-process-submit.tex
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-service-cancel.tex
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-service-submit.tex
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-submit.tex
Modified: uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-process-submit.tex
URL: http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-process-submit.tex?rev=1491963&r1=1491962&r2=1491963&view=diff
==============================================================================
--- uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-process-submit.tex (original)
+++ uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-process-submit.tex Tue Jun 11 21:11:05 2013
@@ -1,4 +1,4 @@
-% Create well-known link to this spot for HTML version
+2y% Create well-known link to this spot for HTML version
\ifpdf
\else
\HCode{<a name='DUCC_CLI_PROCESS_SUBMIT'></a>}
@@ -6,11 +6,11 @@
\section{ducc\_process\_submit}
\label{sec:cli.ducc-process-submit}
\paragraph{Description:}
- Usse {\em ducc\_process\_submit} to submit a Managed Reservation, also known as an
- arbitrary process to DUCC. The intention
+ Use {\em ducc\_process\_submit} to submit a Managed Reservation, also known as an
+ {\em arbitrary process} to DUCC. The intention
of this function is an alternative to utilities such as {\em ssh}, in order to allow the
spawned processes to be fully managed by DUCC. This allows the DUCC scheduler to allocate
- the necessary resources (and prevent over-allocation), and the DUCC runtime environment
+ the necessary resources (and prevent over-allocation), and the DUCC run-time environment
to manage process lifetime.
If {\em process\_attach\_console} is specified, Stdin, Stderr, and Stdout of the remote
@@ -31,26 +31,27 @@
(Ctrl-C). If running with {\em--wait\_for\_completion} and this flag is specified,
terminating the submit process will result in the remote process being terminated.
- \item[--description {[text]}] The text is any (quoted) string used to describe the process. It is
- displayed in the Web Server.
+ \item[--description {[text]}] The text is any string used to describe the process. It is
+ displayed in the Web Server. When specified on a command-line the text usually must be
+ surrounded by quotes to protect it from the shell.
\item[--debug ] Prints internal debugging information, intended for DUCC developers or
extended problem determination.
- \item[--environment {[env vars]}] Blank-delimeted list of environment variables. If
- specified, this is used for all DUCC processes in the job.Example:
+ \item[--environment {[env vars]}] Blank-delimited list of environment variables. If
+ specified, this is used for all DUCC processes in the job. Example:
\begin{verbatim}
--environment "TERM=xterm DISPLAY=me.org.net:1.0".
\end{verbatim}
- Note: On Secure Linux systems, the environemnt variable
+ Note: On Secure Linux systems, the environment variable
LD\_LIBRARY\_PATH may not be passed to the user's program. If it is
necessary to pass LD\_LIBRARY\_PATH to the JP or JD processes, it must be
specified as DUCC\_LD\_LIBRARY\_PATH. Ducc (securely) passes this as
LD\_LIBRARY\_PATH, after the JP or JD has assumed the user's identity. For
example:
\begin{verbatim}
--environment TERM=xterm DISPLAY=:1.0 DUCC\_LD\_LIBRARY\_PATH=/my/own/path
+-environment TERM=xterm DISPLAY=:1.0 DUCC_LD_LIBRARY_PATH=/my/own/path
\end{verbatim}
\item[--help] Prints the usage text to the console.
@@ -60,10 +61,10 @@
This specifies the path to the directory for the user logs. If not specified, the default is the
user's home directory. Example:
\begin{verbatim}
---log\_directory /home/bob
+--log_directory /home/bob
\end{verbatim}
- Within this directory DUCC creates a subdirectory for each process, using the numerical
+ Within this directory DUCC creates a sub-directory for each process, using the numerical
ID of the job. The format of the generated log file names as described
\hyperref[chap:job-logs]{here}.
@@ -72,50 +73,34 @@
machines DUCC, sub-directory and file names are generated by DUCC and may
not be directly specified.
- \item[--process\_attach\_console] If specified, redirect remote process (as
- opposed to driver) stdout and stderr to the local submitting console.
+ \item[--process\_attach\_console] If specified, redirect remote process stdio is
+ redirected the local submitting console.
- \item[--process\_environment {[environment]} ] This specifies environment parameters for the
- Job Processes. If present, they are added to the Job Process environment as the process is
- spawned. It must be a quoted, blankdelimeted lsit of name-value pairs. For example:
-\begin{verbatim}
-"--process_environment TERM=xterm DISPLAY=:1.0"
-\end{verbatim}
-
- Note: On Secure Linux systems, the environemnt variable
- LD\_LIBRARY\_PATH may not be passed to the user's program. If it is
- necessary to pass LD\_LIBRARY\_PATH to the JP or JD processes, it must be
- specified as DUCC\_LD\_LIBRARY\_PATH. Ducc (securely) passes this as
- LD\_LIBRARY\_PATH, after the JP or JD has assumed the user's identity. For
- example:
-
-\begin{verbatim}
-"--process_environment TERM=xterm DISPLAY=:1.0 DUCC\_LD\_LIBRARY\\_PATH=/my/own/
-\end{verbatim}
-
\item[--process\_executable {[program name]}] This is the full path to a program to be
executed.
- \item[--process\_executable\_args {[argument list]}] Blank-delimited list of arguments for
- {\em process\_executable}.
+ \item[--process\_executable\_args {[argument list]}] This is a list of arguments for
+ {\em process\_executable}, if any. When specified on a command-line the text usually must be
+ surrounded by quotes to protect it from the shell.
\item[--process\_memory\_size {[size]} ] This specifies the maximum amount of RAM in GB to
- be allocated to each ]rocess. This value is used by the Resource Manager to allocate
+ be allocated to each process. This value is used by the Resource Manager to allocate
resources. if this amount is exceeded by a process the Agent terminates the process with a
ShareSizeExceeded message.
- \item[--scheduling\_class {[classname]} ] This specifies the name of the scheuling class the
+ \item[--scheduling\_class {[classname]} ] This specifies the name of the scheduling class the
RM will use to determine the resource allocation for each process. The names of the
classes are installation dependent. If not specified, the default is taken from the global
DUCC configuration ducc.properties.
- \item[--specifiecaiton {[file]} ] All the parameters used to submit a process may be placed
+ \item[--specification, $-$f {[file]} ] All the parameters used to submit a process may be placed
in a standard Java properties file. This file may then be used to submit the process
(rather than providing all the parameters directory to submit).
For example,
\begin{verbatim}
ducc_process_submit --specification job.props
+ducc_process_submit -f job.props
\end{verbatim}
where the job.props contains:
Modified: uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-service-cancel.tex
URL: http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-service-cancel.tex?rev=1491963&r1=1491962&r2=1491963&view=diff
==============================================================================
--- uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-service-cancel.tex (original)
+++ uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-service-cancel.tex Tue Jun 11 21:11:05 2013
@@ -7,6 +7,12 @@
\label{sec:cli.service-cancel}
\paragraph{Description:}
+ The ducc\_service\_cancel CLI is used to terminate a service instance.
+
+ Note: This command will for for {\em registered} services, but if the service is registered for
+ {\em autostart}, the Service Manager will restart the process. This can be used to advantage, to
+ recycle specific registered services instances when needed.
+
\paragraph{Usage:}
\begin{description}
\item[Executable Jar] java -jar \ducchome/lib/uima-ducc-service-cancel.jar {\em options}
@@ -24,7 +30,7 @@
Prints the usage text to the console.
\item[--role\_administrator] The command is being issued in the role of a DUCC administrator.
If the user is not also a registered administrator this flag is ignored. (This helps to
- protect administrators from inadvertantly canceling work they do not own.)
+ protect administrators from inadvertently canceling work they do not own.)
\end{description}
\paragraph{Notes:}
Modified: uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-service-submit.tex
URL: http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-service-submit.tex?rev=1491963&r1=1491962&r2=1491963&view=diff
==============================================================================
--- uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-service-submit.tex (original)
+++ uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-service-submit.tex Tue Jun 11 21:11:05 2013
@@ -16,11 +16,12 @@
The {\em ducc\_service\_submit} and \hyperref[sec:cli.service-cancel]{{\em
ducc\_service\_cancel}} commands are primarily used by the DUCC Service Manager for starting
- and stopping instances of registered services. It is legal and supported for these to be used
- outside of the context of registered services. DUCC will attempt to restart failed services
- instances up to a configurable limit. DUCC will NOT restart submitted services aftar a
- system failure however. If full restart after system failure is required, one must
- \hyperref[subsec:cli.ducc-services.register]{register} the service.
+ and stopping instances of registered services but they are supported for general use as well.
+
+ DUCC will attempt to restart failed services instances up to a configurable limit. DUCC will
+ NOT restart submitted services aftar a system failure however. If full restart after system
+ failure is required, one must \hyperref[subsec:cli.ducc-services.register]{register} the
+ service.
\paragraph{Usage:}
@@ -33,23 +34,27 @@
\paragraph{Options:}
\begin{description}
- \item[--classpath] The classpath used for the service, if the service is a
- \hyperref[sec:services.types]{UIMA-AS services}.
+ \item[$--$classpath] The CLASSPATH used for the service, if the service is a
+ \hyperref[sec:services.types]{UIMA-AS services}. If not specified, the CLASSPATH used
+ by the {\em ducc\_service\_submit} command is used.
- \item[--classpath\_order {[UserBeforeDucc | DuccBeforeUser]} ] When DUCC deploys a process,
- set the user-supplied classpath before DUCC-supplied classpath, or the reverse, if the
- service is a \hyperref[sec:services.types]{UIMA-AS services}.
+ \Item[$--$classpath\_order {[UserBeforeDucc | DuccBeforeUser]} ] When DUCC deploys a process,
+ set the user-supplied classpath before DUCC-supplied classpath, or the reverse. This is
+ only valid for \hyperref[sec:services.types]{UIMA-AS services}.
- \item[--debug ]
+ \item[$--$debug ]
Enable debugging messages. This is primarily for debugging DUCC itself.
- \item[--description {[text]}] The text is any quoted string used to describe the job. It is
+ \item[$--$description {[text]}] The text is any quoted string used to describe the job. It is
displayed in the Web Server.
-
- \item[--environment {[env vars]}]
+
+ Note: When used as a CLI option, the description string must usually be quoted to protect
+ it from the shell.
+
+ \item[$--$environment {[env vars]}]
This specifies environment parameters for the Service. If present, they are added
- to the Job Process environment as the process is spawned. It must be a quoted, blankdelimeted
- lsit of name-value pairs. For example:
+ to the Job Process environment as the process is spawned. It is a blank-delimited
+ list of name-value pairs. For example:
\begin{verbatim}
--environment 'TERM=xterm DISPLAY=:1.0'
@@ -61,14 +66,27 @@
specified as DUCC\_LD\_LIBRARY\_PATH. Ducc (securely) passes this as
LD\_LIBRARY\_PATH, after the JP or JD has assumed the user's identity. For
example:
+\begin{verbatim}
+--environment TERM=xterm DISPLAY=:1.0 DUCC\_LD\_LIBRARY\_PATH=/my/own/path
+\end{verbatim}
+
+ Note: When used as a CLI option, the environment string must usually be
+ quoted to protect it from the shell.
+
+ \item[$--$help ] This prints the usage text to the console.
- \item[--help ] This prints the usage text to the console.
-
- \item[--jvm {[path-to-java]}] This specifies the JVM to use for
+ \item[$--$jvm {[path-to-java]}] This specifies the JVM to use for
\hyperref[sec:services.types]{UIMA-AS services}. If not
- specified, the same JVM used by the Agents is used.
+ specified, the same JVM used by the Agents is used.
+
+ Note: The path must be the full path the the Java executable (not
+ simply the JAVA_HOME environment variable.). Example:
+\begin{verbatim}
+--jvm /share/jdk1.6/bin/java
+\end{verbatim}
+
- \item[--jvm\_args {[list]} ]
+ \item[$--$jvm\_args {[list]} ]
This specifes extra JVM arguments to be provided to the server process for
\hyperref[sec:services.types]{UIMA-AS services}. It is a blank delimeted
list of strings. Example:
@@ -76,7 +94,10 @@
--jvm_args '-Xmx100M -Xms50M'
\end{verbatim}
- \item[--log\_directory {[path-to-log directory]}] This specifies the path to the directory for
+ Note: When used as a CLI option, the argument string must usually be quoted to protect
+ it from the shell.
+
+ \item[$--$log\_directory {[path-to-log directory]}] This specifies the path to the directory for
the individual service instance logs. If not specified, the default is the user's home
directory. Example:
\begin{verbatim}
@@ -87,94 +108,97 @@
ID of the job. The format of the generated log file names as described
\hyperref[chap:job-logs]{here}.
- Note: Note that --log\_directory specifies only the path to a directory where
+ Note: Note that $--$log\_directory specifies only the path to a directory where
logs are to be stored. In order to manage multiple processes running in multiple
machines DUCC, sub-directory and file names are generated by DUCC and may
not be directly specified.
- \item[--process\_DD {[DD descriptor]}]
+ \item[$--$process\_DD {[DD descriptor]}]
This specifies the UIMA Deployment Descriptor for \hyperref[sec:services.types]{UIMA-AS services}.
- \item[--process\_environment {[environment]}] This specifies environment parameters for the Job
- Processes. If present, they are added to the Job Process environment as the process is
- spawned. It must be a quoted, blankdelimeted lsit of name-value pairs.
+ \item[$--$process\_executable {[program-name]}] For \hyperref[sec:services.types]{CUSTOM
+ services}, this specifies the full path of the program to execute.
- Note: On Secure Linux systems, the environemnt variable
- LD\_LIBRARY\_PATH may not be passed to the user's program. If it is
- necessary to pass LD\_LIBRARY\_PATH to the JP or JD processes, it must be
- specified as DUCC\_LD\_LIBRARY\_PATH. Ducc (securely) passes this as
- LD\_LIBRARY\_PATH, after the JP or JD has assumed the user's identity. For
- example:
-\begin{verbatim}
---process_environment TERM=xterm DISPLAY=:1.0 DUCC_LD_LIBRARY_PATH=/my/own/lib.so
-\end{verbatim}
-
- \item[--process\_executable {[program-name]}] For \hyperref[sec:services.types]{CUSTOM
- services}, the full path of the program to execute.
-
- \item[--process\_executable\_args {[list-of-arguments]}] For \hyperref[sec:services.types]{CUSTOM
- services}, the program arguments, if any.
+ \item[$--$process\_executable\_args {[list-of-arguments]}] For \hyperref[sec:services.types]{CUSTOM
+ services}, this specifies the program arguments, if any.
- \item[--process\_failures\_limit {[integer]}]
+ \item[$--$process\_failures\_limit {[integer]}]
This specifies the maximum number of consecutive individual service instance failures that are to be
- tolerated before stopping the service. The default is 5. If the instance is successfully
- restarted, the count is reset to 0, so that the occasional process failure does not cause
- the entire service to be canceled.
+ tolerated before stopping the service. The default is five (5). If the instance is successfully
+ restarted, the count is reset to zero (0), so that the occasional process failure does not cause
+ the entire service to be terminated.
- \item[--process\_memory\_size {[size]}] This specifies the maximum amount of RAM in GB to be
+ \item[$--$process\_memory\_size {[size]}] This specifies the maximum amount of RAM in GB to be
allocated to each Job Process. This value is used by the Resource Manager to allocate
resources.
- \item[--scheduling\_class {[classname]}] This specifies the name of the scheuling class the RM
+ \item[$--$scheduling\_class {[classname]}] This specifies the name of the scheuling class the RM
will use to determine the resource allocation for each process. The names of the classes are
installation dependent. If not specified, the default is taken from the global DUCC
configuration \hyperref[sec:ducc.properties]{ducc.properties.}
- \item[--service\_dependency{[list]}] This specifies a comma-delimeted list of services the job
+ \item[$--$service\_dependency{[list]}] This specifies a comma-delimeted list of services the job
processes are dependent upon. Service dependencies are discussed in detail
\hyperref[sec:service.endpoints]{here}. Example:
\begin{verbatim}
--service_dependency UIMA-AS:RandomSleepAE:tcp:bluej682:61616 UIMA-AS:OtherEp:tcp:bluej123:123
\end{verbatim}
- \item[--service\_linger {[seconds]}] This is the time in milliseconds to wait after last
+ Note: When used as a CLI option, the list must usually be
+ quoted to protect it from the shell.
+
+
+ \item[$--$service\_linger {[seconds]}] This is the time in milliseconds to wait after the last
referring job or service exits before stopping a non-autostarted service.
- \item[--service\_ping\_class {[classname]}] This is the class used to ping a service. If
- omitted and the service is a \hyperref[sec:services.types]{UIMA-AS service}, DUCC will
- supply a default pinger. See \hyperref[sec:service.pingers]{Service Pingers} for a
- discussion of the pingers.
+ \item[$--$service\_ping\_class {[classname]}] This is the Java class used to ping a service.
+
+ This parameter is required for CUSTOM services.
- \item[--service\_ping\_classpath {[classpath]}] If {\em service\_ping\_class} is specified,
+ This parameter may be specified for UIMA-AS services; however, DUCC supplies a default
+ pinger for UIMA-AS services.
+
+ \item[$--$service\_ping\_classpath {[classpath]}] If {\em service\_ping\_class} is specified,
this is the classpath containing service\_custom\_ping class and dependencies. If not
specified, the Agent's classpath is used (which will generally be incorrect.)
- \item[--service\_ping\_dolog {[boolean]}] If specified, write pinger stdout and stderr
+ \item[$--$service\_ping\_dolog {[boolean]}] If specified, write pinger stdout and stderr
messages to a log, else suppress the log. See \hyperref[sec:service.pingers]{Service Pingers}
for details.
- \item[--service\_ping\_jvm\_args {[java-system-property-assignments]}] If
+ \item[$--$service\_ping\_jvm\_args {[java-system-property-assignments]}] If
{\em service\_ping\_class} is specified, these are the arguments
- to pass to jvm when running the pinger.
+ to pass to jvm when running the pinger. The arguments are specified as a blank-delimited
+ list of string. Example:
+\begin{verbatim}
+--process_jvm_args '-Xmx400M -Xms100M'
+\end{verbatim}
+
+ Note: When used as a CLI option, the arguments must usually be
+ quoted to protect them from the shell.
- \item[--service\_ping\_timeout {[time-in-ms]}] This is the time in milliseconds to wait for a
+ \item[$--$service\_ping\_timeout {[time-in-ms]}] This is the time in milliseconds to wait for a
ping to the service. If the timer expires without a response the ping is ``failed''. After
a certain number of consecutive failed pings, the service is considered ``down.'' See
\hyperref[sec:service.pingers]{Service Pingers} for more details.
- \item[--service\_request\_endpoint {[string]}] This specifies the expected service id. The string
- must be in form {\tt queue:broker-url}. This must match the service ID that is derived from
- the {\em --process\_DD}; if it does not match, the submit fails. This is intended for use
- by applications using the service API and CLI as a fail-fast if something is wrong with the
- application.
+ \item[$--$service\_request\_endpoint {[string]}] This specifies the expected service id.
+
+ This string is optional for UIMA-AS services; if specified, however, it must be of the
+ form {\tt UIMA-AS:queue:broker-url}, and both the queue and broker must match those specified in the
+ service DD specifier.
+
+ If the service is CUSTOM, the endpoint is required, and must be of the form
+ {\tt CUSTOM:string} where the contents of the string are determined by the service.
- \item[--specifiecaiton {[file]}] All the parameters used to submit a service may be placed in a
+ \item[$--$specifiecaiton, $-$f {[file]}] All the parameters used to submit a service may be placed in a
standard Java properties file. This file may then be used to submit the service (rather than
providing all the parameters directory to submit).
For example,
\begin{verbatim}
ducc_service_submit --specification svc.props
+ducc_service_submit -f svc.props
\end{verbatim}
where the svc.props contains:
@@ -191,7 +215,7 @@ scheduling_class = fixed
service_dependency = UIMA-AS:FixedSleepAE_4:tcp://agent86:61616
\end{verbatim}
- \item[--working\_directory {[directory-name]}]
+ \item[$--$working\_directory {[directory-name]}]
This specifies the working directory to be set by the Job Driver and Job Process processes.
If not specified, the current directory is used.
\end{description}
Modified: uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-submit.tex
URL: http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-submit.tex?rev=1491963&r1=1491962&r2=1491963&view=diff
==============================================================================
--- uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-submit.tex (original)
+++ uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-submit.tex Tue Jun 11 21:11:05 2013
@@ -260,11 +260,11 @@
\item[$--$process\_jvm\_args {[list]} ] This specifies additional arguments to be passed to
the Job Process JVM as a blank-delimited list of strings. Example:
\begin{verbatim}
---process_jvm_args -Xmx400M -Xms100M
+--process_jvm_args '-Xmx400M -Xms100M'
\end{verbatim}
- Note: When used as a CLI option, the environment string must usually be
- quoted to protect it from the shell.
+ Note: When used as a CLI option, the arguments must usually be
+ quoted to protect them from the shell.
\item[$--$process\_memory\_size {[size]} ] This specifies the maximum amount of RAM in GB
to be allocated to each Job Process. This value is used by the Resource Manager to