You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@uima.apache.org by bu...@apache.org on 2015/10/16 21:25:45 UTC
svn commit: r1709066 - in
/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook:
duccbook.tex part2/cli/ducc-reserve.tex part2/cli/ducc-services.tex
part2/cli/ducc-submit.tex
Author: burn
Date: Fri Oct 16 19:25:44 2015
New Revision: 1709066
URL: http://svn.apache.org/viewvc?rev=1709066&view=rev
Log:
UIMA-4625 Update docs for process_thread_count change
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/duccbook.tex
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-reserve.tex
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-services.tex
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-submit.tex
Modified: uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/duccbook.tex
URL: http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/duccbook.tex?rev=1709066&r1=1709065&r2=1709066&view=diff
==============================================================================
--- uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/duccbook.tex (original)
+++ uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/duccbook.tex Fri Oct 16 19:25:44 2015
@@ -54,7 +54,7 @@ UIMA\texttrademark Development Community
\\
\\
\\
-Version 2.0.0}
+Version 2.1.0}
\date{}
Modified: uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-reserve.tex
URL: http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-reserve.tex?rev=1709066&r1=1709065&r2=1709066&view=diff
==============================================================================
--- uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-reserve.tex (original)
+++ uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-reserve.tex Fri Oct 16 19:25:44 2015
@@ -61,6 +61,8 @@
This specifies the amount of memory the reserved machine must support. After rounding
up it must match the total usable memory on the machine. (Required)
+ Alias: $--$instance\_memory\_size
+
\item[$--$scheduling\_class {[classname]}]
This specifies the name of the scheduling class the RM will use to determine the resource
allocation for each process. It must be one implementing the RESERVE policy.
Modified: uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-services.tex
URL: http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-services.tex?rev=1709066&r1=1709065&r2=1709066&view=diff
==============================================================================
--- uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-services.tex (original)
+++ uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-services.tex Fri Oct 16 19:25:44 2015
@@ -117,6 +117,9 @@
\hyperref[sec:services.types]{UIMA-AS services}. If not specified, the CLASSPATH of the
process invoking this request is used.
+ \item[$--$classpath\_order {[user-before-ducc $|$ ducc-before-user]} ]
+ OBSOLETE - ignored.
+
\item[$--$debug ]
Enable debugging messages. This is primarily for debugging DUCC itself.
Modified: uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-submit.tex
URL: http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-submit.tex?rev=1709066&r1=1709065&r2=1709066&view=diff
==============================================================================
--- uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-submit.tex (original)
+++ uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-submit.tex Fri Oct 16 19:25:44 2015
@@ -50,13 +50,17 @@
\item[$--$cancel\_on\_interrupt] If specified, the job is monitored
and will be canceled if the submit command is interrupted, e.g. with CTRL-C.
- This option always implies $--$wait\_for\_completion.
+ This option always implies {\em $--$wait\_for\_completion}.
\item[$--$classpath {[path-string]}] The CLASSPATH used for the job. If specified, this is used
for both the Job Driver and each Job Process. If not specified, the CLASSPATH of the
process invoking this request is used.
- \item[$--$debug] Enable debugging messages. This is primarily for debugging DUCC itself.
+ \item[$--$classpath\_order {[user-before-ducc $|$ ducc-before-user]} ]
+ OBSOLETE - ignored.
+
+ \item[$--$debug] Enable debugging messages.
+ This is primarily for debugging DUCC itself.
\item[$--$description {[text]}] The text is any string used to describe the job. It is
displayed in the Web Server. When specified on a command-line the text usually
@@ -73,40 +77,34 @@
\item[$--$driver\_descriptor\_CR\_overrides {[list]} ]
This is the Job Driver collection reader configuration overrides. They are specified as
- name/value pairs in a whitespace-delimited list. For example:
+ name/value pairs in a whitespace-delimited list. Example:
\begin{verbatim}
---driver_descriptor_CR_overrides name1=value1 name2=value2...
+ --driver_descriptor_CR_overrides name1=value1 name2=value2...
\end{verbatim}
- \begin{sloppypar}
- \item[$--$driver\_exception\_handler {[classname]}] This specifies a developer-supplied
- exception handler for the Job Driver. It must
- implement org.apache.uima.ducc.IErrorHandler or extend
- org.apache.uima.ducc.ErrorHandler. A default handler is provided.
- \end{sloppypar}
+ \item[$--$driver\_exception\_handler {[classname]}]
+ This specifies a developer-supplied exception handler for the Job Driver.
+ It must implement {\em org.apache.uima.ducc.IErrorHandler} or extend
+ {\em org.apache.uima.ducc.ErrorHandler}. A default handler is provided.
\item[$--$driver\_exception\_handler\_arguments {[argument-string]}] This is a string
containing arguments for the exception handler. The contents of
the string is entirely a function of the specified handler. If not specified,
a {\em null} is passed in.
-
- Note: When used as a CLI option, the string must usually be
- quoted to protect it from the shell, if it contains blanks.
-
- The built-in default exception handler supports an argument string of the following form
+ \\The built-in default exception handler supports an argument string of the following form
(with NO embedded blanks):
-\begin{verbatim}
+ \begin{verbatim}
max_job_errors=15
-\end{verbatim}
+ \end{verbatim}
+ Note: When used as a CLI option, the string must usually be
+ quoted to protect it from the shell, if it contains blanks.
\item[$--$driver\_jvm\_args {[list]} ]
-
This specifies extra JVM arguments to be provided to the Job Driver process. It is a blank-delimited
list of strings. Example:
\begin{verbatim}
---driver_jvm_args -Xmx100M -Xms50M
+ --driver_jvm_args -Xmx100M -Xms50M
\end{verbatim}
-
Note: When used as a CLI option, the list must usually be
quoted to protect it from the shell.
@@ -115,45 +113,38 @@
Entries will be copied from the user's environment if just the variable name is
specified, optionally with a final '*' for those with the same prefix.
If specified, this is used for all DUCC processes in the job. Example:
-\begin{verbatim}
---environment TERM=xterm DISPLAY=:1.0 LANG UIMA_*
-\end{verbatim}
-
+ \begin{verbatim}
+ --environment TERM=xterm DISPLAY=:1.0 LANG UIMA_*
+ \end{verbatim}
Additional entries may be copied from the user's environment based on the setting of
-\begin{verbatim}
-ducc.submit.environment.propagated
-\end{verbatim}
+ {\em ducc.submit.environment.propagated}
in the global DUCC configuration ducc.properties.
-
- Note: When used as a CLI option, the environment string must usually be
+ \\Note: When used as a CLI option, the environment string must usually be
quoted to protect it from the shell.
\item[$--$help ]
-
Prints the usage text to the console.
\item[$--$jvm {[path-to-java]} ]
-
States the JVM to use. If not specified, the same JVM used by the Agents is used. This is
the full path to the JVM, not the JAVA\_HOME.
Example:
-\begin{verbatim}
---jvm /share/jdk1.6/bin/java
-\end{verbatim}
+ \begin{verbatim}
+ --jvm /share/jdk1.6/bin/java
+ \end{verbatim}
\item[$--$log\_directory {[path-to-log-directory]} ]
-
- This specifies the path to the directory for the user logs. If not specified, the default is
- \$HOME/ducc/logs. Example:
+ This specifies the path to the directory for the user logs.
+ If not fully specified the path is made relative to the value of the {\em $--$working\_directory}.
+ If omitted, the default is \$HOME/ducc/logs.
+ Example:
\begin{verbatim}
---log_directory /home/bob
+ --log_directory /home/bob
\end{verbatim}
-
Within this directory DUCC creates a sub-directory for each job, using the unique numerical
ID of the job. The format of the generated log file names as described
\hyperref[chap:job-logs]{here}.
-
- Note: Note that $--$log\_directory specifies only the path to a directory where
+ \\Note: The {\em $--$log\_directory} specifies only the path to a directory where
logs are to be stored. In order to manage multiple processes running in multiple
machines, sub-directory and file names are generated by DUCC and may
not be directly specified.
@@ -164,102 +155,91 @@ ducc.submit.environment.propagated
port.
\item[$--$process\_deployments\_max {[integer]} ]
-
This specifies the maximum number of Job Processes to deploy at any given time. If not
specified, DUCC will attempt to provide the largest number of processes within the
constraints of fair\_share scheduling and the amount of work remaining.
in the job. Example:
\begin{verbatim}
---process_deployments_max 66
+ --process_deployments_max 66
\end{verbatim}
-
\item[$--$process\_descriptor\_AE {[descriptor]} ]
-
This specifies the Analysis Engine descriptor to be deployed in the Job Processes. This
descriptor is a resource that is searched for in the filesystem or Java classpath as described
in the ~\hyperref[par:cli.submit.notes]{notes below}.
- It is mutually exclusive with $--$process\_descriptor\_DD For example:
+ It is mutually exclusive with {\em $--$process\_descriptor\_DD}.
+ Example:
\begin{verbatim}
---process_descriptor_AE /home/billy/resource/AE_foo.xml
+ --process_descriptor_AE /home/billy/resource/AE_foo.xml
\end{verbatim}
- \begin{sloppypar}
\item[$--$process\_descriptor\_AE\_overrides {[list]} ]
-
This specifies AE overrides. It is a whitespace-delimited list of name/value pairs. Example:
\begin{verbatim}
---process_descriptor_AE_Overrides name1=value1 name2=value2
+ --process_descriptor_AE_Overrides name1=value1 name2=value2
\end{verbatim}
- \end{sloppypar}
\item[$--$process\_descriptor\_CC {[descriptor]} ]
-
This specifies the CAS Consumer descriptor to be deployed in the Job Processes. This
descriptor is a resource that is searched for in the filesystem or Java classpath as described
in the ~\hyperref[par:cli.submit.notes]{notes below}.
- It is mutually exclusive with $--$process\_descriptor\_DD For example:
+ It is mutually exclusive with {\em $--$process\_descriptor\_DD}.
+ Example:
\begin{verbatim}
---process_descriptor_CC /home/billy/resourceCCE_foo.xml
+ --process_descriptor_CC /home/billy/resourceCCE_foo.xml
\end{verbatim}
- \begin{sloppypar}
\item[$--$process\_descriptor\_CC\_overrides {[list]} ]
-
This specifies CC overrides. It is a whitespace-delimited list of name/value pairs. Example:
\begin{verbatim}
---process_descriptor_CC_overrides name1=value1 name2=value2
+ --process_descriptor_CC_overrides name1=value1 name2=value2
\end{verbatim}
- \end{sloppypar}
\item[$--$process\_descriptor\_CM {[descriptor]} ]
-
This specifies the CAS Multiplier descriptor to be deployed in the Job Processes. This
descriptor is a resource that is searched for in the filesystem or Java classpath as described
in the ~\hyperref[par:cli.submit.notes]{notes below}.
- It is mutually exclusive with $--$process\_descriptor\_DD For example:
+ It is mutually exclusive with {\em $--$process\_descriptor\_DD}.
+ Example:
\begin{verbatim}
---process_descriptor_CM /home/billy/resource/CM_foo.xml
+ --process_descriptor_CM /home/billy/resource/CM_foo.xml
\end{verbatim}
- \begin{sloppypar}
\item[$--$process\_descriptor\_CM\_overrides {[list]} ]
-
This specifies CM overrides. It is a whitespace-delimited list of name/value pairs. Example:
-\begin{verbatim}
---process_descriptor_CM_overrides name1=value1 name2=value2
-\end{verbatim}
- \end{sloppypar}
+ \begin{verbatim}
+ --process_descriptor_CM_overrides name1=value1 name2=value2
+ \end{verbatim}
\item[$--$process\_descriptor\_DD {[descriptor]} ]
-
This specifies a UIMA Deployment Descriptor for the job processes for DD-style jobs.
- This is mutually exclusive with $--$process\_descriptor\_AE, $--$process\_descriptor\_CM,
- and $--$process\_descriptor\_CC. This
+ This is mutually exclusive with {\em $--$process\_descriptor\_AE}, {\em $--$process\_descriptor\_CM},
+ and {\em $--$process\_descriptor\_CC}. This
descriptor is a resource that is searched for in the filesystem or Java classpath as described
in the ~\hyperref[par:cli.submit.notes]{notes below}.
- For example:
+ Example:
\begin{verbatim}
---process_descriptor_DD /home/billy/resource/DD_foo.xml
+ --process_descriptor_DD /home/billy/resource/DD_foo.xml
\end{verbatim}
+ Alias: $--$process\_DD
\item[$--$process\_failures\_limit {[integer]} ]
This specifies the maximum number of individual Job Process (JP) failures allowed
before killing the job. The default is twenty(20). If this limit is exceeded over the lifetime
of a job DUCC terminates the entire job.
-\begin{verbatim}
---process_failures_limit 23
-\end{verbatim}
+ Example:
+ \begin{verbatim}
+ --process_failures_limit 23
+ \end{verbatim}
\item[$--$process\_initialization\_failures\_cap {[integer]} ] This specifies the maximum
number of failures during a UIMA process's initialization phase. If the number is
exceeded the system will allow processes which are already running to continue, but
will assign no new processes to the job. The default is ninety-nine(99). Example:
\begin{verbatim}
---process_initialization_failures_cap 62
+ --process_initialization_failures_cap 62
\end{verbatim}
-
Note that the job is NOT killed if there are processes that have passed initialization and are
running. If this limit is reached, the only action is to not start new processes for the job.
@@ -270,9 +250,8 @@ ducc.submit.environment.propagated
\item[$--$process\_jvm\_args {[list]} ] This specifies additional arguments to be passed to
all of the job processes as a blank-delimited list of strings. Example:
\begin{verbatim}
---process_jvm_args -Xmx400M -Xms100M
+ --process_jvm_args -Xmx400M -Xms100M
\end{verbatim}
-
Note: When used as a CLI option, the arguments must usually be
quoted to protect them from the shell.
@@ -283,9 +262,9 @@ ducc.submit.environment.propagated
\item[$--$process\_per\_item\_time\_max {[integer]} ] This specifies the maximum time in
minutes that the Job Driver will wait for a Job Processes to process a CAS. If a
timeout occurs the process is terminated and the CAS marked in error (not retried). If
- not specified, the default is 1 minute. Example:
+ not specified, the default is 24 hours. Example:
\begin{verbatim}
---process_per_item_time_max 60
+ --process_per_item_time_max 60
\end{verbatim}
\item[$--$process\_pipeline\_count {[integer]} ] This specifies the number of pipelines per
@@ -295,7 +274,7 @@ ducc.submit.environment.propagated
spawn, and by the Job Driver to determine how many CASs to dispatch. If not specified,
the default is 4. Example:
\begin{verbatim}
---process_pipeline_count 7
+ --process_pipeline_count 7
\end{verbatim}
Alias: $--$process\_thread\_count
@@ -306,30 +285,26 @@ ducc.submit.environment.propagated
described \hyperref[subsubsec:class.configuration]{here.}
Example:
\begin{verbatim}
---scheduling_class normal
+ --scheduling_class normal
\end{verbatim}
- \begin{sloppypar}
\item[$--$service\_dependency{[list]}] This specifies a blank-delimited list of services the job
processes are dependent upon. Service dependencies are discussed in detail
\hyperref[sec:service.endpoints]{here}. Example:
-\begin{verbatim}
---service_dependency UIMA-AS:Service1:tcp:host1:61616 UIMA-AS:Service2:tcp:host2:123
-\end{verbatim}
- \end{sloppypar}
+ \begin{verbatim}
+ --service_dependency UIMA-AS:Service1:tcp:host1:61616 UIMA-AS:Service2:tcp:host2:123
+ \end{verbatim}
\item[$--$specification, $-$f {[file]} ]
-
All the parameters used to submit a job may be placed in a standard Java properties file.
This file may then be used to submit the job (rather than providing all the parameters
directory to submit). The leading $--$ is omitted from the keywords.
For example,
-\begin{verbatim}
-ducc_submit --specification job.props
-ducc_submit -f job.props
-\end{verbatim}
-
+ \begin{verbatim}
+ ducc_submit --specification job.props
+ ducc_submit -f job.props
+ \end{verbatim}
where job.props contains:
\begin{verbatim}
working_directory = /home/bob/projects/ducc/ducc_test/test/bin
@@ -349,8 +324,7 @@ process_jvm_args = -X
scheduling_class = normal
process_memory_size = 15
\end{verbatim}
-
- Note that properties in a specifications file may be overridden by other command-line
+ Note that properties in a specification file may be overridden by other command-line
parameters, as discussed \hyperref[chap:cli]{here}.
\item[$--$suppress\_console\_log] If specified, suppress creation of the log files that
@@ -364,7 +338,7 @@ process_memory_size = 15
If specified, the submit command monitors the job and prints periodic
state and progress information to the console. When the job completes, the monitor
is terminated and the submit command returns. If the command is interrupted, e.g. with CTRL-C,
- the job will not be canceled unless $--$cancel\_on\_interrupt is also specified.
+ the job will not be canceled unless {\em $--$cancel\_on\_interrupt} is also specified.
\item[$--$working\_directory ]
This specifies the working directory to be set by the Job Driver and Job Process processes.