You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@uima.apache.org by bu...@apache.org on 2013/06/26 00:43:32 UTC
svn commit: r1496663 - in
/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook:
part2/cli/ part4/ part4/admin/
Author: burn
Date: Tue Jun 25 22:43:32 2013
New Revision: 1496663
URL: http://svn.apache.org/r1496663
Log:
UIMA-3026 Updated docs for new entry in ducc.properties; fixed a few typos and errors
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-process-submit.tex
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-reserve.tex
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-service-submit.tex
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-submit.tex
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part4/admin/ducc-properties.tex
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part4/sim.tex
Modified: uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-process-submit.tex
URL: http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-process-submit.tex?rev=1496663&r1=1496662&r2=1496663&view=diff
==============================================================================
--- uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-process-submit.tex (original)
+++ uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-process-submit.tex Tue Jun 25 22:43:32 2013
@@ -38,28 +38,24 @@
\item[$--$debug ] Prints internal debugging information, intended for DUCC developers or
extended problem determination.
- \item[$--$environment {[env vars]}] Blank-delimited list of environment variables. If
- specified, this is used for all DUCC processes in the job. Example:
-\begin{verbatim}
- --environment "TERM=xterm DISPLAY=me.org.net:1.0".
-\end{verbatim}
-
- Note: On Secure Linux systems, the environment variable
- LD\_LIBRARY\_PATH may not be passed to the user's program. If it is
- necessary to pass LD\_LIBRARY\_PATH to the JP or JD processes, it must be
- specified as DUCC\_LD\_LIBRARY\_PATH. Ducc (securely) passes this as
- LD\_LIBRARY\_PATH, after the JP or JD has assumed the user's identity. For
- example:
- \begin{verbatim}
--environment TERM=xterm DISPLAY=:1.0 DUCC_LD_LIBRARY_PATH=/my/own/path
- \end{verbatim}
+ \item[$--$environment {[env vars]}] Blank-delimited list of environment variable
+ assignments for the process. Example:
+ \begin{verbatim}
+--environment TERM=xterm DISPLAY=:1.0
+ \end{verbatim}
+
+ Additional entries may be copied from the user's environment based on the setting of
+ ducc.submit.environment.propagated in the global DUCC configuration ducc.properties.
+
+ Note: When used as a CLI option, the environment string must usually be
+ quoted to protect it from the shell.
\item[$--$help] Prints the usage text to the console.
\item[$--$log\_directory {[path-to-log directory]} ]
- This specifies the path to the directory for the user logs. If not specified, the default is the
- user's home directory. Example:
+ This specifies the path to the directory for the user logs. If not specified, the default
+ is \$HOME/ducc/logs. Example:
\begin{verbatim}
--log_directory /home/bob
\end{verbatim}
@@ -103,10 +99,10 @@ ducc_process_submit --specification job.
ducc_process_submit -f job.props
\end{verbatim}
- where the job.props contains:
+ where job.props contains:
\begin{verbatim}
working_directory = /home/bob/projects
-process_environment = AE_INIT_TIME=10000 DUCC_LD_LIBRARY_PATH=/a/bogus/path
+environment = AE_INIT_TIME=10000 LD_LIBRARY_PATH=/a/bogus/path
log_directory = /home/bob/ducc/logs/
description = Simple Process
scheduling_class = fixed
Modified: uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-reserve.tex
URL: http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-reserve.tex?rev=1496663&r1=1496662&r2=1496663&view=diff
==============================================================================
--- uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-reserve.tex (original)
+++ uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-reserve.tex Tue Jun 25 22:43:32 2013
@@ -49,7 +49,7 @@
allocation for each process. The default DUCC distribution provides class "reserve" for full
machine reservations, and "fixed" for partial machine reservations.
- \item[$-$f, $--$specifiecaiton {[file]}]
+ \item[$-$f, $--$specification {[file]}]
All the parameters used to request a reservation may be placed in a standard Java
properties file. This file may then be used to submit the request (rather than providing all
the parameters directory to submit).
Modified: uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-service-submit.tex
URL: http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-service-submit.tex?rev=1496663&r1=1496662&r2=1496663&view=diff
==============================================================================
--- uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-service-submit.tex (original)
+++ uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-service-submit.tex Tue Jun 25 22:43:32 2013
@@ -51,27 +51,17 @@
Note: When used as a CLI option, the description string must usually be quoted to protect
it from the shell.
- \item[$--$environment {[env vars]}]
- This specifies environment parameters for the Service. If present, they are added
- to the Job Process environment as the process is spawned. It is a blank-delimited
- list of name-value pairs. For example:
-\begin{verbatim}
+ \item[$--$environment {[env vars]}] Blank-delimited list of environment variable
+ assignments for the service. Example:
+ \begin{verbatim}
+--environment TERM=xterm DISPLAY=:1.0
+ \end{verbatim}
+
+ Additional entries may be copied from the user's environment based on the setting of
+ ducc.submit.environment.propagated in the global DUCC configuration ducc.properties.
---environment 'TERM=xterm DISPLAY=:1.0'
-\end{verbatim}
-
- Note: On Secure Linux systems, the environemnt variable
- LD\_LIBRARY\_PATH may not be passed to the user's program. If it is
- necessary to pass LD\_LIBRARY\_PATH to the JP or JD processes, it must be
- specified as DUCC\_LD\_LIBRARY\_PATH. Ducc (securely) passes this as
- LD\_LIBRARY\_PATH, after the JP or JD has assumed the user's identity. For
- example:
-\begin{verbatim}
---environment TERM=xterm DISPLAY=:1.0 DUCC\_LD\_LIBRARY\_PATH=/my/own/path
-\end{verbatim}
-
- Note: When used as a CLI option, the environment string must usually be
- quoted to protect it from the shell.
+ Note: When used as a CLI option, the environment string must usually be
+ quoted to protect it from the shell.
\item[$--$help ] This prints the usage text to the console.
@@ -88,20 +78,19 @@
\item[$--$jvm\_args {[list]} ]
This specifes extra JVM arguments to be provided to the server process for
- \hyperref[sec:services.types]{UIMA-AS services}. It is a blank delimeted
+ \hyperref[sec:services.types]{UIMA-AS services}. It is a blank delimited
list of strings. Example:
\begin{verbatim}
---jvm_args '-Xmx100M -Xms50M'
+--jvm_args -Xmx100M -Xms50M
\end{verbatim}
Note: When used as a CLI option, the argument string must usually be quoted to protect
it from the shell.
\item[$--$log\_directory {[path-to-log directory]}] This specifies the path to the directory for
- the individual service instance logs. If not specified, the default is the user's home
- directory. Example:
+ the individual service instance logs. If not specified, the default is \$HOME/ducc/logs. Example:
\begin{verbatim}
---log\_directory /home/bob
+--log_directory /home/bob
\end{verbatim}
Within this directory DUCC creates a subdirectory for each job, using the numerical
@@ -137,7 +126,7 @@
installation dependent. If not specified, the default is taken from the global DUCC
configuration \hyperref[sec:ducc.properties]{ducc.properties.}
- \item[$--$service\_dependency{[list]}] This specifies a comma-delimeted list of services the job
+ \item[$--$service\_dependency{[list]}] This specifies a comma-delimited list of services the job
processes are dependent upon. Service dependencies are discussed in detail
\hyperref[sec:service.endpoints]{here}. Example:
\begin{verbatim}
@@ -171,7 +160,7 @@
to pass to jvm when running the pinger. The arguments are specified as a blank-delimited
list of string. Example:
\begin{verbatim}
---process_jvm_args '-Xmx400M -Xms100M'
+--service_ping_jvm_args -Xmx400M -Xms100M
\end{verbatim}
Note: When used as a CLI option, the arguments must usually be
@@ -191,7 +180,7 @@
If the service is CUSTOM, the endpoint is required, and must be of the form
{\tt CUSTOM:string} where the contents of the string are determined by the service.
- \item[$--$specifiecaiton, $-$f {[file]}] All the parameters used to submit a service may be placed in a
+ \item[$--$specification, $-$f {[file]}] All the parameters used to submit a service may be placed in a
standard Java properties file. This file may then be used to submit the service (rather than
providing all the parameters directory to submit).
For example,
@@ -204,10 +193,10 @@ ducc_service_submit -f svc.props
where the svc.props contains:
\begin{verbatim}
-process_environment = AE_INIT_TIME=5000 AE_INIT_RANGE=1000 INIT_ERROR=0
+environment = AE_INIT_TIME=5000 AE_INIT_RANGE=1000 INIT_ERROR=0
description = Test Service 3
-process_jvm_args = -DdefaultBrokerURL=tcp://agent86:61616
-process_classpath = /home/bob/lib/app.jar
+jvm_args = -DdefaultBrokerURL=tcp://agent86:61616
+classpath = /home/bob/lib/app.jar
process_memory_size = 15
working_directory = /home/bob/services
process_DD = /home/bob/services/service.xml
Modified: uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-submit.tex
URL: http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-submit.tex?rev=1496663&r1=1496662&r2=1496663&view=diff
==============================================================================
--- uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-submit.tex (original)
+++ uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-submit.tex Tue Jun 25 22:43:32 2013
@@ -99,24 +99,17 @@
--driver_jvm_args -Xmx100M -Xms50M
\end{verbatim}
- Note: When used as a CLI option, the environment string must usually be
+ Note: When used as a CLI option, the list must usually be
quoted to protect it from the shell.
- \item[$--$environment {[env vars]}] Blank-delimited list of environment variables. If
- specified, this is used for all DUCC processes in the job.Example:
-\begin{verbatim}
- $--$environment TERM=xterm DISPLAY=me.org.net:1.0
-\end{verbatim}
-
- Note: On Secure Linux systems, the environment variable
- LD\_LIBRARY\_PATH may not be passed to the user's program. If it is
- necessary to pass LD\_LIBRARY\_PATH to the JP or JD processes, it must be
- specified as DUCC\_LD\_LIBRARY\_PATH. Ducc (securely) passes this as
- LD\_LIBRARY\_PATH, after the JP or JD has assumed the user's identity. For
- example:
+ \item[$--$environment {[env vars]}] Blank-delimited list of environment variable
+ assignments. If specified, this is used for all DUCC processes in the job. Example:
\begin{verbatim}
---environment TERM=xterm DISPLAY=:1.0 DUCC\_LD\_LIBRARY\_PATH=/my/own/path
- \end{verbatim}
+--environment TERM=xterm DISPLAY=:1.0
+ \end{verbatim}
+
+ Additional entries may be copied from the user's environment based on the setting of
+ ducc.submit.environment.propagated in the global DUCC configuration ducc.properties.
Note: When used as a CLI option, the environment string must usually be
quoted to protect it from the shell.
@@ -136,8 +129,8 @@
\item[$--$log\_directory {[path-to-log-directory]} ]
- This specifies the path to the directory for the user logs. If not specified, the default is the
- user's home directory. Example:
+ This specifies the path to the directory for the user logs. If not specified, the default is
+ \$HOME/ducc/logs. Example:
\begin{verbatim}
--log_directory /home/bob
\end{verbatim}
@@ -260,7 +253,7 @@
\item[$--$process\_jvm\_args {[list]} ] This specifies additional arguments to be passed to
the Job Process JVM as a blank-delimited list of strings. Example:
\begin{verbatim}
---process_jvm_args '-Xmx400M -Xms100M'
+--process_jvm_args -Xmx400M -Xms100M
\end{verbatim}
Note: When used as a CLI option, the arguments must usually be
@@ -292,7 +285,7 @@
classes are installation dependent. If not specified, the default is taken from the
global DUCC configuration ducc.properties. Example:
\begin{verbatim}
---schedling_class normal
+--scheduling_class normal
\end{verbatim}
@@ -315,14 +308,12 @@ ducc_submit --specification job.props
ducc_submit -f job.props
\end{verbatim}
- where the job.props contains:
+ where job.props contains:
\begin{verbatim}
working_directory = /home/bob/projects/ducc/ducc_test/test/bin
process_failures_limit = 20
driver_descriptor_CR = org.apache.uima.ducc.test.randomsleep.FixedSleepCR
-driver_environment = DUCC_LD_LIBRARY_PATH=/a/other/bogus/path
-process_environment = AE_INIT_TIME=10000 DUCC_LD_LIBRARY_PATH=/a/bogus/path
-driver_classpath = /home/bob/duccapps/ducky_driver.jar
+environment = AE_INIT_TIME=10000 LD_LIBRARY_PATH=/a/bogus/path
log_directory = /home/bob/ducc/logs/
process_thread_count = 1
driver_descriptor_CR_overrides = jobfile:../simple/jobs/1.job,compression:10
@@ -330,7 +321,7 @@ process_initialization_failures_cap = 99
process_per_item_time_max = 60
driver_jvm_args = -Xmx500M
process_descriptor_AE = org.apache.uima.ducc.test.randomsleep.FixedSleepAE
-process_classpath = /home/bob/duccapps/ducky_process.jar
+classpath = /home/bob/duccapps/ducky_process.jar
description = ../simple/jobs/1.job[AE]
process_jvm_args = -Xmx100M -DdefaultBrokerURL=tcp://localhost:61616
scheduling_class = normal
Modified: uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part4/admin/ducc-properties.tex
URL: http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part4/admin/ducc-properties.tex?rev=1496663&r1=1496662&r2=1496663&view=diff
==============================================================================
--- uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part4/admin/ducc-properties.tex (original)
+++ uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part4/admin/ducc-properties.tex Tue Jun 25 22:43:32 2013
@@ -348,6 +348,16 @@
\item[Default] (unconfigured)
\item[Type] Local
\end{description}
+
+ \item[ducc.submit.environment.propagated] \hfill \\
+ This specifies the environmental variables whose values will be merged with the
+ user-specified environment option on job, process and service submissions. A typical setting
+ might be: USER HOME
+
+ \begin{description}
+ \item[Default] (unconfigured)
+ \item[Type] Local
+ \end{description}
\end{description}
Modified: uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part4/sim.tex
URL: http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part4/sim.tex?rev=1496663&r1=1496662&r2=1496663&view=diff
==============================================================================
--- uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part4/sim.tex (original)
+++ uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part4/sim.tex Tue Jun 25 22:43:32 2013
@@ -111,9 +111,9 @@
A properties file similar to a Java properties file is used to configure
simulated nodes. There are three types o entries in this file:
\begin{description}
- \item[nodes] This single entry provides the blank-delimeted names of the physical nodes
+ \item[nodes] This single entry provides the blank-delimited names of the physical nodes
participating in the simulated cluster.
- \item[memory] This single line consists of a blank-delimeted set
+ \item[memory] This single line consists of a blank-delimited set
of numbers. Each number corresponds to some memory size, in
GB, to be simulated.
\item[node descriptions] There are one or more of these. The format
@@ -179,7 +179,7 @@ agentn.79 = 6
\begin{description}
\item[-n, --nodelist {[nodelist]}] where the nodelist is a cluster description as
described above.
- \item[-c --components {[component list]}]. The component list is a blank-delimeted
+ \item[-c --components {[component list]}]. The component list is a blank-delimited
list of components includeing {\em or, rm, sm, pm, ws, broker} to start an
individual component, or {\em all} to start all of the components. NOTE: It is
usually an error to start any of these components more than once. However
@@ -283,7 +283,7 @@ agentn.79 = 6
\item[tod] This specifies a virual ``time of day of submission'', starting from time 0, specified
in units of milliseconds, when the job is to be submitted. During job generation, this may
be used to enforce precise timing of submission of the jobs.
- \item[elapsed] This is a space-delimeted set of numbers. Each number represents the elapsed time,
+ \item[elapsed] This is a space-delimited set of numbers. Each number represents the elapsed time,
in milliseconds, for a single work item. There must be one time for each work item.
These numbers are placed into CASs by the job's Job Driver and deliverd to each Job Process.
For example,
@@ -336,17 +336,17 @@ machines = 11
files.
\item[dest.dir] This is the directory that will contain the updated meta-specification
files.
- \item[scheduling.classes] This is a blank-delimeted list of the scheduling classes to
+ \item[scheduling.classes] This is a blank-delimited list of the scheduling classes to
be randomly assigned to the jobs.
\item[scheduling.classes.{[name]}] Here, {\em name} is the name of one of the
scheduling classes listed above. The value is a weight, to be used to affect
the distribution of scheduling classes among the jobs.
- \item[job.memory] This is a blank-delimeted list of memory sizes to be randomly
+ \item[job.memory] This is a blank-delimited list of memory sizes to be randomly
assigned to each job.
\item[job.memory.{[men]}]] Here, {\em mem} is one of the memory sizes specified
above. The value is a weight, used to affect the distribution of memory sizes
among the jobs.
- \item[job.services] This is a blank-delimeted list of a service id, where the id
+ \item[job.services] This is a blank-delimited list of a service id, where the id
is one of the services specified in the {\em services.boot} conrol file.
\item[job.services.{[id]}] Here {\em id} is one of the ids specified in the
job.services line above. The value is a weight, used to affect the distribution