You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@uima.apache.org by cw...@apache.org on 2013/01/02 20:12:11 UTC
svn commit: r1427917 [2/8] - in
/uima/sandbox/uima-ducc/trunk/uima-ducc-ducbook/docbook: ./ images/
images/ducc-overview/ images/job-manager/ part-admin/ part-admin/admin/
part-introduction/ part-user/ part-user/cli/ unused/
Added: uima/sandbox/uima-ducc/trunk/uima-ducc-ducbook/docbook/part-admin/admin/ducc-properties.xml
URL: http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-ducbook/docbook/part-admin/admin/ducc-properties.xml?rev=1427917&view=auto
==============================================================================
--- uima/sandbox/uima-ducc/trunk/uima-ducc-ducbook/docbook/part-admin/admin/ducc-properties.xml (added)
+++ uima/sandbox/uima-ducc/trunk/uima-ducc-ducbook/docbook/part-admin/admin/ducc-properties.xml Wed Jan 2 19:12:10 2013
@@ -0,0 +1,3733 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+ <!-- ************************************ ducc.properties ********************************** -->
+ <section id="ducc.properties" xreflabel="ducc.properties">
+ <title>ducc.properties</title>
+
+ <para>
+ <emphasis>The source for this chapter is ducc_ducbook/documents/admin/ducc-properties.xml</emphasis>
+ </para>
+
+ <para>
+ The primary configuration file is called <filename>ducc.properties</filename> and
+ always resides in the directory <filename class="directory">ducc_runtime/resources</filename>.
+ </para>
+
+ <para>
+ Some of the properties in <filename>ducc.properties</filename> are intended as
+ the "glue" that brings the various DUCC components together and lets then run as
+ a coherent whole. These types of properties should be modified only by developers
+ of DUCC itself. In the description below these properties are classified as "Private".
+ </para>
+
+ <para>
+ Some of the properties are tuning parameters: timeouts, heartbeat intervals, and so on. These
+ may be modified by DUCC administrators, but only after experience is gained with DUCC, and
+ only to solve specific performance problems. The default tuning parameters have been chosen
+ by the DUCC system developers to provide "best" operation under most reasonable situations.
+ Changing these parameters may create imbalances in the system and result in performance
+ problems or even prevent DUCC from operating at all. In the description below these
+ properties are classified as "Tuning".
+ </para>
+
+ <para>
+ Some of the properties are standard configuration properties: the location of the ActiveMQ
+ broker, the location of the Java JRE, port numbers, etc. These should be modified by the DUCC
+ administrators to configure DUCC to each individual installation. In the description below
+ these properties are classified as "Local".
+ </para>
+
+ <!-- ============================== General DUCC Properties ============================== -->
+ <section>
+ <title>General DUCC Properties</title>
+
+ <variablelist>
+ <!-- ducc.jms.provider -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.jms.provider</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>activemq</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>Declare the type of middleware providing the JMS service used by DUCC.</para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.broker.protocol-->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.broker.protocol</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>tcp</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>Declare the wire protocol used to communicate with ActiveMQ.</para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.broker.hostname-->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.broker.hostname</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>localhost</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Local</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This declares the node name where the ActiveMQ broker resides. It MUST be
+ updated to the actual node where the broker is running as part of
+ DUCC installation. The default value will not work.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.broker.port-->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.broker.port</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>61616</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Local</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This declares the port on which the ActiveMQ broker is listening for messages. It
+ MAY be updated as part of DUCC installation. ActiveMQ ships with port 61616 as the
+ default port, and DUCC uses that default.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.broker.decoration -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.broker.decoration</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>wireFormat.maxInactivityDuration=0</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Local</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ From the ActiveMQ documentation: "The maximum inactivity duration (before which the
+ socket is considered dead) in milliseconds. On some platforms it can take a long
+ time for a socket to appear to die, so we allow the broker to kill connections if
+ they are inactive for a period of time. Use by some transports to enable a keep
+ alive heart beat feature. Set to a value less-than-or-equal0 to disable
+ inactivity monitoring. Declare the wire protocol used to communicate with
+ ActiveMQ."
+ </para>
+ <para>
+ This decoration is used to keep the broker connection alive while a JVM is in a
+ long garbage collection. The applications that DUCC is designed to support can
+ spend significant time in garbage collection, which can cause spurious timeouts.
+ By default the DUCC configuration disables the timeout by setting it to 0.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.broker.name -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.broker.name</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>localhost</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Local</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This is the internal name of the broker, used to locate Broker's MBean in JMX Registry. It is NOT
+ related to any node name. When using the ActiveMQ distribution supplied with DUCC
+ it should always be set to "localhost". When using another broker, this name must
+ match the "brokerName" parameter in the local ActiveMQ configuration.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+
+ <!-- ducc.broker.jmx.port -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.broker.jmx.port</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>1099</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Local</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This is the port used to make JMX connections to the broker. When using the
+ ActiveMQ broker supplied with DUCC this should normally not be changed. If using
+ another ActiveMQ broker this must match the configured JMX port.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.cluster.name -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.cluster.name</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>Welcome To DUCC!</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Local</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This is a string used in the Web Server banner to identify the local cluster. It
+ may be set to anything desired.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.runmode -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.runmode</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>unconfigured.</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Local</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ When set to "Test" this property bypasses userid and authentication checks. It is intended
+ for use ONLY by DUCC developers. It allows developers of DUCC to simulate a multi-user
+ environment without the need for root privileges.
+ </para>
+ <note>
+ <para>
+ WARNING! Enabling this feature in a production DUCC system is a serious security breach. It
+ should only be set by DUCC developers running with an un-privileged <emphasis>ducc_ling.</emphasis>
+ </para>
+ </note>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.locale.language -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.locale.language</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>en</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ Establish the language for national language support of messages. Currently only
+ "en" is supported.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.locale.country -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.locale.country</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>us</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ Establish the country for National Language Support of messages. Currently only
+ "us" is supported.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+
+ <!-- ducc.jvm -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.jvm</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>java</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Local</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ Specifies the full path to the JVM to be used by the DUCC processes. If not specified,
+ "java" must be in the default path for user "ducc".
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.jmx.port -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.jmx.port</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>2099</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ Every process started by DUCC has JMX enabled by default. When more than one process
+ runs on the same machine this can cause port conflicts. The property "ducc.jmx.port" is
+ used as the base port for JMX. If the port is busy, it is incremented internally until a
+ free port is found.
+ </para>
+ <para>
+ The web server's "System -> Daemons" tab is used to find the JMX URL that gets assigned to
+ each of the DUCC management processes. The web server's job details page for each job
+ is used to find the JMX URL that is assigned to each JP.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+
+ <!-- ducc.agent.jvm.args -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.agent.jvm.args</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>Xmx100M</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This specifies the list of arguments passed to the JVM when spawing the Agent.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.orchestrator.jvm.args -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.orchestrator.jvm.args</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>Xmx1G</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This specifies the list of arguments passed to the JVM when spawing the Orchestrator.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.rm.jvm.args -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.rm.jvm.args</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>Xmx1G</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This specifies the list of arguments passed to the JVM when spawing the Resource Manager.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.pm.jvm.args -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.agent.jvm.args</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>Xmx1G</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This specifies the list of arguments passed to the JVM when spawing the Process Manager.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.sm.jvm.args -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.sm.jvm.args</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>Xmx1G</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This specifies the list of arguments passed to the JVM when spawing the Service Manager.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.ws.jvm.args -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.ws.jvm.args</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>Xmx8G</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This specifies the list of arguments passed to the JVM when spawing the Webserver.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+
+
+ <!-- ducc.admin.endpoint -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.admin.endpoint</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>ducc.admin.channel</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>This is the JMS endpoint name used for DUCC administration messages.</para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.admin.endpoint.type -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.admin.endpoint.type</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>topic</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This is the JMS message type used for DUCC administration requests. If changed
+ DUCC admin may not work.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+
+
+
+
+ <!-- ducc.submit.threads.limit -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.submit.threads.limit</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>(unconfigured)</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Local</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This enforces a maximum number of threads per job, amortized over all the
+ processes. No job will have more threads than this dispatched. This limit is
+ disabled by default.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+
+ <!-- ducc.submit.driver.jvm.args -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.driver.jvm.args</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>(unconfigured)</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Local</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ If enabled, the arguments here are automatically added to the JVM arguments
+ specified for the Job Driver process.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+
+ <!-- ducc.submit.process.jvm.args -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.process.jvm.args</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>(unconfigured)</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ If enabled, the arguments here are added by DUCC to the JVM arguments in the user's
+ job processes.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+
+ <!-- ducc.cli.httpclient.sotimeout -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.cli.httpclient.sotimeout</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>0</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This is the timeout used by the CLI to communicate with DUCC, in millisseconds. If no response
+ is heard within this time, the request times out and is aborted. When set to 0 (the default),
+ the request never times out.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.signature.required -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.signature.required</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>on</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ When set, the CLI signs each request so the Orchestrator can be sure the requestor is actually
+ who he claims to be.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+
+
+
+ </variablelist>
+ </section>
+
+ <!-- ============================== Web Server Properties============================== -->
+ <section>
+ <title>Web Server Properties</title>
+ <variablelist>
+ <!-- ducc.ws.configuration.class -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.ws.configuration.class</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>org.apache.uima.ducc.ws.config.WebServerConfiguration</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>The name of the pluggable java class used to implement the Web Server.</para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+
+ <!-- ducc.ws.node -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.ws.node</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>(unconfigured)</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Local</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This is the name of the node the web server is started on. If not specified, the
+ web server is started on the node where <emphasis>start_ducc</emphasis> is run.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+
+ <!-- ducc.ws.ipaddress -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.ws.ipaddress</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>(unconfigured)</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Local</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ In multi-homed systems it may be necessary to specify to which of the multiple
+ addresses the Web Server listens for requests. This property is an IP address that
+ specifies to which address the Web Server listens.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+
+ <!-- ducc.ws.port -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.ws.port</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>42133</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Local</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>This is the port on which the DUCC Web Server listens for requests.</para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+
+ <!-- ducc.ws.port.ssl -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.ws.port.ssl</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>42155</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Local</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>This is the port that the Web Server uses for SSL requests (such as authentication).</para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+
+ <!-- ducc.ws.port.ssl.pw -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.ws.port.ssl.pw</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>quackquack</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Local</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>This is the SSL password used for SSL requests.</para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+
+ <!-- ducc.ws.session.minutes -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.ws.session.minutes</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>60</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Local</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ Once authenticated, this property determines the lifetime of the authenticated session to
+ the Web Server.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+
+ <!-- ducc.ws.max.history.entries -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.ws.max.history.entries</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>200</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Local</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ The Web Server maintains a history of jobs over time. To avoid overloading the
+ system with data about old and obsolete jobs it prunes the history. This property
+ determines the size of the history that is kept.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </section>
+
+ <!-- ============================== Job Driver Properties ============================== -->
+ <section>
+ <title>Job Driver Properties</title>
+ <variablelist>
+
+ <!-- ducc.jd.configuration.class -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.jd.configuration.class</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>org.apache.uima.ducc.jd.config.JobDriverConfiguration</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>The name of the pluggable java class used to implement the Job Driver.</para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.jd.state.update.endpoint -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.jd.state.update.endpoint</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>ducc.jd.state</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>
+ This is the JMS endpoint name by the Job Driver to send state to the Orchestrator.
+ </para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.jd.state.update.endpoint.type -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.jd.state.update.endpoint.type</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>topic</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>This is the JMS message type used to send state to the Orchestrator.</para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.jd.state.publish.rate -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.jd.state.publish.rate</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>15000</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>
+ The frequency in milliseconds that JD publishes its state to the Orchestrator. A higher rate
+ may slightly increase system response but will increase network load. A lower rate will
+ somewhat decrease system response and lower network load.
+ </para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.jd.queue.prefix -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.jd.queue.prefix</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>ducc.jd.queue. </para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>
+ This is a human-readable string used to form queue names for the JMS queues
+ used to pass CASs from the Job Driver to the Job Processes.
+ </para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.jd.host.class -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.jd.host.class</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>JobDriver</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>
+ This is the scheduling class used to request a reservation from the Resource Manager
+ for the machine that will be used to run the Job Driver processes. This class must
+ also be configured in <emphasis>ducc.classes</emphasis> with scheduling policy
+ <emphasis>RESERVE</emphasis>.
+ </para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.jd.host.description -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.jd.host.description</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>Job Driver</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>
+ This is a name to be associated with the reservation that is made for the Job
+ Driver Node. It can be any string and is displayed in
+ the <emphasis>Reservations</emphasis> page on the Web Server.
+ </para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.jd.memory.size -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.jd.memory.size</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>8GB</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>
+ This is the amount of memory that is requested in the Job Driver reservation. It is used
+ in conjunction with the configuration of the class specified for the job driver (by default,
+ <emphasis>JobDriver</emphasis>) to schedule a node. The default configuration for this
+ class uses a node pool instead of memory to allocate the Job Driver node so by default,
+ this parameter is ignored.
+ </para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.jd.number.of.machines -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.jd.number.of.machines</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>1</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This is the number of machines to request for Job Driver nodes. This may be increased if
+ there are many jobs in the system and the load on the JD node is high enough to slow the
+ JD processes.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.jd.host.user -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.jd.host.user</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>System</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>
+ This is the userid that is associated with the Job Driver reservation. It does
+ not need to be a "real" userid as the actual owner of the reservation is user
+ "ducc". It is primarily used as annotation of the reservation in the Web Server
+ and logs.
+ </para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </section>
+
+ <!-- ============================== Service Manager Properties ============================== -->
+ <section>
+ <title>Service Manager Properties</title>
+ <variablelist>
+
+ <!-- ducc.sm.configuration.class -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.sm.configuration.class</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>org.apache.uima.ducc.sm.config.JobDriverConfiguration</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>This is the name of the pluggable java class used to implement the Service Manager.</para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.sm.state.update.endpoint -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.sm.state.update.endpoint</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>ducc.sm.state</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>This is the JMS endpoint name used for state messages sent by the Service Manager.</para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.sm.state.update.endpoint.type -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.sm.state.update.endpoint.type</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>topic</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>This is the JMS message type used for state messages sent by the Service Manager.</para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.sm.meta.ping.rate -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.sm.meta.ping.rate</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>60000</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>This is the time, in milliseconds, between pings by the Servic Manager to each known, running service.</para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.sm.meta.ping.stability -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.sm.meta.ping.stability</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>10</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This is the number consecutive pings that may be missed before a service is
+ considered unavailable.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.sm.meta.ping.timeout -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.sm.meta.ping.timeout</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>5000</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This is the time in milliseconds the SM waits for a response to a ping. If the
+ service does not respond within this time the ping is accounted for as a "missed" ping.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+
+ <!-- ducc.sm.http.port -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.sm.http.port</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>19989</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Local</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This is the HTTP port used by the SM to field requests from the CLI / API.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.sm.http.port -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.sm.http.node</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>localhost</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Local</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This is the node where the service manager runs. It MUST be configured as part of
+ DUCC setup.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.sm.default.linger -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.sm.default.linger</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>300</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This is the length of time, in seconds, that the SM allows a service to remain
+ alive after all referenceing jobs have exited. If no new job enters the system
+ by the time this time has expired, the SM stops the service.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+
+ </variablelist>
+ </section>
+
+ <!-- ============================== Orchestrator Properties ============================== -->
+ <section>
+ <title>Orchestrator Properties</title>
+ <variablelist>
+
+ <!-- ducc.orchestrator.configuration.class -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.orchestrator. configuration.class</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>
+ org.apache.uima.ducc.orchestrator.config.OrchestratorConfiguration
+ </para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>This is the name of the pluggable java class used to implement the DUCC Orchestrator.</para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+
+ <!-- ducc.orchestrator.checkpoint -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.orchestrator.checkpoint</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>on</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>
+ This controls Orchestrator state checkpointing. If set off, no state is saved
+ across restarts of the Orchestrator except for the current job numbering. This
+ should generally be left <emphasis>on</emphasis>.
+ </para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.orchestrator.start.type -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.orchestrator.start.type</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>warm</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This indicates the level of recovery to be taken on restarting a system. In general,
+ if DUCC is fully shutdown, only cold and warm starts make sense because the
+ Job Processes and Job Drivers are terminated during the shutdown. However if
+ a management process died or was terminated by the administrators, most work
+ can be recovered without interruption, allowing for a hot start.
+
+ There are three level of startup:
+ <itemizedlist>
+ <listitem><para>
+ <emphasis role="bold">cold</emphasis>. All reservations are canceled, all
+ currently running jobs (if any) are terminated. All services are terminated.
+ The system starts with no jobs, reservations, or services active.
+ </para></listitem>
+ <listitem><para>
+ <emphasis role="bold">warm</emphasis>. All reservations are restored. All
+ currently running jobs (if any) are terminated. All services are started
+ or restarted as indicated by their state when the system went down. The
+ system starts with no jobs active, but reservations and services are
+ preserved.
+ </para></listitem>
+ <listitem><para>
+ <emphasis role="bold">hot</emphasis>. All reservations are restored. The
+ system attempts to reattach to all jobs that are still running. The system
+ attempts to reattach to any services that are still running. Any services that
+ need to be restarted are restarted.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.orchestrator.state.update.endpoint -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.orchestrator.state.endpoint</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>ducc.orchestrator.state</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This is the name of the JMS endpoint through which the Orchestrator
+ broadcasts its full state messages. These messages include full job
+ information and can be large. This state is used by the Process
+ Manager and the Webserver.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.orchestrator.state.update.endpoint.type -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.orchestrator.state. update.endpoint.type</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>topic</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>
+ This is the JMS endpoint type used for the "full" state messages sent by the Orchestrator.
+ </para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.orchestrator.state.publish.rate -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.orchestrator.state.publish.rate</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>15000</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>
+ This is the frequency in milliseconds that the Orchestrator publishes its
+ non-abbreviated state.
+ </para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.orchestrator.abbreviated.state.update.endpoint -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.orchestrator.abbreviated. state.endpoint</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>ducc.orchestrator.abbreviated.state</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>
+ This is the name of the JMS endpoint through which the Orchestrator
+ broadcasts its abbreviated state. This state state is used by the
+ Resource Manager and Service Manager.
+ </para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.orchestrator.abbreviated.state.update.endpoint.type -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.orchestrator.abbreviated. state.update.endpoint.type</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>topic</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>
+ This is the JMS endpoint type used for the "abbreviated" state messages sent by the Orchestrator.
+ </para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.orchestrator.abbreviated.state.publish.rate -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.orchestrator.abbreviated. state.publish.rate</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>15000</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>
+ This is the frequency in milliseconds that the Orchestrator publishes its
+ abbreviated state.
+ </para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.orchestrator.maintenance.rate -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.orchestrator.maintenance.rate</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>60000</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>
+ This is the frequency in milliseconds that the Orchestrator checks and updates
+ history and state.
+ </para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+
+ <!-- ducc.orchestrator.maintenance.rate -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.orchestrator.job.factory.classpath.order</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>user-before-ducc</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ When the DUCC Agent spawns a process it must set the process's Java CLASSPATH. This
+ CLASSPATH must contain a minimum set of entries, which are supplied by the Agent. However,
+ users may want their own CLASSPATH to take precedence; for example, they may have a different
+ version of some <emphasis>.jar</emphasis> file. In this case the user's CLASSPATH should
+ be set before DUCC's. To control this, set this tuning parameter to one of two values:
+ <itemizedlist>
+ <listitem><para>user-before-ducc</para></listitem>
+ <listitem><para>ducc-before-user</para></listitem>
+ </itemizedlist>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+
+ </variablelist>
+ </section>
+
+ <!-- ============================== Resource Manager Properties ============================== -->
+ <section>
+ <title>Resource Manager Properties</title>
+ <variablelist>
+
+ <!-- ducc.rm.configuration.class -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.rm.configuration.class</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>org.apache.uima.ducc.rm.config.ResourceManagerConfiguration</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>This is the name of the pluggable java class used to implement the DUCC Resource Manager.</para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.rm.state.update.endpoint -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.rm.state.update.endpoint</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>ducc.rm.state</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>
+ This is the name of the JMS endpoint through which the Resource
+ Manager broadcasts its abbreviated state.
+ </para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.rm.state.update.endpoint.type -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.rm.state.update.endpoint.type</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>topic</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem><para>
+ This is the JMS endpoint type used for state messages sent by the Resource Manager..
+ </para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.rm.state.publish.ratio -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.rm.state.publish.rate</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>60000</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This is the rate, in milliseconds, at which the Resource Manager publishes its
+ state to the Orchestrator.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.rm.share.quantum -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.rm.share.quantum</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>15</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ The <emphasis>share quantum</emphasis> is the smallest amount of RAM that is
+ schedulable for jobs, in GB. Jobs are scheduled based entirely on their memory
+ requirements. Memory is allocated in multiples of the <emphasis>share quantum.</emphasis>
+ </para>
+ <para>
+ The job's declared <emphasis>process_memory_size</emphasis> is used to determine the
+ overall memory requirements in terms of share quanta according to the formula:
+ physical_requirement = ciel(process_memory_size / share_quantum) * share_quantum.
+ </para>
+ <para>
+ For example suppose a process declares it's memory requirement to be 20GB. Then
+ physical_requirement = ciel(20 / 15) * 15 = 2 * 15 = 30 GB. The processes for this job are
+ scheduled only on machines with at least 30 GB of reported RAM, and the Resource Manager
+ insures that no other processes are scheduled on the machine that might encroach on
+ this 30 GB.
+ </para>
+ <para>
+ The <emphasis>share quantum</emphasis> is also used to determine each user's
+ <emphasis>fair share</emphasis> of the resources. The scheduler's goal is to
+ ensure that all user's are allocated the same number of quantum shares.
+ Conceptually, the total memory in the system is divided by the <emphasis>share
+ quantum</emphasis> and then allocated in equal portions to all users in the
+ system.
+ </para>
+ <para>
+ Thus, jobs that require less memory will generally have more processes scheduled than
+ jobs that require more memory, but the total memory scheduled is approximately the
+ same for all jobs.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.rm.scheduler -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.rm.scheduler</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>org.apache.uima.ducc.rm.scheduler.NodepoolScheduler</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Private</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ The component that implements the scheduling algorithm is pluggable. This specifies the name of
+ that class.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.rm.class.definitions -->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.rm.class.definitions</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>ducc.classes</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ This specifies the name of the file that contains the site's class definitions.
+ This file is described in detail the section on <xref linkend="ducc.properties"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.rm.default.tasks-->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.rm.default.tasks</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>10</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ In order to calculate the number of processes to allocate to a job, the
+ scheduler must know how many tasks or <emphasis>work items</emphasis> the job
+ will execute. If the job does not declare that
+ number, <emphasis>default.tasks</emphasis> is used.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+
+ </varlistentry>
+
+ <!-- ducc.rm.default.memory-->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.rm.default.memory</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>15</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ If a job does not declare the amount of memory each process requires, the
+ scheduler uses <emphasis>default.memory</emphasis> for scheduling. The unit
+ is GB.
+ </para>
+ <para>
+ Note that the Agents enforce the declared memory, so if a process understates
+ its requirements it will generally be killed.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.rm.default.threads-->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.rm.default.threads</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>4</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>
+ Tuning
+ </para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ Each job process will be dispatched with some number of threads such that DUCC will
+ dispatch <emphasis>work items</emphasis> to these threads. The scheduler uses this
+ number to calculate the number of processes that must be allocated.
+ </para>
+ <para>
+ The maximum number of processes a job requites is determined by the formula:
+ num_processes = ciel(num_work_items / num_threads).
+ </para>
+ <para>
+ Thus, a job that declares 100 work items and 4 threads is assigned
+ a maximum of ciel(100/4) = 25 processes.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.rm.node.stability-->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.rm.node.stability</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>5</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ The RM receives regular "heartbeats" from the DUCC agents in order to know
+ what nodes are available for scheduling. The <emphasis>node.stability</emphasis>
+ property configures the number of consecutive heartbeats that may be missed before
+ the Resource Manager considers the node to be inoperative.
+ </para>
+ <para>
+ If a node becomes inoperative, the Resource Manager deallocates all processes on
+ that node and attempts to reallocate them on other nodes. The node is marked offline
+ and is unusable until its heartbeats start up again.
+ </para>
+ <para>
+ The default configuration declares the agent heartbeats to occur at 1 minute
+ intervals. Therefore heartbeats must be missed for five minutes before the
+ Resource Manager takes corrective action.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.rm.init.stability-->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.rm.init.stability</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>3</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ During DUCC initialization the Resource Manager must wait some period of time
+ for all the nodes in the cluster to check-in via their "heartbeats". If the
+ RM were to start scheduling too soon there would be a period of significant
+ "churn" as the perceived cluster configurations changes rapidly. As well,
+ it would be impossible to recover work in a <emphasis>warm or hot</emphasis>
+ start if the affected nodes had not yet checked in.
+ </para>
+ <para>
+ The <emphasis>init.stability</emphasis> property indicates how many heartbeat
+ intervals the RM must wait before it starts scheduling after initialization.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.rm.eviction.policy-->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.rm.eviction.policy</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>SHRINK_BY_INVESTMENT</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
+ The alternative value is SHRINK_BY_MACHINE.
+ </para>
+ <para>
+ The <emphasis>eviction.policy</emphasis> is a heuristic to choose which
+ processes of a job to preempt because of competition from other jobs.
+ </para>
+ <para>
+ The SHRINK_BY_INVESTMENT policy attempts to preempt processes such that
+ the least amount of work is lost. It chooses candidates for eviction
+ in order of:
+ <orderedlist>
+ <listitem><para>
+ Processes still initializing, with the smallest time spent
+ in the initializing step.
+ </para></listitem>
+ <listitem><para>
+ Processes whose currently active work items have been executing for the
+ shortest time.
+ </para></listitem>
+ </orderedlist>
+ </para>
+ <para>
+ The SHRINK_BY_MACHINE policy attempts to preempt processes so as to
+ minimize fragmentation on machines with large memories that can
+ contain multiple job processes. No consideration of execution time
+ or initialization time is made.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <!-- ducc.rm.initialization.cap-->
+ <varlistentry>
+ <term><emphasis role="bold">ducc.rm.initialization.cap</emphasis></term>
+ <listitem>
+
+ <variablelist>
+ <varlistentry>
+ <term>Default Value</term>
+ <listitem><para>2</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Type</term>
+ <listitem><para>Tuning</para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Purpose</term>
+ <listitem>
+ <para>
[... 1223 lines stripped ...]