You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by na...@apache.org on 2016/08/09 18:06:54 UTC
svn commit: r1755636 [23/23] - in /samza/site: ./ learn/documentation/0.10/
learn/documentation/0.10/api/ learn/documentation/0.10/api/javadocs/
learn/documentation/0.10/api/javadocs/org/apache/samza/
learn/documentation/0.10/api/javadocs/org/apache/sa...
Modified: samza/site/learn/documentation/0.10/jobs/configuration-table.html
URL: http://svn.apache.org/viewvc/samza/site/learn/documentation/0.10/jobs/configuration-table.html?rev=1755636&r1=1755635&r2=1755636&view=diff
==============================================================================
--- samza/site/learn/documentation/0.10/jobs/configuration-table.html (original)
+++ samza/site/learn/documentation/0.10/jobs/configuration-table.html Tue Aug 9 18:06:53 2016
@@ -184,6 +184,27 @@
</tr>
<tr>
+ <td class="property" id="job-coordinator-monitor-partition-change">job.coordinator.<br />monitor-partition-change</td>
+ <td class="default">false</td>
+ <td class="description">
+ If you are using Kafka for coordinator stream, this configuration enables the Job Coordinator to
+ detect partition count difference in Kafka input topics. On detection, it updates a Gauge
+ metric of format <span class="system">system-name</span>.<span class="stream">stream-name</span>.partitionCount,
+ which indicates the difference in the partition count from the initial state. Please note that currently this
+ feature only works for Kafka-based systems.
+ </td>
+ </tr>
+
+ <tr>
+ <td class="property" id="job-coordinator-monitor-partition-change-frequency-ms">job.coordinator.<br />monitor-partition-change.frequency.ms</td>
+ <td class="default">300000</td>
+ <td class="description">
+ The frequency at which the input streams' partition count change should be detected. This check
+ can be tuned to be pretty low as partition increase is not a common event.
+ </td>
+ </tr>
+
+ <tr>
<td class="property" id="job-config-rewriter-class">job.config.rewriter.<br><span class="rewriter">rewriter-name</span>.class</td>
<td class="default"></td>
<td class="description">
@@ -233,6 +254,106 @@
</tr>
<tr>
+ <td class="property" id="job-systemstreampartition-matcher-class">job.systemstreampartition.matcher.class</td>
+ <td class="default"></td>
+ <td class="description">
+ If you want to enable static partition assignment, then this is a <strong>required</strong> configuration.
+ The value of this property is a fully-qualified Java class name that implements the interface
+ <code>org.apache.samza.system.SystemStreamPartitionMatcher</code>.
+ Samza ships with two matcher classes:
+ <dl>
+ <dt><code>org.apache.samza.system.RangeSystemStreamPartitionMatcher</code></dt>
+ <dd>This classes uses a comma separated list of range(s) to determine which partition matches,
+ and thus statically assigned to the Job. For example "2,3,1-2", statically assigns partition
+ 1, 2, and 3 for all the specified system and streams (topics in case of Kafka) to the job.
+ For config validation each element in the comma separated list much conform to one of the
+ following regex:
+ <ul>
+ <li><code>"(\\d+)"</code> or </li>
+ <li><code>"(\\d+-\\d+)"</code> </li>
+ </ul>
+ <code>JobConfig.SSP_MATCHER_CLASS_RANGE</code> constant has the canonical name of this class.
+ </dd>
+ </dl>
+ <dl>
+ <dt><code>org.apache.samza.system.RegexSystemStreamPartitionMatcher</code></dt>
+ <dd>This classes uses a standard Java supported regex to determine which partition matches,
+ and thus statically assigned to the Job. For example "[1-2]", statically assigns partition 1 and 2
+ for all the specified system and streams (topics in case of Kafka) to the job.
+ <code>JobConfig.SSP_MATCHER_CLASS_REGEX</code> constant has the canonical name of this class.</dd>
+ </dl>
+ </td>
+ </tr>
+
+ <tr>
+ <td class="property" id="job_systemstreampartition_matcher_config_range">job.systemstreampartition.matcher.config.range</td>
+ <td class="default"></td>
+ <td class="description">
+ If <code>job.systemstreampartition.matcher.class</code> is specified, and the value of this property is
+ <code>org.apache.samza.system.RangeSystemStreamPartitionMatcher</code>, then this property is a
+ <strong>required</strong> configuration. Specify a comma separated list of range(s) to determine which
+ partition matches, and thus statically assigned to the Job. For example "2,3,11-20", statically assigns
+ partition 2, 3, and 11 to 20 for all the specified system and streams (topics in case of Kafka) to the job.
+ A singel configuration value like "19" is valid as well. This statically assigns partition 19.
+ For config validation each element in the comma separated list much conform to one of the
+ following regex:
+ <ul>
+ <li><code>"(\\d+)"</code> or </li>
+ <li><code>"(\\d+-\\d+)"</code> </li>
+ </ul>
+ </td>
+ </tr>
+
+ <tr>
+ <td class="property" id="job_systemstreampartition_matcher_config_regex">job.systemstreampartition.matcher.config.regex</td>
+ <td class="default"></td>
+ <td class="description">
+ If <code>job.systemstreampartition.matcher.class</code> is specified, and the value of this property is
+ <code>org.apache.samza.system.RegexSystemStreamPartitionMatcher</code>, then this property is a
+ <strong>required</strong> configuration. The value should be a valid Java supported regex. For example "[1-2]",
+ statically assigns partition 1 and 2 for all the specified system and streams (topics in case of Kakfa) to the job.
+ </td>
+ </tr>
+
+ <tr>
+ <td class="property" id="job_systemstreampartition_matcher_co
+ nfig_job_factory_regex">job.systemstreampartition.matcher.config.job.factory.regex</td>
+ <td class="default"></td>
+ <td class="description">
+ This configuration can be used to specify the Java supported regex to match the <code>StreamJobFactory</code>
+ for which the static partition assignment should be enabled. This configuration enables the partition
+ assignment feature to be used for custom <code>StreamJobFactory</code>(ies) as well.
+ <p>
+ This config defaults to the following value:
+ <code>"org\\.apache\\.samza\\.job\\.local(.*ProcessJobFactory|.*ThreadJobFactory)"</code>,
+ which enables static partition assignment when <code>job.factory.class</code> is set to
+ <code>org.apache.samza.job.local.ProcessJobFactory</code> or <code>org.apache.samza.job.local.ThreadJobFactory.</code>
+ </p>
+ </td>
+ </tr>
+
+
+ <tr>
+ <td class="property" id="job-checkpoint-validation-enabled">job.checkpoint.<br>validation.enabled</td>
+ <td class="default">true</td>
+ <td class="description">
+ This setting controls if the job should fail(true) or just warn(false) in case the validation of checkpoint partition number fails. <br/> <b>CAUTION</b>: this configuration needs to be used w/ care. It should only be used as a work-around after the checkpoint has been auto-created with wrong number of partitions by mistake.
+ </td>
+ </tr>
+ <tr>
+ <td class="property" id="job-security-manager-factory">job.security.manager.factory</td>
+ <td class="default"></td>
+ <td class="description">
+ This is the factory class used to create the proper <a href="../api/javadocs/org/apache/samza/container/SecurityManager.html">SecurityManager</a> to handle security for Samza containers when running in a secure environment, such as Yarn with Kerberos eanbled.
+ Samza ships with one security manager by default:
+ <dl>
+ <dt><code>org.apache.samza.job.yarn.SamzaYarnSecurityManagerFactory</code></dt>
+ <dd>Supports Samza containers to run properly in a Kerberos enabled Yarn cluster. Each Samza container, once started, will create a <a href="../api/javadocs/org/apache/samza/job/yarn/SamzaContainerSecurityManager.html">SamzaContainerSecurityManager</a>. SamzaContainerSecurityManager runs on its separate thread and update user's delegation tokens at the interval specified by <a href="#yarn-token-renewal-interval-seconds" class="property">yarn.token.renewal.interval.seconds</a>. See <a href="../yarn/yarn-security.html">Yarn Security</a> for details.</dd>
+ </dl>
+ </td>
+ </tr>
+
+ <tr>
<th colspan="3" class="section" id="task"><a href="../api/overview.html">Task configuration</a></th>
</tr>
@@ -654,6 +775,10 @@
<dd>Encodes <code>java.lang.String</code> objects as UTF-8.</dd>
<dt><code>org.apache.samza.serializers.JsonSerdeFactory</code></dt>
<dd>Encodes nested structures of <code>java.util.Map</code>, <code>java.util.List</code> etc. as JSON.</dd>
+ <dt><code>org.apache.samza.serializers.LongSerdeFactory</code></dt>
+ <dd>Encodes <code>java.lang.Long</code> as binary (8 bytes fixed-length big-endian encoding).</dd>
+ <dt><code>org.apache.samza.serializers.DoubleSerdeFactory</code></dt>
+ <dd>Encodes <code>java.lang.Double</code> as binray (8 bytes double-precision float point).</dd>
<dt><code>org.apache.samza.serializers.MetricsSnapshotSerdeFactory</code></dt>
<dd>Encodes <code>org.apache.samza.metrics.reporter.MetricsSnapshot</code> objects (which are
used for <a href="../container/metrics.html">reporting metrics</a>) as JSON.</dd>
@@ -1405,6 +1530,32 @@
</tr>
<tr>
+ <td class="property" id="yarn-kerberos-principal">yarn.kerberos.principal</td>
+ <td class="default"></td>
+ <td class="description">
+ Principal the Samza job uses to authenticate itself into KDC, when running on a Kerberos enabled YARN cluster.
+ </td>
+ </tr>
+
+ <tr>
+ <td class="property" id="yarn-kerberos-keytab">yarn.kerberos.keytab</td>
+ <td class="default"></td>
+ <td class="description">
+ The full path to the file containing keytab for the principal, specified by <a href="#yarn-kerberos-principal" class="property">yarn.kerberos.principal</a>.
+ The keytab file is uploaded to the staging directory unique to each application on HDFS and the Application Master then uses the keytab and principal to
+ periodically logs in to recreate the delegation tokens.
+ </td>
+ </tr>
+
+ <tr>
+ <td class="property" id="yarn-token-renewal-interval-seconds">yarn.token.renewal.interval.seconds</td>
+ <td class="default"></td>
+ <td class="description">
+ The time interval by which the Application Master re-authenticates and renew the delegation tokens. This value should be smaller than the length of time a delegation token is valid on hadoop namenodes before expiration.
+ </td>
+ </tr>
+
+ <tr>
<th colspan="3" class="section" id="metrics"><a href="../container/metrics.html">Metrics</a></th>
</tr>
@@ -1518,10 +1669,15 @@
<td class="description">A date format (using Java's SimpleDataFormat syntax) appropriate for use in an HDFS Path, which can configure time-based bucketing of output files.</td>
</tr>
<tr>
- <td class="property" id="hdfs-write-batch-size-bytes">systems.*.producer.write.batch.size.bytes</td>
+ <td class="property" id="hdfs-write-batch-size-bytes">systems.*.producer.hdfs.write.batch.size.bytes</td>
<td class="default">268435456</td>
<td class="description">The number of bytes of outgoing messages to write to each HDFS output file before cutting a new file. Defaults to 256MB if not set.</td>
</tr>
+ <tr>
+ <td class="property" id="hdfs-write-batch-size-records">systems.*.producer.hdfs.write.batch.size.records</td>
+ <td class="default">262144</td>
+ <td class="description">The number of outgoing messages to write to each HDFS output file before cutting a new file. Defaults to 262144 if not set.</td>
+ </tr>
<tr>
<th colspan="3" class="section" id="task-migration">
Modified: samza/site/learn/documentation/0.10/jobs/configuration.html
URL: http://svn.apache.org/viewvc/samza/site/learn/documentation/0.10/jobs/configuration.html?rev=1755636&r1=1755635&r2=1755636&view=diff
==============================================================================
--- samza/site/learn/documentation/0.10/jobs/configuration.html (original)
+++ samza/site/learn/documentation/0.10/jobs/configuration.html Tue Aug 9 18:06:53 2016
@@ -99,6 +99,7 @@
<h1><i class="fa fa-history"></i> Archive</h1>
<ul>
<li><a href="/archive/index.html#latest">latest</a></li>
+ <li><a href="/archive/index.html#10">0.10</a></li>
<li><a href="/archive/index.html#09">0.9</a></li>
<li><a href="/archive/index.html#08">0.8</a></li>
<li><a href="/archive/index.html#07">0.7</a></li>
Modified: samza/site/learn/documentation/0.10/jobs/job-runner.html
URL: http://svn.apache.org/viewvc/samza/site/learn/documentation/0.10/jobs/job-runner.html?rev=1755636&r1=1755635&r2=1755636&view=diff
==============================================================================
--- samza/site/learn/documentation/0.10/jobs/job-runner.html (original)
+++ samza/site/learn/documentation/0.10/jobs/job-runner.html Tue Aug 9 18:06:53 2016
@@ -99,6 +99,7 @@
<h1><i class="fa fa-history"></i> Archive</h1>
<ul>
<li><a href="/archive/index.html#latest">latest</a></li>
+ <li><a href="/archive/index.html#10">0.10</a></li>
<li><a href="/archive/index.html#09">0.9</a></li>
<li><a href="/archive/index.html#08">0.8</a></li>
<li><a href="/archive/index.html#07">0.7</a></li>
Modified: samza/site/learn/documentation/0.10/jobs/logging.html
URL: http://svn.apache.org/viewvc/samza/site/learn/documentation/0.10/jobs/logging.html?rev=1755636&r1=1755635&r2=1755636&view=diff
==============================================================================
--- samza/site/learn/documentation/0.10/jobs/logging.html (original)
+++ samza/site/learn/documentation/0.10/jobs/logging.html Tue Aug 9 18:06:53 2016
@@ -99,6 +99,7 @@
<h1><i class="fa fa-history"></i> Archive</h1>
<ul>
<li><a href="/archive/index.html#latest">latest</a></li>
+ <li><a href="/archive/index.html#10">0.10</a></li>
<li><a href="/archive/index.html#09">0.9</a></li>
<li><a href="/archive/index.html#08">0.8</a></li>
<li><a href="/archive/index.html#07">0.7</a></li>
@@ -175,13 +176,14 @@
<div class="highlight"><pre><code class="bash">-Dsamza.container.name<span class="o">=</span>samza-application-master</code></pre></div>
-<p>These settings are very useful if you’re using a file-based appender. For example, you can use a daily rolling appender by configuring log4j.xml like this:</p>
+<p>These settings are very useful if you’re using a file-based appender. For example, you can use a rolling appender to separate log file when it reaches certain size by configuring log4j.xml like this:</p>
-<div class="highlight"><pre><code class="xml"><span class="nt"><appender</span> <span class="na">name=</span><span class="s">"RollingAppender"</span> <span class="na">class=</span><span class="s">"org.apache.log4j.DailyRollingFileAppender"</span><span class="nt">></span>
+<div class="highlight"><pre><code class="xml"><span class="nt"><appender</span> <span class="na">name=</span><span class="s">"RollingAppender"</span> <span class="na">class=</span><span class="s">"org.apache.log4j.RollingFileAppender"</span><span class="nt">></span>
<span class="nt"><param</span> <span class="na">name=</span><span class="s">"File"</span> <span class="na">value=</span><span class="s">"${samza.log.dir}/${samza.container.name}.log"</span> <span class="nt">/></span>
- <span class="nt"><param</span> <span class="na">name=</span><span class="s">"DatePattern"</span> <span class="na">value=</span><span class="s">"'.'yyyy-MM-dd"</span> <span class="nt">/></span>
+ <span class="nt"><param</span> <span class="na">name=</span><span class="s">"MaxFileSize"</span> <span class="na">value=</span><span class="s">"256MB"</span> <span class="nt">/></span>
+ <span class="nt"><param</span> <span class="na">name=</span><span class="s">"MaxBackupIndex"</span> <span class="na">value=</span><span class="s">"20"</span> <span class="nt">/></span>
<span class="nt"><layout</span> <span class="na">class=</span><span class="s">"org.apache.log4j.PatternLayout"</span><span class="nt">></span>
- <span class="nt"><param</span> <span class="na">name=</span><span class="s">"ConversionPattern"</span> <span class="na">value=</span><span class="s">"%d{yyyy-MM-dd HH:mm:ss} %c{1} [%p] %m%n"</span> <span class="nt">/></span>
+ <span class="nt"><param</span> <span class="na">name=</span><span class="s">"ConversionPattern"</span> <span class="na">value=</span><span class="s">"%d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} [%p] %m%n"</span> <span class="nt">/></span>
<span class="nt"></layout></span>
<span class="nt"></appender></span></code></pre></div>
@@ -210,7 +212,7 @@
<span class="c"><!-- optional --></span>
<span class="nt"><param</span> <span class="na">name=</span><span class="s">"StreamName"</span> <span class="na">value=</span><span class="s">"EpicStreamName"</span><span class="nt">/></span>
<span class="nt"><layout</span> <span class="na">class=</span><span class="s">"org.apache.log4j.PatternLayout"</span><span class="nt">></span>
- <span class="nt"><param</span> <span class="na">name=</span><span class="s">"ConversionPattern"</span> <span class="na">value=</span><span class="s">"%X{containerName} %X{jobName} %X{jobId} %d{yyyy-MM-dd HH:mm:ss} %c{1} [%p] %m%n"</span> <span class="nt">/></span>
+ <span class="nt"><param</span> <span class="na">name=</span><span class="s">"ConversionPattern"</span> <span class="na">value=</span><span class="s">"%X{containerName} %X{jobName} %X{jobId} %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} [%p] %m%n"</span> <span class="nt">/></span>
<span class="nt"></layout></span>
<span class="nt"></appender></span></code></pre></div>
Modified: samza/site/learn/documentation/0.10/jobs/packaging.html
URL: http://svn.apache.org/viewvc/samza/site/learn/documentation/0.10/jobs/packaging.html?rev=1755636&r1=1755635&r2=1755636&view=diff
==============================================================================
--- samza/site/learn/documentation/0.10/jobs/packaging.html (original)
+++ samza/site/learn/documentation/0.10/jobs/packaging.html Tue Aug 9 18:06:53 2016
@@ -99,6 +99,7 @@
<h1><i class="fa fa-history"></i> Archive</h1>
<ul>
<li><a href="/archive/index.html#latest">latest</a></li>
+ <li><a href="/archive/index.html#10">0.10</a></li>
<li><a href="/archive/index.html#09">0.9</a></li>
<li><a href="/archive/index.html#08">0.8</a></li>
<li><a href="/archive/index.html#07">0.7</a></li>
Modified: samza/site/learn/documentation/0.10/jobs/reprocessing.html
URL: http://svn.apache.org/viewvc/samza/site/learn/documentation/0.10/jobs/reprocessing.html?rev=1755636&r1=1755635&r2=1755636&view=diff
==============================================================================
--- samza/site/learn/documentation/0.10/jobs/reprocessing.html (original)
+++ samza/site/learn/documentation/0.10/jobs/reprocessing.html Tue Aug 9 18:06:53 2016
@@ -99,6 +99,7 @@
<h1><i class="fa fa-history"></i> Archive</h1>
<ul>
<li><a href="/archive/index.html#latest">latest</a></li>
+ <li><a href="/archive/index.html#10">0.10</a></li>
<li><a href="/archive/index.html#09">0.9</a></li>
<li><a href="/archive/index.html#08">0.8</a></li>
<li><a href="/archive/index.html#07">0.7</a></li>
Modified: samza/site/learn/documentation/0.10/jobs/web-ui-rest-api.html
URL: http://svn.apache.org/viewvc/samza/site/learn/documentation/0.10/jobs/web-ui-rest-api.html?rev=1755636&r1=1755635&r2=1755636&view=diff
==============================================================================
--- samza/site/learn/documentation/0.10/jobs/web-ui-rest-api.html (original)
+++ samza/site/learn/documentation/0.10/jobs/web-ui-rest-api.html Tue Aug 9 18:06:53 2016
@@ -99,6 +99,7 @@
<h1><i class="fa fa-history"></i> Archive</h1>
<ul>
<li><a href="/archive/index.html#latest">latest</a></li>
+ <li><a href="/archive/index.html#10">0.10</a></li>
<li><a href="/archive/index.html#09">0.9</a></li>
<li><a href="/archive/index.html#08">0.8</a></li>
<li><a href="/archive/index.html#07">0.7</a></li>
Modified: samza/site/learn/documentation/0.10/jobs/yarn-jobs.html
URL: http://svn.apache.org/viewvc/samza/site/learn/documentation/0.10/jobs/yarn-jobs.html?rev=1755636&r1=1755635&r2=1755636&view=diff
==============================================================================
--- samza/site/learn/documentation/0.10/jobs/yarn-jobs.html (original)
+++ samza/site/learn/documentation/0.10/jobs/yarn-jobs.html Tue Aug 9 18:06:53 2016
@@ -99,6 +99,7 @@
<h1><i class="fa fa-history"></i> Archive</h1>
<ul>
<li><a href="/archive/index.html#latest">latest</a></li>
+ <li><a href="/archive/index.html#10">0.10</a></li>
<li><a href="/archive/index.html#09">0.9</a></li>
<li><a href="/archive/index.html#08">0.8</a></li>
<li><a href="/archive/index.html#07">0.7</a></li>
@@ -142,7 +143,7 @@
limitations under the License.
-->
-<p>When you define <code>job.factory.class=org.apache.samza.job.yarn.YarnJobFactory</code> in your job’s configuration, Samza will use YARN to execute your job. The YarnJobFactory will use the HADOOP<em>YARN</em>HOME environment variable on the machine that run-job.sh is executed on to get the appropriate YARN configuration, which will define where the YARN resource manager is. The YarnJob will work with the resource manager to get your job started on the YARN cluster.</p>
+<p>When you define <code>job.factory.class=org.apache.samza.job.yarn.YarnJobFactory</code> in your job’s configuration, Samza will use YARN to execute your job. The YarnJobFactory will use the <code>HADOOP_YARN_HOME</code> environment variable on the machine that run-job.sh is executed on to get the appropriate YARN configuration, which will define where the YARN resource manager is. The YarnJob will work with the resource manager to get your job started on the YARN cluster.</p>
<p>If you want to use YARN to run your Samza job, you’ll also need to define the location of your Samza job’s package. For example, you might say:</p>
Modified: samza/site/learn/documentation/0.10/operations/kafka.html
URL: http://svn.apache.org/viewvc/samza/site/learn/documentation/0.10/operations/kafka.html?rev=1755636&r1=1755635&r2=1755636&view=diff
==============================================================================
--- samza/site/learn/documentation/0.10/operations/kafka.html (original)
+++ samza/site/learn/documentation/0.10/operations/kafka.html Tue Aug 9 18:06:53 2016
@@ -99,6 +99,7 @@
<h1><i class="fa fa-history"></i> Archive</h1>
<ul>
<li><a href="/archive/index.html#latest">latest</a></li>
+ <li><a href="/archive/index.html#10">0.10</a></li>
<li><a href="/archive/index.html#09">0.9</a></li>
<li><a href="/archive/index.html#08">0.8</a></li>
<li><a href="/archive/index.html#07">0.7</a></li>
Modified: samza/site/learn/documentation/0.10/operations/security.html
URL: http://svn.apache.org/viewvc/samza/site/learn/documentation/0.10/operations/security.html?rev=1755636&r1=1755635&r2=1755636&view=diff
==============================================================================
--- samza/site/learn/documentation/0.10/operations/security.html (original)
+++ samza/site/learn/documentation/0.10/operations/security.html Tue Aug 9 18:06:53 2016
@@ -99,6 +99,7 @@
<h1><i class="fa fa-history"></i> Archive</h1>
<ul>
<li><a href="/archive/index.html#latest">latest</a></li>
+ <li><a href="/archive/index.html#10">0.10</a></li>
<li><a href="/archive/index.html#09">0.9</a></li>
<li><a href="/archive/index.html#08">0.8</a></li>
<li><a href="/archive/index.html#07">0.7</a></li>
Modified: samza/site/learn/documentation/0.10/yarn/application-master.html
URL: http://svn.apache.org/viewvc/samza/site/learn/documentation/0.10/yarn/application-master.html?rev=1755636&r1=1755635&r2=1755636&view=diff
==============================================================================
--- samza/site/learn/documentation/0.10/yarn/application-master.html (original)
+++ samza/site/learn/documentation/0.10/yarn/application-master.html Tue Aug 9 18:06:53 2016
@@ -99,6 +99,7 @@
<h1><i class="fa fa-history"></i> Archive</h1>
<ul>
<li><a href="/archive/index.html#latest">latest</a></li>
+ <li><a href="/archive/index.html#10">0.10</a></li>
<li><a href="/archive/index.html#09">0.9</a></li>
<li><a href="/archive/index.html#08">0.8</a></li>
<li><a href="/archive/index.html#07">0.7</a></li>
Modified: samza/site/learn/documentation/0.10/yarn/isolation.html
URL: http://svn.apache.org/viewvc/samza/site/learn/documentation/0.10/yarn/isolation.html?rev=1755636&r1=1755635&r2=1755636&view=diff
==============================================================================
--- samza/site/learn/documentation/0.10/yarn/isolation.html (original)
+++ samza/site/learn/documentation/0.10/yarn/isolation.html Tue Aug 9 18:06:53 2016
@@ -99,6 +99,7 @@
<h1><i class="fa fa-history"></i> Archive</h1>
<ul>
<li><a href="/archive/index.html#latest">latest</a></li>
+ <li><a href="/archive/index.html#10">0.10</a></li>
<li><a href="/archive/index.html#09">0.9</a></li>
<li><a href="/archive/index.html#08">0.8</a></li>
<li><a href="/archive/index.html#07">0.7</a></li>
@@ -166,7 +167,7 @@
<p>See <a href="http://riccomini.name/posts/hadoop/2013-06-14-yarn-with-cgroups/">this blog post</a> for details on setting up YARN with CGroups.</p>
-<h2 id="security-»"><a href="../operations/security.html">Security »</a></h2>
+<h2 id="yarn-security-»"><a href="../yarn/yarn-secuirty.html">YARN Security »</a></h2>
</div>
Modified: samza/site/learn/documentation/0.10/yarn/yarn-host-affinity.html
URL: http://svn.apache.org/viewvc/samza/site/learn/documentation/0.10/yarn/yarn-host-affinity.html?rev=1755636&r1=1755635&r2=1755636&view=diff
==============================================================================
--- samza/site/learn/documentation/0.10/yarn/yarn-host-affinity.html (original)
+++ samza/site/learn/documentation/0.10/yarn/yarn-host-affinity.html Tue Aug 9 18:06:53 2016
@@ -99,6 +99,7 @@
<h1><i class="fa fa-history"></i> Archive</h1>
<ul>
<li><a href="/archive/index.html#latest">latest</a></li>
+ <li><a href="/archive/index.html#10">0.10</a></li>
<li><a href="/archive/index.html#09">0.9</a></li>
<li><a href="/archive/index.html#08">0.8</a></li>
<li><a href="/archive/index.html#07">0.7</a></li>
@@ -144,7 +145,7 @@
<p>In Samza, containers are the units of physical parallelism that runs on a set of machines. Each container is essentially a process that runs one or more stream tasks. Each task instance consumes one or more partitions of the input streams and is associated with its own durable data store. </p>
-<p>We define a <em>Stateful Samza Job</em> as the Samza job that uses a key-value store in its implementation, alone with an associated changelog stream. In stateful samza jobs, there is a 1:1 mapping between the task instance and the data store. Since the allocation of containers to machines in the Yarn cluster is completely left to Yarn, Samza does not guarantee that a container (and hence, its associated task(s)) gets deployed on the same machine. Containers can get shuffled in any of the following cases:</p>
+<p>We define a <em>Stateful Samza Job</em> as the Samza job that uses a key-value store in its implementation, along with an associated changelog stream. In stateful samza jobs, a task may be configured to use multiple stores. For each store there is a 1:1 mapping between the task instance and the data store. Since the allocation of containers to machines in the Yarn cluster is completely left to Yarn, Samza does not guarantee that a container (and hence, its associated task(s)) gets deployed on the same machine. Containers can get shuffled in any of the following cases:</p>
<ol>
<li>When a job is upgraded by pointing <code>yarn.package.path</code> to the new package path and re-submitted.</li>
@@ -154,7 +155,7 @@
<p>In any of the above cases, the task’s co-located data needs to be restored every time a container starts-up. Restoring data each time can be expensive, especially for applications that have a large data set. This behavior slows the start-up time for the job so much that the job is no longer “near realtime”. Furthermore, if multiple stateful samza jobs restart around the same time in the cluster and they all share the same changelog system, then it is possible to quickly saturate the changelog system’s network and cause a DDoS.</p>
-<p>For instance, consider a Samza job performing a Stream-Table join. Typically, such a job requires the dataset to be available on all processors before they begin processing the input stream. The dataset is usually large (order > 1TB) read-only data that will be used to join or add attributes to incoming messages. The job may initialize this cache by populated with data directly from a remote store or changelog stream. This cache initialization happens each time the container is restarted. This causes significant latency during job start-up.</p>
+<p>For instance, consider a Samza job performing a Stream-Table join. Typically, such a job requires the dataset to be available on all processors before they begin processing the input stream. The dataset is usually large (order > 1TB) read-only data that will be used to join or add attributes to incoming messages. The job may initialize this cache by populating it with data directly from a remote store or changelog stream. This cache initialization happens each time the container is restarted. This causes significant latency during job start-up.</p>
<p>The solution, then, is to simply persist the state store on the machine in which the container process is executing and re-allocate the same host for the container each time the job is restarted, in order to re-use the persisted state. Thus, the ability of Samza to allocate a container to the same machine across job restarts is referred to as <strong><em>host-affinity</em></strong>. Samza leverages host-affinity to enhance our support for local state re-use.</p>
@@ -171,20 +172,19 @@ ls <span class="k">${</span><span class=
<p><img src="/img/0.10/learn/documentation/yarn/samza-host-affinity.png" alt="samza-host-affinity"></p>
-<p>When a container is <em>cleanly shutdown</em>, Samza also writes the last materialized offset from the changelog stream to the checksumed file on disk. Thus, there is an <em>OFFSET</em> file associated with each state stores' changelog partitions, that is consumed by the tasks in the container.</p>
+<p>Each time a task commits, Samza writes the last materialized offset from the changelog stream to the checksumed file on disk. This is also done on container shutdown. Thus, there is an <em>OFFSET</em> file associated with each state stores' changelog partitions, that is consumed by the tasks in the container.</p>
<div class="highlight"><pre><code class="bash"><span class="k">${</span><span class="nv">LOGGED_STORE_BASE_DIR</span><span class="k">}</span>/<span class="k">${</span><span class="nv">job</span><span class="p">.name</span><span class="k">}</span>-<span class="k">${</span><span class="nv">job</span><span class="p">.id</span><span class="k">}</span>/<span class="k">${</span><span class="nv">store</span><span class="p">.name</span><span class="k">}</span>/<span class="k">${</span><span class="nv">task</span><span class="p">.name</span><span class="k">}</span>/OFFSET</code></pre></div>
-<p>Now, when a container restarts on the same machine after a clean shutdown and the OFFSET file exists, the Samza container:</p>
+<p>Now, when a container restarts on the same machine after the OFFSET file exists, the Samza container:</p>
<ol>
<li>Opens the persisted store on disk</li>
<li>Reads the OFFSET file</li>
-<li>Deletes the OFFSET file</li>
<li>Restores the state store from the OFFSET value</li>
</ol>
-<p>If the OFFSET file doesn’t exist, it creates the state store and consumes from the oldest offset in the changelog to re-create the state. Note that Samza optimistically deletes the OFFSET file in step 3 to prevent data from getting corrupted due to any kind of failure during state restoration. This significantly reduces the state restoration time on container start-up as we no longer consume from the beginning of the changelog stream.</p>
+<p>This significantly reduces the state restoration time on container start-up as we no longer consume from the beginning of the changelog stream. If the OFFSET file doesn’t exist, it creates the state store and consumes from the oldest offset in the changelog to re-create the state. Since the OFFSET file is written on each commit after flushing the store, the recorded offset is guaranteed to correspond to the current contents of the store or some older point, but never newer. This gives at least once semantics for state restore. Therefore, the changelog entries must be idempotent.</p>
<p>It is necessary to periodically clean-up unused or orphaned state stores on the machines to manage disk-space. This feature is being worked on in <a href="https://issues.apache.org/jira/browse/SAMZA-656">SAMZA-656</a>.</p>
@@ -210,14 +210,14 @@ Without this configuration, the state st
<span class="nt"><value></span>true<span class="nt"></value></span>
<span class="nt"></property></span>
<span class="nt"><property></span>
-<span class="nt"><name></span>yarn.schedular.fair.locality-delay-node-ms<span class="nt"></name></span>
+<span class="nt"><name></span>yarn.scheduler.fair.locality-delay-node-ms<span class="nt"></name></span>
<span class="nt"><description></span>Delay time in milliseconds before relaxing locality at node-level<span class="nt"></description></span>
<span class="nt"><value></span>1000<span class="nt"></value></span> <span class="c"><!-- Should be tuned per requirement --></span>
<span class="nt"></property></span>
<span class="nt"><property></span>
-<span class="nt"><name></span>yarn.schedular.fair.locality-delay-rack-ms<span class="nt"></name></span>
+<span class="nt"><name></span>yarn.scheduler.fair.locality-delay-rack-ms<span class="nt"></name></span>
<span class="nt"><description></span>Delay time in milliseconds before relaxing locality at rack-level<span class="nt"></description></span>
-<span class="nt"><value></span>1000*<span class="nt"></value></span> <span class="c"><!-- Should be tuned per requirement --></span>
+<span class="nt"><value></span>1000<span class="nt"></value></span> <span class="c"><!-- Should be tuned per requirement --></span>
<span class="nt"></property></span></code></pre></div></p></li>
<li><p>Configure Yarn Node Manager SIGTERM to SIGKILL timeout to be reasonable time s.t. Node Manager will give Samza Container enough time to perform a clean shutdown in yarn-site.xml
<div class="highlight"><pre><code class="xml"><span class="nt"><property></span>
@@ -225,13 +225,14 @@ Without this configuration, the state st
<span class="nt"><description></span>No. of ms to wait between sending a SIGTERM and SIGKILL to a container<span class="nt"></description></span>
<span class="nt"><value></span>600000<span class="nt"></value></span> <span class="c"><!-- Set it to 10min to allow enough time for clean shutdown of containers --></span>
<span class="nt"></property></span></code></pre></div></p></li>
+<li><p>The Yarn <a href="https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/RackAwareness.html">Rack Awareness</a> feature is not required and does not change the behavior of Samza Host Affinity. However, if Rack Awareness is configured in the cluster, make sure the DNSToSwitchMapping implementation is robust. Any failures could cause container requests to fall back to the defaultRack. This will cause ContainerRequests to not match the preferred host, which will degrade Host Affinity. For details, see <a href="https://issues.apache.org/jira/browse/SAMZA-886">SAMZA-866</a></p></li>
</ol>
<h2 id="configuring-a-samza-job-to-use-host-affinity">Configuring a Samza job to use Host Affinity</h2>
-<p>Any stateful Samza job can leverage this feature to reduce the Mean Time To Restore (MTTR) of it’s state stores by setting <code>yarn.samza.host-affinity</code> to true.</p>
+<p>Any stateful Samza job can leverage this feature to reduce the Mean Time To Restore (MTTR) of its state stores by setting <code>yarn.samza.host-affinity.enabled</code> to true.</p>
-<div class="highlight"><pre><code class="bash">yarn.samza.host-affinity<span class="o">=</span><span class="nb">true</span> <span class="c"># Default: false</span></code></pre></div>
+<div class="highlight"><pre><code class="bash">yarn.samza.host-affinity.enabled<span class="o">=</span><span class="nb">true</span> <span class="c"># Default: false</span></code></pre></div>
<p>Enabling this feature for a stateless Samza job should not have any adverse effect on the job.</p>
@@ -240,7 +241,7 @@ Without this configuration, the state st
<p>As you have observed, host-affinity cannot be guaranteed all the time due to varibale load distribution in the Yarn cluster. Hence, this is a best-effort policy that Samza provides. However, certain scenarios are worth calling out where these guarantees may be hard to achieve or are not applicable.</p>
<ol>
-<li><em>When the number of containers and/or container-task assignment changes across successive application runs</em> - We may be able to re-use local state for a subset of partitions. Currently, there is no logic in the Job Coordinator to handle partitioning of tasks among containers intelligently. Handling this is more involved as relates to <a href="https://issues.apache.org/jira/browse/SAMZA-336">auto-scaling</a> of the containers.</li>
+<li><em>When the number of containers and/or container-task assignment changes across successive application runs</em> - We may be able to re-use local state for a subset of partitions. Currently, there is no logic in the Job Coordinator to handle partitioning of tasks among containers intelligently. Handling this is more involved as relates to <a href="https://issues.apache.org/jira/browse/SAMZA-336">auto-scaling</a> of the containers. However, with <a href="https://issues.apache.org/jira/browse/SAMZA-906">task-container mapping</a>, this will work better for typical container count adjustments.</li>
<li><em>When SystemStreamPartitionGrouper changes across successive application runs</em> - When the grouper logic used to distribute the partitions across containers changes, the data in the Coordinator Stream (for changelog-task partition assignment etc) and the data stores becomes invalid. Thus, to be safe, we should flush out all state-related data from the Coordinator Stream. An alternative is to overwrite the Task-ChangelogPartition assignment message and the Container Locality message in the Coordinator Stream, before starting up the job again.</li>
</ol>
Added: samza/site/learn/documentation/0.10/yarn/yarn-security.html
URL: http://svn.apache.org/viewvc/samza/site/learn/documentation/0.10/yarn/yarn-security.html?rev=1755636&view=auto
==============================================================================
--- samza/site/learn/documentation/0.10/yarn/yarn-security.html (added)
+++ samza/site/learn/documentation/0.10/yarn/yarn-security.html Tue Aug 9 18:06:53 2016
@@ -0,0 +1,91 @@
+<hr>
+
+<p>layout: page</p>
+
+<h2 id="title:-yarn-security">title: YARN Security</h2>
+
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<p>You can run a Samza job on a secure YARN cluster. YARN uses Kerberos as its authentication and authorization mechanism. See <a href="https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-common/SecureMode.html">this article</a> for details on Hadoop in secure mode.</p>
+
+<h3 id="delegation-token-management-strategy">Delegation token management strategy</h3>
+
+<p>One of the challenges for long-lived application running on a secure YARN cluster is its token renewal strategy. Samza takes the following approach to manage token creation and renewal.</p>
+
+<ol>
+<li><p>Client running Samza app needs to kinit into KDC with his credentials and add the HDFS delegation tokens to the launcher context before submitting the application.</p></li>
+<li><p>Next, client prepares the local resources for the application as follows.
+2.1. First, it creates a staging directory on HDFS. This directory is only accessible by the running user and used to store resources required for Application Master (AM) and Containers.
+2.2. Client then adds the keytab as a local resource in the container launcher context for AM.
+2.3. Finally, it sends the corresponding principal and the path to the keytab file in the staging directory to the coordinator stream. Samza currently uses the staging directory to store both the keytab and refreshed tokens because the access to the directory is secured via Kerberos.</p></li>
+<li><p>Once the resource is allocated for the Application Master, the Node Manager will localizes app resources from HDFS using the HDFS delegation tokens in the launcher context. Same rule applies to Container localization too. </p></li>
+<li><p>When Application Master starts, it localizes the keytab file into its working directory and reads the principal from the coordinator stream.</p></li>
+<li><p>The Application Master periodically re-authenticate itself with the given principal and keytab. In each iteration, it creates new delegation tokens and stores them in the given job specific staging directory on HDFS.</p></li>
+<li><p>Each running container will get new delegation tokens from the credentials file on HDFS before the current ones expire.</p></li>
+<li><p>Application Master and Containers don’t communicate with each other for that matter. Each side proceeds independently by reading or writing the tokens on HDFS.</p></li>
+</ol>
+
+<p>By default, any HDFS delegation token has a maximum life of 7 days (configured by <code>dfs.namenode.delegation.token.max-lifetime</code> in hdfs-site.xml) and the token is normally renewed every 24 hours (configured by <code>dfs.namenode.delegation.token.renew-interval</code> in hdfs-site.xml). What if the Application Master dies and needs restarts after 7 days? The original HDFS delegation token stored in the launcher context will be invalid no matter what. Luckily, Samza can rely on Resource Manager to handle this scenario. See the Configuration section below for details. </p>
+
+<h3 id="components">Components</h3>
+
+<h4 id="securitymanager">SecurityManager</h4>
+
+<p>When ApplicationMaster starts, it spawns <code>SamzaAppMasterSecurityManager</code>, which runs on its separate thread. The <code>SamzaAppMasterSecurityManager</code> is responsible for periodically logging in through the given Kerberos keytab and regenerates the HDFS delegation tokens regularly. After each run, it writes new tokens on a pre-defined job specific directory on HDFS. The frequency of this process is determined by <code>yarn.token.renewal.interval.seconds</code>.</p>
+
+<p>Each container, upon start, runs a <code>SamzaContainerSecurityManager</code>. It reads from the credentials file on HDFS and refreshes its delegation tokens at the same interval.</p>
+
+<h3 id="configuration">Configuration</h3>
+
+<ol>
+<li>For the Samza job, the following job configurations are required on a YARN cluster with security enabled.
+# Job
+job.security.manager.factory=org.apache.samza.job.yarn.SamzaYarnSecurityManagerFactory</li>
+</ol>
+
+<h1 id="yarn">YARN</h1>
+
+<div class="highlight"><pre><code class="properties"><span class="na">yarn.kerberos.principal</span><span class="o">=</span><span class="s">user/localhost</span>
+<span class="na">yarn.kerberos.keytab</span><span class="o">=</span><span class="s">/etc/krb5.keytab.user</span>
+<span class="na">yarn.token.renewal.interval.seconds</span><span class="o">=</span><span class="s">86400</span></code></pre></div>
+
+<ol>
+<li>Configure the Hadoop cluster to enable Resource Manager to recreate and renew the delegation token on behalf of the application user. This will address the following 2 scenarios.</li>
+</ol>
+<div class="highlight"><pre><code class="language-text" data-lang="text">* When Application Master dies unexpectedly and needs a restart after 7 days (the default maximum lifespan a delegation token can be renewed).
+
+* When the Samza job terminates and log aggregation is turned on for the job. Node managers need to be able to upload all the local application logs to HDFS.
+
+1. Enable the resource manager as a privileged user in yarn-site.xml.
+</code></pre></div>
+<div class="highlight"><pre><code class="xml"><span class="nt"><property></span>
+ <span class="nt"><name></span>yarn.resourcemanager.proxy-user-privileges.enabled<span class="nt"></name></span>
+ <span class="nt"><value></span>true<span class="nt"></value></span>
+ <span class="nt"></property></span></code></pre></div>
+<div class="highlight"><pre><code class="language-text" data-lang="text">2. Make `yarn` as a proxy user, in core-site.xml
+</code></pre></div>
+<div class="highlight"><pre><code class="xml"><span class="nt"><property></span>
+ <span class="nt"><name></span>hadoop.proxyuser.yarn.hosts<span class="nt"></name></span>
+ <span class="nt"><value></span>*<span class="nt"></value></span>
+ <span class="nt"></property></span>
+ <span class="nt"><property></span>
+ <span class="nt"><name></span>hadoop.proxyuser.yarn.groups<span class="nt"></name></span>
+ <span class="nt"><value></span>*<span class="nt"></value></span>
+ <span class="nt"></property></span></code></pre></div>
+
+<h2 id="security-»"><a href="../operations/security.html">Security »</a></h2>
Modified: samza/site/learn/tutorials/0.10/deploy-samza-job-from-hdfs.html
URL: http://svn.apache.org/viewvc/samza/site/learn/tutorials/0.10/deploy-samza-job-from-hdfs.html?rev=1755636&r1=1755635&r2=1755636&view=diff
==============================================================================
--- samza/site/learn/tutorials/0.10/deploy-samza-job-from-hdfs.html (original)
+++ samza/site/learn/tutorials/0.10/deploy-samza-job-from-hdfs.html Tue Aug 9 18:06:53 2016
@@ -99,6 +99,7 @@
<h1><i class="fa fa-history"></i> Archive</h1>
<ul>
<li><a href="/archive/index.html#latest">latest</a></li>
+ <li><a href="/archive/index.html#10">0.10</a></li>
<li><a href="/archive/index.html#09">0.9</a></li>
<li><a href="/archive/index.html#08">0.8</a></li>
<li><a href="/archive/index.html#07">0.7</a></li>
Modified: samza/site/learn/tutorials/0.10/deploy-samza-to-CDH.html
URL: http://svn.apache.org/viewvc/samza/site/learn/tutorials/0.10/deploy-samza-to-CDH.html?rev=1755636&r1=1755635&r2=1755636&view=diff
==============================================================================
--- samza/site/learn/tutorials/0.10/deploy-samza-to-CDH.html (original)
+++ samza/site/learn/tutorials/0.10/deploy-samza-to-CDH.html Tue Aug 9 18:06:53 2016
@@ -99,6 +99,7 @@
<h1><i class="fa fa-history"></i> Archive</h1>
<ul>
<li><a href="/archive/index.html#latest">latest</a></li>
+ <li><a href="/archive/index.html#10">0.10</a></li>
<li><a href="/archive/index.html#09">0.9</a></li>
<li><a href="/archive/index.html#08">0.8</a></li>
<li><a href="/archive/index.html#07">0.7</a></li>
Modified: samza/site/learn/tutorials/0.10/index.html
URL: http://svn.apache.org/viewvc/samza/site/learn/tutorials/0.10/index.html?rev=1755636&r1=1755635&r2=1755636&view=diff
==============================================================================
--- samza/site/learn/tutorials/0.10/index.html (original)
+++ samza/site/learn/tutorials/0.10/index.html Tue Aug 9 18:06:53 2016
@@ -99,6 +99,7 @@
<h1><i class="fa fa-history"></i> Archive</h1>
<ul>
<li><a href="/archive/index.html#latest">latest</a></li>
+ <li><a href="/archive/index.html#10">0.10</a></li>
<li><a href="/archive/index.html#09">0.9</a></li>
<li><a href="/archive/index.html#08">0.8</a></li>
<li><a href="/archive/index.html#07">0.7</a></li>
Modified: samza/site/learn/tutorials/0.10/remote-debugging-samza.html
URL: http://svn.apache.org/viewvc/samza/site/learn/tutorials/0.10/remote-debugging-samza.html?rev=1755636&r1=1755635&r2=1755636&view=diff
==============================================================================
--- samza/site/learn/tutorials/0.10/remote-debugging-samza.html (original)
+++ samza/site/learn/tutorials/0.10/remote-debugging-samza.html Tue Aug 9 18:06:53 2016
@@ -99,6 +99,7 @@
<h1><i class="fa fa-history"></i> Archive</h1>
<ul>
<li><a href="/archive/index.html#latest">latest</a></li>
+ <li><a href="/archive/index.html#10">0.10</a></li>
<li><a href="/archive/index.html#09">0.9</a></li>
<li><a href="/archive/index.html#08">0.8</a></li>
<li><a href="/archive/index.html#07">0.7</a></li>
Modified: samza/site/learn/tutorials/0.10/run-hello-samza-without-internet.html
URL: http://svn.apache.org/viewvc/samza/site/learn/tutorials/0.10/run-hello-samza-without-internet.html?rev=1755636&r1=1755635&r2=1755636&view=diff
==============================================================================
--- samza/site/learn/tutorials/0.10/run-hello-samza-without-internet.html (original)
+++ samza/site/learn/tutorials/0.10/run-hello-samza-without-internet.html Tue Aug 9 18:06:53 2016
@@ -99,6 +99,7 @@
<h1><i class="fa fa-history"></i> Archive</h1>
<ul>
<li><a href="/archive/index.html#latest">latest</a></li>
+ <li><a href="/archive/index.html#10">0.10</a></li>
<li><a href="/archive/index.html#09">0.9</a></li>
<li><a href="/archive/index.html#08">0.8</a></li>
<li><a href="/archive/index.html#07">0.7</a></li>
Modified: samza/site/learn/tutorials/0.10/run-in-multi-node-yarn.html
URL: http://svn.apache.org/viewvc/samza/site/learn/tutorials/0.10/run-in-multi-node-yarn.html?rev=1755636&r1=1755635&r2=1755636&view=diff
==============================================================================
--- samza/site/learn/tutorials/0.10/run-in-multi-node-yarn.html (original)
+++ samza/site/learn/tutorials/0.10/run-in-multi-node-yarn.html Tue Aug 9 18:06:53 2016
@@ -99,6 +99,7 @@
<h1><i class="fa fa-history"></i> Archive</h1>
<ul>
<li><a href="/archive/index.html#latest">latest</a></li>
+ <li><a href="/archive/index.html#10">0.10</a></li>
<li><a href="/archive/index.html#09">0.9</a></li>
<li><a href="/archive/index.html#08">0.8</a></li>
<li><a href="/archive/index.html#07">0.7</a></li>
Modified: samza/site/sitemap.xml
URL: http://svn.apache.org/viewvc/samza/site/sitemap.xml?rev=1755636&r1=1755635&r2=1755636&view=diff
==============================================================================
--- samza/site/sitemap.xml (original)
+++ samza/site/sitemap.xml Tue Aug 9 18:06:53 2016
@@ -20,7 +20,7 @@
<url>
<loc>http://samza.apache.org/</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
<changefreq>daily</changefreq>
<priority>1.0</priority>
</url>
@@ -30,357 +30,364 @@
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/yarn/application-master.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/introduction/architecture.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/introduction/background.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/container/checkpointing.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/contribute/code.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/contribute/coding-guide.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/community/committers.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/introduction/concepts.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/jobs/configuration.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/container/coordinator-stream.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/tutorials/versioned/deploy-samza-job-from-hdfs.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/tutorials/versioned/deploy-samza-to-CDH.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/contribute/design-documents.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/container/event-loop.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/index.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/archive/index.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/tutorials/versioned/index.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/index.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/startup/download/index.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/startup/hello-samza/versioned/index.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/comparisons/introduction.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/community/irc.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/yarn/isolation.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/container/jmx.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/jobs/job-runner.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/operations/kafka.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/jobs/logging.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/community/mailing-lists.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/container/metrics.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/comparisons/mupd8.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/api/overview.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/jobs/packaging.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/hdfs/producer.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/contribute/projects.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/tutorials/versioned/remote-debugging-samza.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/jobs/reprocessing.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/contribute/rules.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/tutorials/versioned/run-hello-samza-without-internet.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/tutorials/versioned/run-in-multi-node-yarn.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/container/samza-container.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/operations/security.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/container/serialization.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/comparisons/spark-streaming.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/container/state-management.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/comparisons/storm.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/container/streams.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/contribute/tests.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/jobs/web-ui-rest-api.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/container/windowing.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/yarn/yarn-host-affinity.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
</url>
<url>
<loc>http://samza.apache.org/learn/documentation/versioned/jobs/yarn-jobs.html</loc>
- <lastmod>2016-04-20</lastmod>
+ <lastmod>2016-08-09</lastmod>
+
+
+ </url>
+
+ <url>
+ <loc>http://samza.apache.org/learn/documentation/versioned/yarn/yarn-security.html</loc>
+ <lastmod>2016-08-09</lastmod>
</url>
Modified: samza/site/startup/download/index.html
URL: http://svn.apache.org/viewvc/samza/site/startup/download/index.html?rev=1755636&r1=1755635&r2=1755636&view=diff
==============================================================================
--- samza/site/startup/download/index.html (original)
+++ samza/site/startup/download/index.html Tue Aug 9 18:06:53 2016
@@ -141,7 +141,7 @@
<p>Samza is released as a source artifact, and also through Maven.</p>
-<p>If you just want to play around with Samza for the first time, go to <a href="/startup/hello-samza/latest">Hello Samza</a>.</p>
+<p>If you just want to play around with Samza for the first time, go to <a href="/startup/hello-samza/0.10">Hello Samza</a>.</p>
<h3 id="jdk-notice">JDK Notice</h3>
@@ -223,7 +223,7 @@
<span class="nt"><version></span>0.8.1<span class="nt"></version></span>
<span class="nt"></dependency></span></code></pre></div>
-<p><a href="/startup/hello-samza/latest">Hello Samza</a> is a working Maven project that illustrates how to build projects that have Samza jobs in them.</p>
+<p><a href="/startup/hello-samza/0.10">Hello Samza</a> is a working Maven project that illustrates how to build projects that have Samza jobs in them.</p>
<h4 id="repositories">Repositories</h4>
Modified: samza/site/startup/hello-samza/0.10/index.html
URL: http://svn.apache.org/viewvc/samza/site/startup/hello-samza/0.10/index.html?rev=1755636&r1=1755635&r2=1755636&view=diff
==============================================================================
--- samza/site/startup/hello-samza/0.10/index.html (original)
+++ samza/site/startup/hello-samza/0.10/index.html Tue Aug 9 18:06:53 2016
@@ -99,6 +99,7 @@
<h1><i class="fa fa-history"></i> Archive</h1>
<ul>
<li><a href="/archive/index.html#latest">latest</a></li>
+ <li><a href="/archive/index.html#10">0.10</a></li>
<li><a href="/archive/index.html#09">0.9</a></li>
<li><a href="/archive/index.html#08">0.8</a></li>
<li><a href="/archive/index.html#07">0.7</a></li>
@@ -177,7 +178,7 @@
<div class="highlight"><pre><code class="bash">mvn clean package
mkdir -p deploy/samza
-tar -xvf ./target/hello-samza-0.10.0-dist.tar.gz -C deploy/samza</code></pre></div>
+tar -xvf ./target/hello-samza-0.10.1-SNAPSHOT-dist.tar.gz -C deploy/samza</code></pre></div>
<h3 id="run-a-samza-job">Run a Samza Job</h3>