You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by ja...@apache.org on 2019/02/01 19:41:53 UTC
svn commit: r1852770 [13/14] - in /samza/site: ./ archive/ blog/
case-studies/ community/ contribute/ learn/documentation/1.0.0/
learn/documentation/1.0.0/api/ learn/documentation/1.0.0/api/javadocs/
learn/documentation/1.0.0/api/javadocs/org/apache/sa...
Modified: samza/site/releases/1.0.0.html
URL: http://svn.apache.org/viewvc/samza/site/releases/1.0.0.html?rev=1852770&r1=1852769&r2=1852770&view=diff
==============================================================================
--- samza/site/releases/1.0.0.html (original)
+++ samza/site/releases/1.0.0.html Fri Feb 1 19:41:48 2019
@@ -81,7 +81,7 @@
</div>
<div class="main-navigation__items" data-menu-opened>
<a class="main-navigation__item" href="/">Home</a>
- <a class="main-navigation__item" href="/learn/documentation/latest/core-concepts/core-concepts.html">Docs</a>
+ <a class="main-navigation__item" href="/learn/documentation/1.0.0/core-concepts/core-concepts.html">Docs</a>
<a class="main-navigation__item" href="/powered-by/">Powered By</a>
<a class="main-navigation__item" href="/startup/download/">Downloads</a>
<a class="main-navigation__item" href="/blog/">Blog</a>
@@ -152,10 +152,10 @@
- <a class="side-navigation__group-item" data-match-active="" href="/startup/quick-start/latest/">QuickStart</a>
+ <a class="side-navigation__group-item" data-match-active="" href="/startup/quick-start/1.0.0/">QuickStart</a>
- <a class="side-navigation__group-item" data-match-active="" href="/startup/code-examples/latest/">Code Examples</a>
+ <a class="side-navigation__group-item" data-match-active="" href="/startup/code-examples/1.0.0/">Code Examples</a>
@@ -187,7 +187,7 @@
<i class="side-navigation__group-title-icon icon ion-md-arrow-dropdown"></i>
Documentation
</div>
- <div class="side-navigation__group-items side-navigation__group-has-submenus" data-sub-menu data-documentation="/learn/documentation/latest/">
+ <div class="side-navigation__group-items side-navigation__group-has-submenus" data-sub-menu data-documentation="/learn/documentation/1.0.0/">
<!-- Handle sub navigation items from data -->
@@ -958,7 +958,7 @@ make a change, since it now only has a s
<div class="footer__items">
<a class="footer__item" href="/meetups/">Meetups</a>
<a class="footer__item" href="/blog/">Blog</a>
- <a class="footer__item" href="/learn/documentation/latest/introduction/background.html">About</a>
+ <a class="footer__item" href="/learn/documentation/1.0.0/introduction/background.html">About</a>
</div>
</div>
<div>
Modified: samza/site/releases/older.html
URL: http://svn.apache.org/viewvc/samza/site/releases/older.html?rev=1852770&r1=1852769&r2=1852770&view=diff
==============================================================================
--- samza/site/releases/older.html (original)
+++ samza/site/releases/older.html Fri Feb 1 19:41:48 2019
@@ -81,7 +81,7 @@
</div>
<div class="main-navigation__items" data-menu-opened>
<a class="main-navigation__item" href="/">Home</a>
- <a class="main-navigation__item" href="/learn/documentation/latest/core-concepts/core-concepts.html">Docs</a>
+ <a class="main-navigation__item" href="/learn/documentation/1.0.0/core-concepts/core-concepts.html">Docs</a>
<a class="main-navigation__item" href="/powered-by/">Powered By</a>
<a class="main-navigation__item" href="/startup/download/">Downloads</a>
<a class="main-navigation__item" href="/blog/">Blog</a>
@@ -152,10 +152,10 @@
- <a class="side-navigation__group-item" data-match-active="" href="/startup/quick-start/latest/">QuickStart</a>
+ <a class="side-navigation__group-item" data-match-active="" href="/startup/quick-start/1.0.0/">QuickStart</a>
- <a class="side-navigation__group-item" data-match-active="" href="/startup/code-examples/latest/">Code Examples</a>
+ <a class="side-navigation__group-item" data-match-active="" href="/startup/code-examples/1.0.0/">Code Examples</a>
@@ -187,7 +187,7 @@
<i class="side-navigation__group-title-icon icon ion-md-arrow-dropdown"></i>
Documentation
</div>
- <div class="side-navigation__group-items side-navigation__group-has-submenus" data-sub-menu data-documentation="/learn/documentation/latest/">
+ <div class="side-navigation__group-items side-navigation__group-has-submenus" data-sub-menu data-documentation="/learn/documentation/1.0.0/">
<!-- Handle sub navigation items from data -->
@@ -561,7 +561,7 @@
<div class="footer__items">
<a class="footer__item" href="/meetups/">Meetups</a>
<a class="footer__item" href="/blog/">Blog</a>
- <a class="footer__item" href="/learn/documentation/latest/introduction/background.html">About</a>
+ <a class="footer__item" href="/learn/documentation/1.0.0/introduction/background.html">About</a>
</div>
</div>
<div>
Modified: samza/site/startup/code-examples/1.0.0/index.html
URL: http://svn.apache.org/viewvc/samza/site/startup/code-examples/1.0.0/index.html?rev=1852770&r1=1852769&r2=1852770&view=diff
==============================================================================
--- samza/site/startup/code-examples/1.0.0/index.html (original)
+++ samza/site/startup/code-examples/1.0.0/index.html Fri Feb 1 19:41:48 2019
@@ -546,6 +546,72 @@ These include:</p>
<li><p><a href="https://github.com/apache/samza-hello-samza/tree/master/src/main/java/samza/examples/kinesis">Amazon Kinesis</a> and <a href="https://github.com/apache/samza-hello-samza/tree/latest/src/main/java/samza/examples/azure">Azure Eventhubs</a> examples that cover how to consume input data from the respective systems.</p></li>
</ul>
+<h4 id="low-level-api-examples">Low-level API examples</h4>
+
+<p>The <a href="https://github.com/apache/samza-hello-samza/tree/latest/src/main/java/samza/examples/wikipedia/task/application">Wikipedia Parser (low-level API)</a>:
+Same example that builds a streaming pipeline consuming a live-feed of
+wikipedia edits, parsing each message and generating statistics from them, but
+using low-level APIs. </p>
+
+<h4 id="samza-sql-api-examples">Samza SQL API examples</h4>
+
+<p>You can easily create a Samza job declaratively using
+<a href="https://samza.apache.org/learn/tutorials/0.14/samza-sql.html">Samza SQL</a>.</p>
+
+<h4 id="apache-beam-api-examples">Apache Beam API examples</h4>
+
+<p>The easiest way to get a copy of the WordCount examples in Beam API is to use <a href="http://maven.apache.org/download.cgi">Apache Maven</a>. After installing Maven, please run the following command:</p>
+
+<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>> mvn archetype:generate <span class="se">\</span>
+ -DarchetypeGroupId<span class="o">=</span>org.apache.beam <span class="se">\</span>
+ -DarchetypeArtifactId<span class="o">=</span>beam-sdks-java-maven-archetypes-examples <span class="se">\</span>
+ -DarchetypeVersion<span class="o">=</span><span class="m">2</span>.9.0 <span class="se">\</span>
+ -DgroupId<span class="o">=</span>org.example <span class="se">\</span>
+ -DartifactId<span class="o">=</span>word-count-beam <span class="se">\</span>
+ -Dversion<span class="o">=</span><span class="s2">"0.1"</span> <span class="se">\</span>
+ -Dpackage<span class="o">=</span>org.apache.beam.examples <span class="se">\</span>
+ -DinteractiveMode<span class="o">=</span><span class="nb">false</span></code></pre></figure>
+
+<p>This command creates a maven project <code>word-count-beam</code> which contains a series of example pipelines that count words in text files:</p>
+
+<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>> <span class="nb">cd</span> word-count-beam/
+
+> ls src/main/java/org/apache/beam/examples/
+DebuggingWordCount.java WindowedWordCount.java common
+MinimalWordCount.java WordCount.java</code></pre></figure>
+
+<p>To use SamzaRunner, please add the following <code>samza-runner</code> profile to <code>pom.xml</code> under the “profiles” section, same as in <a href="https://github.com/apache/beam/blob/master/sdks/java/maven-archetypes/examples/src/main/resources/archetype-resources/pom.xml">here</a>.</p>
+
+<figure class="highlight"><pre><code class="language-xml" data-lang="xml"><span></span> ...
+ <span class="nt"><profile></span>
+ <span class="nt"><id></span>samza-runner<span class="nt"></id></span>
+ <span class="nt"><dependencies></span>
+ <span class="nt"><dependency></span>
+ <span class="nt"><groupId></span>org.apache.beam<span class="nt"></groupId></span>
+ <span class="nt"><artifactId></span>beam-runners-samza<span class="nt"></artifactId></span>
+ <span class="nt"><version></span>${beam.version}<span class="nt"></version></span>
+ <span class="nt"><scope></span>runtime<span class="nt"></scope></span>
+ <span class="nt"></dependency></span>
+ <span class="nt"></dependencies></span>
+ <span class="nt"></profile></span>
+ ....</code></pre></figure>
+
+<p>Now we can run the wordcount example with Samza using the following command:</p>
+
+<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>>mvn compile exec:java -Dexec.mainClass<span class="o">=</span>org.apache.beam.examples.WordCount <span class="se">\</span>
+ -Dexec.args<span class="o">=</span><span class="s2">"--inputFile=pom.xml --output=/tmp/counts --runner=SamzaRunner"</span> -Psamza-runner</code></pre></figure>
+
+<p>After the pipeline finishes, you can check out the output counts files in /tmp folder. Note Beam generates multiple output files for parallel processing. If you prefer a single output, please update the code to use TextIO.write().withoutSharding().</p>
+
+<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>>more /tmp/counts*
+AS: <span class="m">1</span>
+IO: <span class="m">2</span>
+IS: <span class="m">1</span>
+OF: <span class="m">1</span>
+...</code></pre></figure>
+
+<p>A walkthrough of the example code can be found <a href="https://beam.apache.org/get-started/wordcount-example/">here</a>. Feel free to play with other examples in the project or write your own. Please don’t hesitate to <a href="https://samza.apache.org/community/contact-us.html">reach out</a> if you encounter any issues.</p>
+
</div>
</div>
Modified: samza/site/startup/download/index.html
URL: http://svn.apache.org/viewvc/samza/site/startup/download/index.html?rev=1852770&r1=1852769&r2=1852770&view=diff
==============================================================================
--- samza/site/startup/download/index.html (original)
+++ samza/site/startup/download/index.html Fri Feb 1 19:41:48 2019
@@ -81,7 +81,7 @@
</div>
<div class="main-navigation__items" data-menu-opened>
<a class="main-navigation__item" href="/">Home</a>
- <a class="main-navigation__item" href="/learn/documentation/latest/core-concepts/core-concepts.html">Docs</a>
+ <a class="main-navigation__item" href="/learn/documentation/1.0.0/core-concepts/core-concepts.html">Docs</a>
<a class="main-navigation__item" href="/powered-by/">Powered By</a>
<a class="main-navigation__item" href="/startup/download/">Downloads</a>
<a class="main-navigation__item" href="/blog/">Blog</a>
@@ -152,10 +152,10 @@
- <a class="side-navigation__group-item" data-match-active="" href="/startup/quick-start/latest/">QuickStart</a>
+ <a class="side-navigation__group-item" data-match-active="" href="/startup/quick-start/1.0.0/">QuickStart</a>
- <a class="side-navigation__group-item" data-match-active="" href="/startup/code-examples/latest/">Code Examples</a>
+ <a class="side-navigation__group-item" data-match-active="" href="/startup/code-examples/1.0.0/">Code Examples</a>
@@ -187,7 +187,7 @@
<i class="side-navigation__group-title-icon icon ion-md-arrow-dropdown"></i>
Documentation
</div>
- <div class="side-navigation__group-items side-navigation__group-has-submenus" data-sub-menu data-documentation="/learn/documentation/latest/">
+ <div class="side-navigation__group-items side-navigation__group-has-submenus" data-sub-menu data-documentation="/learn/documentation/1.0.0/">
<!-- Handle sub navigation items from data -->
@@ -523,7 +523,7 @@
<p>Samza is released as a source artifact, and also through Maven.</p>
-<p>If you just want to play around with Samza for the first time, go to <a href="/startup/hello-samza/latest">Hello Samza</a>.</p>
+<p>If you just want to play around with Samza for the first time, go to <a href="/startup/hello-samza/1.0.0">Hello Samza</a>.</p>
<h3 id="jdk-notice">JDK Notice</h3>
@@ -628,7 +628,7 @@
<span class="nt"><version></span>0.8.1<span class="nt"></version></span>
<span class="nt"></dependency></span></code></pre></figure>
-<p><a href="/startup/hello-samza/latest">Hello Samza</a> is a working Maven project that illustrates how to build projects that have Samza jobs in them.</p>
+<p><a href="/startup/hello-samza/1.0.0">Hello Samza</a> is a working Maven project that illustrates how to build projects that have Samza jobs in them.</p>
<h4 id="repositories">Repositories</h4>
@@ -690,7 +690,7 @@
<div class="footer__items">
<a class="footer__item" href="/meetups/">Meetups</a>
<a class="footer__item" href="/blog/">Blog</a>
- <a class="footer__item" href="/learn/documentation/latest/introduction/background.html">About</a>
+ <a class="footer__item" href="/learn/documentation/1.0.0/introduction/background.html">About</a>
</div>
</div>
<div>
Modified: samza/site/startup/preview/index.html
URL: http://svn.apache.org/viewvc/samza/site/startup/preview/index.html?rev=1852770&r1=1852769&r2=1852770&view=diff
==============================================================================
--- samza/site/startup/preview/index.html (original)
+++ samza/site/startup/preview/index.html Fri Feb 1 19:41:48 2019
@@ -81,7 +81,7 @@
</div>
<div class="main-navigation__items" data-menu-opened>
<a class="main-navigation__item" href="/">Home</a>
- <a class="main-navigation__item" href="/learn/documentation/latest/core-concepts/core-concepts.html">Docs</a>
+ <a class="main-navigation__item" href="/learn/documentation/1.0.0/core-concepts/core-concepts.html">Docs</a>
<a class="main-navigation__item" href="/powered-by/">Powered By</a>
<a class="main-navigation__item" href="/startup/download/">Downloads</a>
<a class="main-navigation__item" href="/blog/">Blog</a>
@@ -152,10 +152,10 @@
- <a class="side-navigation__group-item" data-match-active="" href="/startup/quick-start/latest/">QuickStart</a>
+ <a class="side-navigation__group-item" data-match-active="" href="/startup/quick-start/1.0.0/">QuickStart</a>
- <a class="side-navigation__group-item" data-match-active="" href="/startup/code-examples/latest/">Code Examples</a>
+ <a class="side-navigation__group-item" data-match-active="" href="/startup/code-examples/1.0.0/">Code Examples</a>
@@ -187,7 +187,7 @@
<i class="side-navigation__group-title-icon icon ion-md-arrow-dropdown"></i>
Documentation
</div>
- <div class="side-navigation__group-items side-navigation__group-has-submenus" data-sub-menu data-documentation="/learn/documentation/latest/">
+ <div class="side-navigation__group-items side-navigation__group-has-submenus" data-sub-menu data-documentation="/learn/documentation/1.0.0/">
<!-- Handle sub navigation items from data -->
@@ -542,9 +542,9 @@
<p>Want to skip all the details and get some hands on experience? There are three tutorials to help you get acquainted with running Samza applications in both YARN and embedded modes and programming with the high level API:</p>
<ul>
-<li><a href="/learn/tutorials/latest/hello-samza-high-level-yarn.html">Yarn Deployment</a> - run a pre-existing Wikipedia application on YARN and observe the output.</li>
-<li><a href="/learn/tutorials/latest/hello-samza-high-level-code.html">High Level API Code Walkthrough</a> - walk through building the Wikipedia application, step by step.</li>
-<li><a href="/learn/tutorials/latest/hello-samza-high-level-zk.html">ZooKeeper Deployment</a> - run a pre-existing Wikipedia application with ZooKeeper coordination and observe the output.</li>
+<li><a href="/learn/tutorials/1.0.0/hello-samza-high-level-yarn.html">Yarn Deployment</a> - run a pre-existing Wikipedia application on YARN and observe the output.</li>
+<li><a href="/learn/tutorials/1.0.0/hello-samza-high-level-code.html">High Level API Code Walkthrough</a> - walk through building the Wikipedia application, step by step.</li>
+<li><a href="/learn/tutorials/1.0.0/hello-samza-high-level-zk.html">ZooKeeper Deployment</a> - run a pre-existing Wikipedia application with ZooKeeper coordination and observe the output.</li>
</ul>
<hr>
@@ -559,19 +559,19 @@
<p>The Samza architecture has been overhauled with distinct layers to handle each stage of application development. The following diagram shows an overview of Apache Samza architecture with the high level API.</p>
-<p><img src="/img/latest/learn/documentation/introduction/layered-arch.png" alt="Architecture diagram" style="max-width: 100%; height: auto;" onclick="window.open(this.src)"></p>
+<p><img src="/img/1.0.0/learn/documentation/introduction/layered-arch.png" alt="Architecture diagram" style="max-width: 100%; height: auto;" onclick="window.open(this.src)"></p>
<p>There are four layers in the architecture. The following sections describe each of the layers.</p>
<h4 id="i-high-level-api">I. High Level API</h4>
-<p>The high level API provides the libraries to define your application logic. The <a href="/learn/documentation/latest/api/javadocs/org/apache/samza/application/StreamApplication.html">StreamApplication</a> is the central abstraction which your application must implement. You start by declaring your inputs as instances of <a href="/learn/documentation/latest/api/javadocs/org/apache/samza/operators/MessageStream.html">MessageStream</a>. Then you can apply operators on each MessageStream like map, filter, window, and join to define the whole end-to-end data processing in a single program.</p>
+<p>The high level API provides the libraries to define your application logic. The <a href="/learn/documentation/1.0.0/api/javadocs/org/apache/samza/application/StreamApplication.html">StreamApplication</a> is the central abstraction which your application must implement. You start by declaring your inputs as instances of <a href="/learn/documentation/1.0.0/api/javadocs/org/apache/samza/operators/MessageStream.html">MessageStream</a>. Then you can apply operators on each MessageStream like map, filter, window, and join to define the whole end-to-end data processing in a single program.</p>
<p>For a deeper dive into the high level API, see <a href="#high-level-api">high level API section</a> below.</p>
<h4 id="ii-applicationrunner">II. ApplicationRunner</h4>
-<p>Samza uses an <a href="/learn/documentation/latest/api/javadocs/org/apache/samza/runtime/ApplicationRunner.html">ApplicationRunner</a> to run a stream application. The ApplicationRunner generates the configs (such as input/output streams), creates intermediate streams, and starts the execution. There are two types of ApplicationRunner:</p>
+<p>Samza uses an <a href="/learn/documentation/1.0.0/api/javadocs/org/apache/samza/runtime/ApplicationRunner.html">ApplicationRunner</a> to run a stream application. The ApplicationRunner generates the configs (such as input/output streams), creates intermediate streams, and starts the execution. There are two types of ApplicationRunner:</p>
<p><strong>RemoteApplicationRunner</strong> - submits the application to a remote cluster. This runner is invoked via the <em>run-app.sh</em> script. To use RemoteApplicationRunner, set the following configurations</p>
@@ -579,7 +579,7 @@
<span class="na">app.class</span><span class="o">=</span><span class="s">com.company.job.YourStreamApplication</span>
<span class="na">job.factory.class</span><span class="o">=</span><span class="s">org.apache.samza.job.yarn.YarnJobFactory</span></code></pre></figure>
-<p>Then use <em>run-app.sh</em> to run the application in the remote cluster. The script will invoke the RemoteApplicationRunner, which will launch one or more jobs using the factory specified with <em>job.factory.class</em>. Follow the <a href="/learn/tutorials/latest/hello-samza-high-level-yarn.html">yarn deployment tutorial</a> to try it out.</p>
+<p>Then use <em>run-app.sh</em> to run the application in the remote cluster. The script will invoke the RemoteApplicationRunner, which will launch one or more jobs using the factory specified with <em>job.factory.class</em>. Follow the <a href="/learn/tutorials/1.0.0/hello-samza-high-level-yarn.html">yarn deployment tutorial</a> to try it out.</p>
<p><strong>LocalApplicationRunner</strong> - runs the application in the JVM process of the runner. For example, to launch your application on multiple machines using ZooKeeper for coordination, you can run multiple instances of LocalApplicationRunner on various machines. After the applications load they will start cordinatinating their actions through ZooKeeper. Here is an example to run the StreamApplication in your program using the LocalApplicationRunner:</p>
@@ -595,7 +595,7 @@
<span class="n">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"Application completed with status "</span> <span class="o">+</span> <span class="n">localRunner</span><span class="o">.</span><span class="na">status</span><span class="o">(</span><span class="n">app</span><span class="o">));</span>
<span class="o">}</span></code></pre></figure>
-<p>Follow the <a href="/learn/tutorials/latest/hello-samza-high-level-zk.html">ZooKeeper deployment tutorial</a> to try it out.</p>
+<p>Follow the <a href="/learn/tutorials/1.0.0/hello-samza-high-level-zk.html">ZooKeeper deployment tutorial</a> to try it out.</p>
<h5 id="execution-plan">Execution Plan</h5>
@@ -608,7 +608,7 @@
<p>To view the plan, open the <em>bin/plan.html</em> file in a browser. Here’s a sample plan visualization:</p>
-<p><img src="/img/latest/learn/documentation/introduction/execution-plan.png" alt="Execution plan" style="max-width: 100%; height: auto;" onclick="window.open(this.src)"/></p>
+<p><img src="/img/1.0.0/learn/documentation/introduction/execution-plan.png" alt="Execution plan" style="max-width: 100%; height: auto;" onclick="window.open(this.src)"/></p>
<h4 id="iii-execution-models">III. Execution Models</h4>
@@ -627,7 +627,7 @@
<h4 id="iv-processor">IV. Processor</h4>
-<p>The lowest execution unit of a Samza application is the processor. It reads the configs generated from the <a href="/learn/documentation/latest/api/javadocs/org/apache/samza/runtime/ApplicationRunner.html">ApplicationRunner</a> and processes the input stream partitions assigned by the JobCoordinator. It can access local state using a <a href="/learn/documentation/latest/api/javadocs/org/apache/samza/storage/kv/KeyValueStore.html">KeyValueStore</a> implementation (e.g. RocksDB or in-memory) and remote state (e.g. REST service) using multithreading.</p>
+<p>The lowest execution unit of a Samza application is the processor. It reads the configs generated from the <a href="/learn/documentation/1.0.0/api/javadocs/org/apache/samza/runtime/ApplicationRunner.html">ApplicationRunner</a> and processes the input stream partitions assigned by the JobCoordinator. It can access local state using a <a href="/learn/documentation/1.0.0/api/javadocs/org/apache/samza/storage/kv/KeyValueStore.html">KeyValueStore</a> implementation (e.g. RocksDB or in-memory) and remote state (e.g. REST service) using multithreading.</p>
<hr>
@@ -650,7 +650,7 @@
<h3 id="streamapplication">StreamApplication</h3>
-<p>When writing your stream processing application using the Samza high-level API, you should implement a <a href="/learn/documentation/latest/api/javadocs/org/apache/samza/application/StreamApplication.html">StreamApplication</a> and define your processing logic in the init method.</p>
+<p>When writing your stream processing application using the Samza high-level API, you should implement a <a href="/learn/documentation/1.0.0/api/javadocs/org/apache/samza/application/StreamApplication.html">StreamApplication</a> and define your processing logic in the init method.</p>
<figure class="highlight"><pre><code class="language-java" data-lang="java"><span></span><span class="kd">public</span> <span class="kt">void</span> <span class="nf">init</span><span class="o">(</span><span class="n">StreamGraph</span> <span class="n">graph</span><span class="o">,</span> <span class="n">Config</span> <span class="n">config</span><span class="o">)</span> <span class="o">{</span> <span class="err">â¦</span> <span class="o">}</span></code></pre></figure>
@@ -723,11 +723,11 @@
<h2 id="operators">Operators</h2>
-<p>The high level API supports common operators like map, flatmap, filter, merge, joins, and windowing on streams. Most of these operators accept corresponding Functions and these functions are <a href="/learn/documentation/latest/api/javadocs/org/apache/samza/operators/functions/InitableFunction.html">Initable</a>.</p>
+<p>The high level API supports common operators like map, flatmap, filter, merge, joins, and windowing on streams. Most of these operators accept corresponding Functions and these functions are <a href="/learn/documentation/1.0.0/api/javadocs/org/apache/samza/operators/functions/InitableFunction.html">Initable</a>.</p>
<h3 id="map">Map</h3>
-<p>Applies the provided 1:1 <a href="/learn/documentation/latest/api/javadocs/org/apache/samza/operators/functions/MapFunction.html">MapFunction</a> to each element in the MessageStream and returns the transformed MessageStream. The MapFunction takes in a single message and returns a single message (potentially of a different type).</p>
+<p>Applies the provided 1:1 <a href="/learn/documentation/1.0.0/api/javadocs/org/apache/samza/operators/functions/MapFunction.html">MapFunction</a> to each element in the MessageStream and returns the transformed MessageStream. The MapFunction takes in a single message and returns a single message (potentially of a different type).</p>
<figure class="highlight"><pre><code class="language-java" data-lang="java"><span></span><span class="n">MessageStream</span><span class="o"><</span><span class="n">Integer</span><span class="o">></span> <span class="n">numbers</span> <span class="o">=</span> <span class="o">...</span>
<span class="n">MessageStream</span><span class="o"><</span><span class="n">Integer</span><span class="o">></span> <span class="n">tripled</span><span class="o">=</span> <span class="n">numbers</span><span class="o">.</span><span class="na">map</span><span class="o">(</span><span class="n">m</span> <span class="o">-></span> <span class="n">m</span> <span class="o">*</span> <span class="mi">3</span><span class="o">)</span>
@@ -735,7 +735,7 @@
<h3 id="flatmap">Flatmap</h3>
-<p>Applies the provided 1:n <a href="/learn/documentation/latest/api/javadocs/org/apache/samza/operators/functions/FlatMapFunction.html">FlatMapFunction</a> to each element in the MessageStream and returns the transformed MessageStream. The FlatMapFunction takes in a single message and returns zero or more messages.</p>
+<p>Applies the provided 1:n <a href="/learn/documentation/1.0.0/api/javadocs/org/apache/samza/operators/functions/FlatMapFunction.html">FlatMapFunction</a> to each element in the MessageStream and returns the transformed MessageStream. The FlatMapFunction takes in a single message and returns zero or more messages.</p>
<figure class="highlight"><pre><code class="language-java" data-lang="java"><span></span><span class="n">MessageStream</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">sentence</span> <span class="o">=</span> <span class="o">...</span>
<span class="c1">// Parse the sentence into its individual words splitting by space</span>
@@ -744,7 +744,7 @@
<h3 id="filter">Filter</h3>
-<p>Applies the provided <a href="/learn/documentation/latest/api/javadocs/org/apache/samza/operators/functions/FilterFunction.html">FilterFunction</a> to the MessageStream and returns the filtered MessageStream. The FilterFunction is a predicate that specifies whether a message should be retained in the filtered stream. Messages for which the FilterFunction returns false are filtered out.</p>
+<p>Applies the provided <a href="/learn/documentation/1.0.0/api/javadocs/org/apache/samza/operators/functions/FilterFunction.html">FilterFunction</a> to the MessageStream and returns the filtered MessageStream. The FilterFunction is a predicate that specifies whether a message should be retained in the filtered stream. Messages for which the FilterFunction returns false are filtered out.</p>
<figure class="highlight"><pre><code class="language-java" data-lang="java"><span></span><span class="n">MessageStream</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">words</span> <span class="o">=</span> <span class="o">...</span>
<span class="c1">// Extract only the long words</span>
@@ -777,7 +777,7 @@
<p>The merge transform preserves the order of each MessageStream, so if message <code>m1</code> appears before <code>m2</code> in any provided stream, then, <code>m1</code> also appears before <code>m2</code> in the merged stream.</p>
-<p>As an alternative to the <code>merge</code> instance method, you also can use the <a href="/learn/documentation/latest/api/javadocs/org/apache/samza/operators/MessageStream.html#mergeAll-java.util.Collection-">MessageStream#mergeAll</a> static method to merge MessageStreams without operating on an initial stream.</p>
+<p>As an alternative to the <code>merge</code> instance method, you also can use the <a href="/learn/documentation/1.0.0/api/javadocs/org/apache/samza/operators/MessageStream.html#mergeAll-java.util.Collection-">MessageStream#mergeAll</a> static method to merge MessageStreams without operating on an initial stream.</p>
<h3 id="sendto-stream">SendTo (stream)</h3>
@@ -802,7 +802,7 @@
<h3 id="sink">Sink</h3>
-<p>Allows sending messages from this MessageStream to an output system using the provided <a href="/learn/documentation/latest/api/javadocs/org/apache/samza/operators/functions/SinkFunction.html">SinkFunction</a>.</p>
+<p>Allows sending messages from this MessageStream to an output system using the provided <a href="/learn/documentation/1.0.0/api/javadocs/org/apache/samza/operators/functions/SinkFunction.html">SinkFunction</a>.</p>
<p>This offers more control than sendTo since the SinkFunction has access to the <code>MessageCollector</code> and the <code>TaskCoordinator</code>. For instance, you can choose to manually commit offsets, or shut-down the job using the TaskCoordinator APIs. This operator can also be used to send messages to non-Samza systems (e.g. remote databases, REST services, etc.)</p>
@@ -816,7 +816,7 @@
<h3 id="join-stream-stream">Join (stream-stream)</h3>
-<p>The stream-stream Join operator joins messages from two MessageStreams using the provided pairwise <a href="/learn/documentation/latest/api/javadocs/org/apache/samza/operators/functions/JoinFunction.html">JoinFunction</a>. Messages are joined when the keys extracted from messages from the first stream match keys extracted from messages in the second stream. Messages in each stream are retained for the provided ttl duration and join results are emitted as matches are found.</p>
+<p>The stream-stream Join operator joins messages from two MessageStreams using the provided pairwise <a href="/learn/documentation/1.0.0/api/javadocs/org/apache/samza/operators/functions/JoinFunction.html">JoinFunction</a>. Messages are joined when the keys extracted from messages from the first stream match keys extracted from messages in the second stream. Messages in each stream are retained for the provided ttl duration and join results are emitted as matches are found.</p>
<figure class="highlight"><pre><code class="language-java" data-lang="java"><span></span><span class="c1">// Joins a stream of OrderRecord with a stream of ShipmentRecord by orderId with a TTL of 20 minutes.</span>
<span class="c1">// Results are produced to a new stream of FulfilledOrderRecord.</span>
@@ -849,7 +849,7 @@
<h3 id="join-stream-table">Join (stream-table)</h3>
-<p>The stream-table Join operator joins messages from a MessageStream using the provided <a href="/learn/documentation/latest/api/javadocs/org/apache/samza/operators/functions/StreamTableJoinFunction.html">StreamTableJoinFunction</a>. Messages from the input stream are joined with record in table using key extracted from input messages. The join function is invoked with both the message and the record. If a record is not found in the table, a null value is provided; the join function can choose to return null (inner join) or an output message (left outer join). For join to function properly, it is important to ensure the input stream and table are partitioned using the same key as this impacts the physical placement of data. </p>
+<p>The stream-table Join operator joins messages from a MessageStream using the provided <a href="/learn/documentation/1.0.0/api/javadocs/org/apache/samza/operators/functions/StreamTableJoinFunction.html">StreamTableJoinFunction</a>. Messages from the input stream are joined with record in table using key extracted from input messages. The join function is invoked with both the message and the record. If a record is not found in the table, a null value is provided; the join function can choose to return null (inner join) or an output message (left outer join). For join to function properly, it is important to ensure the input stream and table are partitioned using the same key as this impacts the physical placement of data. </p>
<figure class="highlight"><pre><code class="language-java" data-lang="java"><span></span> <span class="n">streamGraph</span><span class="o">.</span><span class="na">getInputStream</span><span class="o">(</span><span class="s">"PageView"</span><span class="o">,</span> <span class="k">new</span> <span class="n">NoOpSerde</span><span class="o"><</span><span class="n">PageView</span><span class="o">>())</span>
<span class="o">.</span><span class="na">partitionBy</span><span class="o">(</span><span class="n">PageView</span><span class="o">::</span><span class="n">getMemberId</span><span class="o">,</span> <span class="n">v</span> <span class="o">-></span> <span class="n">v</span><span class="o">,</span> <span class="s">"p1"</span><span class="o">)</span>
@@ -882,9 +882,9 @@
<p><strong>Windows, Triggers, and WindowPanes</strong>: The window operator groups incoming messages in the MessageStream into finite windows. Each emitted result contains one or more messages in the window and is called a WindowPane.</p>
-<p>A window can have one or more associated triggers which determine when results from the window are emitted. Triggers can be either <a href="/learn/documentation/latest/api/javadocs/org/apache/samza/operators/windows/Window.html#setEarlyTrigger-org.apache.samza.operators.triggers.Trigger-">early triggers</a> that allow emitting results speculatively before all data for the window has arrived, or late triggers that allow handling late messages for the window.</p>
+<p>A window can have one or more associated triggers which determine when results from the window are emitted. Triggers can be either <a href="/learn/documentation/1.0.0/api/javadocs/org/apache/samza/operators/windows/Window.html#setEarlyTrigger-org.apache.samza.operators.triggers.Trigger-">early triggers</a> that allow emitting results speculatively before all data for the window has arrived, or late triggers that allow handling late messages for the window.</p>
-<p><strong>Aggregator Function</strong>: By default, the emitted WindowPane will contain all the messages for the window. Instead of retaining all messages, you typically define a more compact data structure for the WindowPane and update it incrementally as new messages arrive, e.g. for keeping a count of messages in the window. To do this, you can provide an aggregating <a href="/learn/documentation/latest/api/javadocs/org/apache/samza/operators/functions/FoldLeftFunction.html">FoldLeftFunction</a> which is invoked for each incoming message added to the window and defines how to update the WindowPane for that message.</p>
+<p><strong>Aggregator Function</strong>: By default, the emitted WindowPane will contain all the messages for the window. Instead of retaining all messages, you typically define a more compact data structure for the WindowPane and update it incrementally as new messages arrive, e.g. for keeping a count of messages in the window. To do this, you can provide an aggregating <a href="/learn/documentation/1.0.0/api/javadocs/org/apache/samza/operators/functions/FoldLeftFunction.html">FoldLeftFunction</a> which is invoked for each incoming message added to the window and defines how to update the WindowPane for that message.</p>
<p><strong>Accumulation Mode</strong>: A windowâs accumulation mode determines how results emitted from a window relate to previously emitted results for the same window. This is particularly useful when the window is configured with early or late triggers. The accumulation mode can either be discarding or accumulating.</p>
@@ -982,7 +982,7 @@
<h4 id="external-partition-management">External Partition Management</h4>
-<p>With external partition management, Samza doesnât manage the partitioning by itself. Instead it uses a <code>PassthroughJobCoordinator</code> which honors whatever partition mapping is provided by the <a href="/learn/documentation/latest/api/javadocs/org/apache/samza/container/grouper/stream/SystemStreamPartitionGrouper.html">SystemStreamPartitionGrouper</a>. There are two common patterns for external partition management:</p>
+<p>With external partition management, Samza doesnât manage the partitioning by itself. Instead it uses a <code>PassthroughJobCoordinator</code> which honors whatever partition mapping is provided by the <a href="/learn/documentation/1.0.0/api/javadocs/org/apache/samza/container/grouper/stream/SystemStreamPartitionGrouper.html">SystemStreamPartitionGrouper</a>. There are two common patterns for external partition management:</p>
<ul>
<li><strong>Using high level Kafka consumer</strong> - partition assignment is done by the high level Kafka consumer itself. To use this model, you need to implement and configure a SystemFactory which provides the Kafka high level consumer. Then you need to configure <em>job.systemstreampartition.grouper.factory</em> to <em>org.apache.samza.container.grouper.stream.AllSspToSingleTaskGrouper</em> so Kafka’s partition assignments all go to one task.</li>
@@ -1026,7 +1026,7 @@ There is one leader processor which gene
<p>The following diagram shows the relationships of the coordinators in the ZooKeeper coordination service implementation.</p>
-<p><img src="/img/latest/learn/documentation/introduction/coordination-service.png" alt="Coordination service diagram" style="max-width: 100%; height: auto;" onclick="window.open(this.src)"></p>
+<p><img src="/img/1.0.0/learn/documentation/introduction/coordination-service.png" alt="Coordination service diagram" style="max-width: 100%; height: auto;" onclick="window.open(this.src)"></p>
<p>Here are a few important details about the coordination service:</p>
@@ -1040,7 +1040,7 @@ There is one leader processor which gene
<p>Embedded deployment is designed to help users who want more control over the deployment of their application. So it is the user’s responsibility to configure and deploy the processors. In case of ZooKeeper coordination, you also need to configure the URL for an instance of ZooKeeper.</p>
-<p>Additionally, each processor requires a unique ID to be used with the coordination service. If location affinity is important, this ID should be unique for each processor on a specific hostname (assuming local Storage services). To address this requirement, Samza uses a <a href="/learn/documentation/latest/api/javadocs/org/apache/samza/runtime/ProcessorIdGenerator.html">ProcessorIdGenerator</a> to provide the ID for each processor. If no generator is explicitly configured, the default one will create a UUID for each processor.</p>
+<p>Additionally, each processor requires a unique ID to be used with the coordination service. If location affinity is important, this ID should be unique for each processor on a specific hostname (assuming local Storage services). To address this requirement, Samza uses a <a href="/learn/documentation/1.0.0/api/javadocs/org/apache/samza/runtime/ProcessorIdGenerator.html">ProcessorIdGenerator</a> to provide the ID for each processor. If no generator is explicitly configured, the default one will create a UUID for each processor.</p>
<h4 id="configuration">Configuration</h4>
@@ -1080,7 +1080,7 @@ There is one leader processor which gene
<p>In the code above, <code>WikipediaApplication</code> is an application written with the <a href="#high-level-api">high level API</a>.</p>
-<p>Check out the <a href="/learn/tutorials/latest/hello-samza-high-level-zk.html">tutorial</a> to run this application with ZooKeeper coordination on your machine now.</p>
+<p>Check out the <a href="/learn/tutorials/1.0.0/hello-samza-high-level-zk.html">tutorial</a> to run this application with ZooKeeper coordination on your machine now.</p>
<h4 id="deployment-and-scaling">Deployment and Scaling</h4>
@@ -1139,7 +1139,7 @@ There is one leader processor which gene
<div class="footer__items">
<a class="footer__item" href="/meetups/">Meetups</a>
<a class="footer__item" href="/blog/">Blog</a>
- <a class="footer__item" href="/learn/documentation/latest/introduction/background.html">About</a>
+ <a class="footer__item" href="/learn/documentation/1.0.0/introduction/background.html">About</a>
</div>
</div>
<div>
Modified: samza/site/startup/quick-start/1.0.0/index.html
URL: http://svn.apache.org/viewvc/samza/site/startup/quick-start/1.0.0/index.html?rev=1852770&r1=1852769&r2=1852770&view=diff
==============================================================================
--- samza/site/startup/quick-start/1.0.0/index.html (original)
+++ samza/site/startup/quick-start/1.0.0/index.html Fri Feb 1 19:41:48 2019
@@ -521,192 +521,9 @@
limitations under the License.
-->
-<p>In this tutorial, we will create our first Samza application - <code>WordCount</code>. This application will consume messages from a Kafka stream, tokenize them into individual words and count the frequency of each word. Let us download the entire project from <a href="https://github.com/apache/samza-hello-samza/blob/latest/quickstart/wordcount.tar.gz">here</a>.</p>
+<p><a href="samza.html">Samza Tutorial</a></p>
-<h3 id="setting-up-a-java-project">Setting up a Java Project</h3>
-
-<p>Observe the project structure as follows:</p>
-
-<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>wordcount
-<span class="p">|</span>-- build.gradle
-<span class="p">|</span>-- gradle.properties
-<span class="p">|</span>-- scripts
-<span class="p">|</span>-- src
- <span class="p">|</span>-- main
- <span class="p">|</span>-- config
- <span class="p">|</span>-- java
- <span class="p">|</span>-- samzaapp
- <span class="p">|</span>-- WordCount.java</code></pre></figure>
-
-<p>You can build the project anytime by running:</p>
-
-<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>> <span class="nb">cd</span> wordcount
-> gradle wrapper --gradle-version <span class="m">4</span>.9
-> ./gradlew build</code></pre></figure>
-
-<h3 id="create-a-samza-streamapplication">Create a Samza StreamApplication</h3>
-
-<p>Now letâs write some code! An application written using Samza’s <a href="/learn/documentation/1.0.0/api/api/high-level-api.html">high-level API</a> implements the <a href="/learn/documentation/1.0.0/api/javadocs/org/apache/samza/application/StreamApplication.html">StreamApplication</a> interface:</p>
-
-<figure class="highlight"><pre><code class="language-java" data-lang="java"><span></span><span class="kn">package</span> <span class="nn">samzaapp</span><span class="o">;</span>
-
-<span class="kn">import</span> <span class="nn">org.apache.samza.application.StreamApplication</span><span class="o">;</span>
-<span class="kn">import</span> <span class="nn">org.apache.samza.application.descriptors.StreamApplicationDescriptor</span><span class="o">;</span>
-
-<span class="kd">public</span> <span class="kd">class</span> <span class="nc">WordCount</span> <span class="kd">implements</span> <span class="n">StreamApplication</span> <span class="o">{</span>
- <span class="nd">@Override</span>
- <span class="kd">public</span> <span class="kt">void</span> <span class="nf">describe</span><span class="o">(</span><span class="n">StreamApplicationDescriptor</span> <span class="n">streamApplicationDescriptor</span><span class="o">)</span> <span class="o">{</span>
- <span class="o">}</span>
-<span class="o">}</span></code></pre></figure>
-
-<p>The interface provides a single method named <code>describe()</code>, which allows us to define our inputs, the processing logic and outputs for our application. </p>
-
-<h3 id="describe-your-inputs-and-outputs">Describe your inputs and outputs</h3>
-
-<p>To interact with Kafka, we will first create a <code>KafkaSystemDescriptor</code> by providing the coordinates of the Kafka cluster. For each Kafka topic our application reads from, we create a <code>KafkaInputDescriptor</code> with the name of the topic and a serializer. Likewise, for each output topic, we instantiate a corresponding <code>KafkaOutputDescriptor</code>. </p>
-
-<figure class="highlight"><pre><code class="language-java" data-lang="java"><span></span><span class="kd">public</span> <span class="kd">class</span> <span class="nc">WordCount</span> <span class="kd">implements</span> <span class="n">StreamApplication</span> <span class="o">{</span>
- <span class="kd">private</span> <span class="kd">static</span> <span class="kd">final</span> <span class="n">String</span> <span class="n">KAFKA_SYSTEM_NAME</span> <span class="o">=</span> <span class="s">"kafka"</span><span class="o">;</span>
- <span class="kd">private</span> <span class="kd">static</span> <span class="kd">final</span> <span class="n">List</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">KAFKA_CONSUMER_ZK_CONNECT</span> <span class="o">=</span> <span class="n">ImmutableList</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="s">"localhost:2181"</span><span class="o">);</span>
- <span class="kd">private</span> <span class="kd">static</span> <span class="kd">final</span> <span class="n">List</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">KAFKA_PRODUCER_BOOTSTRAP_SERVERS</span> <span class="o">=</span> <span class="n">ImmutableList</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="s">"localhost:9092"</span><span class="o">);</span>
- <span class="kd">private</span> <span class="kd">static</span> <span class="kd">final</span> <span class="n">Map</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">String</span><span class="o">></span> <span class="n">KAFKA_DEFAULT_STREAM_CONFIGS</span> <span class="o">=</span> <span class="n">ImmutableMap</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="s">"replication.factor"</span><span class="o">,</span> <span class="s">"1"</span><span class="o">);</span>
-
- <span class="kd">private</span> <span class="kd">static</span> <span class="kd">final</span> <span class="n">String</span> <span class="n">INPUT_STREAM_ID</span> <span class="o">=</span> <span class="s">"sample-text"</span><span class="o">;</span>
- <span class="kd">private</span> <span class="kd">static</span> <span class="kd">final</span> <span class="n">String</span> <span class="n">OUTPUT_STREAM_ID</span> <span class="o">=</span> <span class="s">"word-count-output"</span><span class="o">;</span>
-
- <span class="nd">@Override</span>
- <span class="kd">public</span> <span class="kt">void</span> <span class="nf">describe</span><span class="o">(</span><span class="n">StreamApplicationDescriptor</span> <span class="n">streamApplicationDescriptor</span><span class="o">)</span> <span class="o">{</span>
- <span class="c1">// Create a KafkaSystemDescriptor providing properties of the cluster</span>
- <span class="n">KafkaSystemDescriptor</span> <span class="n">kafkaSystemDescriptor</span> <span class="o">=</span> <span class="k">new</span> <span class="n">KafkaSystemDescriptor</span><span class="o">(</span><span class="n">KAFKA_SYSTEM_NAME</span><span class="o">)</span>
- <span class="o">.</span><span class="na">withConsumerZkConnect</span><span class="o">(</span><span class="n">KAFKA_CONSUMER_ZK_CONNECT</span><span class="o">)</span>
- <span class="o">.</span><span class="na">withProducerBootstrapServers</span><span class="o">(</span><span class="n">KAFKA_PRODUCER_BOOTSTRAP_SERVERS</span><span class="o">)</span>
- <span class="o">.</span><span class="na">withDefaultStreamConfigs</span><span class="o">(</span><span class="n">KAFKA_DEFAULT_STREAM_CONFIGS</span><span class="o">);</span>
-
- <span class="c1">// For each input or output stream, create a KafkaInput/Output descriptor</span>
- <span class="n">KafkaInputDescriptor</span><span class="o"><</span><span class="n">KV</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">String</span><span class="o">>></span> <span class="n">inputDescriptor</span> <span class="o">=</span>
- <span class="n">kafkaSystemDescriptor</span><span class="o">.</span><span class="na">getInputDescriptor</span><span class="o">(</span><span class="n">INPUT_STREAM_ID</span><span class="o">,</span>
- <span class="n">KVSerde</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="k">new</span> <span class="n">StringSerde</span><span class="o">(),</span> <span class="k">new</span> <span class="n">StringSerde</span><span class="o">()));</span>
- <span class="n">KafkaOutputDescriptor</span><span class="o"><</span><span class="n">KV</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">String</span><span class="o">>></span> <span class="n">outputDescriptor</span> <span class="o">=</span>
- <span class="n">kafkaSystemDescriptor</span><span class="o">.</span><span class="na">getOutputDescriptor</span><span class="o">(</span><span class="n">OUTPUT_STREAM_ID</span><span class="o">,</span>
- <span class="n">KVSerde</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="k">new</span> <span class="n">StringSerde</span><span class="o">(),</span> <span class="k">new</span> <span class="n">StringSerde</span><span class="o">()));</span>
-
- <span class="c1">// Obtain a handle to a MessageStream that you can chain operations on</span>
- <span class="n">MessageStream</span><span class="o"><</span><span class="n">KV</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">String</span><span class="o">>></span> <span class="n">lines</span> <span class="o">=</span> <span class="n">streamApplicationDescriptor</span><span class="o">.</span><span class="na">getInputStream</span><span class="o">(</span><span class="n">inputDescriptor</span><span class="o">);</span>
- <span class="n">OutputStream</span><span class="o"><</span><span class="n">KV</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">String</span><span class="o">>></span> <span class="n">counts</span> <span class="o">=</span> <span class="n">streamApplicationDescriptor</span><span class="o">.</span><span class="na">getOutputStream</span><span class="o">(</span><span class="n">outputDescriptor</span><span class="o">);</span>
- <span class="o">}</span>
-<span class="o">}</span></code></pre></figure>
-
-<p>The above example creates a <a href="/learn/documentation/1.0.0/api/javadocs/org/apache/samza/operators/MessageStream.html">MessageStream</a> which reads from an input topic named <code>sample-text</code>. It also defines an output stream that emits results to a topic named <code>word-count-output</code>. Next letâs add our processing logic. </p>
-
-<h3 id="add-word-count-processing-logic">Add word count processing logic</h3>
-
-<p>Kafka messages typically have a key and a value. Since we only care about the value here, we will apply the <code>map</code> operator on the input stream to extract the value. </p>
-
-<figure class="highlight"><pre><code class="language-java" data-lang="java"><span></span><span class="n">lines</span><span class="o">.</span><span class="na">map</span><span class="o">(</span><span class="n">kv</span> <span class="o">-></span> <span class="n">kv</span><span class="o">.</span><span class="na">value</span><span class="o">)</span></code></pre></figure>
-
-<p>Next, we will tokenize the message into individual words using the <code>flatmap</code> operator.</p>
-
-<figure class="highlight"><pre><code class="language-java" data-lang="java"><span></span><span class="o">.</span><span class="na">flatMap</span><span class="o">(</span><span class="n">s</span> <span class="o">-></span> <span class="n">Arrays</span><span class="o">.</span><span class="na">asList</span><span class="o">(</span><span class="n">s</span><span class="o">.</span><span class="na">split</span><span class="o">(</span><span class="s">"\\W+"</span><span class="o">)))</span></code></pre></figure>
-
-<p>We now need to group the words, aggregate their respective counts and periodically emit our results. For this, we will use Samza’s session-windowing feature.</p>
-
-<figure class="highlight"><pre><code class="language-java" data-lang="java"><span></span><span class="o">.</span><span class="na">window</span><span class="o">(</span><span class="n">Windows</span><span class="o">.</span><span class="na">keyedSessionWindow</span><span class="o">(</span>
- <span class="n">w</span> <span class="o">-></span> <span class="n">w</span><span class="o">,</span> <span class="n">Duration</span><span class="o">.</span><span class="na">ofSeconds</span><span class="o">(</span><span class="mi">5</span><span class="o">),</span> <span class="o">()</span> <span class="o">-></span> <span class="mi">0</span><span class="o">,</span> <span class="o">(</span><span class="n">m</span><span class="o">,</span> <span class="n">prevCount</span><span class="o">)</span> <span class="o">-></span> <span class="n">prevCount</span> <span class="o">+</span> <span class="mi">1</span><span class="o">,</span>
- <span class="k">new</span> <span class="n">StringSerde</span><span class="o">(),</span> <span class="k">new</span> <span class="n">IntegerSerde</span><span class="o">()),</span> <span class="s">"count"</span><span class="o">)</span></code></pre></figure>
-
-<p>Let’s walk through each of the parameters to the above <code>window</code> function:
-The first parameter is a “key function”, which defines the key to group messages by. In our case, we can simply use the word as the key. The second parameter is the windowing interval, which is set to 5 seconds. The third parameter is a function which provides the initial value for our aggregations. We can start with an initial count of zero for each word. The fourth parameter is an aggregation function for computing counts. The next two parameters specify the key and value serializers for our window. </p>
-
-<p>The output from the window operator is captured in a <a href="/learn/documentation/1.0.0/api/javadocs/org/apache/samza/operators/windows/WindowPane.html">WindowPane</a> type, which contains the word as the key and its count as the value. We add a further <code>map</code> to format this into a <code>KV</code>, that we can send to our Kafka topic. To write our results to the output topic, we use the <code>sendTo</code> operator in Samza.</p>
-
-<figure class="highlight"><pre><code class="language-java" data-lang="java"><span></span><span class="o">.</span><span class="na">map</span><span class="o">(</span><span class="n">windowPane</span> <span class="o">-></span>
- <span class="n">KV</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">windowPane</span><span class="o">.</span><span class="na">getKey</span><span class="o">().</span><span class="na">getKey</span><span class="o">(),</span>
- <span class="n">windowPane</span><span class="o">.</span><span class="na">getKey</span><span class="o">().</span><span class="na">getKey</span><span class="o">()</span> <span class="o">+</span> <span class="s">": "</span> <span class="o">+</span> <span class="n">windowPane</span><span class="o">.</span><span class="na">getMessage</span><span class="o">().</span><span class="na">toString</span><span class="o">()))</span>
-<span class="o">.</span><span class="na">sendTo</span><span class="o">(</span><span class="n">counts</span><span class="o">);</span></code></pre></figure>
-
-<p>The full processing logic looks like the following:</p>
-
-<figure class="highlight"><pre><code class="language-java" data-lang="java"><span></span><span class="n">lines</span>
- <span class="o">.</span><span class="na">map</span><span class="o">(</span><span class="n">kv</span> <span class="o">-></span> <span class="n">kv</span><span class="o">.</span><span class="na">value</span><span class="o">)</span>
- <span class="o">.</span><span class="na">flatMap</span><span class="o">(</span><span class="n">s</span> <span class="o">-></span> <span class="n">Arrays</span><span class="o">.</span><span class="na">asList</span><span class="o">(</span><span class="n">s</span><span class="o">.</span><span class="na">split</span><span class="o">(</span><span class="s">"\\W+"</span><span class="o">)))</span>
- <span class="o">.</span><span class="na">window</span><span class="o">(</span><span class="n">Windows</span><span class="o">.</span><span class="na">keyedSessionWindow</span><span class="o">(</span>
- <span class="n">w</span> <span class="o">-></span> <span class="n">w</span><span class="o">,</span> <span class="n">Duration</span><span class="o">.</span><span class="na">ofSeconds</span><span class="o">(</span><span class="mi">5</span><span class="o">),</span> <span class="o">()</span> <span class="o">-></span> <span class="mi">0</span><span class="o">,</span> <span class="o">(</span><span class="n">m</span><span class="o">,</span> <span class="n">prevCount</span><span class="o">)</span> <span class="o">-></span> <span class="n">prevCount</span> <span class="o">+</span> <span class="mi">1</span><span class="o">,</span>
- <span class="k">new</span> <span class="n">StringSerde</span><span class="o">(),</span> <span class="k">new</span> <span class="n">IntegerSerde</span><span class="o">()),</span> <span class="s">"count"</span><span class="o">)</span>
- <span class="o">.</span><span class="na">map</span><span class="o">(</span><span class="n">windowPane</span> <span class="o">-></span>
- <span class="n">KV</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">windowPane</span><span class="o">.</span><span class="na">getKey</span><span class="o">().</span><span class="na">getKey</span><span class="o">(),</span>
- <span class="n">windowPane</span><span class="o">.</span><span class="na">getKey</span><span class="o">().</span><span class="na">getKey</span><span class="o">()</span> <span class="o">+</span> <span class="s">": "</span> <span class="o">+</span> <span class="n">windowPane</span><span class="o">.</span><span class="na">getMessage</span><span class="o">().</span><span class="na">toString</span><span class="o">()))</span>
- <span class="o">.</span><span class="na">sendTo</span><span class="o">(</span><span class="n">counts</span><span class="o">);</span></code></pre></figure>
-
-<h3 id="configure-your-application">Configure your application</h3>
-
-<p>In this section, we will configure our word count example to run locally in a single JVM. Let us add a file named âword-count.propertiesâ under the config folder. </p>
-
-<figure class="highlight"><pre><code class="language-jproperties" data-lang="jproperties"><span></span><span class="na">job.name</span><span class="o">=</span><span class="s">word-count</span>
-<span class="c"># Use a PassthroughJobCoordinator since there is no coordination needed</span>
-<span class="na">job.coordinator.factory</span><span class="o">=</span><span class="s">org.apache.samza.standalone.PassthroughJobCoordinatorFactory</span>
-<span class="na">job.coordination.utils.factory</span><span class="o">=</span><span class="s">org.apache.samza.standalone.PassthroughCoordinationUtilsFactory</span>
-
-<span class="na">job.changelog.system</span><span class="o">=</span><span class="s">kafka</span>
-
-<span class="c"># Use a single container to process all of the data</span>
-<span class="na">task.name.grouper.factory</span><span class="o">=</span><span class="s">org.apache.samza.container.grouper.task.SingleContainerGrouperFactory</span>
-<span class="na">processor.id</span><span class="o">=</span><span class="s">0</span>
-
-<span class="c"># Read from the beginning of the topic</span>
-<span class="na">systems.kafka.default.stream.samza.offset.default</span><span class="o">=</span><span class="s">oldest</span></code></pre></figure>
-
-<p>For more details on Samza’s configs, feel free to check out the latest <a href="/learn/documentation/1.0.0/jobs/configuration-table.html">configuration reference</a>.</p>
-
-<h3 id="run-your-application">Run your application</h3>
-
-<p>We are ready to add a <code>main()</code> function to the <code>WordCount</code> class. It parses the command-line arguments and instantiates a <code>LocalApplicationRunner</code> to execute the application locally.</p>
-
-<figure class="highlight"><pre><code class="language-java" data-lang="java"><span></span><span class="kd">public</span> <span class="kd">static</span> <span class="kt">void</span> <span class="nf">main</span><span class="o">(</span><span class="n">String</span><span class="o">[]</span> <span class="n">args</span><span class="o">)</span> <span class="o">{</span>
- <span class="n">CommandLine</span> <span class="n">cmdLine</span> <span class="o">=</span> <span class="k">new</span> <span class="n">CommandLine</span><span class="o">();</span>
- <span class="n">OptionSet</span> <span class="n">options</span> <span class="o">=</span> <span class="n">cmdLine</span><span class="o">.</span><span class="na">parser</span><span class="o">().</span><span class="na">parse</span><span class="o">(</span><span class="n">args</span><span class="o">);</span>
- <span class="n">Config</span> <span class="n">config</span> <span class="o">=</span> <span class="n">cmdLine</span><span class="o">.</span><span class="na">loadConfig</span><span class="o">(</span><span class="n">options</span><span class="o">);</span>
- <span class="n">LocalApplicationRunner</span> <span class="n">runner</span> <span class="o">=</span> <span class="k">new</span> <span class="n">LocalApplicationRunner</span><span class="o">(</span><span class="k">new</span> <span class="n">WordCount</span><span class="o">(),</span> <span class="n">config</span><span class="o">);</span>
- <span class="n">runner</span><span class="o">.</span><span class="na">run</span><span class="o">();</span>
- <span class="n">runner</span><span class="o">.</span><span class="na">waitForFinish</span><span class="o">();</span>
-<span class="o">}</span></code></pre></figure>
-
-<p>Before running <code>main()</code>, we will create our input Kafka topic and populate it with sample data. You can download the scripts to interact with Kafka along with the sample data from <a href="https://github.com/apache/samza-hello-samza/blob/latest/quickstart/wordcount.tar.gz">here</a>.</p>
-
-<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>> ./scripts/grid install zookeeper <span class="o">&&</span> ./scripts/grid start zookeeper
-> ./scripts/grid install kafka <span class="o">&&</span> ./scripts/grid start kafka</code></pre></figure>
-
-<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>> ./deploy/kafka/bin/kafka-topics.sh --create --zookeeper localhost:2181 --topic sample-text --partition <span class="m">1</span> --replication-factor <span class="m">1</span>
-> ./deploy/kafka/bin/kafka-console-producer.sh --topic sample-text --broker localhost:9092 < ./sample-text.txt</code></pre></figure>
-
-<p>Letâs kick off our application and use gradle to run it. Alternately, you can also run it directly from your IDE, with the same program arguments.</p>
-
-<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>> <span class="nb">export</span> <span class="nv">BASE_DIR</span><span class="o">=</span><span class="sb">`</span><span class="nb">pwd</span><span class="sb">`</span>
-> ./gradlew run --args<span class="o">=</span><span class="s2">"--config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://</span><span class="nv">$BASE_DIR</span><span class="s2">/src/main/config/word-count.properties"</span></code></pre></figure>
-
-<p>The application will output to a Kafka topic named “word-count-output”. We will now fire up a Kafka consumer to read from this topic:</p>
-
-<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>> ./deploy/kafka/bin/kafka-console-consumer.sh --topic word-count-output --zookeeper localhost:2181 --from-beginning</code></pre></figure>
-
-<p>It will show the counts for each word like the following:</p>
-
-<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>well: <span class="m">4</span>
-complaining: <span class="m">1</span>
-die: <span class="m">3</span>
-but: <span class="m">22</span>
-not: <span class="m">50</span>
-truly: <span class="m">5</span>
-murmuring: <span class="m">1</span>
-satisfied: <span class="m">3</span>
-the: <span class="m">120</span>
-thy: <span class="m">8</span>
-gods: <span class="m">8</span>
-thankful: <span class="m">1</span>
-and: <span class="m">243</span>
-from: <span class="m">16</span></code></pre></figure>
-
-<p>Congratulations! You’ve successfully run your first Samza application.</p>
-
-<h3 id="more-examples"><a href="/startup/code-examples/1.0.0">More Examples >></a></h3>
+<p><a href="samza-sql.html">Samza SQL Tutorial</a></p>
</div>