You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@storm.apache.org by pt...@apache.org on 2018/02/14 20:57:33 UTC
[09/51] [partial] storm-site git commit: regenerate site for download
page changes
http://git-wip-us.apache.org/repos/asf/storm-site/blob/a7a19afb/content/releases/1.0.4/Serialization-(prior-to-0.6.0).html
----------------------------------------------------------------------
diff --git a/content/releases/1.0.4/Serialization-(prior-to-0.6.0).html b/content/releases/1.0.4/Serialization-(prior-to-0.6.0).html
index 28ea21d..bf1764f 100644
--- a/content/releases/1.0.4/Serialization-(prior-to-0.6.0).html
+++ b/content/releases/1.0.4/Serialization-(prior-to-0.6.0).html
@@ -82,18 +82,16 @@
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
+ <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
+
<li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
<li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
@@ -104,6 +102,8 @@
+
+
<li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
@@ -144,7 +144,7 @@
<li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
</ul>
</li>
- <li><a href="/2017/09/15/storm105-released.html" id="news">News</a></li>
+ <li><a href="/2018/02/14/storm106-released.html" id="news">News</a></li>
</ul>
</nav>
</div>
@@ -182,9 +182,9 @@
<p>The interface looks like this:</p>
<div class="highlight"><pre><code class="language-java" data-lang="java"><span class="kd">public</span> <span class="kd">interface</span> <span class="nc">ISerialization</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="o">{</span>
- <span class="kd">public</span> <span class="kt">boolean</span> <span class="n">accept</span><span class="o">(</span><span class="n">Class</span> <span class="n">c</span><span class="o">);</span>
- <span class="kd">public</span> <span class="kt">void</span> <span class="n">serialize</span><span class="o">(</span><span class="n">T</span> <span class="n">object</span><span class="o">,</span> <span class="n">DataOutputStream</span> <span class="n">stream</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">IOException</span><span class="o">;</span>
- <span class="kd">public</span> <span class="n">T</span> <span class="n">deserialize</span><span class="o">(</span><span class="n">DataInputStream</span> <span class="n">stream</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">IOException</span><span class="o">;</span>
+ <span class="kd">public</span> <span class="kt">boolean</span> <span class="nf">accept</span><span class="o">(</span><span class="n">Class</span> <span class="n">c</span><span class="o">);</span>
+ <span class="kd">public</span> <span class="kt">void</span> <span class="nf">serialize</span><span class="o">(</span><span class="n">T</span> <span class="n">object</span><span class="o">,</span> <span class="n">DataOutputStream</span> <span class="n">stream</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">IOException</span><span class="o">;</span>
+ <span class="kd">public</span> <span class="n">T</span> <span class="nf">deserialize</span><span class="o">(</span><span class="n">DataInputStream</span> <span class="n">stream</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">IOException</span><span class="o">;</span>
<span class="o">}</span>
</code></pre></div>
<p>Storm uses the <code>accept</code> method to determine if a type can be serialized by this serializer. Remember, Storm's tuples are dynamically typed so Storm determines what serializer to use at runtime.</p>
http://git-wip-us.apache.org/repos/asf/storm-site/blob/a7a19afb/content/releases/1.0.4/Serialization.html
----------------------------------------------------------------------
diff --git a/content/releases/1.0.4/Serialization.html b/content/releases/1.0.4/Serialization.html
index 483a2ea..e6f9ffb 100644
--- a/content/releases/1.0.4/Serialization.html
+++ b/content/releases/1.0.4/Serialization.html
@@ -82,18 +82,16 @@
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
+ <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
+
<li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
<li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
@@ -104,6 +102,8 @@
+
+
<li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
@@ -144,7 +144,7 @@
<li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
</ul>
</li>
- <li><a href="/2017/09/15/storm105-released.html" id="news">News</a></li>
+ <li><a href="/2018/02/14/storm106-released.html" id="news">News</a></li>
</ul>
</nav>
</div>
http://git-wip-us.apache.org/repos/asf/storm-site/blob/a7a19afb/content/releases/1.0.4/Serializers.html
----------------------------------------------------------------------
diff --git a/content/releases/1.0.4/Serializers.html b/content/releases/1.0.4/Serializers.html
index 06a500e..89e4d95 100644
--- a/content/releases/1.0.4/Serializers.html
+++ b/content/releases/1.0.4/Serializers.html
@@ -82,18 +82,16 @@
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
+ <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
+
<li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
<li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
@@ -104,6 +102,8 @@
+
+
<li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
@@ -144,7 +144,7 @@
<li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
</ul>
</li>
- <li><a href="/2017/09/15/storm105-released.html" id="news">News</a></li>
+ <li><a href="/2018/02/14/storm106-released.html" id="news">News</a></li>
</ul>
</nav>
</div>
http://git-wip-us.apache.org/repos/asf/storm-site/blob/a7a19afb/content/releases/1.0.4/Setting-up-a-Storm-cluster.html
----------------------------------------------------------------------
diff --git a/content/releases/1.0.4/Setting-up-a-Storm-cluster.html b/content/releases/1.0.4/Setting-up-a-Storm-cluster.html
index b7723b6..a355a33 100644
--- a/content/releases/1.0.4/Setting-up-a-Storm-cluster.html
+++ b/content/releases/1.0.4/Setting-up-a-Storm-cluster.html
@@ -82,18 +82,16 @@
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
+ <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
+
<li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
<li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
@@ -104,6 +102,8 @@
+
+
<li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
@@ -144,7 +144,7 @@
<li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
</ul>
</li>
- <li><a href="/2017/09/15/storm105-released.html" id="news">News</a></li>
+ <li><a href="/2018/02/14/storm106-released.html" id="news">News</a></li>
</ul>
</nav>
</div>
@@ -251,7 +251,7 @@ The time to allow any given healthcheck script to run before it is marked failed
<p>If you need support from external libraries or custom plugins, you can place such jars into the extlib/ and extlib-daemon/ directories. Note that the extlib-daemon/ directory stores jars used only by daemons (Nimbus, Supervisor, DRPC, UI, Logviewer), e.g., HDFS and customized scheduling libraries. Accordingly, two environmental variables STORM_EXT_CLASSPATH and STORM_EXT_CLASSPATH_DAEMON can be configured by users for including the external classpath and daemon-only external classpath.</p>
-<h3 id="launch-daemons-under-supervision-using-quot-storm-quot-script-and-a-supervisor-of-your-choice">Launch daemons under supervision using "storm" script and a supervisor of your choice</h3>
+<h3 id="launch-daemons-under-supervision-using-storm-script-and-a-supervisor-of-your-choice">Launch daemons under supervision using "storm" script and a supervisor of your choice</h3>
<p>The last step is to launch all the Storm daemons. It is critical that you run each of these daemons under supervision. Storm is a <strong>fail-fast</strong> system which means the processes will halt whenever an unexpected error is encountered. Storm is designed so that it can safely halt at any point and recover correctly when the process is restarted. This is why Storm keeps no state in-process -- if Nimbus or the Supervisors restart, the running topologies are unaffected. Here's how to run the Storm daemons:</p>
http://git-wip-us.apache.org/repos/asf/storm-site/blob/a7a19afb/content/releases/1.0.4/Setting-up-development-environment.html
----------------------------------------------------------------------
diff --git a/content/releases/1.0.4/Setting-up-development-environment.html b/content/releases/1.0.4/Setting-up-development-environment.html
index 569a0e3..049525b 100644
--- a/content/releases/1.0.4/Setting-up-development-environment.html
+++ b/content/releases/1.0.4/Setting-up-development-environment.html
@@ -82,18 +82,16 @@
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
+ <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
+
<li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
<li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
@@ -104,6 +102,8 @@
+
+
<li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
@@ -144,7 +144,7 @@
<li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
</ul>
</li>
- <li><a href="/2017/09/15/storm105-released.html" id="news">News</a></li>
+ <li><a href="/2018/02/14/storm106-released.html" id="news">News</a></li>
</ul>
</nav>
</div>
http://git-wip-us.apache.org/repos/asf/storm-site/blob/a7a19afb/content/releases/1.0.4/Spout-implementations.html
----------------------------------------------------------------------
diff --git a/content/releases/1.0.4/Spout-implementations.html b/content/releases/1.0.4/Spout-implementations.html
index 509f1f8..d58c34e 100644
--- a/content/releases/1.0.4/Spout-implementations.html
+++ b/content/releases/1.0.4/Spout-implementations.html
@@ -82,18 +82,16 @@
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
+ <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
+
<li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
<li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
@@ -104,6 +102,8 @@
+
+
<li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
@@ -144,7 +144,7 @@
<li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
</ul>
</li>
- <li><a href="/2017/09/15/storm105-released.html" id="news">News</a></li>
+ <li><a href="/2018/02/14/storm106-released.html" id="news">News</a></li>
</ul>
</nav>
</div>
http://git-wip-us.apache.org/repos/asf/storm-site/blob/a7a19afb/content/releases/1.0.4/State-checkpointing.html
----------------------------------------------------------------------
diff --git a/content/releases/1.0.4/State-checkpointing.html b/content/releases/1.0.4/State-checkpointing.html
index efc6950..1a3db29 100644
--- a/content/releases/1.0.4/State-checkpointing.html
+++ b/content/releases/1.0.4/State-checkpointing.html
@@ -82,18 +82,16 @@
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
+ <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
+
<li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
<li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
@@ -104,6 +102,8 @@
+
+
<li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
@@ -144,7 +144,7 @@
<li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
</ul>
</li>
- <li><a href="/2017/09/15/storm105-released.html" id="news">News</a></li>
+ <li><a href="/2018/02/14/storm106-released.html" id="news">News</a></li>
</ul>
</nav>
</div>
@@ -187,15 +187,15 @@ last committed by the framework during the previous run.</li>
<span class="kd">private</span> <span class="n">OutputCollector</span> <span class="n">collector</span><span class="o">;</span>
<span class="o">...</span>
<span class="nd">@Override</span>
- <span class="kd">public</span> <span class="kt">void</span> <span class="n">prepare</span><span class="o">(</span><span class="n">Map</span> <span class="n">stormConf</span><span class="o">,</span> <span class="n">TopologyContext</span> <span class="n">context</span><span class="o">,</span> <span class="n">OutputCollector</span> <span class="n">collector</span><span class="o">)</span> <span class="o">{</span>
+ <span class="kd">public</span> <span class="kt">void</span> <span class="nf">prepare</span><span class="o">(</span><span class="n">Map</span> <span class="n">stormConf</span><span class="o">,</span> <span class="n">TopologyContext</span> <span class="n">context</span><span class="o">,</span> <span class="n">OutputCollector</span> <span class="n">collector</span><span class="o">)</span> <span class="o">{</span>
<span class="k">this</span><span class="o">.</span><span class="na">collector</span> <span class="o">=</span> <span class="n">collector</span><span class="o">;</span>
<span class="o">}</span>
<span class="nd">@Override</span>
- <span class="kd">public</span> <span class="kt">void</span> <span class="n">initState</span><span class="o">(</span><span class="n">KeyValueState</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Long</span><span class="o">></span> <span class="n">state</span><span class="o">)</span> <span class="o">{</span>
+ <span class="kd">public</span> <span class="kt">void</span> <span class="nf">initState</span><span class="o">(</span><span class="n">KeyValueState</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Long</span><span class="o">></span> <span class="n">state</span><span class="o">)</span> <span class="o">{</span>
<span class="n">wordCounts</span> <span class="o">=</span> <span class="n">state</span><span class="o">;</span>
<span class="o">}</span>
<span class="nd">@Override</span>
- <span class="kd">public</span> <span class="kt">void</span> <span class="n">execute</span><span class="o">(</span><span class="n">Tuple</span> <span class="n">tuple</span><span class="o">)</span> <span class="o">{</span>
+ <span class="kd">public</span> <span class="kt">void</span> <span class="nf">execute</span><span class="o">(</span><span class="n">Tuple</span> <span class="n">tuple</span><span class="o">)</span> <span class="o">{</span>
<span class="n">String</span> <span class="n">word</span> <span class="o">=</span> <span class="n">tuple</span><span class="o">.</span><span class="na">getString</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span>
<span class="n">Integer</span> <span class="n">count</span> <span class="o">=</span> <span class="n">wordCounts</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="n">word</span><span class="o">,</span> <span class="mi">0</span><span class="o">);</span>
<span class="n">count</span><span class="o">++;</span>
@@ -216,20 +216,20 @@ in the extlib directory.</li>
<li>The state provider properties can be overridden by setting <code>topology.state.provider.config</code>. For Redis state this is a
json config with the following properties.</li>
</ol>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"keyClass"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Optional fully qualified class name of the Key type."</span><span class="p">,</span><span class="w">
- </span><span class="nt">"valueClass"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Optional fully qualified class name of the Value type."</span><span class="p">,</span><span class="w">
- </span><span class="nt">"keySerializerClass"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Optional Key serializer implementation class."</span><span class="p">,</span><span class="w">
- </span><span class="nt">"valueSerializerClass"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Optional Value Serializer implementation class."</span><span class="p">,</span><span class="w">
- </span><span class="nt">"jedisPoolConfig"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"host"</span><span class="p">:</span><span class="w"> </span><span class="s2">"localhost"</span><span class="p">,</span><span class="w">
- </span><span class="nt">"port"</span><span class="p">:</span><span class="w"> </span><span class="mi">6379</span><span class="p">,</span><span class="w">
- </span><span class="nt">"timeout"</span><span class="p">:</span><span class="w"> </span><span class="mi">2000</span><span class="p">,</span><span class="w">
- </span><span class="nt">"database"</span><span class="p">:</span><span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w">
- </span><span class="nt">"password"</span><span class="p">:</span><span class="w"> </span><span class="s2">"xyz"</span><span class="w">
- </span><span class="p">}</span><span class="w">
- </span><span class="p">}</span><span class="w">
-</span></code></pre></div>
+<div class="highlight"><pre><code class="language-" data-lang=""> {
+ "keyClass": "Optional fully qualified class name of the Key type.",
+ "valueClass": "Optional fully qualified class name of the Value type.",
+ "keySerializerClass": "Optional Key serializer implementation class.",
+ "valueSerializerClass": "Optional Value Serializer implementation class.",
+ "jedisPoolConfig": {
+ "host": "localhost",
+ "port": 6379,
+ "timeout": 2000,
+ "database": 0,
+ "password": "xyz"
+ }
+ }
+</code></pre></div>
<h2 id="checkpoint-mechanism">Checkpoint mechanism</h2>
<p>Checkpoint is triggered by an internal checkpoint spout at the specified <code>topology.state.checkpoint.interval.ms</code>. If there is
http://git-wip-us.apache.org/repos/asf/storm-site/blob/a7a19afb/content/releases/1.0.4/Storm-Scheduler.html
----------------------------------------------------------------------
diff --git a/content/releases/1.0.4/Storm-Scheduler.html b/content/releases/1.0.4/Storm-Scheduler.html
index 40fabbd..7d6d770 100644
--- a/content/releases/1.0.4/Storm-Scheduler.html
+++ b/content/releases/1.0.4/Storm-Scheduler.html
@@ -82,18 +82,16 @@
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
+ <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
+
<li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
<li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
@@ -104,6 +102,8 @@
+
+
<li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
@@ -144,7 +144,7 @@
<li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
</ul>
</li>
- <li><a href="/2017/09/15/storm105-released.html" id="news">News</a></li>
+ <li><a href="/2018/02/14/storm106-released.html" id="news">News</a></li>
</ul>
</nav>
</div>
http://git-wip-us.apache.org/repos/asf/storm-site/blob/a7a19afb/content/releases/1.0.4/Storm-multi-language-protocol-(versions-0.7.0-and-below).html
----------------------------------------------------------------------
diff --git a/content/releases/1.0.4/Storm-multi-language-protocol-(versions-0.7.0-and-below).html b/content/releases/1.0.4/Storm-multi-language-protocol-(versions-0.7.0-and-below).html
index 0ef2189..b4cf896 100644
--- a/content/releases/1.0.4/Storm-multi-language-protocol-(versions-0.7.0-and-below).html
+++ b/content/releases/1.0.4/Storm-multi-language-protocol-(versions-0.7.0-and-below).html
@@ -82,18 +82,16 @@
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
+ <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
+
<li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
<li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
@@ -104,6 +102,8 @@
+
+
<li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
@@ -144,7 +144,7 @@
<li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
</ul>
</li>
- <li><a href="/2017/09/15/storm105-released.html" id="news">News</a></li>
+ <li><a href="/2018/02/14/storm106-released.html" id="news">News</a></li>
</ul>
</nav>
</div>
@@ -210,56 +210,56 @@ file lets the supervisor know the PID so it can shutdown the process later on.</
<li>The rest happens in a while(true) loop</li>
<li>STDIN: A tuple! This is a JSON encoded structure like this:</li>
</ul>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">tuple's</span><span class="w"> </span><span class="err">id</span><span class="w">
- </span><span class="nt">"id"</span><span class="p">:</span><span class="w"> </span><span class="mi">-6955786537413359385</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">of</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">component</span><span class="w"> </span><span class="err">that</span><span class="w"> </span><span class="err">created</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">tuple</span><span class="w">
- </span><span class="nt">"comp"</span><span class="p">:</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">of</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">stream</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">tuple</span><span class="w"> </span><span class="err">was</span><span class="w"> </span><span class="err">emitted</span><span class="w"> </span><span class="err">to</span><span class="w">
- </span><span class="nt">"stream"</span><span class="p">:</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">of</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">task</span><span class="w"> </span><span class="err">that</span><span class="w"> </span><span class="err">created</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">tuple</span><span class="w">
- </span><span class="nt">"task"</span><span class="p">:</span><span class="w"> </span><span class="mi">9</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">All</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">values</span><span class="w"> </span><span class="err">in</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">tuple</span><span class="w">
- </span><span class="nt">"tuple"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"snow white and the seven dwarfs"</span><span class="p">,</span><span class="w"> </span><span class="s2">"field2"</span><span class="p">,</span><span class="w"> </span><span class="mi">3</span><span class="p">]</span><span class="w">
-</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
+<div class="highlight"><pre><code class="language-" data-lang="">{
+ // The tuple's id
+ "id": -6955786537413359385,
+ // The id of the component that created this tuple
+ "comp": 1,
+ // The id of the stream this tuple was emitted to
+ "stream": 1,
+ // The id of the task that created this tuple
+ "task": 9,
+ // All the values in this tuple
+ "tuple": ["snow white and the seven dwarfs", "field2", 3]
+}
+</code></pre></div>
<ul>
<li>STDOUT: The results of your bolt, JSON encoded. This can be a sequence of acks, fails, emits, and/or logs. Emits look like:</li>
</ul>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="w">
- </span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"emit"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">ids</span><span class="w"> </span><span class="err">of</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">tuples</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">output</span><span class="w"> </span><span class="err">tuples</span><span class="w"> </span><span class="err">should</span><span class="w"> </span><span class="err">be</span><span class="w"> </span><span class="err">anchored</span><span class="w"> </span><span class="err">to</span><span class="w">
- </span><span class="nt">"anchors"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="mi">1231231</span><span class="p">,</span><span class="w"> </span><span class="mi">-234234234</span><span class="p">],</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">of</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">stream</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">tuple</span><span class="w"> </span><span class="err">was</span><span class="w"> </span><span class="err">emitted</span><span class="w"> </span><span class="err">to.</span><span class="w"> </span><span class="err">Leave</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">empty</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">emit</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">default</span><span class="w"> </span><span class="err">stream.</span><sp
an class="w">
- </span><span class="nt">"stream"</span><span class="p">:</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">If</span><span class="w"> </span><span class="err">doing</span><span class="w"> </span><span class="err">an</span><span class="w"> </span><span class="err">emit</span><span class="w"> </span><span class="err">direct,</span><span class="w"> </span><span class="err">indicate</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">task</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">sent</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">tuple</span><span class="w"> </span><span class="err">to</span><span class="w">
- </span><span class="nt">"task"</span><span class="p">:</span><span class="w"> </span><span class="mi">9</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">All</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">values</span><span class="w"> </span><span class="err">in</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">tuple</span><span class="w">
- </span><span class="nt">"tuple"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"field1"</span><span class="p">,</span><span class="w"> </span><span class="mi">2</span><span class="p">,</span><span class="w"> </span><span class="mi">3</span><span class="p">]</span><span class="w">
-</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
+<div class="highlight"><pre><code class="language-" data-lang="">{
+ "command": "emit",
+ // The ids of the tuples this output tuples should be anchored to
+ "anchors": [1231231, -234234234],
+ // The id of the stream this tuple was emitted to. Leave this empty to emit to default stream.
+ "stream": 1,
+ // If doing an emit direct, indicate the task to sent the tuple to
+ "task": 9,
+ // All the values in this tuple
+ "tuple": ["field1", 2, 3]
+}
+</code></pre></div>
<p>An ack looks like:</p>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="w">
- </span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"ack"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">of</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">tuple</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">ack</span><span class="w">
- </span><span class="nt">"id"</span><span class="p">:</span><span class="w"> </span><span class="mi">123123</span><span class="w">
-</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
+<div class="highlight"><pre><code class="language-" data-lang="">{
+ "command": "ack",
+ // the id of the tuple to ack
+ "id": 123123
+}
+</code></pre></div>
<p>A fail looks like:</p>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="w">
- </span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"fail"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">of</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">tuple</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">fail</span><span class="w">
- </span><span class="nt">"id"</span><span class="p">:</span><span class="w"> </span><span class="mi">123123</span><span class="w">
-</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
+<div class="highlight"><pre><code class="language-" data-lang="">{
+ "command": "fail",
+ // the id of the tuple to fail
+ "id": 123123
+}
+</code></pre></div>
<p>A "log" will log a message in the worker log. It looks like:</p>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="w">
- </span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"log"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">message</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">log</span><span class="w">
- </span><span class="nt">"msg"</span><span class="p">:</span><span class="w"> </span><span class="s2">"hello world!"</span><span class="w">
+<div class="highlight"><pre><code class="language-" data-lang="">{
+ "command": "log",
+ // the message to log
+ "msg": "hello world!"
-</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
+}
+</code></pre></div>
<ul>
<li>STDOUT: emit "sync" as a single line by itself when the bolt has finished emitting/acking/failing and is ready for the next input</li>
</ul>
http://git-wip-us.apache.org/repos/asf/storm-site/blob/a7a19afb/content/releases/1.0.4/Structure-of-the-codebase.html
----------------------------------------------------------------------
diff --git a/content/releases/1.0.4/Structure-of-the-codebase.html b/content/releases/1.0.4/Structure-of-the-codebase.html
index 200bc46..43c59e7 100644
--- a/content/releases/1.0.4/Structure-of-the-codebase.html
+++ b/content/releases/1.0.4/Structure-of-the-codebase.html
@@ -82,18 +82,16 @@
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
+ <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
+
<li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
<li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
@@ -104,6 +102,8 @@
+
+
<li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
@@ -144,7 +144,7 @@
<li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
</ul>
</li>
- <li><a href="/2017/09/15/storm105-released.html" id="news">News</a></li>
+ <li><a href="/2018/02/14/storm106-released.html" id="news">News</a></li>
</ul>
</nav>
</div>
http://git-wip-us.apache.org/repos/asf/storm-site/blob/a7a19afb/content/releases/1.0.4/Support-for-non-java-languages.html
----------------------------------------------------------------------
diff --git a/content/releases/1.0.4/Support-for-non-java-languages.html b/content/releases/1.0.4/Support-for-non-java-languages.html
index 34c823b..ea628e2 100644
--- a/content/releases/1.0.4/Support-for-non-java-languages.html
+++ b/content/releases/1.0.4/Support-for-non-java-languages.html
@@ -82,18 +82,16 @@
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
+ <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
+
<li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
<li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
@@ -104,6 +102,8 @@
+
+
<li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
@@ -144,7 +144,7 @@
<li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
</ul>
</li>
- <li><a href="/2017/09/15/storm105-released.html" id="news">News</a></li>
+ <li><a href="/2018/02/14/storm106-released.html" id="news">News</a></li>
</ul>
</nav>
</div>
http://git-wip-us.apache.org/repos/asf/storm-site/blob/a7a19afb/content/releases/1.0.4/Transactional-topologies.html
----------------------------------------------------------------------
diff --git a/content/releases/1.0.4/Transactional-topologies.html b/content/releases/1.0.4/Transactional-topologies.html
index a5f0245..913ff4e 100644
--- a/content/releases/1.0.4/Transactional-topologies.html
+++ b/content/releases/1.0.4/Transactional-topologies.html
@@ -82,18 +82,16 @@
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
+ <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
+
<li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
<li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
@@ -104,6 +102,8 @@
+
+
<li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
@@ -144,7 +144,7 @@
<li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
</ul>
</li>
- <li><a href="/2017/09/15/storm105-released.html" id="news">News</a></li>
+ <li><a href="/2018/02/14/storm106-released.html" id="news">News</a></li>
</ul>
</nav>
</div>
@@ -211,7 +211,7 @@
<p>After bolt 1 finishes its portion of the processing, it will be idle until the rest of the bolts finish and the next batch can be emitted from the spout.</p>
-<h3 id="design-3-storm-39-s-design">Design 3 (Storm's design)</h3>
+<h3 id="design-3-storms-design">Design 3 (Storm's design)</h3>
<p>A key realization is that not all the work for processing batches of tuples needs to be strongly ordered. For example, when computing a global count, there's two parts to the computation:</p>
@@ -268,23 +268,23 @@
<span class="kt">int</span> <span class="n">_count</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span>
<span class="nd">@Override</span>
- <span class="kd">public</span> <span class="kt">void</span> <span class="n">prepare</span><span class="o">(</span><span class="n">Map</span> <span class="n">conf</span><span class="o">,</span> <span class="n">TopologyContext</span> <span class="n">context</span><span class="o">,</span> <span class="n">BatchOutputCollector</span> <span class="n">collector</span><span class="o">,</span> <span class="n">Object</span> <span class="n">id</span><span class="o">)</span> <span class="o">{</span>
+ <span class="kd">public</span> <span class="kt">void</span> <span class="nf">prepare</span><span class="o">(</span><span class="n">Map</span> <span class="n">conf</span><span class="o">,</span> <span class="n">TopologyContext</span> <span class="n">context</span><span class="o">,</span> <span class="n">BatchOutputCollector</span> <span class="n">collector</span><span class="o">,</span> <span class="n">Object</span> <span class="n">id</span><span class="o">)</span> <span class="o">{</span>
<span class="n">_collector</span> <span class="o">=</span> <span class="n">collector</span><span class="o">;</span>
<span class="n">_id</span> <span class="o">=</span> <span class="n">id</span><span class="o">;</span>
<span class="o">}</span>
<span class="nd">@Override</span>
- <span class="kd">public</span> <span class="kt">void</span> <span class="n">execute</span><span class="o">(</span><span class="n">Tuple</span> <span class="n">tuple</span><span class="o">)</span> <span class="o">{</span>
+ <span class="kd">public</span> <span class="kt">void</span> <span class="nf">execute</span><span class="o">(</span><span class="n">Tuple</span> <span class="n">tuple</span><span class="o">)</span> <span class="o">{</span>
<span class="n">_count</span><span class="o">++;</span>
<span class="o">}</span>
<span class="nd">@Override</span>
- <span class="kd">public</span> <span class="kt">void</span> <span class="n">finishBatch</span><span class="o">()</span> <span class="o">{</span>
+ <span class="kd">public</span> <span class="kt">void</span> <span class="nf">finishBatch</span><span class="o">()</span> <span class="o">{</span>
<span class="n">_collector</span><span class="o">.</span><span class="na">emit</span><span class="o">(</span><span class="k">new</span> <span class="n">Values</span><span class="o">(</span><span class="n">_id</span><span class="o">,</span> <span class="n">_count</span><span class="o">));</span>
<span class="o">}</span>
<span class="nd">@Override</span>
- <span class="kd">public</span> <span class="kt">void</span> <span class="n">declareOutputFields</span><span class="o">(</span><span class="n">OutputFieldsDeclarer</span> <span class="n">declarer</span><span class="o">)</span> <span class="o">{</span>
+ <span class="kd">public</span> <span class="kt">void</span> <span class="nf">declareOutputFields</span><span class="o">(</span><span class="n">OutputFieldsDeclarer</span> <span class="n">declarer</span><span class="o">)</span> <span class="o">{</span>
<span class="n">declarer</span><span class="o">.</span><span class="na">declare</span><span class="o">(</span><span class="k">new</span> <span class="n">Fields</span><span class="o">(</span><span class="s">"id"</span><span class="o">,</span> <span class="s">"count"</span><span class="o">));</span>
<span class="o">}</span>
<span class="o">}</span>
@@ -313,18 +313,18 @@
<span class="kt">int</span> <span class="n">_sum</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span>
<span class="nd">@Override</span>
- <span class="kd">public</span> <span class="kt">void</span> <span class="n">prepare</span><span class="o">(</span><span class="n">Map</span> <span class="n">conf</span><span class="o">,</span> <span class="n">TopologyContext</span> <span class="n">context</span><span class="o">,</span> <span class="n">BatchOutputCollector</span> <span class="n">collector</span><span class="o">,</span> <span class="n">TransactionAttempt</span> <span class="n">attempt</span><span class="o">)</span> <span class="o">{</span>
+ <span class="kd">public</span> <span class="kt">void</span> <span class="nf">prepare</span><span class="o">(</span><span class="n">Map</span> <span class="n">conf</span><span class="o">,</span> <span class="n">TopologyContext</span> <span class="n">context</span><span class="o">,</span> <span class="n">BatchOutputCollector</span> <span class="n">collector</span><span class="o">,</span> <span class="n">TransactionAttempt</span> <span class="n">attempt</span><span class="o">)</span> <span class="o">{</span>
<span class="n">_collector</span> <span class="o">=</span> <span class="n">collector</span><span class="o">;</span>
<span class="n">_attempt</span> <span class="o">=</span> <span class="n">attempt</span><span class="o">;</span>
<span class="o">}</span>
<span class="nd">@Override</span>
- <span class="kd">public</span> <span class="kt">void</span> <span class="n">execute</span><span class="o">(</span><span class="n">Tuple</span> <span class="n">tuple</span><span class="o">)</span> <span class="o">{</span>
+ <span class="kd">public</span> <span class="kt">void</span> <span class="nf">execute</span><span class="o">(</span><span class="n">Tuple</span> <span class="n">tuple</span><span class="o">)</span> <span class="o">{</span>
<span class="n">_sum</span><span class="o">+=</span><span class="n">tuple</span><span class="o">.</span><span class="na">getInteger</span><span class="o">(</span><span class="mi">1</span><span class="o">);</span>
<span class="o">}</span>
<span class="nd">@Override</span>
- <span class="kd">public</span> <span class="kt">void</span> <span class="n">finishBatch</span><span class="o">()</span> <span class="o">{</span>
+ <span class="kd">public</span> <span class="kt">void</span> <span class="nf">finishBatch</span><span class="o">()</span> <span class="o">{</span>
<span class="n">Value</span> <span class="n">val</span> <span class="o">=</span> <span class="n">DATABASE</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="n">GLOBAL_COUNT_KEY</span><span class="o">);</span>
<span class="n">Value</span> <span class="n">newval</span><span class="o">;</span>
<span class="k">if</span><span class="o">(</span><span class="n">val</span> <span class="o">==</span> <span class="kc">null</span> <span class="o">||</span> <span class="o">!</span><span class="n">val</span><span class="o">.</span><span class="na">txid</span><span class="o">.</span><span class="na">equals</span><span class="o">(</span><span class="n">_attempt</span><span class="o">.</span><span class="na">getTransactionId</span><span class="o">()))</span> <span class="o">{</span>
@@ -343,7 +343,7 @@
<span class="o">}</span>
<span class="nd">@Override</span>
- <span class="kd">public</span> <span class="kt">void</span> <span class="n">declareOutputFields</span><span class="o">(</span><span class="n">OutputFieldsDeclarer</span> <span class="n">declarer</span><span class="o">)</span> <span class="o">{</span>
+ <span class="kd">public</span> <span class="kt">void</span> <span class="nf">declareOutputFields</span><span class="o">(</span><span class="n">OutputFieldsDeclarer</span> <span class="n">declarer</span><span class="o">)</span> <span class="o">{</span>
<span class="n">declarer</span><span class="o">.</span><span class="na">declare</span><span class="o">(</span><span class="k">new</span> <span class="n">Fields</span><span class="o">(</span><span class="s">"id"</span><span class="o">,</span> <span class="s">"sum"</span><span class="o">));</span>
<span class="o">}</span>
<span class="o">}</span>
@@ -421,7 +421,7 @@
<li><em>Number of active batches permissible at once:</em> You must set a limit to the number of batches that can be processed at once. You configure this using the "topology.max.spout.pending" config. If you don't set this config, it will default to 1.</li>
</ol>
-<h2 id="what-if-you-can-39-t-emit-the-same-batch-of-tuples-for-a-given-transaction-id">What if you can't emit the same batch of tuples for a given transaction id?</h2>
+<h2 id="what-if-you-cant-emit-the-same-batch-of-tuples-for-a-given-transaction-id">What if you can't emit the same batch of tuples for a given transaction id?</h2>
<p>So far the discussion around transactional topologies has assumed that you can always emit the exact same batch of tuples for the same transaction id. So what do you do if this is not possible?</p>