You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@storm.apache.org by pt...@apache.org on 2018/02/14 20:57:35 UTC
[11/51] [partial] storm-site git commit: regenerate site for download
page changes
http://git-wip-us.apache.org/repos/asf/storm-site/blob/a7a19afb/content/releases/1.0.4/Multilang-protocol.html
----------------------------------------------------------------------
diff --git a/content/releases/1.0.4/Multilang-protocol.html b/content/releases/1.0.4/Multilang-protocol.html
index 7d755cc..843c15c 100644
--- a/content/releases/1.0.4/Multilang-protocol.html
+++ b/content/releases/1.0.4/Multilang-protocol.html
@@ -82,18 +82,16 @@
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
+ <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
+
<li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
<li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
@@ -104,6 +102,8 @@
+
+
<li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
@@ -144,7 +144,7 @@
<li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
</ul>
</li>
- <li><a href="/2017/09/15/storm105-released.html" id="news">News</a></li>
+ <li><a href="/2018/02/14/storm106-released.html" id="news">News</a></li>
</ul>
</nav>
</div>
@@ -211,44 +211,44 @@ STDIN and STDOUT.</li>
<ul>
<li>STDIN: Setup info. This is a JSON object with the Storm configuration, a PID directory, and a topology context, like this:</li>
</ul>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="w">
- </span><span class="nt">"conf"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"topology.message.timeout.secs"</span><span class="p">:</span><span class="w"> </span><span class="mi">3</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">etc</span><span class="w">
- </span><span class="err">}</span><span class="p">,</span><span class="w">
- </span><span class="nt">"pidDir"</span><span class="p">:</span><span class="w"> </span><span class="s2">"..."</span><span class="p">,</span><span class="w">
- </span><span class="nt">"context"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"task->component"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"1"</span><span class="p">:</span><span class="w"> </span><span class="s2">"example-spout"</span><span class="p">,</span><span class="w">
- </span><span class="nt">"2"</span><span class="p">:</span><span class="w"> </span><span class="s2">"__acker"</span><span class="p">,</span><span class="w">
- </span><span class="nt">"3"</span><span class="p">:</span><span class="w"> </span><span class="s2">"example-bolt1"</span><span class="p">,</span><span class="w">
- </span><span class="nt">"4"</span><span class="p">:</span><span class="w"> </span><span class="s2">"example-bolt2"</span><span class="w">
- </span><span class="p">},</span><span class="w">
- </span><span class="nt">"taskid"</span><span class="p">:</span><span class="w"> </span><span class="mi">3</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">Everything</span><span class="w"> </span><span class="err">below</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">line</span><span class="w"> </span><span class="err">is</span><span class="w"> </span><span class="err">only</span><span class="w"> </span><span class="err">available</span><span class="w"> </span><span class="err">in</span><span class="w"> </span><span class="err">Storm</span><span class="w"> </span><span class="err">0.10.0+</span><span class="w">
- </span><span class="nt">"componentid"</span><span class="p">:</span><span class="w"> </span><span class="s2">"example-bolt"</span><span class="w">
- </span><span class="s2">"stream->target->grouping"</span><span class="err">:</span><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"default"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"example-bolt2"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"SHUFFLE"</span><span class="p">}}},</span><span class="w">
- </span><span class="nt">"streams"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"default"</span><span class="p">],</span><span class="w">
- </span><span class="nt">"stream->outputfields"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="nt">"default"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"word"</span><span class="p">]},</span><span class="w">
- </span><span class="nt">"source->stream->grouping"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"example-spout"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"default"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"FIELDS"</span><span class="p">,</span><span class="w">
- </span><span class="nt">"fields"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"word"</span><span class="p">]</span><span class="w">
- </span><span class="p">}</span><span class="w">
- </span><span class="p">}</span><span class="w">
- </span><span class="p">}</span><span class="w">
- </span><span class="s2">"source->stream->fields"</span><span class="err">:</span><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"example-spout"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"default"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"word"</span><span class="p">]</span><span class="w">
- </span><span class="p">}</span><span class="w">
- </span><span class="p">}</span><span class="w">
- </span><span class="p">}</span><span class="w">
-</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
+<div class="highlight"><pre><code class="language-" data-lang="">{
+ "conf": {
+ "topology.message.timeout.secs": 3,
+ // etc
+ },
+ "pidDir": "...",
+ "context": {
+ "task->component": {
+ "1": "example-spout",
+ "2": "__acker",
+ "3": "example-bolt1",
+ "4": "example-bolt2"
+ },
+ "taskid": 3,
+ // Everything below this line is only available in Storm 0.10.0+
+ "componentid": "example-bolt"
+ "stream->target->grouping": {
+ "default": {
+ "example-bolt2": {
+ "type": "SHUFFLE"}}},
+ "streams": ["default"],
+ "stream->outputfields": {"default": ["word"]},
+ "source->stream->grouping": {
+ "example-spout": {
+ "default": {
+ "type": "FIELDS",
+ "fields": ["word"]
+ }
+ }
+ }
+ "source->stream->fields": {
+ "example-spout": {
+ "default": ["word"]
+ }
+ }
+ }
+}
+</code></pre></div>
<p>Your script should create an empty file named with its PID in this directory. e.g.
the PID is 1234, so an empty file named 1234 is created in the directory. This
file lets the supervisor know the PID so it can shutdown the process later on.</p>
@@ -277,47 +277,47 @@ to specify which fields are involved in a <code>FIELDS</code> grouping.</p>
</ul>
<p>"next" is the equivalent of ISpout's <code>nextTuple</code>. It looks like:</p>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"next"</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
+<div class="highlight"><pre><code class="language-" data-lang="">{"command": "next"}
+</code></pre></div>
<p>"ack" looks like:</p>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"ack"</span><span class="p">,</span><span class="w"> </span><span class="nt">"id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"1231231"</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
+<div class="highlight"><pre><code class="language-" data-lang="">{"command": "ack", "id": "1231231"}
+</code></pre></div>
<p>"fail" looks like:</p>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"fail"</span><span class="p">,</span><span class="w"> </span><span class="nt">"id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"1231231"</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
+<div class="highlight"><pre><code class="language-" data-lang="">{"command": "fail", "id": "1231231"}
+</code></pre></div>
<ul>
<li>STDOUT: The results of your spout for the previous command. This can
be a sequence of emits and logs.</li>
</ul>
<p>An emit looks like:</p>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="w">
- </span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"emit"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">for</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">tuple.</span><span class="w"> </span><span class="err">Leave</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">out</span><span class="w"> </span><span class="err">for</span><span class="w"> </span><span class="err">an</span><span class="w"> </span><span class="err">unreliable</span><span class="w"> </span><span class="err">emit.</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">can</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">be</span><span class="w"> </span><span class="err">a</span><span class="w"> </span><span class="err">string</span><span class="w"> </span><span class="err">or</span><span class="w"> </span><span class="err">a</span><span class="w"> </span><span class="err">number.</span><span class="w">
- </span><span class="nt">"id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"1231231"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">of</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">stream</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">tuple</span><span class="w"> </span><span class="err">was</span><span class="w"> </span><span class="err">emitted</span><span class="w"> </span><span class="err">to.</span><span class="w"> </span><span class="err">Leave</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">empty</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">emit</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">default</span><span class="w"> </span><span class="err">stream.</span><sp
an class="w">
- </span><span class="nt">"stream"</span><span class="p">:</span><span class="w"> </span><span class="s2">"1"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">If</span><span class="w"> </span><span class="err">doing</span><span class="w"> </span><span class="err">an</span><span class="w"> </span><span class="err">emit</span><span class="w"> </span><span class="err">direct,</span><span class="w"> </span><span class="err">indicate</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">task</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">send</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">tuple</span><span class="w"> </span><span class="err">to</span><span class="w">
- </span><span class="nt">"task"</span><span class="p">:</span><span class="w"> </span><span class="mi">9</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">All</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">values</span><span class="w"> </span><span class="err">in</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">tuple</span><span class="w">
- </span><span class="nt">"tuple"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"field1"</span><span class="p">,</span><span class="w"> </span><span class="mi">2</span><span class="p">,</span><span class="w"> </span><span class="mi">3</span><span class="p">]</span><span class="w">
-</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
+<div class="highlight"><pre><code class="language-" data-lang="">{
+ "command": "emit",
+ // The id for the tuple. Leave this out for an unreliable emit. The id can
+ // be a string or a number.
+ "id": "1231231",
+ // The id of the stream this tuple was emitted to. Leave this empty to emit to default stream.
+ "stream": "1",
+ // If doing an emit direct, indicate the task to send the tuple to
+ "task": 9,
+ // All the values in this tuple
+ "tuple": ["field1", 2, 3]
+}
+</code></pre></div>
<p>If not doing an emit direct, you will immediately receive the task ids to which the tuple was emitted on STDIN as a JSON array.</p>
<p>A "log" will log a message in the worker log. It looks like:</p>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="w">
- </span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"log"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">message</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">log</span><span class="w">
- </span><span class="nt">"msg"</span><span class="p">:</span><span class="w"> </span><span class="s2">"hello world!"</span><span class="w">
-</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
+<div class="highlight"><pre><code class="language-" data-lang="">{
+ "command": "log",
+ // the message to log
+ "msg": "hello world!"
+}
+</code></pre></div>
<ul>
<li>STDOUT: a "sync" command ends the sequence of emits and logs. It looks like:</li>
</ul>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"sync"</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
+<div class="highlight"><pre><code class="language-" data-lang="">{"command": "sync"}
+</code></pre></div>
<p>After you sync, ShellSpout will not read your output until it sends another next, ack, or fail command.</p>
<p>Note that, similarly to ISpout, all of the spouts in the worker will be locked up after a next, ack, or fail, until you sync. Also like ISpout, if you have no tuples to emit for a next, you should sleep for a small amount of time before syncing. ShellSpout will not automatically sleep for you.</p>
@@ -329,34 +329,34 @@ be a sequence of emits and logs.</li>
<ul>
<li>STDIN: A tuple! This is a JSON encoded structure like this:</li>
</ul>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">tuple's</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">-</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">is</span><span class="w"> </span><span class="err">a</span><span class="w"> </span><span class="err">string</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">support</span><span class="w"> </span><span class="err">languages</span><span class="w"> </span><span class="err">lacking</span><span class="w"> </span><span class="err">64-bit</span><span class="w"> </span><span class="err">precision</span><span class="w">
- </span><span class="nt">"id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"-6955786537413359385"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">of</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">component</span><span class="w"> </span><span class="err">that</span><span class="w"> </span><span class="err">created</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">tuple</span><span class="w">
- </span><span class="nt">"comp"</span><span class="p">:</span><span class="w"> </span><span class="s2">"1"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">of</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">stream</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">tuple</span><span class="w"> </span><span class="err">was</span><span class="w"> </span><span class="err">emitted</span><span class="w"> </span><span class="err">to</span><span class="w">
- </span><span class="nt">"stream"</span><span class="p">:</span><span class="w"> </span><span class="s2">"1"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">of</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">task</span><span class="w"> </span><span class="err">that</span><span class="w"> </span><span class="err">created</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">tuple</span><span class="w">
- </span><span class="nt">"task"</span><span class="p">:</span><span class="w"> </span><span class="mi">9</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">All</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">values</span><span class="w"> </span><span class="err">in</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">tuple</span><span class="w">
- </span><span class="nt">"tuple"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"snow white and the seven dwarfs"</span><span class="p">,</span><span class="w"> </span><span class="s2">"field2"</span><span class="p">,</span><span class="w"> </span><span class="mi">3</span><span class="p">]</span><span class="w">
-</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
+<div class="highlight"><pre><code class="language-" data-lang="">{
+ // The tuple's id - this is a string to support languages lacking 64-bit precision
+ "id": "-6955786537413359385",
+ // The id of the component that created this tuple
+ "comp": "1",
+ // The id of the stream this tuple was emitted to
+ "stream": "1",
+ // The id of the task that created this tuple
+ "task": 9,
+ // All the values in this tuple
+ "tuple": ["snow white and the seven dwarfs", "field2", 3]
+}
+</code></pre></div>
<ul>
<li>STDOUT: An ack, fail, emit, or log. Emits look like:</li>
</ul>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="w">
- </span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"emit"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">ids</span><span class="w"> </span><span class="err">of</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">tuples</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">output</span><span class="w"> </span><span class="err">tuples</span><span class="w"> </span><span class="err">should</span><span class="w"> </span><span class="err">be</span><span class="w"> </span><span class="err">anchored</span><span class="w"> </span><span class="err">to</span><span class="w">
- </span><span class="nt">"anchors"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"1231231"</span><span class="p">,</span><span class="w"> </span><span class="s2">"-234234234"</span><span class="p">],</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">of</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">stream</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">tuple</span><span class="w"> </span><span class="err">was</span><span class="w"> </span><span class="err">emitted</span><span class="w"> </span><span class="err">to.</span><span class="w"> </span><span class="err">Leave</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">empty</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">emit</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">default</span><span class="w"> </span><span class="err">stream.</span><sp
an class="w">
- </span><span class="nt">"stream"</span><span class="p">:</span><span class="w"> </span><span class="s2">"1"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">If</span><span class="w"> </span><span class="err">doing</span><span class="w"> </span><span class="err">an</span><span class="w"> </span><span class="err">emit</span><span class="w"> </span><span class="err">direct,</span><span class="w"> </span><span class="err">indicate</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">task</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">send</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">tuple</span><span class="w"> </span><span class="err">to</span><span class="w">
- </span><span class="nt">"task"</span><span class="p">:</span><span class="w"> </span><span class="mi">9</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">All</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">values</span><span class="w"> </span><span class="err">in</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">tuple</span><span class="w">
- </span><span class="nt">"tuple"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"field1"</span><span class="p">,</span><span class="w"> </span><span class="mi">2</span><span class="p">,</span><span class="w"> </span><span class="mi">3</span><span class="p">]</span><span class="w">
-</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
+<div class="highlight"><pre><code class="language-" data-lang="">{
+ "command": "emit",
+ // The ids of the tuples this output tuples should be anchored to
+ "anchors": ["1231231", "-234234234"],
+ // The id of the stream this tuple was emitted to. Leave this empty to emit to default stream.
+ "stream": "1",
+ // If doing an emit direct, indicate the task to send the tuple to
+ "task": 9,
+ // All the values in this tuple
+ "tuple": ["field1", 2, 3]
+}
+</code></pre></div>
<p>If not doing an emit direct, you will receive the task ids to which
the tuple was emitted on STDIN as a JSON array. Note that, due to the
asynchronous nature of the shell bolt protocol, when you read after
@@ -366,26 +366,26 @@ receive the task id lists in the same order as their corresponding
emits, however.</p>
<p>An ack looks like:</p>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="w">
- </span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"ack"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">of</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">tuple</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">ack</span><span class="w">
- </span><span class="nt">"id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"123123"</span><span class="w">
-</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
+<div class="highlight"><pre><code class="language-" data-lang="">{
+ "command": "ack",
+ // the id of the tuple to ack
+ "id": "123123"
+}
+</code></pre></div>
<p>A fail looks like:</p>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="w">
- </span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"fail"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">of</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">tuple</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">fail</span><span class="w">
- </span><span class="nt">"id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"123123"</span><span class="w">
-</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
+<div class="highlight"><pre><code class="language-" data-lang="">{
+ "command": "fail",
+ // the id of the tuple to fail
+ "id": "123123"
+}
+</code></pre></div>
<p>A "log" will log a message in the worker log. It looks like:</p>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="w">
- </span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"log"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">message</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">log</span><span class="w">
- </span><span class="nt">"msg"</span><span class="p">:</span><span class="w"> </span><span class="s2">"hello world!"</span><span class="w">
-</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
+<div class="highlight"><pre><code class="language-" data-lang="">{
+ "command": "log",
+ // the message to log
+ "msg": "hello world!"
+}
+</code></pre></div>
<ul>
<li>Note that, as of version 0.7.1, there is no longer any need for a
shell bolt to 'sync'.</li>
@@ -409,15 +409,15 @@ timeout during <code>next()</code>.</p>
<p>Shell bolts are asynchronous, so a ShellBolt will send heartbeat tuples to its
subprocess periodically. Heartbeat tuple looks like:</p>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="w">
- </span><span class="nt">"id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"-6955786537413359385"</span><span class="p">,</span><span class="w">
- </span><span class="nt">"comp"</span><span class="p">:</span><span class="w"> </span><span class="s2">"1"</span><span class="p">,</span><span class="w">
- </span><span class="nt">"stream"</span><span class="p">:</span><span class="w"> </span><span class="s2">"__heartbeat"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">shell</span><span class="w"> </span><span class="err">bolt's</span><span class="w"> </span><span class="err">system</span><span class="w"> </span><span class="err">task</span><span class="w"> </span><span class="err">id</span><span class="w">
- </span><span class="nt">"task"</span><span class="p">:</span><span class="w"> </span><span class="mi">-1</span><span class="p">,</span><span class="w">
- </span><span class="nt">"tuple"</span><span class="p">:</span><span class="w"> </span><span class="p">[]</span><span class="w">
-</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
+<div class="highlight"><pre><code class="language-" data-lang="">{
+ "id": "-6955786537413359385",
+ "comp": "1",
+ "stream": "__heartbeat",
+ // this shell bolt's system task id
+ "task": -1,
+ "tuple": []
+}
+</code></pre></div>
<p>When subprocess receives heartbeat tuple, it must send a <code>sync</code> command back to
ShellBolt.</p>
http://git-wip-us.apache.org/repos/asf/storm-site/blob/a7a19afb/content/releases/1.0.4/Pacemaker.html
----------------------------------------------------------------------
diff --git a/content/releases/1.0.4/Pacemaker.html b/content/releases/1.0.4/Pacemaker.html
index d6b6921..cb37689 100644
--- a/content/releases/1.0.4/Pacemaker.html
+++ b/content/releases/1.0.4/Pacemaker.html
@@ -82,18 +82,16 @@
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
+ <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
+
<li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
<li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
@@ -104,6 +102,8 @@
+
+
<li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
@@ -144,7 +144,7 @@
<li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
</ul>
</li>
- <li><a href="/2017/09/15/storm105-released.html" id="news">News</a></li>
+ <li><a href="/2018/02/14/storm106-released.html" id="news">News</a></li>
</ul>
</nav>
</div>
http://git-wip-us.apache.org/repos/asf/storm-site/blob/a7a19afb/content/releases/1.0.4/Powered-By.html
----------------------------------------------------------------------
diff --git a/content/releases/1.0.4/Powered-By.html b/content/releases/1.0.4/Powered-By.html
index 73bdca7..fc79fe5 100644
--- a/content/releases/1.0.4/Powered-By.html
+++ b/content/releases/1.0.4/Powered-By.html
@@ -82,18 +82,16 @@
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
+ <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
+
<li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
<li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
@@ -104,6 +102,8 @@
+
+
<li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
@@ -144,7 +144,7 @@
<li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
</ul>
</li>
- <li><a href="/2017/09/15/storm105-released.html" id="news">News</a></li>
+ <li><a href="/2018/02/14/storm106-released.html" id="news">News</a></li>
</ul>
</nav>
</div>
http://git-wip-us.apache.org/repos/asf/storm-site/blob/a7a19afb/content/releases/1.0.4/Project-ideas.html
----------------------------------------------------------------------
diff --git a/content/releases/1.0.4/Project-ideas.html b/content/releases/1.0.4/Project-ideas.html
index fade760..2dad069 100644
--- a/content/releases/1.0.4/Project-ideas.html
+++ b/content/releases/1.0.4/Project-ideas.html
@@ -82,18 +82,16 @@
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
+ <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
+
<li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
<li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
@@ -104,6 +102,8 @@
+
+
<li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
@@ -144,7 +144,7 @@
<li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
</ul>
</li>
- <li><a href="/2017/09/15/storm105-released.html" id="news">News</a></li>
+ <li><a href="/2018/02/14/storm106-released.html" id="news">News</a></li>
</ul>
</nav>
</div>
http://git-wip-us.apache.org/repos/asf/storm-site/blob/a7a19afb/content/releases/1.0.4/Rationale.html
----------------------------------------------------------------------
diff --git a/content/releases/1.0.4/Rationale.html b/content/releases/1.0.4/Rationale.html
index afea959..4e0bf37 100644
--- a/content/releases/1.0.4/Rationale.html
+++ b/content/releases/1.0.4/Rationale.html
@@ -82,18 +82,16 @@
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
+ <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
+
<li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
<li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
@@ -104,6 +102,8 @@
+
+
<li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
@@ -144,7 +144,7 @@
<li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
</ul>
</li>
- <li><a href="/2017/09/15/storm105-released.html" id="news">News</a></li>
+ <li><a href="/2018/02/14/storm106-released.html" id="news">News</a></li>
</ul>
</nav>
</div>
http://git-wip-us.apache.org/repos/asf/storm-site/blob/a7a19afb/content/releases/1.0.4/Resource_Aware_Scheduler_overview.html
----------------------------------------------------------------------
diff --git a/content/releases/1.0.4/Resource_Aware_Scheduler_overview.html b/content/releases/1.0.4/Resource_Aware_Scheduler_overview.html
index f38e99e..367ee53 100644
--- a/content/releases/1.0.4/Resource_Aware_Scheduler_overview.html
+++ b/content/releases/1.0.4/Resource_Aware_Scheduler_overview.html
@@ -82,18 +82,16 @@
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
+ <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
+
<li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
<li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
@@ -104,6 +102,8 @@
+
+
<li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
@@ -144,7 +144,7 @@
<li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
</ul>
</li>
- <li><a href="/2017/09/15/storm105-released.html" id="news">News</a></li>
+ <li><a href="/2018/02/14/storm106-released.html" id="news">News</a></li>
</ul>
</nav>
</div>
http://git-wip-us.apache.org/repos/asf/storm-site/blob/a7a19afb/content/releases/1.0.4/Running-topologies-on-a-production-cluster.html
----------------------------------------------------------------------
diff --git a/content/releases/1.0.4/Running-topologies-on-a-production-cluster.html b/content/releases/1.0.4/Running-topologies-on-a-production-cluster.html
index f6121c4..594c850 100644
--- a/content/releases/1.0.4/Running-topologies-on-a-production-cluster.html
+++ b/content/releases/1.0.4/Running-topologies-on-a-production-cluster.html
@@ -82,18 +82,16 @@
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
+ <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
+
<li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
<li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
@@ -104,6 +102,8 @@
+
+
<li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
@@ -144,7 +144,7 @@
<li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
</ul>
</li>
- <li><a href="/2017/09/15/storm105-released.html" id="news">News</a></li>
+ <li><a href="/2018/02/14/storm106-released.html" id="news">News</a></li>
</ul>
</nav>
</div>
http://git-wip-us.apache.org/repos/asf/storm-site/blob/a7a19afb/content/releases/1.0.4/SECURITY.html
----------------------------------------------------------------------
diff --git a/content/releases/1.0.4/SECURITY.html b/content/releases/1.0.4/SECURITY.html
index 35f2525..00c37a5 100644
--- a/content/releases/1.0.4/SECURITY.html
+++ b/content/releases/1.0.4/SECURITY.html
@@ -82,18 +82,16 @@
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
+ <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
+
<li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
<li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
@@ -104,6 +102,8 @@
+
+
<li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
@@ -144,7 +144,7 @@
<li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
</ul>
</li>
- <li><a href="/2017/09/15/storm105-released.html" id="news">News</a></li>
+ <li><a href="/2018/02/14/storm106-released.html" id="news">News</a></li>
</ul>
</nav>
</div>
@@ -281,7 +281,7 @@ curl -i --negotiate -u:anyUser -b ~/cookiejar.txt -c ~/cookiejar.txt <a href=
<ol>
<li>Firefox: Goto about:config and search for network.negotiate-auth.trusted-uris double-click to add value "<a href="http://storm-ui-hostname:8080">http://storm-ui-hostname:8080</a>"</li>
-<li>Google-chrome: start from command line with: google-chrome --auth-server-whitelist="<em>storm-ui-hostname" --auth-negotiate-delegate-whitelist="</em>storm-ui-hostname"<br></li>
+<li>Google-chrome: start from command line with: google-chrome --auth-server-whitelist="*storm-ui-hostname" --auth-negotiate-delegate-whitelist="*storm-ui-hostname"<br></li>
<li>IE: Configure trusted websites to include "storm-ui-hostname" and allow negotiation for that website</li>
</ol>
@@ -351,14 +351,14 @@ The supervisors and UI also need a principal to run as, but because they are out
The following is an example of how to setup kerberos principals, but the
details may vary depending on your KDC and OS.</p>
<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># Zookeeper (Will need one of these for each box in teh Zk ensamble)</span>
-sudo kadmin.local -q <span class="s1">'addprinc zookeeper/zk1.example.com@STORM.EXAMPLE.COM'</span>
-sudo kadmin.local -q <span class="s2">"ktadd -k /tmp/zk.keytab zookeeper/zk1.example.com@STORM.EXAMPLE.COM"</span>
+<span class="nb">sudo </span>kadmin.local <span class="nt">-q</span> <span class="s1">'addprinc zookeeper/zk1.example.com@STORM.EXAMPLE.COM'</span>
+<span class="nb">sudo </span>kadmin.local <span class="nt">-q</span> <span class="s2">"ktadd -k /tmp/zk.keytab zookeeper/zk1.example.com@STORM.EXAMPLE.COM"</span>
<span class="c"># Nimbus and DRPC</span>
-sudo kadmin.local -q <span class="s1">'addprinc storm/storm.example.com@STORM.EXAMPLE.COM'</span>
-sudo kadmin.local -q <span class="s2">"ktadd -k /tmp/storm.keytab storm/storm.example.com@STORM.EXAMPLE.COM"</span>
+<span class="nb">sudo </span>kadmin.local <span class="nt">-q</span> <span class="s1">'addprinc storm/storm.example.com@STORM.EXAMPLE.COM'</span>
+<span class="nb">sudo </span>kadmin.local <span class="nt">-q</span> <span class="s2">"ktadd -k /tmp/storm.keytab storm/storm.example.com@STORM.EXAMPLE.COM"</span>
<span class="c"># All UI logviewer and Supervisors</span>
-sudo kadmin.local -q <span class="s1">'addprinc storm@STORM.EXAMPLE.COM'</span>
-sudo kadmin.local -q <span class="s2">"ktadd -k /tmp/storm.keytab storm@STORM.EXAMPLE.COM"</span>
+<span class="nb">sudo </span>kadmin.local <span class="nt">-q</span> <span class="s1">'addprinc storm@STORM.EXAMPLE.COM'</span>
+<span class="nb">sudo </span>kadmin.local <span class="nt">-q</span> <span class="s2">"ktadd -k /tmp/storm.keytab storm@STORM.EXAMPLE.COM"</span>
</code></pre></div>
<p>be sure to distribute the keytab(s) to the appropriate boxes and set the FS permissions so that only the headless user running ZK, or storm has access to them.</p>
@@ -542,7 +542,7 @@ Be aware that many of the features of this scheduler rely on storm authenticatio
<h3 id="run-worker-processes-as-user-who-submitted-the-topology">Run worker processes as user who submitted the topology</h3>
<p>By default storm runs workers as the user that is running the supervisor. This is not ideal for security. To make storm run the topologies as the user that launched them set.</p>
-<div class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="s">supervisor.run.worker.as.user</span><span class="pi">:</span> <span class="s">true</span>
+<div class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="s">supervisor.run.worker.as.user</span><span class="pi">:</span> <span class="no">true</span>
</code></pre></div>
<p>There are several files that go along with this that are needed to be configured properly to make storm secure.</p>
@@ -565,15 +565,15 @@ to get a nimbus client as some other user and perform any nimbus action(i.e. kil
The <code>ImpersonationAuthorizer</code> uses <code>nimbus.impersonation.acl</code> as the acl to authorize users. Following is a sample nimbus config for supporting impersonation:</p>
<div class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="s">nimbus.impersonation.authorizer</span><span class="pi">:</span> <span class="s">org.apache.storm.security.auth.authorizer.ImpersonationAuthorizer</span>
<span class="s">nimbus.impersonation.acl</span><span class="pi">:</span>
- <span class="s">impersonating_user1</span><span class="pi">:</span>
- <span class="s">hosts</span><span class="pi">:</span>
+ <span class="na">impersonating_user1</span><span class="pi">:</span>
+ <span class="na">hosts</span><span class="pi">:</span>
<span class="pi">[</span><span class="nv">comma separated list of hosts from which impersonating_user1 is allowed to impersonate other users</span><span class="pi">]</span>
- <span class="s">groups</span><span class="pi">:</span>
+ <span class="na">groups</span><span class="pi">:</span>
<span class="pi">[</span><span class="nv">comma separated list of groups whose users impersonating_user1 is allowed to impersonate</span><span class="pi">]</span>
- <span class="s">impersonating_user2</span><span class="pi">:</span>
- <span class="s">hosts</span><span class="pi">:</span>
+ <span class="na">impersonating_user2</span><span class="pi">:</span>
+ <span class="na">hosts</span><span class="pi">:</span>
<span class="pi">[</span><span class="nv">comma separated list of hosts from which impersonating_user2 is allowed to impersonate other users</span><span class="pi">]</span>
- <span class="s">groups</span><span class="pi">:</span>
+ <span class="na">groups</span><span class="pi">:</span>
<span class="pi">[</span><span class="nv">comma separated list of groups whose users impersonating_user2 is allowed to impersonate</span><span class="pi">]</span>
</code></pre></div>
<p>To support the oozie use case following config can be supplied: