You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@storm.apache.org by pt...@apache.org on 2017/09/15 17:00:23 UTC
[44/51] [partial] storm-site git commit: test git pubsub
http://git-wip-us.apache.org/repos/asf/storm-site/blob/3730cb62/content/releases/0.10.0/Metrics.html
----------------------------------------------------------------------
diff --git a/content/releases/0.10.0/Metrics.html b/content/releases/0.10.0/Metrics.html
deleted file mode 100644
index e720428..0000000
--- a/content/releases/0.10.0/Metrics.html
+++ /dev/null
@@ -1,353 +0,0 @@
-<!DOCTYPE html>
-<html>
- <head>
- <meta charset="utf-8">
- <meta http-equiv="X-UA-Compatible" content="IE=edge">
- <meta name="viewport" content="width=device-width, initial-scale=1">
-
- <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
- <link rel="icon" href="/favicon.ico" type="image/x-icon">
-
- <title>Storm Metrics</title>
-
- <!-- Bootstrap core CSS -->
- <link href="/assets/css/bootstrap.min.css" rel="stylesheet">
- <!-- Bootstrap theme -->
- <link href="/assets/css/bootstrap-theme.min.css" rel="stylesheet">
-
- <!-- Custom styles for this template -->
- <link rel="stylesheet" href="http://fortawesome.github.io/Font-Awesome/assets/font-awesome/css/font-awesome.css">
- <link href="/css/style.css" rel="stylesheet">
- <link href="/assets/css/owl.theme.css" rel="stylesheet">
- <link href="/assets/css/owl.carousel.css" rel="stylesheet">
- <script type="text/javascript" src="/assets/js/jquery.min.js"></script>
- <script type="text/javascript" src="/assets/js/bootstrap.min.js"></script>
- <script type="text/javascript" src="/assets/js/owl.carousel.min.js"></script>
- <script type="text/javascript" src="/assets/js/storm.js"></script>
- <!-- Just for debugging purposes. Don't actually copy these 2 lines! -->
- <!--[if lt IE 9]><script src="../../assets/js/ie8-responsive-file-warning.js"></script><![endif]-->
-
- <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
- <!--[if lt IE 9]>
- <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
- <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
- <![endif]-->
- </head>
-
-
- <body>
- <header>
- <div class="container-fluid">
- <div class="row">
- <div class="col-md-5">
- <a href="/index.html"><img src="/images/logo.png" class="logo" /></a>
- </div>
- <div class="col-md-5">
-
- <h1>Version: 0.10.0</h1>
-
- </div>
- <div class="col-md-2">
- <a href="/downloads.html" class="btn-std btn-block btn-download">Download</a>
- </div>
- </div>
- </div>
-</header>
-<!--Header End-->
-<!--Navigation Begin-->
-<div class="navbar" role="banner">
- <div class="container-fluid">
- <div class="navbar-header">
- <button class="navbar-toggle" type="button" data-toggle="collapse" data-target=".bs-navbar-collapse">
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- </button>
- </div>
- <nav class="collapse navbar-collapse bs-navbar-collapse" role="navigation">
- <ul class="nav navbar-nav">
- <li><a href="/index.html" id="home">Home</a></li>
- <li><a href="/getting-help.html" id="getting-help">Getting Help</a></li>
- <li><a href="/about/integrates.html" id="project-info">Project Information</a></li>
- <li class="dropdown">
- <a href="#" class="dropdown-toggle" data-toggle="dropdown" id="documentation">Documentation <b class="caret"></b></a>
- <ul class="dropdown-menu">
-
-
- <li><a href="/releases/2.0.0-SNAPSHOT/index.html">2.0.0-SNAPSHOT</a></li>
-
-
-
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
-
-
-
-
- <li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
-
-
-
- <li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
-
-
-
- <li><a href="/releases/1.0.2/index.html">1.0.2</a></li>
-
-
-
- <li><a href="/releases/1.0.1/index.html">1.0.1</a></li>
-
-
-
- <li><a href="/releases/1.0.0/index.html">1.0.0</a></li>
-
-
-
- <li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
-
-
-
- <li><a href="/releases/0.10.1/index.html">0.10.1</a></li>
-
-
-
-
-
- <li><a href="/releases/0.10.0/index.html">0.10.0</a></li>
-
-
-
- <li><a href="/releases/0.9.7/index.html">0.9.7</a></li>
-
-
-
- <li><a href="/releases/0.9.6/index.html">0.9.6</a></li>
-
-
-
-
-
-
-
-
-
-
-
-
- </ul>
- </li>
- <li><a href="/talksAndVideos.html">Talks and Slideshows</a></li>
- <li class="dropdown">
- <a href="#" class="dropdown-toggle" data-toggle="dropdown" id="contribute">Community <b class="caret"></b></a>
- <ul class="dropdown-menu">
- <li><a href="/contribute/Contributing-to-Storm.html">Contributing</a></li>
- <li><a href="/contribute/People.html">People</a></li>
- <li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
- </ul>
- </li>
- <li><a href="/2017/07/28/storm104-released.html" id="news">News</a></li>
- </ul>
- </nav>
- </div>
-</div>
-
-
-
- <div class="container-fluid">
- <h1 class="page-title">Storm Metrics</h1>
- <div class="row">
- <div class="col-md-12">
- <!-- Documentation -->
-
-<p class="post-meta"></p>
-
-<p>Storm exposes a metrics interface to report summary statistics across the full topology.
-It's used internally to track the numbers you see in the Nimbus UI console: counts of executes and acks; average process latency per bolt; worker heap usage; and so forth.</p>
-
-<h3 id="metric-types">Metric Types</h3>
-
-<p>Metrics have to implement <a href="http://github.com/apache/storm/blob/v0.10.0/storm-core/src/jvm/org/apache/storm/metric/api/IMetric.java"><code>IMetric</code></a> which contains just one method, <code>getValueAndReset</code> -- do any remaining work to find the summary value, and reset back to an initial state. For example, the MeanReducer divides the running total by its running count to find the mean, then initializes both values back to zero.</p>
-
-<p>Storm gives you these metric types:</p>
-
-<ul>
-<li><a href="http://github.com/apache/storm/blob/v0.10.0/storm-core/src/jvm/org/apache/storm/metric/api/AssignableMetric.java">AssignableMetric</a> -- set the metric to the explicit value you supply. Useful if it's an external value or in the case that you are already calculating the summary statistic yourself.</li>
-<li><a href="http://github.com/apache/storm/blob/v0.10.0/storm-core/src/jvm/org/apache/storm/metric/api/CombinedMetric.java">CombinedMetric</a> -- generic interface for metrics that can be updated associatively. </li>
-<li><a href="http://github.com/apache/storm/blob/v0.10.0/storm-core/src/jvm/org/apache/storm/metric/api/CountMetric.java">CountMetric</a> -- a running total of the supplied values. Call <code>incr()</code> to increment by one, <code>incrBy(n)</code> to add/subtract the given number.
-
-<ul>
-<li><a href="http://github.com/apache/storm/blob/v0.10.0/storm-core/src/jvm/org/apache/storm/metric/api/MultiCountMetric.java">MultiCountMetric</a> -- a hashmap of count metrics.</li>
-</ul></li>
-<li><a href="http://github.com/apache/storm/blob/v0.10.0/storm-core/src/jvm/org/apache/storm/metric/api/ReducedMetric.java">ReducedMetric</a>
-
-<ul>
-<li><a href="http://github.com/apache/storm/blob/v0.10.0/storm-core/src/jvm/org/apache/storm/metric/api/MeanReducer.java">MeanReducer</a> -- track a running average of values given to its <code>reduce()</code> method. (It accepts <code>Double</code>, <code>Integer</code> or <code>Long</code> values, and maintains the internal average as a <code>Double</code>.) Despite his reputation, the MeanReducer is actually a pretty nice guy in person.</li>
-<li><a href="http://github.com/apache/storm/blob/v0.10.0/storm-core/src/jvm/org/apache/storm/metric/api/MultiReducedMetric.java">MultiReducedMetric</a> -- a hashmap of reduced metrics.</li>
-</ul></li>
-</ul>
-
-<h3 id="metrics-consumer">Metrics Consumer</h3>
-
-<p>You can listen and handle the topology metrics via registering Metrics Consumer to your topology. </p>
-
-<p>To register metrics consumer to your topology, add to your topology's configuration like:</p>
-<div class="highlight"><pre><code class="language-java" data-lang="java"><span class="n">conf</span><span class="o">.</span><span class="na">registerMetricsConsumer</span><span class="o">(</span><span class="n">org</span><span class="o">.</span><span class="na">apache</span><span class="o">.</span><span class="na">storm</span><span class="o">.</span><span class="na">metric</span><span class="o">.</span><span class="na">LoggingMetricsConsumer</span><span class="o">.</span><span class="na">class</span><span class="o">,</span> <span class="mi">1</span><span class="o">);</span>
-</code></pre></div>
-<p>You can refer <a href="javadocs/org/apache/storm/Config.html#registerMetricsConsumer-java.lang.Class-">Config#registerMetricsConsumer</a> and overloaded methods from javadoc.</p>
-
-<p>Otherwise edit the storm.yaml config file:</p>
-<div class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="s">topology.metrics.consumer.register</span><span class="pi">:</span>
- <span class="pi">-</span> <span class="s">class</span><span class="pi">:</span> <span class="s2">"</span><span class="s">org.apache.storm.metric.LoggingMetricsConsumer"</span>
- <span class="s">parallelism.hint</span><span class="pi">:</span> <span class="s">1</span>
- <span class="pi">-</span> <span class="s">class</span><span class="pi">:</span> <span class="s2">"</span><span class="s">org.apache.storm.metric.HttpForwardingMetricsConsumer"</span>
- <span class="s">parallelism.hint</span><span class="pi">:</span> <span class="s">1</span>
- <span class="s">argument</span><span class="pi">:</span> <span class="s2">"</span><span class="s">http://example.com:8080/metrics/my-topology/"</span>
-</code></pre></div>
-<p>Storm appends MetricsConsumerBolt to your topology per each registered metrics consumer internally, and each MetricsConsumerBolt subscribes to receive metrics from all tasks. The parallelism for that Bolt is set to <code>parallelism.hint</code> and <code>component id</code> for that Bolt is set to <code>__metrics_<metrics consumer class name></code>. If you register same class name more than once, postfix <code>#<sequence number></code> is appended to component id.</p>
-
-<p>Storm provides some built-in metrics consumers for you to try out to see which metrics are provided in your topology.</p>
-
-<ul>
-<li><a href="http://github.com/apache/storm/blob/v0.10.0/storm-core/src/jvm/org/apache/storm/metric/LoggingMetricsConsumer.java"><code>LoggingMetricsConsumer</code></a> -- listens for all metrics and dumps them to log file with TSV (Tab Separated Values).</li>
-<li><a href="http://github.com/apache/storm/blob/v0.10.0/storm-core/src/jvm/org/apache/storm/metric/HttpForwardingMetricsConsumer.java"><code>HttpForwardingMetricsConsumer</code></a> -- listens for all metrics and POSTs them serialized to a configured URL via HTTP. Storm also provides <a href="http://github.com/apache/storm/blob/v0.10.0/storm-core/src/jvm/org/apache/storm/metric/HttpForwardingMetricsServer.java"><code>HttpForwardingMetricsServer</code></a> as abstract class so you can extend this class and run as a HTTP server, and handle metrics sent by HttpForwardingMetricsConsumer.</li>
-</ul>
-
-<p>Also, Storm exposes the interface <a href="http://github.com/apache/storm/blob/v0.10.0/storm-core/src/jvm/org/apache/storm/metric/api/IMetricsConsumer.java"><code>IMetricsConsumer</code></a> for implementing Metrics Consumer so you can create custom metrics consumers and attach to their topologies, or use other great implementation of Metrics Consumers provided by Storm community. Some of examples are <a href="https://github.com/verisign/storm-graphite">versign/storm-graphite</a>, and <a href="https://github.com/endgameinc/storm-metrics-statsd">storm-metrics-statsd</a>.</p>
-
-<p>When you implement your own metrics consumer, <code>argument</code> is passed to Object when <a href="javadocs/org/apache/storm/metric/api/IMetricsConsumer.html#prepare-java.util.Map-java.lang.Object-org.apache.storm.task.TopologyContext-org.apache.storm.task.IErrorReporter-">IMetricsConsumer#prepare</a> is called, so you need to infer the Java type of configured value on yaml, and do explicit type casting.</p>
-
-<p>Please keep in mind that MetricsConsumerBolt is just a kind of Bolt, so whole throughput of the topology will go down when registered metrics consumers cannot keep up handling incoming metrics, so you may want to take care of those Bolts like normal Bolts. One of idea to avoid this is making your implementation of Metrics Consumer as <code>non-blocking</code> fashion.</p>
-
-<h3 id="build-your-own-metric-task-level">Build your own metric (task level)</h3>
-
-<p>You can measure your own metric by registering <code>IMetric</code> to Metric Registry. </p>
-
-<p>Suppose we would like to measure execution count of Bolt#execute. Let's start with defining metric instance. CountMetric seems to fit our use case.</p>
-<div class="highlight"><pre><code class="language-java" data-lang="java"><span class="kd">private</span> <span class="kd">transient</span> <span class="n">CountMetric</span> <span class="n">countMetric</span><span class="o">;</span>
-</code></pre></div>
-<p>Notice we define it as transient. IMertic is not Serializable so we defined as transient to avoid any serialization issues.</p>
-
-<p>Next, let's initialize and register the metric instance.</p>
-<div class="highlight"><pre><code class="language-java" data-lang="java"><span class="nd">@Override</span>
-<span class="kd">public</span> <span class="kt">void</span> <span class="nf">prepare</span><span class="p">(</span><span class="n">Map</span> <span class="n">conf</span><span class="o">,</span> <span class="n">TopologyContext</span> <span class="n">context</span><span class="o">,</span> <span class="n">OutputCollector</span> <span class="n">collector</span><span class="o">)</span> <span class="o">{</span>
- <span class="c1">// other intialization here.</span>
- <span class="n">countMetric</span> <span class="o">=</span> <span class="k">new</span> <span class="n">CountMetric</span><span class="o">();</span>
- <span class="n">context</span><span class="o">.</span><span class="na">registerMetric</span><span class="o">(</span><span class="s">"execute_count"</span><span class="o">,</span> <span class="n">countMetric</span><span class="o">,</span> <span class="mi">60</span><span class="o">);</span>
-<span class="o">}</span>
-</code></pre></div>
-<p>The meaning of first and second parameters are straightforward, metric name and instance of IMetric. Third parameter of <a href="javadocs/org/apache/storm/task/TopologyContext.html#registerMetric-java.lang.String-T-int-">TopologyContext#registerMetric</a> is the period (seconds) to publish and reset the metric.</p>
-
-<p>Last, let's increment the value when Bolt.execute() is executed.</p>
-<div class="highlight"><pre><code class="language-java" data-lang="java"><span class="kd">public</span> <span class="kt">void</span> <span class="nf">execute</span><span class="p">(</span><span class="n">Tuple</span> <span class="n">input</span><span class="o">)</span> <span class="o">{</span>
- <span class="n">countMetric</span><span class="o">.</span><span class="na">incr</span><span class="o">();</span>
- <span class="c1">// handle tuple here. </span>
-<span class="o">}</span>
-</code></pre></div>
-<p>Note that sample rate for topology metrics is not applied to custom metrics since we're calling incr() ourselves.</p>
-
-<p>Done! <code>countMetric.getValueAndReset()</code> is called every 60 seconds as we registered as period, and pair of ("execute_count", value) will be pushed to MetricsConsumer.</p>
-
-<h3 id="build-your-own-metrics-worker-level">Build your own metrics (worker level)</h3>
-
-<p>You can register your own worker level metrics by adding them to <code>Config.WORKER_METRICS</code> for all workers in cluster, or <code>Config.TOPOLOGY_WORKER_METRICS</code> for all workers in specific topology.</p>
-
-<p>For example, we can add <code>worker.metrics</code> to storm.yaml in cluster,</p>
-<div class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="s">worker.metrics</span><span class="pi">:</span>
- <span class="s">metricA</span><span class="pi">:</span> <span class="s2">"</span><span class="s">aaa.bbb.ccc.ddd.MetricA"</span>
- <span class="s">metricB</span><span class="pi">:</span> <span class="s2">"</span><span class="s">aaa.bbb.ccc.ddd.MetricB"</span>
- <span class="s">...</span>
-</code></pre></div>
-<p>or put <code>Map<String, String></code> (metric name, metric class name) with key <code>Config.TOPOLOGY_WORKER_METRICS</code> to config map.</p>
-
-<p>There're some restrictions for worker level metric instances: </p>
-
-<p>A) Metrics for worker level should be kind of gauge since it is initialized and registered from SystemBolt and not exposed to user tasks.</p>
-
-<p>B) Metrics will be initialized with default constructor, and no injection for configuration or object will be performed.</p>
-
-<p>C) Bucket size (seconds) for metrics is fixed to <code>Config.TOPOLOGY_BUILTIN_METRICS_BUCKET_SIZE_SECS</code>.</p>
-
-<h3 id="builtin-metrics">Builtin Metrics</h3>
-
-<p>The <a href="http://github.com/apache/storm/blob/v0.10.0/storm-core/src/clj/org/apache/storm/daemon/builtin_metrics.clj">builtin metrics</a> instrument Storm itself.</p>
-
-<p><a href="http://github.com/apache/storm/blob/v0.10.0/storm-core/src/clj/org/apache/storm/daemon/builtin_metrics.clj">builtin_metrics.clj</a> sets up data structures for the built-in metrics, and facade methods that the other framework components can use to update them. The metrics themselves are calculated in the calling code -- see for example <a href="http://github.com/apache/storm/blob/v0.10.0/storm-core/src/clj/org/apache/storm/daemon/executor.clj#358"><code>ack-spout-msg</code></a> in <code>clj/b/s/daemon/daemon/executor.clj</code></p>
-
-
-
- </div>
- </div>
- </div>
-<footer>
- <div class="container-fluid">
- <div class="row">
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>Meetups</h5>
- <ul class="latest-news">
-
- <li><a href="http://www.meetup.com/Apache-Storm-Apache-Kafka/">Apache Storm & Apache Kafka</a> <span class="small">(Sunnyvale, CA)</span></li>
-
- <li><a href="http://www.meetup.com/Apache-Storm-Kafka-Users/">Apache Storm & Kafka Users</a> <span class="small">(Seattle, WA)</span></li>
-
- <li><a href="http://www.meetup.com/New-York-City-Storm-User-Group/">NYC Storm User Group</a> <span class="small">(New York, NY)</span></li>
-
- <li><a href="http://www.meetup.com/Bay-Area-Stream-Processing">Bay Area Stream Processing</a> <span class="small">(Emeryville, CA)</span></li>
-
- <li><a href="http://www.meetup.com/Boston-Storm-Users/">Boston Realtime Data</a> <span class="small">(Boston, MA)</span></li>
-
- <li><a href="http://www.meetup.com/storm-london">London Storm User Group</a> <span class="small">(London, UK)</span></li>
-
- <!-- <li><a href="http://www.meetup.com/Apache-Storm-Kafka-Users/">Seatle, WA</a> <span class="small">(27 Jun 2015)</span></li> -->
- </ul>
- </div>
- </div>
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>About Storm</h5>
- <p>Storm integrates with any queueing system and any database system. Storm's spout abstraction makes it easy to integrate a new queuing system. Likewise, integrating Storm with database systems is easy.</p>
- </div>
- </div>
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>First Look</h5>
- <ul class="footer-list">
- <li><a href="/releases/current/Rationale.html">Rationale</a></li>
- <li><a href="/releases/current/Tutorial.html">Tutorial</a></li>
- <li><a href="/releases/current/Setting-up-development-environment.html">Setting up development environment</a></li>
- <li><a href="/releases/current/Creating-a-new-Storm-project.html">Creating a new Storm project</a></li>
- </ul>
- </div>
- </div>
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>Documentation</h5>
- <ul class="footer-list">
- <li><a href="/releases/current/index.html">Index</a></li>
- <li><a href="/releases/current/javadocs/index.html">Javadoc</a></li>
- <li><a href="/releases/current/FAQ.html">FAQ</a></li>
- </ul>
- </div>
- </div>
- </div>
- <hr/>
- <div class="row">
- <div class="col-md-12">
- <p align="center">Copyright © 2015 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved.
- <br>Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation.
- <br>All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
- </div>
- </div>
- </div>
-</footer>
-<!--Footer End-->
-<!-- Scroll to top -->
-<span class="totop"><a href="#"><i class="fa fa-angle-up"></i></a></span>
-
-</body>
-
-</html>
-
http://git-wip-us.apache.org/repos/asf/storm-site/blob/3730cb62/content/releases/0.10.0/Multilang-protocol.html
----------------------------------------------------------------------
diff --git a/content/releases/0.10.0/Multilang-protocol.html b/content/releases/0.10.0/Multilang-protocol.html
deleted file mode 100644
index 4c37c7c..0000000
--- a/content/releases/0.10.0/Multilang-protocol.html
+++ /dev/null
@@ -1,502 +0,0 @@
-<!DOCTYPE html>
-<html>
- <head>
- <meta charset="utf-8">
- <meta http-equiv="X-UA-Compatible" content="IE=edge">
- <meta name="viewport" content="width=device-width, initial-scale=1">
-
- <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
- <link rel="icon" href="/favicon.ico" type="image/x-icon">
-
- <title>Multi-Lang Protocol</title>
-
- <!-- Bootstrap core CSS -->
- <link href="/assets/css/bootstrap.min.css" rel="stylesheet">
- <!-- Bootstrap theme -->
- <link href="/assets/css/bootstrap-theme.min.css" rel="stylesheet">
-
- <!-- Custom styles for this template -->
- <link rel="stylesheet" href="http://fortawesome.github.io/Font-Awesome/assets/font-awesome/css/font-awesome.css">
- <link href="/css/style.css" rel="stylesheet">
- <link href="/assets/css/owl.theme.css" rel="stylesheet">
- <link href="/assets/css/owl.carousel.css" rel="stylesheet">
- <script type="text/javascript" src="/assets/js/jquery.min.js"></script>
- <script type="text/javascript" src="/assets/js/bootstrap.min.js"></script>
- <script type="text/javascript" src="/assets/js/owl.carousel.min.js"></script>
- <script type="text/javascript" src="/assets/js/storm.js"></script>
- <!-- Just for debugging purposes. Don't actually copy these 2 lines! -->
- <!--[if lt IE 9]><script src="../../assets/js/ie8-responsive-file-warning.js"></script><![endif]-->
-
- <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
- <!--[if lt IE 9]>
- <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
- <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
- <![endif]-->
- </head>
-
-
- <body>
- <header>
- <div class="container-fluid">
- <div class="row">
- <div class="col-md-5">
- <a href="/index.html"><img src="/images/logo.png" class="logo" /></a>
- </div>
- <div class="col-md-5">
-
- <h1>Version: 0.10.0</h1>
-
- </div>
- <div class="col-md-2">
- <a href="/downloads.html" class="btn-std btn-block btn-download">Download</a>
- </div>
- </div>
- </div>
-</header>
-<!--Header End-->
-<!--Navigation Begin-->
-<div class="navbar" role="banner">
- <div class="container-fluid">
- <div class="navbar-header">
- <button class="navbar-toggle" type="button" data-toggle="collapse" data-target=".bs-navbar-collapse">
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- </button>
- </div>
- <nav class="collapse navbar-collapse bs-navbar-collapse" role="navigation">
- <ul class="nav navbar-nav">
- <li><a href="/index.html" id="home">Home</a></li>
- <li><a href="/getting-help.html" id="getting-help">Getting Help</a></li>
- <li><a href="/about/integrates.html" id="project-info">Project Information</a></li>
- <li class="dropdown">
- <a href="#" class="dropdown-toggle" data-toggle="dropdown" id="documentation">Documentation <b class="caret"></b></a>
- <ul class="dropdown-menu">
-
-
- <li><a href="/releases/2.0.0-SNAPSHOT/index.html">2.0.0-SNAPSHOT</a></li>
-
-
-
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
-
-
-
-
- <li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
-
-
-
- <li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
-
-
-
- <li><a href="/releases/1.0.2/index.html">1.0.2</a></li>
-
-
-
- <li><a href="/releases/1.0.1/index.html">1.0.1</a></li>
-
-
-
- <li><a href="/releases/1.0.0/index.html">1.0.0</a></li>
-
-
-
- <li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
-
-
-
- <li><a href="/releases/0.10.1/index.html">0.10.1</a></li>
-
-
-
-
-
- <li><a href="/releases/0.10.0/index.html">0.10.0</a></li>
-
-
-
- <li><a href="/releases/0.9.7/index.html">0.9.7</a></li>
-
-
-
- <li><a href="/releases/0.9.6/index.html">0.9.6</a></li>
-
-
-
-
-
-
-
-
-
-
-
-
- </ul>
- </li>
- <li><a href="/talksAndVideos.html">Talks and Slideshows</a></li>
- <li class="dropdown">
- <a href="#" class="dropdown-toggle" data-toggle="dropdown" id="contribute">Community <b class="caret"></b></a>
- <ul class="dropdown-menu">
- <li><a href="/contribute/Contributing-to-Storm.html">Contributing</a></li>
- <li><a href="/contribute/People.html">People</a></li>
- <li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
- </ul>
- </li>
- <li><a href="/2017/07/28/storm104-released.html" id="news">News</a></li>
- </ul>
- </nav>
- </div>
-</div>
-
-
-
- <div class="container-fluid">
- <h1 class="page-title">Multi-Lang Protocol</h1>
- <div class="row">
- <div class="col-md-12">
- <!-- Documentation -->
-
-<p class="post-meta"></p>
-
-<p>This page explains the multilang protocol as of Storm 0.7.1. Versions prior to 0.7.1 used a somewhat different protocol, documented [here](Storm-multi-language-protocol-(versions-0.7.0-and-below).html).</p>
-
-<h1 id="storm-multi-language-protocol">Storm Multi-Language Protocol</h1>
-
-<h2 id="shell-components">Shell Components</h2>
-
-<p>Support for multiple languages is implemented via the ShellBolt,
-ShellSpout, and ShellProcess classes. These classes implement the
-IBolt and ISpout interfaces and the protocol for executing a script or
-program via the shell using Java's ProcessBuilder class.</p>
-
-<h2 id="output-fields">Output fields</h2>
-
-<p>Output fields are part of the Thrift definition of the topology. This means that when you multilang in Java, you need to create a bolt that extends ShellBolt, implements IRichBolt, and declare the fields in <code>declareOutputFields</code> (similarly for ShellSpout).</p>
-
-<p>You can learn more about this on <a href="Concepts.html">Concepts</a></p>
-
-<h2 id="protocol-preamble">Protocol Preamble</h2>
-
-<p>A simple protocol is implemented via the STDIN and STDOUT of the
-executed script or program. All data exchanged with the process is
-encoded in JSON, making support possible for pretty much any language.</p>
-
-<h1 id="packaging-your-stuff">Packaging Your Stuff</h1>
-
-<p>To run a shell component on a cluster, the scripts that are shelled
-out to must be in the <code>resources/</code> directory within the jar submitted
-to the master.</p>
-
-<p>However, during development or testing on a local machine, the resources
-directory just needs to be on the classpath.</p>
-
-<h2 id="the-protocol">The Protocol</h2>
-
-<p>Notes:</p>
-
-<ul>
-<li>Both ends of this protocol use a line-reading mechanism, so be sure to
-trim off newlines from the input and to append them to your output.</li>
-<li>All JSON inputs and outputs are terminated by a single line containing "end". Note that this delimiter is not itself JSON encoded.</li>
-<li>The bullet points below are written from the perspective of the script writer's
-STDIN and STDOUT.</li>
-</ul>
-
-<h3 id="initial-handshake">Initial Handshake</h3>
-
-<p>The initial handshake is the same for both types of shell components:</p>
-
-<ul>
-<li>STDIN: Setup info. This is a JSON object with the Storm configuration, a PID directory, and a topology context, like this:</li>
-</ul>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="w">
- </span><span class="nt">"conf"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"topology.message.timeout.secs"</span><span class="p">:</span><span class="w"> </span><span class="mi">3</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">etc</span><span class="w">
- </span><span class="err">}</span><span class="p">,</span><span class="w">
- </span><span class="nt">"pidDir"</span><span class="p">:</span><span class="w"> </span><span class="s2">"..."</span><span class="p">,</span><span class="w">
- </span><span class="nt">"context"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"task->component"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"1"</span><span class="p">:</span><span class="w"> </span><span class="s2">"example-spout"</span><span class="p">,</span><span class="w">
- </span><span class="nt">"2"</span><span class="p">:</span><span class="w"> </span><span class="s2">"__acker"</span><span class="p">,</span><span class="w">
- </span><span class="nt">"3"</span><span class="p">:</span><span class="w"> </span><span class="s2">"example-bolt1"</span><span class="p">,</span><span class="w">
- </span><span class="nt">"4"</span><span class="p">:</span><span class="w"> </span><span class="s2">"example-bolt2"</span><span class="w">
- </span><span class="p">},</span><span class="w">
- </span><span class="nt">"taskid"</span><span class="p">:</span><span class="w"> </span><span class="mi">3</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">Everything</span><span class="w"> </span><span class="err">below</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">line</span><span class="w"> </span><span class="err">is</span><span class="w"> </span><span class="err">only</span><span class="w"> </span><span class="err">available</span><span class="w"> </span><span class="err">in</span><span class="w"> </span><span class="err">Storm</span><span class="w"> </span><span class="err">0.10.0+</span><span class="w">
- </span><span class="nt">"componentid"</span><span class="p">:</span><span class="w"> </span><span class="s2">"example-bolt"</span><span class="w">
- </span><span class="s2">"stream->target->grouping"</span><span class="err">:</span><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"default"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"example-bolt2"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"SHUFFLE"</span><span class="p">}}},</span><span class="w">
- </span><span class="nt">"streams"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"default"</span><span class="p">],</span><span class="w">
- </span><span class="nt">"stream->outputfields"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="nt">"default"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"word"</span><span class="p">]},</span><span class="w">
- </span><span class="nt">"source->stream->grouping"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"example-spout"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"default"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"FIELDS"</span><span class="p">,</span><span class="w">
- </span><span class="nt">"fields"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"word"</span><span class="p">]</span><span class="w">
- </span><span class="p">}</span><span class="w">
- </span><span class="p">}</span><span class="w">
- </span><span class="p">}</span><span class="w">
- </span><span class="s2">"source->stream->fields"</span><span class="err">:</span><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"example-spout"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
- </span><span class="nt">"default"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"word"</span><span class="p">]</span><span class="w">
- </span><span class="p">}</span><span class="w">
- </span><span class="p">}</span><span class="w">
- </span><span class="p">}</span><span class="w">
-</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
-<p>Your script should create an empty file named with its PID in this directory. e.g.
-the PID is 1234, so an empty file named 1234 is created in the directory. This
-file lets the supervisor know the PID so it can shutdown the process later on.</p>
-
-<p>As of Storm 0.10.0, the context sent by Storm to shell components has been
-enhanced substantially to include all aspects of the topology context available
-to JVM components. One key addition is the ability to determine a shell
-component's source and targets (i.e., inputs and outputs) in the topology via
-the <code>stream->target->grouping</code> and <code>source->stream->grouping</code> dictionaries. At
-the innermost level of these nested dictionaries, groupings are represented as
-a dictionary that minimally has a <code>type</code> key, but can also have a <code>fields</code> key
-to specify which fields are involved in a <code>FIELDS</code> grouping.</p>
-
-<ul>
-<li>STDOUT: Your PID, in a JSON object, like <code>{"pid": 1234}</code>. The shell component will log the PID to its log.</li>
-</ul>
-
-<p>What happens next depends on the type of component:</p>
-
-<h3 id="spouts">Spouts</h3>
-
-<p>Shell spouts are synchronous. The rest happens in a while(true) loop:</p>
-
-<ul>
-<li>STDIN: Either a next, ack, or fail command.</li>
-</ul>
-
-<p>"next" is the equivalent of ISpout's <code>nextTuple</code>. It looks like:</p>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"next"</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
-<p>"ack" looks like:</p>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"ack"</span><span class="p">,</span><span class="w"> </span><span class="nt">"id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"1231231"</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
-<p>"fail" looks like:</p>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"fail"</span><span class="p">,</span><span class="w"> </span><span class="nt">"id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"1231231"</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
-<ul>
-<li>STDOUT: The results of your spout for the previous command. This can
-be a sequence of emits and logs.</li>
-</ul>
-
-<p>An emit looks like:</p>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="w">
- </span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"emit"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">for</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">tuple.</span><span class="w"> </span><span class="err">Leave</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">out</span><span class="w"> </span><span class="err">for</span><span class="w"> </span><span class="err">an</span><span class="w"> </span><span class="err">unreliable</span><span class="w"> </span><span class="err">emit.</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">can</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">be</span><span class="w"> </span><span class="err">a</span><span class="w"> </span><span class="err">string</span><span class="w"> </span><span class="err">or</span><span class="w"> </span><span class="err">a</span><span class="w"> </span><span class="err">number.</span><span class="w">
- </span><span class="nt">"id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"1231231"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">of</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">stream</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">tuple</span><span class="w"> </span><span class="err">was</span><span class="w"> </span><span class="err">emitted</span><span class="w"> </span><span class="err">to.</span><span class="w"> </span><span class="err">Leave</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">empty</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">emit</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">default</span><span class="w"> </span><span class="err">stream.</span><sp
an class="w">
- </span><span class="nt">"stream"</span><span class="p">:</span><span class="w"> </span><span class="s2">"1"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">If</span><span class="w"> </span><span class="err">doing</span><span class="w"> </span><span class="err">an</span><span class="w"> </span><span class="err">emit</span><span class="w"> </span><span class="err">direct,</span><span class="w"> </span><span class="err">indicate</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">task</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">send</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">tuple</span><span class="w"> </span><span class="err">to</span><span class="w">
- </span><span class="nt">"task"</span><span class="p">:</span><span class="w"> </span><span class="mi">9</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">All</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">values</span><span class="w"> </span><span class="err">in</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">tuple</span><span class="w">
- </span><span class="nt">"tuple"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"field1"</span><span class="p">,</span><span class="w"> </span><span class="mi">2</span><span class="p">,</span><span class="w"> </span><span class="mi">3</span><span class="p">]</span><span class="w">
-</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
-<p>If not doing an emit direct, you will immediately receive the task ids to which the tuple was emitted on STDIN as a JSON array.</p>
-
-<p>A "log" will log a message in the worker log. It looks like:</p>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="w">
- </span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"log"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">message</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">log</span><span class="w">
- </span><span class="nt">"msg"</span><span class="p">:</span><span class="w"> </span><span class="s2">"hello world!"</span><span class="w">
-</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
-<ul>
-<li>STDOUT: a "sync" command ends the sequence of emits and logs. It looks like:</li>
-</ul>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"sync"</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
-<p>After you sync, ShellSpout will not read your output until it sends another next, ack, or fail command.</p>
-
-<p>Note that, similarly to ISpout, all of the spouts in the worker will be locked up after a next, ack, or fail, until you sync. Also like ISpout, if you have no tuples to emit for a next, you should sleep for a small amount of time before syncing. ShellSpout will not automatically sleep for you.</p>
-
-<h3 id="bolts">Bolts</h3>
-
-<p>The shell bolt protocol is asynchronous. You will receive tuples on STDIN as soon as they are available, and you may emit, ack, and fail, and log at any time by writing to STDOUT, as follows:</p>
-
-<ul>
-<li>STDIN: A tuple! This is a JSON encoded structure like this:</li>
-</ul>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">tuple's</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">-</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">is</span><span class="w"> </span><span class="err">a</span><span class="w"> </span><span class="err">string</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">support</span><span class="w"> </span><span class="err">languages</span><span class="w"> </span><span class="err">lacking</span><span class="w"> </span><span class="err">64-bit</span><span class="w"> </span><span class="err">precision</span><span class="w">
- </span><span class="nt">"id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"-6955786537413359385"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">of</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">component</span><span class="w"> </span><span class="err">that</span><span class="w"> </span><span class="err">created</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">tuple</span><span class="w">
- </span><span class="nt">"comp"</span><span class="p">:</span><span class="w"> </span><span class="s2">"1"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">of</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">stream</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">tuple</span><span class="w"> </span><span class="err">was</span><span class="w"> </span><span class="err">emitted</span><span class="w"> </span><span class="err">to</span><span class="w">
- </span><span class="nt">"stream"</span><span class="p">:</span><span class="w"> </span><span class="s2">"1"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">of</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">task</span><span class="w"> </span><span class="err">that</span><span class="w"> </span><span class="err">created</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">tuple</span><span class="w">
- </span><span class="nt">"task"</span><span class="p">:</span><span class="w"> </span><span class="mi">9</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">All</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">values</span><span class="w"> </span><span class="err">in</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">tuple</span><span class="w">
- </span><span class="nt">"tuple"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"snow white and the seven dwarfs"</span><span class="p">,</span><span class="w"> </span><span class="s2">"field2"</span><span class="p">,</span><span class="w"> </span><span class="mi">3</span><span class="p">]</span><span class="w">
-</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
-<ul>
-<li>STDOUT: An ack, fail, emit, or log. Emits look like:</li>
-</ul>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="w">
- </span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"emit"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">ids</span><span class="w"> </span><span class="err">of</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">tuples</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">output</span><span class="w"> </span><span class="err">tuples</span><span class="w"> </span><span class="err">should</span><span class="w"> </span><span class="err">be</span><span class="w"> </span><span class="err">anchored</span><span class="w"> </span><span class="err">to</span><span class="w">
- </span><span class="nt">"anchors"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"1231231"</span><span class="p">,</span><span class="w"> </span><span class="s2">"-234234234"</span><span class="p">],</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">The</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">of</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">stream</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">tuple</span><span class="w"> </span><span class="err">was</span><span class="w"> </span><span class="err">emitted</span><span class="w"> </span><span class="err">to.</span><span class="w"> </span><span class="err">Leave</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">empty</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">emit</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">default</span><span class="w"> </span><span class="err">stream.</span><sp
an class="w">
- </span><span class="nt">"stream"</span><span class="p">:</span><span class="w"> </span><span class="s2">"1"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">If</span><span class="w"> </span><span class="err">doing</span><span class="w"> </span><span class="err">an</span><span class="w"> </span><span class="err">emit</span><span class="w"> </span><span class="err">direct,</span><span class="w"> </span><span class="err">indicate</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">task</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">send</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">tuple</span><span class="w"> </span><span class="err">to</span><span class="w">
- </span><span class="nt">"task"</span><span class="p">:</span><span class="w"> </span><span class="mi">9</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">All</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">values</span><span class="w"> </span><span class="err">in</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">tuple</span><span class="w">
- </span><span class="nt">"tuple"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"field1"</span><span class="p">,</span><span class="w"> </span><span class="mi">2</span><span class="p">,</span><span class="w"> </span><span class="mi">3</span><span class="p">]</span><span class="w">
-</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
-<p>If not doing an emit direct, you will receive the task ids to which
-the tuple was emitted on STDIN as a JSON array. Note that, due to the
-asynchronous nature of the shell bolt protocol, when you read after
-emitting, you may not receive the task ids. You may instead read the
-task ids for a previous emit or a new tuple to process. You will
-receive the task id lists in the same order as their corresponding
-emits, however.</p>
-
-<p>An ack looks like:</p>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="w">
- </span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"ack"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">of</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">tuple</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">ack</span><span class="w">
- </span><span class="nt">"id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"123123"</span><span class="w">
-</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
-<p>A fail looks like:</p>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="w">
- </span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"fail"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">id</span><span class="w"> </span><span class="err">of</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">tuple</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">fail</span><span class="w">
- </span><span class="nt">"id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"123123"</span><span class="w">
-</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
-<p>A "log" will log a message in the worker log. It looks like:</p>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="w">
- </span><span class="nt">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"log"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">the</span><span class="w"> </span><span class="err">message</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">log</span><span class="w">
- </span><span class="nt">"msg"</span><span class="p">:</span><span class="w"> </span><span class="s2">"hello world!"</span><span class="w">
-</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
-<ul>
-<li>Note that, as of version 0.7.1, there is no longer any need for a
-shell bolt to 'sync'.</li>
-</ul>
-
-<h3 id="handling-heartbeats-0-9-3-and-later">Handling Heartbeats (0.9.3 and later)</h3>
-
-<p>As of Storm 0.9.3, heartbeats have been between ShellSpout/ShellBolt and their
-multi-lang subprocesses to detect hanging/zombie subprocesses. Any libraries
-for interfacing with Storm via multi-lang must take the following actions
-regarding hearbeats:</p>
-
-<h4 id="spout">Spout</h4>
-
-<p>Shell spouts are synchronous, so subprocesses always send <code>sync</code> commands at the
-end of <code>next()</code>, so you should not have to do much to support heartbeats for
-spouts. That said, you must not let subprocesses sleep more than the worker
-timeout during <code>next()</code>.</p>
-
-<h4 id="bolt">Bolt</h4>
-
-<p>Shell bolts are asynchronous, so a ShellBolt will send heartbeat tuples to its
-subprocess periodically. Heartbeat tuple looks like:</p>
-<div class="highlight"><pre><code class="language-" data-lang=""><span class="p">{</span><span class="w">
- </span><span class="nt">"id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"-6955786537413359385"</span><span class="p">,</span><span class="w">
- </span><span class="nt">"comp"</span><span class="p">:</span><span class="w"> </span><span class="s2">"1"</span><span class="p">,</span><span class="w">
- </span><span class="nt">"stream"</span><span class="p">:</span><span class="w"> </span><span class="s2">"__heartbeat"</span><span class="p">,</span><span class="w">
- </span><span class="err">//</span><span class="w"> </span><span class="err">this</span><span class="w"> </span><span class="err">shell</span><span class="w"> </span><span class="err">bolt's</span><span class="w"> </span><span class="err">system</span><span class="w"> </span><span class="err">task</span><span class="w"> </span><span class="err">id</span><span class="w">
- </span><span class="nt">"task"</span><span class="p">:</span><span class="w"> </span><span class="mi">-1</span><span class="p">,</span><span class="w">
- </span><span class="nt">"tuple"</span><span class="p">:</span><span class="w"> </span><span class="p">[]</span><span class="w">
-</span><span class="p">}</span><span class="w">
-</span></code></pre></div>
-<p>When subprocess receives heartbeat tuple, it must send a <code>sync</code> command back to
-ShellBolt.</p>
-
-
-
- </div>
- </div>
- </div>
-<footer>
- <div class="container-fluid">
- <div class="row">
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>Meetups</h5>
- <ul class="latest-news">
-
- <li><a href="http://www.meetup.com/Apache-Storm-Apache-Kafka/">Apache Storm & Apache Kafka</a> <span class="small">(Sunnyvale, CA)</span></li>
-
- <li><a href="http://www.meetup.com/Apache-Storm-Kafka-Users/">Apache Storm & Kafka Users</a> <span class="small">(Seattle, WA)</span></li>
-
- <li><a href="http://www.meetup.com/New-York-City-Storm-User-Group/">NYC Storm User Group</a> <span class="small">(New York, NY)</span></li>
-
- <li><a href="http://www.meetup.com/Bay-Area-Stream-Processing">Bay Area Stream Processing</a> <span class="small">(Emeryville, CA)</span></li>
-
- <li><a href="http://www.meetup.com/Boston-Storm-Users/">Boston Realtime Data</a> <span class="small">(Boston, MA)</span></li>
-
- <li><a href="http://www.meetup.com/storm-london">London Storm User Group</a> <span class="small">(London, UK)</span></li>
-
- <!-- <li><a href="http://www.meetup.com/Apache-Storm-Kafka-Users/">Seatle, WA</a> <span class="small">(27 Jun 2015)</span></li> -->
- </ul>
- </div>
- </div>
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>About Storm</h5>
- <p>Storm integrates with any queueing system and any database system. Storm's spout abstraction makes it easy to integrate a new queuing system. Likewise, integrating Storm with database systems is easy.</p>
- </div>
- </div>
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>First Look</h5>
- <ul class="footer-list">
- <li><a href="/releases/current/Rationale.html">Rationale</a></li>
- <li><a href="/releases/current/Tutorial.html">Tutorial</a></li>
- <li><a href="/releases/current/Setting-up-development-environment.html">Setting up development environment</a></li>
- <li><a href="/releases/current/Creating-a-new-Storm-project.html">Creating a new Storm project</a></li>
- </ul>
- </div>
- </div>
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>Documentation</h5>
- <ul class="footer-list">
- <li><a href="/releases/current/index.html">Index</a></li>
- <li><a href="/releases/current/javadocs/index.html">Javadoc</a></li>
- <li><a href="/releases/current/FAQ.html">FAQ</a></li>
- </ul>
- </div>
- </div>
- </div>
- <hr/>
- <div class="row">
- <div class="col-md-12">
- <p align="center">Copyright © 2015 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved.
- <br>Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation.
- <br>All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
- </div>
- </div>
- </div>
-</footer>
-<!--Footer End-->
-<!-- Scroll to top -->
-<span class="totop"><a href="#"><i class="fa fa-angle-up"></i></a></span>
-
-</body>
-
-</html>
-