You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@storm.apache.org by pt...@apache.org on 2017/09/15 17:00:24 UTC
[45/51] [partial] storm-site git commit: test git pubsub
http://git-wip-us.apache.org/repos/asf/storm-site/blob/3730cb62/content/releases/0.10.0/Kestrel-and-Storm.html
----------------------------------------------------------------------
diff --git a/content/releases/0.10.0/Kestrel-and-Storm.html b/content/releases/0.10.0/Kestrel-and-Storm.html
deleted file mode 100644
index c1b3c19..0000000
--- a/content/releases/0.10.0/Kestrel-and-Storm.html
+++ /dev/null
@@ -1,432 +0,0 @@
-<!DOCTYPE html>
-<html>
- <head>
- <meta charset="utf-8">
- <meta http-equiv="X-UA-Compatible" content="IE=edge">
- <meta name="viewport" content="width=device-width, initial-scale=1">
-
- <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
- <link rel="icon" href="/favicon.ico" type="image/x-icon">
-
- <title>Storm and Kestrel</title>
-
- <!-- Bootstrap core CSS -->
- <link href="/assets/css/bootstrap.min.css" rel="stylesheet">
- <!-- Bootstrap theme -->
- <link href="/assets/css/bootstrap-theme.min.css" rel="stylesheet">
-
- <!-- Custom styles for this template -->
- <link rel="stylesheet" href="http://fortawesome.github.io/Font-Awesome/assets/font-awesome/css/font-awesome.css">
- <link href="/css/style.css" rel="stylesheet">
- <link href="/assets/css/owl.theme.css" rel="stylesheet">
- <link href="/assets/css/owl.carousel.css" rel="stylesheet">
- <script type="text/javascript" src="/assets/js/jquery.min.js"></script>
- <script type="text/javascript" src="/assets/js/bootstrap.min.js"></script>
- <script type="text/javascript" src="/assets/js/owl.carousel.min.js"></script>
- <script type="text/javascript" src="/assets/js/storm.js"></script>
- <!-- Just for debugging purposes. Don't actually copy these 2 lines! -->
- <!--[if lt IE 9]><script src="../../assets/js/ie8-responsive-file-warning.js"></script><![endif]-->
-
- <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
- <!--[if lt IE 9]>
- <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
- <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
- <![endif]-->
- </head>
-
-
- <body>
- <header>
- <div class="container-fluid">
- <div class="row">
- <div class="col-md-5">
- <a href="/index.html"><img src="/images/logo.png" class="logo" /></a>
- </div>
- <div class="col-md-5">
-
- <h1>Version: 0.10.0</h1>
-
- </div>
- <div class="col-md-2">
- <a href="/downloads.html" class="btn-std btn-block btn-download">Download</a>
- </div>
- </div>
- </div>
-</header>
-<!--Header End-->
-<!--Navigation Begin-->
-<div class="navbar" role="banner">
- <div class="container-fluid">
- <div class="navbar-header">
- <button class="navbar-toggle" type="button" data-toggle="collapse" data-target=".bs-navbar-collapse">
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- </button>
- </div>
- <nav class="collapse navbar-collapse bs-navbar-collapse" role="navigation">
- <ul class="nav navbar-nav">
- <li><a href="/index.html" id="home">Home</a></li>
- <li><a href="/getting-help.html" id="getting-help">Getting Help</a></li>
- <li><a href="/about/integrates.html" id="project-info">Project Information</a></li>
- <li class="dropdown">
- <a href="#" class="dropdown-toggle" data-toggle="dropdown" id="documentation">Documentation <b class="caret"></b></a>
- <ul class="dropdown-menu">
-
-
- <li><a href="/releases/2.0.0-SNAPSHOT/index.html">2.0.0-SNAPSHOT</a></li>
-
-
-
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
-
-
-
-
- <li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
-
-
-
- <li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
-
-
-
- <li><a href="/releases/1.0.2/index.html">1.0.2</a></li>
-
-
-
- <li><a href="/releases/1.0.1/index.html">1.0.1</a></li>
-
-
-
- <li><a href="/releases/1.0.0/index.html">1.0.0</a></li>
-
-
-
- <li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
-
-
-
- <li><a href="/releases/0.10.1/index.html">0.10.1</a></li>
-
-
-
-
-
- <li><a href="/releases/0.10.0/index.html">0.10.0</a></li>
-
-
-
- <li><a href="/releases/0.9.7/index.html">0.9.7</a></li>
-
-
-
- <li><a href="/releases/0.9.6/index.html">0.9.6</a></li>
-
-
-
-
-
-
-
-
-
-
-
-
- </ul>
- </li>
- <li><a href="/talksAndVideos.html">Talks and Slideshows</a></li>
- <li class="dropdown">
- <a href="#" class="dropdown-toggle" data-toggle="dropdown" id="contribute">Community <b class="caret"></b></a>
- <ul class="dropdown-menu">
- <li><a href="/contribute/Contributing-to-Storm.html">Contributing</a></li>
- <li><a href="/contribute/People.html">People</a></li>
- <li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
- </ul>
- </li>
- <li><a href="/2017/07/28/storm104-released.html" id="news">News</a></li>
- </ul>
- </nav>
- </div>
-</div>
-
-
-
- <div class="container-fluid">
- <h1 class="page-title">Storm and Kestrel</h1>
- <div class="row">
- <div class="col-md-12">
- <!-- Documentation -->
-
-<p class="post-meta"></p>
-
-<p>This page explains how to use to Storm to consume items from a Kestrel cluster.</p>
-
-<h2 id="preliminaries">Preliminaries</h2>
-
-<h3 id="storm">Storm</h3>
-
-<p>This tutorial uses examples from the <a href="https://github.com/nathanmarz/storm-kestrel">storm-kestrel</a> project and the <a href="http://github.com/apache/storm/blob/0.10.0/examples/storm-starter">storm-starter</a> project. It's recommended that you clone those projects and follow along with the examples. Read <a href="Setting-up-development-environment.html">Setting up development environment</a> and <a href="Creating-a-new-Storm-project.html">Creating a new Storm project</a> to get your machine set up.</p>
-
-<h3 id="kestrel">Kestrel</h3>
-
-<p>It assumes you are able to run locally a Kestrel server as described <a href="https://github.com/nathanmarz/storm-kestrel">here</a>.</p>
-
-<h2 id="kestrel-server-and-queue">Kestrel Server and Queue</h2>
-
-<p>A single kestrel server has a set of queues. A Kestrel queue is a very simple message queue that runs on the JVM and uses the memcache protocol (with some extensions) to talk to clients. For details, look at the implementation of the <a href="https://github.com/nathanmarz/storm-kestrel/blob/master/src/jvm/backtype/storm/spout/KestrelThriftClient.java">KestrelThriftClient</a> class provided in <a href="https://github.com/nathanmarz/storm-kestrel">storm-kestrel</a> project.</p>
-
-<p>Each queue is strictly ordered following the FIFO (first in, first out) principle. To keep up with performance items are cached in system memory; though, only the first 128MB is kept in memory. When stopping the server, the queue state is stored in a journal file.</p>
-
-<p>Further, details can be found <a href="https://github.com/nathanmarz/kestrel/blob/master/docs/guide.md">here</a>.</p>
-
-<p>Kestrel is:
-* fast
-* small
-* durable
-* reliable</p>
-
-<p>For instance, Twitter uses Kestrel as the backbone of its messaging infrastructure as described <a href="http://bhavin.directi.com/notes-on-kestrel-the-open-source-twitter-queue/">here</a>.</p>
-
-<h2 id="add-items-to-kestrel">Add items to Kestrel</h2>
-
-<p>At first, we need to have a program that can add items to a Kestrel queue. The following method takes benefit of the KestrelClient implementation in <a href="https://github.com/nathanmarz/storm-kestrel">storm-kestrel</a>. It adds sentences into a Kestrel queue randomly chosen out of an array that holds five possible sentences.</p>
-<div class="highlight"><pre><code class="language-" data-lang=""> private static void queueSentenceItems(KestrelClient kestrelClient, String queueName)
- throws ParseError, IOException {
-
- String[] sentences = new String[] {
- "the cow jumped over the moon",
- "an apple a day keeps the doctor away",
- "four score and seven years ago",
- "snow white and the seven dwarfs",
- "i am at two with nature"};
-
- Random _rand = new Random();
-
- for(int i=1; i<=10; i++){
-
- String sentence = sentences[_rand.nextInt(sentences.length)];
-
- String val = "ID " + i + " " + sentence;
-
- boolean queueSucess = kestrelClient.queue(queueName, val);
-
- System.out.println("queueSucess=" +queueSucess+ " [" + val +"]");
- }
- }
-</code></pre></div>
-<h2 id="remove-items-from-kestrel">Remove items from Kestrel</h2>
-
-<p>This method dequeues items from a queue without removing them.
-```
- private static void dequeueItems(KestrelClient kestrelClient, String queueName) throws IOException, ParseError
- {
- for(int i=1; i<=12; i++){</p>
-<div class="highlight"><pre><code class="language-" data-lang=""> Item item = kestrelClient.dequeue(queueName);
-
- if(item==null){
- System.out.println("The queue (" + queueName + ") contains no items.");
- }
- else
- {
- byte[] data = item._data;
-
- String receivedVal = new String(data);
-
- System.out.println("receivedItem=" + receivedVal);
- }
- }
-</code></pre></div><div class="highlight"><pre><code class="language-" data-lang="">
-This method dequeues items from a queue and then removes them.
-</code></pre></div><div class="highlight"><pre><code class="language-" data-lang="">private static void dequeueAndRemoveItems(KestrelClient kestrelClient, String queueName)
-throws IOException, ParseError
- {
- for(int i=1; i<=12; i++){
-
- Item item = kestrelClient.dequeue(queueName);
-
-
- if(item==null){
- System.out.println("The queue (" + queueName + ") contains no items.");
- }
- else
- {
- int itemID = item._id;
-
-
- byte[] data = item._data;
-
- String receivedVal = new String(data);
-
- kestrelClient.ack(queueName, itemID);
-
- System.out.println("receivedItem=" + receivedVal);
- }
- }
-}
-</code></pre></div><div class="highlight"><pre><code class="language-" data-lang="">
-## Add Items continuously to Kestrel
-
-This is our final program to run in order to add continuously sentence items to a queue called **sentence_queue** of a locally running Kestrel server.
-
-In order to stop it type a closing bracket char ']' in console and hit 'Enter'.
-
-</code></pre></div><div class="highlight"><pre><code class="language-" data-lang="">import java.io.IOException;
-import java.io.InputStream;
-import java.util.Random;
-
-import backtype.storm.spout.KestrelClient;
-import backtype.storm.spout.KestrelClient.Item;
-import backtype.storm.spout.KestrelClient.ParseError;
-
-public class AddSentenceItemsToKestrel {
-
- /**
- * @param args
- */
- public static void main(String[] args) {
-
- InputStream is = System.in;
-
- char closing_bracket = ']';
-
- int val = closing_bracket;
-
- boolean aux = true;
-
- try {
-
- KestrelClient kestrelClient = null;
- String queueName = "sentence_queue";
-
- while(aux){
-
- kestrelClient = new KestrelClient("localhost",22133);
-
- queueSentenceItems(kestrelClient, queueName);
-
- kestrelClient.close();
-
- Thread.sleep(1000);
-
- if(is.available()>0){
- if(val==is.read())
- aux=false;
- }
- }
- } catch (IOException e) {
- // TODO Auto-generated catch block
- e.printStackTrace();
- }
- catch (ParseError e) {
- // TODO Auto-generated catch block
- e.printStackTrace();
- } catch (InterruptedException e) {
- // TODO Auto-generated catch block
- e.printStackTrace();
- }
-
- System.out.println("end");
-
- }
-}
-</code></pre></div><div class="highlight"><pre><code class="language-" data-lang="">## Using KestrelSpout
-
-This topology reads sentences off of a Kestrel queue using KestrelSpout, splits the sentences into its constituent words (Bolt: SplitSentence), and then emits for each word the number of times it has seen that word before (Bolt: WordCount). How data is processed is described in detail in [Guaranteeing message processing](Guaranteeing-message-processing.html).
-
-</code></pre></div><div class="highlight"><pre><code class="language-" data-lang="">TopologyBuilder builder = new TopologyBuilder();
-builder.setSpout("sentences", new KestrelSpout("localhost",22133,"sentence_queue",new StringScheme()));
-builder.setBolt("split", new SplitSentence(), 10)
- .shuffleGrouping("sentences");
-builder.setBolt("count", new WordCount(), 20)
- .fieldsGrouping("split", new Fields("word"));
-</code></pre></div><div class="highlight"><pre><code class="language-" data-lang="">
-## Execution
-
-At first, start your local kestrel server in production or development mode.
-
-Than, wait about 5 seconds in order to avoid a ConnectException.
-
-Now execute the program to add items to the queue and launch the Storm topology. The order in which you launch the programs is of no importance.
-
-If you run the topology with TOPOLOGY_DEBUG you should see tuples being emitted in the topology.
-</code></pre></div>
-
-
- </div>
- </div>
- </div>
-<footer>
- <div class="container-fluid">
- <div class="row">
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>Meetups</h5>
- <ul class="latest-news">
-
- <li><a href="http://www.meetup.com/Apache-Storm-Apache-Kafka/">Apache Storm & Apache Kafka</a> <span class="small">(Sunnyvale, CA)</span></li>
-
- <li><a href="http://www.meetup.com/Apache-Storm-Kafka-Users/">Apache Storm & Kafka Users</a> <span class="small">(Seattle, WA)</span></li>
-
- <li><a href="http://www.meetup.com/New-York-City-Storm-User-Group/">NYC Storm User Group</a> <span class="small">(New York, NY)</span></li>
-
- <li><a href="http://www.meetup.com/Bay-Area-Stream-Processing">Bay Area Stream Processing</a> <span class="small">(Emeryville, CA)</span></li>
-
- <li><a href="http://www.meetup.com/Boston-Storm-Users/">Boston Realtime Data</a> <span class="small">(Boston, MA)</span></li>
-
- <li><a href="http://www.meetup.com/storm-london">London Storm User Group</a> <span class="small">(London, UK)</span></li>
-
- <!-- <li><a href="http://www.meetup.com/Apache-Storm-Kafka-Users/">Seatle, WA</a> <span class="small">(27 Jun 2015)</span></li> -->
- </ul>
- </div>
- </div>
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>About Storm</h5>
- <p>Storm integrates with any queueing system and any database system. Storm's spout abstraction makes it easy to integrate a new queuing system. Likewise, integrating Storm with database systems is easy.</p>
- </div>
- </div>
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>First Look</h5>
- <ul class="footer-list">
- <li><a href="/releases/current/Rationale.html">Rationale</a></li>
- <li><a href="/releases/current/Tutorial.html">Tutorial</a></li>
- <li><a href="/releases/current/Setting-up-development-environment.html">Setting up development environment</a></li>
- <li><a href="/releases/current/Creating-a-new-Storm-project.html">Creating a new Storm project</a></li>
- </ul>
- </div>
- </div>
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>Documentation</h5>
- <ul class="footer-list">
- <li><a href="/releases/current/index.html">Index</a></li>
- <li><a href="/releases/current/javadocs/index.html">Javadoc</a></li>
- <li><a href="/releases/current/FAQ.html">FAQ</a></li>
- </ul>
- </div>
- </div>
- </div>
- <hr/>
- <div class="row">
- <div class="col-md-12">
- <p align="center">Copyright © 2015 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved.
- <br>Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation.
- <br>All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
- </div>
- </div>
- </div>
-</footer>
-<!--Footer End-->
-<!-- Scroll to top -->
-<span class="totop"><a href="#"><i class="fa fa-angle-up"></i></a></span>
-
-</body>
-
-</html>
-
http://git-wip-us.apache.org/repos/asf/storm-site/blob/3730cb62/content/releases/0.10.0/Lifecycle-of-a-topology.html
----------------------------------------------------------------------
diff --git a/content/releases/0.10.0/Lifecycle-of-a-topology.html b/content/releases/0.10.0/Lifecycle-of-a-topology.html
deleted file mode 100644
index 940ee49..0000000
--- a/content/releases/0.10.0/Lifecycle-of-a-topology.html
+++ /dev/null
@@ -1,359 +0,0 @@
-<!DOCTYPE html>
-<html>
- <head>
- <meta charset="utf-8">
- <meta http-equiv="X-UA-Compatible" content="IE=edge">
- <meta name="viewport" content="width=device-width, initial-scale=1">
-
- <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
- <link rel="icon" href="/favicon.ico" type="image/x-icon">
-
- <title>Lifecycle of a Storm Topology</title>
-
- <!-- Bootstrap core CSS -->
- <link href="/assets/css/bootstrap.min.css" rel="stylesheet">
- <!-- Bootstrap theme -->
- <link href="/assets/css/bootstrap-theme.min.css" rel="stylesheet">
-
- <!-- Custom styles for this template -->
- <link rel="stylesheet" href="http://fortawesome.github.io/Font-Awesome/assets/font-awesome/css/font-awesome.css">
- <link href="/css/style.css" rel="stylesheet">
- <link href="/assets/css/owl.theme.css" rel="stylesheet">
- <link href="/assets/css/owl.carousel.css" rel="stylesheet">
- <script type="text/javascript" src="/assets/js/jquery.min.js"></script>
- <script type="text/javascript" src="/assets/js/bootstrap.min.js"></script>
- <script type="text/javascript" src="/assets/js/owl.carousel.min.js"></script>
- <script type="text/javascript" src="/assets/js/storm.js"></script>
- <!-- Just for debugging purposes. Don't actually copy these 2 lines! -->
- <!--[if lt IE 9]><script src="../../assets/js/ie8-responsive-file-warning.js"></script><![endif]-->
-
- <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
- <!--[if lt IE 9]>
- <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
- <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
- <![endif]-->
- </head>
-
-
- <body>
- <header>
- <div class="container-fluid">
- <div class="row">
- <div class="col-md-5">
- <a href="/index.html"><img src="/images/logo.png" class="logo" /></a>
- </div>
- <div class="col-md-5">
-
- <h1>Version: 0.10.0</h1>
-
- </div>
- <div class="col-md-2">
- <a href="/downloads.html" class="btn-std btn-block btn-download">Download</a>
- </div>
- </div>
- </div>
-</header>
-<!--Header End-->
-<!--Navigation Begin-->
-<div class="navbar" role="banner">
- <div class="container-fluid">
- <div class="navbar-header">
- <button class="navbar-toggle" type="button" data-toggle="collapse" data-target=".bs-navbar-collapse">
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- </button>
- </div>
- <nav class="collapse navbar-collapse bs-navbar-collapse" role="navigation">
- <ul class="nav navbar-nav">
- <li><a href="/index.html" id="home">Home</a></li>
- <li><a href="/getting-help.html" id="getting-help">Getting Help</a></li>
- <li><a href="/about/integrates.html" id="project-info">Project Information</a></li>
- <li class="dropdown">
- <a href="#" class="dropdown-toggle" data-toggle="dropdown" id="documentation">Documentation <b class="caret"></b></a>
- <ul class="dropdown-menu">
-
-
- <li><a href="/releases/2.0.0-SNAPSHOT/index.html">2.0.0-SNAPSHOT</a></li>
-
-
-
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
-
-
-
-
- <li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
-
-
-
- <li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
-
-
-
- <li><a href="/releases/1.0.2/index.html">1.0.2</a></li>
-
-
-
- <li><a href="/releases/1.0.1/index.html">1.0.1</a></li>
-
-
-
- <li><a href="/releases/1.0.0/index.html">1.0.0</a></li>
-
-
-
- <li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
-
-
-
- <li><a href="/releases/0.10.1/index.html">0.10.1</a></li>
-
-
-
-
-
- <li><a href="/releases/0.10.0/index.html">0.10.0</a></li>
-
-
-
- <li><a href="/releases/0.9.7/index.html">0.9.7</a></li>
-
-
-
- <li><a href="/releases/0.9.6/index.html">0.9.6</a></li>
-
-
-
-
-
-
-
-
-
-
-
-
- </ul>
- </li>
- <li><a href="/talksAndVideos.html">Talks and Slideshows</a></li>
- <li class="dropdown">
- <a href="#" class="dropdown-toggle" data-toggle="dropdown" id="contribute">Community <b class="caret"></b></a>
- <ul class="dropdown-menu">
- <li><a href="/contribute/Contributing-to-Storm.html">Contributing</a></li>
- <li><a href="/contribute/People.html">People</a></li>
- <li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
- </ul>
- </li>
- <li><a href="/2017/07/28/storm104-released.html" id="news">News</a></li>
- </ul>
- </nav>
- </div>
-</div>
-
-
-
- <div class="container-fluid">
- <h1 class="page-title">Lifecycle of a Storm Topology</h1>
- <div class="row">
- <div class="col-md-12">
- <!-- Documentation -->
-
-<p class="post-meta"></p>
-
-<p>(<strong>NOTE</strong>: this page is based on the 0.7.1 code; many things have changed since then, including a split between tasks and executors, and a reorganization of the code under <code>storm-core/src</code> rather than <code>src/</code>.)</p>
-
-<p>This page explains in detail the lifecycle of a topology from running the "storm jar" command to uploading the topology to Nimbus to the supervisors starting/stopping workers to workers and tasks setting themselves up. It also explains how Nimbus monitors topologies and how topologies are shutdown when they are killed.</p>
-
-<p>First a couple of important notes about topologies:</p>
-
-<ol>
-<li>The actual topology that runs is different than the topology the user specifies. The actual topology has implicit streams and an implicit "acker" bolt added to manage the acking framework (used to guarantee data processing). The implicit topology is created via the <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/common.clj#L188">system-topology!</a> function.</li>
-<li><code>system-topology!</code> is used in two places:
-
-<ul>
-<li>when Nimbus is creating tasks for the topology <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L316">code</a></li>
-<li>in the worker so it knows where it needs to route messages to <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/worker.clj#L90">code</a></li>
-</ul></li>
-</ol>
-
-<h2 id="starting-a-topology">Starting a topology</h2>
-
-<ul>
-<li>"storm jar" command executes your class with the specified arguments. The only special thing that "storm jar" does is set the "storm.jar" environment variable for use by <code>StormSubmitter</code> later. <a href="https://github.com/apache/storm/blob/0.7.1/bin/storm#L101">code</a></li>
-<li><p>When your code uses <code>StormSubmitter.submitTopology</code>, <code>StormSubmitter</code> takes the following actions:</p>
-
-<ul>
-<li>First, <code>StormSubmitter</code> uploads the jar if it hasn't been uploaded before. <a href="https://github.com/apache/storm/blob/0.7.1/src/jvm/backtype/storm/StormSubmitter.java#L83">code</a></li>
-<li>Jar uploading is done via Nimbus's Thrift interface <a href="https://github.com/apache/storm/blob/0.7.1/src/storm.thrift#L200">code</a></li>
-<li><code>beginFileUpload</code> returns a path in Nimbus's inbox</li>
-<li>15 kilobytes are uploaded at a time through <code>uploadChunk</code></li>
-<li><code>finishFileUpload</code> is called when it's finished uploading</li>
-<li>Here is Nimbus's implementation of those Thrift methods: <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L694">code</a></li>
-<li>Second, <code>StormSubmitter</code> calls <code>submitTopology</code> on the Nimbus thrift interface <a href="https://github.com/apache/storm/blob/0.7.1/src/jvm/backtype/storm/StormSubmitter.java#L60">code</a></li>
-<li>The topology config is serialized using JSON (JSON is used so that writing DSL's in any language is as easy as possible)</li>
-<li>Notice that the Thrift <code>submitTopology</code> call takes in the Nimbus inbox path where the jar was uploaded</li>
-</ul></li>
-<li><p>Nimbus receives the topology submission. <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L639">code</a></p></li>
-<li><p>Nimbus normalizes the topology configuration. The main purpose of normalization is to ensure that every single task will have the same serialization registrations, which is critical for getting serialization working correctly. <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L557">code</a></p></li>
-<li><p>Nimbus sets up the static state for the topology <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L661">code</a></p>
-
-<ul>
-<li>Jars and configs are kept on local filesystem because they're too big for Zookeeper. The jar and configs are copied into the path {nimbus local dir}/stormdist/{topology id}</li>
-<li><code>setup-storm-static</code> writes task -> component mapping into ZK</li>
-<li><code>setup-heartbeats</code> creates a ZK "directory" in which tasks can heartbeat</li>
-</ul></li>
-<li><p>Nimbus calls <code>mk-assignment</code> to assign tasks to machines <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L458">code</a></p>
-
-<ul>
-<li>Assignment record definition is here: <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/common.clj#L25">code</a></li>
-<li>Assignment contains:
-
-<ul>
-<li><code>master-code-dir</code>: used by supervisors to download the correct jars/configs for the topology from Nimbus</li>
-<li><code>task->node+port</code>: Map from a task id to the worker that task should be running on. (A worker is identified by a node/port pair)</li>
-<li><code>node->host</code>: A map from node id to hostname. This is used so workers know which machines to connect to to communicate with other workers. Node ids are used to identify supervisors so that multiple supervisors can be run on one machine. One place this is done is with Mesos integration.</li>
-<li><code>task->start-time-secs</code>: Contains a map from task id to the timestamp at which Nimbus launched that task. This is used by Nimbus when monitoring topologies, as tasks are given a longer timeout to heartbeat when they're first launched (the launch timeout is configured by "nimbus.task.launch.secs" config)</li>
-</ul></li>
-</ul></li>
-<li><p>Once topologies are assigned, they're initially in a deactivated mode. <code>start-storm</code> writes data into Zookeeper so that the cluster knows the topology is active and can start emitting tuples from spouts. <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L504">code</a></p></li>
-<li><p>TODO cluster state diagram (show all nodes and what's kept everywhere)</p></li>
-<li><p>Supervisor runs two functions in the background:</p>
-
-<ul>
-<li><code>synchronize-supervisor</code>: This is called whenever assignments in Zookeeper change and also every 10 seconds. <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/supervisor.clj#L241">code</a>
-
-<ul>
-<li>Downloads code from Nimbus for topologies assigned to this machine for which it doesn't have the code yet. <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/supervisor.clj#L258">code</a></li>
-<li>Writes into local filesystem what this node is supposed to be running. It writes a map from port -> LocalAssignment. LocalAssignment contains a topology id as well as the list of task ids for that worker. <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/supervisor.clj#L13">code</a></li>
-</ul></li>
-<li><code>sync-processes</code>: Reads from the LFS what <code>synchronize-supervisor</code> wrote and compares that to what's actually running on the machine. It then starts/stops worker processes as necessary to synchronize. <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/supervisor.clj#L177">code</a></li>
-</ul></li>
-<li><p>Worker processes start up through the <code>mk-worker</code> function <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/worker.clj#L67">code</a></p>
-
-<ul>
-<li>Worker connects to other workers and starts a thread to monitor for changes. So if a worker gets reassigned, the worker will automatically reconnect to the other worker's new location. <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/worker.clj#L123">code</a></li>
-<li>Monitors whether a topology is active or not and stores that state in the <code>storm-active-atom</code> variable. This variable is used by tasks to determine whether or not to call <code>nextTuple</code> on the spouts. <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/worker.clj#L155">code</a></li>
-<li>The worker launches the actual tasks as threads within it <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/worker.clj#L178">code</a></li>
-</ul></li>
-<li><p>Tasks are set up through the <code>mk-task</code> function <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/task.clj#L160">code</a></p>
-
-<ul>
-<li>Tasks set up routing function which takes in a stream and an output tuple and returns a list of task ids to send the tuple to <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/task.clj#L207">code</a> (there's also a 3-arity version used for direct streams)</li>
-<li>Tasks set up the spout-specific or bolt-specific code with <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/task.clj#L241">code</a></li>
-</ul></li>
-</ul>
-
-<h2 id="topology-monitoring">Topology Monitoring</h2>
-
-<ul>
-<li>Nimbus monitors the topology during its lifetime
-
-<ul>
-<li>Schedules recurring task on the timer thread to check the topologies <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L623">code</a></li>
-<li>Nimbus's behavior is represented as a finite state machine <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L98">code</a></li>
-<li>The "monitor" event is called on a topology every "nimbus.monitor.freq.secs", which calls <code>reassign-topology</code> through <code>reassign-transition</code> <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L497">code</a></li>
-<li><code>reassign-topology</code> calls <code>mk-assignments</code>, the same function used to assign the topology the first time. <code>mk-assignments</code> is also capable of incrementally updating a topology
-
-<ul>
-<li><code>mk-assignments</code> checks heartbeats and reassigns workers as necessary</li>
-<li>Any reassignments change the state in ZK, which will trigger supervisors to synchronize and start/stop workers</li>
-</ul></li>
-</ul></li>
-</ul>
-
-<h2 id="killing-a-topology">Killing a topology</h2>
-
-<ul>
-<li>"storm kill" command runs this code which just calls the Nimbus Thrift interface to kill the topology: <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/command/kill_topology.clj">code</a></li>
-<li>Nimbus receives the kill command <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L671">code</a></li>
-<li>Nimbus applies the "kill" transition to the topology <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L676">code</a></li>
-<li>The kill transition function changes the status of the topology to "killed" and schedules the "remove" event to run "wait time seconds" in the future. <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L63">code</a>
-
-<ul>
-<li>The wait time defaults to the topology message timeout but can be overridden with the -w flag in the "storm kill" command</li>
-<li>This causes the topology to be deactivated for the wait time before its actually shut down. This gives the topology a chance to finish processing what it's currently processing before shutting down the workers</li>
-<li>Changing the status during the kill transition ensures that the kill protocol is fault-tolerant to Nimbus crashing. On startup, if the status of the topology is "killed", Nimbus schedules the remove event to run "wait time seconds" in the future <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L111">code</a></li>
-</ul></li>
-<li>Removing a topology cleans out the assignment and static information from ZK <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L116">code</a></li>
-<li>A separate cleanup thread runs the <code>do-cleanup</code> function which will clean up the heartbeat dir and the jars/configs stored locally. <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L577">code</a></li>
-</ul>
-
-
-
- </div>
- </div>
- </div>
-<footer>
- <div class="container-fluid">
- <div class="row">
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>Meetups</h5>
- <ul class="latest-news">
-
- <li><a href="http://www.meetup.com/Apache-Storm-Apache-Kafka/">Apache Storm & Apache Kafka</a> <span class="small">(Sunnyvale, CA)</span></li>
-
- <li><a href="http://www.meetup.com/Apache-Storm-Kafka-Users/">Apache Storm & Kafka Users</a> <span class="small">(Seattle, WA)</span></li>
-
- <li><a href="http://www.meetup.com/New-York-City-Storm-User-Group/">NYC Storm User Group</a> <span class="small">(New York, NY)</span></li>
-
- <li><a href="http://www.meetup.com/Bay-Area-Stream-Processing">Bay Area Stream Processing</a> <span class="small">(Emeryville, CA)</span></li>
-
- <li><a href="http://www.meetup.com/Boston-Storm-Users/">Boston Realtime Data</a> <span class="small">(Boston, MA)</span></li>
-
- <li><a href="http://www.meetup.com/storm-london">London Storm User Group</a> <span class="small">(London, UK)</span></li>
-
- <!-- <li><a href="http://www.meetup.com/Apache-Storm-Kafka-Users/">Seatle, WA</a> <span class="small">(27 Jun 2015)</span></li> -->
- </ul>
- </div>
- </div>
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>About Storm</h5>
- <p>Storm integrates with any queueing system and any database system. Storm's spout abstraction makes it easy to integrate a new queuing system. Likewise, integrating Storm with database systems is easy.</p>
- </div>
- </div>
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>First Look</h5>
- <ul class="footer-list">
- <li><a href="/releases/current/Rationale.html">Rationale</a></li>
- <li><a href="/releases/current/Tutorial.html">Tutorial</a></li>
- <li><a href="/releases/current/Setting-up-development-environment.html">Setting up development environment</a></li>
- <li><a href="/releases/current/Creating-a-new-Storm-project.html">Creating a new Storm project</a></li>
- </ul>
- </div>
- </div>
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>Documentation</h5>
- <ul class="footer-list">
- <li><a href="/releases/current/index.html">Index</a></li>
- <li><a href="/releases/current/javadocs/index.html">Javadoc</a></li>
- <li><a href="/releases/current/FAQ.html">FAQ</a></li>
- </ul>
- </div>
- </div>
- </div>
- <hr/>
- <div class="row">
- <div class="col-md-12">
- <p align="center">Copyright © 2015 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved.
- <br>Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation.
- <br>All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
- </div>
- </div>
- </div>
-</footer>
-<!--Footer End-->
-<!-- Scroll to top -->
-<span class="totop"><a href="#"><i class="fa fa-angle-up"></i></a></span>
-
-</body>
-
-</html>
-
http://git-wip-us.apache.org/repos/asf/storm-site/blob/3730cb62/content/releases/0.10.0/Local-mode.html
----------------------------------------------------------------------
diff --git a/content/releases/0.10.0/Local-mode.html b/content/releases/0.10.0/Local-mode.html
deleted file mode 100644
index 68769db..0000000
--- a/content/releases/0.10.0/Local-mode.html
+++ /dev/null
@@ -1,262 +0,0 @@
-<!DOCTYPE html>
-<html>
- <head>
- <meta charset="utf-8">
- <meta http-equiv="X-UA-Compatible" content="IE=edge">
- <meta name="viewport" content="width=device-width, initial-scale=1">
-
- <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
- <link rel="icon" href="/favicon.ico" type="image/x-icon">
-
- <title>Local Mode</title>
-
- <!-- Bootstrap core CSS -->
- <link href="/assets/css/bootstrap.min.css" rel="stylesheet">
- <!-- Bootstrap theme -->
- <link href="/assets/css/bootstrap-theme.min.css" rel="stylesheet">
-
- <!-- Custom styles for this template -->
- <link rel="stylesheet" href="http://fortawesome.github.io/Font-Awesome/assets/font-awesome/css/font-awesome.css">
- <link href="/css/style.css" rel="stylesheet">
- <link href="/assets/css/owl.theme.css" rel="stylesheet">
- <link href="/assets/css/owl.carousel.css" rel="stylesheet">
- <script type="text/javascript" src="/assets/js/jquery.min.js"></script>
- <script type="text/javascript" src="/assets/js/bootstrap.min.js"></script>
- <script type="text/javascript" src="/assets/js/owl.carousel.min.js"></script>
- <script type="text/javascript" src="/assets/js/storm.js"></script>
- <!-- Just for debugging purposes. Don't actually copy these 2 lines! -->
- <!--[if lt IE 9]><script src="../../assets/js/ie8-responsive-file-warning.js"></script><![endif]-->
-
- <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
- <!--[if lt IE 9]>
- <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
- <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
- <![endif]-->
- </head>
-
-
- <body>
- <header>
- <div class="container-fluid">
- <div class="row">
- <div class="col-md-5">
- <a href="/index.html"><img src="/images/logo.png" class="logo" /></a>
- </div>
- <div class="col-md-5">
-
- <h1>Version: 0.10.0</h1>
-
- </div>
- <div class="col-md-2">
- <a href="/downloads.html" class="btn-std btn-block btn-download">Download</a>
- </div>
- </div>
- </div>
-</header>
-<!--Header End-->
-<!--Navigation Begin-->
-<div class="navbar" role="banner">
- <div class="container-fluid">
- <div class="navbar-header">
- <button class="navbar-toggle" type="button" data-toggle="collapse" data-target=".bs-navbar-collapse">
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- </button>
- </div>
- <nav class="collapse navbar-collapse bs-navbar-collapse" role="navigation">
- <ul class="nav navbar-nav">
- <li><a href="/index.html" id="home">Home</a></li>
- <li><a href="/getting-help.html" id="getting-help">Getting Help</a></li>
- <li><a href="/about/integrates.html" id="project-info">Project Information</a></li>
- <li class="dropdown">
- <a href="#" class="dropdown-toggle" data-toggle="dropdown" id="documentation">Documentation <b class="caret"></b></a>
- <ul class="dropdown-menu">
-
-
- <li><a href="/releases/2.0.0-SNAPSHOT/index.html">2.0.0-SNAPSHOT</a></li>
-
-
-
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
-
-
-
-
- <li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
-
-
-
- <li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
-
-
-
- <li><a href="/releases/1.0.2/index.html">1.0.2</a></li>
-
-
-
- <li><a href="/releases/1.0.1/index.html">1.0.1</a></li>
-
-
-
- <li><a href="/releases/1.0.0/index.html">1.0.0</a></li>
-
-
-
- <li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
-
-
-
- <li><a href="/releases/0.10.1/index.html">0.10.1</a></li>
-
-
-
-
-
- <li><a href="/releases/0.10.0/index.html">0.10.0</a></li>
-
-
-
- <li><a href="/releases/0.9.7/index.html">0.9.7</a></li>
-
-
-
- <li><a href="/releases/0.9.6/index.html">0.9.6</a></li>
-
-
-
-
-
-
-
-
-
-
-
-
- </ul>
- </li>
- <li><a href="/talksAndVideos.html">Talks and Slideshows</a></li>
- <li class="dropdown">
- <a href="#" class="dropdown-toggle" data-toggle="dropdown" id="contribute">Community <b class="caret"></b></a>
- <ul class="dropdown-menu">
- <li><a href="/contribute/Contributing-to-Storm.html">Contributing</a></li>
- <li><a href="/contribute/People.html">People</a></li>
- <li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
- </ul>
- </li>
- <li><a href="/2017/07/28/storm104-released.html" id="news">News</a></li>
- </ul>
- </nav>
- </div>
-</div>
-
-
-
- <div class="container-fluid">
- <h1 class="page-title">Local Mode</h1>
- <div class="row">
- <div class="col-md-12">
- <!-- Documentation -->
-
-<p class="post-meta"></p>
-
-<p>Local mode simulates a Storm cluster in process and is useful for developing and testing topologies. Running topologies in local mode is similar to running topologies <a href="Running-topologies-on-a-production-cluster.html">on a cluster</a>. </p>
-
-<p>To create an in-process cluster, simply use the <code>LocalCluster</code> class. For example:</p>
-<div class="highlight"><pre><code class="language-java" data-lang="java"><span class="kn">import</span> <span class="nn">backtype.storm.LocalCluster</span><span class="o">;</span>
-
-<span class="n">LocalCluster</span> <span class="n">cluster</span> <span class="o">=</span> <span class="k">new</span> <span class="n">LocalCluster</span><span class="o">();</span>
-</code></pre></div>
-<p>You can then submit topologies using the <code>submitTopology</code> method on the <code>LocalCluster</code> object. Just like the corresponding method on <a href="javadocs/backtype/storm/StormSubmitter.html">StormSubmitter</a>, <code>submitTopology</code> takes a name, a topology configuration, and the topology object. You can then kill a topology using the <code>killTopology</code> method which takes the topology name as an argument.</p>
-
-<p>To shutdown a local cluster, simple call:</p>
-<div class="highlight"><pre><code class="language-java" data-lang="java"><span class="n">cluster</span><span class="o">.</span><span class="na">shutdown</span><span class="o">();</span>
-</code></pre></div>
-<h3 id="common-configurations-for-local-mode">Common configurations for local mode</h3>
-
-<p>You can see a full list of configurations <a href="javadocs/backtype/storm/Config.html">here</a>.</p>
-
-<ol>
-<li><strong>Config.TOPOLOGY_MAX_TASK_PARALLELISM</strong>: This config puts a ceiling on the number of threads spawned for a single component. Oftentimes production topologies have a lot of parallelism (hundreds of threads) which places unreasonable load when trying to test the topology in local mode. This config lets you easy control that parallelism.</li>
-<li><strong>Config.TOPOLOGY_DEBUG</strong>: When this is set to true, Storm will log a message every time a tuple is emitted from any spout or bolt. This is extremely useful for debugging.</li>
-</ol>
-
-
-
- </div>
- </div>
- </div>
-<footer>
- <div class="container-fluid">
- <div class="row">
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>Meetups</h5>
- <ul class="latest-news">
-
- <li><a href="http://www.meetup.com/Apache-Storm-Apache-Kafka/">Apache Storm & Apache Kafka</a> <span class="small">(Sunnyvale, CA)</span></li>
-
- <li><a href="http://www.meetup.com/Apache-Storm-Kafka-Users/">Apache Storm & Kafka Users</a> <span class="small">(Seattle, WA)</span></li>
-
- <li><a href="http://www.meetup.com/New-York-City-Storm-User-Group/">NYC Storm User Group</a> <span class="small">(New York, NY)</span></li>
-
- <li><a href="http://www.meetup.com/Bay-Area-Stream-Processing">Bay Area Stream Processing</a> <span class="small">(Emeryville, CA)</span></li>
-
- <li><a href="http://www.meetup.com/Boston-Storm-Users/">Boston Realtime Data</a> <span class="small">(Boston, MA)</span></li>
-
- <li><a href="http://www.meetup.com/storm-london">London Storm User Group</a> <span class="small">(London, UK)</span></li>
-
- <!-- <li><a href="http://www.meetup.com/Apache-Storm-Kafka-Users/">Seatle, WA</a> <span class="small">(27 Jun 2015)</span></li> -->
- </ul>
- </div>
- </div>
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>About Storm</h5>
- <p>Storm integrates with any queueing system and any database system. Storm's spout abstraction makes it easy to integrate a new queuing system. Likewise, integrating Storm with database systems is easy.</p>
- </div>
- </div>
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>First Look</h5>
- <ul class="footer-list">
- <li><a href="/releases/current/Rationale.html">Rationale</a></li>
- <li><a href="/releases/current/Tutorial.html">Tutorial</a></li>
- <li><a href="/releases/current/Setting-up-development-environment.html">Setting up development environment</a></li>
- <li><a href="/releases/current/Creating-a-new-Storm-project.html">Creating a new Storm project</a></li>
- </ul>
- </div>
- </div>
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>Documentation</h5>
- <ul class="footer-list">
- <li><a href="/releases/current/index.html">Index</a></li>
- <li><a href="/releases/current/javadocs/index.html">Javadoc</a></li>
- <li><a href="/releases/current/FAQ.html">FAQ</a></li>
- </ul>
- </div>
- </div>
- </div>
- <hr/>
- <div class="row">
- <div class="col-md-12">
- <p align="center">Copyright © 2015 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved.
- <br>Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation.
- <br>All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
- </div>
- </div>
- </div>
-</footer>
-<!--Footer End-->
-<!-- Scroll to top -->
-<span class="totop"><a href="#"><i class="fa fa-angle-up"></i></a></span>
-
-</body>
-
-</html>
-
http://git-wip-us.apache.org/repos/asf/storm-site/blob/3730cb62/content/releases/0.10.0/Maven.html
----------------------------------------------------------------------
diff --git a/content/releases/0.10.0/Maven.html b/content/releases/0.10.0/Maven.html
deleted file mode 100644
index 25768b0..0000000
--- a/content/releases/0.10.0/Maven.html
+++ /dev/null
@@ -1,255 +0,0 @@
-<!DOCTYPE html>
-<html>
- <head>
- <meta charset="utf-8">
- <meta http-equiv="X-UA-Compatible" content="IE=edge">
- <meta name="viewport" content="width=device-width, initial-scale=1">
-
- <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
- <link rel="icon" href="/favicon.ico" type="image/x-icon">
-
- <title>Maven</title>
-
- <!-- Bootstrap core CSS -->
- <link href="/assets/css/bootstrap.min.css" rel="stylesheet">
- <!-- Bootstrap theme -->
- <link href="/assets/css/bootstrap-theme.min.css" rel="stylesheet">
-
- <!-- Custom styles for this template -->
- <link rel="stylesheet" href="http://fortawesome.github.io/Font-Awesome/assets/font-awesome/css/font-awesome.css">
- <link href="/css/style.css" rel="stylesheet">
- <link href="/assets/css/owl.theme.css" rel="stylesheet">
- <link href="/assets/css/owl.carousel.css" rel="stylesheet">
- <script type="text/javascript" src="/assets/js/jquery.min.js"></script>
- <script type="text/javascript" src="/assets/js/bootstrap.min.js"></script>
- <script type="text/javascript" src="/assets/js/owl.carousel.min.js"></script>
- <script type="text/javascript" src="/assets/js/storm.js"></script>
- <!-- Just for debugging purposes. Don't actually copy these 2 lines! -->
- <!--[if lt IE 9]><script src="../../assets/js/ie8-responsive-file-warning.js"></script><![endif]-->
-
- <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
- <!--[if lt IE 9]>
- <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
- <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
- <![endif]-->
- </head>
-
-
- <body>
- <header>
- <div class="container-fluid">
- <div class="row">
- <div class="col-md-5">
- <a href="/index.html"><img src="/images/logo.png" class="logo" /></a>
- </div>
- <div class="col-md-5">
-
- <h1>Version: 0.10.0</h1>
-
- </div>
- <div class="col-md-2">
- <a href="/downloads.html" class="btn-std btn-block btn-download">Download</a>
- </div>
- </div>
- </div>
-</header>
-<!--Header End-->
-<!--Navigation Begin-->
-<div class="navbar" role="banner">
- <div class="container-fluid">
- <div class="navbar-header">
- <button class="navbar-toggle" type="button" data-toggle="collapse" data-target=".bs-navbar-collapse">
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- </button>
- </div>
- <nav class="collapse navbar-collapse bs-navbar-collapse" role="navigation">
- <ul class="nav navbar-nav">
- <li><a href="/index.html" id="home">Home</a></li>
- <li><a href="/getting-help.html" id="getting-help">Getting Help</a></li>
- <li><a href="/about/integrates.html" id="project-info">Project Information</a></li>
- <li class="dropdown">
- <a href="#" class="dropdown-toggle" data-toggle="dropdown" id="documentation">Documentation <b class="caret"></b></a>
- <ul class="dropdown-menu">
-
-
- <li><a href="/releases/2.0.0-SNAPSHOT/index.html">2.0.0-SNAPSHOT</a></li>
-
-
-
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
-
-
-
-
- <li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
-
-
-
- <li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
-
-
-
- <li><a href="/releases/1.0.2/index.html">1.0.2</a></li>
-
-
-
- <li><a href="/releases/1.0.1/index.html">1.0.1</a></li>
-
-
-
- <li><a href="/releases/1.0.0/index.html">1.0.0</a></li>
-
-
-
- <li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
-
-
-
- <li><a href="/releases/0.10.1/index.html">0.10.1</a></li>
-
-
-
-
-
- <li><a href="/releases/0.10.0/index.html">0.10.0</a></li>
-
-
-
- <li><a href="/releases/0.9.7/index.html">0.9.7</a></li>
-
-
-
- <li><a href="/releases/0.9.6/index.html">0.9.6</a></li>
-
-
-
-
-
-
-
-
-
-
-
-
- </ul>
- </li>
- <li><a href="/talksAndVideos.html">Talks and Slideshows</a></li>
- <li class="dropdown">
- <a href="#" class="dropdown-toggle" data-toggle="dropdown" id="contribute">Community <b class="caret"></b></a>
- <ul class="dropdown-menu">
- <li><a href="/contribute/Contributing-to-Storm.html">Contributing</a></li>
- <li><a href="/contribute/People.html">People</a></li>
- <li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
- </ul>
- </li>
- <li><a href="/2017/07/28/storm104-released.html" id="news">News</a></li>
- </ul>
- </nav>
- </div>
-</div>
-
-
-
- <div class="container-fluid">
- <h1 class="page-title">Maven</h1>
- <div class="row">
- <div class="col-md-12">
- <!-- Documentation -->
-
-<p class="post-meta"></p>
-
-<p>To develop topologies, you'll need the Storm jars on your classpath. You should either include the unpacked jars in the classpath for your project or use Maven to include Storm as a development dependency. Storm is hosted on Maven Central. To include Storm in your project as a development dependency, add the following to your pom.xml:</p>
-<div class="highlight"><pre><code class="language-xml" data-lang="xml"><span class="nt"><dependency></span>
- <span class="nt"><groupId></span>org.apache.storm<span class="nt"></groupId></span>
- <span class="nt"><artifactId></span>storm-core<span class="nt"></artifactId></span>
- <span class="nt"><version></span>0.10.0<span class="nt"></version></span>
- <span class="nt"><scope></span>provided<span class="nt"></scope></span>
-<span class="nt"></dependency></span>
-</code></pre></div>
-<p><a href="http://github.com/apache/storm/blob/v0.10.0/examples/storm-starter/pom.xml">Here's an example</a> of a pom.xml for a Storm project.</p>
-
-<h3 id="developing-storm">Developing Storm</h3>
-
-<p>Please refer to <a href="http://github.com/apache/storm/blob/v0.10.0/DEVELOPER.md">DEVELOPER.md</a> for more details.</p>
-
-
-
- </div>
- </div>
- </div>
-<footer>
- <div class="container-fluid">
- <div class="row">
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>Meetups</h5>
- <ul class="latest-news">
-
- <li><a href="http://www.meetup.com/Apache-Storm-Apache-Kafka/">Apache Storm & Apache Kafka</a> <span class="small">(Sunnyvale, CA)</span></li>
-
- <li><a href="http://www.meetup.com/Apache-Storm-Kafka-Users/">Apache Storm & Kafka Users</a> <span class="small">(Seattle, WA)</span></li>
-
- <li><a href="http://www.meetup.com/New-York-City-Storm-User-Group/">NYC Storm User Group</a> <span class="small">(New York, NY)</span></li>
-
- <li><a href="http://www.meetup.com/Bay-Area-Stream-Processing">Bay Area Stream Processing</a> <span class="small">(Emeryville, CA)</span></li>
-
- <li><a href="http://www.meetup.com/Boston-Storm-Users/">Boston Realtime Data</a> <span class="small">(Boston, MA)</span></li>
-
- <li><a href="http://www.meetup.com/storm-london">London Storm User Group</a> <span class="small">(London, UK)</span></li>
-
- <!-- <li><a href="http://www.meetup.com/Apache-Storm-Kafka-Users/">Seatle, WA</a> <span class="small">(27 Jun 2015)</span></li> -->
- </ul>
- </div>
- </div>
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>About Storm</h5>
- <p>Storm integrates with any queueing system and any database system. Storm's spout abstraction makes it easy to integrate a new queuing system. Likewise, integrating Storm with database systems is easy.</p>
- </div>
- </div>
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>First Look</h5>
- <ul class="footer-list">
- <li><a href="/releases/current/Rationale.html">Rationale</a></li>
- <li><a href="/releases/current/Tutorial.html">Tutorial</a></li>
- <li><a href="/releases/current/Setting-up-development-environment.html">Setting up development environment</a></li>
- <li><a href="/releases/current/Creating-a-new-Storm-project.html">Creating a new Storm project</a></li>
- </ul>
- </div>
- </div>
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>Documentation</h5>
- <ul class="footer-list">
- <li><a href="/releases/current/index.html">Index</a></li>
- <li><a href="/releases/current/javadocs/index.html">Javadoc</a></li>
- <li><a href="/releases/current/FAQ.html">FAQ</a></li>
- </ul>
- </div>
- </div>
- </div>
- <hr/>
- <div class="row">
- <div class="col-md-12">
- <p align="center">Copyright © 2015 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved.
- <br>Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation.
- <br>All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
- </div>
- </div>
- </div>
-</footer>
-<!--Footer End-->
-<!-- Scroll to top -->
-<span class="totop"><a href="#"><i class="fa fa-angle-up"></i></a></span>
-
-</body>
-
-</html>
-
http://git-wip-us.apache.org/repos/asf/storm-site/blob/3730cb62/content/releases/0.10.0/Message-passing-implementation.html
----------------------------------------------------------------------
diff --git a/content/releases/0.10.0/Message-passing-implementation.html b/content/releases/0.10.0/Message-passing-implementation.html
deleted file mode 100644
index c5d0ec8..0000000
--- a/content/releases/0.10.0/Message-passing-implementation.html
+++ /dev/null
@@ -1,284 +0,0 @@
-<!DOCTYPE html>
-<html>
- <head>
- <meta charset="utf-8">
- <meta http-equiv="X-UA-Compatible" content="IE=edge">
- <meta name="viewport" content="width=device-width, initial-scale=1">
-
- <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
- <link rel="icon" href="/favicon.ico" type="image/x-icon">
-
- <title>Message Passing Implementation</title>
-
- <!-- Bootstrap core CSS -->
- <link href="/assets/css/bootstrap.min.css" rel="stylesheet">
- <!-- Bootstrap theme -->
- <link href="/assets/css/bootstrap-theme.min.css" rel="stylesheet">
-
- <!-- Custom styles for this template -->
- <link rel="stylesheet" href="http://fortawesome.github.io/Font-Awesome/assets/font-awesome/css/font-awesome.css">
- <link href="/css/style.css" rel="stylesheet">
- <link href="/assets/css/owl.theme.css" rel="stylesheet">
- <link href="/assets/css/owl.carousel.css" rel="stylesheet">
- <script type="text/javascript" src="/assets/js/jquery.min.js"></script>
- <script type="text/javascript" src="/assets/js/bootstrap.min.js"></script>
- <script type="text/javascript" src="/assets/js/owl.carousel.min.js"></script>
- <script type="text/javascript" src="/assets/js/storm.js"></script>
- <!-- Just for debugging purposes. Don't actually copy these 2 lines! -->
- <!--[if lt IE 9]><script src="../../assets/js/ie8-responsive-file-warning.js"></script><![endif]-->
-
- <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
- <!--[if lt IE 9]>
- <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
- <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
- <![endif]-->
- </head>
-
-
- <body>
- <header>
- <div class="container-fluid">
- <div class="row">
- <div class="col-md-5">
- <a href="/index.html"><img src="/images/logo.png" class="logo" /></a>
- </div>
- <div class="col-md-5">
-
- <h1>Version: 0.10.0</h1>
-
- </div>
- <div class="col-md-2">
- <a href="/downloads.html" class="btn-std btn-block btn-download">Download</a>
- </div>
- </div>
- </div>
-</header>
-<!--Header End-->
-<!--Navigation Begin-->
-<div class="navbar" role="banner">
- <div class="container-fluid">
- <div class="navbar-header">
- <button class="navbar-toggle" type="button" data-toggle="collapse" data-target=".bs-navbar-collapse">
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- </button>
- </div>
- <nav class="collapse navbar-collapse bs-navbar-collapse" role="navigation">
- <ul class="nav navbar-nav">
- <li><a href="/index.html" id="home">Home</a></li>
- <li><a href="/getting-help.html" id="getting-help">Getting Help</a></li>
- <li><a href="/about/integrates.html" id="project-info">Project Information</a></li>
- <li class="dropdown">
- <a href="#" class="dropdown-toggle" data-toggle="dropdown" id="documentation">Documentation <b class="caret"></b></a>
- <ul class="dropdown-menu">
-
-
- <li><a href="/releases/2.0.0-SNAPSHOT/index.html">2.0.0-SNAPSHOT</a></li>
-
-
-
- <li><a href="/releases/1.1.0/index.html">1.1.0</a></li>
-
-
-
-
-
- <li><a href="/releases/1.0.4/index.html">1.0.4</a></li>
-
-
-
-
-
- <li><a href="/releases/1.0.3/index.html">1.0.3</a></li>
-
-
-
- <li><a href="/releases/1.0.2/index.html">1.0.2</a></li>
-
-
-
- <li><a href="/releases/1.0.1/index.html">1.0.1</a></li>
-
-
-
- <li><a href="/releases/1.0.0/index.html">1.0.0</a></li>
-
-
-
- <li><a href="/releases/0.10.2/index.html">0.10.2</a></li>
-
-
-
- <li><a href="/releases/0.10.1/index.html">0.10.1</a></li>
-
-
-
-
-
- <li><a href="/releases/0.10.0/index.html">0.10.0</a></li>
-
-
-
- <li><a href="/releases/0.9.7/index.html">0.9.7</a></li>
-
-
-
- <li><a href="/releases/0.9.6/index.html">0.9.6</a></li>
-
-
-
-
-
-
-
-
-
-
-
-
- </ul>
- </li>
- <li><a href="/talksAndVideos.html">Talks and Slideshows</a></li>
- <li class="dropdown">
- <a href="#" class="dropdown-toggle" data-toggle="dropdown" id="contribute">Community <b class="caret"></b></a>
- <ul class="dropdown-menu">
- <li><a href="/contribute/Contributing-to-Storm.html">Contributing</a></li>
- <li><a href="/contribute/People.html">People</a></li>
- <li><a href="/contribute/BYLAWS.html">ByLaws</a></li>
- </ul>
- </li>
- <li><a href="/2017/07/28/storm104-released.html" id="news">News</a></li>
- </ul>
- </nav>
- </div>
-</div>
-
-
-
- <div class="container-fluid">
- <h1 class="page-title">Message Passing Implementation</h1>
- <div class="row">
- <div class="col-md-12">
- <!-- Documentation -->
-
-<p class="post-meta"></p>
-
-<p>(Note: this walkthrough is out of date as of 0.8.0. 0.8.0 revamped the message passing infrastructure to be based on the Disruptor)</p>
-
-<p>This page walks through how emitting and transferring tuples works in Storm.</p>
-
-<ul>
-<li>Worker is responsible for message transfer
-
-<ul>
-<li><code>refresh-connections</code> is called every "task.refresh.poll.secs" or whenever assignment in ZK changes. It manages connections to other workers and maintains a mapping from task -> worker <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/worker.clj#L123">code</a></li>
-<li>Provides a "transfer function" that is used by tasks to send tuples to other tasks. The transfer function takes in a task id and a tuple, and it serializes the tuple and puts it onto a "transfer queue". There is a single transfer queue for each worker. <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/worker.clj#L56">code</a></li>
-<li>The serializer is thread-safe <a href="https://github.com/apache/storm/blob/0.7.1/src/jvm/backtype/storm/serialization/KryoTupleSerializer.java#L26">code</a></li>
-<li>The worker has a single thread which drains the transfer queue and sends the messages to other workers <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/worker.clj#L185">code</a></li>
-<li>Message sending happens through this protocol: <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/messaging/protocol.clj">code</a></li>
-<li>The implementation for distributed mode uses ZeroMQ <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/messaging/zmq.clj">code</a></li>
-<li>The implementation for local mode uses in memory Java queues (so that it's easy to use Storm locally without needing to get ZeroMQ installed) <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/messaging/local.clj">code</a></li>
-</ul></li>
-<li>Receiving messages in tasks works differently in local mode and distributed mode
-
-<ul>
-<li>In local mode, the tuple is sent directly to an in-memory queue for the receiving task <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/messaging/local.clj#L21">code</a></li>
-<li>In distributed mode, each worker listens on a single TCP port for incoming messages and then routes those messages in-memory to tasks. The TCP port is called a "virtual port", because it receives [task id, message] and then routes it to the actual task. <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/worker.clj#L204">code</a>
-
-<ul>
-<li>The virtual port implementation is here: <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/zilch/virtual_port.clj">code</a></li>
-<li>Tasks listen on an in-memory ZeroMQ port for messages from the virtual port <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/task.clj#L201">code</a></li>
-<li>Bolts listen here: <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/task.clj#L489">code</a></li>
-<li>Spouts listen here: <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/task.clj#L382">code</a></li>
-</ul></li>
-</ul></li>
-<li>Tasks are responsible for message routing. A tuple is emitted either to a direct stream (where the task id is specified) or a regular stream. In direct streams, the message is only sent if that bolt subscribes to that direct stream. In regular streams, the stream grouping functions are used to determine the task ids to send the tuple to.
-
-<ul>
-<li>Tasks have a routing map from {stream id} -> {component id} -> {stream grouping function} <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/task.clj#L198">code</a></li>
-<li>The "tasks-fn" returns the task ids to send the tuples to for either regular stream emit or direct stream emit <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/task.clj#L207">code</a></li>
-<li>After getting the output task ids, bolts and spouts use the transfer-fn provided by the worker to actually transfer the tuples
-
-<ul>
-<li>Bolt transfer code here: <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/task.clj#L429">code</a></li>
-<li>Spout transfer code here: <a href="https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/task.clj#L329">code</a></li>
-</ul></li>
-</ul></li>
-</ul>
-
-
-
- </div>
- </div>
- </div>
-<footer>
- <div class="container-fluid">
- <div class="row">
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>Meetups</h5>
- <ul class="latest-news">
-
- <li><a href="http://www.meetup.com/Apache-Storm-Apache-Kafka/">Apache Storm & Apache Kafka</a> <span class="small">(Sunnyvale, CA)</span></li>
-
- <li><a href="http://www.meetup.com/Apache-Storm-Kafka-Users/">Apache Storm & Kafka Users</a> <span class="small">(Seattle, WA)</span></li>
-
- <li><a href="http://www.meetup.com/New-York-City-Storm-User-Group/">NYC Storm User Group</a> <span class="small">(New York, NY)</span></li>
-
- <li><a href="http://www.meetup.com/Bay-Area-Stream-Processing">Bay Area Stream Processing</a> <span class="small">(Emeryville, CA)</span></li>
-
- <li><a href="http://www.meetup.com/Boston-Storm-Users/">Boston Realtime Data</a> <span class="small">(Boston, MA)</span></li>
-
- <li><a href="http://www.meetup.com/storm-london">London Storm User Group</a> <span class="small">(London, UK)</span></li>
-
- <!-- <li><a href="http://www.meetup.com/Apache-Storm-Kafka-Users/">Seatle, WA</a> <span class="small">(27 Jun 2015)</span></li> -->
- </ul>
- </div>
- </div>
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>About Storm</h5>
- <p>Storm integrates with any queueing system and any database system. Storm's spout abstraction makes it easy to integrate a new queuing system. Likewise, integrating Storm with database systems is easy.</p>
- </div>
- </div>
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>First Look</h5>
- <ul class="footer-list">
- <li><a href="/releases/current/Rationale.html">Rationale</a></li>
- <li><a href="/releases/current/Tutorial.html">Tutorial</a></li>
- <li><a href="/releases/current/Setting-up-development-environment.html">Setting up development environment</a></li>
- <li><a href="/releases/current/Creating-a-new-Storm-project.html">Creating a new Storm project</a></li>
- </ul>
- </div>
- </div>
- <div class="col-md-3">
- <div class="footer-widget">
- <h5>Documentation</h5>
- <ul class="footer-list">
- <li><a href="/releases/current/index.html">Index</a></li>
- <li><a href="/releases/current/javadocs/index.html">Javadoc</a></li>
- <li><a href="/releases/current/FAQ.html">FAQ</a></li>
- </ul>
- </div>
- </div>
- </div>
- <hr/>
- <div class="row">
- <div class="col-md-12">
- <p align="center">Copyright © 2015 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved.
- <br>Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation.
- <br>All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
- </div>
- </div>
- </div>
-</footer>
-<!--Footer End-->
-<!-- Scroll to top -->
-<span class="totop"><a href="#"><i class="fa fa-angle-up"></i></a></span>
-
-</body>
-
-</html>
-