You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@storm.apache.org by bo...@apache.org on 2014/12/23 16:59:19 UTC
svn commit: r1647601 [10/20] - in /storm/site/publish: ./ 2012/08/02/
2012/09/06/ 2013/01/11/ 2013/12/08/ 2014/04/10/ 2014/04/17/ 2014/04/19/
2014/04/21/ 2014/04/22/ 2014/04/23/ 2014/04/27/ 2014/04/29/ 2014/05/27/
2014/06/17/ 2014/06/25/ 2014/10/20/ 20...
Modified: storm/site/publish/documentation/Configuration.html
URL: http://svn.apache.org/viewvc/storm/site/publish/documentation/Configuration.html?rev=1647601&r1=1647600&r2=1647601&view=diff
==============================================================================
--- storm/site/publish/documentation/Configuration.html (original)
+++ storm/site/publish/documentation/Configuration.html Tue Dec 23 15:59:17 2014
@@ -1,73 +1,81 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
-<head>
- <meta http-equiv="content-type" content="text/html;charset=utf-8" />
- <meta name="description" content="Storm is a distributed and fault-tolerant realtime computation system. Similar to how Hadoop provides a set of general primitives for doing batch processing, Storm provides a set of general primitives for doing realtime computation. Storm is simple, can be used with any programming language, and is a lot of fun to use!" />
- <meta name="keywords" content="storm, hadoop, realtime, stream, mapreduce, java, computation, scalability, clojure, scala, jvm, processing" />
- <title>Storm Documentation</title>
- <link rel="stylesheet" type="text/css" href="/css/style.css" media="screen" />
-<script type="text/javascript">
-
- var _gaq = _gaq || [];
- _gaq.push(['_setAccount', 'UA-32530285-1']);
- _gaq.push(['_trackPageview']);
-
- (function() {
- var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
- ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
- var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
- })();
-
-</script>
-</head>
-
-<body>
-<div id="wrap">
-<div id="top">
-<div id="projecttitle">
-<h2><a href="/" title="Back to main page">Storm</a></h2>
-<p id="slogan">Distributed and fault-tolerant realtime computation</p>
-</div>
-<div id="menu">
-<ul>
-<li><a href="/about/integrates.html">about</a></li>
-
-<!--
-<li><a href="/blog.html">blog</a></li>
--->
-
-<li><a href="/documentation/Home.html">documentation</a></li>
-<li><a href="/blog.html">blog</a></li>
-<li><a href="/downloads.html">downloads</a></li>
-<li><a href="/community.html">community</a></li>
-</ul>
-</div>
-</div>
-<div id="content">
-<div id="sidebar">
- <ul>
-
-<li><a class="" href="/documentation/Home.html">Documentation Home</a></li>
-
-<li><a class="" href="/documentation/Rationale.html">Rationale</a></li>
-
-<li><a class="" href="/documentation/Tutorial.html">Tutorial</a></li>
-
-<li><a class="" href="/documentation/Documentation.html">Manual</a></li>
-
-<li><a class="" href="/apidocs/">Javadoc</a></li>
-
-<li><a class="" href="/documentation/FAQ.html">FAQ</a></li>
-
-<li><a class="" href="/documentation/Contributing-to-Storm.html">Contributing</a></li>
-
- </ul>
-</div>
-<div id="aboutcontent">
-<p>Storm has a variety of configurations for tweaking the behavior of nimbus, supervisors, and running topologies. Some configurations are system configurations and cannot be modified on a topology by topology basis, whereas other configurations can be modified per topology. </p>
+<!DOCTYPE html>
+<html>
+
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <meta name="description" content="">
+ <meta name="author" content="">
+ <link rel="icon" href="/assets/favicon.ico">
+
+ <title>Configuration</title>
+
+ <!-- Bootstrap core CSS -->
+ <link href="/assets/css/bootstrap.min.css" rel="stylesheet">
+ <!-- Bootstrap theme -->
+ <link href="/assets/css/bootstrap-theme.min.css" rel="stylesheet">
+
+ <!-- Custom styles for this template -->
+ <link href="/assets/css/theme.css" rel="stylesheet">
+
+ <link href="/css/main.css" rel="stylesheet">
+
+ <!-- Just for debugging purposes. Don't actually copy these 2 lines! -->
+ <!--[if lt IE 9]><script src="../../assets/js/ie8-responsive-file-warning.js"></script><![endif]-->
+ <script src="/assets/js/ie-emulation-modes-warning.js"></script>
+
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ </head>
+
+
+
+ <body>
+ <div class="container">
+ <!-- Fixed navbar -->
+ <nav class="navbar navbar-default navbar-fixed-top" role="navigation">
+ <div class="container">
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a class="navbar-brand" href="/">Apache Storm<!-- <img alt="Apache Storm" src="/images/storm_logo_tagline_color.png"></a> -->
+ </div>
+ <div id="navbar" class="navbar-collapse collapse">
+ <ul class="nav navbar-nav">
+ <li><a href="/">Home</a></li>
+ <li><a href="/about/integrates.html">About</a></li>
+ <li><a href="/downloads.html">Download</a></li>
+ <li><a href="/documentation/Home.html">Documentation</a></li>
+ <li><a href="/news.html">News</a></li>
+ </ul>
+ </div><!--/.nav-collapse -->
+ </div>
+ </nav>
+
+
+
+
+
+ <!-- Documentation -->
+<!-- <div class="container"> -->
+
+ <header class="post-header">
+ <h1 class="post-title">Configuration</h1>
+ <p class="post-meta"></p>
+ </header>
+
+ <article class="post-content">
+ <p>Storm has a variety of configurations for tweaking the behavior of nimbus, supervisors, and running topologies. Some configurations are system configurations and cannot be modified on a topology by topology basis, whereas other configurations can be modified per topology. </p>
-<p>Every configuration has a default value defined in <a href="https://github.com/apache/incubator-storm/blob/master/conf/defaults.yaml">defaults.yaml</a> in the Storm codebase. You can override these configurations by defining a storm.yaml in the classpath of Nimbus and the supervisors. Finally, you can define a topology-specific configuration that you submit along with your topology when using <a href="/apidocs/backtype/storm/StormSubmitter.html">StormSubmitter</a>. However, the topology-specific configuration can only override configs prefixed with "TOPOLOGY".</p>
+<p>Every configuration has a default value defined in <a href="https://github.com/apache/storm/blob/master/conf/defaults.yaml">defaults.yaml</a> in the Storm codebase. You can override these configurations by defining a storm.yaml in the classpath of Nimbus and the supervisors. Finally, you can define a topology-specific configuration that you submit along with your topology when using <a href="/apidocs/backtype/storm/StormSubmitter.html">StormSubmitter</a>. However, the topology-specific configuration can only override configs prefixed with "TOPOLOGY".</p>
<p>Storm 0.7.0 and onwards lets you override configuration on a per-bolt/per-spout basis. The only configurations that can be overriden this way are:</p>
@@ -91,20 +99,36 @@
<ul>
<li><a href="/apidocs/backtype/storm/Config.html">Config</a>: a listing of all configurations as well as a helper class for creating topology specific configurations</li>
-<li><a href="https://github.com/apache/incubator-storm/blob/master/conf/defaults.yaml">defaults.yaml</a>: the default values for all configurations</li>
+<li><a href="https://github.com/apache/storm/blob/master/conf/defaults.yaml">defaults.yaml</a>: the default values for all configurations</li>
<li><a href="Setting-up-a-Storm-cluster.html">Setting up a Storm cluster</a>: explains how to create and configure a Storm cluster</li>
<li><a href="Running-topologies-on-a-production-cluster.html">Running topologies on a production cluster</a>: lists useful configurations when running topologies on a cluster</li>
<li><a href="Local-mode.html">Local mode</a>: lists useful configurations when using local mode</li>
</ul>
+ </article>
+
+<!-- </div> -->
+
+
+
+ <hr/>
+<div id="footer" class="container text-center">
+
+ <p class="text-muted credit"><p>
+Copyright © 2014 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved. Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
+
</div>
-</div>
-<div id="clear"></div></div>
-<div id="footer">
- <p>
-Copyright © 2014 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved. Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
-</div>
-</div>
-
-</body>
-</html>
\ No newline at end of file
+
+ <!-- Bootstrap core JavaScript
+ ================================================== -->
+ <!-- Placed at the end of the document so the pages load faster -->
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>
+ <script src="/assets/js/bootstrap.min.js"></script>
+ <script src="/assets/js/docs.min.js"></script>
+ <!-- IE10 viewport hack for Surface/desktop Windows 8 bug -->
+ <script src="/assets/js/ie10-viewport-bug-workaround.js"></script>
+ </div>
+
+ </body>
+
+</html>
Modified: storm/site/publish/documentation/Contributing-to-Storm.html
URL: http://svn.apache.org/viewvc/storm/site/publish/documentation/Contributing-to-Storm.html?rev=1647601&r1=1647600&r2=1647601&view=diff
==============================================================================
--- storm/site/publish/documentation/Contributing-to-Storm.html (original)
+++ storm/site/publish/documentation/Contributing-to-Storm.html Tue Dec 23 15:59:17 2014
@@ -1,79 +1,87 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
-<head>
- <meta http-equiv="content-type" content="text/html;charset=utf-8" />
- <meta name="description" content="Storm is a distributed and fault-tolerant realtime computation system. Similar to how Hadoop provides a set of general primitives for doing batch processing, Storm provides a set of general primitives for doing realtime computation. Storm is simple, can be used with any programming language, and is a lot of fun to use!" />
- <meta name="keywords" content="storm, hadoop, realtime, stream, mapreduce, java, computation, scalability, clojure, scala, jvm, processing" />
- <title>Storm Documentation</title>
- <link rel="stylesheet" type="text/css" href="/css/style.css" media="screen" />
-<script type="text/javascript">
-
- var _gaq = _gaq || [];
- _gaq.push(['_setAccount', 'UA-32530285-1']);
- _gaq.push(['_trackPageview']);
-
- (function() {
- var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
- ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
- var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
- })();
-
-</script>
-</head>
-
-<body>
-<div id="wrap">
-<div id="top">
-<div id="projecttitle">
-<h2><a href="/" title="Back to main page">Storm</a></h2>
-<p id="slogan">Distributed and fault-tolerant realtime computation</p>
-</div>
-<div id="menu">
-<ul>
-<li><a href="/about/integrates.html">about</a></li>
-
-<!--
-<li><a href="/blog.html">blog</a></li>
--->
-
-<li><a href="/documentation/Home.html">documentation</a></li>
-<li><a href="/blog.html">blog</a></li>
-<li><a href="/downloads.html">downloads</a></li>
-<li><a href="/community.html">community</a></li>
-</ul>
-</div>
-</div>
-<div id="content">
-<div id="sidebar">
- <ul>
-
-<li><a class="" href="/documentation/Home.html">Documentation Home</a></li>
-
-<li><a class="" href="/documentation/Rationale.html">Rationale</a></li>
-
-<li><a class="" href="/documentation/Tutorial.html">Tutorial</a></li>
-
-<li><a class="" href="/documentation/Documentation.html">Manual</a></li>
-
-<li><a class="" href="/apidocs/">Javadoc</a></li>
-
-<li><a class="" href="/documentation/FAQ.html">FAQ</a></li>
-
-<li><a class="current" href="/documentation/Contributing-to-Storm.html">Contributing</a></li>
-
- </ul>
-</div>
-<div id="aboutcontent">
-<h3 id="toc_0">Getting started with contributing</h3>
+<!DOCTYPE html>
+<html>
+
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <meta name="description" content="">
+ <meta name="author" content="">
+ <link rel="icon" href="/assets/favicon.ico">
+
+ <title>Contributing</title>
+
+ <!-- Bootstrap core CSS -->
+ <link href="/assets/css/bootstrap.min.css" rel="stylesheet">
+ <!-- Bootstrap theme -->
+ <link href="/assets/css/bootstrap-theme.min.css" rel="stylesheet">
+
+ <!-- Custom styles for this template -->
+ <link href="/assets/css/theme.css" rel="stylesheet">
+
+ <link href="/css/main.css" rel="stylesheet">
+
+ <!-- Just for debugging purposes. Don't actually copy these 2 lines! -->
+ <!--[if lt IE 9]><script src="../../assets/js/ie8-responsive-file-warning.js"></script><![endif]-->
+ <script src="/assets/js/ie-emulation-modes-warning.js"></script>
+
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ </head>
+
+
+
+ <body>
+ <div class="container">
+ <!-- Fixed navbar -->
+ <nav class="navbar navbar-default navbar-fixed-top" role="navigation">
+ <div class="container">
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a class="navbar-brand" href="/">Apache Storm<!-- <img alt="Apache Storm" src="/images/storm_logo_tagline_color.png"></a> -->
+ </div>
+ <div id="navbar" class="navbar-collapse collapse">
+ <ul class="nav navbar-nav">
+ <li><a href="/">Home</a></li>
+ <li><a href="/about/integrates.html">About</a></li>
+ <li><a href="/downloads.html">Download</a></li>
+ <li><a href="/documentation/Home.html">Documentation</a></li>
+ <li><a href="/news.html">News</a></li>
+ </ul>
+ </div><!--/.nav-collapse -->
+ </div>
+ </nav>
+
+
+
+
+
+ <!-- Documentation -->
+<!-- <div class="container"> -->
+
+ <header class="post-header">
+ <h1 class="post-title">Contributing</h1>
+ <p class="post-meta"></p>
+ </header>
+
+ <article class="post-content">
+ <h3 id="getting-started-with-contributing">Getting started with contributing</h3>
<p>Some of the issues on the <a href="https://issues.apache.org/jira/browse/STORM">issue tracker</a> are marked with the "Newbie" label. If you're interesting in contributing to Storm but don't know where to begin, these are good issues to start with. These issues are a great way to get your feet wet with learning the codebase because they require learning about only an isolated portion of the codebase and are a relatively small amount of work.</p>
-<h3 id="toc_1">Learning the codebase</h3>
+<h3 id="learning-the-codebase">Learning the codebase</h3>
<p>The <a href="Implementation-docs.html">Implementation docs</a> section of the wiki gives detailed walkthroughs of the codebase. Reading through these docs is highly recommended to understand the codebase.</p>
-<h3 id="toc_2">Contribution process</h3>
+<h3 id="contribution-process">Contribution process</h3>
<p>Contributions to the Storm codebase should be sent as GitHub pull requests. If there's any problems to the pull request we can iterate on it using GitHub's commenting features.</p>
@@ -86,22 +94,38 @@
<li>Implement your issue, submit a pull request, and iterate from there.</li>
</ol>
-<h3 id="toc_3">Modules built on top of Storm</h3>
+<h3 id="modules-built-on-top-of-storm">Modules built on top of Storm</h3>
<p>Modules built on top of Storm (like spouts, bolts, etc) that aren't appropriate for Storm core can be done as your own project or as part of <a href="https://github.com/stormprocessor">@stormprocessor</a>. To be part of @stormprocessor put your project on your own Github and then send an email to the mailing list proposing to make it part of @stormprocessor. Then the community can discuss whether it's useful enough to be part of @stormprocessor. Then you'll be added to the @stormprocessor organization and can maintain your project there. The advantage of hosting your module in @stormprocessor is that it will be easier for potential users to find your project.</p>
-<h3 id="toc_4">Contributing documentation</h3>
+<h3 id="contributing-documentation">Contributing documentation</h3>
<p>Documentation contributions are very welcome! The best way to send contributions is as emails through the mailing list.</p>
+ </article>
+
+<!-- </div> -->
+
+
+
+ <hr/>
+<div id="footer" class="container text-center">
+
+ <p class="text-muted credit"><p>
+Copyright © 2014 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved. Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
+
</div>
-</div>
-<div id="clear"></div></div>
-<div id="footer">
- <p>
-Copyright © 2014 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved. Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
-</div>
-</div>
-
-</body>
-</html>
\ No newline at end of file
+
+ <!-- Bootstrap core JavaScript
+ ================================================== -->
+ <!-- Placed at the end of the document so the pages load faster -->
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>
+ <script src="/assets/js/bootstrap.min.js"></script>
+ <script src="/assets/js/docs.min.js"></script>
+ <!-- IE10 viewport hack for Surface/desktop Windows 8 bug -->
+ <script src="/assets/js/ie10-viewport-bug-workaround.js"></script>
+ </div>
+
+ </body>
+
+</html>
Modified: storm/site/publish/documentation/Creating-a-new-Storm-project.html
URL: http://svn.apache.org/viewvc/storm/site/publish/documentation/Creating-a-new-Storm-project.html?rev=1647601&r1=1647600&r2=1647601&view=diff
==============================================================================
--- storm/site/publish/documentation/Creating-a-new-Storm-project.html (original)
+++ storm/site/publish/documentation/Creating-a-new-Storm-project.html Tue Dec 23 15:59:17 2014
@@ -1,88 +1,94 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
-<head>
- <meta http-equiv="content-type" content="text/html;charset=utf-8" />
- <meta name="description" content="Storm is a distributed and fault-tolerant realtime computation system. Similar to how Hadoop provides a set of general primitives for doing batch processing, Storm provides a set of general primitives for doing realtime computation. Storm is simple, can be used with any programming language, and is a lot of fun to use!" />
- <meta name="keywords" content="storm, hadoop, realtime, stream, mapreduce, java, computation, scalability, clojure, scala, jvm, processing" />
- <title>Storm Documentation</title>
- <link rel="stylesheet" type="text/css" href="/css/style.css" media="screen" />
-<script type="text/javascript">
-
- var _gaq = _gaq || [];
- _gaq.push(['_setAccount', 'UA-32530285-1']);
- _gaq.push(['_trackPageview']);
-
- (function() {
- var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
- ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
- var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
- })();
-
-</script>
-</head>
-
-<body>
-<div id="wrap">
-<div id="top">
-<div id="projecttitle">
-<h2><a href="/" title="Back to main page">Storm</a></h2>
-<p id="slogan">Distributed and fault-tolerant realtime computation</p>
-</div>
-<div id="menu">
-<ul>
-<li><a href="/about/integrates.html">about</a></li>
-
-<!--
-<li><a href="/blog.html">blog</a></li>
--->
-
-<li><a href="/documentation/Home.html">documentation</a></li>
-<li><a href="/blog.html">blog</a></li>
-<li><a href="/downloads.html">downloads</a></li>
-<li><a href="/community.html">community</a></li>
-</ul>
-</div>
-</div>
-<div id="content">
-<div id="sidebar">
- <ul>
-
-<li><a class="" href="/documentation/Home.html">Documentation Home</a></li>
-
-<li><a class="" href="/documentation/Rationale.html">Rationale</a></li>
-
-<li><a class="" href="/documentation/Tutorial.html">Tutorial</a></li>
-
-<li><a class="" href="/documentation/Documentation.html">Manual</a></li>
-
-<li><a class="" href="/apidocs/">Javadoc</a></li>
-
-<li><a class="" href="/documentation/FAQ.html">FAQ</a></li>
-
-<li><a class="" href="/documentation/Contributing-to-Storm.html">Contributing</a></li>
-
- </ul>
-</div>
-<div id="aboutcontent">
-<p>This page outlines how to set up a Storm project for development. The steps are:</p>
+<!DOCTYPE html>
+<html>
+
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <meta name="description" content="">
+ <meta name="author" content="">
+ <link rel="icon" href="/assets/favicon.ico">
+
+ <title>Creating a New Storm Project</title>
+
+ <!-- Bootstrap core CSS -->
+ <link href="/assets/css/bootstrap.min.css" rel="stylesheet">
+ <!-- Bootstrap theme -->
+ <link href="/assets/css/bootstrap-theme.min.css" rel="stylesheet">
+
+ <!-- Custom styles for this template -->
+ <link href="/assets/css/theme.css" rel="stylesheet">
+
+ <link href="/css/main.css" rel="stylesheet">
+
+ <!-- Just for debugging purposes. Don't actually copy these 2 lines! -->
+ <!--[if lt IE 9]><script src="../../assets/js/ie8-responsive-file-warning.js"></script><![endif]-->
+ <script src="/assets/js/ie-emulation-modes-warning.js"></script>
+
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ </head>
+
+
+
+ <body>
+ <div class="container">
+ <!-- Fixed navbar -->
+ <nav class="navbar navbar-default navbar-fixed-top" role="navigation">
+ <div class="container">
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a class="navbar-brand" href="/">Apache Storm<!-- <img alt="Apache Storm" src="/images/storm_logo_tagline_color.png"></a> -->
+ </div>
+ <div id="navbar" class="navbar-collapse collapse">
+ <ul class="nav navbar-nav">
+ <li><a href="/">Home</a></li>
+ <li><a href="/about/integrates.html">About</a></li>
+ <li><a href="/downloads.html">Download</a></li>
+ <li><a href="/documentation/Home.html">Documentation</a></li>
+ <li><a href="/news.html">News</a></li>
+ </ul>
+ </div><!--/.nav-collapse -->
+ </div>
+ </nav>
+
+
+
+
+
+ <!-- Documentation -->
+<!-- <div class="container"> -->
+
+ <header class="post-header">
+ <h1 class="post-title">Creating a New Storm Project</h1>
+ <p class="post-meta"></p>
+ </header>
+
+ <article class="post-content">
+ <p>This page outlines how to set up a Storm project for development. The steps are:</p>
<ol>
<li>Add Storm jars to classpath</li>
<li>If using multilang, add multilang dir to classpath</li>
</ol>
-<p>Follow along to see how to set up the <a href="http://github.com/nathanmarz/storm-starter">storm-starter</a> project in Eclipse.</p>
+<p>Follow along to see how to set up the <a href="https://github.com/apache/storm/blob/master/examples/storm-starter">storm-starter</a> project in Eclipse.</p>
-<h3 id="toc_0">Add Storm jars to classpath</h3>
+<h3 id="add-storm-jars-to-classpath">Add Storm jars to classpath</h3>
-<p>You'll need the Storm jars on your classpath to develop Storm topologies. Using <a href="Maven.html">Maven</a> is highly recommended. <a href="https://github.com/nathanmarz/storm-starter/blob/master/m2-pom.xml">Here's an example</a> of how to setup your pom.xml for a Storm project. If you don't want to use Maven, you can include the jars from the Storm release on your classpath. </p>
-
-<p><a href="http://github.com/nathanmarz/storm-starter">storm-starter</a> uses <a href="http://github.com/technomancy/leiningen">Leiningen</a> for build and dependency resolution. You can install leiningen by downloading <a href="https://raw.github.com/technomancy/leiningen/stable/bin/lein">this script</a>, placing it on your path, and making it executable. To retrieve the dependencies for Storm, simply run <code>lein deps</code> in the project root.</p>
+<p>You'll need the Storm jars on your classpath to develop Storm topologies. Using <a href="Maven.html">Maven</a> is highly recommended. <a href="https://github.com/apache/storm/blob/master/examples/storm-starter/pom.xml">Here's an example</a> of how to setup your pom.xml for a Storm project. If you don't want to use Maven, you can include the jars from the Storm release on your classpath.</p>
<p>To set up the classpath in Eclipse, create a new Java project, include <code>src/jvm/</code> as a source path, and make sure all the jars in <code>lib/</code> and <code>lib/dev/</code> are in the <code>Referenced Libraries</code> section of the project.</p>
-<h3 id="toc_1">If using multilang, add multilang dir to classpath</h3>
+<h3 id="if-using-multilang,-add-multilang-dir-to-classpath">If using multilang, add multilang dir to classpath</h3>
<p>If you implement spouts or bolts in languages other than Java, then those implementations should be under the <code>multilang/resources/</code> directory of the project. For Storm to find these files in local mode, the <code>resources/</code> dir needs to be on the classpath. You can do this in Eclipse by adding <code>multilang/</code> as a source folder. You may also need to add multilang/resources as a source directory.</p>
@@ -90,14 +96,30 @@
<p>To test that everything is working in Eclipse, you should now be able to <code>Run</code> the <code>WordCountTopology.java</code> file. You will see messages being emitted at the console for 10 seconds.</p>
+ </article>
+
+<!-- </div> -->
+
+
+
+ <hr/>
+<div id="footer" class="container text-center">
+
+ <p class="text-muted credit"><p>
+Copyright © 2014 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved. Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
+
</div>
-</div>
-<div id="clear"></div></div>
-<div id="footer">
- <p>
-Copyright © 2014 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved. Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
-</div>
-</div>
-
-</body>
-</html>
\ No newline at end of file
+
+ <!-- Bootstrap core JavaScript
+ ================================================== -->
+ <!-- Placed at the end of the document so the pages load faster -->
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>
+ <script src="/assets/js/bootstrap.min.js"></script>
+ <script src="/assets/js/docs.min.js"></script>
+ <!-- IE10 viewport hack for Surface/desktop Windows 8 bug -->
+ <script src="/assets/js/ie10-viewport-bug-workaround.js"></script>
+ </div>
+
+ </body>
+
+</html>
Modified: storm/site/publish/documentation/DSLs-and-multilang-adapters.html
URL: http://svn.apache.org/viewvc/storm/site/publish/documentation/DSLs-and-multilang-adapters.html?rev=1647601&r1=1647600&r2=1647601&view=diff
==============================================================================
--- storm/site/publish/documentation/DSLs-and-multilang-adapters.html (original)
+++ storm/site/publish/documentation/DSLs-and-multilang-adapters.html Tue Dec 23 15:59:17 2014
@@ -1,87 +1,110 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
-<head>
- <meta http-equiv="content-type" content="text/html;charset=utf-8" />
- <meta name="description" content="Storm is a distributed and fault-tolerant realtime computation system. Similar to how Hadoop provides a set of general primitives for doing batch processing, Storm provides a set of general primitives for doing realtime computation. Storm is simple, can be used with any programming language, and is a lot of fun to use!" />
- <meta name="keywords" content="storm, hadoop, realtime, stream, mapreduce, java, computation, scalability, clojure, scala, jvm, processing" />
- <title>Storm Documentation</title>
- <link rel="stylesheet" type="text/css" href="/css/style.css" media="screen" />
-<script type="text/javascript">
-
- var _gaq = _gaq || [];
- _gaq.push(['_setAccount', 'UA-32530285-1']);
- _gaq.push(['_trackPageview']);
-
- (function() {
- var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
- ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
- var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
- })();
-
-</script>
-</head>
-
-<body>
-<div id="wrap">
-<div id="top">
-<div id="projecttitle">
-<h2><a href="/" title="Back to main page">Storm</a></h2>
-<p id="slogan">Distributed and fault-tolerant realtime computation</p>
-</div>
-<div id="menu">
-<ul>
-<li><a href="/about/integrates.html">about</a></li>
-
-<!--
-<li><a href="/blog.html">blog</a></li>
--->
-
-<li><a href="/documentation/Home.html">documentation</a></li>
-<li><a href="/blog.html">blog</a></li>
-<li><a href="/downloads.html">downloads</a></li>
-<li><a href="/community.html">community</a></li>
-</ul>
-</div>
-</div>
-<div id="content">
-<div id="sidebar">
- <ul>
-
-<li><a class="" href="/documentation/Home.html">Documentation Home</a></li>
-
-<li><a class="" href="/documentation/Rationale.html">Rationale</a></li>
-
-<li><a class="" href="/documentation/Tutorial.html">Tutorial</a></li>
-
-<li><a class="" href="/documentation/Documentation.html">Manual</a></li>
-
-<li><a class="" href="/apidocs/">Javadoc</a></li>
-
-<li><a class="" href="/documentation/FAQ.html">FAQ</a></li>
-
-<li><a class="" href="/documentation/Contributing-to-Storm.html">Contributing</a></li>
-
- </ul>
-</div>
-<div id="aboutcontent">
-<ul>
+<!DOCTYPE html>
+<html>
+
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <meta name="description" content="">
+ <meta name="author" content="">
+ <link rel="icon" href="/assets/favicon.ico">
+
+ <title>Storm DSLs and Multi-Lang Adapters</title>
+
+ <!-- Bootstrap core CSS -->
+ <link href="/assets/css/bootstrap.min.css" rel="stylesheet">
+ <!-- Bootstrap theme -->
+ <link href="/assets/css/bootstrap-theme.min.css" rel="stylesheet">
+
+ <!-- Custom styles for this template -->
+ <link href="/assets/css/theme.css" rel="stylesheet">
+
+ <link href="/css/main.css" rel="stylesheet">
+
+ <!-- Just for debugging purposes. Don't actually copy these 2 lines! -->
+ <!--[if lt IE 9]><script src="../../assets/js/ie8-responsive-file-warning.js"></script><![endif]-->
+ <script src="/assets/js/ie-emulation-modes-warning.js"></script>
+
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ </head>
+
+
+
+ <body>
+ <div class="container">
+ <!-- Fixed navbar -->
+ <nav class="navbar navbar-default navbar-fixed-top" role="navigation">
+ <div class="container">
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a class="navbar-brand" href="/">Apache Storm<!-- <img alt="Apache Storm" src="/images/storm_logo_tagline_color.png"></a> -->
+ </div>
+ <div id="navbar" class="navbar-collapse collapse">
+ <ul class="nav navbar-nav">
+ <li><a href="/">Home</a></li>
+ <li><a href="/about/integrates.html">About</a></li>
+ <li><a href="/downloads.html">Download</a></li>
+ <li><a href="/documentation/Home.html">Documentation</a></li>
+ <li><a href="/news.html">News</a></li>
+ </ul>
+ </div><!--/.nav-collapse -->
+ </div>
+ </nav>
+
+
+
+
+
+ <!-- Documentation -->
+<!-- <div class="container"> -->
+
+ <header class="post-header">
+ <h1 class="post-title">Storm DSLs and Multi-Lang Adapters</h1>
+ <p class="post-meta"></p>
+ </header>
+
+ <article class="post-content">
+ <ul>
<li><a href="https://github.com/velvia/ScalaStorm">Scala DSL</a></li>
<li><a href="https://github.com/colinsurprenant/redstorm">JRuby DSL</a></li>
<li><a href="Clojure-DSL.html">Clojure DSL</a></li>
<li><a href="https://github.com/tomdz/storm-esper">Storm/Esper integration</a>: Streaming SQL on top of Storm</li>
-<li><a href="https://github.com/gphat/io-storm">io-storm</a>: Perl multilang adapter</li>
-<li><a href="https://github.com/lazyshot/storm-php">storm-php</a>: PHP multilang adapter</li>
+<li><a href="https://github.com/dan-blanchard/io-storm">io-storm</a>: Perl multilang adapter</li>
</ul>
+ </article>
+
+<!-- </div> -->
+
+
+
+ <hr/>
+<div id="footer" class="container text-center">
+
+ <p class="text-muted credit"><p>
+Copyright © 2014 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved. Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
+
</div>
-</div>
-<div id="clear"></div></div>
-<div id="footer">
- <p>
-Copyright © 2014 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved. Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
-</div>
-</div>
-
-</body>
-</html>
\ No newline at end of file
+
+ <!-- Bootstrap core JavaScript
+ ================================================== -->
+ <!-- Placed at the end of the document so the pages load faster -->
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>
+ <script src="/assets/js/bootstrap.min.js"></script>
+ <script src="/assets/js/docs.min.js"></script>
+ <!-- IE10 viewport hack for Surface/desktop Windows 8 bug -->
+ <script src="/assets/js/ie10-viewport-bug-workaround.js"></script>
+ </div>
+
+ </body>
+
+</html>
Modified: storm/site/publish/documentation/Defining-a-non-jvm-language-dsl-for-storm.html
URL: http://svn.apache.org/viewvc/storm/site/publish/documentation/Defining-a-non-jvm-language-dsl-for-storm.html?rev=1647601&r1=1647600&r2=1647601&view=diff
==============================================================================
--- storm/site/publish/documentation/Defining-a-non-jvm-language-dsl-for-storm.html (original)
+++ storm/site/publish/documentation/Defining-a-non-jvm-language-dsl-for-storm.html Tue Dec 23 15:59:17 2014
@@ -1,74 +1,82 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
-<head>
- <meta http-equiv="content-type" content="text/html;charset=utf-8" />
- <meta name="description" content="Storm is a distributed and fault-tolerant realtime computation system. Similar to how Hadoop provides a set of general primitives for doing batch processing, Storm provides a set of general primitives for doing realtime computation. Storm is simple, can be used with any programming language, and is a lot of fun to use!" />
- <meta name="keywords" content="storm, hadoop, realtime, stream, mapreduce, java, computation, scalability, clojure, scala, jvm, processing" />
- <title>Storm Documentation</title>
- <link rel="stylesheet" type="text/css" href="/css/style.css" media="screen" />
-<script type="text/javascript">
-
- var _gaq = _gaq || [];
- _gaq.push(['_setAccount', 'UA-32530285-1']);
- _gaq.push(['_trackPageview']);
-
- (function() {
- var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
- ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
- var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
- })();
-
-</script>
-</head>
-
-<body>
-<div id="wrap">
-<div id="top">
-<div id="projecttitle">
-<h2><a href="/" title="Back to main page">Storm</a></h2>
-<p id="slogan">Distributed and fault-tolerant realtime computation</p>
-</div>
-<div id="menu">
-<ul>
-<li><a href="/about/integrates.html">about</a></li>
-
-<!--
-<li><a href="/blog.html">blog</a></li>
--->
-
-<li><a href="/documentation/Home.html">documentation</a></li>
-<li><a href="/blog.html">blog</a></li>
-<li><a href="/downloads.html">downloads</a></li>
-<li><a href="/community.html">community</a></li>
-</ul>
-</div>
-</div>
-<div id="content">
-<div id="sidebar">
- <ul>
-
-<li><a class="" href="/documentation/Home.html">Documentation Home</a></li>
-
-<li><a class="" href="/documentation/Rationale.html">Rationale</a></li>
-
-<li><a class="" href="/documentation/Tutorial.html">Tutorial</a></li>
-
-<li><a class="" href="/documentation/Documentation.html">Manual</a></li>
-
-<li><a class="" href="/apidocs/">Javadoc</a></li>
-
-<li><a class="" href="/documentation/FAQ.html">FAQ</a></li>
-
-<li><a class="" href="/documentation/Contributing-to-Storm.html">Contributing</a></li>
-
- </ul>
-</div>
-<div id="aboutcontent">
-<p>The right place to start to learn how to make a non-JVM DSL for Storm is <a href="https://github.com/apache/incubator-storm/blob/master/storm-core/src/storm.thrift">storm-core/src/storm.thrift</a>. Since Storm topologies are just Thrift structures, and Nimbus is a Thrift daemon, you can create and submit topologies in any language.</p>
+<!DOCTYPE html>
+<html>
+
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <meta name="description" content="">
+ <meta name="author" content="">
+ <link rel="icon" href="/assets/favicon.ico">
+
+ <title>Defining a Non-JVM DSL for Storm</title>
+
+ <!-- Bootstrap core CSS -->
+ <link href="/assets/css/bootstrap.min.css" rel="stylesheet">
+ <!-- Bootstrap theme -->
+ <link href="/assets/css/bootstrap-theme.min.css" rel="stylesheet">
+
+ <!-- Custom styles for this template -->
+ <link href="/assets/css/theme.css" rel="stylesheet">
+
+ <link href="/css/main.css" rel="stylesheet">
+
+ <!-- Just for debugging purposes. Don't actually copy these 2 lines! -->
+ <!--[if lt IE 9]><script src="../../assets/js/ie8-responsive-file-warning.js"></script><![endif]-->
+ <script src="/assets/js/ie-emulation-modes-warning.js"></script>
+
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ </head>
+
+
+
+ <body>
+ <div class="container">
+ <!-- Fixed navbar -->
+ <nav class="navbar navbar-default navbar-fixed-top" role="navigation">
+ <div class="container">
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a class="navbar-brand" href="/">Apache Storm<!-- <img alt="Apache Storm" src="/images/storm_logo_tagline_color.png"></a> -->
+ </div>
+ <div id="navbar" class="navbar-collapse collapse">
+ <ul class="nav navbar-nav">
+ <li><a href="/">Home</a></li>
+ <li><a href="/about/integrates.html">About</a></li>
+ <li><a href="/downloads.html">Download</a></li>
+ <li><a href="/documentation/Home.html">Documentation</a></li>
+ <li><a href="/news.html">News</a></li>
+ </ul>
+ </div><!--/.nav-collapse -->
+ </div>
+ </nav>
+
+
+
+
+
+ <!-- Documentation -->
+<!-- <div class="container"> -->
+
+ <header class="post-header">
+ <h1 class="post-title">Defining a Non-JVM DSL for Storm</h1>
+ <p class="post-meta"></p>
+ </header>
+
+ <article class="post-content">
+ <p>The right place to start to learn how to make a non-JVM DSL for Storm is <a href="https://github.com/apache/storm/blob/master/storm-core/src/storm.thrift">storm-core/src/storm.thrift</a>. Since Storm topologies are just Thrift structures, and Nimbus is a Thrift daemon, you can create and submit topologies in any language.</p>
<p>When you create the Thrift structs for spouts and bolts, the code for the spout or bolt is specified in the ComponentObject struct:</p>
-<div class="highlight"><pre><code class="text language-text" data-lang="text">union ComponentObject {
+<div class="highlight"><pre><code class="language-text" data-lang="text">union ComponentObject {
1: binary serialized_java;
2: ShellComponent shell;
3: JavaObject java_object;
@@ -77,24 +85,40 @@
<p>For a Python DSL, you would want to make use of "2" and "3". ShellComponent lets you specify a script to run that component (e.g., your python code). And JavaObject lets you specify native java spouts and bolts for the component (and Storm will use reflection to create that spout or bolt).</p>
<p>There's a "storm shell" command that will help with submitting a topology. Its usage is like this:</p>
-<div class="highlight"><pre><code class="text language-text" data-lang="text">storm shell resources/ python topology.py arg1 arg2
+<div class="highlight"><pre><code class="language-text" data-lang="text">storm shell resources/ python topology.py arg1 arg2
</code></pre></div>
<p>storm shell will then package resources/ into a jar, upload the jar to Nimbus, and call your topology.py script like this:</p>
-<div class="highlight"><pre><code class="text language-text" data-lang="text">python topology.py arg1 arg2 {nimbus-host} {nimbus-port} {uploaded-jar-location}
+<div class="highlight"><pre><code class="language-text" data-lang="text">python topology.py arg1 arg2 {nimbus-host} {nimbus-port} {uploaded-jar-location}
</code></pre></div>
<p>Then you can connect to Nimbus using the Thrift API and submit the topology, passing {uploaded-jar-location} into the submitTopology method. For reference, here's the submitTopology definition:</p>
-<div class="highlight"><pre><code class="java language-java" data-lang="java"><span class="kt">void</span> <span class="nf">submitTopology</span><span class="o">(</span><span class="mi">1</span><span class="o">:</span> <span class="n">string</span> <span class="n">name</span><span class="o">,</span> <span class="mi">2</span><span class="o">:</span> <span class="n">string</span> <span class="n">uploadedJarLocation</span><span class="o">,</span> <span class="mi">3</span><span class="o">:</span> <span class="n">string</span> <span class="n">jsonConf</span><span class="o">,</span> <span class="mi">4</span><span class="o">:</span> <span class="n">StormTopology</span> <span class="n">topology</span><span class="o">)</span> <span class="kd">throws</span> <span class="o">(</span><span class="mi">1</span><span class="o">:</span> <span class="n">AlreadyAliveException</span> <span class="n">e</span><span class="o">,</span> <span class="mi">2</span><span class="o">:</span> <span class="n">Inval
idTopologyException</span> <span class="n">ite</span><span class="o">);</span>
+<div class="highlight"><pre><code class="language-java" data-lang="java"><span class="kt">void</span> <span class="nf">submitTopology</span><span class="o">(</span><span class="mi">1</span><span class="o">:</span> <span class="n">string</span> <span class="n">name</span><span class="o">,</span> <span class="mi">2</span><span class="o">:</span> <span class="n">string</span> <span class="n">uploadedJarLocation</span><span class="o">,</span> <span class="mi">3</span><span class="o">:</span> <span class="n">string</span> <span class="n">jsonConf</span><span class="o">,</span> <span class="mi">4</span><span class="o">:</span> <span class="n">StormTopology</span> <span class="n">topology</span><span class="o">)</span> <span class="kd">throws</span> <span class="o">(</span><span class="mi">1</span><span class="o">:</span> <span class="n">AlreadyAliveException</span> <span class="n">e</span><span class="o">,</span> <span class="mi">2</span><span class="o">:</span> <span class="n">InvalidTop
ologyException</span> <span class="n">ite</span><span class="o">);</span>
</code></pre></div>
<p>Finally, one of the key things to do in a non-JVM DSL is make it easy to define the entire topology in one file (the bolts, spouts, and the definition of the topology).</p>
+ </article>
+
+<!-- </div> -->
+
+
+
+ <hr/>
+<div id="footer" class="container text-center">
+
+ <p class="text-muted credit"><p>
+Copyright © 2014 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved. Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
+
</div>
-</div>
-<div id="clear"></div></div>
-<div id="footer">
- <p>
-Copyright © 2014 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved. Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
-</div>
-</div>
-
-</body>
-</html>
\ No newline at end of file
+
+ <!-- Bootstrap core JavaScript
+ ================================================== -->
+ <!-- Placed at the end of the document so the pages load faster -->
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>
+ <script src="/assets/js/bootstrap.min.js"></script>
+ <script src="/assets/js/docs.min.js"></script>
+ <!-- IE10 viewport hack for Surface/desktop Windows 8 bug -->
+ <script src="/assets/js/ie10-viewport-bug-workaround.js"></script>
+ </div>
+
+ </body>
+
+</html>
Modified: storm/site/publish/documentation/Distributed-RPC.html
URL: http://svn.apache.org/viewvc/storm/site/publish/documentation/Distributed-RPC.html?rev=1647601&r1=1647600&r2=1647601&view=diff
==============================================================================
--- storm/site/publish/documentation/Distributed-RPC.html (original)
+++ storm/site/publish/documentation/Distributed-RPC.html Tue Dec 23 15:59:17 2014
@@ -1,78 +1,86 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
-<head>
- <meta http-equiv="content-type" content="text/html;charset=utf-8" />
- <meta name="description" content="Storm is a distributed and fault-tolerant realtime computation system. Similar to how Hadoop provides a set of general primitives for doing batch processing, Storm provides a set of general primitives for doing realtime computation. Storm is simple, can be used with any programming language, and is a lot of fun to use!" />
- <meta name="keywords" content="storm, hadoop, realtime, stream, mapreduce, java, computation, scalability, clojure, scala, jvm, processing" />
- <title>Storm Documentation</title>
- <link rel="stylesheet" type="text/css" href="/css/style.css" media="screen" />
-<script type="text/javascript">
-
- var _gaq = _gaq || [];
- _gaq.push(['_setAccount', 'UA-32530285-1']);
- _gaq.push(['_trackPageview']);
-
- (function() {
- var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
- ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
- var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
- })();
-
-</script>
-</head>
-
-<body>
-<div id="wrap">
-<div id="top">
-<div id="projecttitle">
-<h2><a href="/" title="Back to main page">Storm</a></h2>
-<p id="slogan">Distributed and fault-tolerant realtime computation</p>
-</div>
-<div id="menu">
-<ul>
-<li><a href="/about/integrates.html">about</a></li>
-
-<!--
-<li><a href="/blog.html">blog</a></li>
--->
-
-<li><a href="/documentation/Home.html">documentation</a></li>
-<li><a href="/blog.html">blog</a></li>
-<li><a href="/downloads.html">downloads</a></li>
-<li><a href="/community.html">community</a></li>
-</ul>
-</div>
-</div>
-<div id="content">
-<div id="sidebar">
- <ul>
-
-<li><a class="" href="/documentation/Home.html">Documentation Home</a></li>
-
-<li><a class="" href="/documentation/Rationale.html">Rationale</a></li>
-
-<li><a class="" href="/documentation/Tutorial.html">Tutorial</a></li>
-
-<li><a class="" href="/documentation/Documentation.html">Manual</a></li>
-
-<li><a class="" href="/apidocs/">Javadoc</a></li>
-
-<li><a class="" href="/documentation/FAQ.html">FAQ</a></li>
-
-<li><a class="" href="/documentation/Contributing-to-Storm.html">Contributing</a></li>
-
- </ul>
-</div>
-<div id="aboutcontent">
-<p>The idea behind distributed RPC (DRPC) is to parallelize the computation of really intense functions on the fly using Storm. The Storm topology takes in as input a stream of function arguments, and it emits an output stream of the results for each of those function calls. </p>
+<!DOCTYPE html>
+<html>
+
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <meta name="description" content="">
+ <meta name="author" content="">
+ <link rel="icon" href="/assets/favicon.ico">
+
+ <title>Distributed RPC</title>
+
+ <!-- Bootstrap core CSS -->
+ <link href="/assets/css/bootstrap.min.css" rel="stylesheet">
+ <!-- Bootstrap theme -->
+ <link href="/assets/css/bootstrap-theme.min.css" rel="stylesheet">
+
+ <!-- Custom styles for this template -->
+ <link href="/assets/css/theme.css" rel="stylesheet">
+
+ <link href="/css/main.css" rel="stylesheet">
+
+ <!-- Just for debugging purposes. Don't actually copy these 2 lines! -->
+ <!--[if lt IE 9]><script src="../../assets/js/ie8-responsive-file-warning.js"></script><![endif]-->
+ <script src="/assets/js/ie-emulation-modes-warning.js"></script>
+
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ </head>
+
+
+
+ <body>
+ <div class="container">
+ <!-- Fixed navbar -->
+ <nav class="navbar navbar-default navbar-fixed-top" role="navigation">
+ <div class="container">
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a class="navbar-brand" href="/">Apache Storm<!-- <img alt="Apache Storm" src="/images/storm_logo_tagline_color.png"></a> -->
+ </div>
+ <div id="navbar" class="navbar-collapse collapse">
+ <ul class="nav navbar-nav">
+ <li><a href="/">Home</a></li>
+ <li><a href="/about/integrates.html">About</a></li>
+ <li><a href="/downloads.html">Download</a></li>
+ <li><a href="/documentation/Home.html">Documentation</a></li>
+ <li><a href="/news.html">News</a></li>
+ </ul>
+ </div><!--/.nav-collapse -->
+ </div>
+ </nav>
+
+
+
+
+
+ <!-- Documentation -->
+<!-- <div class="container"> -->
+
+ <header class="post-header">
+ <h1 class="post-title">Distributed RPC</h1>
+ <p class="post-meta"></p>
+ </header>
+
+ <article class="post-content">
+ <p>The idea behind distributed RPC (DRPC) is to parallelize the computation of really intense functions on the fly using Storm. The Storm topology takes in as input a stream of function arguments, and it emits an output stream of the results for each of those function calls. </p>
<p>DRPC is not so much a feature of Storm as it is a pattern expressed from Storm's primitives of streams, spouts, bolts, and topologies. DRPC could have been packaged as a separate library from Storm, but it's so useful that it's bundled with Storm.</p>
-<h3 id="toc_0">High level overview</h3>
+<h3 id="high-level-overview">High level overview</h3>
<p>Distributed RPC is coordinated by a "DRPC server" (Storm comes packaged with an implementation of this). The DRPC server coordinates receiving an RPC request, sending the request to the Storm topology, receiving the results from the Storm topology, and sending the results back to the waiting client. From a client's perspective, a distributed RPC call looks just like a regular RPC call. For example, here's how a client would compute the results for the "reach" function with the argument "<a href="http://twitter.com%22:">http://twitter.com":</a></p>
-<div class="highlight"><pre><code class="java language-java" data-lang="java"><span class="n">DRPCClient</span> <span class="n">client</span> <span class="o">=</span> <span class="k">new</span> <span class="n">DRPCClient</span><span class="o">(</span><span class="s">"drpc-host"</span><span class="o">,</span> <span class="mi">3772</span><span class="o">);</span>
+<div class="highlight"><pre><code class="language-java" data-lang="java"><span class="n">DRPCClient</span> <span class="n">client</span> <span class="o">=</span> <span class="k">new</span> <span class="nf">DRPCClient</span><span class="o">(</span><span class="s">"drpc-host"</span><span class="o">,</span> <span class="mi">3772</span><span class="o">);</span>
<span class="n">String</span> <span class="n">result</span> <span class="o">=</span> <span class="n">client</span><span class="o">.</span><span class="na">execute</span><span class="o">(</span><span class="s">"reach"</span><span class="o">,</span> <span class="s">"http://twitter.com"</span><span class="o">);</span>
</code></pre></div>
<p>The distributed RPC workflow looks like this:</p>
@@ -81,7 +89,7 @@
<p>A client sends the DRPC server the name of the function to execute and the arguments to that function. The topology implementing that function uses a <code>DRPCSpout</code> to receive a function invocation stream from the DRPC server. Each function invocation is tagged with a unique id by the DRPC server. The topology then computes the result and at the end of the topology a bolt called <code>ReturnResults</code> connects to the DRPC server and gives it the result for the function invocation id. The DRPC server then uses the id to match up that result with which client is waiting, unblocks the waiting client, and sends it the result.</p>
-<h3 id="toc_1">LinearDRPCTopologyBuilder</h3>
+<h3 id="lineardrpctopologybuilder">LinearDRPCTopologyBuilder</h3>
<p>Storm comes with a topology builder called <a href="/apidocs/backtype/storm/drpc/LinearDRPCTopologyBuilder.html">LinearDRPCTopologyBuilder</a> that automates almost all the steps involved for doing DRPC. These include:</p>
@@ -92,20 +100,20 @@
</ol>
<p>Let's look at a simple example. Here's the implementation of a DRPC topology that returns its input argument with a "!" appended:</p>
-<div class="highlight"><pre><code class="java language-java" data-lang="java"><span class="kd">public</span> <span class="kd">static</span> <span class="kd">class</span> <span class="nc">ExclaimBolt</span> <span class="kd">extends</span> <span class="n">BaseBasicBolt</span> <span class="o">{</span>
+<div class="highlight"><pre><code class="language-java" data-lang="java"><span class="kd">public</span> <span class="kd">static</span> <span class="kd">class</span> <span class="nc">ExclaimBolt</span> <span class="kd">extends</span> <span class="n">BaseBasicBolt</span> <span class="o">{</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">execute</span><span class="o">(</span><span class="n">Tuple</span> <span class="n">tuple</span><span class="o">,</span> <span class="n">BasicOutputCollector</span> <span class="n">collector</span><span class="o">)</span> <span class="o">{</span>
<span class="n">String</span> <span class="n">input</span> <span class="o">=</span> <span class="n">tuple</span><span class="o">.</span><span class="na">getString</span><span class="o">(</span><span class="mi">1</span><span class="o">);</span>
- <span class="n">collector</span><span class="o">.</span><span class="na">emit</span><span class="o">(</span><span class="k">new</span> <span class="n">Values</span><span class="o">(</span><span class="n">tuple</span><span class="o">.</span><span class="na">getValue</span><span class="o">(</span><span class="mi">0</span><span class="o">),</span> <span class="n">input</span> <span class="o">+</span> <span class="s">"!"</span><span class="o">));</span>
+ <span class="n">collector</span><span class="o">.</span><span class="na">emit</span><span class="o">(</span><span class="k">new</span> <span class="nf">Values</span><span class="o">(</span><span class="n">tuple</span><span class="o">.</span><span class="na">getValue</span><span class="o">(</span><span class="mi">0</span><span class="o">),</span> <span class="n">input</span> <span class="o">+</span> <span class="s">"!"</span><span class="o">));</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">declareOutputFields</span><span class="o">(</span><span class="n">OutputFieldsDeclarer</span> <span class="n">declarer</span><span class="o">)</span> <span class="o">{</span>
- <span class="n">declarer</span><span class="o">.</span><span class="na">declare</span><span class="o">(</span><span class="k">new</span> <span class="n">Fields</span><span class="o">(</span><span class="s">"id"</span><span class="o">,</span> <span class="s">"result"</span><span class="o">));</span>
+ <span class="n">declarer</span><span class="o">.</span><span class="na">declare</span><span class="o">(</span><span class="k">new</span> <span class="nf">Fields</span><span class="o">(</span><span class="s">"id"</span><span class="o">,</span> <span class="s">"result"</span><span class="o">));</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="kt">void</span> <span class="nf">main</span><span class="o">(</span><span class="n">String</span><span class="o">[]</span> <span class="n">args</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
- <span class="n">LinearDRPCTopologyBuilder</span> <span class="n">builder</span> <span class="o">=</span> <span class="k">new</span> <span class="n">LinearDRPCTopologyBuilder</span><span class="o">(</span><span class="s">"exclamation"</span><span class="o">);</span>
- <span class="n">builder</span><span class="o">.</span><span class="na">addBolt</span><span class="o">(</span><span class="k">new</span> <span class="n">ExclaimBolt</span><span class="o">(),</span> <span class="mi">3</span><span class="o">);</span>
+ <span class="n">LinearDRPCTopologyBuilder</span> <span class="n">builder</span> <span class="o">=</span> <span class="k">new</span> <span class="nf">LinearDRPCTopologyBuilder</span><span class="o">(</span><span class="s">"exclamation"</span><span class="o">);</span>
+ <span class="n">builder</span><span class="o">.</span><span class="na">addBolt</span><span class="o">(</span><span class="k">new</span> <span class="nf">ExclaimBolt</span><span class="o">(),</span> <span class="mi">3</span><span class="o">);</span>
<span class="c1">// ...</span>
<span class="o">}</span>
</code></pre></div>
@@ -113,11 +121,11 @@
<p>In this example, <code>ExclaimBolt</code> simply appends a "!" to the second field of the tuple. <code>LinearDRPCTopologyBuilder</code> handles the rest of the coordination of connecting to the DRPC server and sending results back.</p>
-<h3 id="toc_2">Local mode DRPC</h3>
+<h3 id="local-mode-drpc">Local mode DRPC</h3>
<p>DRPC can be run in local mode. Here's how to run the above example in local mode:</p>
-<div class="highlight"><pre><code class="java language-java" data-lang="java"><span class="n">LocalDRPC</span> <span class="n">drpc</span> <span class="o">=</span> <span class="k">new</span> <span class="n">LocalDRPC</span><span class="o">();</span>
-<span class="n">LocalCluster</span> <span class="n">cluster</span> <span class="o">=</span> <span class="k">new</span> <span class="n">LocalCluster</span><span class="o">();</span>
+<div class="highlight"><pre><code class="language-java" data-lang="java"><span class="n">LocalDRPC</span> <span class="n">drpc</span> <span class="o">=</span> <span class="k">new</span> <span class="nf">LocalDRPC</span><span class="o">();</span>
+<span class="n">LocalCluster</span> <span class="n">cluster</span> <span class="o">=</span> <span class="k">new</span> <span class="nf">LocalCluster</span><span class="o">();</span>
<span class="n">cluster</span><span class="o">.</span><span class="na">submitTopology</span><span class="o">(</span><span class="s">"drpc-demo"</span><span class="o">,</span> <span class="n">conf</span><span class="o">,</span> <span class="n">builder</span><span class="o">.</span><span class="na">createLocalTopology</span><span class="o">(</span><span class="n">drpc</span><span class="o">));</span>
@@ -130,7 +138,7 @@
<p>After launching the topology, you can do DRPC invocations using the <code>execute</code> method on <code>LocalDRPC</code>.</p>
-<h3 id="toc_3">Remote mode DRPC</h3>
+<h3 id="remote-mode-drpc">Remote mode DRPC</h3>
<p>Using DRPC on an actual cluster is also straightforward. There's three steps:</p>
@@ -141,19 +149,19 @@
</ol>
<p>Launching a DRPC server can be done with the <code>storm</code> script and is just like launching Nimbus or the UI:</p>
-<div class="highlight"><pre><code class="text language-text" data-lang="text">bin/storm drpc
+<div class="highlight"><pre><code class="language-text" data-lang="text">bin/storm drpc
</code></pre></div>
<p>Next, you need to configure your Storm cluster to know the locations of the DRPC server(s). This is how <code>DRPCSpout</code> knows from where to read function invocations. This can be done through the <code>storm.yaml</code> file or the topology configurations. Configuring this through the <code>storm.yaml</code> looks something like this:</p>
-<div class="highlight"><pre><code class="yaml language-yaml" data-lang="yaml"><span class="l-Scalar-Plain">drpc.servers</span><span class="p-Indicator">:</span>
+<div class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="l-Scalar-Plain">drpc.servers</span><span class="p-Indicator">:</span>
<span class="p-Indicator">-</span> <span class="s">"drpc1.foo.com"</span>
<span class="p-Indicator">-</span> <span class="s">"drpc2.foo.com"</span>
</code></pre></div>
<p>Finally, you launch DRPC topologies using <code>StormSubmitter</code> just like you launch any other topology. To run the above example in remote mode, you do something like this:</p>
-<div class="highlight"><pre><code class="java language-java" data-lang="java"><span class="n">StormSubmitter</span><span class="o">.</span><span class="na">submitTopology</span><span class="o">(</span><span class="s">"exclamation-drpc"</span><span class="o">,</span> <span class="n">conf</span><span class="o">,</span> <span class="n">builder</span><span class="o">.</span><span class="na">createRemoteTopology</span><span class="o">());</span>
+<div class="highlight"><pre><code class="language-java" data-lang="java"><span class="n">StormSubmitter</span><span class="o">.</span><span class="na">submitTopology</span><span class="o">(</span><span class="s">"exclamation-drpc"</span><span class="o">,</span> <span class="n">conf</span><span class="o">,</span> <span class="n">builder</span><span class="o">.</span><span class="na">createRemoteTopology</span><span class="o">());</span>
</code></pre></div>
<p><code>createRemoteTopology</code> is used to create topologies suitable for Storm clusters.</p>
-<h3 id="toc_4">A more complex example</h3>
+<h3 id="a-more-complex-example">A more complex example</h3>
<p>The exclamation DRPC example was a toy example for illustrating the concepts of DRPC. Let's look at a more complex example which really needs the parallelism a Storm cluster provides for computing the DRPC function. The example we'll look at is computing the reach of a URL on Twitter.</p>
@@ -168,15 +176,15 @@
<p>A single reach computation can involve thousands of database calls and tens of millions of follower records during the computation. It's a really, really intense computation. As you're about to see, implementing this function on top of Storm is dead simple. On a single machine, reach can take minutes to compute; on a Storm cluster, you can compute reach for even the hardest URLs in a couple seconds.</p>
-<p>A sample reach topology is defined in storm-starter <a href="https://github.com/nathanmarz/storm-starter/blob/master/src/jvm/storm/starter/ReachTopology.java">here</a>. Here's how you define the reach topology:</p>
-<div class="highlight"><pre><code class="java language-java" data-lang="java"><span class="n">LinearDRPCTopologyBuilder</span> <span class="n">builder</span> <span class="o">=</span> <span class="k">new</span> <span class="n">LinearDRPCTopologyBuilder</span><span class="o">(</span><span class="s">"reach"</span><span class="o">);</span>
-<span class="n">builder</span><span class="o">.</span><span class="na">addBolt</span><span class="o">(</span><span class="k">new</span> <span class="n">GetTweeters</span><span class="o">(),</span> <span class="mi">3</span><span class="o">);</span>
-<span class="n">builder</span><span class="o">.</span><span class="na">addBolt</span><span class="o">(</span><span class="k">new</span> <span class="n">GetFollowers</span><span class="o">(),</span> <span class="mi">12</span><span class="o">)</span>
+<p>A sample reach topology is defined in storm-starter <a href="https://github.com/apache/storm/blob/master/examples/storm-starter/src/jvm/storm/starter/ReachTopology.java">here</a>. Here's how you define the reach topology:</p>
+<div class="highlight"><pre><code class="language-java" data-lang="java"><span class="n">LinearDRPCTopologyBuilder</span> <span class="n">builder</span> <span class="o">=</span> <span class="k">new</span> <span class="nf">LinearDRPCTopologyBuilder</span><span class="o">(</span><span class="s">"reach"</span><span class="o">);</span>
+<span class="n">builder</span><span class="o">.</span><span class="na">addBolt</span><span class="o">(</span><span class="k">new</span> <span class="nf">GetTweeters</span><span class="o">(),</span> <span class="mi">3</span><span class="o">);</span>
+<span class="n">builder</span><span class="o">.</span><span class="na">addBolt</span><span class="o">(</span><span class="k">new</span> <span class="nf">GetFollowers</span><span class="o">(),</span> <span class="mi">12</span><span class="o">)</span>
<span class="o">.</span><span class="na">shuffleGrouping</span><span class="o">();</span>
-<span class="n">builder</span><span class="o">.</span><span class="na">addBolt</span><span class="o">(</span><span class="k">new</span> <span class="n">PartialUniquer</span><span class="o">(),</span> <span class="mi">6</span><span class="o">)</span>
- <span class="o">.</span><span class="na">fieldsGrouping</span><span class="o">(</span><span class="k">new</span> <span class="n">Fields</span><span class="o">(</span><span class="s">"id"</span><span class="o">,</span> <span class="s">"follower"</span><span class="o">));</span>
-<span class="n">builder</span><span class="o">.</span><span class="na">addBolt</span><span class="o">(</span><span class="k">new</span> <span class="n">CountAggregator</span><span class="o">(),</span> <span class="mi">2</span><span class="o">)</span>
- <span class="o">.</span><span class="na">fieldsGrouping</span><span class="o">(</span><span class="k">new</span> <span class="n">Fields</span><span class="o">(</span><span class="s">"id"</span><span class="o">));</span>
+<span class="n">builder</span><span class="o">.</span><span class="na">addBolt</span><span class="o">(</span><span class="k">new</span> <span class="nf">PartialUniquer</span><span class="o">(),</span> <span class="mi">6</span><span class="o">)</span>
+ <span class="o">.</span><span class="na">fieldsGrouping</span><span class="o">(</span><span class="k">new</span> <span class="nf">Fields</span><span class="o">(</span><span class="s">"id"</span><span class="o">,</span> <span class="s">"follower"</span><span class="o">));</span>
+<span class="n">builder</span><span class="o">.</span><span class="na">addBolt</span><span class="o">(</span><span class="k">new</span> <span class="nf">CountAggregator</span><span class="o">(),</span> <span class="mi">2</span><span class="o">)</span>
+ <span class="o">.</span><span class="na">fieldsGrouping</span><span class="o">(</span><span class="k">new</span> <span class="nf">Fields</span><span class="o">(</span><span class="s">"id"</span><span class="o">));</span>
</code></pre></div>
<p>The topology executes as four steps:</p>
@@ -188,7 +196,7 @@
</ol>
<p>Let's take a look at the <code>PartialUniquer</code> bolt:</p>
-<div class="highlight"><pre><code class="java language-java" data-lang="java"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">PartialUniquer</span> <span class="kd">extends</span> <span class="n">BaseBatchBolt</span> <span class="o">{</span>
+<div class="highlight"><pre><code class="language-java" data-lang="java"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">PartialUniquer</span> <span class="kd">extends</span> <span class="n">BaseBatchBolt</span> <span class="o">{</span>
<span class="n">BatchOutputCollector</span> <span class="n">_collector</span><span class="o">;</span>
<span class="n">Object</span> <span class="n">_id</span><span class="o">;</span>
<span class="n">Set</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">_followers</span> <span class="o">=</span> <span class="k">new</span> <span class="n">HashSet</span><span class="o"><</span><span class="n">String</span><span class="o">>();</span>
@@ -206,12 +214,12 @@
<span class="nd">@Override</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">finishBatch</span><span class="o">()</span> <span class="o">{</span>
- <span class="n">_collector</span><span class="o">.</span><span class="na">emit</span><span class="o">(</span><span class="k">new</span> <span class="n">Values</span><span class="o">(</span><span class="n">_id</span><span class="o">,</span> <span class="n">_followers</span><span class="o">.</span><span class="na">size</span><span class="o">()));</span>
+ <span class="n">_collector</span><span class="o">.</span><span class="na">emit</span><span class="o">(</span><span class="k">new</span> <span class="nf">Values</span><span class="o">(</span><span class="n">_id</span><span class="o">,</span> <span class="n">_followers</span><span class="o">.</span><span class="na">size</span><span class="o">()));</span>
<span class="o">}</span>
<span class="nd">@Override</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">declareOutputFields</span><span class="o">(</span><span class="n">OutputFieldsDeclarer</span> <span class="n">declarer</span><span class="o">)</span> <span class="o">{</span>
- <span class="n">declarer</span><span class="o">.</span><span class="na">declare</span><span class="o">(</span><span class="k">new</span> <span class="n">Fields</span><span class="o">(</span><span class="s">"id"</span><span class="o">,</span> <span class="s">"partial-count"</span><span class="o">));</span>
+ <span class="n">declarer</span><span class="o">.</span><span class="na">declare</span><span class="o">(</span><span class="k">new</span> <span class="nf">Fields</span><span class="o">(</span><span class="s">"id"</span><span class="o">,</span> <span class="s">"partial-count"</span><span class="o">));</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div>
@@ -225,11 +233,11 @@
<p>The rest of the topology should be self-explanatory. As you can see, every single step of the reach computation is done in parallel, and defining the DRPC topology was extremely simple.</p>
-<h3 id="toc_5">Non-linear DRPC topologies</h3>
+<h3 id="non-linear-drpc-topologies">Non-linear DRPC topologies</h3>
<p><code>LinearDRPCTopologyBuilder</code> only handles "linear" DRPC topologies, where the computation is expressed as a sequence of steps (like reach). It's not hard to imagine functions that would require a more complicated topology with branching and merging of the bolts. For now, to do this you'll need to drop down into using <code>CoordinatedBolt</code> directly. Be sure to talk about your use case for non-linear DRPC topologies on the mailing list to inform the construction of more general abstractions for DRPC topologies.</p>
-<h3 id="toc_6">How LinearDRPCTopologyBuilder works</h3>
+<h3 id="how-lineardrpctopologybuilder-works">How LinearDRPCTopologyBuilder works</h3>
<ul>
<li>DRPCSpout emits [args, return-info]. return-info is the host and port of the DRPC server as well as the id generated by the DRPC server</li>
@@ -245,21 +253,37 @@
<li>LinearDRPCTopologyBuilder is a good example of a higher level abstraction built on top of Storm's primitives</li>
</ul>
-<h3 id="toc_7">Advanced</h3>
+<h3 id="advanced">Advanced</h3>
<ul>
<li>KeyedFairBolt for weaving the processing of multiple requests at the same time</li>
<li>How to use <code>CoordinatedBolt</code> directly</li>
</ul>
+ </article>
+
+<!-- </div> -->
+
+
+
+ <hr/>
+<div id="footer" class="container text-center">
+
+ <p class="text-muted credit"><p>
+Copyright © 2014 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved. Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
+
</div>
-</div>
-<div id="clear"></div></div>
-<div id="footer">
- <p>
-Copyright © 2014 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved. Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
-</div>
-</div>
-
-</body>
-</html>
\ No newline at end of file
+
+ <!-- Bootstrap core JavaScript
+ ================================================== -->
+ <!-- Placed at the end of the document so the pages load faster -->
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>
+ <script src="/assets/js/bootstrap.min.js"></script>
+ <script src="/assets/js/docs.min.js"></script>
+ <!-- IE10 viewport hack for Surface/desktop Windows 8 bug -->
+ <script src="/assets/js/ie10-viewport-bug-workaround.js"></script>
+ </div>
+
+ </body>
+
+</html>
Modified: storm/site/publish/documentation/Documentation.html
URL: http://svn.apache.org/viewvc/storm/site/publish/documentation/Documentation.html?rev=1647601&r1=1647600&r2=1647601&view=diff
==============================================================================
--- storm/site/publish/documentation/Documentation.html (original)
+++ storm/site/publish/documentation/Documentation.html Tue Dec 23 15:59:17 2014
@@ -1,71 +1,79 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
-<head>
- <meta http-equiv="content-type" content="text/html;charset=utf-8" />
- <meta name="description" content="Storm is a distributed and fault-tolerant realtime computation system. Similar to how Hadoop provides a set of general primitives for doing batch processing, Storm provides a set of general primitives for doing realtime computation. Storm is simple, can be used with any programming language, and is a lot of fun to use!" />
- <meta name="keywords" content="storm, hadoop, realtime, stream, mapreduce, java, computation, scalability, clojure, scala, jvm, processing" />
- <title>Storm Documentation</title>
- <link rel="stylesheet" type="text/css" href="/css/style.css" media="screen" />
-<script type="text/javascript">
-
- var _gaq = _gaq || [];
- _gaq.push(['_setAccount', 'UA-32530285-1']);
- _gaq.push(['_trackPageview']);
-
- (function() {
- var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
- ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
- var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
- })();
-
-</script>
-</head>
-
-<body>
-<div id="wrap">
-<div id="top">
-<div id="projecttitle">
-<h2><a href="/" title="Back to main page">Storm</a></h2>
-<p id="slogan">Distributed and fault-tolerant realtime computation</p>
-</div>
-<div id="menu">
-<ul>
-<li><a href="/about/integrates.html">about</a></li>
-
-<!--
-<li><a href="/blog.html">blog</a></li>
--->
-
-<li><a href="/documentation/Home.html">documentation</a></li>
-<li><a href="/blog.html">blog</a></li>
-<li><a href="/downloads.html">downloads</a></li>
-<li><a href="/community.html">community</a></li>
-</ul>
-</div>
-</div>
-<div id="content">
-<div id="sidebar">
- <ul>
-
-<li><a class="" href="/documentation/Home.html">Documentation Home</a></li>
-
-<li><a class="" href="/documentation/Rationale.html">Rationale</a></li>
-
-<li><a class="" href="/documentation/Tutorial.html">Tutorial</a></li>
-
-<li><a class="current" href="/documentation/Documentation.html">Manual</a></li>
-
-<li><a class="" href="/apidocs/">Javadoc</a></li>
-
-<li><a class="" href="/documentation/FAQ.html">FAQ</a></li>
-
-<li><a class="" href="/documentation/Contributing-to-Storm.html">Contributing</a></li>
-
- </ul>
-</div>
-<div id="aboutcontent">
-<h3 id="toc_0">Basics of Storm</h3>
+<!DOCTYPE html>
+<html>
+
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <meta name="description" content="">
+ <meta name="author" content="">
+ <link rel="icon" href="/assets/favicon.ico">
+
+ <title>Documentation</title>
+
+ <!-- Bootstrap core CSS -->
+ <link href="/assets/css/bootstrap.min.css" rel="stylesheet">
+ <!-- Bootstrap theme -->
+ <link href="/assets/css/bootstrap-theme.min.css" rel="stylesheet">
+
+ <!-- Custom styles for this template -->
+ <link href="/assets/css/theme.css" rel="stylesheet">
+
+ <link href="/css/main.css" rel="stylesheet">
+
+ <!-- Just for debugging purposes. Don't actually copy these 2 lines! -->
+ <!--[if lt IE 9]><script src="../../assets/js/ie8-responsive-file-warning.js"></script><![endif]-->
+ <script src="/assets/js/ie-emulation-modes-warning.js"></script>
+
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ </head>
+
+
+
+ <body>
+ <div class="container">
+ <!-- Fixed navbar -->
+ <nav class="navbar navbar-default navbar-fixed-top" role="navigation">
+ <div class="container">
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a class="navbar-brand" href="/">Apache Storm<!-- <img alt="Apache Storm" src="/images/storm_logo_tagline_color.png"></a> -->
+ </div>
+ <div id="navbar" class="navbar-collapse collapse">
+ <ul class="nav navbar-nav">
+ <li><a href="/">Home</a></li>
+ <li><a href="/about/integrates.html">About</a></li>
+ <li><a href="/downloads.html">Download</a></li>
+ <li><a href="/documentation/Home.html">Documentation</a></li>
+ <li><a href="/news.html">News</a></li>
+ </ul>
+ </div><!--/.nav-collapse -->
+ </div>
+ </nav>
+
+
+
+
+
+ <!-- Documentation -->
+<!-- <div class="container"> -->
+
+ <header class="post-header">
+ <h1 class="post-title">Documentation</h1>
+ <p class="post-meta"></p>
+ </header>
+
+ <article class="post-content">
+ <h3 id="basics-of-storm">Basics of Storm</h3>
<ul>
<li><a href="http://nathanmarz.github.com/storm">Javadoc</a></li>
@@ -78,7 +86,7 @@
<li><a href="FAQ.html">FAQ</a></li>
</ul>
-<h3 id="toc_1">Trident</h3>
+<h3 id="trident">Trident</h3>
<p>Trident is an alternative interface to Storm. It provides exactly-once processing, "transactional" datastore persistence, and a set of common stream analytics operations.</p>
@@ -89,7 +97,7 @@
<li><a href="Trident-spouts.html">Trident spouts</a> -- transactional and non-transactional data intake</li>
</ul>
-<h3 id="toc_2">Setup and deploying</h3>
+<h3 id="setup-and-deploying">Setup and deploying</h3>
<ul>
<li><a href="Setting-up-a-Storm-cluster.html">Setting up a Storm cluster</a></li>
@@ -99,7 +107,7 @@
<li><a href="Maven.html">Building Storm</a> with Maven</li>
</ul>
-<h3 id="toc_3">Intermediate</h3>
+<h3 id="intermediate">Intermediate</h3>
<ul>
<li><a href="Serialization.html">Serialization</a></li>
@@ -115,7 +123,7 @@
<li><a href="">Lifecycle of a trident tuple</a></li>
</ul>
-<h3 id="toc_4">Advanced</h3>
+<h3 id="advanced">Advanced</h3>
<ul>
<li><a href="Defining-a-non-jvm-language-dsl-for-storm.html">Defining a non-JVM language DSL for Storm</a></li>
@@ -123,14 +131,30 @@
<li><a href="Implementation-docs.html">Implementation docs</a></li>
</ul>
+ </article>
+
+<!-- </div> -->
+
+
+
+ <hr/>
+<div id="footer" class="container text-center">
+
+ <p class="text-muted credit"><p>
+Copyright © 2014 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved. Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
+
</div>
-</div>
-<div id="clear"></div></div>
-<div id="footer">
- <p>
-Copyright © 2014 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved. Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
-</div>
-</div>
-
-</body>
-</html>
\ No newline at end of file
+
+ <!-- Bootstrap core JavaScript
+ ================================================== -->
+ <!-- Placed at the end of the document so the pages load faster -->
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>
+ <script src="/assets/js/bootstrap.min.js"></script>
+ <script src="/assets/js/docs.min.js"></script>
+ <!-- IE10 viewport hack for Surface/desktop Windows 8 bug -->
+ <script src="/assets/js/ie10-viewport-bug-workaround.js"></script>
+ </div>
+
+ </body>
+
+</html>