You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@storm.apache.org by bo...@apache.org on 2014/12/23 16:38:07 UTC
[09/15] storm git commit: Revert "Update repository link now that
Storm has graduated from Incubator"
Revert "Update repository link now that Storm has graduated from Incubator"
There're critical replacements which breaks link.
This reverts commit 39de7bc9d4a073c2c2caed12a21244fa0cc245e9.
Conflicts:
docs/about/multi-language.md
Project: http://git-wip-us.apache.org/repos/asf/storm/repo
Commit: http://git-wip-us.apache.org/repos/asf/storm/commit/e48f5050
Tree: http://git-wip-us.apache.org/repos/asf/storm/tree/e48f5050
Diff: http://git-wip-us.apache.org/repos/asf/storm/diff/e48f5050
Branch: refs/heads/master
Commit: e48f5050399bc2d673ba09c64c5195c93419f5fe
Parents: 0bacffb
Author: Jungtaek Lim <ka...@gmail.com>
Authored: Sat Dec 6 13:12:12 2014 +0900
Committer: Jungtaek Lim <ka...@gmail.com>
Committed: Sat Dec 6 13:12:12 2014 +0900
----------------------------------------------------------------------
docs/about/integrates.md | 2 +-
docs/about/multi-language.md | 6 +-
docs/about/simple-api.md | 2 +-
.../Acking-framework-implementation.md | 8 +--
docs/documentation/Clojure-DSL.md | 2 +-
docs/documentation/Configuration.md | 4 +-
...Defining-a-non-jvm-language-dsl-for-storm.md | 2 +-
docs/documentation/FAQ.md | 4 +-
docs/documentation/Kestrel-and-Storm.md | 2 +-
docs/documentation/Lifecycle-of-a-topology.md | 67 ++++++++++--------
.../Serialization-(prior-to-0.6.0).md | 2 +-
.../documentation/Setting-up-a-Storm-cluster.md | 2 +-
.../Setting-up-development-environment.md | 2 +-
docs/documentation/Structure-of-the-codebase.md | 72 +++++++++++---------
docs/documentation/Transactional-topologies.md | 2 +-
docs/documentation/Trident-API-Overview.md | 2 +-
docs/documentation/Trident-spouts.md | 6 +-
docs/documentation/Trident-state.md | 6 +-
docs/downloads.html | 2 +-
19 files changed, 106 insertions(+), 89 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/storm/blob/e48f5050/docs/about/integrates.md
----------------------------------------------------------------------
diff --git a/docs/about/integrates.md b/docs/about/integrates.md
index 8071bdb..bd9a236 100644
--- a/docs/about/integrates.md
+++ b/docs/about/integrates.md
@@ -6,7 +6,7 @@ Storm integrates with any queueing system and any database system. Storm's [spou
1. [Kestrel](https://github.com/nathanmarz/storm-kestrel)
2. [RabbitMQ / AMQP](https://github.com/Xorlev/storm-amqp-spout)
-3. [Kafka](https://github.com/apache/storm/tree/master/external/storm-kafka)
+3. [Kafka](https://github.com/apache/incubator-storm/tree/master/external/storm-kafka)
4. [JMS](https://github.com/ptgoetz/storm-jms)
5. [Amazon Kinesis](https://github.com/awslabs/kinesis-storm-spout)
http://git-wip-us.apache.org/repos/asf/storm/blob/e48f5050/docs/about/multi-language.md
----------------------------------------------------------------------
diff --git a/docs/about/multi-language.md b/docs/about/multi-language.md
index dbf9d70..d431b56 100644
--- a/docs/about/multi-language.md
+++ b/docs/about/multi-language.md
@@ -2,8 +2,8 @@
layout: about
---
-Storm was designed from the ground up to be usable with any programming language. At the core of Storm is a [Thrift](http://thrift.apache.org/) [definition](https://github.com/apache/storm/blob/master/storm-core/src/storm.thrift) for defining and submitting topologies. Since Thrift can be used in any language, topologies can be defined and submitted from any language.
+Storm was designed from the ground up to be usable with any programming language. At the core of Storm is a [Thrift](http://thrift.apache.org/) [definition](https://github.com/apache/incubator-storm/blob/master/storm-core/src/storm.thrift) for defining and submitting topologies. Since Thrift can be used in any language, topologies can be defined and submitted from any language.
-Similarly, spouts and bolts can be defined in any language. Non-JVM spouts and bolts communicate to Storm over a [JSON-based protocol](/documentation/Multilang-protocol.html) over stdin/stdout. Adapters that implement this protocol exist for [Ruby](https://gihttps://github.com/apache/stormer/storm-core/src/multilang/rb/storm.rb), [Python](https://github.com/apache/storm/blob/master/storm-core/src/multilang/py/storm.py), [Javascript](https://github.com/apache/storm/blob/master/storm-core/src/multilang/js/storm.js), [Perl](https://github.com/dan-blanchard/io-storm).
+Similarly, spouts and bolts can be defined in any language. Non-JVM spouts and bolts communicate to Storm over a [JSON-based protocol](/documentation/Multilang-protocol.html) over stdin/stdout. Adapters that implement this protocol exist for [Ruby](https://github.com/apache/incubator-storm/blob/master/storm-core/src/multilang/rb/storm.rb), [Python](https://github.com/apache/incubator-storm/blob/master/storm-core/src/multilang/py/storm.py), [Javascript](https://github.com/apache/storm/blob/master/storm-core/src/multilang/js/storm.js), [Perl](https://github.com/dan-blanchard/io-storm).
-*storm-starter* has an [example topology](https://github.com/apache/storm/blob/master/examples/storm-starter/src/jvm/storm/starter/WordCountTopology.java) that implements one of the bolts in Python.
\ No newline at end of file
+*storm-starter* has an [example topology](https://github.com/apache/storm/blob/master/examples/storm-starter/src/jvm/storm/starter/WordCountTopology.java) that implements one of the bolts in Python.
http://git-wip-us.apache.org/repos/asf/storm/blob/e48f5050/docs/about/simple-api.md
----------------------------------------------------------------------
diff --git a/docs/about/simple-api.md b/docs/about/simple-api.md
index 7fc8c10..4aee6d5 100644
--- a/docs/about/simple-api.md
+++ b/docs/about/simple-api.md
@@ -12,4 +12,4 @@ A **topology** is a network of spouts and bolts, with each edge in the network r
Storm has a "local mode" where a Storm cluster is simulated in-process. This is useful for development and testing. The "storm" command line client is used when ready to submit a topology for execution on an actual cluster.
-The [storm-starter](https://github.com/apache/storm/tree/master/examples/storm-starter) project contains example topologies for learning the basics of Storm. Learn more about how to use Storm by reading the [tutorial](/documentation/Tutorial.html) and the [documentation](/documentation/Home.html).
\ No newline at end of file
+The [storm-starter](https://github.com/apache/incubator-storm/tree/master/examples/storm-starter) project contains example topologies for learning the basics of Storm. Learn more about how to use Storm by reading the [tutorial](/documentation/Tutorial.html) and the [documentation](/documentation/Home.html).
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/storm/blob/e48f5050/docs/documentation/Acking-framework-implementation.md
----------------------------------------------------------------------
diff --git a/docs/documentation/Acking-framework-implementation.md b/docs/documentation/Acking-framework-implementation.md
index 4099219..75cef3c 100644
--- a/docs/documentation/Acking-framework-implementation.md
+++ b/docs/documentation/Acking-framework-implementation.md
@@ -3,13 +3,13 @@ title: Acking Framework Implementation
layout: documentation
documentation: true
---
-[Storm's acker](https://github.com/apache/storm/blob/46c3ba7/storm-core/src/clj/backtype/storm/daemon/acker.clj#L28) tracks completion of each tupletree with a checksum hash: each time a tuple is sent, its value is XORed into the checksum, and each time a tuple is acked its value is XORed in again. If all tuples have been successfully acked, the checksum will be zero (the odds that the checksum will be zero otherwise are vanishingly small).
+[Storm's acker](https://github.com/apache/incubator-storm/blob/46c3ba7/storm-core/src/clj/backtype/storm/daemon/acker.clj#L28) tracks completion of each tupletree with a checksum hash: each time a tuple is sent, its value is XORed into the checksum, and each time a tuple is acked its value is XORed in again. If all tuples have been successfully acked, the checksum will be zero (the odds that the checksum will be zero otherwise are vanishingly small).
You can read a bit more about the [reliability mechanism](Guaranteeing-message-processing.html#what-is-storms-reliability-api) elsewhere on the wiki -- this explains the internal details.
### acker `execute()`
-The acker is actually a regular bolt, with its [execute method](https://github.com/apache/storm/blob/46c3ba7/storm-core/src/clj/backtype/storm/daemon/acker.clj#L36) defined withing `mk-acker-bolt`. When a new tupletree is born, the spout sends the XORed edge-ids of each tuple recipient, which the acker records in its `pending` ledger. Every time an executor acks a tuple, the acker receives a partial checksum that is the XOR of the tuple's own edge-id (clearing it from the ledger) and the edge-id of each downstream tuple the executor emitted (thus entering them into the ledger).
+The acker is actually a regular bolt, with its [execute method](https://github.com/apache/incubator-storm/blob/46c3ba7/storm-core/src/clj/backtype/storm/daemon/acker.clj#L36) defined withing `mk-acker-bolt`. When a new tupletree is born, the spout sends the XORed edge-ids of each tuple recipient, which the acker records in its `pending` ledger. Every time an executor acks a tuple, the acker receives a partial checksum that is the XOR of the tuple's own edge-id (clearing it from the ledger) and the edge-id of each downstream tuple the executor emitted (thus entering them into the ledger).
This is accomplished as follows.
@@ -19,7 +19,7 @@ On a tick tuple, just advance pending tupletree checksums towards death and retu
* on ack: xor the partial checksum into the existing checksum value
* on fail: just mark it as failed
-Next, [put the record](https://github.com/apache/storm/blob/46c3ba7/storm-core/src/clj/backtype/storm/daemon/acker.clj#L50)), into the RotatingMap (thus resetting is countdown to expiry) and take action:
+Next, [put the record](https://github.com/apache/incubator-storm/blob/46c3ba7/storm-core/src/clj/backtype/storm/daemon/acker.clj#L50)), into the RotatingMap (thus resetting is countdown to expiry) and take action:
* if the total checksum is zero, the tupletree is complete: remove it from the pending collection and notify the spout of success
* if the tupletree has failed, it is also complete: remove it from the pending collection and notify the spout of failure
@@ -28,7 +28,7 @@ Finally, pass on an ack of our own.
### Pending tuples and the `RotatingMap`
-The acker stores pending tuples in a [`RotatingMap`](https://github.com/apache/storm/blob/master/storm-core/src/jvm/backtype/storm/utils/RotatingMap.java#L19), a simple device used in several places within Storm to efficiently time-expire a process.
+The acker stores pending tuples in a [`RotatingMap`](https://github.com/apache/incubator-storm/blob/master/storm-core/src/jvm/backtype/storm/utils/RotatingMap.java#L19), a simple device used in several places within Storm to efficiently time-expire a process.
The RotatingMap behaves as a HashMap, and offers the same O(1) access guarantees.
http://git-wip-us.apache.org/repos/asf/storm/blob/e48f5050/docs/documentation/Clojure-DSL.md
----------------------------------------------------------------------
diff --git a/docs/documentation/Clojure-DSL.md b/docs/documentation/Clojure-DSL.md
index 5c8b65f..c75c5df 100644
--- a/docs/documentation/Clojure-DSL.md
+++ b/docs/documentation/Clojure-DSL.md
@@ -3,7 +3,7 @@ title: Clojure DSL
layout: documentation
documentation: true
---
-Storm comes with a Clojure DSL for defining spouts, bolts, and topologies. The Clojure DSL has access to everything the Java API exposes, so if you're a Clojure user you can code Storm topologies without touching Java at all. The Clojure DSL is defined in the source in the [backtype.storm.clojure](https://github.com/apache/storm/blob/0.5.3/src/clj/backtype/storm/clojure.clj) namespace.
+Storm comes with a Clojure DSL for defining spouts, bolts, and topologies. The Clojure DSL has access to everything the Java API exposes, so if you're a Clojure user you can code Storm topologies without touching Java at all. The Clojure DSL is defined in the source in the [backtype.storm.clojure](https://github.com/apache/incubator-storm/blob/0.5.3/src/clj/backtype/storm/clojure.clj) namespace.
This page outlines all the pieces of the Clojure DSL, including:
http://git-wip-us.apache.org/repos/asf/storm/blob/e48f5050/docs/documentation/Configuration.md
----------------------------------------------------------------------
diff --git a/docs/documentation/Configuration.md b/docs/documentation/Configuration.md
index 01c136e..8738aee 100644
--- a/docs/documentation/Configuration.md
+++ b/docs/documentation/Configuration.md
@@ -5,7 +5,7 @@ documentation: true
---
Storm has a variety of configurations for tweaking the behavior of nimbus, supervisors, and running topologies. Some configurations are system configurations and cannot be modified on a topology by topology basis, whereas other configurations can be modified per topology.
-Every configuration has a default value defined in [defaults.yaml](https://github.com/apache/storm/blob/master/conf/defaults.yaml) in the Storm codebase. You can override these configurations by defining a storm.yaml in the classpath of Nimbus and the supervisors. Finally, you can define a topology-specific configuration that you submit along with your topology when using [StormSubmitter](/apidocs/backtype/storm/StormSubmitter.html). However, the topology-specific configuration can only override configs prefixed with "TOPOLOGY".
+Every configuration has a default value defined in [defaults.yaml](https://github.com/apache/incubator-storm/blob/master/conf/defaults.yaml) in the Storm codebase. You can override these configurations by defining a storm.yaml in the classpath of Nimbus and the supervisors. Finally, you can define a topology-specific configuration that you submit along with your topology when using [StormSubmitter](/apidocs/backtype/storm/StormSubmitter.html). However, the topology-specific configuration can only override configs prefixed with "TOPOLOGY".
Storm 0.7.0 and onwards lets you override configuration on a per-bolt/per-spout basis. The only configurations that can be overriden this way are:
@@ -25,7 +25,7 @@ The preference order for configuration values is defaults.yaml < storm.yaml < to
**Resources:**
* [Config](/apidocs/backtype/storm/Config.html): a listing of all configurations as well as a helper class for creating topology specific configurations
-* [defaults.yaml](https://github.com/apache/storm/blob/master/conf/defaults.yaml): the default values for all configurations
+* [defaults.yaml](https://github.com/apache/incubator-storm/blob/master/conf/defaults.yaml): the default values for all configurations
* [Setting up a Storm cluster](Setting-up-a-Storm-cluster.html): explains how to create and configure a Storm cluster
* [Running topologies on a production cluster](Running-topologies-on-a-production-cluster.html): lists useful configurations when running topologies on a cluster
* [Local mode](Local-mode.html): lists useful configurations when using local mode
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/storm/blob/e48f5050/docs/documentation/Defining-a-non-jvm-language-dsl-for-storm.md
----------------------------------------------------------------------
diff --git a/docs/documentation/Defining-a-non-jvm-language-dsl-for-storm.md b/docs/documentation/Defining-a-non-jvm-language-dsl-for-storm.md
index 77eb392..b01a8b5 100644
--- a/docs/documentation/Defining-a-non-jvm-language-dsl-for-storm.md
+++ b/docs/documentation/Defining-a-non-jvm-language-dsl-for-storm.md
@@ -3,7 +3,7 @@ title: Defining a Non-JVM DSL for Storm
layout: documentation
documentation: true
---
-The right place to start to learn how to make a non-JVM DSL for Storm is [storm-core/src/storm.thrift](https://github.com/apache/storm/blob/master/storm-core/src/storm.thrift). Since Storm topologies are just Thrift structures, and Nimbus is a Thrift daemon, you can create and submit topologies in any language.
+The right place to start to learn how to make a non-JVM DSL for Storm is [storm-core/src/storm.thrift](https://github.com/apache/incubator-storm/blob/master/storm-core/src/storm.thrift). Since Storm topologies are just Thrift structures, and Nimbus is a Thrift daemon, you can create and submit topologies in any language.
When you create the Thrift structs for spouts and bolts, the code for the spout or bolt is specified in the ComponentObject struct:
http://git-wip-us.apache.org/repos/asf/storm/blob/e48f5050/docs/documentation/FAQ.md
----------------------------------------------------------------------
diff --git a/docs/documentation/FAQ.md b/docs/documentation/FAQ.md
index b292b2f..b193256 100644
--- a/docs/documentation/FAQ.md
+++ b/docs/documentation/FAQ.md
@@ -74,11 +74,11 @@ You should only store static data, and as little of it as possible, into the met
### How often is the 'emitPartitionBatchNew' function called?
-Since the MBC is the actual spout, all the tuples in a batch are just members of its tupletree. That means storm's "max spout pending" config effectively defines the number of concurrent batches trident runs. The MBC emits a new batch if it has fewer than max-spending tuples pending and if at least one [trident batch interval](https://github.com/apache/storm/blob/master/conf/defaults.yaml#L115)'s worth of seconds has passed since the last batch.
+Since the MBC is the actual spout, all the tuples in a batch are just members of its tupletree. That means storm's "max spout pending" config effectively defines the number of concurrent batches trident runs. The MBC emits a new batch if it has fewer than max-spending tuples pending and if at least one [trident batch interval](https://github.com/apache/incubator-storm/blob/master/conf/defaults.yaml#L115)'s worth of seconds has passed since the last batch.
### If nothing was emitted does Trident slow down the calls?
-Yes, there's a pluggable "spout wait strategy"; the default is to sleep for a [configurable amount of time](https://github.com/apache/storm/blob/master/conf/defaults.yaml#L110)
+Yes, there's a pluggable "spout wait strategy"; the default is to sleep for a [configurable amount of time](https://github.com/apache/incubator-storm/blob/master/conf/defaults.yaml#L110)
### OK, then what is the trident batch interval for?
http://git-wip-us.apache.org/repos/asf/storm/blob/e48f5050/docs/documentation/Kestrel-and-Storm.md
----------------------------------------------------------------------
diff --git a/docs/documentation/Kestrel-and-Storm.md b/docs/documentation/Kestrel-and-Storm.md
index 9191018..26492d5 100644
--- a/docs/documentation/Kestrel-and-Storm.md
+++ b/docs/documentation/Kestrel-and-Storm.md
@@ -7,7 +7,7 @@ This page explains how to use to Storm to consume items from a Kestrel cluster.
## Preliminaries
### Storm
-This tutorial uses examples from the [storm-kestrel](https://github.com/nathanmarz/storm-kestrel) project and the [storm-starter](http://github.com/apache/storm/blob/master/examples/storm-starter) project. It's recommended that you clone those projects and follow along with the examples. Read [Setting up development environment](https://github.com/apache/storm/wiki/Setting-up-development-environment) and [Creating a new Storm project](https://github.com/apache/incubator-storm/wiki/Creating-a-new-Storm-project) to get your machine set up.
+This tutorial uses examples from the [storm-kestrel](https://github.com/nathanmarz/storm-kestrel) project and the [storm-starter](http://github.com/apache/storm/blob/master/examples/storm-starter) project. It's recommended that you clone those projects and follow along with the examples. Read [Setting up development environment](https://github.com/apache/incubator-storm/wiki/Setting-up-development-environment) and [Creating a new Storm project](https://github.com/apache/incubator-storm/wiki/Creating-a-new-Storm-project) to get your machine set up.
### Kestrel
It assumes you are able to run locally a Kestrel server as described [here](https://github.com/nathanmarz/storm-kestrel).
http://git-wip-us.apache.org/repos/asf/storm/blob/e48f5050/docs/documentation/Lifecycle-of-a-topology.md
----------------------------------------------------------------------
diff --git a/docs/documentation/Lifecycle-of-a-topology.md b/docs/documentation/Lifecycle-of-a-topology.md
index af535b2..e9f47a8 100644
--- a/docs/documentation/Lifecycle-of-a-topology.md
+++ b/docs/documentation/Lifecycle-of-a-topology.md
@@ -9,65 +9,74 @@ This page explains in detail the lifecycle of a topology from running the "storm
First a couple of important notes about topologies:
-1. The actual topology that runs is different than the topology the user specifies. The actual topology has implicit streams and an implicit "acker" bolt added to manage the acking framework (used to guarantee data processing). The implicit topology is created via the [system-topology!](https://github.com/apache/storm/blob/0.7.1/src/clj/backtype/storm/daemon/common.clj#L188) function.
+1. The actual topology that runs is different than the topology the user specifies. The actual topology has implicit streams and an implicit "acker" bolt added to manage the acking framework (used to guarantee data processing). The implicit topology is created via the [system-topology!](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/common.clj#L188) function.
2. `system-topology!` is used in two places:
- - when Nimbus is creating tasks for the topology [code](https://gihttps://github.com/apache/storm1/src/clj/backtype/storm/daemon/nimbus.clj#L316)
- - in the worker so it knows where it needs to route messages to [code](https://github.com/ahttps://github.com/apache/stormbacktype/storm/daemon/worker.clj#L90)
+ - when Nimbus is creating tasks for the topology [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L316)
+ - in the worker so it knows where it needs to route messages to [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/worker.clj#L90)
## Starting a topology
-- "storm jar" command executes your class with the specified arguments. The only special thing that "storm jar" does is set the "storm.jar" environment variable for use by `StormSubmitter` later. [code](https://github.com/apache/incuhttps://github.com/apache/stormWhen your code uses `StormSubmitter.submitTopology`, `StormSubmitter` takes the following actions:
- - First, `StormSubmitter` uploads the jar if it hasn't been uploaded before. [code](https://github.com/apache/incubator-storhttps://github.com/apache/stormSubmitter.java#L83)
- - Jar uploading is done via Nimbus's Thrift interface [code](https://github.com/apache/incubator-storm/blob/0.7https://github.com/apache/stormeUpload` returns a path in Nimbus's inbox
+- "storm jar" command executes your class with the specified arguments. The only special thing that "storm jar" does is set the "storm.jar" environment variable for use by `StormSubmitter` later. [code](https://github.com/apache/incubator-storm/blob/0.7.1/bin/storm#L101)
+- When your code uses `StormSubmitter.submitTopology`, `StormSubmitter` takes the following actions:
+ - First, `StormSubmitter` uploads the jar if it hasn't been uploaded before. [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/jvm/backtype/storm/StormSubmitter.java#L83)
+ - Jar uploading is done via Nimbus's Thrift interface [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/storm.thrift#L200)
+ - `beginFileUpload` returns a path in Nimbus's inbox
- 15 kilobytes are uploaded at a time through `uploadChunk`
- `finishFileUpload` is called when it's finished uploading
- - Here is Nimbus's implementation of those Thrift methods: [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/cljhttps://github.com/apache/storm - Second, `StormSubmitter` calls `submitTopology` on the Nimbus thrift interface [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/jvm/backtype/https://github.com/apache/stormtopology config is serialized using JSON (JSON is used so that writing DSL's in any language is as easy as possible)
+ - Here is Nimbus's implementation of those Thrift methods: [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L694)
+ - Second, `StormSubmitter` calls `submitTopology` on the Nimbus thrift interface [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/jvm/backtype/storm/StormSubmitter.java#L60)
+ - The topology config is serialized using JSON (JSON is used so that writing DSL's in any language is as easy as possible)
- Notice that the Thrift `submitTopology` call takes in the Nimbus inbox path where the jar was uploaded
-- Nimbus receives the topology submission. [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemhttps://github.com/apache/stormhe topology configuration. The main purpose of normalization is to ensure that every single task will have the same serialization registrations, which is critical for getting serialization working correctly. [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.https://github.com/apache/stormate for the topology [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L661)
-https://github.com/apache/stormfilesystem because they're too big for Zookeeper. The jar and configs are copied into the path {nimbus local dir}/stormdist/{topology id}
+- Nimbus receives the topology submission. [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L639)
+- Nimbus normalizes the topology configuration. The main purpose of normalization is to ensure that every single task will have the same serialization registrations, which is critical for getting serialization working correctly. [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L557)
+- Nimbus sets up the static state for the topology [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L661)
+ - Jars and configs are kept on local filesystem because they're too big for Zookeeper. The jar and configs are copied into the path {nimbus local dir}/stormdist/{topology id}
- `setup-storm-static` writes task -> component mapping into ZK
- `setup-heartbeats` creates a ZK "directory" in which tasks can heartbeat
- Nimbus calls `mk-assignment` to assign tasks to machines [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L458)
- - Asshttps://github.com/apache/storm(https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/common.clj#L25)
- - Assignment conthttps://github.com/apache/stormsupervisors to download the correct jars/configs for the topology from Nimbus
+ - Assignment record definition is here: [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/common.clj#L25)
+ - Assignment contains:
+ - `master-code-dir`: used by supervisors to download the correct jars/configs for the topology from Nimbus
- `task->node+port`: Map from a task id to the worker that task should be running on. (A worker is identified by a node/port pair)
- `node->host`: A map from node id to hostname. This is used so workers know which machines to connect to to communicate with other workers. Node ids are used to identify supervisors so that multiple supervisors can be run on one machine. One place this is done is with Mesos integration.
- `task->start-time-secs`: Contains a map from task id to the timestamp at which Nimbus launched that task. This is used by Nimbus when monitoring topologies, as tasks are given a longer timeout to heartbeat when they're first launched (the launch timeout is configured by "nimbus.task.launch.secs" config)
- Once topologies are assigned, they're initially in a deactivated mode. `start-storm` writes data into Zookeeper so that the cluster knows the topology is active and can start emitting tuples from spouts. [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L504)
-- TODO cluster state diagram https://github.com/apache/storme)
+- TODO cluster state diagram (show all nodes and what's kept everywhere)
- Supervisor runs two functions in the background:
- `synchronize-supervisor`: This is called whenever assignments in Zookeeper change and also every 10 seconds. [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/supervisor.clj#L241)
- - Downloads code from Nimbus fhttps://github.com/apache/stormr which it doesn't have the code yet. [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/supervisor.clj#L258)
- - Writes into local filesystem what thishttps://github.com/apache/storms a map from port -> LocalAssignment. LocalAssignment contains a topology id as well as the list of task ids for that worker. [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/supervisor.clj#L13)
- - `sync-processes`: Reads from the LFS what `synchronhttps://github.com/apache/stormo what's actually running on the machine. It then starts/stops worker processes as necessary to synchronize. [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/supervisor.clj#L177)
+ - Downloads code from Nimbus for topologies assigned to this machine for which it doesn't have the code yet. [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/supervisor.clj#L258)
+ - Writes into local filesystem what this node is supposed to be running. It writes a map from port -> LocalAssignment. LocalAssignment contains a topology id as well as the list of task ids for that worker. [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/supervisor.clj#L13)
+ - `sync-processes`: Reads from the LFS what `synchronize-supervisor` wrote and compares that to what's actually running on the machine. It then starts/stops worker processes as necessary to synchronize. [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/supervisor.clj#L177)
-- Worker processes start up through the `mk-worker` function https://github.com/apache/stormr-storm/blob/0.7.1/src/clj/backtype/storm/daemon/worker.clj#L67)
- - Worker connects to other workers and starts a thread to monitor for changes. https://github.com/apache/stormr will automatically reconnect to the other worker's new location. [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/worker.clj#L123)
- - Monitors whether a topology is active or not and stores that state in the `storm-activhttps://github.com/apache/stormy tasks to determine whether or not to call `nextTuple` on the spouts. [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/worker.clj#L155)
- - The worker launches the actual tasks as threads within it [code](https://github.com/apache/incubhttps://github.com/apache/stormorm/daemon/worker.clj#L178)
-- Tasks are set up through the `mk-task` function [code](https://github.com/apache/incubator-storm/blob/0.7.1/https://github.com/apache/storm60)
- - Tasks set up routing function which takes in a stream and an output tuple and returns a list of task ids to send the thttps://github.com/apache/stormincubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/task.clj#L207) (there's also a 3-arity version used for direct streams)
- - Tasks set up the spout-specific or bolt-specific code with [code](httpshttps://github.com/apache/storm/0.7.1/src/clj/backtype/storm/daemon/task.clj#L241)
+- Worker processes start up through the `mk-worker` function [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/worker.clj#L67)
+ - Worker connects to other workers and starts a thread to monitor for changes. So if a worker gets reassigned, the worker will automatically reconnect to the other worker's new location. [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/worker.clj#L123)
+ - Monitors whether a topology is active or not and stores that state in the `storm-active-atom` variable. This variable is used by tasks to determine whether or not to call `nextTuple` on the spouts. [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/worker.clj#L155)
+ - The worker launches the actual tasks as threads within it [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/worker.clj#L178)
+- Tasks are set up through the `mk-task` function [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/task.clj#L160)
+ - Tasks set up routing function which takes in a stream and an output tuple and returns a list of task ids to send the tuple to [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/task.clj#L207) (there's also a 3-arity version used for direct streams)
+ - Tasks set up the spout-specific or bolt-specific code with [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/task.clj#L241)
## Topology Monitoring
- Nimbus monitors the topology during its lifetime
- - Schedules recurring task on the timer thread to check the https://github.com/apache/stormhe/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L623)
- - Nimbus's behavior is represented as a finite state machine [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daehttps://github.com/apache/storment is called on a topology every "nimbus.monitor.freq.secs", which calls `reassign-topology` through `reassign-transition` [code](https://ghttps://github.com/apache/storm.1/src/clj/backtype/storm/daemon/nimbus.clj#L497)
- - `reassign-topology` calls `mk-assignments`, the same function used to assign the topology the first time. `mk-assignments` is also capable of incrementally updating https://github.com/apache/storms heartbeats and reassigns workers as necessary
+ - Schedules recurring task on the timer thread to check the topologies [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L623)
+ - Nimbus's behavior is represented as a finite state machine [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L98)
+ - The "monitor" event is called on a topology every "nimbus.monitor.freq.secs", which calls `reassign-topology` through `reassign-transition` [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L497)
+ - `reassign-topology` calls `mk-assignments`, the same function used to assign the topology the first time. `mk-assignments` is also capable of incrementally updating a topology
+ - `mk-assignments` checks heartbeats and reassigns workers as necessary
- Any reassignments change the state in ZK, which will trigger supervisors to synchronize and start/stop workers
## Killing a topology
- "storm kill" command runs this code which just calls the Nimbus Thrift interface to kill the topology: [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/command/kill_topology.clj)
- Nimbus receives the kill command [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L671)
-- Nimbus applies the "kill" transithttps://github.com/apache/storm.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L676)
-- The kill transition functihttps://github.com/apache/storm"killed" and schedules the "remove" event to run "wait time seconds" in the future. [code](https://github.com/apache/incubator-storhttps://github.com/apache/stormn/nimbus.clj#L63)
+- Nimbus applies the "kill" transition to the topology [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L676)
+- The kill transition function changes the status of the topology to "killed" and schedules the "remove" event to run "wait time seconds" in the future. [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L63)
- The wait time defaults to the topology message timeout but can be overridden with the -w flag in the "storm kill" command
- - This causes the topology to be deactivated for the wait time before its actuallhttps://github.com/apache/stormance to finish processing what it's currently processing before shutting down the workers
+ - This causes the topology to be deactivated for the wait time before its actually shut down. This gives the topology a chance to finish processing what it's currently processing before shutting down the workers
- Changing the status during the kill transition ensures that the kill protocol is fault-tolerant to Nimbus crashing. On startup, if the status of the topology is "killed", Nimbus schedules the remove event to run "wait time seconds" in the future [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L111)
- Removing a topology cleans out the assignment and static information from ZK [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L116)
-- A separate cleanup thread runs thttps://github.com/apache/storm up the heartbeat dir and the jars/configs stored locally. [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L577)
\ No newline at end of file
+- A separate cleanup thread runs the `do-cleanup` function which will clean up the heartbeat dir and the jars/configs stored locally. [code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L577)
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/storm/blob/e48f5050/docs/documentation/Serialization-(prior-to-0.6.0).md
----------------------------------------------------------------------
diff --git a/docs/documentation/Serialization-(prior-to-0.6.0).md b/docs/documentation/Serialization-(prior-to-0.6.0).md
index 71037a8..7b9047d 100644
--- a/docs/documentation/Serialization-(prior-to-0.6.0).md
+++ b/docs/documentation/Serialization-(prior-to-0.6.0).md
@@ -39,7 +39,7 @@ Storm uses the `accept` method to determine if a type can be serialized by this
`deserialize` reads the serialized object off of the stream and returns it.
-You can see example serialization implementations in the source for [SerializationFactory](https://github.com/apache/storm/blob/0.5.4/src/jvm/backtype/storm/serialization/SerializationFactory.java)
+You can see example serialization implementations in the source for [SerializationFactory](https://github.com/apache/incubator-storm/blob/0.5.4/src/jvm/backtype/storm/serialization/SerializationFactory.java)
#### Registering a serializer
http://git-wip-us.apache.org/repos/asf/storm/blob/e48f5050/docs/documentation/Setting-up-a-Storm-cluster.md
----------------------------------------------------------------------
diff --git a/docs/documentation/Setting-up-a-Storm-cluster.md b/docs/documentation/Setting-up-a-Storm-cluster.md
index 80b9b27..53a92f8 100644
--- a/docs/documentation/Setting-up-a-Storm-cluster.md
+++ b/docs/documentation/Setting-up-a-Storm-cluster.md
@@ -40,7 +40,7 @@ Next, download a Storm release and extract the zip file somewhere on Nimbus and
### Fill in mandatory configurations into storm.yaml
-The Storm release contains a file at `conf/storm.yaml` that configures the Storm daemons. You can see the default configuration values [here](https://github.com/apache/storm/blob/master/conf/defaults.yaml). storm.yaml overrides anything in defaults.yaml. There's a few configurations that are mandatory to get a working cluster:
+The Storm release contains a file at `conf/storm.yaml` that configures the Storm daemons. You can see the default configuration values [here](https://github.com/apache/incubator-storm/blob/master/conf/defaults.yaml). storm.yaml overrides anything in defaults.yaml. There's a few configurations that are mandatory to get a working cluster:
1) **storm.zookeeper.servers**: This is a list of the hosts in the Zookeeper cluster for your Storm cluster. It should look something like:
http://git-wip-us.apache.org/repos/asf/storm/blob/e48f5050/docs/documentation/Setting-up-development-environment.md
----------------------------------------------------------------------
diff --git a/docs/documentation/Setting-up-development-environment.md b/docs/documentation/Setting-up-development-environment.md
index 194ece3..cb49961 100644
--- a/docs/documentation/Setting-up-development-environment.md
+++ b/docs/documentation/Setting-up-development-environment.md
@@ -20,7 +20,7 @@ Let's quickly go over the relationship between your machine and a remote cluster
### Installing a Storm release locally
-If you want to be able to submit topologies to a remote cluster from your machine, you should install a Storm release locally. Installing a Storm release will give you the `storm` client that you can use to interact with remote clusters. To install Storm locally, download a release [from here](https://github.com/apache/storm/downloads) and unzip it somewhere on your computer. Then add the unpacked `bin/` directory onto your `PATH` and make sure the `bin/storm` script is executable.
+If you want to be able to submit topologies to a remote cluster from your machine, you should install a Storm release locally. Installing a Storm release will give you the `storm` client that you can use to interact with remote clusters. To install Storm locally, download a release [from here](https://github.com/apache/incubator-storm/downloads) and unzip it somewhere on your computer. Then add the unpacked `bin/` directory onto your `PATH` and make sure the `bin/storm` script is executable.
Installing a Storm release locally is only for interacting with remote clusters. For developing and testing topologies in local mode, it is recommended that you use Maven to include Storm as a dev dependency for your project. You can read more about using Maven for this purpose on [Maven](Maven.html).
http://git-wip-us.apache.org/repos/asf/storm/blob/e48f5050/docs/documentation/Structure-of-the-codebase.md
----------------------------------------------------------------------
diff --git a/docs/documentation/Structure-of-the-codebase.md b/docs/documentation/Structure-of-the-codebase.md
index 25d87b7..27333eb 100644
--- a/docs/documentation/Structure-of-the-codebase.md
+++ b/docs/documentation/Structure-of-the-codebase.md
@@ -15,18 +15,18 @@ The following sections explain each of these layers in more detail.
### storm.thrift
-The first place to look to understand the structure of Storm's codebase is the [storm.thrift](https://github.com/apache/storm/blob/master/storm-core/src/storm.thrift) file.
+The first place to look to understand the structure of Storm's codebase is the [storm.thrift](https://github.com/apache/incubator-storm/blob/master/storm-core/src/storm.thrift) file.
Storm uses [this fork](https://github.com/nathanmarz/thrift/tree/storm) of Thrift (branch 'storm') to produce the generated code. This "fork" is actually Thrift 7 with all the Java packages renamed to be `org.apache.thrift7`. Otherwise, it's identical to Thrift 7. This fork was done because of the lack of backwards compatibility in Thrift and the need for many people to use other versions of Thrift in their Storm topologies.
-Every spout or bolt in a topology is given a user-specified identifier called the "component id". The component id is used to specify subscriptions from a bolt to the output streams of other spouts or bolts. A [StormTopology](https://gihttps://github.com/apache/stormer/storm-core/src/storm.thrift#L91) structure contains a map from component id to component for each type of component (spouts and bolts).
+Every spout or bolt in a topology is given a user-specified identifier called the "component id". The component id is used to specify subscriptions from a bolt to the output streams of other spouts or bolts. A [StormTopology](https://github.com/apache/incubator-storm/blob/master/storm-core/src/storm.thrift#L91) structure contains a map from component id to component for each type of component (spouts and bolts).
-Spouts and bolts have the same Thrift definition, so let's just take a look at the [Thrift definition for bolts](https://github.com/ahttps://github.com/apache/stormore/src/storm.thrift#L79). It contains a `ComponentObject` struct and a `ComponentCommon` struct.
+Spouts and bolts have the same Thrift definition, so let's just take a look at the [Thrift definition for bolts](https://github.com/apache/incubator-storm/blob/master/storm-core/src/storm.thrift#L79). It contains a `ComponentObject` struct and a `ComponentCommon` struct.
The `ComponentObject` defines the implementation for the bolt. It can be one of three types:
-1. A serialized java object (that implements [IBolt](https://github.com/apache/incuhttps://github.com/apache/stormm/backtype/storm/task/IBolt.java))
-2. A `ShellComponent` object that indicates the implementation is in another language. Specifying a bolt this way will cause Storm to instantiate a [ShellBolt](https://github.com/apache/incubator-storhttps://github.com/apache/storm/storm/task/ShellBolt.java) object to handle the communication between the JVM-based worker process and the non-JVM-based implementation of the component.
+1. A serialized java object (that implements [IBolt](https://github.com/apache/incubator-storm/blob/master/storm-core/src/jvm/backtype/storm/task/IBolt.java))
+2. A `ShellComponent` object that indicates the implementation is in another language. Specifying a bolt this way will cause Storm to instantiate a [ShellBolt](https://github.com/apache/incubator-storm/blob/master/storm-core/src/jvm/backtype/storm/task/ShellBolt.java) object to handle the communication between the JVM-based worker process and the non-JVM-based implementation of the component.
3. A `JavaObject` structure which tells Storm the classname and constructor arguments to use to instantiate that bolt. This is useful if you want to define a topology in a non-JVM language. This way, you can make use of JVM-based spouts and bolts without having to create and serialize a Java object yourself.
`ComponentCommon` defines everything else for this component. This includes:
@@ -34,7 +34,9 @@ The `ComponentObject` defines the implementation for the bolt. It can be one of
1. What streams this component emits and the metadata for each stream (whether it's a direct stream, the fields declaration)
2. What streams this component consumes (specified as a map from component_id:stream_id to the stream grouping to use)
3. The parallelism for this component
-4. The component-specific [configuration](https://github.com/apache/incubator-storm/wiki/Conhttps://github.com/apache/storm the structure spouts also have a `ComponentCommon` field, and so spouts can also have declarations to consume other input streams. Yet the Storm Java API does not provide a way for spouts to consume other streams, and if you put any input declarations there for a spout you would get an error when you tried to submit the topology. The reason that spouts have an input declarations field is not for users to use, but for Storm itself to use. Storm adds implicit streams and bolts to the topology to set up the [acking framework](https://github.com/apache/incubator-storm/wiki/Guaranteeing-https://github.com/apache/stormlicit streams are from the acker bolt to each spout in the topology. The acker sends "ack" or "fail" messages along these streams whenever a tuple tree is detected to be completed or failed. The code that transforms the user's topology into the runtime
topology is located [here](https://github.com/apache/incubator-storm/blob/master/storm-core/src/chttps://github.com/apache/storm.
+4. The component-specific [configuration](https://github.com/apache/incubator-storm/wiki/Configuration) for this component
+
+Note that the structure spouts also have a `ComponentCommon` field, and so spouts can also have declarations to consume other input streams. Yet the Storm Java API does not provide a way for spouts to consume other streams, and if you put any input declarations there for a spout you would get an error when you tried to submit the topology. The reason that spouts have an input declarations field is not for users to use, but for Storm itself to use. Storm adds implicit streams and bolts to the topology to set up the [acking framework](https://github.com/apache/incubator-storm/wiki/Guaranteeing-message-processing), and two of these implicit streams are from the acker bolt to each spout in the topology. The acker sends "ack" or "fail" messages along these streams whenever a tuple tree is detected to be completed or failed. The code that transforms the user's topology into the runtime topology is located [here](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/back
type/storm/daemon/common.clj#L279).
### Java interfaces
@@ -53,81 +55,87 @@ You can see this strategy at work with the [BaseRichSpout](/apidocs/backtype/sto
Spouts and bolts are serialized into the Thrift definition of the topology as described above.
-One subtle aspect of the interfaces is the difference between `IBolt` and `ISpout` vs. `IRichBolt` and `IRichSpout`. The main difference between them is the addition of the `declareOutputFields` method in the "Rich" versions of the interfaces. The reason for the split is that the output fields declaration for each output stream needs to be part of the Thrift struct (so it can be specified from any language), but as a user you want to be able to declare the streams as part of your class. What `TopologyBuilder` does when constructing the Thrift representation is call `declareOutputFields` to get the declaration and convert it into the Thrift structure. The conversion happens [at this portion](https://github.com/apache/incubator-storm/blob/master/storm-core/src/jvm/backtyphttps://github.com/apache/storm5) of the `TopologyBuilder` code.
+One subtle aspect of the interfaces is the difference between `IBolt` and `ISpout` vs. `IRichBolt` and `IRichSpout`. The main difference between them is the addition of the `declareOutputFields` method in the "Rich" versions of the interfaces. The reason for the split is that the output fields declaration for each output stream needs to be part of the Thrift struct (so it can be specified from any language), but as a user you want to be able to declare the streams as part of your class. What `TopologyBuilder` does when constructing the Thrift representation is call `declareOutputFields` to get the declaration and convert it into the Thrift structure. The conversion happens [at this portion](https://github.com/apache/incubator-storm/blob/master/storm-core/src/jvm/backtype/storm/topology/TopologyBuilder.java#L205) of the `TopologyBuilder` code.
### Implementation
Specifying all the functionality via Java interfaces ensures that every feature of Storm is available via Java. Moreso, the focus on Java interfaces ensures that the user experience from Java-land is pleasant as well.
-The implementation of Storm, on the other hand, is primarily in Clojure. While the codebase is about 50% Java and 50% Clojure in terms of LOC, most of the implementation logic is in Clojure. There are two notable exceptions to this, and that is the [DRPC](https://github.com/apache/incubator-storm/wiki/Distributed-RPC) and [transactional topologhttps://github.com/apache/stormstorm/wiki/Transactional-topologies) implementations. These are https://github.com/apache/storm to serve as an illustration for how to implement a higher level abstraction on Storm. The DRPC and transactional topologies implementations are in the [backtype.storm.coordination](https://github.com/apache/incubator-storm/tree/master/storm-core/src/jvm/backtype/storm/coordination), [backtyhttps://github.com/apache/stormincubator-storm/tree/master/storm-core/src/jvm/backtype/storm/drpc), and [backtype.storm.transhttps://github.com/apache/stormator-storm/tree/master/storm-core/src/jvm/backtype/storm/transactional) packa
ges.
+The implementation of Storm, on the other hand, is primarily in Clojure. While the codebase is about 50% Java and 50% Clojure in terms of LOC, most of the implementation logic is in Clojure. There are two notable exceptions to this, and that is the [DRPC](https://github.com/apache/incubator-storm/wiki/Distributed-RPC) and [transactional topologies](https://github.com/apache/incubator-storm/wiki/Transactional-topologies) implementations. These are implemented purely in Java. This was done to serve as an illustration for how to implement a higher level abstraction on Storm. The DRPC and transactional topologies implementations are in the [backtype.storm.coordination](https://github.com/apache/incubator-storm/tree/master/storm-core/src/jvm/backtype/storm/coordination), [backtype.storm.drpc](https://github.com/apache/incubator-storm/tree/master/storm-core/src/jvm/backtype/storm/drpc), and [backtype.storm.transactional](https://github.com/apache/incubator-storm/tree/master/storm-core/src
/jvm/backtype/storm/transactional) packages.
-Here's a summaryhttps://github.com/apache/storm and Clojure namespace:
+Here's a summary of the purpose of the main Java packages and Clojure namespace:
#### Java packages
-[backtype.storm.coordination](https://github.com/apache/incubator-storm/tree/master/storm-core/src/jvm/backtype/storm/coordination): Implements the pieces required to coohttps://github.com/apache/storm which both DRPC and transactional topologies use. `CoordinatedBolt` is the most important class here.
+[backtype.storm.coordination](https://github.com/apache/incubator-storm/tree/master/storm-core/src/jvm/backtype/storm/coordination): Implements the pieces required to coordinate batch-processing on top of Storm, which both DRPC and transactional topologies use. `CoordinatedBolt` is the most important class here.
[backtype.storm.drpc](https://github.com/apache/incubator-storm/tree/master/storm-core/src/jvm/backtype/storm/drpc): Implementation of the DRPC higher level abstraction
-[bhttps://github.com/apache/stormom/apache/incubator-storm/tree/master/storm-core/src/jvm/backtype/storm/generated): The generated Thrift code for Storm (generated using [this fhttps://github.com/apache/storm) of Thrift, which simply renames the packages to org.apache.thrift7 to avoid conflicts with other Thrift versions)
+[backtype.storm.generated](https://github.com/apache/incubator-storm/tree/master/storm-core/src/jvm/backtype/storm/generated): The generated Thrift code for Storm (generated using [this fork](https://github.com/nathanmarz/thrift) of Thrift, which simply renames the packages to org.apache.thrift7 to avoid conflicts with other Thrift versions)
[backtype.storm.grouping](https://github.com/apache/incubator-storm/tree/master/storm-core/src/jvm/backtype/storm/grouping): Contains interface for making custom stream groupings
-[backtype.storm.https://github.com/apache/stormr-storm/tree/master/storm-core/src/jvm/backtype/storm/hooks): Interfaces for hooking into various events in Storm, such as when tasks emit tuples,https://github.com/apache/stormor hooks is [here](https://github.com/apache/incubator-storm/wiki/Hooks).
+[backtype.storm.hooks](https://github.com/apache/incubator-storm/tree/master/storm-core/src/jvm/backtype/storm/hooks): Interfaces for hooking into various events in Storm, such as when tasks emit tuples, when tuples are acked, etc. User guide for hooks is [here](https://github.com/apache/incubator-storm/wiki/Hooks).
+
+[backtype.storm.serialization](https://github.com/apache/incubator-storm/tree/master/storm-core/src/jvm/backtype/storm/serialization): Implementation of how Storm serializes/deserializes tuples. Built on top of [Kryo](http://code.google.com/p/kryo/).
-[backtype.storm.serialization](https://github.com/apache/incubator-storm/tree/master/storm-core/src/jvm/backtype/storm/serialization):https://github.com/apache/stormeserializes tuples. Built on top of [Kryo](http://code.ghttps://github.com/apache/stormt](https://github.com/apache/incubator-storm/tree/master/storm-core/src/jvm/backtype/storm/spout): Definition of spout and associated interfaces (like the `SpoutOutputCollector`). Also contains `ShellSpout` which https://github.com/apache/stormts in non-JVM languages.
+[backtype.storm.spout](https://github.com/apache/incubator-storm/tree/master/storm-core/src/jvm/backtype/storm/spout): Definition of spout and associated interfaces (like the `SpoutOutputCollector`). Also contains `ShellSpout` which implements the protocol for defining spouts in non-JVM languages.
-[backtype.storm.task](https://github.com/apache/incubator-storm/tree/master/storm-core/src/jvm/backtype/storm/task): Definition of bolt and associated interfaces (like `OutputCollector`). Also contains `ShellBolt` which implements the protocohttps://github.com/apache/storm. Finally, `TopologyContext` is defined here as well, which is provided to spouts and bolts so they can get data about the topology and its execution at runtime.
+[backtype.storm.task](https://github.com/apache/incubator-storm/tree/master/storm-core/src/jvm/backtype/storm/task): Definition of bolt and associated interfaces (like `OutputCollector`). Also contains `ShellBolt` which implements the protocol for defining bolts in non-JVM languages. Finally, `TopologyContext` is defined here as well, which is provided to spouts and bolts so they can get data about the topology and its execution at runtime.
[backtype.storm.testing](https://github.com/apache/incubator-storm/tree/master/storm-core/src/jvm/backtype/storm/testing): Contains a variety of test bolts and utilities used in Storm's unit tests.
-[backtype.storm.topology](https://github.com/apache/incuhttps://github.com/apache/stormm/backtype/storm/topology): Java layer over the underlying Thrift structure to provide a clean, pure-Java API to Storm (users don't have to know about Thrift). `Topologyhttps://github.com/apache/stormase classes for the different spouts and bolts. The slightly-higher level `IBasicBolt` interface is here, which is a simpler way to write certain kinds of bolts.
+[backtype.storm.topology](https://github.com/apache/incubator-storm/tree/master/storm-core/src/jvm/backtype/storm/topology): Java layer over the underlying Thrift structure to provide a clean, pure-Java API to Storm (users don't have to know about Thrift). `TopologyBuilder` is here as well as the helpful base classes for the different spouts and bolts. The slightly-higher level `IBasicBolt` interface is here, which is a simpler way to write certain kinds of bolts.
[backtype.storm.transactional](https://github.com/apache/incubator-storm/tree/master/storm-core/src/jvm/backtype/storm/transactional): Implementation of transactional topologies.
-[backtype.storm.tuple](https://github.com/apache/incubator-storm/tree/master/storm-core/src/jvm/backthttps://github.com/apache/storm's tuple data model.
+[backtype.storm.tuple](https://github.com/apache/incubator-storm/tree/master/storm-core/src/jvm/backtype/storm/tuple): Implementation of Storm's tuple data model.
-[backtype.storm.utils](https://github.com/apache/incubator-storm/tree/master/storm-core/src/jvm/backtype/storm/tuple): https://github.com/apache/stormes used throughout the codebase.
+[backtype.storm.utils](https://github.com/apache/incubator-storm/tree/master/storm-core/src/jvm/backtype/storm/tuple): Data structures and miscellaneous utilities used throughout the codebase.
#### Clojure namespaces
-[backtype.storm.bootstrap](https://github.com/apache/incubator-storm/blobhttps://github.com/apache/storm/bootstrap.clj): Contains a helpful macro to import all the classes and namespaces that are used throughout the codebase.
+[backtype.storm.bootstrap](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/bootstrap.clj): Contains a helpful macro to import all the classes and namespaces that are used throughout the codebase.
-[backtype.storm.clojure](https://github.com/apache/incubator-storm/blohttps://github.com/apache/stormm/clojure.clj): Implementation of the Clojure DSL for Storm.
+[backtype.storm.clojure](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/clojure.clj): Implementation of the Clojure DSL for Storm.
-[backtype.storm.cluster](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/cluster.clj): All Zookeeper lhttps://github.com/apache/stormd in this file. This code manages how cluster state (like what tasks are running where, what spout/bolt each task runs as) is mapped to the Zohttps://github.com/apache/stormm.command.*](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/command): These namespaces implement various commands for the `storm` command line client. These implementations are very short.
+[backtype.storm.cluster](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/cluster.clj): All Zookeeper logic used in Storm daemons is encapsulated in this file. This code manages how cluster state (like what tasks are running where, what spout/bolt each task runs as) is mapped to the Zookeeper "filesystem" API.
-[backtype.storm.config](https://github.com/apache/incubator-storm/blob/master/storm-core/srchttps://github.com/apache/stormtation of config reading/parsing code for Clojure. Also has utility functions for determining what local path nimbus/supervisor/daemons should be using for various things. e.g. the `master-inbox` function will rhttps://github.com/apache/stormse when jars are uploaded to it.
+[backtype.storm.command.*](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/command): These namespaces implement various commands for the `storm` command line client. These implementations are very short.
+
+[backtype.storm.config](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/config.clj): Implementation of config reading/parsing code for Clojure. Also has utility functions for determining what local path nimbus/supervisor/daemons should be using for various things. e.g. the `master-inbox` function will return the local path that Nimbus should use when jars are uploaded to it.
[backtype.storm.daemon.acker](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/daemon/acker.clj): Implementation of the "acker" bolt, which is a key part of how Storm guarantees data processing.
-[backtype.storm.daemon.common](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/daemonhttps://github.com/apache/stormnctions used in Storm daemons, like getting the id for a topology based on the name, mapping a user's topology into the one that actually executes (with implicit acking streams and acker bolt added - see `https://github.com/apache/stormons for the various heartbeat and other structures persisted by Storm.
+[backtype.storm.daemon.common](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/daemon/common.clj): Implementation of common functions used in Storm daemons, like getting the id for a topology based on the name, mapping a user's topology into the one that actually executes (with implicit acking streams and acker bolt added - see `system-topology!` function), and definitions for the various heartbeat and other structures persisted by Storm.
[backtype.storm.daemon.drpc](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/daemon/drpc.clj): Implementation of the DRPC server for use with DRPC topologies.
[backtype.storm.daemon.nimbus](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/daemon/nimbus.clj): Implementation of Nimbus.
-[backtype.stohttps://github.com/apache/stormapache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/daemon/supervisor.clj): Implementation of Supervisor.
+[backtype.storm.daemon.supervisor](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/daemon/supervisor.clj): Implementation of Supervisor.
+
+[backtype.storm.daemon.task](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/daemon/task.clj): Implementation of an individual task for a spout or bolt. Handles message routing, serialization, stats collection for the UI, as well as the spout-specific and bolt-specific execution implementations.
-[backtype.storm.daemon.task](https://github.comhttps://github.com/apache/storm-core/src/clj/backtype/storm/daemon/task.clj): Implementation of an individual task for a spout or bolt. Handles message routing, serializahttps://github.com/apache/storml as the spout-specific and bolt-specific execution implementations.
+[backtype.storm.daemon.worker](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/daemon/worker.clj): Implementation of a worker process (which will contain many tasks within). Implements message transferring and task launching.
-[backtype.storm.daemon.worker](https://github.com/apache/incubator-storhttps://github.com/apache/storm/storm/daemon/worker.clj): Implementation of a worker process (which will contain many tasks within). Implements message transferring and task launching.
+[backtype.storm.event](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/event.clj): Implements a simple asynchronous function executor. Used in various places in Nimbus and Supervisor to make functions execute in serial to avoid any race conditions.
-[backtype.storm.event](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/event.clj): Implements a simple asynchronouhttps://github.com/apache/stormes in Nimbus and Supervisor to make functions execute in serial to avoid any race conditions.
+[backtype.storm.log](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/log.clj): Defines the functions used to log messages to log4j.
-[backtype.storm.log](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/log.clj): Defines the fuhttps://github.com/apache/stormbacktype.storm.messaging.*](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/messaging): Defines a higher level interface to implementing point to point messaging. In local mode Storm uses in-memory Java queues to do thttps://github.com/apache/stormneric interface is defined in protocol.clj.
+[backtype.storm.messaging.*](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/messaging): Defines a higher level interface to implementing point to point messaging. In local mode Storm uses in-memory Java queues to do this; on a cluster, it uses ZeroMQ. The generic interface is defined in protocol.clj.
-[backtype.storm.stats](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/https://github.com/apache/storm rollup routines used when sending stats to ZK for use by the UI. Does things like windowed and rolling aggregations at multiple granularities.
+[backtype.storm.stats](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/stats.clj): Implementation of stats rollup routines used when sending stats to ZK for use by the UI. Does things like windowed and rolling aggregations at multiple granularities.
-[backtype.storm.testing](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/testing.clj): Implementation of facilities used thttps://github.com/apache/stormmulation, `complete-topology` for running a fixed set of tuples through a topology and capturing the output, tracker topologies for having fine grained control over detecting when a cluster is "idle", and other utilities.
+[backtype.storm.testing](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/testing.clj): Implementation of facilities used to test Storm topologies. Includes time simulation, `complete-topology` for running a fixed set of tuples through a topology and capturing the output, tracker topologies for having fine grained control over detecting when a cluster is "idle", and other utilities.
-[backtype.storm.thrift](https://github.https://github.com/apache/stormorm-core/src/clj/backtype/storm/thrift.clj): Clojure wrappers around the generated Thrift API to make working with Thrift structures more pleasant.
+[backtype.storm.thrift](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/thrift.clj): Clojure wrappers around the generated Thrift API to make working with Thrift structures more pleasant.
-[backtype.storm.timer](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/timer.clj): Implementation of a background timer to execute functions in the future or on a recurring interval. Storm couldn't use thehttps://github.com/apache/storm4.2/docs/api/java/util/Timer.html) class because it needed integration with time simulation in order to be able to unit test Nimbus and the Supervisor.
+[backtype.storm.timer](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/timer.clj): Implementation of a background timer to execute functions in the future or on a recurring interval. Storm couldn't use the [Timer](http://docs.oracle.com/javase/1.4.2/docs/api/java/util/Timer.html) class because it needed integration with time simulation in order to be able to unit test Nimbus and the Supervisor.
-[backtype.storm.ui.*](https://github.com/apahttps://github.com/apache/storme/src/clj/backtype/storm/ui): Implementation of Storm UI. Completely independent from rest of code base and uses the Nimbus Thrift API to get data.
+[backtype.storm.ui.*](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/ui): Implementation of Storm UI. Completely independent from rest of code base and uses the Nimbus Thrift API to get data.
[backtype.storm.util](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/util.clj): Contains generic utility functions used throughout the code base.
http://git-wip-us.apache.org/repos/asf/storm/blob/e48f5050/docs/documentation/Transactional-topologies.md
----------------------------------------------------------------------
diff --git a/docs/documentation/Transactional-topologies.md b/docs/documentation/Transactional-topologies.md
index 893e2ab..a1f4391 100644
--- a/docs/documentation/Transactional-topologies.md
+++ b/docs/documentation/Transactional-topologies.md
@@ -130,7 +130,7 @@ public static class BatchCount extends BaseBatchBolt {
}
```
-A new instance of this object is created for every batch that's being processed. The actual bolt this runs within is called [BatchBoltExecutor](https://github.com/apache/storm/blob/0.7.0/src/jvm/backtype/storm/coordination/BatchBoltExecutor.java) and manages the creation and cleanup for these objects.
+A new instance of this object is created for every batch that's being processed. The actual bolt this runs within is called [BatchBoltExecutor](https://github.com/apache/incubator-storm/blob/0.7.0/src/jvm/backtype/storm/coordination/BatchBoltExecutor.java) and manages the creation and cleanup for these objects.
The `prepare` method parameterizes this batch bolt with the Storm config, the topology context, an output collector, and the id for this batch of tuples. In the case of transactional topologies, the id will be a [TransactionAttempt](/apidocs/backtype/storm/transactional/TransactionAttempt.html) object. The batch bolt abstraction can be used in Distributed RPC as well which uses a different type of id for the batches. `BatchBolt` can actually be parameterized with the type of the id, so if you only intend to use the batch bolt for transactional topologies, you can extend `BaseTransactionalBolt` which has this definition:
http://git-wip-us.apache.org/repos/asf/storm/blob/e48f5050/docs/documentation/Trident-API-Overview.md
----------------------------------------------------------------------
diff --git a/docs/documentation/Trident-API-Overview.md b/docs/documentation/Trident-API-Overview.md
index da20ca2..fe5812e 100644
--- a/docs/documentation/Trident-API-Overview.md
+++ b/docs/documentation/Trident-API-Overview.md
@@ -270,7 +270,7 @@ mystream.aggregate(new Count(), new Fields("count"))
Like partitionAggregate, aggregators for aggregate can be chained. However, if you chain a CombinerAggregator with a non-CombinerAggregator, Trident is unable to do the partial aggregation optimization.
-You can read more about how to use persistentAggregate in the [Trident state doc](https://github.com/apache/storm/wiki/Trident-state).
+You can read more about how to use persistentAggregate in the [Trident state doc](https://github.com/apache/incubator-storm/wiki/Trident-state).
## Operations on grouped streams
http://git-wip-us.apache.org/repos/asf/storm/blob/e48f5050/docs/documentation/Trident-spouts.md
----------------------------------------------------------------------
diff --git a/docs/documentation/Trident-spouts.md b/docs/documentation/Trident-spouts.md
index 20ca5a5..c6bed45 100644
--- a/docs/documentation/Trident-spouts.md
+++ b/docs/documentation/Trident-spouts.md
@@ -34,9 +34,9 @@ Even while processing multiple batches simultaneously, Trident will order any st
Here are the following spout APIs available:
-1. [ITridentSpout](https://github.com/apache/storm/blob/master/storm-core/src/jvm/storm/trident/spout/ITridentSpout.java): The most general API that can support transactional or opaque transactional semantics. Generally you'll use one of the partitioned flavors of this API rather than this one directly.
-2. [IBatchSpout](https://gihttps://github.com/apache/stormer/storm-core/src/jvm/storm/trident/spout/IBatchSpout.java): A non-transactional spout that emits batches of tuples at a time
-3. [IPartitionedTridentSpout](https://github.com/ahttps://github.com/apache/stormore/src/jvm/storm/trident/spout/IPartitionedTridentSpout.java): A transactional spout that reads from a partitioned data source (like a cluster of Kafka servers)
+1. [ITridentSpout](https://github.com/apache/incubator-storm/blob/master/storm-core/src/jvm/storm/trident/spout/ITridentSpout.java): The most general API that can support transactional or opaque transactional semantics. Generally you'll use one of the partitioned flavors of this API rather than this one directly.
+2. [IBatchSpout](https://github.com/apache/incubator-storm/blob/master/storm-core/src/jvm/storm/trident/spout/IBatchSpout.java): A non-transactional spout that emits batches of tuples at a time
+3. [IPartitionedTridentSpout](https://github.com/apache/incubator-storm/blob/master/storm-core/src/jvm/storm/trident/spout/IPartitionedTridentSpout.java): A transactional spout that reads from a partitioned data source (like a cluster of Kafka servers)
4. [IOpaquePartitionedTridentSpout](https://github.com/apache/incubator-storm/blob/master/storm-core/src/jvm/storm/trident/spout/IOpaquePartitionedTridentSpout.java): An opaque transactional spout that reads from a partitioned data source
And, like mentioned in the beginning of this tutorial, you can use regular IRichSpout's as well.
http://git-wip-us.apache.org/repos/asf/storm/blob/e48f5050/docs/documentation/Trident-state.md
----------------------------------------------------------------------
diff --git a/docs/documentation/Trident-state.md b/docs/documentation/Trident-state.md
index 4c8b181..dc9efe8 100644
--- a/docs/documentation/Trident-state.md
+++ b/docs/documentation/Trident-state.md
@@ -309,7 +309,7 @@ public interface Snapshottable<T> extends State {
}
```
-[MemoryMapState](https://github.com/apache/storm/blob/master/storm-core/src/jvm/storm/trident/testing/MemoryMapState.java) and [MemcachedState](https://github.com/nathanmarz/trident-memcached/blob/master/src/jvm/trident/memcached/MemcachedState.java) each implement both of these interfaces.
+[MemoryMapState](https://github.com/apache/incubator-storm/blob/master/storm-core/src/jvm/storm/trident/testing/MemoryMapState.java) and [MemcachedState](https://github.com/nathanmarz/trident-memcached/blob/master/src/jvm/trident/memcached/MemcachedState.java) each implement both of these interfaces.
## Implementing Map States
@@ -322,9 +322,9 @@ public interface IBackingMap<T> {
}
```
-OpaqueMap's will call multiPut with [OpaqueValue](https://gihttps://github.com/apache/stormer/storm-core/src/jvm/storm/trident/state/OpaqueValue.java)'s for the vals, TransactionalMap's will give [TransactionalValue](https://github.com/ahttps://github.com/apache/stormore/src/jvm/storm/trident/state/TransactionalValue.java)'s for the vals, and NonTransactionalMaps will just pass the objects from the topology through.
+OpaqueMap's will call multiPut with [OpaqueValue](https://github.com/apache/incubator-storm/blob/master/storm-core/src/jvm/storm/trident/state/OpaqueValue.java)'s for the vals, TransactionalMap's will give [TransactionalValue](https://github.com/apache/incubator-storm/blob/master/storm-core/src/jvm/storm/trident/state/TransactionalValue.java)'s for the vals, and NonTransactionalMaps will just pass the objects from the topology through.
-Trident also provides the [CachedMap](https://github.com/apache/incuhttps://github.com/apache/stormm/storm/trident/state/map/CachedMap.java) class to do automatic LRU caching of map key/vals.
+Trident also provides the [CachedMap](https://github.com/apache/incubator-storm/blob/master/storm-core/src/jvm/storm/trident/state/map/CachedMap.java) class to do automatic LRU caching of map key/vals.
Finally, Trident provides the [SnapshottableMap](https://github.com/apache/incubator-storm/blob/master/storm-core/src/jvm/storm/trident/state/map/SnapshottableMap.java) class that turns a MapState into a Snapshottable object, by storing global aggregations into a fixed key.
http://git-wip-us.apache.org/repos/asf/storm/blob/e48f5050/docs/downloads.html
----------------------------------------------------------------------
diff --git a/docs/downloads.html b/docs/downloads.html
index 3e0a9d6..766a0c1 100644
--- a/docs/downloads.html
+++ b/docs/downloads.html
@@ -26,7 +26,7 @@ older:
<h3>Current Release</h3>
The current release is 0.9.3. Source and binary distributions can be found below.
- The list of changes for this release can be found <a href="https://github.com/apache/storm/blob/v0.9.3/CHANGELOG.md">here.</a>
+ The list of changes for this release can be found <a href="https://github.com/apache/incubator-storm/blob/v0.9.3/CHANGELOG.md">here.</a>
<ul>
<li><a href="http://www.apache.org/dyn/closer.cgi/storm/apache-storm-0.9.3/apache-storm-0.9.3.tar.gz">apache-storm-0.9.3.tar.gz</a>