You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@storm.apache.org by pt...@apache.org on 2016/04/21 20:01:41 UTC
[1/3] storm git commit: add scheduler docs
Repository: storm
Updated Branches:
refs/heads/1.x-branch 705951ebf -> 8b3dff85c
add scheduler docs
Project: http://git-wip-us.apache.org/repos/asf/storm/repo
Commit: http://git-wip-us.apache.org/repos/asf/storm/commit/1919ef1f
Tree: http://git-wip-us.apache.org/repos/asf/storm/tree/1919ef1f
Diff: http://git-wip-us.apache.org/repos/asf/storm/diff/1919ef1f
Branch: refs/heads/1.x-branch
Commit: 1919ef1fcfdf774c5154f37264b12b0c231dbb08
Parents: a240df5
Author: Xin Wang <be...@163.com>
Authored: Sun Apr 3 21:41:44 2016 +0800
Committer: Xin Wang <be...@163.com>
Committed: Wed Apr 20 23:24:35 2016 +0800
----------------------------------------------------------------------
docs/Storm-Scheduler.md | 27 +++++++++++++++++++++++++++
docs/index.md | 1 +
2 files changed, 28 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/storm/blob/1919ef1f/docs/Storm-Scheduler.md
----------------------------------------------------------------------
diff --git a/docs/Storm-Scheduler.md b/docs/Storm-Scheduler.md
new file mode 100644
index 0000000..51b3aac
--- /dev/null
+++ b/docs/Storm-Scheduler.md
@@ -0,0 +1,27 @@
+---
+title: Scheduler
+layout: documentation
+documentation: true
+---
+
+Storm now has 4 kinds of built-in schedulers: [DefaultScheduler]({{page.git-blob-base}}/storm-core/src/clj/org/apache/storm/scheduler/DefaultScheduler.clj), [IsolationScheduler]({{page.git-blob-base}}/storm-core/src/clj/org/apache/storm/scheduler/IsolationScheduler.clj), [MultitenantScheduler]({{page.git-blob-base}}/storm-core/src/jvm/org/apache/storm/scheduler/multitenant/MultitenantScheduler.java), [ResourceAwareScheduler](Resource_Aware_Scheduler_overview.html).
+
+## Pluggable scheduler
+You can implement your own scheduler to replace the default scheduler to assign executors to workers. You configure the class to use the "storm.scheduler" config in your storm.yaml, and your scheduler must implement the [IScheduler]({{page.git-blob-base}}/storm-core/src/jvm/org/apache/storm/scheduler/IScheduler.java) interface.
+
+## Isolation Scheduler
+The isolation scheduler makes it easy and safe to share a cluster among many topologies. The isolation scheduler lets you specify which topologies should be "isolated", meaning that they run on a dedicated set of machines within the cluster where no other topologies will be running. These isolated topologies are given priority on the cluster, so resources will be allocated to isolated topologies if there's competition with non-isolated topologies, and resources will be taken away from non-isolated topologies if necessary to get resources for an isolated topology. Once all isolated topologies are allocated, the remaining machines on the cluster are shared among all non-isolated topologies.
+
+You can configure the isolation scheduler in the Nimbus configuration by setting "storm.scheduler" to "org.apache.storm.scheduler.IsolationScheduler". Then, use the "isolation.scheduler.machines" config to specify how many machines each topology should get. This configuration is a map from topology name to the number of isolated machines allocated to this topology. For example:
+
+```
+isolation.scheduler.machines:
+ "my-topology": 8
+ "tiny-topology": 1
+ "some-other-topology": 3
+```
+
+Any topologies submitted to the cluster not listed there will not be isolated. Note that there is no way for a user of Storm to affect their isolation settings – this is only allowed by the administrator of the cluster (this is very much intentional).
+
+The isolation scheduler solves the multi-tenancy problem – avoiding resource contention between topologies – by providing full isolation between topologies. The intention is that "productionized" topologies should be listed in the isolation config, and test or in-development topologies should not. The remaining machines on the cluster serve the dual role of failover for isolated topologies and for running the non-isolated topologies.
+
http://git-wip-us.apache.org/repos/asf/storm/blob/1919ef1f/docs/index.md
----------------------------------------------------------------------
diff --git a/docs/index.md b/docs/index.md
index 14d6ce8..db736e2 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -7,6 +7,7 @@ documentation: true
* [Javadoc](javadocs/index.html)
* [Concepts](Concepts.html)
+* [Scheduler](Storm-Scheduler.html)
* [Configuration](Configuration.html)
* [Guaranteeing message processing](Guaranteeing-message-processing.html)
* [Daemon Fault Tolerance](Daemon-Fault-Tolerance.html)
[2/3] storm git commit: Merge branch 'STORM-1679-1.x' of
github.com:vesense/storm into 1.x-branch
Posted by pt...@apache.org.
Merge branch 'STORM-1679-1.x' of github.com:vesense/storm into 1.x-branch
Project: http://git-wip-us.apache.org/repos/asf/storm/repo
Commit: http://git-wip-us.apache.org/repos/asf/storm/commit/36ab4b31
Tree: http://git-wip-us.apache.org/repos/asf/storm/tree/36ab4b31
Diff: http://git-wip-us.apache.org/repos/asf/storm/diff/36ab4b31
Branch: refs/heads/1.x-branch
Commit: 36ab4b317b28f764bda29c273edbe94b0a45d960
Parents: 705951e 1919ef1
Author: P. Taylor Goetz <pt...@gmail.com>
Authored: Thu Apr 21 13:52:44 2016 -0400
Committer: P. Taylor Goetz <pt...@gmail.com>
Committed: Thu Apr 21 13:52:44 2016 -0400
----------------------------------------------------------------------
docs/Storm-Scheduler.md | 27 +++++++++++++++++++++++++++
docs/index.md | 1 +
2 files changed, 28 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/storm/blob/36ab4b31/docs/index.md
----------------------------------------------------------------------
[3/3] storm git commit: Add STORM-1679 to changelog
Posted by pt...@apache.org.
Add STORM-1679 to changelog
Project: http://git-wip-us.apache.org/repos/asf/storm/repo
Commit: http://git-wip-us.apache.org/repos/asf/storm/commit/8b3dff85
Tree: http://git-wip-us.apache.org/repos/asf/storm/tree/8b3dff85
Diff: http://git-wip-us.apache.org/repos/asf/storm/diff/8b3dff85
Branch: refs/heads/1.x-branch
Commit: 8b3dff85cd480233a5219fae65c805478e38f29c
Parents: 36ab4b3
Author: P. Taylor Goetz <pt...@gmail.com>
Authored: Thu Apr 21 13:53:40 2016 -0400
Committer: P. Taylor Goetz <pt...@gmail.com>
Committed: Thu Apr 21 13:53:40 2016 -0400
----------------------------------------------------------------------
CHANGELOG.md | 1 +
1 file changed, 1 insertion(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/storm/blob/8b3dff85/CHANGELOG.md
----------------------------------------------------------------------
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 7fc5bfb..c734f27 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,4 +1,5 @@
## 1.0.1
+ * STORM-1679: add storm Scheduler documents
* STORM-1704: When logviewer_search.html opens daemon file, next search always show no result
* STORM-1714: StatefulBolts ends up as normal bolts while using TopologyBuilder.setBolt without parallelism
* STORM-1683: only check non-system streams by default