You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@storm.apache.org by ka...@apache.org on 2015/09/19 02:02:25 UTC
[1/3] storm git commit: set storm.local.dir relative values relative
to storm.home rather than where user run storm command
Repository: storm
Updated Branches:
refs/heads/0.10.x-branch 150f7f6e3 -> a01ffbaac
set storm.local.dir relative values relative to storm.home rather than where user run storm command
Project: http://git-wip-us.apache.org/repos/asf/storm/repo
Commit: http://git-wip-us.apache.org/repos/asf/storm/commit/30421554
Tree: http://git-wip-us.apache.org/repos/asf/storm/tree/30421554
Diff: http://git-wip-us.apache.org/repos/asf/storm/diff/30421554
Branch: refs/heads/0.10.x-branch
Commit: 3042155498c64c3298c9e714a472147b20698284
Parents: 150f7f6
Author: renkai <ga...@gmail.com>
Authored: Sun Sep 6 11:47:02 2015 +0800
Committer: Jungtaek Lim <ka...@gmail.com>
Committed: Fri Sep 18 10:05:16 2015 +0900
----------------------------------------------------------------------
docs/documentation/Setting-up-a-Storm-cluster.md | 10 ++++++++--
storm-core/src/clj/backtype/storm/config.clj | 13 +++++++++----
storm-core/src/clj/backtype/storm/util.clj | 4 ++++
3 files changed, 21 insertions(+), 6 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/storm/blob/30421554/docs/documentation/Setting-up-a-Storm-cluster.md
----------------------------------------------------------------------
diff --git a/docs/documentation/Setting-up-a-Storm-cluster.md b/docs/documentation/Setting-up-a-Storm-cluster.md
index 07b4eda..e92061f 100644
--- a/docs/documentation/Setting-up-a-Storm-cluster.md
+++ b/docs/documentation/Setting-up-a-Storm-cluster.md
@@ -52,12 +52,18 @@ storm.zookeeper.servers:
If the port that your Zookeeper cluster uses is different than the default, you should set **storm.zookeeper.port** as well.
-2) **storm.local.dir**: The Nimbus and Supervisor daemons require a directory on the local disk to store small amounts of state (like jars, confs, and things like that). You should create that directory on each machine, give it proper permissions, and then fill in the directory location using this config. For example:
+2) **storm.local.dir**: The Nimbus and Supervisor daemons require a directory on the local disk to store small amounts of state (like jars, confs, and things like that).
+ You should create that directory on each machine, give it proper permissions, and then fill in the directory location using this config. For example:
```yaml
storm.local.dir: "/mnt/storm"
```
-
+If you run storm on windows,it could be:
+```yaml
+storm.local.dir: "C:\\\\storm-local\\"
+```
+If you use a relative path,it will be relative to where you installed storm(STORM_HOME).
+You can leave it empty with default value `$STORM_HOME/storm-local`
3) **nimbus.host**: The worker nodes need to know which machine is the master in order to download topology jars and confs. For example:
```yaml
http://git-wip-us.apache.org/repos/asf/storm/blob/30421554/storm-core/src/clj/backtype/storm/config.clj
----------------------------------------------------------------------
diff --git a/storm-core/src/clj/backtype/storm/config.clj b/storm-core/src/clj/backtype/storm/config.clj
index 7647906..48a13b0 100644
--- a/storm-core/src/clj/backtype/storm/config.clj
+++ b/storm-core/src/clj/backtype/storm/config.clj
@@ -131,9 +131,14 @@
([name]
(read-yaml-config true)))
+(defn absolute-storm-local-dir [conf]
+ (let [storm-home (System/getProperty "storm.home")
+ path (conf STORM-LOCAL-DIR)]
+ (if (is-absolute-path? path) path (str storm-home file-path-separator path))))
+
(defn master-local-dir
[conf]
- (let [ret (str (conf STORM-LOCAL-DIR) file-path-separator "nimbus")]
+ (let [ret (str (absolute-storm-local-dir conf) file-path-separator "nimbus")]
(FileUtils/forceMkdir (File. ret))
ret))
@@ -167,7 +172,7 @@
(defn supervisor-local-dir
[conf]
- (let [ret (str (conf STORM-LOCAL-DIR) file-path-separator "supervisor")]
+ (let [ret (str (absolute-storm-local-dir conf) file-path-separator "supervisor")]
(FileUtils/forceMkdir (File. ret))
ret))
@@ -222,7 +227,7 @@
))
(defn worker-user-root [conf]
- (str (conf STORM-LOCAL-DIR) "/workers-users"))
+ (str (absolute-storm-local-dir conf) "/workers-users"))
(defn worker-user-file [conf worker-id]
(str (worker-user-root conf) "/" worker-id))
@@ -249,7 +254,7 @@
(defn worker-root
([conf]
- (str (conf STORM-LOCAL-DIR) file-path-separator "workers"))
+ (str (absolute-storm-local-dir conf) file-path-separator "workers"))
([conf id]
(str (worker-root conf) file-path-separator id)))
http://git-wip-us.apache.org/repos/asf/storm/blob/30421554/storm-core/src/clj/backtype/storm/util.clj
----------------------------------------------------------------------
diff --git a/storm-core/src/clj/backtype/storm/util.clj b/storm-core/src/clj/backtype/storm/util.clj
index acd0b6b..1762a5f 100644
--- a/storm-core/src/clj/backtype/storm/util.clj
+++ b/storm-core/src/clj/backtype/storm/util.clj
@@ -18,6 +18,7 @@
(:import [java.net InetAddress])
(:import [java.util Map Map$Entry List ArrayList Collection Iterator HashMap])
(:import [java.io FileReader FileNotFoundException])
+ (:import [java.nio.file Paths])
(:import [backtype.storm Config])
(:import [backtype.storm.utils Time Container ClojureTimerTask Utils
MutableObject MutableInt])
@@ -58,6 +59,9 @@
(def class-path-separator
(System/getProperty "path.separator"))
+(defn is-absolute-path? [path]
+ (.isAbsolute (Paths/get path (into-array String []))))
+
(defmacro defalias
"Defines an alias for a var: a new var with the same root binding (if
any) and similar metadata. The metadata of the alias is its initial
[2/3] storm git commit: typo fix
Posted by ka...@apache.org.
typo fix
Project: http://git-wip-us.apache.org/repos/asf/storm/repo
Commit: http://git-wip-us.apache.org/repos/asf/storm/commit/d88b5bb3
Tree: http://git-wip-us.apache.org/repos/asf/storm/tree/d88b5bb3
Diff: http://git-wip-us.apache.org/repos/asf/storm/diff/d88b5bb3
Branch: refs/heads/0.10.x-branch
Commit: d88b5bb342c279f326e31c5a72fe567e46cca29f
Parents: 3042155
Author: renkai <ga...@gmail.com>
Authored: Sun Sep 6 13:07:09 2015 +0800
Committer: Jungtaek Lim <ka...@gmail.com>
Committed: Fri Sep 18 10:05:20 2015 +0900
----------------------------------------------------------------------
docs/documentation/Setting-up-a-Storm-cluster.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/storm/blob/d88b5bb3/docs/documentation/Setting-up-a-Storm-cluster.md
----------------------------------------------------------------------
diff --git a/docs/documentation/Setting-up-a-Storm-cluster.md b/docs/documentation/Setting-up-a-Storm-cluster.md
index e92061f..9fabb27 100644
--- a/docs/documentation/Setting-up-a-Storm-cluster.md
+++ b/docs/documentation/Setting-up-a-Storm-cluster.md
@@ -60,7 +60,7 @@ storm.local.dir: "/mnt/storm"
```
If you run storm on windows,it could be:
```yaml
-storm.local.dir: "C:\\\\storm-local\\"
+storm.local.dir: "C:\\storm-local"
```
If you use a relative path,it will be relative to where you installed storm(STORM_HOME).
You can leave it empty with default value `$STORM_HOME/storm-local`
[3/3] storm git commit: add STORM-1005 to CHANGELOG.md
Posted by ka...@apache.org.
add STORM-1005 to CHANGELOG.md
Project: http://git-wip-us.apache.org/repos/asf/storm/repo
Commit: http://git-wip-us.apache.org/repos/asf/storm/commit/a01ffbaa
Tree: http://git-wip-us.apache.org/repos/asf/storm/tree/a01ffbaa
Diff: http://git-wip-us.apache.org/repos/asf/storm/diff/a01ffbaa
Branch: refs/heads/0.10.x-branch
Commit: a01ffbaacf355ccf54525be1a906f45b1dc62c2d
Parents: d88b5bb
Author: Jungtaek Lim <ka...@gmail.com>
Authored: Sat Sep 19 09:02:01 2015 +0900
Committer: Jungtaek Lim <ka...@gmail.com>
Committed: Sat Sep 19 09:02:01 2015 +0900
----------------------------------------------------------------------
CHANGELOG.md | 1 +
1 file changed, 1 insertion(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/storm/blob/a01ffbaa/CHANGELOG.md
----------------------------------------------------------------------
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c8c60bf..8a3c7dc 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,4 +1,5 @@
## 0.10.0
+ * STORM-1005: Supervisor do not get running workers after restart.
* STORM-803: Cleanup travis-ci build and logs
* STORM-1027: Use overflow buffer for emitting metrics
* STORM-1024: log4j changes leaving ${sys:storm.log.dir} under STORM_HOME dir