You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by ppoulosk <gi...@git.apache.org> on 2016/08/30 23:36:35 UTC
[GitHub] storm pull request #1661: [STORM-2071] Add in retry after rebalance in unit ...
GitHub user ppoulosk opened a pull request:
https://github.com/apache/storm/pull/1661
[STORM-2071] Add in retry after rebalance in unit test
This retries the command a five times after rebalance, as it was failing on some systems.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ppoulosk/storm STORM-2071
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/1661.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1661
----
commit ff6b91a304b6359d9a79cf33ebf10a535c375c6c
Author: Paul Poulosky <pp...@thenperfect-lm.champ.corp.yahoo.com>
Date: 2016-08-30T23:34:44Z
Add in retry after rebalance
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1661: [STORM-2071] Add in retry after rebalance in unit test
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on the issue:
https://github.com/apache/storm/pull/1661
+1 once comments from @abellina are addressed. Nice finding.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request #1661: [STORM-2071] Add in retry after rebalance in unit ...
Posted by ppoulosk <gi...@git.apache.org>.
Github user ppoulosk commented on a diff in the pull request:
https://github.com/apache/storm/pull/1661#discussion_r76999135
--- Diff: storm-core/test/clj/org/apache/storm/nimbus_test.clj ---
@@ -1218,8 +1218,16 @@
(submit-local-topology nimbus "t1" {} topology)
;; Instead of sleeping until topology is scheduled, rebalance topology so mk-assignments is called.
(.rebalance nimbus "t1" (doto (RebalanceOptions.) (.set_wait_secs 0)))
- (Thread/sleep 1000)
- (.deactivate nimbus "t1")
+ ;; One second doesn't work on all systems. Try it five times
+ (loop [n 5]
+ (if-let [result (try
+ [(Thread/sleep 1000)
--- End diff --
Agreed. Will fix. Thanks, @abellina.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1661: [STORM-2071] Add in retry after rebalance in unit test
Posted by ppoulosk <gi...@git.apache.org>.
Github user ppoulosk commented on the issue:
https://github.com/apache/storm/pull/1661
Coded fix, retested, and pushed. Thanks!!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1661: [STORM-2071] Add in retry after rebalance in unit test
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on the issue:
https://github.com/apache/storm/pull/1661
@ppoulosk Could we close this as Supervisor V2 was merged? Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request #1661: [STORM-2071] Add in retry after rebalance in unit ...
Posted by ppoulosk <gi...@git.apache.org>.
Github user ppoulosk closed the pull request at:
https://github.com/apache/storm/pull/1661
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1661: [STORM-2071] Add in retry after rebalance in unit test
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on the issue:
https://github.com/apache/storm/pull/1661
@ppoulosk Do you have any update on this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1661: [STORM-2071] Add in retry after rebalance in unit test
Posted by ppoulosk <gi...@git.apache.org>.
Github user ppoulosk commented on the issue:
https://github.com/apache/storm/pull/1661
Yes. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1661: [STORM-2071] Add in retry after rebalance in unit test
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on the issue:
https://github.com/apache/storm/pull/1661
From JIRA comment @revans2 said he fixed this issue from Supervisor V2 patch.
@revans2 Since we saw builds failing (high chance from my dev.) how about addressing build issue before than Supervisor V2?
@ppoulosk @revans2
Please have a cross look at each patch. We need to choose one which will be merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request #1661: [STORM-2071] Add in retry after rebalance in unit ...
Posted by abellina <gi...@git.apache.org>.
Github user abellina commented on a diff in the pull request:
https://github.com/apache/storm/pull/1661#discussion_r76916977
--- Diff: storm-core/test/clj/org/apache/storm/nimbus_test.clj ---
@@ -1218,8 +1218,16 @@
(submit-local-topology nimbus "t1" {} topology)
;; Instead of sleeping until topology is scheduled, rebalance topology so mk-assignments is called.
(.rebalance nimbus "t1" (doto (RebalanceOptions.) (.set_wait_secs 0)))
- (Thread/sleep 1000)
- (.deactivate nimbus "t1")
+ ;; One second doesn't work on all systems. Try it five times
+ (loop [n 5]
+ (if-let [result (try
+ [(Thread/sleep 1000)
--- End diff --
how about:
(try
(do
(Thread/sleep 1000)
(.deactivate nimbus "t1"))
(catch ...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1661: [STORM-2071] Add in retry after rebalance in unit test
Posted by revans2 <gi...@git.apache.org>.
Github user revans2 commented on the issue:
https://github.com/apache/storm/pull/1661
I would be happy to pull that fix back into a separate patch. @ppoulosk this patch looks fine to me, my only concern would be documenting it a little more explaining that the state change might not have completed yet, and until it does we cannot rebalance. Alternatively if you want to pull in my changes, by replacing sleep with the following I would be OK with that too.
```
(defn wait-for-status [nimbus name status]
(while-timeout 5000
(let [topo-summary (first (filter (fn [topo] (= name (.get_name topo))) (.get_topologies (.getClusterInfo nimbus))))
topo-status (if topo-summary (.get_status topo-summary) "NOT-RUNNING")]
(log-message "WAITING FOR "name" TO BE " status " CURRENT " topo-status)
(not= topo-status status))
(Thread/sleep 100)))
...
(wait-for-status nimbus "t1" "ACTIVE")
```
Either solution is fine with me.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1661: [STORM-2071] Add in retry after rebalance in unit test
Posted by abellina <gi...@git.apache.org>.
Github user abellina commented on the issue:
https://github.com/apache/storm/pull/1661
+1 thanks @ppoulosk
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1661: [STORM-2071] Add in retry after rebalance in unit test
Posted by ppoulosk <gi...@git.apache.org>.
Github user ppoulosk commented on the issue:
https://github.com/apache/storm/pull/1661
@HeartSaVioR @revans2 I have no preference which solution gets pulled in first.
I fixed the comment, retested, squashed the three commits into one and pushed it to my branch.
Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request #1661: [STORM-2071] Add in retry after rebalance in unit ...
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/1661#discussion_r76932681
--- Diff: storm-core/test/clj/org/apache/storm/nimbus_test.clj ---
@@ -1218,8 +1218,16 @@
(submit-local-topology nimbus "t1" {} topology)
;; Instead of sleeping until topology is scheduled, rebalance topology so mk-assignments is called.
(.rebalance nimbus "t1" (doto (RebalanceOptions.) (.set_wait_secs 0)))
- (Thread/sleep 1000)
- (.deactivate nimbus "t1")
+ ;; One second doesn't work on all systems. Try it five times
+ (loop [n 5]
+ (if-let [result (try
+ [(Thread/sleep 1000)
--- End diff --
+1 to this one. Seems clearer.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request #1661: [STORM-2071] Add in retry after rebalance in unit ...
Posted by abellina <gi...@git.apache.org>.
Github user abellina commented on a diff in the pull request:
https://github.com/apache/storm/pull/1661#discussion_r76917059
--- Diff: storm-core/test/clj/org/apache/storm/nimbus_test.clj ---
@@ -1218,8 +1218,16 @@
(submit-local-topology nimbus "t1" {} topology)
;; Instead of sleeping until topology is scheduled, rebalance topology so mk-assignments is called.
(.rebalance nimbus "t1" (doto (RebalanceOptions.) (.set_wait_secs 0)))
- (Thread/sleep 1000)
- (.deactivate nimbus "t1")
+ ;; One second doesn't work on all systems. Try it five times
+ (loop [n 5]
+ (if-let [result (try
+ [(Thread/sleep 1000)
+ (.deactivate nimbus "t1")]
+ (catch Exception e
+ (when (zero? n)
+ (throw e))))]
+ (result 0)
--- End diff --
these should be aligned with (if-let +2 spaces from the opening parenthesis.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---