You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by ppoulosk <gi...@git.apache.org> on 2016/08/30 23:36:35 UTC

[GitHub] storm pull request #1661: [STORM-2071] Add in retry after rebalance in unit ...

GitHub user ppoulosk opened a pull request:

    https://github.com/apache/storm/pull/1661

    [STORM-2071]  Add in retry after rebalance in unit test

    This retries the command a five times after rebalance, as it was failing on some systems.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/ppoulosk/storm STORM-2071

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/storm/pull/1661.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1661
    
----
commit ff6b91a304b6359d9a79cf33ebf10a535c375c6c
Author: Paul Poulosky <pp...@thenperfect-lm.champ.corp.yahoo.com>
Date:   2016-08-30T23:34:44Z

    Add in retry after rebalance

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1661: [STORM-2071] Add in retry after rebalance in unit test

Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on the issue:

    https://github.com/apache/storm/pull/1661
  
    +1 once comments from @abellina are addressed. Nice finding.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1661: [STORM-2071] Add in retry after rebalance in unit ...

Posted by ppoulosk <gi...@git.apache.org>.
Github user ppoulosk commented on a diff in the pull request:

    https://github.com/apache/storm/pull/1661#discussion_r76999135
  
    --- Diff: storm-core/test/clj/org/apache/storm/nimbus_test.clj ---
    @@ -1218,8 +1218,16 @@
                   (submit-local-topology nimbus "t1" {} topology)
                   ;; Instead of sleeping until topology is scheduled, rebalance topology so mk-assignments is called.
                   (.rebalance nimbus "t1" (doto (RebalanceOptions.) (.set_wait_secs 0)))
    -              (Thread/sleep 1000)
    -              (.deactivate nimbus "t1")
    +              ;; One second doesn't work on all systems.  Try it five times
    +              (loop [n 5]
    +                (if-let [result (try
    +                                  [(Thread/sleep 1000)
    --- End diff --
    
    Agreed.  Will fix.  Thanks, @abellina.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1661: [STORM-2071] Add in retry after rebalance in unit test

Posted by ppoulosk <gi...@git.apache.org>.
Github user ppoulosk commented on the issue:

    https://github.com/apache/storm/pull/1661
  
    Coded fix, retested, and pushed.  Thanks!!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1661: [STORM-2071] Add in retry after rebalance in unit test

Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on the issue:

    https://github.com/apache/storm/pull/1661
  
    @ppoulosk Could we close this as Supervisor V2 was merged? Thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1661: [STORM-2071] Add in retry after rebalance in unit ...

Posted by ppoulosk <gi...@git.apache.org>.
Github user ppoulosk closed the pull request at:

    https://github.com/apache/storm/pull/1661


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1661: [STORM-2071] Add in retry after rebalance in unit test

Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on the issue:

    https://github.com/apache/storm/pull/1661
  
    @ppoulosk Do you have any update on this?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1661: [STORM-2071] Add in retry after rebalance in unit test

Posted by ppoulosk <gi...@git.apache.org>.
Github user ppoulosk commented on the issue:

    https://github.com/apache/storm/pull/1661
  
    Yes.  Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1661: [STORM-2071] Add in retry after rebalance in unit test

Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on the issue:

    https://github.com/apache/storm/pull/1661
  
    From JIRA comment @revans2 said he fixed this issue from Supervisor V2 patch.
    
    @revans2 Since we saw builds failing (high chance from my dev.) how about addressing build issue before than Supervisor V2?
    
    @ppoulosk @revans2 
    Please have a cross look at each patch. We need to choose one which will be merged.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1661: [STORM-2071] Add in retry after rebalance in unit ...

Posted by abellina <gi...@git.apache.org>.
Github user abellina commented on a diff in the pull request:

    https://github.com/apache/storm/pull/1661#discussion_r76916977
  
    --- Diff: storm-core/test/clj/org/apache/storm/nimbus_test.clj ---
    @@ -1218,8 +1218,16 @@
                   (submit-local-topology nimbus "t1" {} topology)
                   ;; Instead of sleeping until topology is scheduled, rebalance topology so mk-assignments is called.
                   (.rebalance nimbus "t1" (doto (RebalanceOptions.) (.set_wait_secs 0)))
    -              (Thread/sleep 1000)
    -              (.deactivate nimbus "t1")
    +              ;; One second doesn't work on all systems.  Try it five times
    +              (loop [n 5]
    +                (if-let [result (try
    +                                  [(Thread/sleep 1000)
    --- End diff --
    
    how about:
    (try
      (do
         (Thread/sleep 1000)
         (.deactivate nimbus "t1"))
      (catch ...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1661: [STORM-2071] Add in retry after rebalance in unit test

Posted by revans2 <gi...@git.apache.org>.
Github user revans2 commented on the issue:

    https://github.com/apache/storm/pull/1661
  
    I would be happy to pull that fix back into a separate patch.  @ppoulosk this patch looks fine to me, my only concern would be documenting it a little more explaining that the state change might not have completed yet, and until it does we cannot rebalance.  Alternatively if you want to pull in my changes, by replacing sleep with the following I would be OK with that too.
    
    ```
    (defn wait-for-status [nimbus name status]
      (while-timeout 5000
        (let [topo-summary (first (filter (fn [topo] (= name (.get_name topo))) (.get_topologies (.getClusterInfo nimbus))))
              topo-status (if topo-summary (.get_status topo-summary) "NOT-RUNNING")]
          (log-message "WAITING FOR "name" TO BE " status " CURRENT " topo-status)
          (not= topo-status status))
        (Thread/sleep 100)))
    ...
    (wait-for-status nimbus "t1" "ACTIVE") 
    ```
    
    Either solution is fine with me.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1661: [STORM-2071] Add in retry after rebalance in unit test

Posted by abellina <gi...@git.apache.org>.
Github user abellina commented on the issue:

    https://github.com/apache/storm/pull/1661
  
    +1 thanks @ppoulosk 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1661: [STORM-2071] Add in retry after rebalance in unit test

Posted by ppoulosk <gi...@git.apache.org>.
Github user ppoulosk commented on the issue:

    https://github.com/apache/storm/pull/1661
  
    @HeartSaVioR @revans2 I have no preference which solution gets pulled in first.  
    
    I fixed the comment, retested, squashed the three commits into one and pushed it to my branch.
    
    Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1661: [STORM-2071] Add in retry after rebalance in unit ...

Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on a diff in the pull request:

    https://github.com/apache/storm/pull/1661#discussion_r76932681
  
    --- Diff: storm-core/test/clj/org/apache/storm/nimbus_test.clj ---
    @@ -1218,8 +1218,16 @@
                   (submit-local-topology nimbus "t1" {} topology)
                   ;; Instead of sleeping until topology is scheduled, rebalance topology so mk-assignments is called.
                   (.rebalance nimbus "t1" (doto (RebalanceOptions.) (.set_wait_secs 0)))
    -              (Thread/sleep 1000)
    -              (.deactivate nimbus "t1")
    +              ;; One second doesn't work on all systems.  Try it five times
    +              (loop [n 5]
    +                (if-let [result (try
    +                                  [(Thread/sleep 1000)
    --- End diff --
    
    +1 to this one. Seems clearer.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1661: [STORM-2071] Add in retry after rebalance in unit ...

Posted by abellina <gi...@git.apache.org>.
Github user abellina commented on a diff in the pull request:

    https://github.com/apache/storm/pull/1661#discussion_r76917059
  
    --- Diff: storm-core/test/clj/org/apache/storm/nimbus_test.clj ---
    @@ -1218,8 +1218,16 @@
                   (submit-local-topology nimbus "t1" {} topology)
                   ;; Instead of sleeping until topology is scheduled, rebalance topology so mk-assignments is called.
                   (.rebalance nimbus "t1" (doto (RebalanceOptions.) (.set_wait_secs 0)))
    -              (Thread/sleep 1000)
    -              (.deactivate nimbus "t1")
    +              ;; One second doesn't work on all systems.  Try it five times
    +              (loop [n 5]
    +                (if-let [result (try
    +                                  [(Thread/sleep 1000)
    +                                   (.deactivate nimbus "t1")]
    +                                  (catch Exception e
    +                                    (when (zero? n)
    +                                      (throw e))))]
    +                         (result 0)
    --- End diff --
    
    these should be aligned with (if-let +2 spaces from the opening parenthesis.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---