You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by HeartSaVioR <gi...@git.apache.org> on 2016/07/18 03:32:21 UTC
[GitHub] storm pull request #1572: STORM-1976 Remove cleanup-corrupt-topologies! (1.x...
GitHub user HeartSaVioR opened a pull request:
https://github.com/apache/storm/pull/1572
STORM-1976 Remove cleanup-corrupt-topologies! (1.x)
For master branch: #1571
* cleanup-corrupt-topologies! was born from non-H/A Nimbus age
* Keeping this with Nimbus H/A makes various issues which in result topology going away
Please refer [STORM-1976](https://issues.apache.org/jira/browse/STORM-1976) for more details.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/HeartSaVioR/storm STORM-1976-1.x
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/1572.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1572
----
commit d677f54b4d6281733169e493eb3330bbde95bbe7
Author: Jungtaek Lim <ka...@gmail.com>
Date: 2016-07-17T22:48:55Z
STORM-1976 Remove cleanup-corrupt-topologies!
* cleanup-corrupt-topologies! was born from non-H/A Nimbus age
* Keeping this with Nimbus H/A makes various issues which in result topology going away
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1572: STORM-1976 Remove cleanup-corrupt-topologies! (1.x)
Posted by revans2 <gi...@git.apache.org>.
Github user revans2 commented on the issue:
https://github.com/apache/storm/pull/1572
+1, but I would like to see a tool that can kill a corrupted topology without nimbus being involved, or instructions somewhere on how to do it by logging directly into ZooKeeper.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request #1572: STORM-1976 Remove cleanup-corrupt-topologies! (1.x...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/storm/pull/1572
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1572: STORM-1976 Remove cleanup-corrupt-topologies! (1.x)
Posted by harshach <gi...@git.apache.org>.
Github user harshach commented on the issue:
https://github.com/apache/storm/pull/1572
@HeartSaVioR @revans2 are we ok with merging this in and file a follow-up JIRA to have a discussion around the HA and expectations from user.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1572: STORM-1976 Remove cleanup-corrupt-topologies! (1.x)
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on the issue:
https://github.com/apache/storm/pull/1572
Regarding tool, I filed an issue: https://issues.apache.org/jira/browse/STORM-1985
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1572: STORM-1976 Remove cleanup-corrupt-topologies! (1.x)
Posted by revans2 <gi...@git.apache.org>.
Github user revans2 commented on the issue:
https://github.com/apache/storm/pull/1572
@HeartSaVioR actually I have thought about it more and I am probably overreacting. If you want to call these blockers it is not that big of a deal. I would just like the community have a follow on discussion about how we really want HA to behave in these situations now that we have had some experience with people running with HA.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1572: STORM-1976 Remove cleanup-corrupt-topologies! (1.x)
Posted by harshach <gi...@git.apache.org>.
Github user harshach commented on the issue:
https://github.com/apache/storm/pull/1572
+1 . build failures are not related to the patch.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1572: STORM-1976 Remove cleanup-corrupt-topologies! (1.x)
Posted by revans2 <gi...@git.apache.org>.
Github user revans2 commented on the issue:
https://github.com/apache/storm/pull/1572
Yes I am good with merging this in.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1572: STORM-1976 Remove cleanup-corrupt-topologies! (1.x)
Posted by harshach <gi...@git.apache.org>.
Github user harshach commented on the issue:
https://github.com/apache/storm/pull/1572
@revans2 @HeartSaVioR why can't we enforce the min.replication.count by looking nimbus.seeds if there are more than one host in that list we should set the min.replication.count to 2. User can override the setting by specifying it in storm.yaml. My understanding is they are not doing it and we are relying on min.replication.count to be 1.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1572: STORM-1976 Remove cleanup-corrupt-topologies! (1.x)
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on the issue:
https://github.com/apache/storm/pull/1572
@revans2
Yes agreed. After removing this code we can't remove corrupted topology in normal way since no nimbus can gain leadership. It would be nice to have a tool to show replication count of topologies *without leader nimbus involved* to determine corrupted topology, and kill one if users want to.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1572: STORM-1976 Remove cleanup-corrupt-topologies! (1.x)
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on the issue:
https://github.com/apache/storm/pull/1572
Yes I think discussion of Nimbus H/A and BlobStore would be longer, so we need to apply short-term fix first. There're lots of critical bugfixes in 1.0.2 so we shouldn't drag the release because of struggling long-term fix.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1572: STORM-1976 Remove cleanup-corrupt-topologies! (1.x)
Posted by revans2 <gi...@git.apache.org>.
Github user revans2 commented on the issue:
https://github.com/apache/storm/pull/1572
@harshach I don't think that would fix the issue, although it is an improvement. If I think about it more I think what happened was that they were not running with HA/expecting HA to work, but ran into a bad situation where they really would have wanted to be running with HA, and then when they tried to recover, by bringing up a new nimbus instance, all of the topologies were killed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---