You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@fluo.apache.org by GitBox <gi...@apache.org> on 2020/01/10 22:46:48 UTC
[GitHub] [fluo-muchos] shannawaz opened a new pull request #306: Implement
terminate and wipe actions for Azure clusters
shannawaz opened a new pull request #306: Implement terminate and wipe actions for Azure clusters
URL: https://github.com/apache/fluo-muchos/pull/306
Also adds wipe for Hub, Maven and Fluo YARN
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [fluo-muchos] keith-turner commented on issue #306: Implement
terminate and wipe actions for Azure clusters
Posted by GitBox <gi...@apache.org>.
keith-turner commented on issue #306: Implement terminate and wipe actions for Azure clusters
URL: https://github.com/apache/fluo-muchos/pull/306#issuecomment-576730804
> Moved files to azure folder
Thanks @shannawaz I think this will make the code easier to maintain going forward.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [fluo-muchos] keith-turner commented on a change in pull request
#306: Implement terminate and wipe actions for Azure clusters
Posted by GitBox <gi...@apache.org>.
keith-turner commented on a change in pull request #306: Implement terminate and wipe actions for Azure clusters
URL: https://github.com/apache/fluo-muchos/pull/306#discussion_r369062918
##########
File path: lib/muchos/existing.py
##########
@@ -241,6 +241,10 @@ def perform(self, action):
elif action == 'cancel_shutdown':
print("Cancelling automatic shutdown of {0} cluster".format(self.config.cluster_name))
self.execute_playbook(action + ".yml")
+ # In case of Azure, also check and if necessary wipe the storage accounts
+ # It is necessary to do this after the common kill and wipe to ensure that processes are all killed prior to this
+ if action == 'wipe' and self.config.get("general", "cluster_type") == 'azure':
Review comment:
> Kindly please test wipe for EC2
Not sure I would have time to do that in the short term, not sure if anyone else could. Looking at the diffs, it seems `wipe.yml` is called in the same way before and after this change. So it seems likely that EC2 will work the same as before.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [fluo-muchos] shannawaz commented on a change in pull request #306:
Implement terminate and wipe actions for Azure clusters
Posted by GitBox <gi...@apache.org>.
shannawaz commented on a change in pull request #306: Implement terminate and wipe actions for Azure clusters
URL: https://github.com/apache/fluo-muchos/pull/306#discussion_r368176735
##########
File path: lib/muchos/existing.py
##########
@@ -241,6 +241,10 @@ def perform(self, action):
elif action == 'cancel_shutdown':
print("Cancelling automatic shutdown of {0} cluster".format(self.config.cluster_name))
self.execute_playbook(action + ".yml")
+ # In case of Azure, also check and if necessary wipe the storage accounts
+ # It is necessary to do this after the common kill and wipe to ensure that processes are all killed prior to this
+ if action == 'wipe' and self.config.get("general", "cluster_type") == 'azure':
Review comment:
Made the changes as per recommendation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [fluo-muchos] keith-turner commented on a change in pull request
#306: Implement terminate and wipe actions for Azure clusters
Posted by GitBox <gi...@apache.org>.
keith-turner commented on a change in pull request #306: Implement terminate and wipe actions for Azure clusters
URL: https://github.com/apache/fluo-muchos/pull/306#discussion_r366043111
##########
File path: lib/muchos/existing.py
##########
@@ -241,6 +241,10 @@ def perform(self, action):
elif action == 'cancel_shutdown':
print("Cancelling automatic shutdown of {0} cluster".format(self.config.cluster_name))
self.execute_playbook(action + ".yml")
+ # In case of Azure, also check and if necessary wipe the storage accounts
+ # It is necessary to do this after the common kill and wipe to ensure that processes are all killed prior to this
+ if action == 'wipe' and self.config.get("general", "cluster_type") == 'azure':
Review comment:
Would be nice to put azure specific code in azure.py if possible. One possible way to do this is to create a `wipe()` method in existing.py that does nothing, override it in azure.py, and call `wipe()` here. Alternatively, would it make sense for wipe.yml to call azure-wipe-adlsg2.yml in ansible instead of doing it from Pytyhon?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [fluo-muchos] shannawaz commented on issue #306: Implement
terminate and wipe actions for Azure clusters
Posted by GitBox <gi...@apache.org>.
shannawaz commented on issue #306: Implement terminate and wipe actions for Azure clusters
URL: https://github.com/apache/fluo-muchos/pull/306#issuecomment-575833294
> There are multiple azure files under the ansible dir now. Would it make sense to organize these into a directory?
Moved files to azure folder
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [fluo-muchos] arvindshmicrosoft merged pull request #306: Implement
terminate and wipe actions for Azure clusters
Posted by GitBox <gi...@apache.org>.
arvindshmicrosoft merged pull request #306: Implement terminate and wipe actions for Azure clusters
URL: https://github.com/apache/fluo-muchos/pull/306
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [fluo-muchos] arvindshmicrosoft commented on a change in pull
request #306: Implement terminate and wipe actions for Azure clusters
Posted by GitBox <gi...@apache.org>.
arvindshmicrosoft commented on a change in pull request #306: Implement terminate and wipe actions for Azure clusters
URL: https://github.com/apache/fluo-muchos/pull/306#discussion_r365874438
##########
File path: lib/muchos/existing.py
##########
@@ -230,13 +230,19 @@ def perform(self, action):
self.setup()
elif action == 'ssh':
self.ssh()
- elif action in ('wipe', 'kill', 'cancel_shutdown'):
+ elif action == 'wipe':
+ # First execute kill and wipe routines on the attached disks
Review comment:
I believe we need the validation for a valid hosts file here as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [fluo-muchos] shannawaz commented on a change in pull request #306:
Implement terminate and wipe actions for Azure clusters
Posted by GitBox <gi...@apache.org>.
shannawaz commented on a change in pull request #306: Implement terminate and wipe actions for Azure clusters
URL: https://github.com/apache/fluo-muchos/pull/306#discussion_r368180098
##########
File path: lib/muchos/existing.py
##########
@@ -241,6 +241,10 @@ def perform(self, action):
elif action == 'cancel_shutdown':
print("Cancelling automatic shutdown of {0} cluster".format(self.config.cluster_name))
self.execute_playbook(action + ".yml")
+ # In case of Azure, also check and if necessary wipe the storage accounts
+ # It is necessary to do this after the common kill and wipe to ensure that processes are all killed prior to this
+ if action == 'wipe' and self.config.get("general", "cluster_type") == 'azure':
Review comment:
We are unable to test wipe changes for EC2. Kindly please test wipe for EC2
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services