You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by "virajjasani (via GitHub)" <gi...@apache.org> on 2023/05/24 23:10:20 UTC
[GitHub] [ambari] virajjasani opened a new pull request, #3702: AMBARI-25949 regionserver graceful start/stop should be compatible for hbase 2
virajjasani opened a new pull request, #3702:
URL: https://github.com/apache/ambari/pull/3702
Jira: AMBARI-25949
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
For additional commands, e-mail: dev-help@ambari.apache.org
[GitHub] [ambari] timyuer commented on pull request #3702: AMBARI-25949 regionserver graceful start/stop should be compatible for hbase 2
Posted by "timyuer (via GitHub)" <gi...@apache.org>.
timyuer commented on PR #3702:
URL: https://github.com/apache/ambari/pull/3702#issuecomment-1614245169
Apart from the minor issues above, this PR is good looks at me.
+1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
For additional commands, e-mail: dev-help@ambari.apache.org
[GitHub] [ambari] timyuer commented on a diff in pull request #3702: AMBARI-25949 regionserver graceful start/stop should be compatible for hbase 2
Posted by "timyuer (via GitHub)" <gi...@apache.org>.
timyuer commented on code in PR #3702:
URL: https://github.com/apache/ambari/pull/3702#discussion_r1247514610
##########
ambari-server/src/main/resources/stacks/BIGTOP/3.2.0/services/HBASE/package/scripts/hbase_regionserver.py:
##########
@@ -51,6 +55,57 @@ def configure(self, env):
def decommission(self, env):
print "Decommission not yet implemented!"
+ def graceful_stop(self, env, upgrade_type=None):
+ import params
+
+ # Mark Draining ZNode
+ params.hbase_drain_only = False
+ params.hbase_excluded_hosts = params.hostname
+ env.set_params(params)
+ hbase_decommission(env)
+
+ # Stop RegionServer
+ hbase_service('regionserver', action='stop')
+
+ # Remove from Draining ZNode to make host useable on restarting regionserver
+ params.hbase_drain_only = True
+ env.set_params(params)
+ hbase_decommission(env)
+
+ def graceful_start(self, env, upgrade_type=None):
+ import params
+ env.set_params(params)
+
+ #Start RegionServer
+ hbase_service('regionserver', action='start')
+
+ # Load Regions back
+ kinit_cmd = params.kinit_cmd_master
+ host = params.hostname
+
+ regionmover_cmd = format(
+ "{kinit_cmd} HBASE_SERVER_JAAS_OPTS=\"{master_security_config}\" {hbase_cmd} --config {hbase_conf_dir} {hbase_decommission_auth_config} org.jruby.Main {region_mover} -m 24 load {host}")
+
+ try:
+ Execute(regionmover_cmd,
+ user=params.hbase_user,
+ logoutput=True
+ )
+ except Exception as e:
+ Logger.info("HBase 1: region_mover failed while loading regions back to source RS." + str(e))
+ # Execute HBase 2 scripts if HBase 1 scripts fail.
+ # If the Exception is genuine, it will fail here because HBase 1 scripts work only for HBase 1
+ # and HBase 2 scripts work only for HBase 2 cluster.
+ try:
+ regionmover_cmd = format(
+ "{kinit_cmd} HBASE_SERVER_JAAS_OPTS=\"{master_security_config}\" {hbase_cmd} --config {hbase_conf_dir} {hbase_decommission_auth_config} org.jruby.Main {region_mover} -m 24 -o load -r {host}")
+ Execute(regionmover_cmd,
+ user=params.hbase_user,
+ logoutput=True
+ )
Review Comment:
@virajjasani Thank you for your work.
Why don't we use HBase 2 scripts directly because HBase 1 scripts will fail in the current stack (BIGTOP-3.2.0)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
For additional commands, e-mail: dev-help@ambari.apache.org
[GitHub] [ambari] virajjasani commented on pull request #3702: AMBARI-25949 regionserver graceful start/stop should be compatible for hbase 2
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on PR #3702:
URL: https://github.com/apache/ambari/pull/3702#issuecomment-1639185543
thanks for the review @timyuer
sorry for the delay, let me address the changes soon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
For additional commands, e-mail: dev-help@ambari.apache.org
[GitHub] [ambari] timyuer commented on a diff in pull request #3702: AMBARI-25949 regionserver graceful start/stop should be compatible for hbase 2
Posted by "timyuer (via GitHub)" <gi...@apache.org>.
timyuer commented on code in PR #3702:
URL: https://github.com/apache/ambari/pull/3702#discussion_r1247526948
##########
ambari-server/src/main/resources/stacks/BIGTOP/3.2.0/services/HBASE/package/scripts/hbase_decommission.py:
##########
@@ -57,32 +69,65 @@ def hbase_decommission(env):
if params.hbase_drain_only:
for host in hosts:
if host:
- regiondrainer_cmd = format(
- "{kinit_cmd} HBASE_SERVER_JAAS_OPTS=\"{master_security_config}\" {hbase_cmd} --config {hbase_conf_dir} {hbase_decommission_auth_config} org.jruby.Main {region_drainer} remove {host}")
- Execute(regiondrainer_cmd,
- user=params.hbase_user,
- logoutput=True
- )
- pass
+ try:
+ regiondrainer_cmd = format(
+ "{kinit_cmd} HBASE_SERVER_JAAS_OPTS=\"{master_security_config}\" {hbase_cmd} --config {hbase_conf_dir} {hbase_decommission_auth_config} org.jruby.Main {region_drainer} remove {host}")
+ Execute(regiondrainer_cmd,
+ user=params.hbase_user,
+ logoutput=True
+ )
+ pass
+ except Exception as e:
+ # Execute HBase 2 scripts if HBase 1 scripts fail.
+ # If the Exception is genuine, it will fail here because HBase 1 scripts work only for HBase 1
+ # and HBase 2 scripts work only for HBase 2 cluster.
+ Logger.info("HBase 1 RegionMover failed. Will try with HBase 2 RegionMover." + str(e))
+ regiondrainer_cmd = format(
+ "{kinit_cmd} HBASE_SERVER_JAAS_OPTS=\"{master_security_config}\" {hbase_cmd} --config {hbase_conf_dir} {hbase_decommission_auth_config} org.jruby.Main {region_drainer2} remove {host}")
+ Execute(regiondrainer_cmd,
+ user=params.hbase_user,
+ logoutput=True
+ )
+ pass
pass
else:
for host in hosts:
if host:
- regiondrainer_cmd = format(
- "{kinit_cmd} HBASE_SERVER_JAAS_OPTS=\"{master_security_config}\" {hbase_cmd} --config {hbase_conf_dir} {hbase_decommission_auth_config} org.jruby.Main {region_drainer} add {host}")
- regionmover_cmd = format(
- "{kinit_cmd} HBASE_SERVER_JAAS_OPTS=\"{master_security_config}\" {hbase_cmd} --config {hbase_conf_dir} {hbase_decommission_auth_config} org.jruby.Main {region_mover} -o unload -r {host}")
-
- Execute(regiondrainer_cmd,
- user=params.hbase_user,
- logoutput=True
- )
-
- Execute(regionmover_cmd,
- user=params.hbase_user,
- logoutput=True
- )
+ try:
+ regiondrainer_cmd = format(
+ "{kinit_cmd} HBASE_SERVER_JAAS_OPTS=\"{master_security_config}\" {hbase_cmd} --config {hbase_conf_dir} {hbase_decommission_auth_config} org.jruby.Main {region_drainer} add {host}")
+ regionmover_cmd = format(
+ "{kinit_cmd} HBASE_SERVER_JAAS_OPTS=\"{master_security_config}\" {hbase_cmd} --config {hbase_conf_dir} {hbase_decommission_auth_config} org.jruby.Main {region_mover} -m 24 unload {host}")
+
+ Execute(regiondrainer_cmd,
+ user=params.hbase_user,
+ logoutput=True
+ )
+
+ Execute(regionmover_cmd,
+ user=params.hbase_user,
+ logoutput=True
+ )
Review Comment:
Same comment above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
For additional commands, e-mail: dev-help@ambari.apache.org
[GitHub] [ambari] virajjasani merged pull request #3702: AMBARI-25949 regionserver graceful start/stop should be compatible for hbase 2
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani merged PR #3702:
URL: https://github.com/apache/ambari/pull/3702
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
For additional commands, e-mail: dev-help@ambari.apache.org
[GitHub] [ambari] virajjasani commented on a diff in pull request #3702: AMBARI-25949 regionserver graceful start/stop should be compatible for hbase 2
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on code in PR #3702:
URL: https://github.com/apache/ambari/pull/3702#discussion_r1248208396
##########
ambari-server/src/main/resources/stacks/BIGTOP/3.2.0/services/HBASE/package/scripts/hbase_regionserver.py:
##########
@@ -51,6 +55,57 @@ def configure(self, env):
def decommission(self, env):
print "Decommission not yet implemented!"
+ def graceful_stop(self, env, upgrade_type=None):
+ import params
+
+ # Mark Draining ZNode
+ params.hbase_drain_only = False
+ params.hbase_excluded_hosts = params.hostname
+ env.set_params(params)
+ hbase_decommission(env)
+
+ # Stop RegionServer
+ hbase_service('regionserver', action='stop')
+
+ # Remove from Draining ZNode to make host useable on restarting regionserver
+ params.hbase_drain_only = True
+ env.set_params(params)
+ hbase_decommission(env)
+
+ def graceful_start(self, env, upgrade_type=None):
+ import params
+ env.set_params(params)
+
+ #Start RegionServer
+ hbase_service('regionserver', action='start')
+
+ # Load Regions back
+ kinit_cmd = params.kinit_cmd_master
+ host = params.hostname
+
+ regionmover_cmd = format(
+ "{kinit_cmd} HBASE_SERVER_JAAS_OPTS=\"{master_security_config}\" {hbase_cmd} --config {hbase_conf_dir} {hbase_decommission_auth_config} org.jruby.Main {region_mover} -m 24 load {host}")
+
+ try:
+ Execute(regionmover_cmd,
+ user=params.hbase_user,
+ logoutput=True
+ )
+ except Exception as e:
+ Logger.info("HBase 1: region_mover failed while loading regions back to source RS." + str(e))
+ # Execute HBase 2 scripts if HBase 1 scripts fail.
+ # If the Exception is genuine, it will fail here because HBase 1 scripts work only for HBase 1
+ # and HBase 2 scripts work only for HBase 2 cluster.
+ try:
+ regionmover_cmd = format(
+ "{kinit_cmd} HBASE_SERVER_JAAS_OPTS=\"{master_security_config}\" {hbase_cmd} --config {hbase_conf_dir} {hbase_decommission_auth_config} org.jruby.Main {region_mover} -m 24 -o load -r {host}")
+ Execute(regionmover_cmd,
+ user=params.hbase_user,
+ logoutput=True
+ )
Review Comment:
thanks a lot for taking a look @timyuer
> Why don't we use HBase 2 scripts directly because HBase 1 scripts will fail in the current stack (BIGTOP-3.2.0)?
sure that also works. at least for branch-2.7, i was planning to provide this fallback but you are right that maybe we don't need fallback for trunk and we can directly use hbase 2 script only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
For additional commands, e-mail: dev-help@ambari.apache.org
[GitHub] [ambari] timyuer commented on a diff in pull request #3702: AMBARI-25949 regionserver graceful start/stop should be compatible for hbase 2
Posted by "timyuer (via GitHub)" <gi...@apache.org>.
timyuer commented on code in PR #3702:
URL: https://github.com/apache/ambari/pull/3702#discussion_r1247527157
##########
ambari-server/src/main/resources/stacks/BIGTOP/3.2.0/services/HBASE/package/scripts/hbase_decommission.py:
##########
@@ -57,32 +69,65 @@ def hbase_decommission(env):
if params.hbase_drain_only:
for host in hosts:
if host:
- regiondrainer_cmd = format(
- "{kinit_cmd} HBASE_SERVER_JAAS_OPTS=\"{master_security_config}\" {hbase_cmd} --config {hbase_conf_dir} {hbase_decommission_auth_config} org.jruby.Main {region_drainer} remove {host}")
- Execute(regiondrainer_cmd,
- user=params.hbase_user,
- logoutput=True
- )
- pass
+ try:
+ regiondrainer_cmd = format(
+ "{kinit_cmd} HBASE_SERVER_JAAS_OPTS=\"{master_security_config}\" {hbase_cmd} --config {hbase_conf_dir} {hbase_decommission_auth_config} org.jruby.Main {region_drainer} remove {host}")
+ Execute(regiondrainer_cmd,
+ user=params.hbase_user,
+ logoutput=True
+ )
+ pass
Review Comment:
Same comment above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
For additional commands, e-mail: dev-help@ambari.apache.org
[GitHub] [ambari] virajjasani commented on pull request #3702: AMBARI-25949 regionserver graceful start/stop should be compatible for hbase 2
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on PR #3702:
URL: https://github.com/apache/ambari/pull/3702#issuecomment-1656959848
incorporated the review comments
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
For additional commands, e-mail: dev-help@ambari.apache.org