You are viewing a plain text version of this content. The canonical link for it is here.
Posted to codereview@trafodion.apache.org by narendragoyal <gi...@git.apache.org> on 2016/03/25 02:54:52 UTC
[GitHub] incubator-trafodion pull request: [TRAFODION-1617]
GitHub user narendragoyal opened a pull request:
https://github.com/apache/incubator-trafodion/pull/402
[TRAFODION-1617]
Add a few convenience scripts (and associated helper scripts)
Note: All the scripts with an "ilh_" prefix work if one has used install_local_hadoop
ilh_trafinit: ckillall, restarts HBase(clobbers HBase data), sqstart and initializes trafodion
ilh_traf_restart: ckillall, restarts HBase, sqstart
trafnodestatus: Prints the trafodion-status (whether Trafodion is UP or DOWN on a node)
of the nodes in the Trafodion cluster. Has a '-j' switch to print the
output in JSON format.
Note: There is another script 'sqnodestatus' with a similar output but that
prints the status of the nodes from a Linux perspective.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/narendragoyal/incubator-trafodion master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-trafodion/pull/402.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #402
----
commit 79670d0b2784618e38fdc750a4aec47fdf5a7047
Author: Narendra Goyal <na...@esgyn.com>
Date: 2016-03-25T01:39:46Z
[TRAFODION-1617]
Add a few convenience scripts (and associated helper scripts)
Note: All the scripts with an "ilh_" prefix work if one has used install_local_hadoop
ilh_trafinit: ckillall, restarts HBase(clobbers HBase data), sqstart and initializes trafodion
ilh_traf_restart: ckillall, restarts HBase, sqstart
trafnodestatus: Prints the trafodion-status (whether Trafodion is UP or DOWN on a node)
of the nodes in the Trafodion cluster. Has a '-j' switch to print the
output in JSON format.
Note: There is another script 'sqnodestatus' with a similar output but that
prints the status of the nodes from a Linux perspective.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: [TRAFODION-1617]
Posted by narendragoyal <gi...@git.apache.org>.
Github user narendragoyal commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/402#discussion_r57468441
--- Diff: core/sqf/sql/scripts/ilh_traf_restart ---
@@ -0,0 +1,117 @@
+#!/bin/bash
+# @@@ START COPYRIGHT @@@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# @@@ END COPYRIGHT @@@
+
+####################################################
+#
+# Executes the following:
+#
+# - Kills the Trafodion Env
+# - Kills the HBase Master process and the zookeeper data
+# (uses ilh_cleanhb 1)
+# - Restarts HBase
+# - Repairs HBase (ilh_hbase_repair)
+# - Checks whether HBase is ready (uses ilh_hbcheck)
+# - Deletes the Trafodion env file: $MY_SQROOT/etc/ms.env
+# - Regenerates Config (sqgen)
+# - Starts up Trafodion (sqstart)
+#
+####################################################
+
+if [ ! -z $MY_SQROOT ]; then
+ cd $MY_SQROOT/sql/scripts
+else
+ echo "The environment variable MY_SQROOT does not exist."
+ echo "Please ensure sqenv.sh has been sourced."
+ echo
+ exit 1;
+fi
+
+lv_ilh_cleanhb_mode=1
+
+echo 'This script will do the following:'
+echo ' - Kill the Trafodion env (pkillall)'
+echo " - Kill the HBase env and delete the zookeeper data directory (ilh_cleanhb ${lv_ilh_cleanhb_mode})"
+echo ' - Restart HBase (swstarthbase)'
+echo ' - Delete the env file: $MY_SQROOT/etc/ms.env'
+echo ' - sqgen'
+echo ' - sqstart'
+
+if [[ -z $1 ]]; then
+ read -p 'Do you want to proceed? y/n : ' lv_ans
+else
+ lv_ans=$1
+fi
+
+if [ "$lv_ans" != "y" ]; then
+ echo "Not proceeding. Exitting..."
+ exit 0
+else
+ echo "Ok...going ahead..."
+fi
+
+lv_starttime=`date`
+
+run_util pkillall
+sleep 5
+run_util pkillall
+
+echo "Stopping HBase"
+swstophbase &
+sleep 45
+kill %1
+
+run_util 'ilh_cleanhb 1'
+
+run_util swstarthbase
+
+echo "Sleeping for 15 seconds before checking the status of HBase"
+sleep 15
+
+echo "--------------------------------------"
+if [ -e $MY_SQROOT/sql/scripts/hbcheck ]; then
+ run_util hbcheck 4 10
+else
+ run_util ilh_hbcheck
+fi
+echo "--------------------------------------"
+
+echo "Repairing HBase (hbase hbck -repair)"
+ilh_hbase_repair y
+
--- End diff --
On a single node env (with HMaster, RegionServer, Zookeeper in a single process), I saw instances where the HMaster would get stuck at startup. Restarting HBase multiple times wouldn't help but executing 'hbase hbck -repair' somehow took care of the issue. I am not sure what it cleaned up in the HBase space.
So, I wrapped this in a little script. Currently, I would not suggest that 'ilh_hbase_repair' be used in a clustered/production env by itself.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: [TRAFODION-1617]
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-trafodion/pull/402
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: [TRAFODION-1617]
Posted by amandamoran <gi...@git.apache.org>.
Github user amandamoran commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/402#discussion_r57464316
--- Diff: core/sqf/sql/scripts/ilh_traf_restart ---
@@ -0,0 +1,117 @@
+#!/bin/bash
+# @@@ START COPYRIGHT @@@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# @@@ END COPYRIGHT @@@
+
+####################################################
+#
+# Executes the following:
+#
+# - Kills the Trafodion Env
+# - Kills the HBase Master process and the zookeeper data
+# (uses ilh_cleanhb 1)
+# - Restarts HBase
+# - Repairs HBase (ilh_hbase_repair)
+# - Checks whether HBase is ready (uses ilh_hbcheck)
+# - Deletes the Trafodion env file: $MY_SQROOT/etc/ms.env
+# - Regenerates Config (sqgen)
+# - Starts up Trafodion (sqstart)
+#
+####################################################
+
+if [ ! -z $MY_SQROOT ]; then
+ cd $MY_SQROOT/sql/scripts
+else
+ echo "The environment variable MY_SQROOT does not exist."
+ echo "Please ensure sqenv.sh has been sourced."
+ echo
+ exit 1;
+fi
+
+lv_ilh_cleanhb_mode=1
+
+echo 'This script will do the following:'
+echo ' - Kill the Trafodion env (pkillall)'
+echo " - Kill the HBase env and delete the zookeeper data directory (ilh_cleanhb ${lv_ilh_cleanhb_mode})"
+echo ' - Restart HBase (swstarthbase)'
+echo ' - Delete the env file: $MY_SQROOT/etc/ms.env'
+echo ' - sqgen'
+echo ' - sqstart'
+
+if [[ -z $1 ]]; then
+ read -p 'Do you want to proceed? y/n : ' lv_ans
+else
+ lv_ans=$1
+fi
+
+if [ "$lv_ans" != "y" ]; then
+ echo "Not proceeding. Exitting..."
+ exit 0
+else
+ echo "Ok...going ahead..."
+fi
+
+lv_starttime=`date`
+
+run_util pkillall
+sleep 5
+run_util pkillall
+
+echo "Stopping HBase"
+swstophbase &
+sleep 45
+kill %1
+
+run_util 'ilh_cleanhb 1'
+
+run_util swstarthbase
+
+echo "Sleeping for 15 seconds before checking the status of HBase"
+sleep 15
+
+echo "--------------------------------------"
+if [ -e $MY_SQROOT/sql/scripts/hbcheck ]; then
+ run_util hbcheck 4 10
+else
+ run_util ilh_hbcheck
+fi
+echo "--------------------------------------"
+
+echo "Repairing HBase (hbase hbck -repair)"
+ilh_hbase_repair y
+
--- End diff --
Can you explain, why a hbase -repair is needed? Is this only needed for install_local_hadoop but not in a clusters environment?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---