svn commit: r1294115 - /hbase/branches/0.89-fb/bin/hbase-daemon.sh

[mb...@apache.org]

Author: mbautin
Date: Mon Feb 27 11:37:59 2012
New Revision: 1294115

URL: http://svn.apache.org/viewvc?rev=1294115&view=rev
Log:
[master] porting "ghost pid" issue to hbase-daemon.sh

Summary:
Much like the already-corrected issue in hadoop-daemon.sh, we sometimes see
false alarms for hbase processes when another thread responds to kill -0 $pid --
this can lead to clusters starting up slower than ideal.

hadoop-daemon.sh patch where I stole this from:
https://phabricator.fb.com/D350607

I would like someone from hbase-eng to shepherd this patch to the various other
places it needs to be committed and take care of any merging to the open source
processes.

prefixign with [master] as per kannan

Test Plan:
found a host which was exhibiting this behaviour, applied the new .sh file

[root@hbase233.ash3 ~]# hadoopctl start
INFO: start SEARCHHBASE002-ASH3-HBASE:regionserver (timeout=180s)
WARN: Non-zero code 1 received from start script.
      Stdout:
          regionserver running as process 4735. Stop it first.
      Stderr:

Modified:
    hbase/branches/0.89-fb/bin/hbase-daemon.sh

Modified: hbase/branches/0.89-fb/bin/hbase-daemon.sh
URL: http://svn.apache.org/viewvc/hbase/branches/0.89-fb/bin/hbase-daemon.sh?rev=1294115&r1=1294114&r2=1294115&view=diff
==============================================================================
--- hbase/branches/0.89-fb/bin/hbase-daemon.sh (original)
+++ hbase/branches/0.89-fb/bin/hbase-daemon.sh Mon Feb 27 11:37:59 2012
@@ -131,8 +131,17 @@ case $startStop in
     mkdir -p "$HBASE_PID_DIR"
     if [ -f $pid ]; then
       if kill -0 `cat $pid` > /dev/null 2>&1; then
-        echo $command running as process `cat $pid`.  Stop it first.
-        exit 1
+        # On Linux, process pids and thread pids are indistinguishable to
+        # signals. It's possible that the pid in our pidfile is now a thread
+        # owned by another process. Let's check to make sure our pid is
+        # actually a running process.
+        ps -e -o pid | egrep "^`cat $pid`$" >/dev/null 2>&1
+        if [ $? -eq 0 ]; then
+          echo $command running as process `cat $pid`.  Stop it first.
+          exit 1
+        else
+          rm $pid
+        fi
       fi
     fi