You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by ya...@apache.org on 2014/06/25 02:04:40 UTC

git commit: updated refs/heads/4.4-forward to a6381f5

Repository: cloudstack
Updated Branches:
  refs/heads/4.4-forward 5cf8edd7e -> a6381f580


CLOUDSTACK-6989: Add 3 strikes rule for RvR freezing detection

Sometime in VR ntpd would move time backward to keep sync with NTP server, which
can result in false alarm of keepalived monitering process.

This patch adds 3 strikes for keepalived process dead detection to avoid falsely
shutdown keepalived process due to time adjustment for only once.


Project: http://git-wip-us.apache.org/repos/asf/cloudstack/repo
Commit: http://git-wip-us.apache.org/repos/asf/cloudstack/commit/a6381f58
Tree: http://git-wip-us.apache.org/repos/asf/cloudstack/tree/a6381f58
Diff: http://git-wip-us.apache.org/repos/asf/cloudstack/diff/a6381f58

Branch: refs/heads/4.4-forward
Commit: a6381f5809253c4fe2a2a6ac932a4d8861de8de2
Parents: 5cf8edd
Author: Sheng Yang <sh...@citrix.com>
Authored: Tue Jun 24 16:58:55 2014 -0700
Committer: Sheng Yang <sh...@citrix.com>
Committed: Tue Jun 24 16:58:55 2014 -0700

----------------------------------------------------------------------
 .../root/redundant_router/check_heartbeat.sh.templ   | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/cloudstack/blob/a6381f58/systemvm/patches/debian/config/root/redundant_router/check_heartbeat.sh.templ
----------------------------------------------------------------------
diff --git a/systemvm/patches/debian/config/root/redundant_router/check_heartbeat.sh.templ b/systemvm/patches/debian/config/root/redundant_router/check_heartbeat.sh.templ
index 95cabd6b..d6bdc5d 100755
--- a/systemvm/patches/debian/config/root/redundant_router/check_heartbeat.sh.templ
+++ b/systemvm/patches/debian/config/root/redundant_router/check_heartbeat.sh.templ
@@ -17,13 +17,28 @@
 # under the License.
 
 
+STRIKE_FILE="[RROUTER_BIN_PATH]/keepalived.strikes"
+
 if [ -e [RROUTER_BIN_PATH]/keepalived.ts2 ]
 then
     lasttime=$(cat [RROUTER_BIN_PATH]/keepalived.ts2)
     thistime=$(cat [RROUTER_BIN_PATH]/keepalived.ts)
     diff=$(($thistime - $lasttime))
+    s=0
     if [ $diff -lt 30 ]
     then
+        if [ -e $STRIKE_FILE ]
+        then
+            s=`cat $STRIKE_FILE 2>/dev/null`
+        fi
+        s=$(($s+1))
+        echo $s > $STRIKE_FILE
+    else
+        rm $STRIKE_FILE
+    fi
+    #3 strikes rule
+    if [ $s -gt 2 ]
+    then
         echo Keepalived process is dead! >> [RROUTER_LOG]
         [RROUTER_BIN_PATH]/services.sh stop >> [RROUTER_LOG] 2>&1
         [RROUTER_BIN_PATH]/disable_pubip.sh >> [RROUTER_LOG] 2>&1