Cannot (re)start ZooKeeper via zkServer.sh over PLINK

[Erhhung Yuan <er...@gmail.com>]

I have a working ZooKeeper 3.3.3 installed on a standard
32-bit Amazon EC2 Linux AMI, and the server starts fine
with the following line added to my /etc/rc.local script:

/var/lib/zookeeper/bin/zkServer.sh start

I'm managing the server over SSH on Windows over PuTTY,
and everything works wonderfully. However, I'd like to
automate basic admin tasks via scripts, so I use PLINK
to login and run the zkServer.sh script with restart:

plink.exe -t -I key.ppk ec2-user@ec-host
    sudo /var/lib/zookeeper/bin/zkServer.sh restart

PLINK connects fine using this command, and I get this
encouraging output:

JMX enabled by default
Using config: /var/lib/zookeeper/bin/../conf/zoo.cfg
JMX enabled by default
Using config: /var/lib/zookeeper/bin/../conf/zoo.cfg
Stopping zookeeper ...
/var/lib/zookeeper/bin/zkServer.sh: line 90: kill:
   (3877) - No such process
STOPPED
JMX enabled by default
Using config: /var/lib/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ...
STARTED

However, when PLINK connection ends right afterwards,
I don't see the java process running. I'm able to use
PLINK to manage many other services that are normally
controlled using the 'service' command, so is there
something I need to do different to run java from a
PLINK connection? The -t option is supposed to supply
pseudo TTY (sudo won't even work without a TTY), but
zkServer.sh works fine over a PuTTY session, after I
exit and reconnect.

Re: Cannot (re)start ZooKeeper via zkServer.sh over PLINK

[Erhhung Yuan <er...@gmail.com>]

Alright, I've figured out a solution on my own, though I haven't dug deep
enough to understand why a shell process created via PLINK causes child
processes to exit when itself exits even when they're run in the background.

As a added benefit, I've created an init.d script and registered it with
chkconfig, so I can just run ZooKeeper via the 'service' command. The key
change in the zkServer.sh script (which I have under
'/var/lib/zookeeper/bin) was running java using the daemon() function in
the init.d script library and combining it with the 'nohup' command so as
to not hang up on the child processes.

Here's the script that others may find useful. This was on a non-Debian
distro (basic Amazon EC2 Linux AMI, actually), so it may need minor
adjustments in Ubuntu. Create a symlink from '/etc/init.d/zookeeper' to
'/var/lib/zookeeper/bin/zkServer,sh' (or wherever you installed ZooKeeper),
and chmod it to 755, and then run 'chkconfig --add zookeeper' and
'chkconfig zookeeper on' to make it auto-run.

Erhhung

#!/bin/sh
#
# Startup/shutdown script for Apache ZooKeeper
#
# This file must stay as '/var/lib/zookeeper/bin/zkServer.sh'
# but can be symbolically linked from '/etc/init.d/zookeeper'
#
### BEGIN INIT INFO
# Provides: zookeeper
# Defalt-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Description: Apache ZooKeeper
### END INIT INFO

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version
# 2.0 (the "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# If this scripted is run out of /usr/bin or some other system bin
# directory it should be linked to and not copied. Things like java
# jar files are found relative to the canonical path of this script.

# Import function library
if [ -f /etc/init.d/functions ] ; then
    . /etc/init.d/functions
else
    exit 1
fi

# See the following page for extensive details on setting
# up the JVM to accept JMX remote management:
# http://java.sun.com/javase/6/docs/technotes/guides/management/agent.html
# by default we allow local JMX connections
if [ "x$JMXLOCALONLY" = "x" ] ; then
    JMXLOCALONLY=false
fi

if [ "x$JMXDISABLE" = "x" ] ; then
    echo "JMX enabled by default"
    # for some reason these two options are necessary on jdk6 on Ubuntu
    # accord to the docs they are not necessary, but otw jconsole cannot
    # do a local attach
    ZOOMAIN="-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.local.only=$JMXLOCALONLY
org.apache.zookeeper.server.quorum.QuorumPeerMain"
else
    echo "JMX disabled by user request"
    ZOOMAIN="org.apache.zookeeper.server.quorum.QuorumPeerMain"
fi

# Only follow symlinks if readlink supports
if readlink -f "$0" > /dev/null 2>&1 ; then
    ZOOBIN=`readlink -f "$0"`
else
    ZOOBIN="$0"
fi

ZOOBINDIR=`dirname "$ZOOBIN"`
. "$ZOOBINDIR"/zkEnv.sh

if [ "x$2" != "x" ] ; then
    ZOOCFG="$ZOOCFGDIR/$2"
fi

if $cygwin ; then
    ZOOCFG=`cygpath -wp "$ZOOCFG"`
    # cygwin has a "kill" in the shell itself, gets confused
    KILL=/bin/kill
else
    KILL=kill
fi

echo "Using config: $ZOOCFG"

ZOOPIDFILE=$(grep dataDir "$ZOOCFG" | sed -e 's/.*=//')/zookeeper_server.pid

case "$1" in
start)
    echo -n "Starting zookeeper: "
    daemon 'nohup java "-Dzookeeper.log.dir='${ZOO_LOG_DIR}'"
"-Dzookeeper.root.logger='${ZOO_LOG4J_PROP}'" -cp "'$CLASSPATH'"
'$JVMFLAGS' '$ZOOMAIN' "'$ZOOCFG'" > /dev/null &'

    pid=`pidofproc java`
    if [ -n "$pid" ] ; then
        echo $pid > "$ZOOPIDFILE"
        echo STARTED
    fi
    ;;
stop)
    echo -n "Stopping zookeeper: "
    if [ ! -f "$ZOOPIDFILE" ] ; then
        echo "ERROR: Could not find file $ZOOPIDFILE"
        exit 1
    else
        $KILL -9 $(cat "$ZOOPIDFILE")
        rm -f "$ZOOPIDFILE"
        echo STOPPED
    fi
    ;;
upgrade)
    shift
    echo "Upgrading the servers to 3.*"
    java "-Dzookeeper.log.dir=${ZOO_LOG_DIR}"
"-Dzookeeper.root.logger=${ZOO_LOG4J_PROP}" -cp "$CLASSPATH" $JVMFLAGS
org.apache.zookeeper.server.upgrade.UpgradeMain ${@}
    echo "Upgrading ... "
    ;;
restart)
    shift
    "$0" stop ${@}
    sleep 3
    "$0" start ${@}
    ;;
status)
    STAT=`echo stat | nc localhost $(grep clientPort "$ZOOCFG" | sed -e
's/.*=//') 2> /dev/null | grep Mode`
    if [ "x$STAT" = "x" ] ; then
        echo "Error contacting service. It's probably not running."
    else
        echo $STAT
    fi
    ;;
*)
    echo "Usage: $0 {start|stop|restart|status}" >&2
esac


On Wed, Feb 1, 2012 at 5:13 PM, Erhhung Yuan <er...@gmail.com> wrote:

> I have a working ZooKeeper 3.3.3 installed on a standard
> 32-bit Amazon EC2 Linux AMI, and the server starts fine
> with the following line added to my /etc/rc.local script:
>
> /var/lib/zookeeper/bin/zkServer.sh start
>
> I'm managing the server over SSH on Windows over PuTTY,
> and everything works wonderfully. However, I'd like to
> automate basic admin tasks via scripts, so I use PLINK
> to login and run the zkServer.sh script with restart:
>
> plink.exe -t -I key.ppk ec2-user@ec-host
>    sudo /var/lib/zookeeper/bin/zkServer.sh restart
>
> PLINK connects fine using this command, and I get this
> encouraging output:
>
> JMX enabled by default
> Using config: /var/lib/zookeeper/bin/../conf/zoo.cfg
> JMX enabled by default
> Using config: /var/lib/zookeeper/bin/../conf/zoo.cfg
> Stopping zookeeper ...
> /var/lib/zookeeper/bin/zkServer.sh: line 90: kill:
>   (3877) - No such process
> STOPPED
> JMX enabled by default
> Using config: /var/lib/zookeeper/bin/../conf/zoo.cfg
> Starting zookeeper ...
> STARTED
>
> However, when PLINK connection ends right afterwards,
> I don't see the java process running. I'm able to use
> PLINK to manage many other services that are normally
> controlled using the 'service' command, so is there
> something I need to do different to run java from a
> PLINK connection? The -t option is supposed to supply
> pseudo TTY (sudo won't even work without a TTY), but
> zkServer.sh works fine over a PuTTY session, after I
> exit and reconnect.
>
>
>