You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by 刘明敏 <di...@gmail.com> on 2012/07/20 05:12:50 UTC
one broker fail to re-join the cluster after been killed
In my kafka cluster ,there are 2 brokers,
One of them was killed by accident yesterday, after that ,I tried to
restart the broker,but seems that it failed to join the cluster.
In zookeeper,it successfully register itself,but after sending logs, no
data show up in kafka data directory
This is the log of the broker after restart it:
[2012-07-19 17:59:21,154] INFO The number of partitions for topic
no_appkey : 1 (kafka.utils.Utils$)
[2012-07-19 17:59:21,156] INFO The number of partitions for topic
parse_exception : 1 (kafka.utils.Utils$)
[2012-07-19 17:59:21,160] INFO Starting Kafka server...
(kafka.server.KafkaServer)
[2012-07-19 17:59:21,171] INFO starting log cleaner every 60000 ms
(kafka.log.LogManager)
[2012-07-19 17:59:21,178] INFO connecting to ZK: XX.XX.XX.XX:2181/kafka
(kafka.server.KafkaZooKeeper)
[2012-07-19 17:59:21,187] INFO Starting ZkClient event thread.
(org.I0Itec.zkclient.ZkEventThread)
[2012-07-19 17:59:21,193] INFO Client
environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT
(org.apache.zookeeper.ZooKeeper)
[2012-07-19 17:59:21,193] INFO Client environment:host.name=mobile-1
(org.apache.zookeeper.ZooKeeper)
[2012-07-19 17:59:21,193] INFO Client environment:java.version=1.6.0_17
(org.apache.zookeeper.ZooKeeper)
[2012-07-19 17:59:21,193] INFO Client environment:java.vendor=Sun
Microsystems Inc. (org.apache.zookeeper.ZooKeeper)
[2012-07-19 17:59:21,193] INFO Client
environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre
(org.apache.zookeeper.ZooKeeper)
[2012-07-19 17:59:21,193] INFO Client
environment:java.class.path=:/home/OUR-SERVER/kafka/libs/jopt-simple-3.2.jar:/home/OUR-SERVER/kafka/libs/log4j-1.2.15.jar:/home/OUR-SERVER/kafka/libs/scala-compiler.jar:/home/OUR-SERVER/kafka/libs/scala-library.jar:/home/OUR-SERVER/kafka/libs/snappy-java-1.0.4.1.jar:/home/OUR-SERVER/kafka/libs/zkclient-0.1.jar:/home/OUR-SERVER/kafka/libs/zookeeper-3.3.4.jar:/home/OUR-SERVER/kafka/kafka-trunk-2a59ad76c6_scala-2.8.0.jar
(org.apache.zookeeper.ZooKeeper)
[2012-07-19 17:59:21,193] INFO Client
environment:java.library.path=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
(org.apache.zookeeper.ZooKeeper)
[2012-07-19 17:59:21,193] INFO Client environment:java.io.tmpdir=/tmp
(org.apache.zookeeper.ZooKeeper)
[2012-07-19 17:59:21,193] INFO Client environment:java.compiler=<NA>
(org.apache.zookeeper.ZooKeeper)
[2012-07-19 17:59:21,193] INFO Client environment:os.name=Linux
(org.apache.zookeeper.ZooKeeper)
[2012-07-19 17:59:21,193] INFO Client environment:os.arch=amd64
(org.apache.zookeeper.ZooKeeper)
[2012-07-19 17:59:21,193] INFO Client
environment:os.version=2.6.32-71.29.1.el6.x86_64
(org.apache.zookeeper.ZooKeeper)
[2012-07-19 17:59:21,193] INFO Client environment:user.name=OUR-SERVER
(org.apache.zookeeper.ZooKeeper)
[2012-07-19 17:59:21,193] INFO Client
environment:user.home=/home/OUR-SERVER (org.apache.zookeeper.ZooKeeper)
[2012-07-19 17:59:21,193] INFO Client
environment:user.dir=/home/OUR-SERVER/kafka (org.apache.zookeeper.ZooKeeper)
[2012-07-19 17:59:21,194] INFO Initiating client connection,
connectString=XX.XX.XX.XX:2181/kafka sessionTimeout=6000
watcher=org.I0Itec.zkclient.ZkClient@4a48edb5(org.apache.zookeeper.ZooKeeper)
[2012-07-19 17:59:21,211] INFO Opening socket connection to server
/XX.XX.XX.XX:2181 (org.apache.zookeeper.ClientCnxn)
[2012-07-19 17:59:21,215] INFO Socket connection established to
mobile-1/XX.XX.XX.XX:2181, initiating session
(org.apache.zookeeper.ClientCnxn)
[2012-07-19 17:59:21,280] INFO Session establishment complete on server
mobile-1/XX.XX.XX.XX:2181, sessionid = 0x1389e95409a0023, negotiated
timeout = 6000 (org.apache.zookeeper.ClientCnxn)
[2012-07-19 17:59:21,282] INFO zookeeper state changed (SyncConnected)
(org.I0Itec.zkclient.ZkClient)
[2012-07-19 17:59:21,380] INFO Awaiting connections on port 9092
(kafka.network.Acceptor)
[2012-07-19 17:59:21,382] INFO Will not load MX4J, mx4j-tools.jar is not in
the classpath (kafka.utils.Mx4jLoader$)
[2012-07-19 17:59:21,383] INFO Registering broker /brokers/ids/1
(kafka.server.KafkaZooKeeper)
[2012-07-19 17:59:21,404] INFO Registering broker /brokers/ids/1 succeeded
with id:1,creatorId:XX.XX.XX.XX-1342691961383,host:XX.XX.XX.XX,port:9092
(kafka.server.KafkaZooKeeper)
[2012-07-19 17:59:21,417] INFO Starting log flusher every 1000 ms with the
following overrides Map() (kafka.log.LogManager)
[2012-07-19 17:59:21,419] INFO Kafka server started.
(kafka.server.KafkaServer)
This is the error from producer:
ERR [20120719-17:53:56.312] exception: Server Exception. : Event queue is
full of unsent messages, could not send event: {OUR-LOG}
ERR [20120719-17:53:56.312] exception:
kafka.producer.async.AsyncProducer.send(AsyncProducer.scala:110)
ERR [20120719-17:53:56.312] exception:
kafka.producer.ProducerPool$$anonfun$send$1$$anonfun$apply$mcVI$sp$5$$anonfun$apply$2.apply(ProducerPool.scala:129)
ERR [20120719-17:53:56.312] exception:
kafka.producer.ProducerPool$$anonfun$send$1$$anonfun$apply$mcVI$sp$5$$anonfun$apply$2.apply(ProducerPool.scala:129)
ERR [20120719-17:53:56.312] exception:
scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
ERR [20120719-17:53:56.312] exception:
scala.collection.immutable.List.foreach(List.scala:45)
ERR [20120719-17:53:56.312] exception:
kafka.producer.ProducerPool$$anonfun$send$1$$anonfun$apply$mcVI$sp$5.apply(ProducerPool.scala:129)
ERR [20120719-17:53:56.312] exception:
kafka.producer.ProducerPool$$anonfun$send$1$$anonfun$apply$mcVI$sp$5.apply(ProducerPool.scala:128)
ERR [20120719-17:53:56.312] exception:
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:60)
ERR [20120719-17:53:56.312] exception:
scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:44)
ERR [20120719-17:53:56.312] exception:
kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:128)
ERR [20120719-17:53:56.312] exception:
kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:100)
ERR [20120719-17:53:56.312] exception:
kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:100)
ERR [20120719-17:53:56.312] exception:
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:60)
ERR [20120719-17:53:56.312] exception:
scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:44)
ERR [20120719-17:53:56.312] exception:
kafka.producer.ProducerPool.send(ProducerPool.scala:100)
ERR [20120719-17:53:56.312] exception:
kafka.producer.Producer.zkSend(Producer.scala:137)
ERR [20120719-17:53:56.312] exception:
kafka.producer.Producer.send(Producer.scala:99)
I intentionally shut down the other broker,make the restarted broker the
only one in the cluster,seems that the producer can not find the broker and
not sending any events
--
Best Regards
----------------------
刘明敏 | mmLiu
Re: one broker fail to re-join the cluster after been killed
Posted by 刘明敏 <di...@gmail.com>.
1.the version of kafka we use is trunk 2a59ad76c6
2.I file a JIRA here:https://issues.apache.org/jira/browse/KAFKA-410 with
broker and producer log
On Fri, Jul 20, 2012 at 1:42 PM, Neha Narkhede <ne...@gmail.com>wrote:
> Couple of questions -
>
> 1. Which version of Kafka are you using ?
> 2. Can you please file a JIRA and attach the full producer and broker log
> at TRACE level ?
>
> Thanks,
> Neha
>
> On Thu, Jul 19, 2012 at 8:12 PM, 刘明敏 <di...@gmail.com> wrote:
>
> > In my kafka cluster ,there are 2 brokers,
> >
> > One of them was killed by accident yesterday, after that ,I tried to
> > restart the broker,but seems that it failed to join the cluster.
> >
> > In zookeeper,it successfully register itself,but after sending logs, no
> > data show up in kafka data directory
> >
> > This is the log of the broker after restart it:
> >
> > [2012-07-19 17:59:21,154] INFO The number of partitions for topic
> > no_appkey : 1 (kafka.utils.Utils$)
> > [2012-07-19 17:59:21,156] INFO The number of partitions for topic
> > parse_exception : 1 (kafka.utils.Utils$)
> > [2012-07-19 17:59:21,160] INFO Starting Kafka server...
> > (kafka.server.KafkaServer)
> > [2012-07-19 17:59:21,171] INFO starting log cleaner every 60000 ms
> > (kafka.log.LogManager)
> > [2012-07-19 17:59:21,178] INFO connecting to ZK: XX.XX.XX.XX:2181/kafka
> > (kafka.server.KafkaZooKeeper)
> > [2012-07-19 17:59:21,187] INFO Starting ZkClient event thread.
> > (org.I0Itec.zkclient.ZkEventThread)
> > [2012-07-19 17:59:21,193] INFO Client
> > environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47
> GMT
> > (org.apache.zookeeper.ZooKeeper)
> > [2012-07-19 17:59:21,193] INFO Client environment:host.name=mobile-1
> > (org.apache.zookeeper.ZooKeeper)
> > [2012-07-19 17:59:21,193] INFO Client environment:java.version=1.6.0_17
> > (org.apache.zookeeper.ZooKeeper)
> > [2012-07-19 17:59:21,193] INFO Client environment:java.vendor=Sun
> > Microsystems Inc. (org.apache.zookeeper.ZooKeeper)
> > [2012-07-19 17:59:21,193] INFO Client
> > environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre
> > (org.apache.zookeeper.ZooKeeper)
> > [2012-07-19 17:59:21,193] INFO Client
> >
> >
> environment:java.class.path=:/home/OUR-SERVER/kafka/libs/jopt-simple-3.2.jar:/home/OUR-SERVER/kafka/libs/log4j-1.2.15.jar:/home/OUR-SERVER/kafka/libs/scala-compiler.jar:/home/OUR-SERVER/kafka/libs/scala-library.jar:/home/OUR-SERVER/kafka/libs/snappy-java-1.0.4.1.jar:/home/OUR-SERVER/kafka/libs/zkclient-0.1.jar:/home/OUR-SERVER/kafka/libs/zookeeper-3.3.4.jar:/home/OUR-SERVER/kafka/kafka-trunk-2a59ad76c6_scala-2.8.0.jar
> > (org.apache.zookeeper.ZooKeeper)
> > [2012-07-19 17:59:21,193] INFO Client
> >
> >
> environment:java.library.path=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> > (org.apache.zookeeper.ZooKeeper)
> > [2012-07-19 17:59:21,193] INFO Client environment:java.io.tmpdir=/tmp
> > (org.apache.zookeeper.ZooKeeper)
> > [2012-07-19 17:59:21,193] INFO Client environment:java.compiler=<NA>
> > (org.apache.zookeeper.ZooKeeper)
> > [2012-07-19 17:59:21,193] INFO Client environment:os.name=Linux
> > (org.apache.zookeeper.ZooKeeper)
> > [2012-07-19 17:59:21,193] INFO Client environment:os.arch=amd64
> > (org.apache.zookeeper.ZooKeeper)
> > [2012-07-19 17:59:21,193] INFO Client
> > environment:os.version=2.6.32-71.29.1.el6.x86_64
> > (org.apache.zookeeper.ZooKeeper)
> > [2012-07-19 17:59:21,193] INFO Client environment:user.name=OUR-SERVER
> > (org.apache.zookeeper.ZooKeeper)
> > [2012-07-19 17:59:21,193] INFO Client
> > environment:user.home=/home/OUR-SERVER (org.apache.zookeeper.ZooKeeper)
> > [2012-07-19 17:59:21,193] INFO Client
> > environment:user.dir=/home/OUR-SERVER/kafka
> > (org.apache.zookeeper.ZooKeeper)
> > [2012-07-19 17:59:21,194] INFO Initiating client connection,
> > connectString=XX.XX.XX.XX:2181/kafka sessionTimeout=6000
> > watcher=org.I0Itec.zkclient.ZkClient@4a48edb5
> > (org.apache.zookeeper.ZooKeeper)
> > [2012-07-19 17:59:21,211] INFO Opening socket connection to server
> > /XX.XX.XX.XX:2181 (org.apache.zookeeper.ClientCnxn)
> > [2012-07-19 17:59:21,215] INFO Socket connection established to
> > mobile-1/XX.XX.XX.XX:2181, initiating session
> > (org.apache.zookeeper.ClientCnxn)
> > [2012-07-19 17:59:21,280] INFO Session establishment complete on server
> > mobile-1/XX.XX.XX.XX:2181, sessionid = 0x1389e95409a0023, negotiated
> > timeout = 6000 (org.apache.zookeeper.ClientCnxn)
> > [2012-07-19 17:59:21,282] INFO zookeeper state changed (SyncConnected)
> > (org.I0Itec.zkclient.ZkClient)
> > [2012-07-19 17:59:21,380] INFO Awaiting connections on port 9092
> > (kafka.network.Acceptor)
> > [2012-07-19 17:59:21,382] INFO Will not load MX4J, mx4j-tools.jar is not
> in
> > the classpath (kafka.utils.Mx4jLoader$)
> > [2012-07-19 17:59:21,383] INFO Registering broker /brokers/ids/1
> > (kafka.server.KafkaZooKeeper)
> > [2012-07-19 17:59:21,404] INFO Registering broker /brokers/ids/1
> succeeded
> > with id:1,creatorId:XX.XX.XX.XX-1342691961383,host:XX.XX.XX.XX,port:9092
> > (kafka.server.KafkaZooKeeper)
> > [2012-07-19 17:59:21,417] INFO Starting log flusher every 1000 ms with
> the
> > following overrides Map() (kafka.log.LogManager)
> > [2012-07-19 17:59:21,419] INFO Kafka server started.
> > (kafka.server.KafkaServer)
> >
> >
> > This is the error from producer:
> >
> > ERR [20120719-17:53:56.312] exception: Server Exception. : Event queue is
> > full of unsent messages, could not send event: {OUR-LOG}
> > ERR [20120719-17:53:56.312] exception:
> > kafka.producer.async.AsyncProducer.send(AsyncProducer.scala:110)
> > ERR [20120719-17:53:56.312] exception:
> >
> >
> kafka.producer.ProducerPool$$anonfun$send$1$$anonfun$apply$mcVI$sp$5$$anonfun$apply$2.apply(ProducerPool.scala:129)
> > ERR [20120719-17:53:56.312] exception:
> >
> >
> kafka.producer.ProducerPool$$anonfun$send$1$$anonfun$apply$mcVI$sp$5$$anonfun$apply$2.apply(ProducerPool.scala:129)
> > ERR [20120719-17:53:56.312] exception:
> >
> >
> scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
> > ERR [20120719-17:53:56.312] exception:
> > scala.collection.immutable.List.foreach(List.scala:45)
> > ERR [20120719-17:53:56.312] exception:
> >
> >
> kafka.producer.ProducerPool$$anonfun$send$1$$anonfun$apply$mcVI$sp$5.apply(ProducerPool.scala:129)
> > ERR [20120719-17:53:56.312] exception:
> >
> >
> kafka.producer.ProducerPool$$anonfun$send$1$$anonfun$apply$mcVI$sp$5.apply(ProducerPool.scala:128)
> > ERR [20120719-17:53:56.312] exception:
> >
> >
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:60)
> > ERR [20120719-17:53:56.312] exception:
> > scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:44)
> > ERR [20120719-17:53:56.312] exception:
> >
> >
> kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:128)
> > ERR [20120719-17:53:56.312] exception:
> > kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:100)
> > ERR [20120719-17:53:56.312] exception:
> > kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:100)
> > ERR [20120719-17:53:56.312] exception:
> >
> >
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:60)
> > ERR [20120719-17:53:56.312] exception:
> > scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:44)
> > ERR [20120719-17:53:56.312] exception:
> > kafka.producer.ProducerPool.send(ProducerPool.scala:100)
> > ERR [20120719-17:53:56.312] exception:
> > kafka.producer.Producer.zkSend(Producer.scala:137)
> > ERR [20120719-17:53:56.312] exception:
> > kafka.producer.Producer.send(Producer.scala:99)
> >
> > I intentionally shut down the other broker,make the restarted broker the
> > only one in the cluster,seems that the producer can not find the broker
> and
> > not sending any events
> >
> > --
> > Best Regards
> >
> > ----------------------
> > 刘明敏 | mmLiu
> >
>
--
Best Regards
----------------------
刘明敏 | mmLiu
Re: one broker fail to re-join the cluster after been killed
Posted by Neha Narkhede <ne...@gmail.com>.
Couple of questions -
1. Which version of Kafka are you using ?
2. Can you please file a JIRA and attach the full producer and broker log
at TRACE level ?
Thanks,
Neha
On Thu, Jul 19, 2012 at 8:12 PM, 刘明敏 <di...@gmail.com> wrote:
> In my kafka cluster ,there are 2 brokers,
>
> One of them was killed by accident yesterday, after that ,I tried to
> restart the broker,but seems that it failed to join the cluster.
>
> In zookeeper,it successfully register itself,but after sending logs, no
> data show up in kafka data directory
>
> This is the log of the broker after restart it:
>
> [2012-07-19 17:59:21,154] INFO The number of partitions for topic
> no_appkey : 1 (kafka.utils.Utils$)
> [2012-07-19 17:59:21,156] INFO The number of partitions for topic
> parse_exception : 1 (kafka.utils.Utils$)
> [2012-07-19 17:59:21,160] INFO Starting Kafka server...
> (kafka.server.KafkaServer)
> [2012-07-19 17:59:21,171] INFO starting log cleaner every 60000 ms
> (kafka.log.LogManager)
> [2012-07-19 17:59:21,178] INFO connecting to ZK: XX.XX.XX.XX:2181/kafka
> (kafka.server.KafkaZooKeeper)
> [2012-07-19 17:59:21,187] INFO Starting ZkClient event thread.
> (org.I0Itec.zkclient.ZkEventThread)
> [2012-07-19 17:59:21,193] INFO Client
> environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT
> (org.apache.zookeeper.ZooKeeper)
> [2012-07-19 17:59:21,193] INFO Client environment:host.name=mobile-1
> (org.apache.zookeeper.ZooKeeper)
> [2012-07-19 17:59:21,193] INFO Client environment:java.version=1.6.0_17
> (org.apache.zookeeper.ZooKeeper)
> [2012-07-19 17:59:21,193] INFO Client environment:java.vendor=Sun
> Microsystems Inc. (org.apache.zookeeper.ZooKeeper)
> [2012-07-19 17:59:21,193] INFO Client
> environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre
> (org.apache.zookeeper.ZooKeeper)
> [2012-07-19 17:59:21,193] INFO Client
>
> environment:java.class.path=:/home/OUR-SERVER/kafka/libs/jopt-simple-3.2.jar:/home/OUR-SERVER/kafka/libs/log4j-1.2.15.jar:/home/OUR-SERVER/kafka/libs/scala-compiler.jar:/home/OUR-SERVER/kafka/libs/scala-library.jar:/home/OUR-SERVER/kafka/libs/snappy-java-1.0.4.1.jar:/home/OUR-SERVER/kafka/libs/zkclient-0.1.jar:/home/OUR-SERVER/kafka/libs/zookeeper-3.3.4.jar:/home/OUR-SERVER/kafka/kafka-trunk-2a59ad76c6_scala-2.8.0.jar
> (org.apache.zookeeper.ZooKeeper)
> [2012-07-19 17:59:21,193] INFO Client
>
> environment:java.library.path=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> (org.apache.zookeeper.ZooKeeper)
> [2012-07-19 17:59:21,193] INFO Client environment:java.io.tmpdir=/tmp
> (org.apache.zookeeper.ZooKeeper)
> [2012-07-19 17:59:21,193] INFO Client environment:java.compiler=<NA>
> (org.apache.zookeeper.ZooKeeper)
> [2012-07-19 17:59:21,193] INFO Client environment:os.name=Linux
> (org.apache.zookeeper.ZooKeeper)
> [2012-07-19 17:59:21,193] INFO Client environment:os.arch=amd64
> (org.apache.zookeeper.ZooKeeper)
> [2012-07-19 17:59:21,193] INFO Client
> environment:os.version=2.6.32-71.29.1.el6.x86_64
> (org.apache.zookeeper.ZooKeeper)
> [2012-07-19 17:59:21,193] INFO Client environment:user.name=OUR-SERVER
> (org.apache.zookeeper.ZooKeeper)
> [2012-07-19 17:59:21,193] INFO Client
> environment:user.home=/home/OUR-SERVER (org.apache.zookeeper.ZooKeeper)
> [2012-07-19 17:59:21,193] INFO Client
> environment:user.dir=/home/OUR-SERVER/kafka
> (org.apache.zookeeper.ZooKeeper)
> [2012-07-19 17:59:21,194] INFO Initiating client connection,
> connectString=XX.XX.XX.XX:2181/kafka sessionTimeout=6000
> watcher=org.I0Itec.zkclient.ZkClient@4a48edb5
> (org.apache.zookeeper.ZooKeeper)
> [2012-07-19 17:59:21,211] INFO Opening socket connection to server
> /XX.XX.XX.XX:2181 (org.apache.zookeeper.ClientCnxn)
> [2012-07-19 17:59:21,215] INFO Socket connection established to
> mobile-1/XX.XX.XX.XX:2181, initiating session
> (org.apache.zookeeper.ClientCnxn)
> [2012-07-19 17:59:21,280] INFO Session establishment complete on server
> mobile-1/XX.XX.XX.XX:2181, sessionid = 0x1389e95409a0023, negotiated
> timeout = 6000 (org.apache.zookeeper.ClientCnxn)
> [2012-07-19 17:59:21,282] INFO zookeeper state changed (SyncConnected)
> (org.I0Itec.zkclient.ZkClient)
> [2012-07-19 17:59:21,380] INFO Awaiting connections on port 9092
> (kafka.network.Acceptor)
> [2012-07-19 17:59:21,382] INFO Will not load MX4J, mx4j-tools.jar is not in
> the classpath (kafka.utils.Mx4jLoader$)
> [2012-07-19 17:59:21,383] INFO Registering broker /brokers/ids/1
> (kafka.server.KafkaZooKeeper)
> [2012-07-19 17:59:21,404] INFO Registering broker /brokers/ids/1 succeeded
> with id:1,creatorId:XX.XX.XX.XX-1342691961383,host:XX.XX.XX.XX,port:9092
> (kafka.server.KafkaZooKeeper)
> [2012-07-19 17:59:21,417] INFO Starting log flusher every 1000 ms with the
> following overrides Map() (kafka.log.LogManager)
> [2012-07-19 17:59:21,419] INFO Kafka server started.
> (kafka.server.KafkaServer)
>
>
> This is the error from producer:
>
> ERR [20120719-17:53:56.312] exception: Server Exception. : Event queue is
> full of unsent messages, could not send event: {OUR-LOG}
> ERR [20120719-17:53:56.312] exception:
> kafka.producer.async.AsyncProducer.send(AsyncProducer.scala:110)
> ERR [20120719-17:53:56.312] exception:
>
> kafka.producer.ProducerPool$$anonfun$send$1$$anonfun$apply$mcVI$sp$5$$anonfun$apply$2.apply(ProducerPool.scala:129)
> ERR [20120719-17:53:56.312] exception:
>
> kafka.producer.ProducerPool$$anonfun$send$1$$anonfun$apply$mcVI$sp$5$$anonfun$apply$2.apply(ProducerPool.scala:129)
> ERR [20120719-17:53:56.312] exception:
>
> scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
> ERR [20120719-17:53:56.312] exception:
> scala.collection.immutable.List.foreach(List.scala:45)
> ERR [20120719-17:53:56.312] exception:
>
> kafka.producer.ProducerPool$$anonfun$send$1$$anonfun$apply$mcVI$sp$5.apply(ProducerPool.scala:129)
> ERR [20120719-17:53:56.312] exception:
>
> kafka.producer.ProducerPool$$anonfun$send$1$$anonfun$apply$mcVI$sp$5.apply(ProducerPool.scala:128)
> ERR [20120719-17:53:56.312] exception:
>
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:60)
> ERR [20120719-17:53:56.312] exception:
> scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:44)
> ERR [20120719-17:53:56.312] exception:
>
> kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:128)
> ERR [20120719-17:53:56.312] exception:
> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:100)
> ERR [20120719-17:53:56.312] exception:
> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:100)
> ERR [20120719-17:53:56.312] exception:
>
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:60)
> ERR [20120719-17:53:56.312] exception:
> scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:44)
> ERR [20120719-17:53:56.312] exception:
> kafka.producer.ProducerPool.send(ProducerPool.scala:100)
> ERR [20120719-17:53:56.312] exception:
> kafka.producer.Producer.zkSend(Producer.scala:137)
> ERR [20120719-17:53:56.312] exception:
> kafka.producer.Producer.send(Producer.scala:99)
>
> I intentionally shut down the other broker,make the restarted broker the
> only one in the cluster,seems that the producer can not find the broker and
> not sending any events
>
> --
> Best Regards
>
> ----------------------
> 刘明敏 | mmLiu
>