You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by SuoNayi <su...@163.com> on 2012/02/07 03:26:43 UTC
Bad user experience when out of box
Hi all,
I'm new to kafka.I set up zooker server and kafka server on my test server by following the quickstart.
I also success running the example list in the quickstart,such as step 3 and step 4.
Now I set up a project on my pc to run examples of directory named kafka-0.7.0-incubating-src\examples.
Of course, I changed addresses and ports of zk server and kafka that both pointed at my test server.
When start the main function of class KafkaConsumerProducerDemo,it throws the following exception:
Exception in thread "main" org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 6000
at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:876)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
at kafka.producer.ZKBrokerPartitionInfo.<init>(ZKBrokerPartitionInfo.scala:63)
at kafka.producer.Producer.<init>(Producer.scala:53)
at kafka.javaapi.producer.Producer.<init>(Producer.scala:33)
at kafka.javaapi.producer.Producer.<init>(Producer.scala:40)
at kafka.examples.Producer.<init>(Producer.java:35)
at kafka.examples.KafkaConsumerProducerDemo.main(KafkaConsumerProducerDemo.java:23)
I have to specify the timeout value for zk,like this,zk.connectiontimeout.ms=10000 and the client could connect to zk server successfully.
But another new exception is thrown again:
Exception in thread "Thread-0" java.nio.channels.ClosedChannelException
at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(Unknown Source)
at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
at kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:48)
at kafka.network.Send$class.writeCompletely(Transmission.scala:76)
at kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend.scala:25)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:88)
at kafka.producer.SyncProducer.send(SyncProducer.scala:87)
at kafka.producer.SyncProducer.send(SyncProducer.scala:117)
at kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:116)
at kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
at kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:57)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:43)
at kafka.producer.ProducerPool.send(ProducerPool.scala:102)
at kafka.producer.Producer.zkSend(Producer.scala:143)
at kafka.producer.Producer.send(Producer.scala:105)
at kafka.javaapi.producer.Producer.send(Producer.scala:104)
at kafka.examples.Producer.run(Producer.java:45)
Hope someone could help me, thanks very much!
SuoNayi
Re: Reply:Re: Reply:Bad user experience when out of box
Posted by Neha Narkhede <ne...@gmail.com>.
Hi,
The producer closes the socket connection to the broker and
reconnects. The frequency of this reconnect is based on the
reconnect.interval config parameter in the producer.
By default, it is set to 30 sec, if you want you can increase it. This
is mainly to ensure better load balancing on the Kafka servers. It
shouldn't cause any overhead on the Kafka cluster, is there any
particular reason you would like the Producer to not reconnect to the
brokers ?
Thanks,
Neha
2012/2/7 SuoNayi <su...@163.com>:
> Yes, zk server and kafka server are now running normally.
> My client sends messages successfully now.
> Why kafka sever keeps closing connections from my client all the time,
> I can see information of log like this:
> INFO Closing socket connection to /2x3.1x9.x0.x. (kafka.network.Processor)
> Whichever configuration such as zk or static configuration to discovery kafka sever my client uses ,
> kafka server will log these informations all the time.
> I know that kafka client will create a connection pool keeping connect to brokers,
> each broker each connection, in my case, my client has connected to the single broker,
> so why kafka server keeps closing connection from my client?
>
> At 2012-02-07 18:42:17,"Joe Stein" <cr...@gmail.com> wrote:
>>SuoNayi,
>>
>>Do you have the Kafka Server running? The Producer will need to find a
>>Broker that is registered within ZK for it to publish messages.
>>
>>So like in the Quick Start this is still a required step
>>
>>*bin/kafka-server-start.sh config/server.properties*
>>
>>
>>On Tue, Feb 7, 2012 at 5:00 AM, SuoNayi <su...@163.com> wrote:
>>
>>> Sorry for that I have found that hostname of the test server is of
>>> misconfigure.
>>> Here is new question when I send messages with 20 producers that each
>>> workes in it's own thread,
>>>
>>> My client throws exception of timeout:
>>> Exception in thread "Thread-2" java.net.ConnectException: Connection timed
>>> out: connect
>>> at sun.nio.ch.Net.connect(Native Method)
>>> at sun.nio.ch.SocketChannelImpl.connect(Unknown Source)
>>> at kafka.producer.SyncProducer.connect(SyncProducer.scala:172)
>>> at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
>>> at kafka.producer.SyncProducer.send(SyncProducer.scala:117)
>>> at
>>> kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:116)
>>> at
>>> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
>>> at
>>> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
>>> at
>>> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:57)
>>> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:43)
>>> at kafka.producer.ProducerPool.send(ProducerPool.scala:102)
>>> at kafka.producer.Producer.configSend(Producer.scala:167)
>>> at kafka.producer.Producer.send(Producer.scala:106)
>>> at kafka.javaapi.producer.Producer.send(Producer.scala:104)
>>> at test.kafka.TestOutboundKfa$Sender.run(TestOutboundKfa.java:87)
>>>
>>> At 2012-02-07 10:26:43,SuoNayi <su...@163.com> wrote:
>>> >Hi all,
>>> > I'm new to kafka.I set up zooker server and kafka server on my test
>>> server by following the quickstart.
>>> > I also success running the example list in the quickstart,such as step
>>> 3 and step 4.
>>> > Now I set up a project on my pc to run examples of directory named
>>> kafka-0.7.0-incubating-src\examples.
>>> > Of course, I changed addresses and ports of zk server and kafka that
>>> both pointed at my test server.
>>> > When start the main function of class KafkaConsumerProducerDemo,it
>>> throws the following exception:
>>> >Exception in thread "main"
>>> org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to
>>> zookeeper server within timeout: 6000
>>> >at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:876)
>>> >at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
>>> >at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
>>> >at
>>> kafka.producer.ZKBrokerPartitionInfo.<init>(ZKBrokerPartitionInfo.scala:63)
>>> >at kafka.producer.Producer.<init>(Producer.scala:53)
>>> >at kafka.javaapi.producer.Producer.<init>(Producer.scala:33)
>>> >at kafka.javaapi.producer.Producer.<init>(Producer.scala:40)
>>> >at kafka.examples.Producer.<init>(Producer.java:35)
>>> >at
>>> kafka.examples.KafkaConsumerProducerDemo.main(KafkaConsumerProducerDemo.java:23)
>>> > I have to specify the timeout value for zk,like this,
>>> zk.connectiontimeout.ms=10000 and the client could connect to zk server
>>> successfully.
>>> > But another new exception is thrown again:
>>> >Exception in thread "Thread-0" java.nio.channels.ClosedChannelException
>>> >at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(Unknown Source)
>>> >at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
>>> >at
>>> kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:48)
>>> >at kafka.network.Send$class.writeCompletely(Transmission.scala:76)
>>> >at
>>> kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend.scala:25)
>>> >at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:88)
>>> >at kafka.producer.SyncProducer.send(SyncProducer.scala:87)
>>> >at kafka.producer.SyncProducer.send(SyncProducer.scala:117)
>>> >at
>>> kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:116)
>>> >at
>>> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
>>> >at
>>> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
>>> >at
>>> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:57)
>>> >at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:43)
>>> >at kafka.producer.ProducerPool.send(ProducerPool.scala:102)
>>> >at kafka.producer.Producer.zkSend(Producer.scala:143)
>>> >at kafka.producer.Producer.send(Producer.scala:105)
>>> >at kafka.javaapi.producer.Producer.send(Producer.scala:104)
>>> >at kafka.examples.Producer.run(Producer.java:45)
>>> > Hope someone could help me, thanks very much!
>>> >
>>> >
>>> >SuoNayi
>>> >
>>> >
>>> >
>>>
>>
>>
>>
>>--
>>
>>/*
>>Joe Stein
>>http://www.linkedin.com/in/charmalloc
>>Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>>*/
Re: Reply:Re: Reply:Bad user experience when out of box
Posted by Jay Kreps <ja...@gmail.com>.
The clients periodically reconnect. The parameter that controls this is
reconnect.interval in http://incubator.apache.org/kafka/configuration.html
When the client closes the connection, the server logs it, I think. If you
don't like this you can make the reconnect interval higher. The intention
of this feature is to allow a level 4 load balancer to work with kafka.
Since this works at the tcp connection level, the client needs to reconnect
to balance load.
-Jay
2012/2/7 SuoNayi <su...@163.com>
> Yes, zk server and kafka server are now running normally.
> My client sends messages successfully now.
> Why kafka sever keeps closing connections from my client all the time,
> I can see information of log like this:
> INFO Closing socket connection to /2x3.1x9.x0.x. (kafka.network.Processor)
> Whichever configuration such as zk or static configuration to discovery
> kafka sever my client uses ,
> kafka server will log these informations all the time.
> I know that kafka client will create a connection pool keeping connect to
> brokers,
> each broker each connection, in my case, my client has connected to the
> single broker,
> so why kafka server keeps closing connection from my client?
>
> At 2012-02-07 18:42:17,"Joe Stein" <cr...@gmail.com> wrote:
> >SuoNayi,
> >
> >Do you have the Kafka Server running? The Producer will need to find a
> >Broker that is registered within ZK for it to publish messages.
> >
> >So like in the Quick Start this is still a required step
> >
> >*bin/kafka-server-start.sh config/server.properties*
> >
> >
> >On Tue, Feb 7, 2012 at 5:00 AM, SuoNayi <su...@163.com> wrote:
> >
> >> Sorry for that I have found that hostname of the test server is of
> >> misconfigure.
> >> Here is new question when I send messages with 20 producers that each
> >> workes in it's own thread,
> >>
> >> My client throws exception of timeout:
> >> Exception in thread "Thread-2" java.net.ConnectException: Connection
> timed
> >> out: connect
> >> at sun.nio.ch.Net.connect(Native Method)
> >> at sun.nio.ch.SocketChannelImpl.connect(Unknown Source)
> >> at kafka.producer.SyncProducer.connect(SyncProducer.scala:172)
> >> at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
> >> at kafka.producer.SyncProducer.send(SyncProducer.scala:117)
> >> at
> >>
> kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:116)
> >> at
> >>
> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
> >> at
> >>
> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
> >> at
> >>
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:57)
> >> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:43)
> >> at kafka.producer.ProducerPool.send(ProducerPool.scala:102)
> >> at kafka.producer.Producer.configSend(Producer.scala:167)
> >> at kafka.producer.Producer.send(Producer.scala:106)
> >> at kafka.javaapi.producer.Producer.send(Producer.scala:104)
> >> at test.kafka.TestOutboundKfa$Sender.run(TestOutboundKfa.java:87)
> >>
> >> At 2012-02-07 10:26:43,SuoNayi <su...@163.com> wrote:
> >> >Hi all,
> >> > I'm new to kafka.I set up zooker server and kafka server on my test
> >> server by following the quickstart.
> >> > I also success running the example list in the quickstart,such as
> step
> >> 3 and step 4.
> >> > Now I set up a project on my pc to run examples of directory named
> >> kafka-0.7.0-incubating-src\examples.
> >> > Of course, I changed addresses and ports of zk server and kafka
> that
> >> both pointed at my test server.
> >> > When start the main function of class KafkaConsumerProducerDemo,it
> >> throws the following exception:
> >> >Exception in thread "main"
> >> org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to
> >> zookeeper server within timeout: 6000
> >> >at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:876)
> >> >at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
> >> >at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
> >> >at
> >>
> kafka.producer.ZKBrokerPartitionInfo.<init>(ZKBrokerPartitionInfo.scala:63)
> >> >at kafka.producer.Producer.<init>(Producer.scala:53)
> >> >at kafka.javaapi.producer.Producer.<init>(Producer.scala:33)
> >> >at kafka.javaapi.producer.Producer.<init>(Producer.scala:40)
> >> >at kafka.examples.Producer.<init>(Producer.java:35)
> >> >at
> >>
> kafka.examples.KafkaConsumerProducerDemo.main(KafkaConsumerProducerDemo.java:23)
> >> > I have to specify the timeout value for zk,like this,
> >> zk.connectiontimeout.ms=10000 and the client could connect to zk server
> >> successfully.
> >> > But another new exception is thrown again:
> >> >Exception in thread "Thread-0" java.nio.channels.ClosedChannelException
> >> >at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(Unknown Source)
> >> >at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> >> >at
> >>
> kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:48)
> >> >at kafka.network.Send$class.writeCompletely(Transmission.scala:76)
> >> >at
> >>
> kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend.scala:25)
> >> >at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:88)
> >> >at kafka.producer.SyncProducer.send(SyncProducer.scala:87)
> >> >at kafka.producer.SyncProducer.send(SyncProducer.scala:117)
> >> >at
> >>
> kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:116)
> >> >at
> >>
> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
> >> >at
> >>
> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
> >> >at
> >>
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:57)
> >> >at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:43)
> >> >at kafka.producer.ProducerPool.send(ProducerPool.scala:102)
> >> >at kafka.producer.Producer.zkSend(Producer.scala:143)
> >> >at kafka.producer.Producer.send(Producer.scala:105)
> >> >at kafka.javaapi.producer.Producer.send(Producer.scala:104)
> >> >at kafka.examples.Producer.run(Producer.java:45)
> >> > Hope someone could help me, thanks very much!
> >> >
> >> >
> >> >SuoNayi
> >> >
> >> >
> >> >
> >>
> >
> >
> >
> >--
> >
> >/*
> >Joe Stein
> >http://www.linkedin.com/in/charmalloc
> >Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> >*/
>
Reply:Re: Reply:Bad user experience when out of box
Posted by SuoNayi <su...@163.com>.
Yes, zk server and kafka server are now running normally.
My client sends messages successfully now.
Why kafka sever keeps closing connections from my client all the time,
I can see information of log like this:
INFO Closing socket connection to /2x3.1x9.x0.x. (kafka.network.Processor)
Whichever configuration such as zk or static configuration to discovery kafka sever my client uses ,
kafka server will log these informations all the time.
I know that kafka client will create a connection pool keeping connect to brokers,
each broker each connection, in my case, my client has connected to the single broker,
so why kafka server keeps closing connection from my client?
At 2012-02-07 18:42:17,"Joe Stein" <cr...@gmail.com> wrote:
>SuoNayi,
>
>Do you have the Kafka Server running? The Producer will need to find a
>Broker that is registered within ZK for it to publish messages.
>
>So like in the Quick Start this is still a required step
>
>*bin/kafka-server-start.sh config/server.properties*
>
>
>On Tue, Feb 7, 2012 at 5:00 AM, SuoNayi <su...@163.com> wrote:
>
>> Sorry for that I have found that hostname of the test server is of
>> misconfigure.
>> Here is new question when I send messages with 20 producers that each
>> workes in it's own thread,
>>
>> My client throws exception of timeout:
>> Exception in thread "Thread-2" java.net.ConnectException: Connection timed
>> out: connect
>> at sun.nio.ch.Net.connect(Native Method)
>> at sun.nio.ch.SocketChannelImpl.connect(Unknown Source)
>> at kafka.producer.SyncProducer.connect(SyncProducer.scala:172)
>> at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
>> at kafka.producer.SyncProducer.send(SyncProducer.scala:117)
>> at
>> kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:116)
>> at
>> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
>> at
>> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
>> at
>> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:57)
>> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:43)
>> at kafka.producer.ProducerPool.send(ProducerPool.scala:102)
>> at kafka.producer.Producer.configSend(Producer.scala:167)
>> at kafka.producer.Producer.send(Producer.scala:106)
>> at kafka.javaapi.producer.Producer.send(Producer.scala:104)
>> at test.kafka.TestOutboundKfa$Sender.run(TestOutboundKfa.java:87)
>>
>> At 2012-02-07 10:26:43,SuoNayi <su...@163.com> wrote:
>> >Hi all,
>> > I'm new to kafka.I set up zooker server and kafka server on my test
>> server by following the quickstart.
>> > I also success running the example list in the quickstart,such as step
>> 3 and step 4.
>> > Now I set up a project on my pc to run examples of directory named
>> kafka-0.7.0-incubating-src\examples.
>> > Of course, I changed addresses and ports of zk server and kafka that
>> both pointed at my test server.
>> > When start the main function of class KafkaConsumerProducerDemo,it
>> throws the following exception:
>> >Exception in thread "main"
>> org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to
>> zookeeper server within timeout: 6000
>> >at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:876)
>> >at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
>> >at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
>> >at
>> kafka.producer.ZKBrokerPartitionInfo.<init>(ZKBrokerPartitionInfo.scala:63)
>> >at kafka.producer.Producer.<init>(Producer.scala:53)
>> >at kafka.javaapi.producer.Producer.<init>(Producer.scala:33)
>> >at kafka.javaapi.producer.Producer.<init>(Producer.scala:40)
>> >at kafka.examples.Producer.<init>(Producer.java:35)
>> >at
>> kafka.examples.KafkaConsumerProducerDemo.main(KafkaConsumerProducerDemo.java:23)
>> > I have to specify the timeout value for zk,like this,
>> zk.connectiontimeout.ms=10000 and the client could connect to zk server
>> successfully.
>> > But another new exception is thrown again:
>> >Exception in thread "Thread-0" java.nio.channels.ClosedChannelException
>> >at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(Unknown Source)
>> >at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
>> >at
>> kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:48)
>> >at kafka.network.Send$class.writeCompletely(Transmission.scala:76)
>> >at
>> kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend.scala:25)
>> >at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:88)
>> >at kafka.producer.SyncProducer.send(SyncProducer.scala:87)
>> >at kafka.producer.SyncProducer.send(SyncProducer.scala:117)
>> >at
>> kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:116)
>> >at
>> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
>> >at
>> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
>> >at
>> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:57)
>> >at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:43)
>> >at kafka.producer.ProducerPool.send(ProducerPool.scala:102)
>> >at kafka.producer.Producer.zkSend(Producer.scala:143)
>> >at kafka.producer.Producer.send(Producer.scala:105)
>> >at kafka.javaapi.producer.Producer.send(Producer.scala:104)
>> >at kafka.examples.Producer.run(Producer.java:45)
>> > Hope someone could help me, thanks very much!
>> >
>> >
>> >SuoNayi
>> >
>> >
>> >
>>
>
>
>
>--
>
>/*
>Joe Stein
>http://www.linkedin.com/in/charmalloc
>Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>*/
Re: Reply:Bad user experience when out of box
Posted by Joe Stein <cr...@gmail.com>.
SuoNayi,
Do you have the Kafka Server running? The Producer will need to find a
Broker that is registered within ZK for it to publish messages.
So like in the Quick Start this is still a required step
*bin/kafka-server-start.sh config/server.properties*
On Tue, Feb 7, 2012 at 5:00 AM, SuoNayi <su...@163.com> wrote:
> Sorry for that I have found that hostname of the test server is of
> misconfigure.
> Here is new question when I send messages with 20 producers that each
> workes in it's own thread,
>
> My client throws exception of timeout:
> Exception in thread "Thread-2" java.net.ConnectException: Connection timed
> out: connect
> at sun.nio.ch.Net.connect(Native Method)
> at sun.nio.ch.SocketChannelImpl.connect(Unknown Source)
> at kafka.producer.SyncProducer.connect(SyncProducer.scala:172)
> at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
> at kafka.producer.SyncProducer.send(SyncProducer.scala:117)
> at
> kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:116)
> at
> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
> at
> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
> at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:57)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:43)
> at kafka.producer.ProducerPool.send(ProducerPool.scala:102)
> at kafka.producer.Producer.configSend(Producer.scala:167)
> at kafka.producer.Producer.send(Producer.scala:106)
> at kafka.javaapi.producer.Producer.send(Producer.scala:104)
> at test.kafka.TestOutboundKfa$Sender.run(TestOutboundKfa.java:87)
>
> At 2012-02-07 10:26:43,SuoNayi <su...@163.com> wrote:
> >Hi all,
> > I'm new to kafka.I set up zooker server and kafka server on my test
> server by following the quickstart.
> > I also success running the example list in the quickstart,such as step
> 3 and step 4.
> > Now I set up a project on my pc to run examples of directory named
> kafka-0.7.0-incubating-src\examples.
> > Of course, I changed addresses and ports of zk server and kafka that
> both pointed at my test server.
> > When start the main function of class KafkaConsumerProducerDemo,it
> throws the following exception:
> >Exception in thread "main"
> org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to
> zookeeper server within timeout: 6000
> >at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:876)
> >at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
> >at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
> >at
> kafka.producer.ZKBrokerPartitionInfo.<init>(ZKBrokerPartitionInfo.scala:63)
> >at kafka.producer.Producer.<init>(Producer.scala:53)
> >at kafka.javaapi.producer.Producer.<init>(Producer.scala:33)
> >at kafka.javaapi.producer.Producer.<init>(Producer.scala:40)
> >at kafka.examples.Producer.<init>(Producer.java:35)
> >at
> kafka.examples.KafkaConsumerProducerDemo.main(KafkaConsumerProducerDemo.java:23)
> > I have to specify the timeout value for zk,like this,
> zk.connectiontimeout.ms=10000 and the client could connect to zk server
> successfully.
> > But another new exception is thrown again:
> >Exception in thread "Thread-0" java.nio.channels.ClosedChannelException
> >at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(Unknown Source)
> >at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> >at
> kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:48)
> >at kafka.network.Send$class.writeCompletely(Transmission.scala:76)
> >at
> kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend.scala:25)
> >at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:88)
> >at kafka.producer.SyncProducer.send(SyncProducer.scala:87)
> >at kafka.producer.SyncProducer.send(SyncProducer.scala:117)
> >at
> kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:116)
> >at
> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
> >at
> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
> >at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:57)
> >at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:43)
> >at kafka.producer.ProducerPool.send(ProducerPool.scala:102)
> >at kafka.producer.Producer.zkSend(Producer.scala:143)
> >at kafka.producer.Producer.send(Producer.scala:105)
> >at kafka.javaapi.producer.Producer.send(Producer.scala:104)
> >at kafka.examples.Producer.run(Producer.java:45)
> > Hope someone could help me, thanks very much!
> >
> >
> >SuoNayi
> >
> >
> >
>
--
/*
Joe Stein
http://www.linkedin.com/in/charmalloc
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
*/
Re: Reply:Bad user experience when out of box
Posted by Jun Rao <ju...@gmail.com>.
Try telnet to the broker and the port directly see if you can make the
connection. If not, it's very likely that the server is not bound to the
ip/host that you want. See #2 in http://incubator.apache.org/kafka/faq.html
Thanks,
Jun
2012/2/7 SuoNayi <su...@163.com>
> Sorry for that I have found that hostname of the test server is of
> misconfigure.
> Here is new question when I send messages with 20 producers that each
> workes in it's own thread,
>
> My client throws exception of timeout:
> Exception in thread "Thread-2" java.net.ConnectException: Connection timed
> out: connect
> at sun.nio.ch.Net.connect(Native Method)
> at sun.nio.ch.SocketChannelImpl.connect(Unknown Source)
> at kafka.producer.SyncProducer.connect(SyncProducer.scala:172)
> at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
> at kafka.producer.SyncProducer.send(SyncProducer.scala:117)
> at
> kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:116)
> at
> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
> at
> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
> at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:57)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:43)
> at kafka.producer.ProducerPool.send(ProducerPool.scala:102)
> at kafka.producer.Producer.configSend(Producer.scala:167)
> at kafka.producer.Producer.send(Producer.scala:106)
> at kafka.javaapi.producer.Producer.send(Producer.scala:104)
> at test.kafka.TestOutboundKfa$Sender.run(TestOutboundKfa.java:87)
>
> At 2012-02-07 10:26:43,SuoNayi <su...@163.com> wrote:
> >Hi all,
> > I'm new to kafka.I set up zooker server and kafka server on my test
> server by following the quickstart.
> > I also success running the example list in the quickstart,such as step
> 3 and step 4.
> > Now I set up a project on my pc to run examples of directory named
> kafka-0.7.0-incubating-src\examples.
> > Of course, I changed addresses and ports of zk server and kafka that
> both pointed at my test server.
> > When start the main function of class KafkaConsumerProducerDemo,it
> throws the following exception:
> >Exception in thread "main"
> org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to
> zookeeper server within timeout: 6000
> >at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:876)
> >at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
> >at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
> >at
> kafka.producer.ZKBrokerPartitionInfo.<init>(ZKBrokerPartitionInfo.scala:63)
> >at kafka.producer.Producer.<init>(Producer.scala:53)
> >at kafka.javaapi.producer.Producer.<init>(Producer.scala:33)
> >at kafka.javaapi.producer.Producer.<init>(Producer.scala:40)
> >at kafka.examples.Producer.<init>(Producer.java:35)
> >at
> kafka.examples.KafkaConsumerProducerDemo.main(KafkaConsumerProducerDemo.java:23)
> > I have to specify the timeout value for zk,like this,
> zk.connectiontimeout.ms=10000 and the client could connect to zk server
> successfully.
> > But another new exception is thrown again:
> >Exception in thread "Thread-0" java.nio.channels.ClosedChannelException
> >at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(Unknown Source)
> >at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> >at
> kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:48)
> >at kafka.network.Send$class.writeCompletely(Transmission.scala:76)
> >at
> kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend.scala:25)
> >at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:88)
> >at kafka.producer.SyncProducer.send(SyncProducer.scala:87)
> >at kafka.producer.SyncProducer.send(SyncProducer.scala:117)
> >at
> kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:116)
> >at
> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
> >at
> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
> >at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:57)
> >at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:43)
> >at kafka.producer.ProducerPool.send(ProducerPool.scala:102)
> >at kafka.producer.Producer.zkSend(Producer.scala:143)
> >at kafka.producer.Producer.send(Producer.scala:105)
> >at kafka.javaapi.producer.Producer.send(Producer.scala:104)
> >at kafka.examples.Producer.run(Producer.java:45)
> > Hope someone could help me, thanks very much!
> >
> >
> >SuoNayi
> >
> >
> >
>
Reply:Bad user experience when out of box
Posted by SuoNayi <su...@163.com>.
Sorry for that I have found that hostname of the test server is of misconfigure.
Here is new question when I send messages with 20 producers that each workes in it's own thread,
My client throws exception of timeout:
Exception in thread "Thread-2" java.net.ConnectException: Connection timed out: connect
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(Unknown Source)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:172)
at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
at kafka.producer.SyncProducer.send(SyncProducer.scala:117)
at kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:116)
at kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
at kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:57)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:43)
at kafka.producer.ProducerPool.send(ProducerPool.scala:102)
at kafka.producer.Producer.configSend(Producer.scala:167)
at kafka.producer.Producer.send(Producer.scala:106)
at kafka.javaapi.producer.Producer.send(Producer.scala:104)
at test.kafka.TestOutboundKfa$Sender.run(TestOutboundKfa.java:87)
At 2012-02-07 10:26:43,SuoNayi <su...@163.com> wrote:
>Hi all,
> I'm new to kafka.I set up zooker server and kafka server on my test server by following the quickstart.
> I also success running the example list in the quickstart,such as step 3 and step 4.
> Now I set up a project on my pc to run examples of directory named kafka-0.7.0-incubating-src\examples.
> Of course, I changed addresses and ports of zk server and kafka that both pointed at my test server.
> When start the main function of class KafkaConsumerProducerDemo,it throws the following exception:
>Exception in thread "main" org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 6000
>at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:876)
>at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
>at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
>at kafka.producer.ZKBrokerPartitionInfo.<init>(ZKBrokerPartitionInfo.scala:63)
>at kafka.producer.Producer.<init>(Producer.scala:53)
>at kafka.javaapi.producer.Producer.<init>(Producer.scala:33)
>at kafka.javaapi.producer.Producer.<init>(Producer.scala:40)
>at kafka.examples.Producer.<init>(Producer.java:35)
>at kafka.examples.KafkaConsumerProducerDemo.main(KafkaConsumerProducerDemo.java:23)
> I have to specify the timeout value for zk,like this,zk.connectiontimeout.ms=10000 and the client could connect to zk server successfully.
> But another new exception is thrown again:
>Exception in thread "Thread-0" java.nio.channels.ClosedChannelException
>at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(Unknown Source)
>at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
>at kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:48)
>at kafka.network.Send$class.writeCompletely(Transmission.scala:76)
>at kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend.scala:25)
>at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:88)
>at kafka.producer.SyncProducer.send(SyncProducer.scala:87)
>at kafka.producer.SyncProducer.send(SyncProducer.scala:117)
>at kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:116)
>at kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
>at kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
>at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:57)
>at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:43)
>at kafka.producer.ProducerPool.send(ProducerPool.scala:102)
>at kafka.producer.Producer.zkSend(Producer.scala:143)
>at kafka.producer.Producer.send(Producer.scala:105)
>at kafka.javaapi.producer.Producer.send(Producer.scala:104)
>at kafka.examples.Producer.run(Producer.java:45)
> Hope someone could help me, thanks very much!
>
>
>SuoNayi
>
>
>
Re: Bad user experience when out of box
Posted by Joe Stein <cr...@gmail.com>.
SuoNayi,
In line 32 of examples/src/main/java/kafka/examples/Producer.java
props.put("zk.connect", "localhost:2181");
change that to zk host and port in real world you can be passing that
structure in like ProducerShell does from properties file
/*
Joe Stein
http://www.linkedin.com/in/charmalloc
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
*/
On Mon, Feb 6, 2012 at 9:26 PM, SuoNayi <su...@163.com> wrote:
> Hi all,
> I'm new to kafka.I set up zooker server and kafka server on my test
> server by following the quickstart.
> I also success running the example list in the quickstart,such as step 3
> and step 4.
> Now I set up a project on my pc to run examples of directory named
> kafka-0.7.0-incubating-src\examples.
> Of course, I changed addresses and ports of zk server and kafka that
> both pointed at my test server.
> When start the main function of class KafkaConsumerProducerDemo,it
> throws the following exception:
> Exception in thread "main"
> org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to
> zookeeper server within timeout: 6000
> at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:876)
> at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
> at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
> at
> kafka.producer.ZKBrokerPartitionInfo.<init>(ZKBrokerPartitionInfo.scala:63)
> at kafka.producer.Producer.<init>(Producer.scala:53)
> at kafka.javaapi.producer.Producer.<init>(Producer.scala:33)
> at kafka.javaapi.producer.Producer.<init>(Producer.scala:40)
> at kafka.examples.Producer.<init>(Producer.java:35)
> at
> kafka.examples.KafkaConsumerProducerDemo.main(KafkaConsumerProducerDemo.java:23)
> I have to specify the timeout value for zk,like this,
> zk.connectiontimeout.ms=10000 and the client could connect to zk server
> successfully.
> But another new exception is thrown again:
> Exception in thread "Thread-0" java.nio.channels.ClosedChannelException
> at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at
> kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:48)
> at kafka.network.Send$class.writeCompletely(Transmission.scala:76)
> at
> kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend.scala:25)
> at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:88)
> at kafka.producer.SyncProducer.send(SyncProducer.scala:87)
> at kafka.producer.SyncProducer.send(SyncProducer.scala:117)
> at
> kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:116)
> at
> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
> at
> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
> at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:57)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:43)
> at kafka.producer.ProducerPool.send(ProducerPool.scala:102)
> at kafka.producer.Producer.zkSend(Producer.scala:143)
> at kafka.producer.Producer.send(Producer.scala:105)
> at kafka.javaapi.producer.Producer.send(Producer.scala:104)
> at kafka.examples.Producer.run(Producer.java:45)
> Hope someone could help me, thanks very much!
>
>
> SuoNayi
>
>
>
>