You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Yashika Gupta <ya...@impetus.co.in> on 2014/04/24 19:12:39 UTC
Kafka Performance Tuning
Hi,
I am working on a POC where I have 1 Zookeeper and 2 Kafka Brokers on my local machine. I am running 8 sets of Kafka consumers and producers running in parallel.
Below are my configurations:
Consumer Configs:
zookeeper.session.timeout.ms=120000
zookeeper.sync.time.ms=2000
zookeeper.connection.timeout.ms=120000
auto.commit.interval.ms=60000
rebalance.backoff.ms=2000
fetch.wait.max.ms=60000
auto.offset.reset=smallest
Producer configs:
key.serializer.class=kafka.serializer.StringEncoder
request.required.acks=-1
message.send.max.retries=3
request.timeout.ms=60000
I have tried various more configuration changes but I am running into the same exception again and again.
17.04.2014 06:31:83216 ERROR pool-5-thread-1 kafka.producer.async.DefaultEventHandler - Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: TOPIC99
17.04.2014 06:31:85616 ERROR pool-4-thread-1 kafka.producer.async.DefaultEventHandler - Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: TOPIC99
I am not able to get to the root cause of the issue.
Appreciate your help.
Regards,
Yashika
________________________________
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
Re: Kafka Performance Tuning
Posted by Jun Rao <ju...@gmail.com>.
The socket timeout may be due to this: MaxWait: 60000 ms. So, you need to
set socket.timeout.ms in consumer config (defaults to 30 secs) to be larger
than 60 secs since your fetch request could take up to 60 secs.
Thanks,
Jun
On Fri, Apr 25, 2014 at 12:11 AM, Yashika Gupta <yashika.gupta@impetus.co.in
> wrote:
> Timothy,
>
> I checked out https://issues.apache.org/jira/browse/KAFKA-1124
> So, I created the topic manually and re-ran my test and has enabled log4j
> properties.
>
> There are no ERROR logs in controller and state change logs.
> And I m still getting the SocketTimeout Exception:
>
> 25.04.2014 03:05:10115 WARN
> ConsumerFetcherThread-consumerDeviceError2_vcmd-devanshu-1398408813021-efe24d49-0-0
> kafka.consumer.ConsumerFetcherThread -
> [ConsumerFetcherThread-consumerDeviceError2_vcmd-devanshu-1398408813021-efe24d49-0-0],
> Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 26; ClientId:
> consumerDeviceError2-ConsumerFetcherThread-consumerDeviceError2_vcmd-devanshu-1398408813021-efe24d49-0-0;
> ReplicaId: -1; MaxWait: 60000 ms; MinBytes: 1 bytes; RequestInfo:
> [LOGLINE10,0] -> PartitionFetchInfo(12,1048576)
> java.net.SocketTimeoutException
> at
> sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:229)
> at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
> at
> java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385)
> at kafka.utils.Utils$.read(Unknown Source)
> at kafka.network.BoundedByteBufferReceive.readFrom(Unknown Source)
> at kafka.network.Receive$class.readCompletely(Unknown Source)
> at kafka.network.BoundedByteBufferReceive.readCompletely(Unknown
> Source)
> at kafka.network.BlockingChannel.receive(Unknown Source)
> at kafka.consumer.SimpleConsumer.liftedTree1$1(Unknown Source)
> at
> kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(Unknown
> Source)
> at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(Unknown
> Source)
> at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
> Source)
> at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
> Source)
> at kafka.metrics.KafkaTimer.time(Unknown Source)
> at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(Unknown Source)
> at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
> Source)
> at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
> Source)
> at kafka.metrics.KafkaTimer.time(Unknown Source)
> at kafka.consumer.SimpleConsumer.fetch(Unknown Source)
> at kafka.server.AbstractFetcherThread.processFetchRequest(Unknown
> Source)
> at kafka.server.AbstractFetcherThread.doWork(Unknown Source)
> at kafka.utils.ShutdownableThread.run(Unknown Source)
>
> My 8 consumers are running in parallel with different consumer group ids
> and trying to read from LOGLINE10.
> 2 of them are able to read the LOGLINE10 topic but the 6 others are
> failing with the above exception.
>
> Yashika
>
>
>
> ________________________________________
> From: Timothy Chen <tn...@gmail.com>
> Sent: Friday, April 25, 2014 11:40:33 AM
> To: users@kafka.apache.org
> Subject: Re: Kafka Performance Tuning
>
> Hi Yashika,
>
> No logs in broker log is not normal, can you verify if you turned off
> logging in your log4j properties file?
>
> If it is please enable it and try again, and see what is in the logs.
>
> Tim
>
> On Thu, Apr 24, 2014 at 10:53 PM, Yashika Gupta
> <ya...@impetus.co.in> wrote:
> > Jun,
> >
> > I am using Kafka 2.8.0- 0.8.0 version.
> > There are no logs for the past month in the controller and state-change
> log.
> >
> > Though I can see dome gc logs in the kafka-home-dir/logs folder.
> > zookeeper-gc.log
> > kafkaServer-gc.log
> >
> >
> > Yashika
> > __________________________
> > From: Jun Rao <ju...@gmail.com>
> > Sent: Friday, April 25, 2014 9:03 AM
> > To: users@kafka.apache.org
> > Subject: Re: Kafka Performance Tuning
> >
> > Which version of Kafka are you using? Any error in the controller and
> > state-change log?
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Thu, Apr 24, 2014 at 7:37 PM, Yashika Gupta
> > <ya...@impetus.co.in>wrote:
> >
> >> I am running a single broker and the leader column has 0 as the value.
> >>
> >> pushkar priyadarshi <pr...@gmail.com> wrote:
> >>
> >>
> >> you can use the kafka-list-topic.sh to find out if leader for particual
> >> topic is available.-1 in leader column might indicate trouble.
> >>
> >>
> >> On Fri, Apr 25, 2014 at 6:34 AM, Guozhang Wang <wa...@gmail.com>
> wrote:
> >>
> >> > Could you double check if the topic LOGFILE04 is already created on
> the
> >> > servers?
> >> >
> >> > Guozhang
> >> >
> >> >
> >> > On Thu, Apr 24, 2014 at 10:46 AM, Yashika Gupta <
> >> > yashika.gupta@impetus.co.in
> >> > > wrote:
> >> >
> >> > > Jun,
> >> > >
> >> > > The detailed logs are as follows:
> >> > >
> >> > > 24.04.2014 13:37:31812 INFO main kafka.producer.SyncProducer -
> >> > > Disconnecting from localhost:9092
> >> > > 24.04.2014 13:37:38612 WARN main
> kafka.producer.BrokerPartitionInfo -
> >> > > Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
> >> > > No partition metadata for topic LOGFILE04 due to
> >> > > kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]:
> class
> >> > > kafka.common.LeaderNotAvailableException
> >> > > 24.04.2014 13:37:40712 INFO main kafka.client.ClientUtils$ -
> Fetching
> >> > > metadata from broker id:0,host:localhost,port:9092 with correlation
> id
> >> 1
> >> > > for 1 topic(s) Set(LOGFILE04)
> >> > > 24.04.2014 13:37:41212 INFO main kafka.producer.SyncProducer -
> >> Connected
> >> > > to localhost:9092 for producing
> >> > > 24.04.2014 13:37:48812 INFO main kafka.producer.SyncProducer -
> >> > > Disconnecting from localhost:9092
> >> > > 24.04.2014 13:37:48912 WARN main kafka.producer.BrokerPartitionInfo
> -
> >> > > Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
> >> > > No partition metadata for topic LOGFILE04 due to
> >> > > kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]:
> class
> >> > > kafka.common.LeaderNotAvailableException
> >> > > 24.04.2014 13:37:49012 ERROR main
> >> > kafka.producer.async.DefaultEventHandler
> >> > > - Failed to collate messages by topic, partition due to: Failed to
> >> fetch
> >> > > topic metadata for topic: LOGFILE04
> >> > >
> >> > >
> >> > > 24.04.2014 13:39:96513 WARN
> >> > >
> >> >
> >>
> ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0
> >> > > kafka.consumer.ConsumerFetcherThread -
> >> > >
> >> >
> >>
> [ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0],
> >> > > Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 4;
> >> > ClientId:
> >> > >
> >> >
> >>
> produceLogLine2-ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0;
> >> > > ReplicaId: -1; MaxWait: 60000 ms; MinBytes: 1 bytes; RequestInfo:
> >> > > [LOGFILE04,0] -> PartitionFetchInfo(2,1048576)
> >> > > java.net.SocketTimeoutException
> >> > > at
> >> > >
> sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:229)
> >> > > at
> >> > sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
> >> > > at
> >> > >
> >> >
> >>
> java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385)
> >> > > at kafka.utils.Utils$.read(Unknown Source)
> >> > > at kafka.network.BoundedByteBufferReceive.readFrom(Unknown
> >> > Source)
> >> > > at kafka.network.Receive$class.readCompletely(Unknown
> Source)
> >> > > at
> >> kafka.network.BoundedByteBufferReceive.readCompletely(Unknown
> >> > > Source)
> >> > > at kafka.network.BlockingChannel.receive(Unknown Source)
> >> > > at kafka.consumer.SimpleConsumer.liftedTree1$1(Unknown
> Source)
> >> > > at
> >> > >
> >> >
> >>
> kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(Unknown
> >> > > Source)
> >> > > at
> >> > >
> >> >
> >>
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(Unknown
> >> > > Source)
> >> > > at
> >> > >
> >> >
> >>
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
> >> > > Source)
> >> > > at
> >> > >
> >> >
> >>
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
> >> > > Source)
> >> > > at kafka.metrics.KafkaTimer.time(Unknown Source)
> >> > > at
> >> > > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(Unknown
> >> > Source)
> >> > > at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
> >> > > Source)
> >> > > at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
> >> > > Source)
> >> > > at kafka.metrics.KafkaTimer.time(Unknown Source)
> >> > > at kafka.consumer.SimpleConsumer.fetch(Unknown Source)
> >> > > at
> >> kafka.server.AbstractFetcherThread.processFetchRequest(Unknown
> >> > > Source)
> >> > > at kafka.server.AbstractFetcherThread.doWork(Unknown Source)
> >> > > at kafka.utils.ShutdownableThread.run(Unknown Source)
> >> > >
> >> > >
> >> > > Regards,
> >> > > Yashika
> >> > > ________________________________________
> >> > > From: Jun Rao <ju...@gmail.com>
> >> > > Sent: Thursday, April 24, 2014 10:49 PM
> >> > > To: users@kafka.apache.org
> >> > > Subject: Re: Kafka Performance Tuning
> >> > >
> >> > > Before that error messge, the log should tell you the cause of the
> >> error.
> >> > > Could you dig that out?
> >> > >
> >> > > Thanks,
> >> > >
> >> > > Jun
> >> > >
> >> > >
> >> > > On Thu, Apr 24, 2014 at 10:12 AM, Yashika Gupta <
> >> > > yashika.gupta@impetus.co.in
> >> > > > wrote:
> >> > >
> >> > > > Hi,
> >> > > >
> >> > > > I am working on a POC where I have 1 Zookeeper and 2 Kafka
> Brokers on
> >> > my
> >> > > > local machine. I am running 8 sets of Kafka consumers and
> producers
> >> > > running
> >> > > > in parallel.
> >> > > >
> >> > > > Below are my configurations:
> >> > > > Consumer Configs:
> >> > > > zookeeper.session.timeout.ms=120000
> >> > > > zookeeper.sync.time.ms=2000
> >> > > > zookeeper.connection.timeout.ms=120000
> >> > > > auto.commit.interval.ms=60000
> >> > > > rebalance.backoff.ms=2000
> >> > > > fetch.wait.max.ms=60000
> >> > > > auto.offset.reset=smallest
> >> > > >
> >> > > > Producer configs:
> >> > > > key.serializer.class=kafka.serializer.StringEncoder
> >> > > > request.required.acks=-1
> >> > > > message.send.max.retries=3
> >> > > > request.timeout.ms=60000
> >> > > >
> >> > > > I have tried various more configuration changes but I am running
> into
> >> > the
> >> > > > same exception again and again.
> >> > > >
> >> > > > 17.04.2014 06:31:83216 ERROR pool-5-thread-1
> >> > > > kafka.producer.async.DefaultEventHandler - Failed to collate
> messages
> >> > by
> >> > > > topic, partition due to: Failed to fetch topic metadata for topic:
> >> > > TOPIC99
> >> > > > 17.04.2014 06:31:85616 ERROR pool-4-thread-1
> >> > > > kafka.producer.async.DefaultEventHandler - Failed to collate
> messages
> >> > by
> >> > > > topic, partition due to: Failed to fetch topic metadata for topic:
> >> > > TOPIC99
> >> > > >
> >> > > > I am not able to get to the root cause of the issue.
> >> > > > Appreciate your help.
> >> > > >
> >> > > > Regards,
> >> > > > Yashika
> >> > > >
> >> > > > ________________________________
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > NOTE: This message may contain information that is confidential,
> >> > > > proprietary, privileged or otherwise protected by law. The
> message is
> >> > > > intended solely for the named addressee. If received in error,
> please
> >> > > > destroy and notify the sender. Any use of this email is prohibited
> >> when
> >> > > > received in error. Impetus does not represent, warrant and/or
> >> > guarantee,
> >> > > > that the integrity of this communication has been maintained nor
> that
> >> > the
> >> > > > communication is free of errors, virus, interception or
> interference.
> >> > > >
> >> > >
> >> > > ________________________________
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > NOTE: This message may contain information that is confidential,
> >> > > proprietary, privileged or otherwise protected by law. The message
> is
> >> > > intended solely for the named addressee. If received in error,
> please
> >> > > destroy and notify the sender. Any use of this email is prohibited
> when
> >> > > received in error. Impetus does not represent, warrant and/or
> >> guarantee,
> >> > > that the integrity of this communication has been maintained nor
> that
> >> the
> >> > > communication is free of errors, virus, interception or
> interference.
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > -- Guozhang
> >> >
> >>
> >> ________________________________
> >>
> >>
> >>
> >>
> >>
> >>
> >> NOTE: This message may contain information that is confidential,
> >> proprietary, privileged or otherwise protected by law. The message is
> >> intended solely for the named addressee. If received in error, please
> >> destroy and notify the sender. Any use of this email is prohibited when
> >> received in error. Impetus does not represent, warrant and/or guarantee,
> >> that the integrity of this communication has been maintained nor that
> the
> >> communication is free of errors, virus, interception or interference.
> >>
> >
> > ________________________________
> >
> >
> >
> >
> >
> >
> > NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>
> ________________________________
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>
RE: Kafka Performance Tuning
Posted by Yashika Gupta <ya...@impetus.co.in>.
Timothy,
I checked out https://issues.apache.org/jira/browse/KAFKA-1124
So, I created the topic manually and re-ran my test and has enabled log4j properties.
There are no ERROR logs in controller and state change logs.
And I m still getting the SocketTimeout Exception:
25.04.2014 03:05:10115 WARN ConsumerFetcherThread-consumerDeviceError2_vcmd-devanshu-1398408813021-efe24d49-0-0 kafka.consumer.ConsumerFetcherThread - [ConsumerFetcherThread-consumerDeviceError2_vcmd-devanshu-1398408813021-efe24d49-0-0], Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 26; ClientId: consumerDeviceError2-ConsumerFetcherThread-consumerDeviceError2_vcmd-devanshu-1398408813021-efe24d49-0-0; ReplicaId: -1; MaxWait: 60000 ms; MinBytes: 1 bytes; RequestInfo: [LOGLINE10,0] -> PartitionFetchInfo(12,1048576)
java.net.SocketTimeoutException
at sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:229)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
at java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385)
at kafka.utils.Utils$.read(Unknown Source)
at kafka.network.BoundedByteBufferReceive.readFrom(Unknown Source)
at kafka.network.Receive$class.readCompletely(Unknown Source)
at kafka.network.BoundedByteBufferReceive.readCompletely(Unknown Source)
at kafka.network.BlockingChannel.receive(Unknown Source)
at kafka.consumer.SimpleConsumer.liftedTree1$1(Unknown Source)
at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(Unknown Source)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(Unknown Source)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown Source)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown Source)
at kafka.metrics.KafkaTimer.time(Unknown Source)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(Unknown Source)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown Source)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown Source)
at kafka.metrics.KafkaTimer.time(Unknown Source)
at kafka.consumer.SimpleConsumer.fetch(Unknown Source)
at kafka.server.AbstractFetcherThread.processFetchRequest(Unknown Source)
at kafka.server.AbstractFetcherThread.doWork(Unknown Source)
at kafka.utils.ShutdownableThread.run(Unknown Source)
My 8 consumers are running in parallel with different consumer group ids and trying to read from LOGLINE10.
2 of them are able to read the LOGLINE10 topic but the 6 others are failing with the above exception.
Yashika
________________________________________
From: Timothy Chen <tn...@gmail.com>
Sent: Friday, April 25, 2014 11:40:33 AM
To: users@kafka.apache.org
Subject: Re: Kafka Performance Tuning
Hi Yashika,
No logs in broker log is not normal, can you verify if you turned off
logging in your log4j properties file?
If it is please enable it and try again, and see what is in the logs.
Tim
On Thu, Apr 24, 2014 at 10:53 PM, Yashika Gupta
<ya...@impetus.co.in> wrote:
> Jun,
>
> I am using Kafka 2.8.0- 0.8.0 version.
> There are no logs for the past month in the controller and state-change log.
>
> Though I can see dome gc logs in the kafka-home-dir/logs folder.
> zookeeper-gc.log
> kafkaServer-gc.log
>
>
> Yashika
> __________________________
> From: Jun Rao <ju...@gmail.com>
> Sent: Friday, April 25, 2014 9:03 AM
> To: users@kafka.apache.org
> Subject: Re: Kafka Performance Tuning
>
> Which version of Kafka are you using? Any error in the controller and
> state-change log?
>
> Thanks,
>
> Jun
>
>
> On Thu, Apr 24, 2014 at 7:37 PM, Yashika Gupta
> <ya...@impetus.co.in>wrote:
>
>> I am running a single broker and the leader column has 0 as the value.
>>
>> pushkar priyadarshi <pr...@gmail.com> wrote:
>>
>>
>> you can use the kafka-list-topic.sh to find out if leader for particual
>> topic is available.-1 in leader column might indicate trouble.
>>
>>
>> On Fri, Apr 25, 2014 at 6:34 AM, Guozhang Wang <wa...@gmail.com> wrote:
>>
>> > Could you double check if the topic LOGFILE04 is already created on the
>> > servers?
>> >
>> > Guozhang
>> >
>> >
>> > On Thu, Apr 24, 2014 at 10:46 AM, Yashika Gupta <
>> > yashika.gupta@impetus.co.in
>> > > wrote:
>> >
>> > > Jun,
>> > >
>> > > The detailed logs are as follows:
>> > >
>> > > 24.04.2014 13:37:31812 INFO main kafka.producer.SyncProducer -
>> > > Disconnecting from localhost:9092
>> > > 24.04.2014 13:37:38612 WARN main kafka.producer.BrokerPartitionInfo -
>> > > Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
>> > > No partition metadata for topic LOGFILE04 due to
>> > > kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class
>> > > kafka.common.LeaderNotAvailableException
>> > > 24.04.2014 13:37:40712 INFO main kafka.client.ClientUtils$ - Fetching
>> > > metadata from broker id:0,host:localhost,port:9092 with correlation id
>> 1
>> > > for 1 topic(s) Set(LOGFILE04)
>> > > 24.04.2014 13:37:41212 INFO main kafka.producer.SyncProducer -
>> Connected
>> > > to localhost:9092 for producing
>> > > 24.04.2014 13:37:48812 INFO main kafka.producer.SyncProducer -
>> > > Disconnecting from localhost:9092
>> > > 24.04.2014 13:37:48912 WARN main kafka.producer.BrokerPartitionInfo -
>> > > Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
>> > > No partition metadata for topic LOGFILE04 due to
>> > > kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class
>> > > kafka.common.LeaderNotAvailableException
>> > > 24.04.2014 13:37:49012 ERROR main
>> > kafka.producer.async.DefaultEventHandler
>> > > - Failed to collate messages by topic, partition due to: Failed to
>> fetch
>> > > topic metadata for topic: LOGFILE04
>> > >
>> > >
>> > > 24.04.2014 13:39:96513 WARN
>> > >
>> >
>> ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0
>> > > kafka.consumer.ConsumerFetcherThread -
>> > >
>> >
>> [ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0],
>> > > Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 4;
>> > ClientId:
>> > >
>> >
>> produceLogLine2-ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0;
>> > > ReplicaId: -1; MaxWait: 60000 ms; MinBytes: 1 bytes; RequestInfo:
>> > > [LOGFILE04,0] -> PartitionFetchInfo(2,1048576)
>> > > java.net.SocketTimeoutException
>> > > at
>> > > sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:229)
>> > > at
>> > sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
>> > > at
>> > >
>> >
>> java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385)
>> > > at kafka.utils.Utils$.read(Unknown Source)
>> > > at kafka.network.BoundedByteBufferReceive.readFrom(Unknown
>> > Source)
>> > > at kafka.network.Receive$class.readCompletely(Unknown Source)
>> > > at
>> kafka.network.BoundedByteBufferReceive.readCompletely(Unknown
>> > > Source)
>> > > at kafka.network.BlockingChannel.receive(Unknown Source)
>> > > at kafka.consumer.SimpleConsumer.liftedTree1$1(Unknown Source)
>> > > at
>> > >
>> >
>> kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(Unknown
>> > > Source)
>> > > at
>> > >
>> >
>> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(Unknown
>> > > Source)
>> > > at
>> > >
>> >
>> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
>> > > Source)
>> > > at
>> > >
>> >
>> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
>> > > Source)
>> > > at kafka.metrics.KafkaTimer.time(Unknown Source)
>> > > at
>> > > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(Unknown
>> > Source)
>> > > at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
>> > > Source)
>> > > at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
>> > > Source)
>> > > at kafka.metrics.KafkaTimer.time(Unknown Source)
>> > > at kafka.consumer.SimpleConsumer.fetch(Unknown Source)
>> > > at
>> kafka.server.AbstractFetcherThread.processFetchRequest(Unknown
>> > > Source)
>> > > at kafka.server.AbstractFetcherThread.doWork(Unknown Source)
>> > > at kafka.utils.ShutdownableThread.run(Unknown Source)
>> > >
>> > >
>> > > Regards,
>> > > Yashika
>> > > ________________________________________
>> > > From: Jun Rao <ju...@gmail.com>
>> > > Sent: Thursday, April 24, 2014 10:49 PM
>> > > To: users@kafka.apache.org
>> > > Subject: Re: Kafka Performance Tuning
>> > >
>> > > Before that error messge, the log should tell you the cause of the
>> error.
>> > > Could you dig that out?
>> > >
>> > > Thanks,
>> > >
>> > > Jun
>> > >
>> > >
>> > > On Thu, Apr 24, 2014 at 10:12 AM, Yashika Gupta <
>> > > yashika.gupta@impetus.co.in
>> > > > wrote:
>> > >
>> > > > Hi,
>> > > >
>> > > > I am working on a POC where I have 1 Zookeeper and 2 Kafka Brokers on
>> > my
>> > > > local machine. I am running 8 sets of Kafka consumers and producers
>> > > running
>> > > > in parallel.
>> > > >
>> > > > Below are my configurations:
>> > > > Consumer Configs:
>> > > > zookeeper.session.timeout.ms=120000
>> > > > zookeeper.sync.time.ms=2000
>> > > > zookeeper.connection.timeout.ms=120000
>> > > > auto.commit.interval.ms=60000
>> > > > rebalance.backoff.ms=2000
>> > > > fetch.wait.max.ms=60000
>> > > > auto.offset.reset=smallest
>> > > >
>> > > > Producer configs:
>> > > > key.serializer.class=kafka.serializer.StringEncoder
>> > > > request.required.acks=-1
>> > > > message.send.max.retries=3
>> > > > request.timeout.ms=60000
>> > > >
>> > > > I have tried various more configuration changes but I am running into
>> > the
>> > > > same exception again and again.
>> > > >
>> > > > 17.04.2014 06:31:83216 ERROR pool-5-thread-1
>> > > > kafka.producer.async.DefaultEventHandler - Failed to collate messages
>> > by
>> > > > topic, partition due to: Failed to fetch topic metadata for topic:
>> > > TOPIC99
>> > > > 17.04.2014 06:31:85616 ERROR pool-4-thread-1
>> > > > kafka.producer.async.DefaultEventHandler - Failed to collate messages
>> > by
>> > > > topic, partition due to: Failed to fetch topic metadata for topic:
>> > > TOPIC99
>> > > >
>> > > > I am not able to get to the root cause of the issue.
>> > > > Appreciate your help.
>> > > >
>> > > > Regards,
>> > > > Yashika
>> > > >
>> > > > ________________________________
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > NOTE: This message may contain information that is confidential,
>> > > > proprietary, privileged or otherwise protected by law. The message is
>> > > > intended solely for the named addressee. If received in error, please
>> > > > destroy and notify the sender. Any use of this email is prohibited
>> when
>> > > > received in error. Impetus does not represent, warrant and/or
>> > guarantee,
>> > > > that the integrity of this communication has been maintained nor that
>> > the
>> > > > communication is free of errors, virus, interception or interference.
>> > > >
>> > >
>> > > ________________________________
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > NOTE: This message may contain information that is confidential,
>> > > proprietary, privileged or otherwise protected by law. The message is
>> > > intended solely for the named addressee. If received in error, please
>> > > destroy and notify the sender. Any use of this email is prohibited when
>> > > received in error. Impetus does not represent, warrant and/or
>> guarantee,
>> > > that the integrity of this communication has been maintained nor that
>> the
>> > > communication is free of errors, virus, interception or interference.
>> > >
>> >
>> >
>> >
>> > --
>> > -- Guozhang
>> >
>>
>> ________________________________
>>
>>
>>
>>
>>
>>
>> NOTE: This message may contain information that is confidential,
>> proprietary, privileged or otherwise protected by law. The message is
>> intended solely for the named addressee. If received in error, please
>> destroy and notify the sender. Any use of this email is prohibited when
>> received in error. Impetus does not represent, warrant and/or guarantee,
>> that the integrity of this communication has been maintained nor that the
>> communication is free of errors, virus, interception or interference.
>>
>
> ________________________________
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
________________________________
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
Re: Kafka Performance Tuning
Posted by Timothy Chen <tn...@gmail.com>.
Hi Yashika,
No logs in broker log is not normal, can you verify if you turned off
logging in your log4j properties file?
If it is please enable it and try again, and see what is in the logs.
Tim
On Thu, Apr 24, 2014 at 10:53 PM, Yashika Gupta
<ya...@impetus.co.in> wrote:
> Jun,
>
> I am using Kafka 2.8.0- 0.8.0 version.
> There are no logs for the past month in the controller and state-change log.
>
> Though I can see dome gc logs in the kafka-home-dir/logs folder.
> zookeeper-gc.log
> kafkaServer-gc.log
>
>
> Yashika
> __________________________
> From: Jun Rao <ju...@gmail.com>
> Sent: Friday, April 25, 2014 9:03 AM
> To: users@kafka.apache.org
> Subject: Re: Kafka Performance Tuning
>
> Which version of Kafka are you using? Any error in the controller and
> state-change log?
>
> Thanks,
>
> Jun
>
>
> On Thu, Apr 24, 2014 at 7:37 PM, Yashika Gupta
> <ya...@impetus.co.in>wrote:
>
>> I am running a single broker and the leader column has 0 as the value.
>>
>> pushkar priyadarshi <pr...@gmail.com> wrote:
>>
>>
>> you can use the kafka-list-topic.sh to find out if leader for particual
>> topic is available.-1 in leader column might indicate trouble.
>>
>>
>> On Fri, Apr 25, 2014 at 6:34 AM, Guozhang Wang <wa...@gmail.com> wrote:
>>
>> > Could you double check if the topic LOGFILE04 is already created on the
>> > servers?
>> >
>> > Guozhang
>> >
>> >
>> > On Thu, Apr 24, 2014 at 10:46 AM, Yashika Gupta <
>> > yashika.gupta@impetus.co.in
>> > > wrote:
>> >
>> > > Jun,
>> > >
>> > > The detailed logs are as follows:
>> > >
>> > > 24.04.2014 13:37:31812 INFO main kafka.producer.SyncProducer -
>> > > Disconnecting from localhost:9092
>> > > 24.04.2014 13:37:38612 WARN main kafka.producer.BrokerPartitionInfo -
>> > > Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
>> > > No partition metadata for topic LOGFILE04 due to
>> > > kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class
>> > > kafka.common.LeaderNotAvailableException
>> > > 24.04.2014 13:37:40712 INFO main kafka.client.ClientUtils$ - Fetching
>> > > metadata from broker id:0,host:localhost,port:9092 with correlation id
>> 1
>> > > for 1 topic(s) Set(LOGFILE04)
>> > > 24.04.2014 13:37:41212 INFO main kafka.producer.SyncProducer -
>> Connected
>> > > to localhost:9092 for producing
>> > > 24.04.2014 13:37:48812 INFO main kafka.producer.SyncProducer -
>> > > Disconnecting from localhost:9092
>> > > 24.04.2014 13:37:48912 WARN main kafka.producer.BrokerPartitionInfo -
>> > > Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
>> > > No partition metadata for topic LOGFILE04 due to
>> > > kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class
>> > > kafka.common.LeaderNotAvailableException
>> > > 24.04.2014 13:37:49012 ERROR main
>> > kafka.producer.async.DefaultEventHandler
>> > > - Failed to collate messages by topic, partition due to: Failed to
>> fetch
>> > > topic metadata for topic: LOGFILE04
>> > >
>> > >
>> > > 24.04.2014 13:39:96513 WARN
>> > >
>> >
>> ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0
>> > > kafka.consumer.ConsumerFetcherThread -
>> > >
>> >
>> [ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0],
>> > > Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 4;
>> > ClientId:
>> > >
>> >
>> produceLogLine2-ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0;
>> > > ReplicaId: -1; MaxWait: 60000 ms; MinBytes: 1 bytes; RequestInfo:
>> > > [LOGFILE04,0] -> PartitionFetchInfo(2,1048576)
>> > > java.net.SocketTimeoutException
>> > > at
>> > > sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:229)
>> > > at
>> > sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
>> > > at
>> > >
>> >
>> java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385)
>> > > at kafka.utils.Utils$.read(Unknown Source)
>> > > at kafka.network.BoundedByteBufferReceive.readFrom(Unknown
>> > Source)
>> > > at kafka.network.Receive$class.readCompletely(Unknown Source)
>> > > at
>> kafka.network.BoundedByteBufferReceive.readCompletely(Unknown
>> > > Source)
>> > > at kafka.network.BlockingChannel.receive(Unknown Source)
>> > > at kafka.consumer.SimpleConsumer.liftedTree1$1(Unknown Source)
>> > > at
>> > >
>> >
>> kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(Unknown
>> > > Source)
>> > > at
>> > >
>> >
>> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(Unknown
>> > > Source)
>> > > at
>> > >
>> >
>> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
>> > > Source)
>> > > at
>> > >
>> >
>> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
>> > > Source)
>> > > at kafka.metrics.KafkaTimer.time(Unknown Source)
>> > > at
>> > > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(Unknown
>> > Source)
>> > > at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
>> > > Source)
>> > > at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
>> > > Source)
>> > > at kafka.metrics.KafkaTimer.time(Unknown Source)
>> > > at kafka.consumer.SimpleConsumer.fetch(Unknown Source)
>> > > at
>> kafka.server.AbstractFetcherThread.processFetchRequest(Unknown
>> > > Source)
>> > > at kafka.server.AbstractFetcherThread.doWork(Unknown Source)
>> > > at kafka.utils.ShutdownableThread.run(Unknown Source)
>> > >
>> > >
>> > > Regards,
>> > > Yashika
>> > > ________________________________________
>> > > From: Jun Rao <ju...@gmail.com>
>> > > Sent: Thursday, April 24, 2014 10:49 PM
>> > > To: users@kafka.apache.org
>> > > Subject: Re: Kafka Performance Tuning
>> > >
>> > > Before that error messge, the log should tell you the cause of the
>> error.
>> > > Could you dig that out?
>> > >
>> > > Thanks,
>> > >
>> > > Jun
>> > >
>> > >
>> > > On Thu, Apr 24, 2014 at 10:12 AM, Yashika Gupta <
>> > > yashika.gupta@impetus.co.in
>> > > > wrote:
>> > >
>> > > > Hi,
>> > > >
>> > > > I am working on a POC where I have 1 Zookeeper and 2 Kafka Brokers on
>> > my
>> > > > local machine. I am running 8 sets of Kafka consumers and producers
>> > > running
>> > > > in parallel.
>> > > >
>> > > > Below are my configurations:
>> > > > Consumer Configs:
>> > > > zookeeper.session.timeout.ms=120000
>> > > > zookeeper.sync.time.ms=2000
>> > > > zookeeper.connection.timeout.ms=120000
>> > > > auto.commit.interval.ms=60000
>> > > > rebalance.backoff.ms=2000
>> > > > fetch.wait.max.ms=60000
>> > > > auto.offset.reset=smallest
>> > > >
>> > > > Producer configs:
>> > > > key.serializer.class=kafka.serializer.StringEncoder
>> > > > request.required.acks=-1
>> > > > message.send.max.retries=3
>> > > > request.timeout.ms=60000
>> > > >
>> > > > I have tried various more configuration changes but I am running into
>> > the
>> > > > same exception again and again.
>> > > >
>> > > > 17.04.2014 06:31:83216 ERROR pool-5-thread-1
>> > > > kafka.producer.async.DefaultEventHandler - Failed to collate messages
>> > by
>> > > > topic, partition due to: Failed to fetch topic metadata for topic:
>> > > TOPIC99
>> > > > 17.04.2014 06:31:85616 ERROR pool-4-thread-1
>> > > > kafka.producer.async.DefaultEventHandler - Failed to collate messages
>> > by
>> > > > topic, partition due to: Failed to fetch topic metadata for topic:
>> > > TOPIC99
>> > > >
>> > > > I am not able to get to the root cause of the issue.
>> > > > Appreciate your help.
>> > > >
>> > > > Regards,
>> > > > Yashika
>> > > >
>> > > > ________________________________
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > NOTE: This message may contain information that is confidential,
>> > > > proprietary, privileged or otherwise protected by law. The message is
>> > > > intended solely for the named addressee. If received in error, please
>> > > > destroy and notify the sender. Any use of this email is prohibited
>> when
>> > > > received in error. Impetus does not represent, warrant and/or
>> > guarantee,
>> > > > that the integrity of this communication has been maintained nor that
>> > the
>> > > > communication is free of errors, virus, interception or interference.
>> > > >
>> > >
>> > > ________________________________
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > NOTE: This message may contain information that is confidential,
>> > > proprietary, privileged or otherwise protected by law. The message is
>> > > intended solely for the named addressee. If received in error, please
>> > > destroy and notify the sender. Any use of this email is prohibited when
>> > > received in error. Impetus does not represent, warrant and/or
>> guarantee,
>> > > that the integrity of this communication has been maintained nor that
>> the
>> > > communication is free of errors, virus, interception or interference.
>> > >
>> >
>> >
>> >
>> > --
>> > -- Guozhang
>> >
>>
>> ________________________________
>>
>>
>>
>>
>>
>>
>> NOTE: This message may contain information that is confidential,
>> proprietary, privileged or otherwise protected by law. The message is
>> intended solely for the named addressee. If received in error, please
>> destroy and notify the sender. Any use of this email is prohibited when
>> received in error. Impetus does not represent, warrant and/or guarantee,
>> that the integrity of this communication has been maintained nor that the
>> communication is free of errors, virus, interception or interference.
>>
>
> ________________________________
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
RE: Kafka Performance Tuning
Posted by Yashika Gupta <ya...@impetus.co.in>.
Jun,
I am using Kafka 2.8.0- 0.8.0 version.
There are no logs for the past month in the controller and state-change log.
Though I can see dome gc logs in the kafka-home-dir/logs folder.
zookeeper-gc.log
kafkaServer-gc.log
Yashika
__________________________
From: Jun Rao <ju...@gmail.com>
Sent: Friday, April 25, 2014 9:03 AM
To: users@kafka.apache.org
Subject: Re: Kafka Performance Tuning
Which version of Kafka are you using? Any error in the controller and
state-change log?
Thanks,
Jun
On Thu, Apr 24, 2014 at 7:37 PM, Yashika Gupta
<ya...@impetus.co.in>wrote:
> I am running a single broker and the leader column has 0 as the value.
>
> pushkar priyadarshi <pr...@gmail.com> wrote:
>
>
> you can use the kafka-list-topic.sh to find out if leader for particual
> topic is available.-1 in leader column might indicate trouble.
>
>
> On Fri, Apr 25, 2014 at 6:34 AM, Guozhang Wang <wa...@gmail.com> wrote:
>
> > Could you double check if the topic LOGFILE04 is already created on the
> > servers?
> >
> > Guozhang
> >
> >
> > On Thu, Apr 24, 2014 at 10:46 AM, Yashika Gupta <
> > yashika.gupta@impetus.co.in
> > > wrote:
> >
> > > Jun,
> > >
> > > The detailed logs are as follows:
> > >
> > > 24.04.2014 13:37:31812 INFO main kafka.producer.SyncProducer -
> > > Disconnecting from localhost:9092
> > > 24.04.2014 13:37:38612 WARN main kafka.producer.BrokerPartitionInfo -
> > > Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
> > > No partition metadata for topic LOGFILE04 due to
> > > kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class
> > > kafka.common.LeaderNotAvailableException
> > > 24.04.2014 13:37:40712 INFO main kafka.client.ClientUtils$ - Fetching
> > > metadata from broker id:0,host:localhost,port:9092 with correlation id
> 1
> > > for 1 topic(s) Set(LOGFILE04)
> > > 24.04.2014 13:37:41212 INFO main kafka.producer.SyncProducer -
> Connected
> > > to localhost:9092 for producing
> > > 24.04.2014 13:37:48812 INFO main kafka.producer.SyncProducer -
> > > Disconnecting from localhost:9092
> > > 24.04.2014 13:37:48912 WARN main kafka.producer.BrokerPartitionInfo -
> > > Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
> > > No partition metadata for topic LOGFILE04 due to
> > > kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class
> > > kafka.common.LeaderNotAvailableException
> > > 24.04.2014 13:37:49012 ERROR main
> > kafka.producer.async.DefaultEventHandler
> > > - Failed to collate messages by topic, partition due to: Failed to
> fetch
> > > topic metadata for topic: LOGFILE04
> > >
> > >
> > > 24.04.2014 13:39:96513 WARN
> > >
> >
> ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0
> > > kafka.consumer.ConsumerFetcherThread -
> > >
> >
> [ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0],
> > > Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 4;
> > ClientId:
> > >
> >
> produceLogLine2-ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0;
> > > ReplicaId: -1; MaxWait: 60000 ms; MinBytes: 1 bytes; RequestInfo:
> > > [LOGFILE04,0] -> PartitionFetchInfo(2,1048576)
> > > java.net.SocketTimeoutException
> > > at
> > > sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:229)
> > > at
> > sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
> > > at
> > >
> >
> java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385)
> > > at kafka.utils.Utils$.read(Unknown Source)
> > > at kafka.network.BoundedByteBufferReceive.readFrom(Unknown
> > Source)
> > > at kafka.network.Receive$class.readCompletely(Unknown Source)
> > > at
> kafka.network.BoundedByteBufferReceive.readCompletely(Unknown
> > > Source)
> > > at kafka.network.BlockingChannel.receive(Unknown Source)
> > > at kafka.consumer.SimpleConsumer.liftedTree1$1(Unknown Source)
> > > at
> > >
> >
> kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(Unknown
> > > Source)
> > > at
> > >
> >
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(Unknown
> > > Source)
> > > at
> > >
> >
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
> > > Source)
> > > at
> > >
> >
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
> > > Source)
> > > at kafka.metrics.KafkaTimer.time(Unknown Source)
> > > at
> > > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(Unknown
> > Source)
> > > at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
> > > Source)
> > > at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
> > > Source)
> > > at kafka.metrics.KafkaTimer.time(Unknown Source)
> > > at kafka.consumer.SimpleConsumer.fetch(Unknown Source)
> > > at
> kafka.server.AbstractFetcherThread.processFetchRequest(Unknown
> > > Source)
> > > at kafka.server.AbstractFetcherThread.doWork(Unknown Source)
> > > at kafka.utils.ShutdownableThread.run(Unknown Source)
> > >
> > >
> > > Regards,
> > > Yashika
> > > ________________________________________
> > > From: Jun Rao <ju...@gmail.com>
> > > Sent: Thursday, April 24, 2014 10:49 PM
> > > To: users@kafka.apache.org
> > > Subject: Re: Kafka Performance Tuning
> > >
> > > Before that error messge, the log should tell you the cause of the
> error.
> > > Could you dig that out?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > > On Thu, Apr 24, 2014 at 10:12 AM, Yashika Gupta <
> > > yashika.gupta@impetus.co.in
> > > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I am working on a POC where I have 1 Zookeeper and 2 Kafka Brokers on
> > my
> > > > local machine. I am running 8 sets of Kafka consumers and producers
> > > running
> > > > in parallel.
> > > >
> > > > Below are my configurations:
> > > > Consumer Configs:
> > > > zookeeper.session.timeout.ms=120000
> > > > zookeeper.sync.time.ms=2000
> > > > zookeeper.connection.timeout.ms=120000
> > > > auto.commit.interval.ms=60000
> > > > rebalance.backoff.ms=2000
> > > > fetch.wait.max.ms=60000
> > > > auto.offset.reset=smallest
> > > >
> > > > Producer configs:
> > > > key.serializer.class=kafka.serializer.StringEncoder
> > > > request.required.acks=-1
> > > > message.send.max.retries=3
> > > > request.timeout.ms=60000
> > > >
> > > > I have tried various more configuration changes but I am running into
> > the
> > > > same exception again and again.
> > > >
> > > > 17.04.2014 06:31:83216 ERROR pool-5-thread-1
> > > > kafka.producer.async.DefaultEventHandler - Failed to collate messages
> > by
> > > > topic, partition due to: Failed to fetch topic metadata for topic:
> > > TOPIC99
> > > > 17.04.2014 06:31:85616 ERROR pool-4-thread-1
> > > > kafka.producer.async.DefaultEventHandler - Failed to collate messages
> > by
> > > > topic, partition due to: Failed to fetch topic metadata for topic:
> > > TOPIC99
> > > >
> > > > I am not able to get to the root cause of the issue.
> > > > Appreciate your help.
> > > >
> > > > Regards,
> > > > Yashika
> > > >
> > > > ________________________________
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > NOTE: This message may contain information that is confidential,
> > > > proprietary, privileged or otherwise protected by law. The message is
> > > > intended solely for the named addressee. If received in error, please
> > > > destroy and notify the sender. Any use of this email is prohibited
> when
> > > > received in error. Impetus does not represent, warrant and/or
> > guarantee,
> > > > that the integrity of this communication has been maintained nor that
> > the
> > > > communication is free of errors, virus, interception or interference.
> > > >
> > >
> > > ________________________________
> > >
> > >
> > >
> > >
> > >
> > >
> > > NOTE: This message may contain information that is confidential,
> > > proprietary, privileged or otherwise protected by law. The message is
> > > intended solely for the named addressee. If received in error, please
> > > destroy and notify the sender. Any use of this email is prohibited when
> > > received in error. Impetus does not represent, warrant and/or
> guarantee,
> > > that the integrity of this communication has been maintained nor that
> the
> > > communication is free of errors, virus, interception or interference.
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>
> ________________________________
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>
________________________________
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
Re: Kafka Performance Tuning
Posted by Jun Rao <ju...@gmail.com>.
Which version of Kafka are you using? Any error in the controller and
state-change log?
Thanks,
Jun
On Thu, Apr 24, 2014 at 7:37 PM, Yashika Gupta
<ya...@impetus.co.in>wrote:
> I am running a single broker and the leader column has 0 as the value.
>
> pushkar priyadarshi <pr...@gmail.com> wrote:
>
>
> you can use the kafka-list-topic.sh to find out if leader for particual
> topic is available.-1 in leader column might indicate trouble.
>
>
> On Fri, Apr 25, 2014 at 6:34 AM, Guozhang Wang <wa...@gmail.com> wrote:
>
> > Could you double check if the topic LOGFILE04 is already created on the
> > servers?
> >
> > Guozhang
> >
> >
> > On Thu, Apr 24, 2014 at 10:46 AM, Yashika Gupta <
> > yashika.gupta@impetus.co.in
> > > wrote:
> >
> > > Jun,
> > >
> > > The detailed logs are as follows:
> > >
> > > 24.04.2014 13:37:31812 INFO main kafka.producer.SyncProducer -
> > > Disconnecting from localhost:9092
> > > 24.04.2014 13:37:38612 WARN main kafka.producer.BrokerPartitionInfo -
> > > Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
> > > No partition metadata for topic LOGFILE04 due to
> > > kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class
> > > kafka.common.LeaderNotAvailableException
> > > 24.04.2014 13:37:40712 INFO main kafka.client.ClientUtils$ - Fetching
> > > metadata from broker id:0,host:localhost,port:9092 with correlation id
> 1
> > > for 1 topic(s) Set(LOGFILE04)
> > > 24.04.2014 13:37:41212 INFO main kafka.producer.SyncProducer -
> Connected
> > > to localhost:9092 for producing
> > > 24.04.2014 13:37:48812 INFO main kafka.producer.SyncProducer -
> > > Disconnecting from localhost:9092
> > > 24.04.2014 13:37:48912 WARN main kafka.producer.BrokerPartitionInfo -
> > > Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
> > > No partition metadata for topic LOGFILE04 due to
> > > kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class
> > > kafka.common.LeaderNotAvailableException
> > > 24.04.2014 13:37:49012 ERROR main
> > kafka.producer.async.DefaultEventHandler
> > > - Failed to collate messages by topic, partition due to: Failed to
> fetch
> > > topic metadata for topic: LOGFILE04
> > >
> > >
> > > 24.04.2014 13:39:96513 WARN
> > >
> >
> ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0
> > > kafka.consumer.ConsumerFetcherThread -
> > >
> >
> [ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0],
> > > Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 4;
> > ClientId:
> > >
> >
> produceLogLine2-ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0;
> > > ReplicaId: -1; MaxWait: 60000 ms; MinBytes: 1 bytes; RequestInfo:
> > > [LOGFILE04,0] -> PartitionFetchInfo(2,1048576)
> > > java.net.SocketTimeoutException
> > > at
> > > sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:229)
> > > at
> > sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
> > > at
> > >
> >
> java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385)
> > > at kafka.utils.Utils$.read(Unknown Source)
> > > at kafka.network.BoundedByteBufferReceive.readFrom(Unknown
> > Source)
> > > at kafka.network.Receive$class.readCompletely(Unknown Source)
> > > at
> kafka.network.BoundedByteBufferReceive.readCompletely(Unknown
> > > Source)
> > > at kafka.network.BlockingChannel.receive(Unknown Source)
> > > at kafka.consumer.SimpleConsumer.liftedTree1$1(Unknown Source)
> > > at
> > >
> >
> kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(Unknown
> > > Source)
> > > at
> > >
> >
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(Unknown
> > > Source)
> > > at
> > >
> >
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
> > > Source)
> > > at
> > >
> >
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
> > > Source)
> > > at kafka.metrics.KafkaTimer.time(Unknown Source)
> > > at
> > > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(Unknown
> > Source)
> > > at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
> > > Source)
> > > at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
> > > Source)
> > > at kafka.metrics.KafkaTimer.time(Unknown Source)
> > > at kafka.consumer.SimpleConsumer.fetch(Unknown Source)
> > > at
> kafka.server.AbstractFetcherThread.processFetchRequest(Unknown
> > > Source)
> > > at kafka.server.AbstractFetcherThread.doWork(Unknown Source)
> > > at kafka.utils.ShutdownableThread.run(Unknown Source)
> > >
> > >
> > > Regards,
> > > Yashika
> > > ________________________________________
> > > From: Jun Rao <ju...@gmail.com>
> > > Sent: Thursday, April 24, 2014 10:49 PM
> > > To: users@kafka.apache.org
> > > Subject: Re: Kafka Performance Tuning
> > >
> > > Before that error messge, the log should tell you the cause of the
> error.
> > > Could you dig that out?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > > On Thu, Apr 24, 2014 at 10:12 AM, Yashika Gupta <
> > > yashika.gupta@impetus.co.in
> > > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I am working on a POC where I have 1 Zookeeper and 2 Kafka Brokers on
> > my
> > > > local machine. I am running 8 sets of Kafka consumers and producers
> > > running
> > > > in parallel.
> > > >
> > > > Below are my configurations:
> > > > Consumer Configs:
> > > > zookeeper.session.timeout.ms=120000
> > > > zookeeper.sync.time.ms=2000
> > > > zookeeper.connection.timeout.ms=120000
> > > > auto.commit.interval.ms=60000
> > > > rebalance.backoff.ms=2000
> > > > fetch.wait.max.ms=60000
> > > > auto.offset.reset=smallest
> > > >
> > > > Producer configs:
> > > > key.serializer.class=kafka.serializer.StringEncoder
> > > > request.required.acks=-1
> > > > message.send.max.retries=3
> > > > request.timeout.ms=60000
> > > >
> > > > I have tried various more configuration changes but I am running into
> > the
> > > > same exception again and again.
> > > >
> > > > 17.04.2014 06:31:83216 ERROR pool-5-thread-1
> > > > kafka.producer.async.DefaultEventHandler - Failed to collate messages
> > by
> > > > topic, partition due to: Failed to fetch topic metadata for topic:
> > > TOPIC99
> > > > 17.04.2014 06:31:85616 ERROR pool-4-thread-1
> > > > kafka.producer.async.DefaultEventHandler - Failed to collate messages
> > by
> > > > topic, partition due to: Failed to fetch topic metadata for topic:
> > > TOPIC99
> > > >
> > > > I am not able to get to the root cause of the issue.
> > > > Appreciate your help.
> > > >
> > > > Regards,
> > > > Yashika
> > > >
> > > > ________________________________
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > NOTE: This message may contain information that is confidential,
> > > > proprietary, privileged or otherwise protected by law. The message is
> > > > intended solely for the named addressee. If received in error, please
> > > > destroy and notify the sender. Any use of this email is prohibited
> when
> > > > received in error. Impetus does not represent, warrant and/or
> > guarantee,
> > > > that the integrity of this communication has been maintained nor that
> > the
> > > > communication is free of errors, virus, interception or interference.
> > > >
> > >
> > > ________________________________
> > >
> > >
> > >
> > >
> > >
> > >
> > > NOTE: This message may contain information that is confidential,
> > > proprietary, privileged or otherwise protected by law. The message is
> > > intended solely for the named addressee. If received in error, please
> > > destroy and notify the sender. Any use of this email is prohibited when
> > > received in error. Impetus does not represent, warrant and/or
> guarantee,
> > > that the integrity of this communication has been maintained nor that
> the
> > > communication is free of errors, virus, interception or interference.
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>
> ________________________________
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>
Re: Kafka Performance Tuning
Posted by Yashika Gupta <ya...@impetus.co.in>.
I am running a single broker and the leader column has 0 as the value.
pushkar priyadarshi <pr...@gmail.com> wrote:
you can use the kafka-list-topic.sh to find out if leader for particual
topic is available.-1 in leader column might indicate trouble.
On Fri, Apr 25, 2014 at 6:34 AM, Guozhang Wang <wa...@gmail.com> wrote:
> Could you double check if the topic LOGFILE04 is already created on the
> servers?
>
> Guozhang
>
>
> On Thu, Apr 24, 2014 at 10:46 AM, Yashika Gupta <
> yashika.gupta@impetus.co.in
> > wrote:
>
> > Jun,
> >
> > The detailed logs are as follows:
> >
> > 24.04.2014 13:37:31812 INFO main kafka.producer.SyncProducer -
> > Disconnecting from localhost:9092
> > 24.04.2014 13:37:38612 WARN main kafka.producer.BrokerPartitionInfo -
> > Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
> > No partition metadata for topic LOGFILE04 due to
> > kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class
> > kafka.common.LeaderNotAvailableException
> > 24.04.2014 13:37:40712 INFO main kafka.client.ClientUtils$ - Fetching
> > metadata from broker id:0,host:localhost,port:9092 with correlation id 1
> > for 1 topic(s) Set(LOGFILE04)
> > 24.04.2014 13:37:41212 INFO main kafka.producer.SyncProducer - Connected
> > to localhost:9092 for producing
> > 24.04.2014 13:37:48812 INFO main kafka.producer.SyncProducer -
> > Disconnecting from localhost:9092
> > 24.04.2014 13:37:48912 WARN main kafka.producer.BrokerPartitionInfo -
> > Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
> > No partition metadata for topic LOGFILE04 due to
> > kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class
> > kafka.common.LeaderNotAvailableException
> > 24.04.2014 13:37:49012 ERROR main
> kafka.producer.async.DefaultEventHandler
> > - Failed to collate messages by topic, partition due to: Failed to fetch
> > topic metadata for topic: LOGFILE04
> >
> >
> > 24.04.2014 13:39:96513 WARN
> >
> ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0
> > kafka.consumer.ConsumerFetcherThread -
> >
> [ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0],
> > Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 4;
> ClientId:
> >
> produceLogLine2-ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0;
> > ReplicaId: -1; MaxWait: 60000 ms; MinBytes: 1 bytes; RequestInfo:
> > [LOGFILE04,0] -> PartitionFetchInfo(2,1048576)
> > java.net.SocketTimeoutException
> > at
> > sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:229)
> > at
> sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
> > at
> >
> java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385)
> > at kafka.utils.Utils$.read(Unknown Source)
> > at kafka.network.BoundedByteBufferReceive.readFrom(Unknown
> Source)
> > at kafka.network.Receive$class.readCompletely(Unknown Source)
> > at kafka.network.BoundedByteBufferReceive.readCompletely(Unknown
> > Source)
> > at kafka.network.BlockingChannel.receive(Unknown Source)
> > at kafka.consumer.SimpleConsumer.liftedTree1$1(Unknown Source)
> > at
> >
> kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(Unknown
> > Source)
> > at
> >
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(Unknown
> > Source)
> > at
> >
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
> > Source)
> > at
> >
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
> > Source)
> > at kafka.metrics.KafkaTimer.time(Unknown Source)
> > at
> > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(Unknown
> Source)
> > at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
> > Source)
> > at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
> > Source)
> > at kafka.metrics.KafkaTimer.time(Unknown Source)
> > at kafka.consumer.SimpleConsumer.fetch(Unknown Source)
> > at kafka.server.AbstractFetcherThread.processFetchRequest(Unknown
> > Source)
> > at kafka.server.AbstractFetcherThread.doWork(Unknown Source)
> > at kafka.utils.ShutdownableThread.run(Unknown Source)
> >
> >
> > Regards,
> > Yashika
> > ________________________________________
> > From: Jun Rao <ju...@gmail.com>
> > Sent: Thursday, April 24, 2014 10:49 PM
> > To: users@kafka.apache.org
> > Subject: Re: Kafka Performance Tuning
> >
> > Before that error messge, the log should tell you the cause of the error.
> > Could you dig that out?
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Thu, Apr 24, 2014 at 10:12 AM, Yashika Gupta <
> > yashika.gupta@impetus.co.in
> > > wrote:
> >
> > > Hi,
> > >
> > > I am working on a POC where I have 1 Zookeeper and 2 Kafka Brokers on
> my
> > > local machine. I am running 8 sets of Kafka consumers and producers
> > running
> > > in parallel.
> > >
> > > Below are my configurations:
> > > Consumer Configs:
> > > zookeeper.session.timeout.ms=120000
> > > zookeeper.sync.time.ms=2000
> > > zookeeper.connection.timeout.ms=120000
> > > auto.commit.interval.ms=60000
> > > rebalance.backoff.ms=2000
> > > fetch.wait.max.ms=60000
> > > auto.offset.reset=smallest
> > >
> > > Producer configs:
> > > key.serializer.class=kafka.serializer.StringEncoder
> > > request.required.acks=-1
> > > message.send.max.retries=3
> > > request.timeout.ms=60000
> > >
> > > I have tried various more configuration changes but I am running into
> the
> > > same exception again and again.
> > >
> > > 17.04.2014 06:31:83216 ERROR pool-5-thread-1
> > > kafka.producer.async.DefaultEventHandler - Failed to collate messages
> by
> > > topic, partition due to: Failed to fetch topic metadata for topic:
> > TOPIC99
> > > 17.04.2014 06:31:85616 ERROR pool-4-thread-1
> > > kafka.producer.async.DefaultEventHandler - Failed to collate messages
> by
> > > topic, partition due to: Failed to fetch topic metadata for topic:
> > TOPIC99
> > >
> > > I am not able to get to the root cause of the issue.
> > > Appreciate your help.
> > >
> > > Regards,
> > > Yashika
> > >
> > > ________________________________
> > >
> > >
> > >
> > >
> > >
> > >
> > > NOTE: This message may contain information that is confidential,
> > > proprietary, privileged or otherwise protected by law. The message is
> > > intended solely for the named addressee. If received in error, please
> > > destroy and notify the sender. Any use of this email is prohibited when
> > > received in error. Impetus does not represent, warrant and/or
> guarantee,
> > > that the integrity of this communication has been maintained nor that
> the
> > > communication is free of errors, virus, interception or interference.
> > >
> >
> > ________________________________
> >
> >
> >
> >
> >
> >
> > NOTE: This message may contain information that is confidential,
> > proprietary, privileged or otherwise protected by law. The message is
> > intended solely for the named addressee. If received in error, please
> > destroy and notify the sender. Any use of this email is prohibited when
> > received in error. Impetus does not represent, warrant and/or guarantee,
> > that the integrity of this communication has been maintained nor that the
> > communication is free of errors, virus, interception or interference.
> >
>
>
>
> --
> -- Guozhang
>
________________________________
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
Re: Kafka Performance Tuning
Posted by pushkar priyadarshi <pr...@gmail.com>.
you can use the kafka-list-topic.sh to find out if leader for particual
topic is available.-1 in leader column might indicate trouble.
On Fri, Apr 25, 2014 at 6:34 AM, Guozhang Wang <wa...@gmail.com> wrote:
> Could you double check if the topic LOGFILE04 is already created on the
> servers?
>
> Guozhang
>
>
> On Thu, Apr 24, 2014 at 10:46 AM, Yashika Gupta <
> yashika.gupta@impetus.co.in
> > wrote:
>
> > Jun,
> >
> > The detailed logs are as follows:
> >
> > 24.04.2014 13:37:31812 INFO main kafka.producer.SyncProducer -
> > Disconnecting from localhost:9092
> > 24.04.2014 13:37:38612 WARN main kafka.producer.BrokerPartitionInfo -
> > Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
> > No partition metadata for topic LOGFILE04 due to
> > kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class
> > kafka.common.LeaderNotAvailableException
> > 24.04.2014 13:37:40712 INFO main kafka.client.ClientUtils$ - Fetching
> > metadata from broker id:0,host:localhost,port:9092 with correlation id 1
> > for 1 topic(s) Set(LOGFILE04)
> > 24.04.2014 13:37:41212 INFO main kafka.producer.SyncProducer - Connected
> > to localhost:9092 for producing
> > 24.04.2014 13:37:48812 INFO main kafka.producer.SyncProducer -
> > Disconnecting from localhost:9092
> > 24.04.2014 13:37:48912 WARN main kafka.producer.BrokerPartitionInfo -
> > Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
> > No partition metadata for topic LOGFILE04 due to
> > kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class
> > kafka.common.LeaderNotAvailableException
> > 24.04.2014 13:37:49012 ERROR main
> kafka.producer.async.DefaultEventHandler
> > - Failed to collate messages by topic, partition due to: Failed to fetch
> > topic metadata for topic: LOGFILE04
> >
> >
> > 24.04.2014 13:39:96513 WARN
> >
> ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0
> > kafka.consumer.ConsumerFetcherThread -
> >
> [ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0],
> > Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 4;
> ClientId:
> >
> produceLogLine2-ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0;
> > ReplicaId: -1; MaxWait: 60000 ms; MinBytes: 1 bytes; RequestInfo:
> > [LOGFILE04,0] -> PartitionFetchInfo(2,1048576)
> > java.net.SocketTimeoutException
> > at
> > sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:229)
> > at
> sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
> > at
> >
> java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385)
> > at kafka.utils.Utils$.read(Unknown Source)
> > at kafka.network.BoundedByteBufferReceive.readFrom(Unknown
> Source)
> > at kafka.network.Receive$class.readCompletely(Unknown Source)
> > at kafka.network.BoundedByteBufferReceive.readCompletely(Unknown
> > Source)
> > at kafka.network.BlockingChannel.receive(Unknown Source)
> > at kafka.consumer.SimpleConsumer.liftedTree1$1(Unknown Source)
> > at
> >
> kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(Unknown
> > Source)
> > at
> >
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(Unknown
> > Source)
> > at
> >
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
> > Source)
> > at
> >
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
> > Source)
> > at kafka.metrics.KafkaTimer.time(Unknown Source)
> > at
> > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(Unknown
> Source)
> > at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
> > Source)
> > at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
> > Source)
> > at kafka.metrics.KafkaTimer.time(Unknown Source)
> > at kafka.consumer.SimpleConsumer.fetch(Unknown Source)
> > at kafka.server.AbstractFetcherThread.processFetchRequest(Unknown
> > Source)
> > at kafka.server.AbstractFetcherThread.doWork(Unknown Source)
> > at kafka.utils.ShutdownableThread.run(Unknown Source)
> >
> >
> > Regards,
> > Yashika
> > ________________________________________
> > From: Jun Rao <ju...@gmail.com>
> > Sent: Thursday, April 24, 2014 10:49 PM
> > To: users@kafka.apache.org
> > Subject: Re: Kafka Performance Tuning
> >
> > Before that error messge, the log should tell you the cause of the error.
> > Could you dig that out?
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Thu, Apr 24, 2014 at 10:12 AM, Yashika Gupta <
> > yashika.gupta@impetus.co.in
> > > wrote:
> >
> > > Hi,
> > >
> > > I am working on a POC where I have 1 Zookeeper and 2 Kafka Brokers on
> my
> > > local machine. I am running 8 sets of Kafka consumers and producers
> > running
> > > in parallel.
> > >
> > > Below are my configurations:
> > > Consumer Configs:
> > > zookeeper.session.timeout.ms=120000
> > > zookeeper.sync.time.ms=2000
> > > zookeeper.connection.timeout.ms=120000
> > > auto.commit.interval.ms=60000
> > > rebalance.backoff.ms=2000
> > > fetch.wait.max.ms=60000
> > > auto.offset.reset=smallest
> > >
> > > Producer configs:
> > > key.serializer.class=kafka.serializer.StringEncoder
> > > request.required.acks=-1
> > > message.send.max.retries=3
> > > request.timeout.ms=60000
> > >
> > > I have tried various more configuration changes but I am running into
> the
> > > same exception again and again.
> > >
> > > 17.04.2014 06:31:83216 ERROR pool-5-thread-1
> > > kafka.producer.async.DefaultEventHandler - Failed to collate messages
> by
> > > topic, partition due to: Failed to fetch topic metadata for topic:
> > TOPIC99
> > > 17.04.2014 06:31:85616 ERROR pool-4-thread-1
> > > kafka.producer.async.DefaultEventHandler - Failed to collate messages
> by
> > > topic, partition due to: Failed to fetch topic metadata for topic:
> > TOPIC99
> > >
> > > I am not able to get to the root cause of the issue.
> > > Appreciate your help.
> > >
> > > Regards,
> > > Yashika
> > >
> > > ________________________________
> > >
> > >
> > >
> > >
> > >
> > >
> > > NOTE: This message may contain information that is confidential,
> > > proprietary, privileged or otherwise protected by law. The message is
> > > intended solely for the named addressee. If received in error, please
> > > destroy and notify the sender. Any use of this email is prohibited when
> > > received in error. Impetus does not represent, warrant and/or
> guarantee,
> > > that the integrity of this communication has been maintained nor that
> the
> > > communication is free of errors, virus, interception or interference.
> > >
> >
> > ________________________________
> >
> >
> >
> >
> >
> >
> > NOTE: This message may contain information that is confidential,
> > proprietary, privileged or otherwise protected by law. The message is
> > intended solely for the named addressee. If received in error, please
> > destroy and notify the sender. Any use of this email is prohibited when
> > received in error. Impetus does not represent, warrant and/or guarantee,
> > that the integrity of this communication has been maintained nor that the
> > communication is free of errors, virus, interception or interference.
> >
>
>
>
> --
> -- Guozhang
>
Re: Kafka Performance Tuning
Posted by Yashika Gupta <ya...@impetus.co.in>.
I had cleaned up the topics using the following commands:
Rm -rf /tmp/kafka-logs/*
And verified using the topics list command before executing the script.
Am I missing anything else.
Regards,
Yashika
Guozhang Wang <wa...@gmail.com> wrote:
Could you double check if the topic LOGFILE04 is already created on the
servers?
Guozhang
On Thu, Apr 24, 2014 at 10:46 AM, Yashika Gupta <yashika.gupta@impetus.co.in
> wrote:
> Jun,
>
> The detailed logs are as follows:
>
> 24.04.2014 13:37:31812 INFO main kafka.producer.SyncProducer -
> Disconnecting from localhost:9092
> 24.04.2014 13:37:38612 WARN main kafka.producer.BrokerPartitionInfo -
> Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
> No partition metadata for topic LOGFILE04 due to
> kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class
> kafka.common.LeaderNotAvailableException
> 24.04.2014 13:37:40712 INFO main kafka.client.ClientUtils$ - Fetching
> metadata from broker id:0,host:localhost,port:9092 with correlation id 1
> for 1 topic(s) Set(LOGFILE04)
> 24.04.2014 13:37:41212 INFO main kafka.producer.SyncProducer - Connected
> to localhost:9092 for producing
> 24.04.2014 13:37:48812 INFO main kafka.producer.SyncProducer -
> Disconnecting from localhost:9092
> 24.04.2014 13:37:48912 WARN main kafka.producer.BrokerPartitionInfo -
> Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
> No partition metadata for topic LOGFILE04 due to
> kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class
> kafka.common.LeaderNotAvailableException
> 24.04.2014 13:37:49012 ERROR main kafka.producer.async.DefaultEventHandler
> - Failed to collate messages by topic, partition due to: Failed to fetch
> topic metadata for topic: LOGFILE04
>
>
> 24.04.2014 13:39:96513 WARN
> ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0
> kafka.consumer.ConsumerFetcherThread -
> [ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0],
> Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 4; ClientId:
> produceLogLine2-ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0;
> ReplicaId: -1; MaxWait: 60000 ms; MinBytes: 1 bytes; RequestInfo:
> [LOGFILE04,0] -> PartitionFetchInfo(2,1048576)
> java.net.SocketTimeoutException
> at
> sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:229)
> at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
> at
> java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385)
> at kafka.utils.Utils$.read(Unknown Source)
> at kafka.network.BoundedByteBufferReceive.readFrom(Unknown Source)
> at kafka.network.Receive$class.readCompletely(Unknown Source)
> at kafka.network.BoundedByteBufferReceive.readCompletely(Unknown
> Source)
> at kafka.network.BlockingChannel.receive(Unknown Source)
> at kafka.consumer.SimpleConsumer.liftedTree1$1(Unknown Source)
> at
> kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(Unknown
> Source)
> at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(Unknown
> Source)
> at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
> Source)
> at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
> Source)
> at kafka.metrics.KafkaTimer.time(Unknown Source)
> at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(Unknown Source)
> at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
> Source)
> at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
> Source)
> at kafka.metrics.KafkaTimer.time(Unknown Source)
> at kafka.consumer.SimpleConsumer.fetch(Unknown Source)
> at kafka.server.AbstractFetcherThread.processFetchRequest(Unknown
> Source)
> at kafka.server.AbstractFetcherThread.doWork(Unknown Source)
> at kafka.utils.ShutdownableThread.run(Unknown Source)
>
>
> Regards,
> Yashika
> ________________________________________
> From: Jun Rao <ju...@gmail.com>
> Sent: Thursday, April 24, 2014 10:49 PM
> To: users@kafka.apache.org
> Subject: Re: Kafka Performance Tuning
>
> Before that error messge, the log should tell you the cause of the error.
> Could you dig that out?
>
> Thanks,
>
> Jun
>
>
> On Thu, Apr 24, 2014 at 10:12 AM, Yashika Gupta <
> yashika.gupta@impetus.co.in
> > wrote:
>
> > Hi,
> >
> > I am working on a POC where I have 1 Zookeeper and 2 Kafka Brokers on my
> > local machine. I am running 8 sets of Kafka consumers and producers
> running
> > in parallel.
> >
> > Below are my configurations:
> > Consumer Configs:
> > zookeeper.session.timeout.ms=120000
> > zookeeper.sync.time.ms=2000
> > zookeeper.connection.timeout.ms=120000
> > auto.commit.interval.ms=60000
> > rebalance.backoff.ms=2000
> > fetch.wait.max.ms=60000
> > auto.offset.reset=smallest
> >
> > Producer configs:
> > key.serializer.class=kafka.serializer.StringEncoder
> > request.required.acks=-1
> > message.send.max.retries=3
> > request.timeout.ms=60000
> >
> > I have tried various more configuration changes but I am running into the
> > same exception again and again.
> >
> > 17.04.2014 06:31:83216 ERROR pool-5-thread-1
> > kafka.producer.async.DefaultEventHandler - Failed to collate messages by
> > topic, partition due to: Failed to fetch topic metadata for topic:
> TOPIC99
> > 17.04.2014 06:31:85616 ERROR pool-4-thread-1
> > kafka.producer.async.DefaultEventHandler - Failed to collate messages by
> > topic, partition due to: Failed to fetch topic metadata for topic:
> TOPIC99
> >
> > I am not able to get to the root cause of the issue.
> > Appreciate your help.
> >
> > Regards,
> > Yashika
> >
> > ________________________________
> >
> >
> >
> >
> >
> >
> > NOTE: This message may contain information that is confidential,
> > proprietary, privileged or otherwise protected by law. The message is
> > intended solely for the named addressee. If received in error, please
> > destroy and notify the sender. Any use of this email is prohibited when
> > received in error. Impetus does not represent, warrant and/or guarantee,
> > that the integrity of this communication has been maintained nor that the
> > communication is free of errors, virus, interception or interference.
> >
>
> ________________________________
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>
--
-- Guozhang
________________________________
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
Re: Kafka Performance Tuning
Posted by Guozhang Wang <wa...@gmail.com>.
Could you double check if the topic LOGFILE04 is already created on the
servers?
Guozhang
On Thu, Apr 24, 2014 at 10:46 AM, Yashika Gupta <yashika.gupta@impetus.co.in
> wrote:
> Jun,
>
> The detailed logs are as follows:
>
> 24.04.2014 13:37:31812 INFO main kafka.producer.SyncProducer -
> Disconnecting from localhost:9092
> 24.04.2014 13:37:38612 WARN main kafka.producer.BrokerPartitionInfo -
> Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
> No partition metadata for topic LOGFILE04 due to
> kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class
> kafka.common.LeaderNotAvailableException
> 24.04.2014 13:37:40712 INFO main kafka.client.ClientUtils$ - Fetching
> metadata from broker id:0,host:localhost,port:9092 with correlation id 1
> for 1 topic(s) Set(LOGFILE04)
> 24.04.2014 13:37:41212 INFO main kafka.producer.SyncProducer - Connected
> to localhost:9092 for producing
> 24.04.2014 13:37:48812 INFO main kafka.producer.SyncProducer -
> Disconnecting from localhost:9092
> 24.04.2014 13:37:48912 WARN main kafka.producer.BrokerPartitionInfo -
> Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
> No partition metadata for topic LOGFILE04 due to
> kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class
> kafka.common.LeaderNotAvailableException
> 24.04.2014 13:37:49012 ERROR main kafka.producer.async.DefaultEventHandler
> - Failed to collate messages by topic, partition due to: Failed to fetch
> topic metadata for topic: LOGFILE04
>
>
> 24.04.2014 13:39:96513 WARN
> ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0
> kafka.consumer.ConsumerFetcherThread -
> [ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0],
> Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 4; ClientId:
> produceLogLine2-ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0;
> ReplicaId: -1; MaxWait: 60000 ms; MinBytes: 1 bytes; RequestInfo:
> [LOGFILE04,0] -> PartitionFetchInfo(2,1048576)
> java.net.SocketTimeoutException
> at
> sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:229)
> at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
> at
> java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385)
> at kafka.utils.Utils$.read(Unknown Source)
> at kafka.network.BoundedByteBufferReceive.readFrom(Unknown Source)
> at kafka.network.Receive$class.readCompletely(Unknown Source)
> at kafka.network.BoundedByteBufferReceive.readCompletely(Unknown
> Source)
> at kafka.network.BlockingChannel.receive(Unknown Source)
> at kafka.consumer.SimpleConsumer.liftedTree1$1(Unknown Source)
> at
> kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(Unknown
> Source)
> at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(Unknown
> Source)
> at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
> Source)
> at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
> Source)
> at kafka.metrics.KafkaTimer.time(Unknown Source)
> at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(Unknown Source)
> at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
> Source)
> at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
> Source)
> at kafka.metrics.KafkaTimer.time(Unknown Source)
> at kafka.consumer.SimpleConsumer.fetch(Unknown Source)
> at kafka.server.AbstractFetcherThread.processFetchRequest(Unknown
> Source)
> at kafka.server.AbstractFetcherThread.doWork(Unknown Source)
> at kafka.utils.ShutdownableThread.run(Unknown Source)
>
>
> Regards,
> Yashika
> ________________________________________
> From: Jun Rao <ju...@gmail.com>
> Sent: Thursday, April 24, 2014 10:49 PM
> To: users@kafka.apache.org
> Subject: Re: Kafka Performance Tuning
>
> Before that error messge, the log should tell you the cause of the error.
> Could you dig that out?
>
> Thanks,
>
> Jun
>
>
> On Thu, Apr 24, 2014 at 10:12 AM, Yashika Gupta <
> yashika.gupta@impetus.co.in
> > wrote:
>
> > Hi,
> >
> > I am working on a POC where I have 1 Zookeeper and 2 Kafka Brokers on my
> > local machine. I am running 8 sets of Kafka consumers and producers
> running
> > in parallel.
> >
> > Below are my configurations:
> > Consumer Configs:
> > zookeeper.session.timeout.ms=120000
> > zookeeper.sync.time.ms=2000
> > zookeeper.connection.timeout.ms=120000
> > auto.commit.interval.ms=60000
> > rebalance.backoff.ms=2000
> > fetch.wait.max.ms=60000
> > auto.offset.reset=smallest
> >
> > Producer configs:
> > key.serializer.class=kafka.serializer.StringEncoder
> > request.required.acks=-1
> > message.send.max.retries=3
> > request.timeout.ms=60000
> >
> > I have tried various more configuration changes but I am running into the
> > same exception again and again.
> >
> > 17.04.2014 06:31:83216 ERROR pool-5-thread-1
> > kafka.producer.async.DefaultEventHandler - Failed to collate messages by
> > topic, partition due to: Failed to fetch topic metadata for topic:
> TOPIC99
> > 17.04.2014 06:31:85616 ERROR pool-4-thread-1
> > kafka.producer.async.DefaultEventHandler - Failed to collate messages by
> > topic, partition due to: Failed to fetch topic metadata for topic:
> TOPIC99
> >
> > I am not able to get to the root cause of the issue.
> > Appreciate your help.
> >
> > Regards,
> > Yashika
> >
> > ________________________________
> >
> >
> >
> >
> >
> >
> > NOTE: This message may contain information that is confidential,
> > proprietary, privileged or otherwise protected by law. The message is
> > intended solely for the named addressee. If received in error, please
> > destroy and notify the sender. Any use of this email is prohibited when
> > received in error. Impetus does not represent, warrant and/or guarantee,
> > that the integrity of this communication has been maintained nor that the
> > communication is free of errors, virus, interception or interference.
> >
>
> ________________________________
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>
--
-- Guozhang
Re: Kafka Performance Tuning
Posted by Bert Corderman <be...@gmail.com>.
I had this error before and corrected by increasing nofile limit
add to file an entry for the user running the broker.
/etc/security/limits.conf
kafka - nofile 98304
On Thu, Apr 24, 2014 at 1:46 PM, Yashika Gupta
<ya...@impetus.co.in>wrote:
> Jun,
>
> The detailed logs are as follows:
>
> 24.04.2014 13:37:31812 INFO main kafka.producer.SyncProducer -
> Disconnecting from localhost:9092
> 24.04.2014 13:37:38612 WARN main kafka.producer.BrokerPartitionInfo -
> Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
> No partition metadata for topic LOGFILE04 due to
> kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class
> kafka.common.LeaderNotAvailableException
> 24.04.2014 13:37:40712 INFO main kafka.client.ClientUtils$ - Fetching
> metadata from broker id:0,host:localhost,port:9092 with correlation id 1
> for 1 topic(s) Set(LOGFILE04)
> 24.04.2014 13:37:41212 INFO main kafka.producer.SyncProducer - Connected
> to localhost:9092 for producing
> 24.04.2014 13:37:48812 INFO main kafka.producer.SyncProducer -
> Disconnecting from localhost:9092
> 24.04.2014 13:37:48912 WARN main kafka.producer.BrokerPartitionInfo -
> Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
> No partition metadata for topic LOGFILE04 due to
> kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class
> kafka.common.LeaderNotAvailableException
> 24.04.2014 13:37:49012 ERROR main kafka.producer.async.DefaultEventHandler
> - Failed to collate messages by topic, partition due to: Failed to fetch
> topic metadata for topic: LOGFILE04
>
>
> 24.04.2014 13:39:96513 WARN
> ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0
> kafka.consumer.ConsumerFetcherThread -
> [ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0],
> Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 4; ClientId:
> produceLogLine2-ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0;
> ReplicaId: -1; MaxWait: 60000 ms; MinBytes: 1 bytes; RequestInfo:
> [LOGFILE04,0] -> PartitionFetchInfo(2,1048576)
> java.net.SocketTimeoutException
> at
> sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:229)
> at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
> at
> java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385)
> at kafka.utils.Utils$.read(Unknown Source)
> at kafka.network.BoundedByteBufferReceive.readFrom(Unknown Source)
> at kafka.network.Receive$class.readCompletely(Unknown Source)
> at kafka.network.BoundedByteBufferReceive.readCompletely(Unknown
> Source)
> at kafka.network.BlockingChannel.receive(Unknown Source)
> at kafka.consumer.SimpleConsumer.liftedTree1$1(Unknown Source)
> at
> kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(Unknown
> Source)
> at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(Unknown
> Source)
> at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
> Source)
> at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
> Source)
> at kafka.metrics.KafkaTimer.time(Unknown Source)
> at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(Unknown Source)
> at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
> Source)
> at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
> Source)
> at kafka.metrics.KafkaTimer.time(Unknown Source)
> at kafka.consumer.SimpleConsumer.fetch(Unknown Source)
> at kafka.server.AbstractFetcherThread.processFetchRequest(Unknown
> Source)
> at kafka.server.AbstractFetcherThread.doWork(Unknown Source)
> at kafka.utils.ShutdownableThread.run(Unknown Source)
>
>
> Regards,
> Yashika
> ________________________________________
> From: Jun Rao <ju...@gmail.com>
> Sent: Thursday, April 24, 2014 10:49 PM
> To: users@kafka.apache.org
> Subject: Re: Kafka Performance Tuning
>
> Before that error messge, the log should tell you the cause of the error.
> Could you dig that out?
>
> Thanks,
>
> Jun
>
>
> On Thu, Apr 24, 2014 at 10:12 AM, Yashika Gupta <
> yashika.gupta@impetus.co.in
> > wrote:
>
> > Hi,
> >
> > I am working on a POC where I have 1 Zookeeper and 2 Kafka Brokers on my
> > local machine. I am running 8 sets of Kafka consumers and producers
> running
> > in parallel.
> >
> > Below are my configurations:
> > Consumer Configs:
> > zookeeper.session.timeout.ms=120000
> > zookeeper.sync.time.ms=2000
> > zookeeper.connection.timeout.ms=120000
> > auto.commit.interval.ms=60000
> > rebalance.backoff.ms=2000
> > fetch.wait.max.ms=60000
> > auto.offset.reset=smallest
> >
> > Producer configs:
> > key.serializer.class=kafka.serializer.StringEncoder
> > request.required.acks=-1
> > message.send.max.retries=3
> > request.timeout.ms=60000
> >
> > I have tried various more configuration changes but I am running into the
> > same exception again and again.
> >
> > 17.04.2014 06:31:83216 ERROR pool-5-thread-1
> > kafka.producer.async.DefaultEventHandler - Failed to collate messages by
> > topic, partition due to: Failed to fetch topic metadata for topic:
> TOPIC99
> > 17.04.2014 06:31:85616 ERROR pool-4-thread-1
> > kafka.producer.async.DefaultEventHandler - Failed to collate messages by
> > topic, partition due to: Failed to fetch topic metadata for topic:
> TOPIC99
> >
> > I am not able to get to the root cause of the issue.
> > Appreciate your help.
> >
> > Regards,
> > Yashika
> >
> > ________________________________
> >
> >
> >
> >
> >
> >
> > NOTE: This message may contain information that is confidential,
> > proprietary, privileged or otherwise protected by law. The message is
> > intended solely for the named addressee. If received in error, please
> > destroy and notify the sender. Any use of this email is prohibited when
> > received in error. Impetus does not represent, warrant and/or guarantee,
> > that the integrity of this communication has been maintained nor that the
> > communication is free of errors, virus, interception or interference.
> >
>
> ________________________________
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>
RE: Kafka Performance Tuning
Posted by Yashika Gupta <ya...@impetus.co.in>.
Jun,
The detailed logs are as follows:
24.04.2014 13:37:31812 INFO main kafka.producer.SyncProducer - Disconnecting from localhost:9092
24.04.2014 13:37:38612 WARN main kafka.producer.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
No partition metadata for topic LOGFILE04 due to kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class kafka.common.LeaderNotAvailableException
24.04.2014 13:37:40712 INFO main kafka.client.ClientUtils$ - Fetching metadata from broker id:0,host:localhost,port:9092 with correlation id 1 for 1 topic(s) Set(LOGFILE04)
24.04.2014 13:37:41212 INFO main kafka.producer.SyncProducer - Connected to localhost:9092 for producing
24.04.2014 13:37:48812 INFO main kafka.producer.SyncProducer - Disconnecting from localhost:9092
24.04.2014 13:37:48912 WARN main kafka.producer.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
No partition metadata for topic LOGFILE04 due to kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class kafka.common.LeaderNotAvailableException
24.04.2014 13:37:49012 ERROR main kafka.producer.async.DefaultEventHandler - Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: LOGFILE04
24.04.2014 13:39:96513 WARN ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0 kafka.consumer.ConsumerFetcherThread - [ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0], Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 4; ClientId: produceLogLine2-ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0; ReplicaId: -1; MaxWait: 60000 ms; MinBytes: 1 bytes; RequestInfo: [LOGFILE04,0] -> PartitionFetchInfo(2,1048576)
java.net.SocketTimeoutException
at sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:229)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
at java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385)
at kafka.utils.Utils$.read(Unknown Source)
at kafka.network.BoundedByteBufferReceive.readFrom(Unknown Source)
at kafka.network.Receive$class.readCompletely(Unknown Source)
at kafka.network.BoundedByteBufferReceive.readCompletely(Unknown Source)
at kafka.network.BlockingChannel.receive(Unknown Source)
at kafka.consumer.SimpleConsumer.liftedTree1$1(Unknown Source)
at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(Unknown Source)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(Unknown Source)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown Source)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown Source)
at kafka.metrics.KafkaTimer.time(Unknown Source)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(Unknown Source)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown Source)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown Source)
at kafka.metrics.KafkaTimer.time(Unknown Source)
at kafka.consumer.SimpleConsumer.fetch(Unknown Source)
at kafka.server.AbstractFetcherThread.processFetchRequest(Unknown Source)
at kafka.server.AbstractFetcherThread.doWork(Unknown Source)
at kafka.utils.ShutdownableThread.run(Unknown Source)
Regards,
Yashika
________________________________________
From: Jun Rao <ju...@gmail.com>
Sent: Thursday, April 24, 2014 10:49 PM
To: users@kafka.apache.org
Subject: Re: Kafka Performance Tuning
Before that error messge, the log should tell you the cause of the error.
Could you dig that out?
Thanks,
Jun
On Thu, Apr 24, 2014 at 10:12 AM, Yashika Gupta <yashika.gupta@impetus.co.in
> wrote:
> Hi,
>
> I am working on a POC where I have 1 Zookeeper and 2 Kafka Brokers on my
> local machine. I am running 8 sets of Kafka consumers and producers running
> in parallel.
>
> Below are my configurations:
> Consumer Configs:
> zookeeper.session.timeout.ms=120000
> zookeeper.sync.time.ms=2000
> zookeeper.connection.timeout.ms=120000
> auto.commit.interval.ms=60000
> rebalance.backoff.ms=2000
> fetch.wait.max.ms=60000
> auto.offset.reset=smallest
>
> Producer configs:
> key.serializer.class=kafka.serializer.StringEncoder
> request.required.acks=-1
> message.send.max.retries=3
> request.timeout.ms=60000
>
> I have tried various more configuration changes but I am running into the
> same exception again and again.
>
> 17.04.2014 06:31:83216 ERROR pool-5-thread-1
> kafka.producer.async.DefaultEventHandler - Failed to collate messages by
> topic, partition due to: Failed to fetch topic metadata for topic: TOPIC99
> 17.04.2014 06:31:85616 ERROR pool-4-thread-1
> kafka.producer.async.DefaultEventHandler - Failed to collate messages by
> topic, partition due to: Failed to fetch topic metadata for topic: TOPIC99
>
> I am not able to get to the root cause of the issue.
> Appreciate your help.
>
> Regards,
> Yashika
>
> ________________________________
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>
________________________________
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
Re: Kafka Performance Tuning
Posted by Jun Rao <ju...@gmail.com>.
Before that error messge, the log should tell you the cause of the error.
Could you dig that out?
Thanks,
Jun
On Thu, Apr 24, 2014 at 10:12 AM, Yashika Gupta <yashika.gupta@impetus.co.in
> wrote:
> Hi,
>
> I am working on a POC where I have 1 Zookeeper and 2 Kafka Brokers on my
> local machine. I am running 8 sets of Kafka consumers and producers running
> in parallel.
>
> Below are my configurations:
> Consumer Configs:
> zookeeper.session.timeout.ms=120000
> zookeeper.sync.time.ms=2000
> zookeeper.connection.timeout.ms=120000
> auto.commit.interval.ms=60000
> rebalance.backoff.ms=2000
> fetch.wait.max.ms=60000
> auto.offset.reset=smallest
>
> Producer configs:
> key.serializer.class=kafka.serializer.StringEncoder
> request.required.acks=-1
> message.send.max.retries=3
> request.timeout.ms=60000
>
> I have tried various more configuration changes but I am running into the
> same exception again and again.
>
> 17.04.2014 06:31:83216 ERROR pool-5-thread-1
> kafka.producer.async.DefaultEventHandler - Failed to collate messages by
> topic, partition due to: Failed to fetch topic metadata for topic: TOPIC99
> 17.04.2014 06:31:85616 ERROR pool-4-thread-1
> kafka.producer.async.DefaultEventHandler - Failed to collate messages by
> topic, partition due to: Failed to fetch topic metadata for topic: TOPIC99
>
> I am not able to get to the root cause of the issue.
> Appreciate your help.
>
> Regards,
> Yashika
>
> ________________________________
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>