Help with SocketTimeoutException while reading from Kafka cluster

[venkatesh kavuluri <ve...@gmail.com>]

Hi All,


I am running a high level Kafka consumer and while it works fine against a
smaller staging cluster I am getting socket timeout exceptions while
running against larger LIVE cluster (against a topic with 100 partitions).
I am using "zookeeper.connect" to connect to the cluster.


*Stage cluster (Consumer gets messages)*:

1751 [ConsumerFetcherThread-local-enrich_XXXX-1439334511597-5eb31426-0-1]
INFO  kafka.consumer.ConsumerFetcherThread  -
[ConsumerFetcherThread-local-enrich_XXXX-1439334511597-5eb31426-0-1],
Starting

2344 [local-enrich_XXXX-1439334511597-5eb31426-leader-finder-thread] INFO
 kafka.consumer.ConsumerFetcherManager  -
[ConsumerFetcherManager-1439334511817] Added fetcher for partitions
ArrayBuffer([[fpti_onboarding_events,2], initOffset -1 to broker
id:1,host:XXX,port:9092] , [[fpti_onboarding_events,1], initOffset -1 to
broker id:1,host:XXXX,port:9092] , [[fpti_onboarding_events,0], initOffset
-1 to broker id:1,host:XXXX,port:9092] )

[ 2 ]************** LISTENING TO EVENTS **************

[ 0 ]************** LISTENING TO EVENTS **************

[ 1 ]************** LISTENING TO EVENTS **************


*Live cluster (Socket Timeouts )*:

3645
[ConsumerFetcherThread-c3-onboard_XXXX-2-9571-1439334326956-cfa8b46a-0-5]
INFO  kafka.consumer.ConsumerFetcherThread  -
[ConsumerFetcherThread-c3-onboard_XXXX-2-9571-1439334326956-cfa8b46a-0-5],
Starting

83799 [c3-onboard_XXXX-2-9571-1439334326956-cfa8b46a-leader-finder-thread]
INFO  kafka.consumer.SimpleConsumer  - Reconnect due to socket error:
java.net.SocketTimeoutException

163931 [c3-onboard_XXXX-2-9571-1439334326956-cfa8b46a-leader-finder-thread]
INFO  kafka.consumer.SimpleConsumer  - Reconnect due to socket error:
java.net.SocketTimeoutException

163969
[ConsumerFetcherThread-c3-onboard_XXXX-2-9571-1439334326956-cfa8b46a-0-1]
INFO  kafka.consumer.ConsumerFetcherThread  -
[ConsumerFetcherThread-c3-onboard_XXXX-2-9571-1439334326956-cfa8b46a-0-1],
Starting

Looks like leader-finder-thread is timing out in Live. Do you know why it
would happen.


I have the following timeouts defined in the configuration:

        props.put("zookeeper.session.timeout.ms", "6400");

        props.put("zookeeper.sync.time.ms", "600");

        props.put("auto.offset.reset", "smallest");

        props.put("socket.timeout.ms", "80000");

Re: Help with SocketTimeoutException while reading from Kafka cluster

[Jaikiran Pai <ja...@gmail.com>]

On Wednesday 12 August 2015 04:59 AM, venkatesh kavuluri wrote:
> 83799 [c3-onboard_XXXX-2-9571-1439334326956-cfa8b46a-leader-finder-thread]
> INFO  kafka.consumer.SimpleConsumer  - Reconnect due to socket error:
> java.net.SocketTimeoutException
>
> 163931 [c3-onboard_XXXX-2-9571-1439334326956-cfa8b46a-leader-finder-thread]
> INFO  kafka.consumer.SimpleConsumer  - Reconnect due to socket error:
> java.net.SocketTimeoutException
>
>
There's a patch in the JIRA here which logs the exact reason why the 
exception was thrown https://issues.apache.org/jira/browse/KAFKA-2221. 
It hasn't been merged since SimpleConsumer was considered deprecated. 
But you might want to apply that and see if that helps to narrow down 
the issue.

-Jaikiran