Kafka exception "Unable to find a leader for partitions"

[Shannon Carey <sc...@expedia.com>]

Does anyone have a guess what might cause this exception?


java.lang.RuntimeException: Unable to find a leader for partitions: [FetchPartition {topic=usersignals, partition=1, offset=2825838}]

at org.apache.flink.streaming.connectors.kafka.internals.LegacyFetcher.findLeaderForPartitions(LegacyFetcher.java:323)

at org.apache.flink.streaming.connectors.kafka.internals.LegacyFetcher.run(LegacyFetcher.java:162)

at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.run(FlinkKafkaConsumer08.java:316)

at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:78)

at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:56)

at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:225)

at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)

at java.lang.Thread.run(Thread.java:745)

Re: Kafka exception "Unable to find a leader for partitions"

[Robert Metzger <rm...@apache.org>]

Hi Shannon,

Some questions:

which Flink version are you using?
Can you provide me with some more logs, in particular the log entries
before this event from the Kafka connector.

Also, it is possible that the Kafka broker was in an erroneous state?

Did the error happen after weeks of data consumption, or early?
And last question: how many of the available brokers did you put into the
bootstrap.server property ?

Regards,
Robert

On Wed, Jun 8, 2016 at 7:11 PM, Shannon Carey <sc...@expedia.com> wrote:

> Does anyone have a guess what might cause this exception?
>
>
> java.lang.RuntimeException: Unable to find a leader for partitions:
> [FetchPartition {topic=usersignals, partition=1, offset=2825838}]
>
> at
> org.apache.flink.streaming.connectors.kafka.internals.LegacyFetcher.findLeaderForPartitions(LegacyFetcher.java:323)
>
> at
> org.apache.flink.streaming.connectors.kafka.internals.LegacyFetcher.run(LegacyFetcher.java:162)
>
> at
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.run(FlinkKafkaConsumer08.java:316)
>
> at
> org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:78)
>
> at
> org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:56)
>
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:225)
>
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
>
> at java.lang.Thread.run(Thread.java:745)
>