You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by dizzy0ny <di...@gmail.com> on 2018/02/04 03:39:51 UTC
Store not ready
Hi,We have a recurring problem that I wonder if there is a better way to solve. Currently when we restart our backed Kafka services and then our datastreams app, the app is unable to reach many of the Kafka topic state stores. We currently retry, but more often than not, it requires a restart of the app to clear up.
I think this is because perhaps a partition leader has not been elected when the app starts. Two questions
1. Is there a good way to know when a partition leader has been elected such that our restart script can wait2. Is it possible to have the app deterministically wait/retry until the stores are ready? As mentioned, we have retry logic for up to 200 tries with a few seconds sleep in between, but it seems in some cases we have restart the app as the retries get exceeded.
Thanks for any assistance
Re: Store not ready
Posted by "Matthias J. Sax" <ma...@confluent.io>.
You should upgrade to a never version. 0.10.2 has some issues on broker
restart that got fix in later versions.
Note: you don't need to upgrade your brokers for this. Since 0.10.1,
Kafka Streams is backward compatible to older brokers:
https://docs.confluent.io/current/streams/upgrade-guide.html#compatibility
-Matthias
On 2/16/18 12:50 AM, dizzy0ny wrote:
> We are using kafka version 0.10.2.
>
> -------- Original message --------From: "Matthias J. Sax" <ma...@confluent.io> Date: 2/14/18 4:00 PM (GMT-05:00) To: users@kafka.apache.org Subject: Re: Store not ready
> What version to you use?
>
> Kafka Streams should be able to keep running while you restart you
> brokers. If not, it seems to be a bug in Kafka Streams itself.
>
> -Matthias
>
> On 2/3/18 7:39 PM, dizzy0ny wrote:
>> Hi,We have a recurring problem that I wonder if there is a better way to solve. Currently when we restart our backed Kafka services and then our datastreams app, the app is unable to reach many of the Kafka topic state stores. We currently retry, but more often than not, it requires a restart of the app to clear up.
>> I think this is because perhaps a partition leader has not been elected when the app starts. Two questions
>> 1. Is there a good way to know when a partition leader has been elected such that our restart script can wait2. Is it possible to have the app deterministically wait/retry until the stores are ready? As mentioned, we have retry logic for up to 200 tries with a few seconds sleep in between, but it seems in some cases we have restart the app as the retries get exceeded.
>> Thanks for any assistance
>>
>
Re: Store not ready
Posted by dizzy0ny <di...@gmail.com>.
We are using kafka version 0.10.2.
-------- Original message --------From: "Matthias J. Sax" <ma...@confluent.io> Date: 2/14/18 4:00 PM (GMT-05:00) To: users@kafka.apache.org Subject: Re: Store not ready
What version to you use?
Kafka Streams should be able to keep running while you restart you
brokers. If not, it seems to be a bug in Kafka Streams itself.
-Matthias
On 2/3/18 7:39 PM, dizzy0ny wrote:
> Hi,We have a recurring problem that I wonder if there is a better way to solve. Currently when we restart our backed Kafka services and then our datastreams app, the app is unable to reach many of the Kafka topic state stores. We currently retry, but more often than not, it requires a restart of the app to clear up.
> I think this is because perhaps a partition leader has not been elected when the app starts. Two questions
> 1. Is there a good way to know when a partition leader has been elected such that our restart script can wait2. Is it possible to have the app deterministically wait/retry until the stores are ready? As mentioned, we have retry logic for up to 200 tries with a few seconds sleep in between, but it seems in some cases we have restart the app as the retries get exceeded.
> Thanks for any assistance
>
Re: Store not ready
Posted by "Matthias J. Sax" <ma...@confluent.io>.
What version to you use?
Kafka Streams should be able to keep running while you restart you
brokers. If not, it seems to be a bug in Kafka Streams itself.
-Matthias
On 2/3/18 7:39 PM, dizzy0ny wrote:
> Hi,We have a recurring problem that I wonder if there is a better way to solve. Currently when we restart our backed Kafka services and then our datastreams app, the app is unable to reach many of the Kafka topic state stores. We currently retry, but more often than not, it requires a restart of the app to clear up.
> I think this is because perhaps a partition leader has not been elected when the app starts. Two questions
> 1. Is there a good way to know when a partition leader has been elected such that our restart script can wait2. Is it possible to have the app deterministically wait/retry until the stores are ready? As mentioned, we have retry logic for up to 200 tries with a few seconds sleep in between, but it seems in some cases we have restart the app as the retries get exceeded.
> Thanks for any assistance
>