You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Brenden Cobb <br...@gmail.com> on 2016/01/05 04:18:19 UTC
Topic Deletion Issues
I might have sent this recently, but was not able to receive mail from
this list (fixed)
---
Hello-
We have a use case where we're trying to create a topic, delete, then
recreate with the same topic name.
Running into inconsistant results.
Creating the topic:
/opt/kafka/bin/kafka-topics.sh --create --partitions 3
--replication-factor 3 --topic test-01 --zookeeper zoo01:2181,
zoo02:2181, zoo03:2181
Delete:
/opt/kafka/bin/kafka-topics.sh --delete --topic test-01 --zookeeper
zoo01:2181, zoo02:2181, zoo03:2181
Repeat creation.
The results are inconsistant. Executing the above several times can be
successful, then sporadically we get caught in "topic marked for
deletion" and it does not clear.
This appears to be a Zookeeper issue of sorts as the logs will show:
[2015-12-30 22:32:32,946] WARN Conditional update of path
/brokers/topics/test-01/partitions/0/state with data
{"controller_epoch":21,"leader":2,"version":1,"leader_epoch":1,"isr":[2,0,1]}
and expected version 1 failed due to
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode
= NoNode for /brokers/topics/test-01/partitions/0/state
(kafka.utils.ZkUtils$)
In this instance no subdirectories exist beyond /brokers/topics/test-01
I'd like to know if this is a common occurrance and why the Zookeeper
node isn't "fully" created as Kafka deletion seems stuck without the
expected node path.
We are using Kafka 8.2. Purging is also an option (if achievable
programmatically), we just need to make sure there are no messages
left when resuming the producer.
Appreciate any info/guidance.
Thanks,
BC
Re: Topic Deletion Issues
Posted by Ismael Juma <is...@juma.me.uk>.
By the way, https://issues.apache.org/jira/browse/KAFKA-2937 was filed
recently and it includes a PR.
Ismael
On Tue, Jan 5, 2016 at 7:14 PM, Gwen Shapira <gw...@confluent.io> wrote:
> If you have any way of capturing logs at trace level from the point of the
> failed attempt until 10 minutes after, it will be super useful.
>
> We know there are issues in deleting topics, but it doesn't reproduce well,
> we don't know if its one issue or several and we don't have a good starting
> point on fixing them. So more info will help us move along.
>
> Gwen
>
> On Tue, Jan 5, 2016 at 11:03 AM, Brenden Cobb <br...@gmail.com>
> wrote:
>
> > Hi Gwen- No, unfortunately it doesn't appear to be consistent. One day
> > I had no problem with rapid deletion/recreation, the next I did.
> >
> > Kafka is still obsessing about the last failed deletion attempt and
> > won't move on to subsequent delete requests.
> >
> > I've planned downtime today so I can restart Kafka (clearing the topic
> > info) and test again.
> >
> > On Tue, Jan 5, 2016 at 1:30 PM, Gwen Shapira <gw...@confluent.io> wrote:
> > > Does the issue reproduce consistently?
> > >
> > > On Tue, Jan 5, 2016 at 10:28 AM, Brenden Cobb <br...@gmail.com>
> > wrote:
> > >
> > >> Thanks Alex (and sorry all for the spam)
> > >>
> > >> I'll give it another go as suggested, however in my tests I was just
> > >> using the kafka-topics command and there was definitely a few seconds
> > >> between creation/deletion.
> > >>
> > >> -BC
> > >>
> > >> On Tue, Jan 5, 2016 at 1:24 PM, Alex Loddengaard <al...@confluent.io>
> > >> wrote:
> > >> > Hi Brenden, I sent the wrong permalink. Try this:
> > >> >
> > >> >
> > >>
> >
> http://mail-archives.apache.org/mod_mbox/kafka-users/201512.mbox/%3CCAH7vnfhUQHQcCJ1R3pS_DYn0LvmA8fJtLtvCuvvDboWSqm-NBg%40mail.gmail.com%3E
> > >> >
> > >> > Alex
> > >> >
> > >> > On Tue, Jan 5, 2016 at 10:21 AM, Alex Loddengaard <
> alex@confluent.io>
> > >> wrote:
> > >> >
> > >> >> Hi Brenden, your previous email went through, and you got a
> response
> > >> (that
> > >> >> I believe answers your question). See here:
> > >> >>
> > >> >>
> > >>
> > http://mail-archives.apache.org/mod_mbox/kafka-users/201512.mbox/browser
> > >> >>
> > >> >> Alex
> > >> >>
> > >> >> On Mon, Jan 4, 2016 at 7:18 PM, Brenden Cobb <
> brendencobb@gmail.com>
> > >> >> wrote:
> > >> >>
> > >> >>> I might have sent this recently, but was not able to receive mail
> > from
> > >> >>> this list (fixed)
> > >> >>> ---
> > >> >>>
> > >> >>> Hello-
> > >> >>>
> > >> >>> We have a use case where we're trying to create a topic, delete,
> > then
> > >> >>> recreate with the same topic name.
> > >> >>>
> > >> >>> Running into inconsistant results.
> > >> >>>
> > >> >>> Creating the topic:
> > >> >>> /opt/kafka/bin/kafka-topics.sh --create --partitions 3
> > >> >>> --replication-factor 3 --topic test-01 --zookeeper zoo01:2181,
> > >> >>> zoo02:2181, zoo03:2181
> > >> >>>
> > >> >>> Delete:
> > >> >>> /opt/kafka/bin/kafka-topics.sh --delete --topic test-01
> --zookeeper
> > >> >>> zoo01:2181, zoo02:2181, zoo03:2181
> > >> >>>
> > >> >>> Repeat creation.
> > >> >>>
> > >> >>> The results are inconsistant. Executing the above several times
> can
> > be
> > >> >>> successful, then sporadically we get caught in "topic marked for
> > >> >>> deletion" and it does not clear.
> > >> >>>
> > >> >>> This appears to be a Zookeeper issue of sorts as the logs will
> show:
> > >> >>> [2015-12-30 22:32:32,946] WARN Conditional update of path
> > >> >>> /brokers/topics/test-01/partitions/0/state with data
> > >> >>>
> > >> >>>
> > >>
> >
> {"controller_epoch":21,"leader":2,"version":1,"leader_epoch":1,"isr":[2,0,1]}
> > >> >>> and expected version 1 failed due to
> > >> >>> org.apache.zookeeper.KeeperException$NoNodeException:
> > KeeperErrorCode
> > >> >>> = NoNode for /brokers/topics/test-01/partitions/0/state
> > >> >>> (kafka.utils.ZkUtils$)
> > >> >>>
> > >> >>> In this instance no subdirectories exist beyond
> > /brokers/topics/test-01
> > >> >>>
> > >> >>> I'd like to know if this is a common occurrance and why the
> > Zookeeper
> > >> >>> node isn't "fully" created as Kafka deletion seems stuck without
> the
> > >> >>> expected node path.
> > >> >>>
> > >> >>> We are using Kafka 8.2. Purging is also an option (if achievable
> > >> >>> programmatically), we just need to make sure there are no messages
> > >> >>> left when resuming the producer.
> > >> >>>
> > >> >>> Appreciate any info/guidance.
> > >> >>>
> > >> >>> Thanks,
> > >> >>> BC
> > >> >>>
> > >> >>
> > >> >>
> > >> >>
> > >> >> --
> > >> >> *Alex Loddengaard | **Solutions Architect | Confluent*
> > >> >> *Download Apache Kafka and Confluent Platform:
> > >> www.confluent.io/download
> > >> >> <http://www.confluent.io/download>*
> > >> >>
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > *Alex Loddengaard | **Solutions Architect | Confluent*
> > >> > *Download Apache Kafka and Confluent Platform:
> > www.confluent.io/download
> > >> > <http://www.confluent.io/download>*
> > >>
> >
>
Re: Topic Deletion Issues
Posted by Gwen Shapira <gw...@confluent.io>.
If you have any way of capturing logs at trace level from the point of the
failed attempt until 10 minutes after, it will be super useful.
We know there are issues in deleting topics, but it doesn't reproduce well,
we don't know if its one issue or several and we don't have a good starting
point on fixing them. So more info will help us move along.
Gwen
On Tue, Jan 5, 2016 at 11:03 AM, Brenden Cobb <br...@gmail.com> wrote:
> Hi Gwen- No, unfortunately it doesn't appear to be consistent. One day
> I had no problem with rapid deletion/recreation, the next I did.
>
> Kafka is still obsessing about the last failed deletion attempt and
> won't move on to subsequent delete requests.
>
> I've planned downtime today so I can restart Kafka (clearing the topic
> info) and test again.
>
> On Tue, Jan 5, 2016 at 1:30 PM, Gwen Shapira <gw...@confluent.io> wrote:
> > Does the issue reproduce consistently?
> >
> > On Tue, Jan 5, 2016 at 10:28 AM, Brenden Cobb <br...@gmail.com>
> wrote:
> >
> >> Thanks Alex (and sorry all for the spam)
> >>
> >> I'll give it another go as suggested, however in my tests I was just
> >> using the kafka-topics command and there was definitely a few seconds
> >> between creation/deletion.
> >>
> >> -BC
> >>
> >> On Tue, Jan 5, 2016 at 1:24 PM, Alex Loddengaard <al...@confluent.io>
> >> wrote:
> >> > Hi Brenden, I sent the wrong permalink. Try this:
> >> >
> >> >
> >>
> http://mail-archives.apache.org/mod_mbox/kafka-users/201512.mbox/%3CCAH7vnfhUQHQcCJ1R3pS_DYn0LvmA8fJtLtvCuvvDboWSqm-NBg%40mail.gmail.com%3E
> >> >
> >> > Alex
> >> >
> >> > On Tue, Jan 5, 2016 at 10:21 AM, Alex Loddengaard <al...@confluent.io>
> >> wrote:
> >> >
> >> >> Hi Brenden, your previous email went through, and you got a response
> >> (that
> >> >> I believe answers your question). See here:
> >> >>
> >> >>
> >>
> http://mail-archives.apache.org/mod_mbox/kafka-users/201512.mbox/browser
> >> >>
> >> >> Alex
> >> >>
> >> >> On Mon, Jan 4, 2016 at 7:18 PM, Brenden Cobb <br...@gmail.com>
> >> >> wrote:
> >> >>
> >> >>> I might have sent this recently, but was not able to receive mail
> from
> >> >>> this list (fixed)
> >> >>> ---
> >> >>>
> >> >>> Hello-
> >> >>>
> >> >>> We have a use case where we're trying to create a topic, delete,
> then
> >> >>> recreate with the same topic name.
> >> >>>
> >> >>> Running into inconsistant results.
> >> >>>
> >> >>> Creating the topic:
> >> >>> /opt/kafka/bin/kafka-topics.sh --create --partitions 3
> >> >>> --replication-factor 3 --topic test-01 --zookeeper zoo01:2181,
> >> >>> zoo02:2181, zoo03:2181
> >> >>>
> >> >>> Delete:
> >> >>> /opt/kafka/bin/kafka-topics.sh --delete --topic test-01 --zookeeper
> >> >>> zoo01:2181, zoo02:2181, zoo03:2181
> >> >>>
> >> >>> Repeat creation.
> >> >>>
> >> >>> The results are inconsistant. Executing the above several times can
> be
> >> >>> successful, then sporadically we get caught in "topic marked for
> >> >>> deletion" and it does not clear.
> >> >>>
> >> >>> This appears to be a Zookeeper issue of sorts as the logs will show:
> >> >>> [2015-12-30 22:32:32,946] WARN Conditional update of path
> >> >>> /brokers/topics/test-01/partitions/0/state with data
> >> >>>
> >> >>>
> >>
> {"controller_epoch":21,"leader":2,"version":1,"leader_epoch":1,"isr":[2,0,1]}
> >> >>> and expected version 1 failed due to
> >> >>> org.apache.zookeeper.KeeperException$NoNodeException:
> KeeperErrorCode
> >> >>> = NoNode for /brokers/topics/test-01/partitions/0/state
> >> >>> (kafka.utils.ZkUtils$)
> >> >>>
> >> >>> In this instance no subdirectories exist beyond
> /brokers/topics/test-01
> >> >>>
> >> >>> I'd like to know if this is a common occurrance and why the
> Zookeeper
> >> >>> node isn't "fully" created as Kafka deletion seems stuck without the
> >> >>> expected node path.
> >> >>>
> >> >>> We are using Kafka 8.2. Purging is also an option (if achievable
> >> >>> programmatically), we just need to make sure there are no messages
> >> >>> left when resuming the producer.
> >> >>>
> >> >>> Appreciate any info/guidance.
> >> >>>
> >> >>> Thanks,
> >> >>> BC
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> *Alex Loddengaard | **Solutions Architect | Confluent*
> >> >> *Download Apache Kafka and Confluent Platform:
> >> www.confluent.io/download
> >> >> <http://www.confluent.io/download>*
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > *Alex Loddengaard | **Solutions Architect | Confluent*
> >> > *Download Apache Kafka and Confluent Platform:
> www.confluent.io/download
> >> > <http://www.confluent.io/download>*
> >>
>
Re: Topic Deletion Issues
Posted by Brenden Cobb <br...@gmail.com>.
Hi Gwen- No, unfortunately it doesn't appear to be consistent. One day
I had no problem with rapid deletion/recreation, the next I did.
Kafka is still obsessing about the last failed deletion attempt and
won't move on to subsequent delete requests.
I've planned downtime today so I can restart Kafka (clearing the topic
info) and test again.
On Tue, Jan 5, 2016 at 1:30 PM, Gwen Shapira <gw...@confluent.io> wrote:
> Does the issue reproduce consistently?
>
> On Tue, Jan 5, 2016 at 10:28 AM, Brenden Cobb <br...@gmail.com> wrote:
>
>> Thanks Alex (and sorry all for the spam)
>>
>> I'll give it another go as suggested, however in my tests I was just
>> using the kafka-topics command and there was definitely a few seconds
>> between creation/deletion.
>>
>> -BC
>>
>> On Tue, Jan 5, 2016 at 1:24 PM, Alex Loddengaard <al...@confluent.io>
>> wrote:
>> > Hi Brenden, I sent the wrong permalink. Try this:
>> >
>> >
>> http://mail-archives.apache.org/mod_mbox/kafka-users/201512.mbox/%3CCAH7vnfhUQHQcCJ1R3pS_DYn0LvmA8fJtLtvCuvvDboWSqm-NBg%40mail.gmail.com%3E
>> >
>> > Alex
>> >
>> > On Tue, Jan 5, 2016 at 10:21 AM, Alex Loddengaard <al...@confluent.io>
>> wrote:
>> >
>> >> Hi Brenden, your previous email went through, and you got a response
>> (that
>> >> I believe answers your question). See here:
>> >>
>> >>
>> http://mail-archives.apache.org/mod_mbox/kafka-users/201512.mbox/browser
>> >>
>> >> Alex
>> >>
>> >> On Mon, Jan 4, 2016 at 7:18 PM, Brenden Cobb <br...@gmail.com>
>> >> wrote:
>> >>
>> >>> I might have sent this recently, but was not able to receive mail from
>> >>> this list (fixed)
>> >>> ---
>> >>>
>> >>> Hello-
>> >>>
>> >>> We have a use case where we're trying to create a topic, delete, then
>> >>> recreate with the same topic name.
>> >>>
>> >>> Running into inconsistant results.
>> >>>
>> >>> Creating the topic:
>> >>> /opt/kafka/bin/kafka-topics.sh --create --partitions 3
>> >>> --replication-factor 3 --topic test-01 --zookeeper zoo01:2181,
>> >>> zoo02:2181, zoo03:2181
>> >>>
>> >>> Delete:
>> >>> /opt/kafka/bin/kafka-topics.sh --delete --topic test-01 --zookeeper
>> >>> zoo01:2181, zoo02:2181, zoo03:2181
>> >>>
>> >>> Repeat creation.
>> >>>
>> >>> The results are inconsistant. Executing the above several times can be
>> >>> successful, then sporadically we get caught in "topic marked for
>> >>> deletion" and it does not clear.
>> >>>
>> >>> This appears to be a Zookeeper issue of sorts as the logs will show:
>> >>> [2015-12-30 22:32:32,946] WARN Conditional update of path
>> >>> /brokers/topics/test-01/partitions/0/state with data
>> >>>
>> >>>
>> {"controller_epoch":21,"leader":2,"version":1,"leader_epoch":1,"isr":[2,0,1]}
>> >>> and expected version 1 failed due to
>> >>> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode
>> >>> = NoNode for /brokers/topics/test-01/partitions/0/state
>> >>> (kafka.utils.ZkUtils$)
>> >>>
>> >>> In this instance no subdirectories exist beyond /brokers/topics/test-01
>> >>>
>> >>> I'd like to know if this is a common occurrance and why the Zookeeper
>> >>> node isn't "fully" created as Kafka deletion seems stuck without the
>> >>> expected node path.
>> >>>
>> >>> We are using Kafka 8.2. Purging is also an option (if achievable
>> >>> programmatically), we just need to make sure there are no messages
>> >>> left when resuming the producer.
>> >>>
>> >>> Appreciate any info/guidance.
>> >>>
>> >>> Thanks,
>> >>> BC
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> *Alex Loddengaard | **Solutions Architect | Confluent*
>> >> *Download Apache Kafka and Confluent Platform:
>> www.confluent.io/download
>> >> <http://www.confluent.io/download>*
>> >>
>> >
>> >
>> >
>> > --
>> > *Alex Loddengaard | **Solutions Architect | Confluent*
>> > *Download Apache Kafka and Confluent Platform: www.confluent.io/download
>> > <http://www.confluent.io/download>*
>>
Re: Topic Deletion Issues
Posted by Gwen Shapira <gw...@confluent.io>.
Does the issue reproduce consistently?
On Tue, Jan 5, 2016 at 10:28 AM, Brenden Cobb <br...@gmail.com> wrote:
> Thanks Alex (and sorry all for the spam)
>
> I'll give it another go as suggested, however in my tests I was just
> using the kafka-topics command and there was definitely a few seconds
> between creation/deletion.
>
> -BC
>
> On Tue, Jan 5, 2016 at 1:24 PM, Alex Loddengaard <al...@confluent.io>
> wrote:
> > Hi Brenden, I sent the wrong permalink. Try this:
> >
> >
> http://mail-archives.apache.org/mod_mbox/kafka-users/201512.mbox/%3CCAH7vnfhUQHQcCJ1R3pS_DYn0LvmA8fJtLtvCuvvDboWSqm-NBg%40mail.gmail.com%3E
> >
> > Alex
> >
> > On Tue, Jan 5, 2016 at 10:21 AM, Alex Loddengaard <al...@confluent.io>
> wrote:
> >
> >> Hi Brenden, your previous email went through, and you got a response
> (that
> >> I believe answers your question). See here:
> >>
> >>
> http://mail-archives.apache.org/mod_mbox/kafka-users/201512.mbox/browser
> >>
> >> Alex
> >>
> >> On Mon, Jan 4, 2016 at 7:18 PM, Brenden Cobb <br...@gmail.com>
> >> wrote:
> >>
> >>> I might have sent this recently, but was not able to receive mail from
> >>> this list (fixed)
> >>> ---
> >>>
> >>> Hello-
> >>>
> >>> We have a use case where we're trying to create a topic, delete, then
> >>> recreate with the same topic name.
> >>>
> >>> Running into inconsistant results.
> >>>
> >>> Creating the topic:
> >>> /opt/kafka/bin/kafka-topics.sh --create --partitions 3
> >>> --replication-factor 3 --topic test-01 --zookeeper zoo01:2181,
> >>> zoo02:2181, zoo03:2181
> >>>
> >>> Delete:
> >>> /opt/kafka/bin/kafka-topics.sh --delete --topic test-01 --zookeeper
> >>> zoo01:2181, zoo02:2181, zoo03:2181
> >>>
> >>> Repeat creation.
> >>>
> >>> The results are inconsistant. Executing the above several times can be
> >>> successful, then sporadically we get caught in "topic marked for
> >>> deletion" and it does not clear.
> >>>
> >>> This appears to be a Zookeeper issue of sorts as the logs will show:
> >>> [2015-12-30 22:32:32,946] WARN Conditional update of path
> >>> /brokers/topics/test-01/partitions/0/state with data
> >>>
> >>>
> {"controller_epoch":21,"leader":2,"version":1,"leader_epoch":1,"isr":[2,0,1]}
> >>> and expected version 1 failed due to
> >>> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode
> >>> = NoNode for /brokers/topics/test-01/partitions/0/state
> >>> (kafka.utils.ZkUtils$)
> >>>
> >>> In this instance no subdirectories exist beyond /brokers/topics/test-01
> >>>
> >>> I'd like to know if this is a common occurrance and why the Zookeeper
> >>> node isn't "fully" created as Kafka deletion seems stuck without the
> >>> expected node path.
> >>>
> >>> We are using Kafka 8.2. Purging is also an option (if achievable
> >>> programmatically), we just need to make sure there are no messages
> >>> left when resuming the producer.
> >>>
> >>> Appreciate any info/guidance.
> >>>
> >>> Thanks,
> >>> BC
> >>>
> >>
> >>
> >>
> >> --
> >> *Alex Loddengaard | **Solutions Architect | Confluent*
> >> *Download Apache Kafka and Confluent Platform:
> www.confluent.io/download
> >> <http://www.confluent.io/download>*
> >>
> >
> >
> >
> > --
> > *Alex Loddengaard | **Solutions Architect | Confluent*
> > *Download Apache Kafka and Confluent Platform: www.confluent.io/download
> > <http://www.confluent.io/download>*
>
Re: Topic Deletion Issues
Posted by Brenden Cobb <br...@gmail.com>.
Thanks Alex (and sorry all for the spam)
I'll give it another go as suggested, however in my tests I was just
using the kafka-topics command and there was definitely a few seconds
between creation/deletion.
-BC
On Tue, Jan 5, 2016 at 1:24 PM, Alex Loddengaard <al...@confluent.io> wrote:
> Hi Brenden, I sent the wrong permalink. Try this:
>
> http://mail-archives.apache.org/mod_mbox/kafka-users/201512.mbox/%3CCAH7vnfhUQHQcCJ1R3pS_DYn0LvmA8fJtLtvCuvvDboWSqm-NBg%40mail.gmail.com%3E
>
> Alex
>
> On Tue, Jan 5, 2016 at 10:21 AM, Alex Loddengaard <al...@confluent.io> wrote:
>
>> Hi Brenden, your previous email went through, and you got a response (that
>> I believe answers your question). See here:
>>
>> http://mail-archives.apache.org/mod_mbox/kafka-users/201512.mbox/browser
>>
>> Alex
>>
>> On Mon, Jan 4, 2016 at 7:18 PM, Brenden Cobb <br...@gmail.com>
>> wrote:
>>
>>> I might have sent this recently, but was not able to receive mail from
>>> this list (fixed)
>>> ---
>>>
>>> Hello-
>>>
>>> We have a use case where we're trying to create a topic, delete, then
>>> recreate with the same topic name.
>>>
>>> Running into inconsistant results.
>>>
>>> Creating the topic:
>>> /opt/kafka/bin/kafka-topics.sh --create --partitions 3
>>> --replication-factor 3 --topic test-01 --zookeeper zoo01:2181,
>>> zoo02:2181, zoo03:2181
>>>
>>> Delete:
>>> /opt/kafka/bin/kafka-topics.sh --delete --topic test-01 --zookeeper
>>> zoo01:2181, zoo02:2181, zoo03:2181
>>>
>>> Repeat creation.
>>>
>>> The results are inconsistant. Executing the above several times can be
>>> successful, then sporadically we get caught in "topic marked for
>>> deletion" and it does not clear.
>>>
>>> This appears to be a Zookeeper issue of sorts as the logs will show:
>>> [2015-12-30 22:32:32,946] WARN Conditional update of path
>>> /brokers/topics/test-01/partitions/0/state with data
>>>
>>> {"controller_epoch":21,"leader":2,"version":1,"leader_epoch":1,"isr":[2,0,1]}
>>> and expected version 1 failed due to
>>> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode
>>> = NoNode for /brokers/topics/test-01/partitions/0/state
>>> (kafka.utils.ZkUtils$)
>>>
>>> In this instance no subdirectories exist beyond /brokers/topics/test-01
>>>
>>> I'd like to know if this is a common occurrance and why the Zookeeper
>>> node isn't "fully" created as Kafka deletion seems stuck without the
>>> expected node path.
>>>
>>> We are using Kafka 8.2. Purging is also an option (if achievable
>>> programmatically), we just need to make sure there are no messages
>>> left when resuming the producer.
>>>
>>> Appreciate any info/guidance.
>>>
>>> Thanks,
>>> BC
>>>
>>
>>
>>
>> --
>> *Alex Loddengaard | **Solutions Architect | Confluent*
>> *Download Apache Kafka and Confluent Platform: www.confluent.io/download
>> <http://www.confluent.io/download>*
>>
>
>
>
> --
> *Alex Loddengaard | **Solutions Architect | Confluent*
> *Download Apache Kafka and Confluent Platform: www.confluent.io/download
> <http://www.confluent.io/download>*
Re: Topic Deletion Issues
Posted by Alex Loddengaard <al...@confluent.io>.
Hi Brenden, I sent the wrong permalink. Try this:
http://mail-archives.apache.org/mod_mbox/kafka-users/201512.mbox/%3CCAH7vnfhUQHQcCJ1R3pS_DYn0LvmA8fJtLtvCuvvDboWSqm-NBg%40mail.gmail.com%3E
Alex
On Tue, Jan 5, 2016 at 10:21 AM, Alex Loddengaard <al...@confluent.io> wrote:
> Hi Brenden, your previous email went through, and you got a response (that
> I believe answers your question). See here:
>
> http://mail-archives.apache.org/mod_mbox/kafka-users/201512.mbox/browser
>
> Alex
>
> On Mon, Jan 4, 2016 at 7:18 PM, Brenden Cobb <br...@gmail.com>
> wrote:
>
>> I might have sent this recently, but was not able to receive mail from
>> this list (fixed)
>> ---
>>
>> Hello-
>>
>> We have a use case where we're trying to create a topic, delete, then
>> recreate with the same topic name.
>>
>> Running into inconsistant results.
>>
>> Creating the topic:
>> /opt/kafka/bin/kafka-topics.sh --create --partitions 3
>> --replication-factor 3 --topic test-01 --zookeeper zoo01:2181,
>> zoo02:2181, zoo03:2181
>>
>> Delete:
>> /opt/kafka/bin/kafka-topics.sh --delete --topic test-01 --zookeeper
>> zoo01:2181, zoo02:2181, zoo03:2181
>>
>> Repeat creation.
>>
>> The results are inconsistant. Executing the above several times can be
>> successful, then sporadically we get caught in "topic marked for
>> deletion" and it does not clear.
>>
>> This appears to be a Zookeeper issue of sorts as the logs will show:
>> [2015-12-30 22:32:32,946] WARN Conditional update of path
>> /brokers/topics/test-01/partitions/0/state with data
>>
>> {"controller_epoch":21,"leader":2,"version":1,"leader_epoch":1,"isr":[2,0,1]}
>> and expected version 1 failed due to
>> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode
>> = NoNode for /brokers/topics/test-01/partitions/0/state
>> (kafka.utils.ZkUtils$)
>>
>> In this instance no subdirectories exist beyond /brokers/topics/test-01
>>
>> I'd like to know if this is a common occurrance and why the Zookeeper
>> node isn't "fully" created as Kafka deletion seems stuck without the
>> expected node path.
>>
>> We are using Kafka 8.2. Purging is also an option (if achievable
>> programmatically), we just need to make sure there are no messages
>> left when resuming the producer.
>>
>> Appreciate any info/guidance.
>>
>> Thanks,
>> BC
>>
>
>
>
> --
> *Alex Loddengaard | **Solutions Architect | Confluent*
> *Download Apache Kafka and Confluent Platform: www.confluent.io/download
> <http://www.confluent.io/download>*
>
--
*Alex Loddengaard | **Solutions Architect | Confluent*
*Download Apache Kafka and Confluent Platform: www.confluent.io/download
<http://www.confluent.io/download>*
Re: Topic Deletion Issues
Posted by Alex Loddengaard <al...@confluent.io>.
Hi Brenden, your previous email went through, and you got a response (that
I believe answers your question). See here:
http://mail-archives.apache.org/mod_mbox/kafka-users/201512.mbox/browser
Alex
On Mon, Jan 4, 2016 at 7:18 PM, Brenden Cobb <br...@gmail.com> wrote:
> I might have sent this recently, but was not able to receive mail from
> this list (fixed)
> ---
>
> Hello-
>
> We have a use case where we're trying to create a topic, delete, then
> recreate with the same topic name.
>
> Running into inconsistant results.
>
> Creating the topic:
> /opt/kafka/bin/kafka-topics.sh --create --partitions 3
> --replication-factor 3 --topic test-01 --zookeeper zoo01:2181,
> zoo02:2181, zoo03:2181
>
> Delete:
> /opt/kafka/bin/kafka-topics.sh --delete --topic test-01 --zookeeper
> zoo01:2181, zoo02:2181, zoo03:2181
>
> Repeat creation.
>
> The results are inconsistant. Executing the above several times can be
> successful, then sporadically we get caught in "topic marked for
> deletion" and it does not clear.
>
> This appears to be a Zookeeper issue of sorts as the logs will show:
> [2015-12-30 22:32:32,946] WARN Conditional update of path
> /brokers/topics/test-01/partitions/0/state with data
>
> {"controller_epoch":21,"leader":2,"version":1,"leader_epoch":1,"isr":[2,0,1]}
> and expected version 1 failed due to
> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode
> = NoNode for /brokers/topics/test-01/partitions/0/state
> (kafka.utils.ZkUtils$)
>
> In this instance no subdirectories exist beyond /brokers/topics/test-01
>
> I'd like to know if this is a common occurrance and why the Zookeeper
> node isn't "fully" created as Kafka deletion seems stuck without the
> expected node path.
>
> We are using Kafka 8.2. Purging is also an option (if achievable
> programmatically), we just need to make sure there are no messages
> left when resuming the producer.
>
> Appreciate any info/guidance.
>
> Thanks,
> BC
>
--
*Alex Loddengaard | **Solutions Architect | Confluent*
*Download Apache Kafka and Confluent Platform: www.confluent.io/download
<http://www.confluent.io/download>*