Unable to delete topic - kafka 0.8.0

[Virendra Pratap Singh <vp...@yahoo-inc.com.INVALID>]

I am aware of lack of programmatic way of deleting topics in kafka 0.8.0. So using the sledge hammer approach.
This is what I am doing:

1. Bring whole of my kafka cluster down.
2. Delete all the content on all the kafka clusters pointed via logs.dir setting.
3. Delete the topic metadata from zookeeper : rmr /brokers (note I am not wiping off the whole zookeeper but the znode /brokers where the kafka broker ids and topic metadata is stored)
4. Restart the kafka cluster again.

One would expect that the kafka cluster will come up with no memory of any topic from previous.

But guess what, and this is the place where I need help and need to understand, when the kafka cluster comes back, it somehow is able to obtain the info of the previous topics. It promptly goes ahead creating and assigning partitions/replicas to the brokers for the previous topics. Now I am completely at loss to understand where exactly is kafka able to get the info of previous topics when I have wiped it off the zookeeper and also dropped the logs.dir locations across the kafka cluster.

An insight is much needed here. Where else is the topic meta data store which the kafka server is getting hold of after coming back alive?

Regards,
Virendra

Re: Unable to delete topic - kafka 0.8.0

[Virendra Pratap Singh <vp...@yahoo-inc.com.INVALID>]

No producers were active. The only producer in our pipeline is a storm
topology and had made it a point that the whole cluster was down and no
topology running.
What I did was nothing short of doing a fresh kafka cluster setup (barring
the fact that I didn¹t wiped the zookeepers as its the same setup which
our storm uses. But given the fact that I had dropped /brokers, it should
not be concern).

So the question is where exactly is kafka brokers getting the info about
the topic to recreate when they come up when there is no producers and all
previous info has been wiped out.

Regards,
Virendra

On 6/26/14, 9:29 AM, "Neha Narkhede" <ne...@gmail.com> wrote:

>Firstly, I'm not sure this process of deleting topics completely works,
>especially in corner cases. Having said that, for #3, you should just be
>deleting /brokers/topics/<topics-to-be-deleted>. If producers are sending
>data to the Kafka cluster, it will recreate the topics once the brokers
>come up.
>
>Thanks,
>Neha
>
>
>On Wed, Jun 25, 2014 at 11:53 PM, Virendra Pratap Singh <
>vpsingh@yahoo-inc.com.invalid> wrote:
>
>> I am aware of lack of programmatic way of deleting topics in kafka
>>0.8.0.
>> So using the sledge hammer approach.
>> This is what I am doing:
>>
>> 1. Bring whole of my kafka cluster down.
>> 2. Delete all the content on all the kafka clusters pointed via logs.dir
>> setting.
>> 3. Delete the topic metadata from zookeeper : rmr /brokers (note I am
>>not
>> wiping off the whole zookeeper but the znode /brokers where the kafka
>> broker ids and topic metadata is stored)
>> 4. Restart the kafka cluster again.
>>
>> One would expect that the kafka cluster will come up with no memory of
>>any
>> topic from previous.
>>
>> But guess what, and this is the place where I need help and need to
>> understand, when the kafka cluster comes back, it somehow is able to
>>obtain
>> the info of the previous topics. It promptly goes ahead creating and
>> assigning partitions/replicas to the brokers for the previous topics.
>>Now I
>> am completely at loss to understand where exactly is kafka able to get
>>the
>> info of previous topics when I have wiped it off the zookeeper and also
>> dropped the logs.dir locations across the kafka cluster.
>>
>> An insight is much needed here. Where else is the topic meta data store
>> which the kafka server is getting hold of after coming back alive?
>>
>> Regards,
>> Virendra
>>
>>

Re: Unable to delete topic - kafka 0.8.0

[Neha Narkhede <ne...@gmail.com>]

Firstly, I'm not sure this process of deleting topics completely works,
especially in corner cases. Having said that, for #3, you should just be
deleting /brokers/topics/<topics-to-be-deleted>. If producers are sending
data to the Kafka cluster, it will recreate the topics once the brokers
come up.

Thanks,
Neha


On Wed, Jun 25, 2014 at 11:53 PM, Virendra Pratap Singh <
vpsingh@yahoo-inc.com.invalid> wrote:

> I am aware of lack of programmatic way of deleting topics in kafka 0.8.0.
> So using the sledge hammer approach.
> This is what I am doing:
>
> 1. Bring whole of my kafka cluster down.
> 2. Delete all the content on all the kafka clusters pointed via logs.dir
> setting.
> 3. Delete the topic metadata from zookeeper : rmr /brokers (note I am not
> wiping off the whole zookeeper but the znode /brokers where the kafka
> broker ids and topic metadata is stored)
> 4. Restart the kafka cluster again.
>
> One would expect that the kafka cluster will come up with no memory of any
> topic from previous.
>
> But guess what, and this is the place where I need help and need to
> understand, when the kafka cluster comes back, it somehow is able to obtain
> the info of the previous topics. It promptly goes ahead creating and
> assigning partitions/replicas to the brokers for the previous topics. Now I
> am completely at loss to understand where exactly is kafka able to get the
> info of previous topics when I have wiped it off the zookeeper and also
> dropped the logs.dir locations across the kafka cluster.
>
> An insight is much needed here. Where else is the topic meta data store
> which the kafka server is getting hold of after coming back alive?
>
> Regards,
> Virendra
>
>

Re: Unable to delete topic - kafka 0.8.0

[Virendra Pratap Singh <vp...@yahoo-inc.com.INVALID>]

I am 100% sure nothing was running. However I am not sure of consumers.
Would that make any difference?
I thought producer message write request could only cause kafka to
initiate topic creation, not consumer read. Would consumer read request
even succeed if the topic metadata is non-existent in zookeeper.
Since I brought the kafka brokers down, and deleted all logs and zookeeper
metadata, I would assume consumers will sync with that info and fail, as
now the topic is nonexistent. Not sure how consumer can impact in this
situation.

Compared to clean install, the only thing I see is zookeepers not wiped
off and reinstalled/restarted. But I believe that¹s should not be an issue
as I dropped the /brokers znode.

Virendra

On 6/26/14, 9:28 AM, "Guozhang Wang" <wa...@gmail.com> wrote:

>Hello Virendra,
>
>Did you have any producer/consumer clients running during the whole
>process?
>
>Guozhang
>
>
>On Wed, Jun 25, 2014 at 11:53 PM, Virendra Pratap Singh <
>vpsingh@yahoo-inc.com.invalid> wrote:
>
>> I am aware of lack of programmatic way of deleting topics in kafka
>>0.8.0.
>> So using the sledge hammer approach.
>> This is what I am doing:
>>
>> 1. Bring whole of my kafka cluster down.
>> 2. Delete all the content on all the kafka clusters pointed via logs.dir
>> setting.
>> 3. Delete the topic metadata from zookeeper : rmr /brokers (note I am
>>not
>> wiping off the whole zookeeper but the znode /brokers where the kafka
>> broker ids and topic metadata is stored)
>> 4. Restart the kafka cluster again.
>>
>> One would expect that the kafka cluster will come up with no memory of
>>any
>> topic from previous.
>>
>> But guess what, and this is the place where I need help and need to
>> understand, when the kafka cluster comes back, it somehow is able to
>>obtain
>> the info of the previous topics. It promptly goes ahead creating and
>> assigning partitions/replicas to the brokers for the previous topics.
>>Now I
>> am completely at loss to understand where exactly is kafka able to get
>>the
>> info of previous topics when I have wiped it off the zookeeper and also
>> dropped the logs.dir locations across the kafka cluster.
>>
>> An insight is much needed here. Where else is the topic meta data store
>> which the kafka server is getting hold of after coming back alive?
>>
>> Regards,
>> Virendra
>>
>>
>
>
>-- 
>-- Guozhang

Re: Unable to delete topic - kafka 0.8.0

[Guozhang Wang <wa...@gmail.com>]

Hello Virendra,

Did you have any producer/consumer clients running during the whole process?

Guozhang


On Wed, Jun 25, 2014 at 11:53 PM, Virendra Pratap Singh <
vpsingh@yahoo-inc.com.invalid> wrote:

> I am aware of lack of programmatic way of deleting topics in kafka 0.8.0.
> So using the sledge hammer approach.
> This is what I am doing:
>
> 1. Bring whole of my kafka cluster down.
> 2. Delete all the content on all the kafka clusters pointed via logs.dir
> setting.
> 3. Delete the topic metadata from zookeeper : rmr /brokers (note I am not
> wiping off the whole zookeeper but the znode /brokers where the kafka
> broker ids and topic metadata is stored)
> 4. Restart the kafka cluster again.
>
> One would expect that the kafka cluster will come up with no memory of any
> topic from previous.
>
> But guess what, and this is the place where I need help and need to
> understand, when the kafka cluster comes back, it somehow is able to obtain
> the info of the previous topics. It promptly goes ahead creating and
> assigning partitions/replicas to the brokers for the previous topics. Now I
> am completely at loss to understand where exactly is kafka able to get the
> info of previous topics when I have wiped it off the zookeeper and also
> dropped the logs.dir locations across the kafka cluster.
>
> An insight is much needed here. Where else is the topic meta data store
> which the kafka server is getting hold of after coming back alive?
>
> Regards,
> Virendra
>
>


-- 
-- Guozhang