You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by anthony musyoki <an...@gmail.com> on 2015/03/23 09:29:20 UTC
Check topic exists after deleting it.
On deleting a topic via TopicCommand.deleteTopic()
I get "Topic test-delete is marked for deletion."
I follow up by checking if the topic exists by using
AdminUtils.topicExists()
which suprisingly returns true.
I expected AdminUtils.TopicExists() to check both BrokerTopicsPath
and DeleteTopicsPath before returning a verdict but it only checks
BrokerTopicsPath
Shouldn't a topic marked for deletion return false for topicExists() ?
Re: Check topic exists after deleting it.
Posted by anthony musyoki <an...@gmail.com>.
Thanks for your prompt response.
In my check for topicExists i will add a check for topic in
DeleteTopicsPath.
On Mon, Mar 23, 2015 at 8:21 PM, Harsha <ka...@harsha.io> wrote:
> Just to be clear, one needs to stop producers and consumers that
> writing/reading from a topic “test” if they are trying to delete that
> specific topic “test”. Not all producers and clients.
>
> --
> Harsha
>
> On March 23, 2015 at 10:13:47 AM, Harsha (harshach@fastmail.fm) wrote:
>
> Currently we have auto.create.topics.enable set to true by default. If
> this is set true any one who is making TopicMetadataRequest can create a
> topic . As both producers and consumers can send TopicMetadataRequest which
> will create a topic if the above config is true. So while doing deletion if
> there is producer or consumer running it can re-create a topic thats in
> deletion process. This issue going to be addressed in upcoming versions.
> Meanwhile if you are not creating topics via producer than turn this config
> off or stop producer and consumers while you are trying to delete a topic.
> --
> Harsha
>
>
> On March 23, 2015 at 9:57:53 AM, Grant Henke (ghenke@cloudera.com) wrote:
>
> What happens when producers or consumers are running while the topic
> deleting is going on?
>
> On Mon, Mar 23, 2015 at 10:02 AM, Harsha <ka...@harsha.io> wrote:
>
> > DeleteTopic makes a node in zookeeper to let controller know that there
> is
> > a topic up for deletion. This doesn’t immediately delete the topic it can
> > take time depending if all the partitions of that topic are online and
> > brokers are available as well. Once all the Log files deleted zookeeper
> > node gets deleted as well.
> > Also make sure you don’t have any producers or consumers are running
> while
> > the topic deleting is going on.
> >
> > --
> > Harsha
> >
> >
> > On March 23, 2015 at 1:29:50 AM, anthony musyoki (
> > anthony.musyoki@gmail.com) wrote:
> >
> > On deleting a topic via TopicCommand.deleteTopic()
> >
> > I get "Topic test-delete is marked for deletion."
> >
> > I follow up by checking if the topic exists by using
> > AdminUtils.topicExists()
> > which suprisingly returns true.
> >
> > I expected AdminUtils.TopicExists() to check both BrokerTopicsPath
> > and DeleteTopicsPath before returning a verdict but it only checks
> > BrokerTopicsPath
> >
> > Shouldn't a topic marked for deletion return false for topicExists() ?
> >
>
>
>
> --
> Grant Henke
> Solutions Consultant | Cloudera
> ghenke@cloudera.com | 920-980-8979
> twitter.com/ghenke <http://twitter.com/gchenke> |
> linkedin.com/in/granthenke
>
>
Re: Check topic exists after deleting it.
Posted by Harsha <ka...@harsha.io>.
Just to be clear, one needs to stop producers and consumers that writing/reading from a topic “test” if they are trying to delete that specific topic “test”. Not all producers and clients.
--
Harsha
On March 23, 2015 at 10:13:47 AM, Harsha (harshach@fastmail.fm) wrote:
Currently we have auto.create.topics.enable set to true by default. If this is set true any one who is making TopicMetadataRequest can create a topic . As both producers and consumers can send TopicMetadataRequest which will create a topic if the above config is true. So while doing deletion if there is producer or consumer running it can re-create a topic thats in deletion process. This issue going to be addressed in upcoming versions. Meanwhile if you are not creating topics via producer than turn this config off or stop producer and consumers while you are trying to delete a topic.
--
Harsha
On March 23, 2015 at 9:57:53 AM, Grant Henke (ghenke@cloudera.com) wrote:
What happens when producers or consumers are running while the topic
deleting is going on?
On Mon, Mar 23, 2015 at 10:02 AM, Harsha <ka...@harsha.io> wrote:
> DeleteTopic makes a node in zookeeper to let controller know that there is
> a topic up for deletion. This doesn’t immediately delete the topic it can
> take time depending if all the partitions of that topic are online and
> brokers are available as well. Once all the Log files deleted zookeeper
> node gets deleted as well.
> Also make sure you don’t have any producers or consumers are running while
> the topic deleting is going on.
>
> --
> Harsha
>
>
> On March 23, 2015 at 1:29:50 AM, anthony musyoki (
> anthony.musyoki@gmail.com) wrote:
>
> On deleting a topic via TopicCommand.deleteTopic()
>
> I get "Topic test-delete is marked for deletion."
>
> I follow up by checking if the topic exists by using
> AdminUtils.topicExists()
> which suprisingly returns true.
>
> I expected AdminUtils.TopicExists() to check both BrokerTopicsPath
> and DeleteTopicsPath before returning a verdict but it only checks
> BrokerTopicsPath
>
> Shouldn't a topic marked for deletion return false for topicExists() ?
>
--
Grant Henke
Solutions Consultant | Cloudera
ghenke@cloudera.com | 920-980-8979
twitter.com/ghenke <http://twitter.com/gchenke> | linkedin.com/in/granthenke
Re: Check topic exists after deleting it.
Posted by Harsha <ha...@fastmail.fm>.
Currently we have auto.create.topics.enable set to true by default. If this is set true any one who is making TopicMetadataRequest can create a topic . As both producers and consumers can send TopicMetadataRequest which will create a topic if the above config is true. So while doing deletion if there is producer or consumer running it can re-create a topic thats in deletion process. This issue going to be addressed in upcoming versions. Meanwhile if you are not creating topics via producer than turn this config off or stop producer and consumers while you are trying to delete a topic.
--
Harsha
On March 23, 2015 at 9:57:53 AM, Grant Henke (ghenke@cloudera.com) wrote:
What happens when producers or consumers are running while the topic
deleting is going on?
On Mon, Mar 23, 2015 at 10:02 AM, Harsha <ka...@harsha.io> wrote:
> DeleteTopic makes a node in zookeeper to let controller know that there is
> a topic up for deletion. This doesn’t immediately delete the topic it can
> take time depending if all the partitions of that topic are online and
> brokers are available as well. Once all the Log files deleted zookeeper
> node gets deleted as well.
> Also make sure you don’t have any producers or consumers are running while
> the topic deleting is going on.
>
> --
> Harsha
>
>
> On March 23, 2015 at 1:29:50 AM, anthony musyoki (
> anthony.musyoki@gmail.com) wrote:
>
> On deleting a topic via TopicCommand.deleteTopic()
>
> I get "Topic test-delete is marked for deletion."
>
> I follow up by checking if the topic exists by using
> AdminUtils.topicExists()
> which suprisingly returns true.
>
> I expected AdminUtils.TopicExists() to check both BrokerTopicsPath
> and DeleteTopicsPath before returning a verdict but it only checks
> BrokerTopicsPath
>
> Shouldn't a topic marked for deletion return false for topicExists() ?
>
--
Grant Henke
Solutions Consultant | Cloudera
ghenke@cloudera.com | 920-980-8979
twitter.com/ghenke <http://twitter.com/gchenke> | linkedin.com/in/granthenke
Re: Check topic exists after deleting it.
Posted by Grant Henke <gh...@cloudera.com>.
What happens when producers or consumers are running while the topic
deleting is going on?
On Mon, Mar 23, 2015 at 10:02 AM, Harsha <ka...@harsha.io> wrote:
> DeleteTopic makes a node in zookeeper to let controller know that there is
> a topic up for deletion. This doesn’t immediately delete the topic it can
> take time depending if all the partitions of that topic are online and
> brokers are available as well. Once all the Log files deleted zookeeper
> node gets deleted as well.
> Also make sure you don’t have any producers or consumers are running while
> the topic deleting is going on.
>
> --
> Harsha
>
>
> On March 23, 2015 at 1:29:50 AM, anthony musyoki (
> anthony.musyoki@gmail.com) wrote:
>
> On deleting a topic via TopicCommand.deleteTopic()
>
> I get "Topic test-delete is marked for deletion."
>
> I follow up by checking if the topic exists by using
> AdminUtils.topicExists()
> which suprisingly returns true.
>
> I expected AdminUtils.TopicExists() to check both BrokerTopicsPath
> and DeleteTopicsPath before returning a verdict but it only checks
> BrokerTopicsPath
>
> Shouldn't a topic marked for deletion return false for topicExists() ?
>
--
Grant Henke
Solutions Consultant | Cloudera
ghenke@cloudera.com | 920-980-8979
twitter.com/ghenke <http://twitter.com/gchenke> | linkedin.com/in/granthenke
Re: Check topic exists after deleting it.
Posted by Harsha <ka...@harsha.io>.
DeleteTopic makes a node in zookeeper to let controller know that there is a topic up for deletion. This doesn’t immediately delete the topic it can take time depending if all the partitions of that topic are online and brokers are available as well. Once all the Log files deleted zookeeper node gets deleted as well.
Also make sure you don’t have any producers or consumers are running while the topic deleting is going on.
--
Harsha
On March 23, 2015 at 1:29:50 AM, anthony musyoki (anthony.musyoki@gmail.com) wrote:
On deleting a topic via TopicCommand.deleteTopic()
I get "Topic test-delete is marked for deletion."
I follow up by checking if the topic exists by using
AdminUtils.topicExists()
which suprisingly returns true.
I expected AdminUtils.TopicExists() to check both BrokerTopicsPath
and DeleteTopicsPath before returning a verdict but it only checks
BrokerTopicsPath
Shouldn't a topic marked for deletion return false for topicExists() ?