You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Omnia Ibrahim <o....@gmail.com> on 2021/10/28 13:57:14 UTC
Track topic deletion state without ZK
Hi,
Kafka topicCommand used to report which topic is marked for deletion by
checking the znode on zookeeper; this feature has been deprecated without
replacement as part of KAFKA-12596 Remove deprecated --zookeeper in
topicCommands <https://issues.apache.org/jira/browse/KAFKA-12596>.
Also as far as I can see, there's no equivalent for this with KIP-500 as
well.
Is there any other way to know the state of deletion for Kafka with ZK and
Without ZK?
Is possible to leverage `RemoveTopicRecord` on the metadata topic to
provide the same feature?
Thanks
Omnia
Re: Track topic deletion state without ZK
Posted by Omnia Ibrahim <o....@gmail.com>.
Hi,
To clarify here, what I meant by "stuck" for deletion is regarding the
state of LogDir deletion. When the cluster receives a topic deletion
request, the KRAFT will delete the topic's metadata and eventually delete
all LogDirs on any broker/disk at some point (correct me if I am wrong).
My question is how to confirm the deletion state of these LogDirs. For
example, what will happen for the following case?
- Topic-A has one partition with four replicas on broker1, 2, 3, 4
- The owner of Topic-A fired a topic deletion request.
- Broker1,2,3 have no issues; however, the disk on broker 4 isn't
reachable.
- Broker1,2,3 deleted any LogDir they had for Topic-A.
- Broker 4 can't delete the LogDir (for example, the disk went to
read-only mode).
- Now Topic-A's data isn't entirely deleted until this last LogDir on
broker-4 is deleted.
- For data compliance, we need to confirm the deletion of the data.
Will KRAFT keeps listing the Topic-A until the last LogDir on broker-4 gets
deleted? If this is the case, then this topic should be marked "Pending for
deletion" for transparency; If KRAFT does not list the topic anymore, how
will we confirm that all topic's LogDirs have been deleted?
Thanks,
Omnia
On Tue, Nov 16, 2021 at 9:19 PM Colin McCabe <cm...@apache.org> wrote:
> Hi Omnia,
>
> Topic deletion doesn't get stuck if a broker is down, when using KRaft.
> There is no "deleting" state, only deleted or not deleted.
>
> best,
> Colin
>
> On Tue, Nov 2, 2021, at 09:24, Omnia Ibrahim wrote:
> > Hi Colin, thanks for your response.
> > Regards your point that the topic gets deleted immediately, I got that we
> > do this if the cluster is healthy.
> > However, if there's a hardware failure with the disk or the broker is
> > unreachable and has a replica; In these cases, deleting the log files
> from
> > the failed disk or unreachable broker will be impossible to delete until
> we
> > fix the hardware issue,
> > So during troubleshooting, how will we know which topic is stuck for
> > deletion because we can't delete some replicas because of hardware
> > failures?
> >
> > Thanks
> >
> >
> > On Mon, Nov 1, 2021 at 8:57 PM Colin McCabe <cm...@apache.org> wrote:
> >
> >> Hi Omnia,
> >>
> >> It is not necessary to know which topics are marked for deletion in when
> >> in KRaft mode, because topic deletion happens immediately.
> >>
> >> best,
> >> Colin
> >>
> >> On Thu, Oct 28, 2021, at 06:57, Omnia Ibrahim wrote:
> >> > Hi,
> >> >
> >> > Kafka topicCommand used to report which topic is marked for deletion
> by
> >> > checking the znode on zookeeper; this feature has been deprecated
> without
> >> > replacement as part of KAFKA-12596 Remove deprecated --zookeeper in
> >> > topicCommands <https://issues.apache.org/jira/browse/KAFKA-12596>.
> >> >
> >> > Also as far as I can see, there's no equivalent for this with KIP-500
> as
> >> > well.
> >> >
> >> > Is there any other way to know the state of deletion for Kafka with ZK
> >> and
> >> > Without ZK?
> >> >
> >> > Is possible to leverage `RemoveTopicRecord` on the metadata topic to
> >> > provide the same feature?
> >> >
> >> >
> >> > Thanks
> >> >
> >> > Omnia
> >>
>
Re: Track topic deletion state without ZK
Posted by Colin McCabe <cm...@apache.org>.
Hi Omnia,
Topic deletion doesn't get stuck if a broker is down, when using KRaft. There is no "deleting" state, only deleted or not deleted.
best,
Colin
On Tue, Nov 2, 2021, at 09:24, Omnia Ibrahim wrote:
> Hi Colin, thanks for your response.
> Regards your point that the topic gets deleted immediately, I got that we
> do this if the cluster is healthy.
> However, if there's a hardware failure with the disk or the broker is
> unreachable and has a replica; In these cases, deleting the log files from
> the failed disk or unreachable broker will be impossible to delete until we
> fix the hardware issue,
> So during troubleshooting, how will we know which topic is stuck for
> deletion because we can't delete some replicas because of hardware
> failures?
>
> Thanks
>
>
> On Mon, Nov 1, 2021 at 8:57 PM Colin McCabe <cm...@apache.org> wrote:
>
>> Hi Omnia,
>>
>> It is not necessary to know which topics are marked for deletion in when
>> in KRaft mode, because topic deletion happens immediately.
>>
>> best,
>> Colin
>>
>> On Thu, Oct 28, 2021, at 06:57, Omnia Ibrahim wrote:
>> > Hi,
>> >
>> > Kafka topicCommand used to report which topic is marked for deletion by
>> > checking the znode on zookeeper; this feature has been deprecated without
>> > replacement as part of KAFKA-12596 Remove deprecated --zookeeper in
>> > topicCommands <https://issues.apache.org/jira/browse/KAFKA-12596>.
>> >
>> > Also as far as I can see, there's no equivalent for this with KIP-500 as
>> > well.
>> >
>> > Is there any other way to know the state of deletion for Kafka with ZK
>> and
>> > Without ZK?
>> >
>> > Is possible to leverage `RemoveTopicRecord` on the metadata topic to
>> > provide the same feature?
>> >
>> >
>> > Thanks
>> >
>> > Omnia
>>
Re: Track topic deletion state without ZK
Posted by Omnia Ibrahim <o....@gmail.com>.
Hi Colin, thanks for your response.
Regards your point that the topic gets deleted immediately, I got that we
do this if the cluster is healthy.
However, if there's a hardware failure with the disk or the broker is
unreachable and has a replica; In these cases, deleting the log files from
the failed disk or unreachable broker will be impossible to delete until we
fix the hardware issue,
So during troubleshooting, how will we know which topic is stuck for
deletion because we can't delete some replicas because of hardware
failures?
Thanks
On Mon, Nov 1, 2021 at 8:57 PM Colin McCabe <cm...@apache.org> wrote:
> Hi Omnia,
>
> It is not necessary to know which topics are marked for deletion in when
> in KRaft mode, because topic deletion happens immediately.
>
> best,
> Colin
>
> On Thu, Oct 28, 2021, at 06:57, Omnia Ibrahim wrote:
> > Hi,
> >
> > Kafka topicCommand used to report which topic is marked for deletion by
> > checking the znode on zookeeper; this feature has been deprecated without
> > replacement as part of KAFKA-12596 Remove deprecated --zookeeper in
> > topicCommands <https://issues.apache.org/jira/browse/KAFKA-12596>.
> >
> > Also as far as I can see, there's no equivalent for this with KIP-500 as
> > well.
> >
> > Is there any other way to know the state of deletion for Kafka with ZK
> and
> > Without ZK?
> >
> > Is possible to leverage `RemoveTopicRecord` on the metadata topic to
> > provide the same feature?
> >
> >
> > Thanks
> >
> > Omnia
>
Re: Track topic deletion state without ZK
Posted by Colin McCabe <cm...@apache.org>.
Hi Omnia,
It is not necessary to know which topics are marked for deletion in when in KRaft mode, because topic deletion happens immediately.
best,
Colin
On Thu, Oct 28, 2021, at 06:57, Omnia Ibrahim wrote:
> Hi,
>
> Kafka topicCommand used to report which topic is marked for deletion by
> checking the znode on zookeeper; this feature has been deprecated without
> replacement as part of KAFKA-12596 Remove deprecated --zookeeper in
> topicCommands <https://issues.apache.org/jira/browse/KAFKA-12596>.
>
> Also as far as I can see, there's no equivalent for this with KIP-500 as
> well.
>
> Is there any other way to know the state of deletion for Kafka with ZK and
> Without ZK?
>
> Is possible to leverage `RemoveTopicRecord` on the metadata topic to
> provide the same feature?
>
>
> Thanks
>
> Omnia