Re: [VOTE] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io.INVALID>]

Hello again. Quick update on KIP-516. After much discussion and thought,
I've updated the KIP to include new admin apis and new/updated classes to
support them. The new APIs are:

default DescribeTopicsResult describeTopics(TopicCollection topics);

DescribeTopicsResult describeTopics(TopicCollection topics,
DescribeTopicsOptions options);

default DeleteTopicsResult deleteTopics(TopicCollection topics);

DeleteTopicsResult deleteTopics(TopicCollection topics,
DeleteTopicsOptions options);


As you may notice, there is a new class TopicCollection that can store
topics by name or ID. This is a new class also described in the KIP.
Finally Delete/DescribeTopicsResult classes have been modified to
support topic IDs as well as name and some of the older methods will
be deprecated. For more information, please check out the
KIP.https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-AdminClientSupport

Thanks,
Justine


On Tue, Mar 30, 2021 at 2:38 PM Justine Olshan <jo...@confluent.io> wrote:

> Hi all,
> Another quick update. After some offline discussion with KIP-500 folks,
> I'm making a small tweak to one of the configs in KIP-516.
> Instead of delete.stale.topics.ms, KIP-516 will introduce
> delete.topic.delay.ms which is defined as *"**The minimum amount of time
> to wait before removing a deleted topic's data on every broker."*
> The idea behind this config is to give a configurable window before the
> data is fully deleted and removed from the brokers. This config will apply
> to all topic deletions, not just the "stale topic" case described in
> KIP-516.
>
> Let me know if there are any questions,
> Justine
>
> On Thu, Feb 18, 2021 at 10:16 AM Justine Olshan <jo...@confluent.io>
> wrote:
>
>> Hi all,
>> I realized that the DISCUSS thread got very long, so I'll be posting
>> updates to this thread from now on.
>> Just a quick update to the KIP. As a part of
>> https://issues.apache.org/jira/browse/KAFKA-12332 and
>> https://github.com/apache/kafka/pull/10143, I'm proposing adding a new
>> error.
>> INCONSISTENT_TOPIC_ID will be returned on partitions in
>> LeaderAndIsrResponses where the topic ID in the request did not match the
>> topic ID in the log. This will only occur when a valid topic ID is provided
>> in the request.
>>
>> I've also updated the KIP to reflect this change.
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-LeaderAndIsrRequestv5
>>
>>
>> Please let me know if you have any thoughts or concerns with this change.
>>
>> Thanks,
>> Justine
>>
>> On Mon, Oct 19, 2020 at 8:50 AM Justine Olshan <jo...@confluent.io>
>> wrote:
>>
>>> Thanks everyone for the votes. KIP-516 has been accepted.
>>>
>>> Binding: Jun, Rajini, David
>>> Non-binding: Lucas, Satish, Tom
>>>
>>> Justine
>>>
>>> On Sat, Oct 17, 2020 at 3:22 AM Tom Bentley <tb...@redhat.com> wrote:
>>>
>>>> +1 non-binding. Thanks!
>>>>
>>>> On Sat, Oct 17, 2020 at 7:55 AM David Jacot <da...@gmail.com>
>>>> wrote:
>>>>
>>>> > Hi Justine,
>>>> >
>>>> > Thanks for the KIP! This is a great and long awaited improvement.
>>>> >
>>>> > +1 (binding)
>>>> >
>>>> > Best,
>>>> > David
>>>> >
>>>> > Le ven. 16 oct. 2020 à 17:36, Rajini Sivaram <ra...@gmail.com>
>>>> a
>>>> > écrit :
>>>> >
>>>> > > Hi Justine,
>>>> > >
>>>> > > +1 (binding)
>>>> > >
>>>> > > Thanks for all the work you put into this KIP!
>>>> > >
>>>> > > btw, there is a typo in the DeleteTopics Request/Response schema in
>>>> the
>>>> > > KIP, it says Metadata request.
>>>> > >
>>>> > > Regards,
>>>> > >
>>>> > > Rajini
>>>> > >
>>>> > >
>>>> > > On Fri, Oct 16, 2020 at 4:06 PM Satish Duggana <
>>>> satish.duggana@gmail.com
>>>> > >
>>>> > > wrote:
>>>> > >
>>>> > > > Hi Justine,
>>>> > > > Thanks for the KIP,  +1 (non-binding)
>>>> > > >
>>>> > > > On Thu, Oct 15, 2020 at 10:48 PM Lucas Bradstreet <
>>>> lucas@confluent.io>
>>>> > > > wrote:
>>>> > > > >
>>>> > > > > Hi Justine,
>>>> > > > >
>>>> > > > > +1 (non-binding). Thanks for all your hard work on this KIP!
>>>> > > > >
>>>> > > > > Lucas
>>>> > > > >
>>>> > > > > On Wed, Oct 14, 2020 at 8:59 AM Jun Rao <ju...@confluent.io>
>>>> wrote:
>>>> > > > >
>>>> > > > > > Hi, Justine,
>>>> > > > > >
>>>> > > > > > Thanks for the updated KIP. +1 from me.
>>>> > > > > >
>>>> > > > > > Jun
>>>> > > > > >
>>>> > > > > > On Tue, Oct 13, 2020 at 2:38 PM Jun Rao <ju...@confluent.io>
>>>> wrote:
>>>> > > > > >
>>>> > > > > > > Hi, Justine,
>>>> > > > > > >
>>>> > > > > > > Thanks for starting the vote. Just a few minor comments.
>>>> > > > > > >
>>>> > > > > > > 1. It seems that we should remove the topic field from the
>>>> > > > > > > StopReplicaResponse below?
>>>> > > > > > > StopReplica Response (Version: 4) => error_code [topics]
>>>> > > > > > >   error_code => INT16
>>>> > > > > > >     topics => topic topic_id* [partitions]
>>>> > > > > > >
>>>> > > > > > > 2. "After controller election, upon receiving the result,
>>>> assign
>>>> > > the
>>>> > > > > > > metadata topic its unique topic ID". Will the UUID for the
>>>> > metadata
>>>> > > > topic
>>>> > > > > > > be written to the metadata topic itself?
>>>> > > > > > >
>>>> > > > > > > 3. The vote request is designed to support multiple topics,
>>>> each
>>>> > of
>>>> > > > them
>>>> > > > > > > may require a different sentinel ID. Should we reserve more
>>>> than
>>>> > > one
>>>> > > > > > > sentinel ID for future usage?
>>>> > > > > > >
>>>> > > > > > > 4. UUID.randomUUID(): Could we clarify whether this method
>>>> > returns
>>>> > > > any
>>>> > > > > > > sentinel ID? Also, how do we expect the user to use it?
>>>> > > > > > >
>>>> > > > > > > Thanks,
>>>> > > > > > >
>>>> > > > > > > Jun
>>>> > > > > > >
>>>> > > > > > > On Mon, Oct 12, 2020 at 9:54 AM Justine Olshan <
>>>> > > jolshan@confluent.io
>>>> > > > >
>>>> > > > > > > wrote:
>>>> > > > > > >
>>>> > > > > > >> Hi all,
>>>> > > > > > >>
>>>> > > > > > >> After further discussion and changes to this KIP, I think
>>>> we are
>>>> > > > ready
>>>> > > > > > to
>>>> > > > > > >> restart this vote.
>>>> > > > > > >>
>>>> > > > > > >> Again, here is the KIP:
>>>> > > > > > >>
>>>> > > > > > >>
>>>> > > > > >
>>>> > > >
>>>> > >
>>>> >
>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
>>>> > > > > > >>
>>>> > > > > > >> The discussion thread is here:
>>>> > > > > > >>
>>>> > > > > > >>
>>>> > > > > >
>>>> > > >
>>>> > >
>>>> >
>>>> https://lists.apache.org/thread.html/7efa8cd169cadc7dc9cf86a7c0dbbab1836ddb5024d310fcebacf80c@%3Cdev.kafka.apache.org%3E
>>>> > > > > > >>
>>>> > > > > > >> Please take a look and vote if you have a chance.
>>>> > > > > > >>
>>>> > > > > > >> Thanks,
>>>> > > > > > >> Justine
>>>> > > > > > >>
>>>> > > > > > >> On Tue, Sep 22, 2020 at 8:52 AM Justine Olshan <
>>>> > > > jolshan@confluent.io>
>>>> > > > > > >> wrote:
>>>> > > > > > >>
>>>> > > > > > >> > Hi all,
>>>> > > > > > >> >
>>>> > > > > > >> > I'd like to call a vote on KIP-516: Topic Identifiers.
>>>> Here is
>>>> > > the
>>>> > > > > > KIP:
>>>> > > > > > >> >
>>>> > > > > > >> >
>>>> > > > > > >>
>>>> > > > > >
>>>> > > >
>>>> > >
>>>> >
>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
>>>> > > > > > >> >
>>>> > > > > > >> > The discussion thread is here:
>>>> > > > > > >> >
>>>> > > > > > >> >
>>>> > > > > > >>
>>>> > > > > >
>>>> > > >
>>>> > >
>>>> >
>>>> https://lists.apache.org/thread.html/7efa8cd169cadc7dc9cf86a7c0dbbab1836ddb5024d310fcebacf80c@%3Cdev.kafka.apache.org%3E
>>>> > > > > > >> >
>>>> > > > > > >> > Please take a look and vote if you have a chance.
>>>> > > > > > >> >
>>>> > > > > > >> > Thank you,
>>>> > > > > > >> > Justine
>>>> > > > > > >> >
>>>> > > > > > >>
>>>> > > > > > >
>>>> > > > > >
>>>> > > >
>>>> > >
>>>> >
>>>>
>>>