You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Grant Henke <gh...@cloudera.com> on 2016/06/24 03:32:04 UTC
[VOTE] KIP-4 Delete Topics Schema
I would like to initiate the voting process for the "KIP-4 Delete Topics
Schema changes". This is not a vote for all of KIP-4, but specifically for
the delete topics changes. I have included the exact changes below for
clarity:
>
> Delete Topics Request (KAFKA-2946
> <https://issues.apache.org/jira/browse/KAFKA-2946>)
>
> DeleteTopics Request (Version: 0) => [topics] timeout
> topics => STRING
> timeout => INT32
>
> DeleteTopicsRequest is a batch request to initiate topic deletion.
>
> Request semantics:
>
> 1. Must be sent to the controller broker
> 2. If there are multiple instructions for the same topic in one
> request the extra request will be ingnored
> - This is because the list of topics is modeled server side as a set
> - Multiple deletes results in the same end goal, so handling this
> error for the user should be okay
> 3. When requesting to delete a topic that does not exist, a an
> InvalidTopic error will be returned for that topic.
> 4. When requesting to delete a topic that is already marked for
> deletion, the request will wait up to the timeout until the delete is
> "complete" and return as usual.
> - This is to avoid errors due to concurrent delete requests. The
> end result is the same, the topic is deleted.
> 5. The principal must be authorized to the "Delete" Operation on the
> "Topic" resource to delete the topic.
> - Unauthorized requests will receive a TopicAuthorizationException
> if they are authorized to the "Describe" Operation on the "Topic" resource
> - Otherwise they will receive an InvalidTopicException as if the
> topic does not exist.
> 6. Setting a timeout > 0 will allow the request to block until the
> delete is "complete" on the controller node.
> - Complete means the local topic metadata cache no longer contains
> the topic
> - The topic metadata is updated when the controller sends out
> update metadata requests to the brokers
> - If a timeout error occurs, the topic could still be deleted
> successfully at a later time. Its up to the client to query for the state
> at that point.
> 7. Setting a timeout <= 0 will validate arguments and trigger the
> delete topics and return immediately.
> - This is essentially the fully asynchronous mode we have in the
> Zookeeper tools today.
> - The error code in the response will either contain an argument
> validation exception or a timeout exception. If you receive a timeout
> exception, because you asked for 0 timeout, you can assume the message was
> valid and the topic deletion was triggered.
> 8. The request is not transactional.
> 1. If an error occurs on one topic, the others could still be
> deleted.
> 2. Errors are reported independently.
>
> QA:
>
> - Why is DeleteTopicsRequest a batch request?
> - Scenarios where tools or admins want to delete many topics should
> be able to with fewer requests
> - Example: Removing all cluster topics
> - What happens if some topics error immediately? Will it
> return immediately?
> - The request will block until all topics have either been deleted,
> errors, or the timeout has been hit
> - There is no "short circuiting" where 1 error stops the other
> topics from being deleted
> - Why have a timeout at all? Deletes could take a while?
> - True some deletes may take a while or never finish, however some
> admin tools may want extended blocking regardless.
> - If you don't want any blocking setting a timeout of 0 works.
> - Future changes may make deletes much faster. See the Follow Up
> Changes
> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-follow-up-changes> section
> above.
> - Why implement "partial blocking" instead of fully async or fully
> consistent?
> - See Cluster Consistent Blocking
> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-cluster-consistent-blocking>
> below
> - Why require the request to go to the controller?
> - The controller is responsible for the cluster metadata and its
> propagation
> - See Request Forwarding
> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-request>
> below
>
> Delete Topics Response
>
> DeleteTopics Response (Version: 0) => [topic_error_codes]
> topic_error_codes => topic error_code
> topic => STRING
> error_code => INT16
>
> DeleteTopicsResponse contains a map between topic and topic creation
> result error code (see New Protocol Errors
> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-NewProtocolErrors>
> ).
>
> Response semantics:
>
> 1. When a request hits the timeout, the topics that are not "complete"
> will have the TimeoutException error code.
> - The topics that did complete successfully with have no error.
>
>
The KIP is available here for reference (linked to the Create Topics schema
section):
*https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-DeleteTopicsRequest(KAFKA-2946)
<https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-DeleteTopicsRequest(KAFKA-2946)>*
A sample patch is on github:
https://github.com/granthenke/kafka/tree/delete-wire-new
Note: This branch and patch is based on the CreateTopic request/response
PR. I will open a PR once that patch is complete.
Here is a link to the past discussion on the mailing list:
*http://search-hadoop.com/m/uyzND1fokOBn6uNd2&subj=+DISCUSS+KIP+4+Delete+Topic+Schema
<http://search-hadoop.com/m/uyzND1fokOBn6uNd2&subj=+DISCUSS+KIP+4+Delete+Topic+Schema>*
Thank you,
Grant
--
Grant Henke
Software Engineer | Cloudera
grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Re: [VOTE] KIP-4 Delete Topics Schema
Posted by Grant Henke <gh...@cloudera.com>.
Thanks to all who voted. The KIP-4 Delete Topics changes passed with +4
(binding), and +3 (non-binding).
There is a branch based off the create topics PR available here:
https://github.com/granthenke/kafka/tree/delete-wire-new
I will open a PR once the create topics patch is in:
https://github.com/apache/kafka/pull/1489
On Fri, Jun 24, 2016 at 5:43 PM, Gwen Shapira <gw...@confluent.io> wrote:
> +1
>
> On Thu, Jun 23, 2016 at 8:32 PM, Grant Henke <gh...@cloudera.com> wrote:
> > I would like to initiate the voting process for the "KIP-4 Delete Topics
> > Schema changes". This is not a vote for all of KIP-4, but specifically
> for
> > the delete topics changes. I have included the exact changes below for
> > clarity:
> >>
> >> Delete Topics Request (KAFKA-2946
> >> <https://issues.apache.org/jira/browse/KAFKA-2946>)
> >>
> >> DeleteTopics Request (Version: 0) => [topics] timeout
> >> topics => STRING
> >> timeout => INT32
> >>
> >> DeleteTopicsRequest is a batch request to initiate topic deletion.
> >>
> >> Request semantics:
> >>
> >> 1. Must be sent to the controller broker
> >> 2. If there are multiple instructions for the same topic in one
> >> request the extra request will be ingnored
> >> - This is because the list of topics is modeled server side as a
> set
> >> - Multiple deletes results in the same end goal, so handling this
> >> error for the user should be okay
> >> 3. When requesting to delete a topic that does not exist, a an
> >> InvalidTopic error will be returned for that topic.
> >> 4. When requesting to delete a topic that is already marked for
> >> deletion, the request will wait up to the timeout until the delete is
> >> "complete" and return as usual.
> >> - This is to avoid errors due to concurrent delete requests. The
> >> end result is the same, the topic is deleted.
> >> 5. The principal must be authorized to the "Delete" Operation on the
> >> "Topic" resource to delete the topic.
> >> - Unauthorized requests will receive a TopicAuthorizationException
> >> if they are authorized to the "Describe" Operation on the "Topic"
> resource
> >> - Otherwise they will receive an InvalidTopicException as if the
> >> topic does not exist.
> >> 6. Setting a timeout > 0 will allow the request to block until the
> >> delete is "complete" on the controller node.
> >> - Complete means the local topic metadata cache no longer contains
> >> the topic
> >> - The topic metadata is updated when the controller sends out
> >> update metadata requests to the brokers
> >> - If a timeout error occurs, the topic could still be deleted
> >> successfully at a later time. Its up to the client to query for
> the state
> >> at that point.
> >> 7. Setting a timeout <= 0 will validate arguments and trigger the
> >> delete topics and return immediately.
> >> - This is essentially the fully asynchronous mode we have in the
> >> Zookeeper tools today.
> >> - The error code in the response will either contain an argument
> >> validation exception or a timeout exception. If you receive a
> timeout
> >> exception, because you asked for 0 timeout, you can assume the
> message was
> >> valid and the topic deletion was triggered.
> >> 8. The request is not transactional.
> >> 1. If an error occurs on one topic, the others could still be
> >> deleted.
> >> 2. Errors are reported independently.
> >>
> >> QA:
> >>
> >> - Why is DeleteTopicsRequest a batch request?
> >> - Scenarios where tools or admins want to delete many topics
> should
> >> be able to with fewer requests
> >> - Example: Removing all cluster topics
> >> - What happens if some topics error immediately? Will it
> >> return immediately?
> >> - The request will block until all topics have either been
> deleted,
> >> errors, or the timeout has been hit
> >> - There is no "short circuiting" where 1 error stops the other
> >> topics from being deleted
> >> - Why have a timeout at all? Deletes could take a while?
> >> - True some deletes may take a while or never finish, however some
> >> admin tools may want extended blocking regardless.
> >> - If you don't want any blocking setting a timeout of 0 works.
> >> - Future changes may make deletes much faster. See the Follow Up
> >> Changes
> >> <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-follow-up-changes>
> section
> >> above.
> >> - Why implement "partial blocking" instead of fully async or fully
> >> consistent?
> >> - See Cluster Consistent Blocking
> >> <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-cluster-consistent-blocking
> >
> >> below
> >> - Why require the request to go to the controller?
> >> - The controller is responsible for the cluster metadata and its
> >> propagation
> >> - See Request Forwarding
> >> <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-request
> >
> >> below
> >>
> >> Delete Topics Response
> >>
> >> DeleteTopics Response (Version: 0) => [topic_error_codes]
> >> topic_error_codes => topic error_code
> >> topic => STRING
> >> error_code => INT16
> >>
> >> DeleteTopicsResponse contains a map between topic and topic creation
> >> result error code (see New Protocol Errors
> >> <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-NewProtocolErrors
> >
> >> ).
> >>
> >> Response semantics:
> >>
> >> 1. When a request hits the timeout, the topics that are not
> "complete"
> >> will have the TimeoutException error code.
> >> - The topics that did complete successfully with have no error.
> >>
> >>
> > The KIP is available here for reference (linked to the Create Topics
> schema
> > section):
> > *
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-DeleteTopicsRequest(KAFKA-2946)
> > <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-DeleteTopicsRequest(KAFKA-2946)
> >*
> >
> > A sample patch is on github:
> > https://github.com/granthenke/kafka/tree/delete-wire-new
> > Note: This branch and patch is based on the CreateTopic request/response
> > PR. I will open a PR once that patch is complete.
> >
> > Here is a link to the past discussion on the mailing list:
> >
> > *
> http://search-hadoop.com/m/uyzND1fokOBn6uNd2&subj=+DISCUSS+KIP+4+Delete+Topic+Schema
> > <
> http://search-hadoop.com/m/uyzND1fokOBn6uNd2&subj=+DISCUSS+KIP+4+Delete+Topic+Schema
> >*
> >
> > Thank you,
> > Grant
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
--
Grant Henke
Software Engineer | Cloudera
grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Re: [VOTE] KIP-4 Delete Topics Schema
Posted by Gwen Shapira <gw...@confluent.io>.
+1
On Thu, Jun 23, 2016 at 8:32 PM, Grant Henke <gh...@cloudera.com> wrote:
> I would like to initiate the voting process for the "KIP-4 Delete Topics
> Schema changes". This is not a vote for all of KIP-4, but specifically for
> the delete topics changes. I have included the exact changes below for
> clarity:
>>
>> Delete Topics Request (KAFKA-2946
>> <https://issues.apache.org/jira/browse/KAFKA-2946>)
>>
>> DeleteTopics Request (Version: 0) => [topics] timeout
>> topics => STRING
>> timeout => INT32
>>
>> DeleteTopicsRequest is a batch request to initiate topic deletion.
>>
>> Request semantics:
>>
>> 1. Must be sent to the controller broker
>> 2. If there are multiple instructions for the same topic in one
>> request the extra request will be ingnored
>> - This is because the list of topics is modeled server side as a set
>> - Multiple deletes results in the same end goal, so handling this
>> error for the user should be okay
>> 3. When requesting to delete a topic that does not exist, a an
>> InvalidTopic error will be returned for that topic.
>> 4. When requesting to delete a topic that is already marked for
>> deletion, the request will wait up to the timeout until the delete is
>> "complete" and return as usual.
>> - This is to avoid errors due to concurrent delete requests. The
>> end result is the same, the topic is deleted.
>> 5. The principal must be authorized to the "Delete" Operation on the
>> "Topic" resource to delete the topic.
>> - Unauthorized requests will receive a TopicAuthorizationException
>> if they are authorized to the "Describe" Operation on the "Topic" resource
>> - Otherwise they will receive an InvalidTopicException as if the
>> topic does not exist.
>> 6. Setting a timeout > 0 will allow the request to block until the
>> delete is "complete" on the controller node.
>> - Complete means the local topic metadata cache no longer contains
>> the topic
>> - The topic metadata is updated when the controller sends out
>> update metadata requests to the brokers
>> - If a timeout error occurs, the topic could still be deleted
>> successfully at a later time. Its up to the client to query for the state
>> at that point.
>> 7. Setting a timeout <= 0 will validate arguments and trigger the
>> delete topics and return immediately.
>> - This is essentially the fully asynchronous mode we have in the
>> Zookeeper tools today.
>> - The error code in the response will either contain an argument
>> validation exception or a timeout exception. If you receive a timeout
>> exception, because you asked for 0 timeout, you can assume the message was
>> valid and the topic deletion was triggered.
>> 8. The request is not transactional.
>> 1. If an error occurs on one topic, the others could still be
>> deleted.
>> 2. Errors are reported independently.
>>
>> QA:
>>
>> - Why is DeleteTopicsRequest a batch request?
>> - Scenarios where tools or admins want to delete many topics should
>> be able to with fewer requests
>> - Example: Removing all cluster topics
>> - What happens if some topics error immediately? Will it
>> return immediately?
>> - The request will block until all topics have either been deleted,
>> errors, or the timeout has been hit
>> - There is no "short circuiting" where 1 error stops the other
>> topics from being deleted
>> - Why have a timeout at all? Deletes could take a while?
>> - True some deletes may take a while or never finish, however some
>> admin tools may want extended blocking regardless.
>> - If you don't want any blocking setting a timeout of 0 works.
>> - Future changes may make deletes much faster. See the Follow Up
>> Changes
>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-follow-up-changes> section
>> above.
>> - Why implement "partial blocking" instead of fully async or fully
>> consistent?
>> - See Cluster Consistent Blocking
>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-cluster-consistent-blocking>
>> below
>> - Why require the request to go to the controller?
>> - The controller is responsible for the cluster metadata and its
>> propagation
>> - See Request Forwarding
>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-request>
>> below
>>
>> Delete Topics Response
>>
>> DeleteTopics Response (Version: 0) => [topic_error_codes]
>> topic_error_codes => topic error_code
>> topic => STRING
>> error_code => INT16
>>
>> DeleteTopicsResponse contains a map between topic and topic creation
>> result error code (see New Protocol Errors
>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-NewProtocolErrors>
>> ).
>>
>> Response semantics:
>>
>> 1. When a request hits the timeout, the topics that are not "complete"
>> will have the TimeoutException error code.
>> - The topics that did complete successfully with have no error.
>>
>>
> The KIP is available here for reference (linked to the Create Topics schema
> section):
> *https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-DeleteTopicsRequest(KAFKA-2946)
> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-DeleteTopicsRequest(KAFKA-2946)>*
>
> A sample patch is on github:
> https://github.com/granthenke/kafka/tree/delete-wire-new
> Note: This branch and patch is based on the CreateTopic request/response
> PR. I will open a PR once that patch is complete.
>
> Here is a link to the past discussion on the mailing list:
>
> *http://search-hadoop.com/m/uyzND1fokOBn6uNd2&subj=+DISCUSS+KIP+4+Delete+Topic+Schema
> <http://search-hadoop.com/m/uyzND1fokOBn6uNd2&subj=+DISCUSS+KIP+4+Delete+Topic+Schema>*
>
> Thank you,
> Grant
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Re: [VOTE] KIP-4 Delete Topics Schema
Posted by Ismael Juma <is...@juma.me.uk>.
+1 (binding)
On Fri, Jun 24, 2016 at 7:16 PM, Onur Karaman <okaraman@linkedin.com.invalid
> wrote:
> +1
>
> On Fri, Jun 24, 2016 at 9:52 AM, Jay Kreps <ja...@confluent.io> wrote:
>
> > +1
> >
> > On Thu, Jun 23, 2016 at 8:32 PM, Grant Henke <gh...@cloudera.com>
> wrote:
> >
> > > I would like to initiate the voting process for the "KIP-4 Delete
> Topics
> > > Schema changes". This is not a vote for all of KIP-4, but specifically
> > for
> > > the delete topics changes. I have included the exact changes below for
> > > clarity:
> > > >
> > > > Delete Topics Request (KAFKA-2946
> > > > <https://issues.apache.org/jira/browse/KAFKA-2946>)
> > > >
> > > > DeleteTopics Request (Version: 0) => [topics] timeout
> > > > topics => STRING
> > > > timeout => INT32
> > > >
> > > > DeleteTopicsRequest is a batch request to initiate topic deletion.
> > > >
> > > > Request semantics:
> > > >
> > > > 1. Must be sent to the controller broker
> > > > 2. If there are multiple instructions for the same topic in one
> > > > request the extra request will be ingnored
> > > > - This is because the list of topics is modeled server side as
> a
> > > set
> > > > - Multiple deletes results in the same end goal, so handling
> this
> > > > error for the user should be okay
> > > > 3. When requesting to delete a topic that does not exist, a an
> > > > InvalidTopic error will be returned for that topic.
> > > > 4. When requesting to delete a topic that is already marked for
> > > > deletion, the request will wait up to the timeout until the delete
> > is
> > > > "complete" and return as usual.
> > > > - This is to avoid errors due to concurrent delete requests.
> The
> > > > end result is the same, the topic is deleted.
> > > > 5. The principal must be authorized to the "Delete" Operation on
> the
> > > > "Topic" resource to delete the topic.
> > > > - Unauthorized requests will receive a
> > TopicAuthorizationException
> > > > if they are authorized to the "Describe" Operation on the
> "Topic"
> > > resource
> > > > - Otherwise they will receive an InvalidTopicException as if
> the
> > > > topic does not exist.
> > > > 6. Setting a timeout > 0 will allow the request to block until
> > the
> > > > delete is "complete" on the controller node.
> > > > - Complete means the local topic metadata cache no longer
> > contains
> > > > the topic
> > > > - The topic metadata is updated when the controller sends
> out
> > > > update metadata requests to the brokers
> > > > - If a timeout error occurs, the topic could still be deleted
> > > > successfully at a later time. Its up to the client to query for
> > > the state
> > > > at that point.
> > > > 7. Setting a timeout <= 0 will validate arguments and trigger the
> > > > delete topics and return immediately.
> > > > - This is essentially the fully asynchronous mode we have in
> the
> > > > Zookeeper tools today.
> > > > - The error code in the response will either contain an
> argument
> > > > validation exception or a timeout exception. If you receive a
> > > timeout
> > > > exception, because you asked for 0 timeout, you can assume the
> > > message was
> > > > valid and the topic deletion was triggered.
> > > > 8. The request is not transactional.
> > > > 1. If an error occurs on one topic, the others could still be
> > > > deleted.
> > > > 2. Errors are reported independently.
> > > >
> > > > QA:
> > > >
> > > > - Why is DeleteTopicsRequest a batch request?
> > > > - Scenarios where tools or admins want to delete many topics
> > should
> > > > be able to with fewer requests
> > > > - Example: Removing all cluster topics
> > > > - What happens if some topics error immediately? Will it
> > > > return immediately?
> > > > - The request will block until all topics have either been
> > deleted,
> > > > errors, or the timeout has been hit
> > > > - There is no "short circuiting" where 1 error stops the other
> > > > topics from being deleted
> > > > - Why have a timeout at all? Deletes could take a while?
> > > > - True some deletes may take a while or never finish, however
> > some
> > > > admin tools may want extended blocking regardless.
> > > > - If you don't want any blocking setting a timeout of 0 works.
> > > > - Future changes may make deletes much faster. See the Follow
> Up
> > > > Changes
> > > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-follow-up-changes
> > >
> > > section
> > > > above.
> > > > - Why implement "partial blocking" instead of fully async or fully
> > > > consistent?
> > > > - See Cluster Consistent Blocking
> > > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-cluster-consistent-blocking
> > > >
> > > > below
> > > > - Why require the request to go to the controller?
> > > > - The controller is responsible for the cluster metadata and
> its
> > > > propagation
> > > > - See Request Forwarding
> > > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-request
> > > >
> > > > below
> > > >
> > > > Delete Topics Response
> > > >
> > > > DeleteTopics Response (Version: 0) => [topic_error_codes]
> > > > topic_error_codes => topic error_code
> > > > topic => STRING
> > > > error_code => INT16
> > > >
> > > > DeleteTopicsResponse contains a map between topic and topic creation
> > > > result error code (see New Protocol Errors
> > > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-NewProtocolErrors
> > > >
> > > > ).
> > > >
> > > > Response semantics:
> > > >
> > > > 1. When a request hits the timeout, the topics that are not
> > "complete"
> > > > will have the TimeoutException error code.
> > > > - The topics that did complete successfully with have no error.
> > > >
> > > >
> > > The KIP is available here for reference (linked to the Create Topics
> > schema
> > > section):
> > > *
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-DeleteTopicsRequest(KAFKA-2946)
> > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-DeleteTopicsRequest(KAFKA-2946)
> > > >*
> > >
> > > A sample patch is on github:
> > > https://github.com/granthenke/kafka/tree/delete-wire-new
> > > Note: This branch and patch is based on the CreateTopic
> request/response
> > > PR. I will open a PR once that patch is complete.
> > >
> > > Here is a link to the past discussion on the mailing list:
> > >
> > > *
> > >
> >
> http://search-hadoop.com/m/uyzND1fokOBn6uNd2&subj=+DISCUSS+KIP+4+Delete+Topic+Schema
> > > <
> > >
> >
> http://search-hadoop.com/m/uyzND1fokOBn6uNd2&subj=+DISCUSS+KIP+4+Delete+Topic+Schema
> > > >*
> > >
> > > Thank you,
> > > Grant
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> > >
> >
>
Re: [VOTE] KIP-4 Delete Topics Schema
Posted by Onur Karaman <ok...@linkedin.com.INVALID>.
+1
On Fri, Jun 24, 2016 at 9:52 AM, Jay Kreps <ja...@confluent.io> wrote:
> +1
>
> On Thu, Jun 23, 2016 at 8:32 PM, Grant Henke <gh...@cloudera.com> wrote:
>
> > I would like to initiate the voting process for the "KIP-4 Delete Topics
> > Schema changes". This is not a vote for all of KIP-4, but specifically
> for
> > the delete topics changes. I have included the exact changes below for
> > clarity:
> > >
> > > Delete Topics Request (KAFKA-2946
> > > <https://issues.apache.org/jira/browse/KAFKA-2946>)
> > >
> > > DeleteTopics Request (Version: 0) => [topics] timeout
> > > topics => STRING
> > > timeout => INT32
> > >
> > > DeleteTopicsRequest is a batch request to initiate topic deletion.
> > >
> > > Request semantics:
> > >
> > > 1. Must be sent to the controller broker
> > > 2. If there are multiple instructions for the same topic in one
> > > request the extra request will be ingnored
> > > - This is because the list of topics is modeled server side as a
> > set
> > > - Multiple deletes results in the same end goal, so handling this
> > > error for the user should be okay
> > > 3. When requesting to delete a topic that does not exist, a an
> > > InvalidTopic error will be returned for that topic.
> > > 4. When requesting to delete a topic that is already marked for
> > > deletion, the request will wait up to the timeout until the delete
> is
> > > "complete" and return as usual.
> > > - This is to avoid errors due to concurrent delete requests. The
> > > end result is the same, the topic is deleted.
> > > 5. The principal must be authorized to the "Delete" Operation on the
> > > "Topic" resource to delete the topic.
> > > - Unauthorized requests will receive a
> TopicAuthorizationException
> > > if they are authorized to the "Describe" Operation on the "Topic"
> > resource
> > > - Otherwise they will receive an InvalidTopicException as if the
> > > topic does not exist.
> > > 6. Setting a timeout > 0 will allow the request to block until
> the
> > > delete is "complete" on the controller node.
> > > - Complete means the local topic metadata cache no longer
> contains
> > > the topic
> > > - The topic metadata is updated when the controller sends out
> > > update metadata requests to the brokers
> > > - If a timeout error occurs, the topic could still be deleted
> > > successfully at a later time. Its up to the client to query for
> > the state
> > > at that point.
> > > 7. Setting a timeout <= 0 will validate arguments and trigger the
> > > delete topics and return immediately.
> > > - This is essentially the fully asynchronous mode we have in the
> > > Zookeeper tools today.
> > > - The error code in the response will either contain an argument
> > > validation exception or a timeout exception. If you receive a
> > timeout
> > > exception, because you asked for 0 timeout, you can assume the
> > message was
> > > valid and the topic deletion was triggered.
> > > 8. The request is not transactional.
> > > 1. If an error occurs on one topic, the others could still be
> > > deleted.
> > > 2. Errors are reported independently.
> > >
> > > QA:
> > >
> > > - Why is DeleteTopicsRequest a batch request?
> > > - Scenarios where tools or admins want to delete many topics
> should
> > > be able to with fewer requests
> > > - Example: Removing all cluster topics
> > > - What happens if some topics error immediately? Will it
> > > return immediately?
> > > - The request will block until all topics have either been
> deleted,
> > > errors, or the timeout has been hit
> > > - There is no "short circuiting" where 1 error stops the other
> > > topics from being deleted
> > > - Why have a timeout at all? Deletes could take a while?
> > > - True some deletes may take a while or never finish, however
> some
> > > admin tools may want extended blocking regardless.
> > > - If you don't want any blocking setting a timeout of 0 works.
> > > - Future changes may make deletes much faster. See the Follow Up
> > > Changes
> > > <
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-follow-up-changes
> >
> > section
> > > above.
> > > - Why implement "partial blocking" instead of fully async or fully
> > > consistent?
> > > - See Cluster Consistent Blocking
> > > <
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-cluster-consistent-blocking
> > >
> > > below
> > > - Why require the request to go to the controller?
> > > - The controller is responsible for the cluster metadata and its
> > > propagation
> > > - See Request Forwarding
> > > <
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-request
> > >
> > > below
> > >
> > > Delete Topics Response
> > >
> > > DeleteTopics Response (Version: 0) => [topic_error_codes]
> > > topic_error_codes => topic error_code
> > > topic => STRING
> > > error_code => INT16
> > >
> > > DeleteTopicsResponse contains a map between topic and topic creation
> > > result error code (see New Protocol Errors
> > > <
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-NewProtocolErrors
> > >
> > > ).
> > >
> > > Response semantics:
> > >
> > > 1. When a request hits the timeout, the topics that are not
> "complete"
> > > will have the TimeoutException error code.
> > > - The topics that did complete successfully with have no error.
> > >
> > >
> > The KIP is available here for reference (linked to the Create Topics
> schema
> > section):
> > *
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-DeleteTopicsRequest(KAFKA-2946)
> > <
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-DeleteTopicsRequest(KAFKA-2946)
> > >*
> >
> > A sample patch is on github:
> > https://github.com/granthenke/kafka/tree/delete-wire-new
> > Note: This branch and patch is based on the CreateTopic request/response
> > PR. I will open a PR once that patch is complete.
> >
> > Here is a link to the past discussion on the mailing list:
> >
> > *
> >
> http://search-hadoop.com/m/uyzND1fokOBn6uNd2&subj=+DISCUSS+KIP+4+Delete+Topic+Schema
> > <
> >
> http://search-hadoop.com/m/uyzND1fokOBn6uNd2&subj=+DISCUSS+KIP+4+Delete+Topic+Schema
> > >*
> >
> > Thank you,
> > Grant
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>
Re: [VOTE] KIP-4 Delete Topics Schema
Posted by Jay Kreps <ja...@confluent.io>.
+1
On Thu, Jun 23, 2016 at 8:32 PM, Grant Henke <gh...@cloudera.com> wrote:
> I would like to initiate the voting process for the "KIP-4 Delete Topics
> Schema changes". This is not a vote for all of KIP-4, but specifically for
> the delete topics changes. I have included the exact changes below for
> clarity:
> >
> > Delete Topics Request (KAFKA-2946
> > <https://issues.apache.org/jira/browse/KAFKA-2946>)
> >
> > DeleteTopics Request (Version: 0) => [topics] timeout
> > topics => STRING
> > timeout => INT32
> >
> > DeleteTopicsRequest is a batch request to initiate topic deletion.
> >
> > Request semantics:
> >
> > 1. Must be sent to the controller broker
> > 2. If there are multiple instructions for the same topic in one
> > request the extra request will be ingnored
> > - This is because the list of topics is modeled server side as a
> set
> > - Multiple deletes results in the same end goal, so handling this
> > error for the user should be okay
> > 3. When requesting to delete a topic that does not exist, a an
> > InvalidTopic error will be returned for that topic.
> > 4. When requesting to delete a topic that is already marked for
> > deletion, the request will wait up to the timeout until the delete is
> > "complete" and return as usual.
> > - This is to avoid errors due to concurrent delete requests. The
> > end result is the same, the topic is deleted.
> > 5. The principal must be authorized to the "Delete" Operation on the
> > "Topic" resource to delete the topic.
> > - Unauthorized requests will receive a TopicAuthorizationException
> > if they are authorized to the "Describe" Operation on the "Topic"
> resource
> > - Otherwise they will receive an InvalidTopicException as if the
> > topic does not exist.
> > 6. Setting a timeout > 0 will allow the request to block until the
> > delete is "complete" on the controller node.
> > - Complete means the local topic metadata cache no longer contains
> > the topic
> > - The topic metadata is updated when the controller sends out
> > update metadata requests to the brokers
> > - If a timeout error occurs, the topic could still be deleted
> > successfully at a later time. Its up to the client to query for
> the state
> > at that point.
> > 7. Setting a timeout <= 0 will validate arguments and trigger the
> > delete topics and return immediately.
> > - This is essentially the fully asynchronous mode we have in the
> > Zookeeper tools today.
> > - The error code in the response will either contain an argument
> > validation exception or a timeout exception. If you receive a
> timeout
> > exception, because you asked for 0 timeout, you can assume the
> message was
> > valid and the topic deletion was triggered.
> > 8. The request is not transactional.
> > 1. If an error occurs on one topic, the others could still be
> > deleted.
> > 2. Errors are reported independently.
> >
> > QA:
> >
> > - Why is DeleteTopicsRequest a batch request?
> > - Scenarios where tools or admins want to delete many topics should
> > be able to with fewer requests
> > - Example: Removing all cluster topics
> > - What happens if some topics error immediately? Will it
> > return immediately?
> > - The request will block until all topics have either been deleted,
> > errors, or the timeout has been hit
> > - There is no "short circuiting" where 1 error stops the other
> > topics from being deleted
> > - Why have a timeout at all? Deletes could take a while?
> > - True some deletes may take a while or never finish, however some
> > admin tools may want extended blocking regardless.
> > - If you don't want any blocking setting a timeout of 0 works.
> > - Future changes may make deletes much faster. See the Follow Up
> > Changes
> > <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-follow-up-changes>
> section
> > above.
> > - Why implement "partial blocking" instead of fully async or fully
> > consistent?
> > - See Cluster Consistent Blocking
> > <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-cluster-consistent-blocking
> >
> > below
> > - Why require the request to go to the controller?
> > - The controller is responsible for the cluster metadata and its
> > propagation
> > - See Request Forwarding
> > <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-request
> >
> > below
> >
> > Delete Topics Response
> >
> > DeleteTopics Response (Version: 0) => [topic_error_codes]
> > topic_error_codes => topic error_code
> > topic => STRING
> > error_code => INT16
> >
> > DeleteTopicsResponse contains a map between topic and topic creation
> > result error code (see New Protocol Errors
> > <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-NewProtocolErrors
> >
> > ).
> >
> > Response semantics:
> >
> > 1. When a request hits the timeout, the topics that are not "complete"
> > will have the TimeoutException error code.
> > - The topics that did complete successfully with have no error.
> >
> >
> The KIP is available here for reference (linked to the Create Topics schema
> section):
> *
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-DeleteTopicsRequest(KAFKA-2946)
> <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-DeleteTopicsRequest(KAFKA-2946)
> >*
>
> A sample patch is on github:
> https://github.com/granthenke/kafka/tree/delete-wire-new
> Note: This branch and patch is based on the CreateTopic request/response
> PR. I will open a PR once that patch is complete.
>
> Here is a link to the past discussion on the mailing list:
>
> *
> http://search-hadoop.com/m/uyzND1fokOBn6uNd2&subj=+DISCUSS+KIP+4+Delete+Topic+Schema
> <
> http://search-hadoop.com/m/uyzND1fokOBn6uNd2&subj=+DISCUSS+KIP+4+Delete+Topic+Schema
> >*
>
> Thank you,
> Grant
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
Re: [VOTE] KIP-4 Delete Topics Schema
Posted by Dana Powers <da...@gmail.com>.
+1
Re: [VOTE] KIP-4 Delete Topics Schema
Posted by Guozhang Wang <wa...@gmail.com>.
+1
On Thu, Jun 23, 2016 at 8:32 PM, Grant Henke <gh...@cloudera.com> wrote:
> I would like to initiate the voting process for the "KIP-4 Delete Topics
> Schema changes". This is not a vote for all of KIP-4, but specifically for
> the delete topics changes. I have included the exact changes below for
> clarity:
> >
> > Delete Topics Request (KAFKA-2946
> > <https://issues.apache.org/jira/browse/KAFKA-2946>)
> >
> > DeleteTopics Request (Version: 0) => [topics] timeout
> > topics => STRING
> > timeout => INT32
> >
> > DeleteTopicsRequest is a batch request to initiate topic deletion.
> >
> > Request semantics:
> >
> > 1. Must be sent to the controller broker
> > 2. If there are multiple instructions for the same topic in one
> > request the extra request will be ingnored
> > - This is because the list of topics is modeled server side as a
> set
> > - Multiple deletes results in the same end goal, so handling this
> > error for the user should be okay
> > 3. When requesting to delete a topic that does not exist, a an
> > InvalidTopic error will be returned for that topic.
> > 4. When requesting to delete a topic that is already marked for
> > deletion, the request will wait up to the timeout until the delete is
> > "complete" and return as usual.
> > - This is to avoid errors due to concurrent delete requests. The
> > end result is the same, the topic is deleted.
> > 5. The principal must be authorized to the "Delete" Operation on the
> > "Topic" resource to delete the topic.
> > - Unauthorized requests will receive a TopicAuthorizationException
> > if they are authorized to the "Describe" Operation on the "Topic"
> resource
> > - Otherwise they will receive an InvalidTopicException as if the
> > topic does not exist.
> > 6. Setting a timeout > 0 will allow the request to block until the
> > delete is "complete" on the controller node.
> > - Complete means the local topic metadata cache no longer contains
> > the topic
> > - The topic metadata is updated when the controller sends out
> > update metadata requests to the brokers
> > - If a timeout error occurs, the topic could still be deleted
> > successfully at a later time. Its up to the client to query for
> the state
> > at that point.
> > 7. Setting a timeout <= 0 will validate arguments and trigger the
> > delete topics and return immediately.
> > - This is essentially the fully asynchronous mode we have in the
> > Zookeeper tools today.
> > - The error code in the response will either contain an argument
> > validation exception or a timeout exception. If you receive a
> timeout
> > exception, because you asked for 0 timeout, you can assume the
> message was
> > valid and the topic deletion was triggered.
> > 8. The request is not transactional.
> > 1. If an error occurs on one topic, the others could still be
> > deleted.
> > 2. Errors are reported independently.
> >
> > QA:
> >
> > - Why is DeleteTopicsRequest a batch request?
> > - Scenarios where tools or admins want to delete many topics should
> > be able to with fewer requests
> > - Example: Removing all cluster topics
> > - What happens if some topics error immediately? Will it
> > return immediately?
> > - The request will block until all topics have either been deleted,
> > errors, or the timeout has been hit
> > - There is no "short circuiting" where 1 error stops the other
> > topics from being deleted
> > - Why have a timeout at all? Deletes could take a while?
> > - True some deletes may take a while or never finish, however some
> > admin tools may want extended blocking regardless.
> > - If you don't want any blocking setting a timeout of 0 works.
> > - Future changes may make deletes much faster. See the Follow Up
> > Changes
> > <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-follow-up-changes>
> section
> > above.
> > - Why implement "partial blocking" instead of fully async or fully
> > consistent?
> > - See Cluster Consistent Blocking
> > <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-cluster-consistent-blocking
> >
> > below
> > - Why require the request to go to the controller?
> > - The controller is responsible for the cluster metadata and its
> > propagation
> > - See Request Forwarding
> > <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-request
> >
> > below
> >
> > Delete Topics Response
> >
> > DeleteTopics Response (Version: 0) => [topic_error_codes]
> > topic_error_codes => topic error_code
> > topic => STRING
> > error_code => INT16
> >
> > DeleteTopicsResponse contains a map between topic and topic creation
> > result error code (see New Protocol Errors
> > <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-NewProtocolErrors
> >
> > ).
> >
> > Response semantics:
> >
> > 1. When a request hits the timeout, the topics that are not "complete"
> > will have the TimeoutException error code.
> > - The topics that did complete successfully with have no error.
> >
> >
> The KIP is available here for reference (linked to the Create Topics schema
> section):
> *
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-DeleteTopicsRequest(KAFKA-2946)
> <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-DeleteTopicsRequest(KAFKA-2946)
> >*
>
> A sample patch is on github:
> https://github.com/granthenke/kafka/tree/delete-wire-new
> Note: This branch and patch is based on the CreateTopic request/response
> PR. I will open a PR once that patch is complete.
>
> Here is a link to the past discussion on the mailing list:
>
> *
> http://search-hadoop.com/m/uyzND1fokOBn6uNd2&subj=+DISCUSS+KIP+4+Delete+Topic+Schema
> <
> http://search-hadoop.com/m/uyzND1fokOBn6uNd2&subj=+DISCUSS+KIP+4+Delete+Topic+Schema
> >*
>
> Thank you,
> Grant
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
--
-- Guozhang