You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by David Jacot <dj...@confluent.io> on 2019/09/04 07:24:54 UTC
Re: [VOTE] KIP-496: Administrative API to delete consumer offsets
Hi all,
While implementing the KIP, I have realized that a new error code and
exception is required to notify the caller that offsets of a topic can not
be deleted because the group is actively subscribed to the topic.
I would like to know if there are any concerns with these changes before
updating the KIP.
*Proposed API:*
GROUP_SUBSCRIBED_TO_TOPIC(86, "The consumer group is actively subscribed to
the topic", GroupSubscribedToTopicException::new);
public class GroupSubscribedToTopicException extends ApiException {
public GroupSubscribedToTopicException(String message) {
super(message);
}
}
Best,
David
On Fri, Aug 16, 2019 at 10:58 AM Mickael Maison <mi...@gmail.com>
wrote:
> +1 (non binding)
> Thanks!
>
> On Thu, Aug 15, 2019 at 11:53 PM Colin McCabe <cm...@apache.org> wrote:
> >
> > On Thu, Aug 15, 2019, at 11:47, Jason Gustafson wrote:
> > > Hey Colin, I think deleting all offsets is equivalent to deleting the
> > > group, which can be done with the `deleteConsumerGroups` api. I debated
> > > whether there should be a way to delete partitions for all unsubscribed
> > > topics, but I decided to start with a simple API.
> >
> > That's a fair point-- deleting the group covers the main use-case for
> deleting all offsets. So we might as well keep it simple for now.
> >
> > cheers,
> > Colin
> >
> > >
> > > I'm going to close this vote. The final result is +3 with myself,
> Guozhang,
> > > and Colin voting.
> > >
> > > -Jason
> > >
> > > On Tue, Aug 13, 2019 at 9:21 AM Colin McCabe <cm...@apache.org>
> wrote:
> > >
> > > > Hi Jason,
> > > >
> > > > Thanks for the KIP.
> > > >
> > > > Is there ever a desire to delete all the offsets for a given group?
> > > > Should the protocol and tools support this?
> > > >
> > > > +1 (binding)
> > > >
> > > > best,
> > > > Colin
> > > >
> > > >
> > > > On Mon, Aug 12, 2019, at 10:57, Guozhang Wang wrote:
> > > > > +1 (binding).
> > > > >
> > > > > Thanks Jason!
> > > > >
> > > > > On Wed, Aug 7, 2019 at 11:18 AM Jason Gustafson <
> jason@confluent.io>
> > > > wrote:
> > > > >
> > > > > > Hi All,
> > > > > >
> > > > > > I'd like to start a vote on KIP-496:
> > > > > >
> > > > > >
> > > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-496%3A+Administrative+API+to+delete+consumer+offsets
> > > > > > .
> > > > > > +1
> > > > > > from me of course.
> > > > > >
> > > > > > -Jason
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > -- Guozhang
> > > > >
> > > >
> > >
>
Re: [VOTE] KIP-496: Administrative API to delete consumer offsets
Posted by David Jacot <dj...@confluent.io>.
Great. I have updated the KIP.
Thanks,
David
On Thu, Sep 19, 2019 at 10:56 PM Colin McCabe <cm...@apache.org> wrote:
> Sounds good to me. It makes sense to add this functionality to the
> command line.
>
> best,
> Colin
>
> On Wed, Sep 18, 2019, at 11:26, David Jacot wrote:
> > Indeed, I have forgotten to add the action. There will be a new action «
> > —delete-offsets ». Sorry!
> >
> > *Proposed API*
> > kafka-consumer-groups.sh --bootstrap-server <bootstrap server>
> > —delete-offsets --group <group id> --topic <topic-name>:<partition
> numbers>
> > ex: --bootstrap-server localhost:9092 --group my-group --topic
> > topic1 --topic topic2:0,1,2
> >
> > When partitions is not provided, all partitions are used.
> >
> > Best,
> > David
> >
> > Le mer. 18 sept. 2019 à 20:09, Colin McCabe <cm...@apache.org> a
> écrit :
> >
> > > On Tue, Sep 17, 2019, at 09:07, David Jacot wrote:
> > > > Hi all,
> > > >
> > > > We haven't included the changes in the command line tool to support
> the
> > > new
> > > > API. Therefore,
> > > > I would like to amend the current KIP to cover the changes in the
> > > > `kafka-consumer-groups`
> > > > command line tool. The change is rather small and it does not need
> to add
> > > > any new arguments
> > > > to the command line tool. so it doesn't make sense to create a new
> KIP
> > > for
> > > > it.
> > > >
> > > > *Proposed API*
> > > > kafka-consumer-groups.sh --bootstrap-server <bootstrap server>
> --group
> > > > <group id> --topic <topic-name>:<partition numbers>
> > > > ex: --bootstrap-server localhost:9092 --group my-group --topic topic1
> > > > --topic topic2:0,1,2
> > > >
> > > > When partitions not provided, all partitions are used.
> > >
> > > Hmm. I think I'm missing something here. If you try the command you
> > > specified, you get:
> > >
> > > > Command must include exactly one action: --list, --describe,
> --delete,
> > > --reset-offsets
> > >
> > > Did you mean to add a new action here that was offsets-related?
> > >
> > > best,
> > > Colin
> > >
> > > >
> > > > What do you think?
> > > >
> > > > Best,
> > > > David
> > > >
> > > >
> > > > On Fri, Sep 13, 2019 at 6:42 PM Colin McCabe <cm...@apache.org>
> wrote:
> > > >
> > > > > Hi David,
> > > > >
> > > > > Sounds good.
> > > > >
> > > > > best,
> > > > > Colin
> > > > >
> > > > >
> > > > > On Fri, Sep 13, 2019, at 04:45, David Jacot wrote:
> > > > > > Hi all,
> > > > > >
> > > > > > I would like to do another modification to the proposal. In the
> > > proposal,
> > > > > > the OffsetDeleteResponse
> > > > > > doesn't have a top level error field so I would like to add one.
> Many
> > > > > > errors concern the whole
> > > > > > group (e.g. GROUP_ID_NOT_FOUND) so it would be great to have a
> way to
> > > > > > communicate them
> > > > > > back to the client without having to set such errors for all the
> > > > > requested
> > > > > > partitions. It makes the
> > > > > > error handling on the client easier and cleaner.
> > > > > >
> > > > > > *Proposed API with the ErrorCode:*
> > > > > > {
> > > > > > "apiKey": 47,
> > > > > > "type": "response",
> > > > > > "name": "OffsetDeleteResponse",
> > > > > > "validVersions": "0",
> > > > > > "fields": [
> > > > > > { "name": "ErrorCode", "type": "int16", "versions": "0+",
> > > > > > "about": "The top-level error code, or 0 if there was no
> > > error." },
> > > > > > { "name": "ThrottleTimeMs", "type": "int32", "versions":
> "0+",
> > > > > > "ignorable": true,
> > > > > > "about": "The duration in milliseconds for which the
> request
> > > was
> > > > > > throttled due to a quota violation, or zero if the request did
> not
> > > > > violate
> > > > > > any quota." },
> > > > > > { "name": "Topics", "type": "[]OffsetDeleteResponseTopic",
> > > > > "versions":
> > > > > > "0+",
> > > > > > "about": "The responses for each topic.", "fields": [
> > > > > > { "name": "Name", "type": "string", "versions": "0+",
> > > "mapKey":
> > > > > > true,
> > > > > > "about": "The topic name." },
> > > > > > { "name": "Partitions", "type":
> > > > > "[]OffsetDeleteResponsePartition",
> > > > > > "versions": "0+",
> > > > > > "about": "The responses for each partition in the
> topic.",
> > > > > > "fields": [
> > > > > > { "name": "PartitionIndex", "type": "int32",
> "versions":
> > > > > "0+",
> > > > > > "mapKey": true,
> > > > > > "about": "The partition index." },
> > > > > > { "name": "ErrorCode", "type": "int16", "versions":
> "0+",
> > > > > > "about": "The error code, or 0 if there was no
> error."
> > > }
> > > > > > ]
> > > > > > }
> > > > > > ]
> > > > > > }
> > > > > > ]
> > > > > > }
> > > > > >
> > > > > > I would like to know if there are any concerns or objections
> > > regarding
> > > > > this
> > > > > > change before updating the KIP.
> > > > > >
> > > > > > Best,
> > > > > > David
> > > > > >
> > > > > > On Wed, Sep 4, 2019 at 9:24 AM David Jacot <dj...@confluent.io>
> > > wrote:
> > > > > >
> > > > > > > Hi all,
> > > > > > >
> > > > > > > While implementing the KIP, I have realized that a new error
> code
> > > and
> > > > > > > exception is required to notify the caller that offsets of a
> topic
> > > can
> > > > > not
> > > > > > > be deleted because the group is actively subscribed to the
> topic.
> > > > > > >
> > > > > > > I would like to know if there are any concerns with these
> changes
> > > > > before
> > > > > > > updating the KIP.
> > > > > > >
> > > > > > > *Proposed API:*
> > > > > > > GROUP_SUBSCRIBED_TO_TOPIC(86, "The consumer group is actively
> > > > > subscribed
> > > > > > > to the topic", GroupSubscribedToTopicException::new);
> > > > > > >
> > > > > > > public class GroupSubscribedToTopicException extends
> ApiException {
> > > > > > > public GroupSubscribedToTopicException(String message) {
> > > > > > > super(message);
> > > > > > > }
> > > > > > > }
> > > > > > >
> > > > > > > Best,
> > > > > > > David
> > > > > > >
> > > > > > > On Fri, Aug 16, 2019 at 10:58 AM Mickael Maison <
> > > > > mickael.maison@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > >> +1 (non binding)
> > > > > > >> Thanks!
> > > > > > >>
> > > > > > >> On Thu, Aug 15, 2019 at 11:53 PM Colin McCabe <
> cmccabe@apache.org
> > > >
> > > > > wrote:
> > > > > > >> >
> > > > > > >> > On Thu, Aug 15, 2019, at 11:47, Jason Gustafson wrote:
> > > > > > >> > > Hey Colin, I think deleting all offsets is equivalent to
> > > deleting
> > > > > the
> > > > > > >> > > group, which can be done with the `deleteConsumerGroups`
> api.
> > > I
> > > > > > >> debated
> > > > > > >> > > whether there should be a way to delete partitions for all
> > > > > > >> unsubscribed
> > > > > > >> > > topics, but I decided to start with a simple API.
> > > > > > >> >
> > > > > > >> > That's a fair point-- deleting the group covers the main
> > > use-case
> > > > > for
> > > > > > >> deleting all offsets. So we might as well keep it simple for
> now.
> > > > > > >> >
> > > > > > >> > cheers,
> > > > > > >> > Colin
> > > > > > >> >
> > > > > > >> > >
> > > > > > >> > > I'm going to close this vote. The final result is +3 with
> > > myself,
> > > > > > >> Guozhang,
> > > > > > >> > > and Colin voting.
> > > > > > >> > >
> > > > > > >> > > -Jason
> > > > > > >> > >
> > > > > > >> > > On Tue, Aug 13, 2019 at 9:21 AM Colin McCabe <
> > > cmccabe@apache.org>
> > > > > > >> wrote:
> > > > > > >> > >
> > > > > > >> > > > Hi Jason,
> > > > > > >> > > >
> > > > > > >> > > > Thanks for the KIP.
> > > > > > >> > > >
> > > > > > >> > > > Is there ever a desire to delete all the offsets for a
> given
> > > > > group?
> > > > > > >> > > > Should the protocol and tools support this?
> > > > > > >> > > >
> > > > > > >> > > > +1 (binding)
> > > > > > >> > > >
> > > > > > >> > > > best,
> > > > > > >> > > > Colin
> > > > > > >> > > >
> > > > > > >> > > >
> > > > > > >> > > > On Mon, Aug 12, 2019, at 10:57, Guozhang Wang wrote:
> > > > > > >> > > > > +1 (binding).
> > > > > > >> > > > >
> > > > > > >> > > > > Thanks Jason!
> > > > > > >> > > > >
> > > > > > >> > > > > On Wed, Aug 7, 2019 at 11:18 AM Jason Gustafson <
> > > > > > >> jason@confluent.io>
> > > > > > >> > > > wrote:
> > > > > > >> > > > >
> > > > > > >> > > > > > Hi All,
> > > > > > >> > > > > >
> > > > > > >> > > > > > I'd like to start a vote on KIP-496:
> > > > > > >> > > > > >
> > > > > > >> > > > > >
> > > > > > >> > > >
> > > > > > >>
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-496%3A+Administrative+API+to+delete+consumer+offsets
> > > > > > >> > > > > > .
> > > > > > >> > > > > > +1
> > > > > > >> > > > > > from me of course.
> > > > > > >> > > > > >
> > > > > > >> > > > > > -Jason
> > > > > > >> > > > > >
> > > > > > >> > > > >
> > > > > > >> > > > >
> > > > > > >> > > > > --
> > > > > > >> > > > > -- Guozhang
> > > > > > >> > > > >
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [VOTE] KIP-496: Administrative API to delete consumer offsets
Posted by Colin McCabe <cm...@apache.org>.
Sounds good to me. It makes sense to add this functionality to the command line.
best,
Colin
On Wed, Sep 18, 2019, at 11:26, David Jacot wrote:
> Indeed, I have forgotten to add the action. There will be a new action «
> —delete-offsets ». Sorry!
>
> *Proposed API*
> kafka-consumer-groups.sh --bootstrap-server <bootstrap server>
> —delete-offsets --group <group id> --topic <topic-name>:<partition numbers>
> ex: --bootstrap-server localhost:9092 --group my-group --topic
> topic1 --topic topic2:0,1,2
>
> When partitions is not provided, all partitions are used.
>
> Best,
> David
>
> Le mer. 18 sept. 2019 à 20:09, Colin McCabe <cm...@apache.org> a écrit :
>
> > On Tue, Sep 17, 2019, at 09:07, David Jacot wrote:
> > > Hi all,
> > >
> > > We haven't included the changes in the command line tool to support the
> > new
> > > API. Therefore,
> > > I would like to amend the current KIP to cover the changes in the
> > > `kafka-consumer-groups`
> > > command line tool. The change is rather small and it does not need to add
> > > any new arguments
> > > to the command line tool. so it doesn't make sense to create a new KIP
> > for
> > > it.
> > >
> > > *Proposed API*
> > > kafka-consumer-groups.sh --bootstrap-server <bootstrap server> --group
> > > <group id> --topic <topic-name>:<partition numbers>
> > > ex: --bootstrap-server localhost:9092 --group my-group --topic topic1
> > > --topic topic2:0,1,2
> > >
> > > When partitions not provided, all partitions are used.
> >
> > Hmm. I think I'm missing something here. If you try the command you
> > specified, you get:
> >
> > > Command must include exactly one action: --list, --describe, --delete,
> > --reset-offsets
> >
> > Did you mean to add a new action here that was offsets-related?
> >
> > best,
> > Colin
> >
> > >
> > > What do you think?
> > >
> > > Best,
> > > David
> > >
> > >
> > > On Fri, Sep 13, 2019 at 6:42 PM Colin McCabe <cm...@apache.org> wrote:
> > >
> > > > Hi David,
> > > >
> > > > Sounds good.
> > > >
> > > > best,
> > > > Colin
> > > >
> > > >
> > > > On Fri, Sep 13, 2019, at 04:45, David Jacot wrote:
> > > > > Hi all,
> > > > >
> > > > > I would like to do another modification to the proposal. In the
> > proposal,
> > > > > the OffsetDeleteResponse
> > > > > doesn't have a top level error field so I would like to add one. Many
> > > > > errors concern the whole
> > > > > group (e.g. GROUP_ID_NOT_FOUND) so it would be great to have a way to
> > > > > communicate them
> > > > > back to the client without having to set such errors for all the
> > > > requested
> > > > > partitions. It makes the
> > > > > error handling on the client easier and cleaner.
> > > > >
> > > > > *Proposed API with the ErrorCode:*
> > > > > {
> > > > > "apiKey": 47,
> > > > > "type": "response",
> > > > > "name": "OffsetDeleteResponse",
> > > > > "validVersions": "0",
> > > > > "fields": [
> > > > > { "name": "ErrorCode", "type": "int16", "versions": "0+",
> > > > > "about": "The top-level error code, or 0 if there was no
> > error." },
> > > > > { "name": "ThrottleTimeMs", "type": "int32", "versions": "0+",
> > > > > "ignorable": true,
> > > > > "about": "The duration in milliseconds for which the request
> > was
> > > > > throttled due to a quota violation, or zero if the request did not
> > > > violate
> > > > > any quota." },
> > > > > { "name": "Topics", "type": "[]OffsetDeleteResponseTopic",
> > > > "versions":
> > > > > "0+",
> > > > > "about": "The responses for each topic.", "fields": [
> > > > > { "name": "Name", "type": "string", "versions": "0+",
> > "mapKey":
> > > > > true,
> > > > > "about": "The topic name." },
> > > > > { "name": "Partitions", "type":
> > > > "[]OffsetDeleteResponsePartition",
> > > > > "versions": "0+",
> > > > > "about": "The responses for each partition in the topic.",
> > > > > "fields": [
> > > > > { "name": "PartitionIndex", "type": "int32", "versions":
> > > > "0+",
> > > > > "mapKey": true,
> > > > > "about": "The partition index." },
> > > > > { "name": "ErrorCode", "type": "int16", "versions": "0+",
> > > > > "about": "The error code, or 0 if there was no error."
> > }
> > > > > ]
> > > > > }
> > > > > ]
> > > > > }
> > > > > ]
> > > > > }
> > > > >
> > > > > I would like to know if there are any concerns or objections
> > regarding
> > > > this
> > > > > change before updating the KIP.
> > > > >
> > > > > Best,
> > > > > David
> > > > >
> > > > > On Wed, Sep 4, 2019 at 9:24 AM David Jacot <dj...@confluent.io>
> > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > While implementing the KIP, I have realized that a new error code
> > and
> > > > > > exception is required to notify the caller that offsets of a topic
> > can
> > > > not
> > > > > > be deleted because the group is actively subscribed to the topic.
> > > > > >
> > > > > > I would like to know if there are any concerns with these changes
> > > > before
> > > > > > updating the KIP.
> > > > > >
> > > > > > *Proposed API:*
> > > > > > GROUP_SUBSCRIBED_TO_TOPIC(86, "The consumer group is actively
> > > > subscribed
> > > > > > to the topic", GroupSubscribedToTopicException::new);
> > > > > >
> > > > > > public class GroupSubscribedToTopicException extends ApiException {
> > > > > > public GroupSubscribedToTopicException(String message) {
> > > > > > super(message);
> > > > > > }
> > > > > > }
> > > > > >
> > > > > > Best,
> > > > > > David
> > > > > >
> > > > > > On Fri, Aug 16, 2019 at 10:58 AM Mickael Maison <
> > > > mickael.maison@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > >> +1 (non binding)
> > > > > >> Thanks!
> > > > > >>
> > > > > >> On Thu, Aug 15, 2019 at 11:53 PM Colin McCabe <cmccabe@apache.org
> > >
> > > > wrote:
> > > > > >> >
> > > > > >> > On Thu, Aug 15, 2019, at 11:47, Jason Gustafson wrote:
> > > > > >> > > Hey Colin, I think deleting all offsets is equivalent to
> > deleting
> > > > the
> > > > > >> > > group, which can be done with the `deleteConsumerGroups` api.
> > I
> > > > > >> debated
> > > > > >> > > whether there should be a way to delete partitions for all
> > > > > >> unsubscribed
> > > > > >> > > topics, but I decided to start with a simple API.
> > > > > >> >
> > > > > >> > That's a fair point-- deleting the group covers the main
> > use-case
> > > > for
> > > > > >> deleting all offsets. So we might as well keep it simple for now.
> > > > > >> >
> > > > > >> > cheers,
> > > > > >> > Colin
> > > > > >> >
> > > > > >> > >
> > > > > >> > > I'm going to close this vote. The final result is +3 with
> > myself,
> > > > > >> Guozhang,
> > > > > >> > > and Colin voting.
> > > > > >> > >
> > > > > >> > > -Jason
> > > > > >> > >
> > > > > >> > > On Tue, Aug 13, 2019 at 9:21 AM Colin McCabe <
> > cmccabe@apache.org>
> > > > > >> wrote:
> > > > > >> > >
> > > > > >> > > > Hi Jason,
> > > > > >> > > >
> > > > > >> > > > Thanks for the KIP.
> > > > > >> > > >
> > > > > >> > > > Is there ever a desire to delete all the offsets for a given
> > > > group?
> > > > > >> > > > Should the protocol and tools support this?
> > > > > >> > > >
> > > > > >> > > > +1 (binding)
> > > > > >> > > >
> > > > > >> > > > best,
> > > > > >> > > > Colin
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > > > On Mon, Aug 12, 2019, at 10:57, Guozhang Wang wrote:
> > > > > >> > > > > +1 (binding).
> > > > > >> > > > >
> > > > > >> > > > > Thanks Jason!
> > > > > >> > > > >
> > > > > >> > > > > On Wed, Aug 7, 2019 at 11:18 AM Jason Gustafson <
> > > > > >> jason@confluent.io>
> > > > > >> > > > wrote:
> > > > > >> > > > >
> > > > > >> > > > > > Hi All,
> > > > > >> > > > > >
> > > > > >> > > > > > I'd like to start a vote on KIP-496:
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > >
> > > > > >>
> > > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-496%3A+Administrative+API+to+delete+consumer+offsets
> > > > > >> > > > > > .
> > > > > >> > > > > > +1
> > > > > >> > > > > > from me of course.
> > > > > >> > > > > >
> > > > > >> > > > > > -Jason
> > > > > >> > > > > >
> > > > > >> > > > >
> > > > > >> > > > >
> > > > > >> > > > > --
> > > > > >> > > > > -- Guozhang
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [VOTE] KIP-496: Administrative API to delete consumer offsets
Posted by David Jacot <dj...@confluent.io>.
Indeed, I have forgotten to add the action. There will be a new action «
—delete-offsets ». Sorry!
*Proposed API*
kafka-consumer-groups.sh --bootstrap-server <bootstrap server>
—delete-offsets --group <group id> --topic <topic-name>:<partition numbers>
ex: --bootstrap-server localhost:9092 --group my-group --topic
topic1 --topic topic2:0,1,2
When partitions is not provided, all partitions are used.
Best,
David
Le mer. 18 sept. 2019 à 20:09, Colin McCabe <cm...@apache.org> a écrit :
> On Tue, Sep 17, 2019, at 09:07, David Jacot wrote:
> > Hi all,
> >
> > We haven't included the changes in the command line tool to support the
> new
> > API. Therefore,
> > I would like to amend the current KIP to cover the changes in the
> > `kafka-consumer-groups`
> > command line tool. The change is rather small and it does not need to add
> > any new arguments
> > to the command line tool. so it doesn't make sense to create a new KIP
> for
> > it.
> >
> > *Proposed API*
> > kafka-consumer-groups.sh --bootstrap-server <bootstrap server> --group
> > <group id> --topic <topic-name>:<partition numbers>
> > ex: --bootstrap-server localhost:9092 --group my-group --topic topic1
> > --topic topic2:0,1,2
> >
> > When partitions not provided, all partitions are used.
>
> Hmm. I think I'm missing something here. If you try the command you
> specified, you get:
>
> > Command must include exactly one action: --list, --describe, --delete,
> --reset-offsets
>
> Did you mean to add a new action here that was offsets-related?
>
> best,
> Colin
>
> >
> > What do you think?
> >
> > Best,
> > David
> >
> >
> > On Fri, Sep 13, 2019 at 6:42 PM Colin McCabe <cm...@apache.org> wrote:
> >
> > > Hi David,
> > >
> > > Sounds good.
> > >
> > > best,
> > > Colin
> > >
> > >
> > > On Fri, Sep 13, 2019, at 04:45, David Jacot wrote:
> > > > Hi all,
> > > >
> > > > I would like to do another modification to the proposal. In the
> proposal,
> > > > the OffsetDeleteResponse
> > > > doesn't have a top level error field so I would like to add one. Many
> > > > errors concern the whole
> > > > group (e.g. GROUP_ID_NOT_FOUND) so it would be great to have a way to
> > > > communicate them
> > > > back to the client without having to set such errors for all the
> > > requested
> > > > partitions. It makes the
> > > > error handling on the client easier and cleaner.
> > > >
> > > > *Proposed API with the ErrorCode:*
> > > > {
> > > > "apiKey": 47,
> > > > "type": "response",
> > > > "name": "OffsetDeleteResponse",
> > > > "validVersions": "0",
> > > > "fields": [
> > > > { "name": "ErrorCode", "type": "int16", "versions": "0+",
> > > > "about": "The top-level error code, or 0 if there was no
> error." },
> > > > { "name": "ThrottleTimeMs", "type": "int32", "versions": "0+",
> > > > "ignorable": true,
> > > > "about": "The duration in milliseconds for which the request
> was
> > > > throttled due to a quota violation, or zero if the request did not
> > > violate
> > > > any quota." },
> > > > { "name": "Topics", "type": "[]OffsetDeleteResponseTopic",
> > > "versions":
> > > > "0+",
> > > > "about": "The responses for each topic.", "fields": [
> > > > { "name": "Name", "type": "string", "versions": "0+",
> "mapKey":
> > > > true,
> > > > "about": "The topic name." },
> > > > { "name": "Partitions", "type":
> > > "[]OffsetDeleteResponsePartition",
> > > > "versions": "0+",
> > > > "about": "The responses for each partition in the topic.",
> > > > "fields": [
> > > > { "name": "PartitionIndex", "type": "int32", "versions":
> > > "0+",
> > > > "mapKey": true,
> > > > "about": "The partition index." },
> > > > { "name": "ErrorCode", "type": "int16", "versions": "0+",
> > > > "about": "The error code, or 0 if there was no error."
> }
> > > > ]
> > > > }
> > > > ]
> > > > }
> > > > ]
> > > > }
> > > >
> > > > I would like to know if there are any concerns or objections
> regarding
> > > this
> > > > change before updating the KIP.
> > > >
> > > > Best,
> > > > David
> > > >
> > > > On Wed, Sep 4, 2019 at 9:24 AM David Jacot <dj...@confluent.io>
> wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > While implementing the KIP, I have realized that a new error code
> and
> > > > > exception is required to notify the caller that offsets of a topic
> can
> > > not
> > > > > be deleted because the group is actively subscribed to the topic.
> > > > >
> > > > > I would like to know if there are any concerns with these changes
> > > before
> > > > > updating the KIP.
> > > > >
> > > > > *Proposed API:*
> > > > > GROUP_SUBSCRIBED_TO_TOPIC(86, "The consumer group is actively
> > > subscribed
> > > > > to the topic", GroupSubscribedToTopicException::new);
> > > > >
> > > > > public class GroupSubscribedToTopicException extends ApiException {
> > > > > public GroupSubscribedToTopicException(String message) {
> > > > > super(message);
> > > > > }
> > > > > }
> > > > >
> > > > > Best,
> > > > > David
> > > > >
> > > > > On Fri, Aug 16, 2019 at 10:58 AM Mickael Maison <
> > > mickael.maison@gmail.com>
> > > > > wrote:
> > > > >
> > > > >> +1 (non binding)
> > > > >> Thanks!
> > > > >>
> > > > >> On Thu, Aug 15, 2019 at 11:53 PM Colin McCabe <cmccabe@apache.org
> >
> > > wrote:
> > > > >> >
> > > > >> > On Thu, Aug 15, 2019, at 11:47, Jason Gustafson wrote:
> > > > >> > > Hey Colin, I think deleting all offsets is equivalent to
> deleting
> > > the
> > > > >> > > group, which can be done with the `deleteConsumerGroups` api.
> I
> > > > >> debated
> > > > >> > > whether there should be a way to delete partitions for all
> > > > >> unsubscribed
> > > > >> > > topics, but I decided to start with a simple API.
> > > > >> >
> > > > >> > That's a fair point-- deleting the group covers the main
> use-case
> > > for
> > > > >> deleting all offsets. So we might as well keep it simple for now.
> > > > >> >
> > > > >> > cheers,
> > > > >> > Colin
> > > > >> >
> > > > >> > >
> > > > >> > > I'm going to close this vote. The final result is +3 with
> myself,
> > > > >> Guozhang,
> > > > >> > > and Colin voting.
> > > > >> > >
> > > > >> > > -Jason
> > > > >> > >
> > > > >> > > On Tue, Aug 13, 2019 at 9:21 AM Colin McCabe <
> cmccabe@apache.org>
> > > > >> wrote:
> > > > >> > >
> > > > >> > > > Hi Jason,
> > > > >> > > >
> > > > >> > > > Thanks for the KIP.
> > > > >> > > >
> > > > >> > > > Is there ever a desire to delete all the offsets for a given
> > > group?
> > > > >> > > > Should the protocol and tools support this?
> > > > >> > > >
> > > > >> > > > +1 (binding)
> > > > >> > > >
> > > > >> > > > best,
> > > > >> > > > Colin
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > On Mon, Aug 12, 2019, at 10:57, Guozhang Wang wrote:
> > > > >> > > > > +1 (binding).
> > > > >> > > > >
> > > > >> > > > > Thanks Jason!
> > > > >> > > > >
> > > > >> > > > > On Wed, Aug 7, 2019 at 11:18 AM Jason Gustafson <
> > > > >> jason@confluent.io>
> > > > >> > > > wrote:
> > > > >> > > > >
> > > > >> > > > > > Hi All,
> > > > >> > > > > >
> > > > >> > > > > > I'd like to start a vote on KIP-496:
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > >
> > > > >>
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-496%3A+Administrative+API+to+delete+consumer+offsets
> > > > >> > > > > > .
> > > > >> > > > > > +1
> > > > >> > > > > > from me of course.
> > > > >> > > > > >
> > > > >> > > > > > -Jason
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > > > > --
> > > > >> > > > > -- Guozhang
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >>
> > > > >
> > > >
> > >
> >
>
Re: [VOTE] KIP-496: Administrative API to delete consumer offsets
Posted by Colin McCabe <cm...@apache.org>.
On Tue, Sep 17, 2019, at 09:07, David Jacot wrote:
> Hi all,
>
> We haven't included the changes in the command line tool to support the new
> API. Therefore,
> I would like to amend the current KIP to cover the changes in the
> `kafka-consumer-groups`
> command line tool. The change is rather small and it does not need to add
> any new arguments
> to the command line tool. so it doesn't make sense to create a new KIP for
> it.
>
> *Proposed API*
> kafka-consumer-groups.sh --bootstrap-server <bootstrap server> --group
> <group id> --topic <topic-name>:<partition numbers>
> ex: --bootstrap-server localhost:9092 --group my-group --topic topic1
> --topic topic2:0,1,2
>
> When partitions not provided, all partitions are used.
Hmm. I think I'm missing something here. If you try the command you specified, you get:
> Command must include exactly one action: --list, --describe, --delete, --reset-offsets
Did you mean to add a new action here that was offsets-related?
best,
Colin
>
> What do you think?
>
> Best,
> David
>
>
> On Fri, Sep 13, 2019 at 6:42 PM Colin McCabe <cm...@apache.org> wrote:
>
> > Hi David,
> >
> > Sounds good.
> >
> > best,
> > Colin
> >
> >
> > On Fri, Sep 13, 2019, at 04:45, David Jacot wrote:
> > > Hi all,
> > >
> > > I would like to do another modification to the proposal. In the proposal,
> > > the OffsetDeleteResponse
> > > doesn't have a top level error field so I would like to add one. Many
> > > errors concern the whole
> > > group (e.g. GROUP_ID_NOT_FOUND) so it would be great to have a way to
> > > communicate them
> > > back to the client without having to set such errors for all the
> > requested
> > > partitions. It makes the
> > > error handling on the client easier and cleaner.
> > >
> > > *Proposed API with the ErrorCode:*
> > > {
> > > "apiKey": 47,
> > > "type": "response",
> > > "name": "OffsetDeleteResponse",
> > > "validVersions": "0",
> > > "fields": [
> > > { "name": "ErrorCode", "type": "int16", "versions": "0+",
> > > "about": "The top-level error code, or 0 if there was no error." },
> > > { "name": "ThrottleTimeMs", "type": "int32", "versions": "0+",
> > > "ignorable": true,
> > > "about": "The duration in milliseconds for which the request was
> > > throttled due to a quota violation, or zero if the request did not
> > violate
> > > any quota." },
> > > { "name": "Topics", "type": "[]OffsetDeleteResponseTopic",
> > "versions":
> > > "0+",
> > > "about": "The responses for each topic.", "fields": [
> > > { "name": "Name", "type": "string", "versions": "0+", "mapKey":
> > > true,
> > > "about": "The topic name." },
> > > { "name": "Partitions", "type":
> > "[]OffsetDeleteResponsePartition",
> > > "versions": "0+",
> > > "about": "The responses for each partition in the topic.",
> > > "fields": [
> > > { "name": "PartitionIndex", "type": "int32", "versions":
> > "0+",
> > > "mapKey": true,
> > > "about": "The partition index." },
> > > { "name": "ErrorCode", "type": "int16", "versions": "0+",
> > > "about": "The error code, or 0 if there was no error." }
> > > ]
> > > }
> > > ]
> > > }
> > > ]
> > > }
> > >
> > > I would like to know if there are any concerns or objections regarding
> > this
> > > change before updating the KIP.
> > >
> > > Best,
> > > David
> > >
> > > On Wed, Sep 4, 2019 at 9:24 AM David Jacot <dj...@confluent.io> wrote:
> > >
> > > > Hi all,
> > > >
> > > > While implementing the KIP, I have realized that a new error code and
> > > > exception is required to notify the caller that offsets of a topic can
> > not
> > > > be deleted because the group is actively subscribed to the topic.
> > > >
> > > > I would like to know if there are any concerns with these changes
> > before
> > > > updating the KIP.
> > > >
> > > > *Proposed API:*
> > > > GROUP_SUBSCRIBED_TO_TOPIC(86, "The consumer group is actively
> > subscribed
> > > > to the topic", GroupSubscribedToTopicException::new);
> > > >
> > > > public class GroupSubscribedToTopicException extends ApiException {
> > > > public GroupSubscribedToTopicException(String message) {
> > > > super(message);
> > > > }
> > > > }
> > > >
> > > > Best,
> > > > David
> > > >
> > > > On Fri, Aug 16, 2019 at 10:58 AM Mickael Maison <
> > mickael.maison@gmail.com>
> > > > wrote:
> > > >
> > > >> +1 (non binding)
> > > >> Thanks!
> > > >>
> > > >> On Thu, Aug 15, 2019 at 11:53 PM Colin McCabe <cm...@apache.org>
> > wrote:
> > > >> >
> > > >> > On Thu, Aug 15, 2019, at 11:47, Jason Gustafson wrote:
> > > >> > > Hey Colin, I think deleting all offsets is equivalent to deleting
> > the
> > > >> > > group, which can be done with the `deleteConsumerGroups` api. I
> > > >> debated
> > > >> > > whether there should be a way to delete partitions for all
> > > >> unsubscribed
> > > >> > > topics, but I decided to start with a simple API.
> > > >> >
> > > >> > That's a fair point-- deleting the group covers the main use-case
> > for
> > > >> deleting all offsets. So we might as well keep it simple for now.
> > > >> >
> > > >> > cheers,
> > > >> > Colin
> > > >> >
> > > >> > >
> > > >> > > I'm going to close this vote. The final result is +3 with myself,
> > > >> Guozhang,
> > > >> > > and Colin voting.
> > > >> > >
> > > >> > > -Jason
> > > >> > >
> > > >> > > On Tue, Aug 13, 2019 at 9:21 AM Colin McCabe <cm...@apache.org>
> > > >> wrote:
> > > >> > >
> > > >> > > > Hi Jason,
> > > >> > > >
> > > >> > > > Thanks for the KIP.
> > > >> > > >
> > > >> > > > Is there ever a desire to delete all the offsets for a given
> > group?
> > > >> > > > Should the protocol and tools support this?
> > > >> > > >
> > > >> > > > +1 (binding)
> > > >> > > >
> > > >> > > > best,
> > > >> > > > Colin
> > > >> > > >
> > > >> > > >
> > > >> > > > On Mon, Aug 12, 2019, at 10:57, Guozhang Wang wrote:
> > > >> > > > > +1 (binding).
> > > >> > > > >
> > > >> > > > > Thanks Jason!
> > > >> > > > >
> > > >> > > > > On Wed, Aug 7, 2019 at 11:18 AM Jason Gustafson <
> > > >> jason@confluent.io>
> > > >> > > > wrote:
> > > >> > > > >
> > > >> > > > > > Hi All,
> > > >> > > > > >
> > > >> > > > > > I'd like to start a vote on KIP-496:
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > >
> > > >>
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-496%3A+Administrative+API+to+delete+consumer+offsets
> > > >> > > > > > .
> > > >> > > > > > +1
> > > >> > > > > > from me of course.
> > > >> > > > > >
> > > >> > > > > > -Jason
> > > >> > > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > --
> > > >> > > > > -- Guozhang
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >>
> > > >
> > >
> >
>
Re: [VOTE] KIP-496: Administrative API to delete consumer offsets
Posted by David Jacot <dj...@confluent.io>.
Hi all,
We haven't included the changes in the command line tool to support the new
API. Therefore,
I would like to amend the current KIP to cover the changes in the
`kafka-consumer-groups`
command line tool. The change is rather small and it does not need to add
any new arguments
to the command line tool. so it doesn't make sense to create a new KIP for
it.
*Proposed API*
kafka-consumer-groups.sh --bootstrap-server <bootstrap server> --group
<group id> --topic <topic-name>:<partition numbers>
ex: --bootstrap-server localhost:9092 --group my-group --topic topic1
--topic topic2:0,1,2
When partitions not provided, all partitions are used.
What do you think?
Best,
David
On Fri, Sep 13, 2019 at 6:42 PM Colin McCabe <cm...@apache.org> wrote:
> Hi David,
>
> Sounds good.
>
> best,
> Colin
>
>
> On Fri, Sep 13, 2019, at 04:45, David Jacot wrote:
> > Hi all,
> >
> > I would like to do another modification to the proposal. In the proposal,
> > the OffsetDeleteResponse
> > doesn't have a top level error field so I would like to add one. Many
> > errors concern the whole
> > group (e.g. GROUP_ID_NOT_FOUND) so it would be great to have a way to
> > communicate them
> > back to the client without having to set such errors for all the
> requested
> > partitions. It makes the
> > error handling on the client easier and cleaner.
> >
> > *Proposed API with the ErrorCode:*
> > {
> > "apiKey": 47,
> > "type": "response",
> > "name": "OffsetDeleteResponse",
> > "validVersions": "0",
> > "fields": [
> > { "name": "ErrorCode", "type": "int16", "versions": "0+",
> > "about": "The top-level error code, or 0 if there was no error." },
> > { "name": "ThrottleTimeMs", "type": "int32", "versions": "0+",
> > "ignorable": true,
> > "about": "The duration in milliseconds for which the request was
> > throttled due to a quota violation, or zero if the request did not
> violate
> > any quota." },
> > { "name": "Topics", "type": "[]OffsetDeleteResponseTopic",
> "versions":
> > "0+",
> > "about": "The responses for each topic.", "fields": [
> > { "name": "Name", "type": "string", "versions": "0+", "mapKey":
> > true,
> > "about": "The topic name." },
> > { "name": "Partitions", "type":
> "[]OffsetDeleteResponsePartition",
> > "versions": "0+",
> > "about": "The responses for each partition in the topic.",
> > "fields": [
> > { "name": "PartitionIndex", "type": "int32", "versions":
> "0+",
> > "mapKey": true,
> > "about": "The partition index." },
> > { "name": "ErrorCode", "type": "int16", "versions": "0+",
> > "about": "The error code, or 0 if there was no error." }
> > ]
> > }
> > ]
> > }
> > ]
> > }
> >
> > I would like to know if there are any concerns or objections regarding
> this
> > change before updating the KIP.
> >
> > Best,
> > David
> >
> > On Wed, Sep 4, 2019 at 9:24 AM David Jacot <dj...@confluent.io> wrote:
> >
> > > Hi all,
> > >
> > > While implementing the KIP, I have realized that a new error code and
> > > exception is required to notify the caller that offsets of a topic can
> not
> > > be deleted because the group is actively subscribed to the topic.
> > >
> > > I would like to know if there are any concerns with these changes
> before
> > > updating the KIP.
> > >
> > > *Proposed API:*
> > > GROUP_SUBSCRIBED_TO_TOPIC(86, "The consumer group is actively
> subscribed
> > > to the topic", GroupSubscribedToTopicException::new);
> > >
> > > public class GroupSubscribedToTopicException extends ApiException {
> > > public GroupSubscribedToTopicException(String message) {
> > > super(message);
> > > }
> > > }
> > >
> > > Best,
> > > David
> > >
> > > On Fri, Aug 16, 2019 at 10:58 AM Mickael Maison <
> mickael.maison@gmail.com>
> > > wrote:
> > >
> > >> +1 (non binding)
> > >> Thanks!
> > >>
> > >> On Thu, Aug 15, 2019 at 11:53 PM Colin McCabe <cm...@apache.org>
> wrote:
> > >> >
> > >> > On Thu, Aug 15, 2019, at 11:47, Jason Gustafson wrote:
> > >> > > Hey Colin, I think deleting all offsets is equivalent to deleting
> the
> > >> > > group, which can be done with the `deleteConsumerGroups` api. I
> > >> debated
> > >> > > whether there should be a way to delete partitions for all
> > >> unsubscribed
> > >> > > topics, but I decided to start with a simple API.
> > >> >
> > >> > That's a fair point-- deleting the group covers the main use-case
> for
> > >> deleting all offsets. So we might as well keep it simple for now.
> > >> >
> > >> > cheers,
> > >> > Colin
> > >> >
> > >> > >
> > >> > > I'm going to close this vote. The final result is +3 with myself,
> > >> Guozhang,
> > >> > > and Colin voting.
> > >> > >
> > >> > > -Jason
> > >> > >
> > >> > > On Tue, Aug 13, 2019 at 9:21 AM Colin McCabe <cm...@apache.org>
> > >> wrote:
> > >> > >
> > >> > > > Hi Jason,
> > >> > > >
> > >> > > > Thanks for the KIP.
> > >> > > >
> > >> > > > Is there ever a desire to delete all the offsets for a given
> group?
> > >> > > > Should the protocol and tools support this?
> > >> > > >
> > >> > > > +1 (binding)
> > >> > > >
> > >> > > > best,
> > >> > > > Colin
> > >> > > >
> > >> > > >
> > >> > > > On Mon, Aug 12, 2019, at 10:57, Guozhang Wang wrote:
> > >> > > > > +1 (binding).
> > >> > > > >
> > >> > > > > Thanks Jason!
> > >> > > > >
> > >> > > > > On Wed, Aug 7, 2019 at 11:18 AM Jason Gustafson <
> > >> jason@confluent.io>
> > >> > > > wrote:
> > >> > > > >
> > >> > > > > > Hi All,
> > >> > > > > >
> > >> > > > > > I'd like to start a vote on KIP-496:
> > >> > > > > >
> > >> > > > > >
> > >> > > >
> > >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-496%3A+Administrative+API+to+delete+consumer+offsets
> > >> > > > > > .
> > >> > > > > > +1
> > >> > > > > > from me of course.
> > >> > > > > >
> > >> > > > > > -Jason
> > >> > > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > --
> > >> > > > > -- Guozhang
> > >> > > > >
> > >> > > >
> > >> > >
> > >>
> > >
> >
>
Re: [VOTE] KIP-496: Administrative API to delete consumer offsets
Posted by Colin McCabe <cm...@apache.org>.
Hi David,
Sounds good.
best,
Colin
On Fri, Sep 13, 2019, at 04:45, David Jacot wrote:
> Hi all,
>
> I would like to do another modification to the proposal. In the proposal,
> the OffsetDeleteResponse
> doesn't have a top level error field so I would like to add one. Many
> errors concern the whole
> group (e.g. GROUP_ID_NOT_FOUND) so it would be great to have a way to
> communicate them
> back to the client without having to set such errors for all the requested
> partitions. It makes the
> error handling on the client easier and cleaner.
>
> *Proposed API with the ErrorCode:*
> {
> "apiKey": 47,
> "type": "response",
> "name": "OffsetDeleteResponse",
> "validVersions": "0",
> "fields": [
> { "name": "ErrorCode", "type": "int16", "versions": "0+",
> "about": "The top-level error code, or 0 if there was no error." },
> { "name": "ThrottleTimeMs", "type": "int32", "versions": "0+",
> "ignorable": true,
> "about": "The duration in milliseconds for which the request was
> throttled due to a quota violation, or zero if the request did not violate
> any quota." },
> { "name": "Topics", "type": "[]OffsetDeleteResponseTopic", "versions":
> "0+",
> "about": "The responses for each topic.", "fields": [
> { "name": "Name", "type": "string", "versions": "0+", "mapKey":
> true,
> "about": "The topic name." },
> { "name": "Partitions", "type": "[]OffsetDeleteResponsePartition",
> "versions": "0+",
> "about": "The responses for each partition in the topic.",
> "fields": [
> { "name": "PartitionIndex", "type": "int32", "versions": "0+",
> "mapKey": true,
> "about": "The partition index." },
> { "name": "ErrorCode", "type": "int16", "versions": "0+",
> "about": "The error code, or 0 if there was no error." }
> ]
> }
> ]
> }
> ]
> }
>
> I would like to know if there are any concerns or objections regarding this
> change before updating the KIP.
>
> Best,
> David
>
> On Wed, Sep 4, 2019 at 9:24 AM David Jacot <dj...@confluent.io> wrote:
>
> > Hi all,
> >
> > While implementing the KIP, I have realized that a new error code and
> > exception is required to notify the caller that offsets of a topic can not
> > be deleted because the group is actively subscribed to the topic.
> >
> > I would like to know if there are any concerns with these changes before
> > updating the KIP.
> >
> > *Proposed API:*
> > GROUP_SUBSCRIBED_TO_TOPIC(86, "The consumer group is actively subscribed
> > to the topic", GroupSubscribedToTopicException::new);
> >
> > public class GroupSubscribedToTopicException extends ApiException {
> > public GroupSubscribedToTopicException(String message) {
> > super(message);
> > }
> > }
> >
> > Best,
> > David
> >
> > On Fri, Aug 16, 2019 at 10:58 AM Mickael Maison <mi...@gmail.com>
> > wrote:
> >
> >> +1 (non binding)
> >> Thanks!
> >>
> >> On Thu, Aug 15, 2019 at 11:53 PM Colin McCabe <cm...@apache.org> wrote:
> >> >
> >> > On Thu, Aug 15, 2019, at 11:47, Jason Gustafson wrote:
> >> > > Hey Colin, I think deleting all offsets is equivalent to deleting the
> >> > > group, which can be done with the `deleteConsumerGroups` api. I
> >> debated
> >> > > whether there should be a way to delete partitions for all
> >> unsubscribed
> >> > > topics, but I decided to start with a simple API.
> >> >
> >> > That's a fair point-- deleting the group covers the main use-case for
> >> deleting all offsets. So we might as well keep it simple for now.
> >> >
> >> > cheers,
> >> > Colin
> >> >
> >> > >
> >> > > I'm going to close this vote. The final result is +3 with myself,
> >> Guozhang,
> >> > > and Colin voting.
> >> > >
> >> > > -Jason
> >> > >
> >> > > On Tue, Aug 13, 2019 at 9:21 AM Colin McCabe <cm...@apache.org>
> >> wrote:
> >> > >
> >> > > > Hi Jason,
> >> > > >
> >> > > > Thanks for the KIP.
> >> > > >
> >> > > > Is there ever a desire to delete all the offsets for a given group?
> >> > > > Should the protocol and tools support this?
> >> > > >
> >> > > > +1 (binding)
> >> > > >
> >> > > > best,
> >> > > > Colin
> >> > > >
> >> > > >
> >> > > > On Mon, Aug 12, 2019, at 10:57, Guozhang Wang wrote:
> >> > > > > +1 (binding).
> >> > > > >
> >> > > > > Thanks Jason!
> >> > > > >
> >> > > > > On Wed, Aug 7, 2019 at 11:18 AM Jason Gustafson <
> >> jason@confluent.io>
> >> > > > wrote:
> >> > > > >
> >> > > > > > Hi All,
> >> > > > > >
> >> > > > > > I'd like to start a vote on KIP-496:
> >> > > > > >
> >> > > > > >
> >> > > >
> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-496%3A+Administrative+API+to+delete+consumer+offsets
> >> > > > > > .
> >> > > > > > +1
> >> > > > > > from me of course.
> >> > > > > >
> >> > > > > > -Jason
> >> > > > > >
> >> > > > >
> >> > > > >
> >> > > > > --
> >> > > > > -- Guozhang
> >> > > > >
> >> > > >
> >> > >
> >>
> >
>
Re: [VOTE] KIP-496: Administrative API to delete consumer offsets
Posted by David Jacot <dj...@confluent.io>.
Hi all,
I would like to do another modification to the proposal. In the proposal,
the OffsetDeleteResponse
doesn't have a top level error field so I would like to add one. Many
errors concern the whole
group (e.g. GROUP_ID_NOT_FOUND) so it would be great to have a way to
communicate them
back to the client without having to set such errors for all the requested
partitions. It makes the
error handling on the client easier and cleaner.
*Proposed API with the ErrorCode:*
{
"apiKey": 47,
"type": "response",
"name": "OffsetDeleteResponse",
"validVersions": "0",
"fields": [
{ "name": "ErrorCode", "type": "int16", "versions": "0+",
"about": "The top-level error code, or 0 if there was no error." },
{ "name": "ThrottleTimeMs", "type": "int32", "versions": "0+",
"ignorable": true,
"about": "The duration in milliseconds for which the request was
throttled due to a quota violation, or zero if the request did not violate
any quota." },
{ "name": "Topics", "type": "[]OffsetDeleteResponseTopic", "versions":
"0+",
"about": "The responses for each topic.", "fields": [
{ "name": "Name", "type": "string", "versions": "0+", "mapKey":
true,
"about": "The topic name." },
{ "name": "Partitions", "type": "[]OffsetDeleteResponsePartition",
"versions": "0+",
"about": "The responses for each partition in the topic.",
"fields": [
{ "name": "PartitionIndex", "type": "int32", "versions": "0+",
"mapKey": true,
"about": "The partition index." },
{ "name": "ErrorCode", "type": "int16", "versions": "0+",
"about": "The error code, or 0 if there was no error." }
]
}
]
}
]
}
I would like to know if there are any concerns or objections regarding this
change before updating the KIP.
Best,
David
On Wed, Sep 4, 2019 at 9:24 AM David Jacot <dj...@confluent.io> wrote:
> Hi all,
>
> While implementing the KIP, I have realized that a new error code and
> exception is required to notify the caller that offsets of a topic can not
> be deleted because the group is actively subscribed to the topic.
>
> I would like to know if there are any concerns with these changes before
> updating the KIP.
>
> *Proposed API:*
> GROUP_SUBSCRIBED_TO_TOPIC(86, "The consumer group is actively subscribed
> to the topic", GroupSubscribedToTopicException::new);
>
> public class GroupSubscribedToTopicException extends ApiException {
> public GroupSubscribedToTopicException(String message) {
> super(message);
> }
> }
>
> Best,
> David
>
> On Fri, Aug 16, 2019 at 10:58 AM Mickael Maison <mi...@gmail.com>
> wrote:
>
>> +1 (non binding)
>> Thanks!
>>
>> On Thu, Aug 15, 2019 at 11:53 PM Colin McCabe <cm...@apache.org> wrote:
>> >
>> > On Thu, Aug 15, 2019, at 11:47, Jason Gustafson wrote:
>> > > Hey Colin, I think deleting all offsets is equivalent to deleting the
>> > > group, which can be done with the `deleteConsumerGroups` api. I
>> debated
>> > > whether there should be a way to delete partitions for all
>> unsubscribed
>> > > topics, but I decided to start with a simple API.
>> >
>> > That's a fair point-- deleting the group covers the main use-case for
>> deleting all offsets. So we might as well keep it simple for now.
>> >
>> > cheers,
>> > Colin
>> >
>> > >
>> > > I'm going to close this vote. The final result is +3 with myself,
>> Guozhang,
>> > > and Colin voting.
>> > >
>> > > -Jason
>> > >
>> > > On Tue, Aug 13, 2019 at 9:21 AM Colin McCabe <cm...@apache.org>
>> wrote:
>> > >
>> > > > Hi Jason,
>> > > >
>> > > > Thanks for the KIP.
>> > > >
>> > > > Is there ever a desire to delete all the offsets for a given group?
>> > > > Should the protocol and tools support this?
>> > > >
>> > > > +1 (binding)
>> > > >
>> > > > best,
>> > > > Colin
>> > > >
>> > > >
>> > > > On Mon, Aug 12, 2019, at 10:57, Guozhang Wang wrote:
>> > > > > +1 (binding).
>> > > > >
>> > > > > Thanks Jason!
>> > > > >
>> > > > > On Wed, Aug 7, 2019 at 11:18 AM Jason Gustafson <
>> jason@confluent.io>
>> > > > wrote:
>> > > > >
>> > > > > > Hi All,
>> > > > > >
>> > > > > > I'd like to start a vote on KIP-496:
>> > > > > >
>> > > > > >
>> > > >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-496%3A+Administrative+API+to+delete+consumer+offsets
>> > > > > > .
>> > > > > > +1
>> > > > > > from me of course.
>> > > > > >
>> > > > > > -Jason
>> > > > > >
>> > > > >
>> > > > >
>> > > > > --
>> > > > > -- Guozhang
>> > > > >
>> > > >
>> > >
>>
>