You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Lianet Magrans <li...@apache.org> on 2026/05/24 10:14:55 UTC
Re: [DISCUSS] KIP-1323: Initialization of share group offsets from a specific offset or a file
Hi Andrew, thanks for the KIP!
LM1: about the --from-file option, the KIP shows the example with a single
group, but will it support loading multiple groups from a single file too?
(or loading a single group but from a file with multiple groups?). The
trick is thst the consumer tool allows exporting multiple groups to a
single file with the --all-groups option (so I imagine it may be used with
files with several groups). If the intention is to support multiple groups
as part of this KIP, I imagine the parsing and error handling behaviour
will just be the same as when the consumer-group tool resets from a file
containing multiple groups? (or no multi group support for now?)
LM2: about the --to-offset option, several things could go wrong, I imagine
the behaviour will just be the same as in the consumer-group tool when
resetting to a negative offset, or an offset out of range?
Thanks!
Lianet
On Thu, Apr 30, 2026 at 2:48 PM Andrew Schofield <sc...@apache.org>
wrote:
> Hi Apoorv,
> Thanks for your review.
>
> AM1: I can add --export, but I will need to add --to-current as well. I'll
> update the KIP.
>
> Thanks,
> Andrew
>
> On 2026/04/30 10:59:42 Apoorv Mittal wrote:
> > Hi Andrew,
> > Thanks for the KIP. It will be a great addition. I have one query:
> >
> > AM1: Does --export also make sense for share groups as well? The current
> > definition in KIP works for consumer group -> share group migration, but
> > without --export on share groups, user cannot do:
> > - Share group -> share group migration (e.g., renaming a group)
> > - Share group -> consumer group rollback (if the migration doesn't go
> > well)
> >
> > Hence, KIP should either add --export to share groups or explicitly state
> > why it's being deferred? Please let me know your thoughts.
> >
> > Regards,
> > Apoorv Mittal
> >
> >
> > On Thu, Apr 23, 2026 at 6:18 PM Andrew Schofield <
> andrew_schofield@live.com>
> > wrote:
> >
> > > Hi,
> > > I’d like to start discussion for a new KIP which adds some new options
> for
> > > initialization of share groups. This makes it easier to migrate
> > > applications from a consumer group to a share group from the committed
> > > offsets of the consumer group.
> > >
> > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1323%3A+Initialization+of+share+group+offsets+from+a+specific+offset+or+a+file
> > >
> > > Thanks,
> > > Andrew
> > >
> >
>
Re: [DISCUSS] KIP-1323: Initialization of share group offsets from a specific offset or a file
Posted by Lianet Magrans <li...@apache.org>.
Thanks for the answers and the KIP updates Andrew, lgtm.
Cheers!
Lianet
On Sun, May 24, 2026 at 10:03 AM Andrew Schofield <sc...@apache.org>
wrote:
> Hi Lianet,
> Thanks for your comments.
>
> LM1: kafka-share-groups.sh is only able to reset the offsets for a single
> group at a time. I did not extend that capability in this KIP. I understand
> that kafka-consumer-groups.sh can do that, but the share groups tool cannot.
>
> If the user exports the offsets for multiple consumer groups at once, the
> resulting file will contains 4 columns (group, topic, partition, offset).
> If you then reset share group offsets from such a file, it filters the
> lines by the specified group ID, in exactly the same way as the
> kafka-consumer-groups.sh tool.
>
> LM2: Yes, the offset validation will be just the same.
>
> I've updated the KIP to be more explicit that the behaviour will be the
> same as kafka-consumer-groups.sh, with the exception of being able to work
> on multiple groups at once.
>
> Thanks,
> Andrew
>
> On 2026/05/24 10:14:55 Lianet Magrans wrote:
> > Hi Andrew, thanks for the KIP!
> >
> > LM1: about the --from-file option, the KIP shows the example with a
> single
> > group, but will it support loading multiple groups from a single file
> too?
> > (or loading a single group but from a file with multiple groups?). The
> > trick is thst the consumer tool allows exporting multiple groups to a
> > single file with the --all-groups option (so I imagine it may be used
> with
> > files with several groups). If the intention is to support multiple
> groups
> > as part of this KIP, I imagine the parsing and error handling behaviour
> > will just be the same as when the consumer-group tool resets from a file
> > containing multiple groups? (or no multi group support for now?)
> >
> > LM2: about the --to-offset option, several things could go wrong, I
> imagine
> > the behaviour will just be the same as in the consumer-group tool when
> > resetting to a negative offset, or an offset out of range?
> >
> > Thanks!
> > Lianet
> >
> > On Thu, Apr 30, 2026 at 2:48 PM Andrew Schofield <sc...@apache.org>
> > wrote:
> >
> > > Hi Apoorv,
> > > Thanks for your review.
> > >
> > > AM1: I can add --export, but I will need to add --to-current as well.
> I'll
> > > update the KIP.
> > >
> > > Thanks,
> > > Andrew
> > >
> > > On 2026/04/30 10:59:42 Apoorv Mittal wrote:
> > > > Hi Andrew,
> > > > Thanks for the KIP. It will be a great addition. I have one query:
> > > >
> > > > AM1: Does --export also make sense for share groups as well? The
> current
> > > > definition in KIP works for consumer group -> share group migration,
> but
> > > > without --export on share groups, user cannot do:
> > > > - Share group -> share group migration (e.g., renaming a group)
> > > > - Share group -> consumer group rollback (if the migration doesn't
> go
> > > > well)
> > > >
> > > > Hence, KIP should either add --export to share groups or explicitly
> state
> > > > why it's being deferred? Please let me know your thoughts.
> > > >
> > > > Regards,
> > > > Apoorv Mittal
> > > >
> > > >
> > > > On Thu, Apr 23, 2026 at 6:18 PM Andrew Schofield <
> > > andrew_schofield@live.com>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > > I’d like to start discussion for a new KIP which adds some new
> options
> > > for
> > > > > initialization of share groups. This makes it easier to migrate
> > > > > applications from a consumer group to a share group from the
> committed
> > > > > offsets of the consumer group.
> > > > >
> > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1323%3A+Initialization+of+share+group+offsets+from+a+specific+offset+or+a+file
> > > > >
> > > > > Thanks,
> > > > > Andrew
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] KIP-1323: Initialization of share group offsets from a specific offset or a file
Posted by Andrew Schofield <sc...@apache.org>.
Hi Lianet,
Thanks for your comments.
LM1: kafka-share-groups.sh is only able to reset the offsets for a single group at a time. I did not extend that capability in this KIP. I understand that kafka-consumer-groups.sh can do that, but the share groups tool cannot.
If the user exports the offsets for multiple consumer groups at once, the resulting file will contains 4 columns (group, topic, partition, offset). If you then reset share group offsets from such a file, it filters the lines by the specified group ID, in exactly the same way as the kafka-consumer-groups.sh tool.
LM2: Yes, the offset validation will be just the same.
I've updated the KIP to be more explicit that the behaviour will be the same as kafka-consumer-groups.sh, with the exception of being able to work on multiple groups at once.
Thanks,
Andrew
On 2026/05/24 10:14:55 Lianet Magrans wrote:
> Hi Andrew, thanks for the KIP!
>
> LM1: about the --from-file option, the KIP shows the example with a single
> group, but will it support loading multiple groups from a single file too?
> (or loading a single group but from a file with multiple groups?). The
> trick is thst the consumer tool allows exporting multiple groups to a
> single file with the --all-groups option (so I imagine it may be used with
> files with several groups). If the intention is to support multiple groups
> as part of this KIP, I imagine the parsing and error handling behaviour
> will just be the same as when the consumer-group tool resets from a file
> containing multiple groups? (or no multi group support for now?)
>
> LM2: about the --to-offset option, several things could go wrong, I imagine
> the behaviour will just be the same as in the consumer-group tool when
> resetting to a negative offset, or an offset out of range?
>
> Thanks!
> Lianet
>
> On Thu, Apr 30, 2026 at 2:48 PM Andrew Schofield <sc...@apache.org>
> wrote:
>
> > Hi Apoorv,
> > Thanks for your review.
> >
> > AM1: I can add --export, but I will need to add --to-current as well. I'll
> > update the KIP.
> >
> > Thanks,
> > Andrew
> >
> > On 2026/04/30 10:59:42 Apoorv Mittal wrote:
> > > Hi Andrew,
> > > Thanks for the KIP. It will be a great addition. I have one query:
> > >
> > > AM1: Does --export also make sense for share groups as well? The current
> > > definition in KIP works for consumer group -> share group migration, but
> > > without --export on share groups, user cannot do:
> > > - Share group -> share group migration (e.g., renaming a group)
> > > - Share group -> consumer group rollback (if the migration doesn't go
> > > well)
> > >
> > > Hence, KIP should either add --export to share groups or explicitly state
> > > why it's being deferred? Please let me know your thoughts.
> > >
> > > Regards,
> > > Apoorv Mittal
> > >
> > >
> > > On Thu, Apr 23, 2026 at 6:18 PM Andrew Schofield <
> > andrew_schofield@live.com>
> > > wrote:
> > >
> > > > Hi,
> > > > I’d like to start discussion for a new KIP which adds some new options
> > for
> > > > initialization of share groups. This makes it easier to migrate
> > > > applications from a consumer group to a share group from the committed
> > > > offsets of the consumer group.
> > > >
> > > >
> > > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1323%3A+Initialization+of+share+group+offsets+from+a+specific+offset+or+a+file
> > > >
> > > > Thanks,
> > > > Andrew
> > > >
> > >
> >
>