You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Arseniy Tashoyan <ta...@gmail.com> on 2018/06/01 21:32:34 UTC
[DISCUSS] KIP-308: GetOffsetShell: new KafkaConsumer API, support for
multiple topics, minimize the number of requests to server
Hi,
I have just created KIP that proposes enhancements to GetOffsetShell
command line tool.
KIP:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-308%3A+GetOffsetShell%3A+new+KafkaConsumer+API%2C+support+for+multiple+topics%2C+minimize+the+number+of+requests+to+server
PR: https://github.com/apache/kafka/pull/3051
Suggestions are welcome.
Regards,
Arseniy
Re: [DISCUSS] KIP-308: GetOffsetShell: new KafkaConsumer API, support
for multiple topics, minimize the number of requests to server
Posted by Stanislav Kozlovski <st...@confluent.io>.
Hey Arseniy,
Do you plan on continuing the work for this KIP?
I would gladly take over. I find myself needing the GetOffsetShell tool to
work on secured cluster time and time again, I had even implemented a
duplicate JIRA (KAFKA-7373
<https://issues.apache.org/jira/browse/KAFKA-7373>) about it once.
Thanks,
Stanislav
On Tue, Sep 25, 2018 at 3:09 PM Manikumar <ma...@gmail.com> wrote:
> Hi Arseniy,
>
> It is good have GetOffsetShell tool to work on secure clusters. Passing
> additional configuration to the
> underlying KafkaConsumer is sufficient for the requirement. Can you revive
> this discussion thread?
> Let us know if you need any help.
>
> Thanks,
>
>
> On Mon, Jun 4, 2018 at 3:27 PM Viktor Somogyi <vi...@gmail.com>
> wrote:
>
> > Hi Arseniy,
> >
> > A few comments from me collected in points:
> >
> > 1. Usually the commands use "bootstrap-server" as in singular and not the
> > plural "bootstrap-servers", so I suggest to keep this functionality.
> > Otherwise their behavior is still the same.
> >
> > 2. kafka-get-offsets.sh --bootstrap-servers vm:9092 --topics AAA,ZZZ
> > --partitions 0,1
> > I think that we could try out one of the "--topics AAA:0-3,ZZZ:1" or
> > "--topic-partitions AAA:0-3,ZZZ:1" or "--tp AAA:0-3,ZZZ:1" notations so
> > that asking for specific topic-partitions (or intervals as in AAA) is
> more
> > concise. Perhaps we could enable using your originally proposed and one
> of
> > these. What do you think?
> > Also I would be interested in what others think about this notation. I'd
> > find it useful as we often refer to partitions as "topic-partition" and
> > this might strengthen this kind of thinking.
> >
> > Viktor
> >
> >
> > On Fri, Jun 1, 2018 at 11:32 PM, Arseniy Tashoyan <ta...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > I have just created KIP that proposes enhancements to GetOffsetShell
> > > command line tool.
> > > KIP:
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 308%3A+GetOffsetShell%3A+new+KafkaConsumer+API%2C+support+
> > > for+multiple+topics%2C+minimize+the+number+of+requests+to+server
> > >
> > > PR: https://github.com/apache/kafka/pull/3051
> > >
> > > Suggestions are welcome.
> > >
> > > Regards,
> > > Arseniy
> > >
> >
>
--
Best,
Stanislav
Re: [DISCUSS] KIP-308: GetOffsetShell: new KafkaConsumer API, support
for multiple topics, minimize the number of requests to server
Posted by Manikumar <ma...@gmail.com>.
Hi Arseniy,
It is good have GetOffsetShell tool to work on secure clusters. Passing
additional configuration to the
underlying KafkaConsumer is sufficient for the requirement. Can you revive
this discussion thread?
Let us know if you need any help.
Thanks,
On Mon, Jun 4, 2018 at 3:27 PM Viktor Somogyi <vi...@gmail.com>
wrote:
> Hi Arseniy,
>
> A few comments from me collected in points:
>
> 1. Usually the commands use "bootstrap-server" as in singular and not the
> plural "bootstrap-servers", so I suggest to keep this functionality.
> Otherwise their behavior is still the same.
>
> 2. kafka-get-offsets.sh --bootstrap-servers vm:9092 --topics AAA,ZZZ
> --partitions 0,1
> I think that we could try out one of the "--topics AAA:0-3,ZZZ:1" or
> "--topic-partitions AAA:0-3,ZZZ:1" or "--tp AAA:0-3,ZZZ:1" notations so
> that asking for specific topic-partitions (or intervals as in AAA) is more
> concise. Perhaps we could enable using your originally proposed and one of
> these. What do you think?
> Also I would be interested in what others think about this notation. I'd
> find it useful as we often refer to partitions as "topic-partition" and
> this might strengthen this kind of thinking.
>
> Viktor
>
>
> On Fri, Jun 1, 2018 at 11:32 PM, Arseniy Tashoyan <ta...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I have just created KIP that proposes enhancements to GetOffsetShell
> > command line tool.
> > KIP:
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 308%3A+GetOffsetShell%3A+new+KafkaConsumer+API%2C+support+
> > for+multiple+topics%2C+minimize+the+number+of+requests+to+server
> >
> > PR: https://github.com/apache/kafka/pull/3051
> >
> > Suggestions are welcome.
> >
> > Regards,
> > Arseniy
> >
>
Re: [DISCUSS] KIP-308: GetOffsetShell: new KafkaConsumer API, support
for multiple topics, minimize the number of requests to server
Posted by Viktor Somogyi <vi...@gmail.com>.
Hi Arseniy,
A few comments from me collected in points:
1. Usually the commands use "bootstrap-server" as in singular and not the
plural "bootstrap-servers", so I suggest to keep this functionality.
Otherwise their behavior is still the same.
2. kafka-get-offsets.sh --bootstrap-servers vm:9092 --topics AAA,ZZZ
--partitions 0,1
I think that we could try out one of the "--topics AAA:0-3,ZZZ:1" or
"--topic-partitions AAA:0-3,ZZZ:1" or "--tp AAA:0-3,ZZZ:1" notations so
that asking for specific topic-partitions (or intervals as in AAA) is more
concise. Perhaps we could enable using your originally proposed and one of
these. What do you think?
Also I would be interested in what others think about this notation. I'd
find it useful as we often refer to partitions as "topic-partition" and
this might strengthen this kind of thinking.
Viktor
On Fri, Jun 1, 2018 at 11:32 PM, Arseniy Tashoyan <ta...@gmail.com>
wrote:
> Hi,
>
> I have just created KIP that proposes enhancements to GetOffsetShell
> command line tool.
> KIP:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 308%3A+GetOffsetShell%3A+new+KafkaConsumer+API%2C+support+
> for+multiple+topics%2C+minimize+the+number+of+requests+to+server
>
> PR: https://github.com/apache/kafka/pull/3051
>
> Suggestions are welcome.
>
> Regards,
> Arseniy
>