You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by 一直以来 <27...@qq.com> on 2020/04/22 06:37:23 UTC
thank you ! which java-client api can has same effect about kafka-consumer-groups.sh command ?
./kafka-consumer-groups.sh --bootstrap-server localhost:9081 --describe --group test
use describeConsumerGroups method ??
private static void print() throws InterruptedException, ExecutionException {
Properties props = new Properties();
props.setProperty("bootstrap.servers",
"192.168.1.100:9081,192.168.1.100:9082,192.168.1.100:9083,,192.168.1.100:9087,,192.168.1.100:9088");
AdminClient client = KafkaAdminClient.create(props);
DescribeConsumerGroupsResult describeConsumerGroups = client.describeConsumerGroups(Arrays.asList("test"));
Map<String, KafkaFuture<ConsumerGroupDescription>> describedGroups = describeConsumerGroups.describedGroups();
Iterator<String> iterator = describedGroups.keySet().iterator();
while (iterator.hasNext()) {
String key = iterator.next();
KafkaFuture<ConsumerGroupDescription> value = describedGroups.get(key);
ConsumerGroupDescription consumerGroupDescription = value.get();
Collection<MemberDescription> members = consumerGroupDescription.members();
}
}
but i can't find about any method about bottom column:
GROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID
By its three info :
PARTITION CURRENT-OFFSET LOG-END-OFFSET
Re: thank you ! which java-client api can has same effect about
kafka-consumer-groups.sh command ?
Posted by Liam Clarke-Hutchinson <li...@adscale.co.nz>.
Ah, with you now. You'll also need to use the results of
AdminClient.listOffsets which takes TopicPartition objects as an argument.
On Wed, Apr 22, 2020 at 7:43 PM 一直以来 <27...@qq.com> wrote:
> i use :
> private static void printConsumerGroupOffsets() throws
> InterruptedException, ExecutionException {
> Properties props = new Properties();
> props.setProperty("bootstrap.servers",
> "192.168.1.100:9081,192.168.1.100:9082,
> 192.168.1.100:9083,,192.168.1.100:9087,,192.168.1.100:9088");
> AdminClient client = KafkaAdminClient.create(props);
>
>
> ListConsumerGroupOffsetsResult listConsumerGroupOffsets =
> client.listConsumerGroupOffsets("test");
> KafkaFuture<Map<TopicPartition, OffsetAndMetadata>>
> partitionsToOffsetAndMetadata = listConsumerGroupOffsets
> .partitionsToOffsetAndMetadata();
> Map<TopicPartition, OffsetAndMetadata> map =
> partitionsToOffsetAndMetadata.get();
> Iterator<TopicPartition> iterator =
> map.keySet().iterator();
> while (iterator.hasNext()) {
> TopicPartition key = iterator.next();
> OffsetAndMetadata value = map.get(key);
> System.out.println(key.toString() + " " +
> value.toString());
> }
> }
>
> but i not find PARTITION,CURRENT-OFFSET,LOG-END-OFFSET
> Corresponding java method
>
>
> ------------------ 原始邮件 ------------------
> 发件人: "Liam Clarke-Hutchinson"<liam.clarke@adscale.co.nz>;
> 发送时间: 2020年4月22日(星期三) 下午3:35
> 收件人: "users"<users@kafka.apache.org>;
>
> 主题: Re: thank you ! which java-client api can has same effect about
> kafka-consumer-groups.sh command ?
>
>
>
> Looking at the source code, try listConsumerGroupOffsets(String
> groupId, ListConsumerGroupOffsetsOptions options) instead?
>
> On Wed, Apr 22, 2020 at 6:40 PM 一直以来 <279377921@qq.com> wrote:
>
> > ./kafka-consumer-groups.sh --bootstrap-server localhost:9081
> --describe
> > --group test
> >
> >
> > use describeConsumerGroups method ??
> >
> >
> > private static void
> print() throws InterruptedException,
> > ExecutionException {
> >
> Properties props = new Properties();
> >
> props.setProperty("bootstrap.servers",
> >
> "192.168.1.100:9081,192.168.1.100:9082,
> > 192.168.1.100:9083,,192.168.1.100:9087,,192.168.1.100:9088");
> >
> AdminClient client = KafkaAdminClient.create(props);
> >
> DescribeConsumerGroupsResult describeConsumerGroups =
> > client.describeConsumerGroups(Arrays.asList("test"));
> >
> Map<String, KafkaFuture<ConsumerGroupDescription&gt;&gt;
> > describedGroups = describeConsumerGroups.describedGroups();
> >
> Iterator<String&gt; iterator =
> > describedGroups.keySet().iterator();
> >
> while (iterator.hasNext()) {
> >
> String key = iterator.next();
> >
> KafkaFuture<ConsumerGroupDescription&gt; value =
> > describedGroups.get(key);
> >
> ConsumerGroupDescription consumerGroupDescription
> > = value.get();
> >
> Collection<MemberDescription&gt; members =
> > consumerGroupDescription.members();
> >
> }
> >
> >
> > }
> >
> >
> >
> > but i can't find about any method about bottom column:
> > GROUP&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
> &nbsp;TOPIC&nbsp; &nbsp; &nbsp;
> > &nbsp; &nbsp; &nbsp;PARTITION&nbsp;
> CURRENT-OFFSET&nbsp;
> > LOG-END-OFFSET&nbsp; LAG&nbsp; &nbsp; &nbsp;
> &nbsp; &nbsp; &nbsp;
> > &nbsp;CONSUMER-ID&nbsp; &nbsp; &nbsp;HOST&nbsp;
> &nbsp; &nbsp; &nbsp; &nbsp;
> > &nbsp; CLIENT-ID
> >
> >
> >
> > By its three info :
> > PARTITION&nbsp; CURRENT-OFFSET&nbsp; LOG-END-OFFSET&nbsp;
回复: thank you ! which java-client api can has same effect about kafka-consumer-groups.sh command ?
Posted by 一直以来 <27...@qq.com>.
i use :
private static void printConsumerGroupOffsets() throws InterruptedException, ExecutionException {
Properties props = new Properties();
props.setProperty("bootstrap.servers",
"192.168.1.100:9081,192.168.1.100:9082,192.168.1.100:9083,,192.168.1.100:9087,,192.168.1.100:9088");
AdminClient client = KafkaAdminClient.create(props);
ListConsumerGroupOffsetsResult listConsumerGroupOffsets = client.listConsumerGroupOffsets("test");
KafkaFuture<Map<TopicPartition, OffsetAndMetadata>> partitionsToOffsetAndMetadata = listConsumerGroupOffsets
.partitionsToOffsetAndMetadata();
Map<TopicPartition, OffsetAndMetadata> map = partitionsToOffsetAndMetadata.get();
Iterator<TopicPartition> iterator = map.keySet().iterator();
while (iterator.hasNext()) {
TopicPartition key = iterator.next();
OffsetAndMetadata value = map.get(key);
System.out.println(key.toString() + " " + value.toString());
}
}
but i not find PARTITION,CURRENT-OFFSET,LOG-END-OFFSET Corresponding java method
------------------ 原始邮件 ------------------
发件人: "Liam Clarke-Hutchinson"<liam.clarke@adscale.co.nz>;
发送时间: 2020年4月22日(星期三) 下午3:35
收件人: "users"<users@kafka.apache.org>;
主题: Re: thank you ! which java-client api can has same effect about kafka-consumer-groups.sh command ?
Looking at the source code, try listConsumerGroupOffsets(String
groupId, ListConsumerGroupOffsetsOptions options) instead?
On Wed, Apr 22, 2020 at 6:40 PM 一直以来 <279377921@qq.com> wrote:
> ./kafka-consumer-groups.sh --bootstrap-server localhost:9081 --describe
> --group test
>
>
> use describeConsumerGroups method ??
>
>
> private static void print() throws InterruptedException,
> ExecutionException {
> Properties props = new Properties();
> props.setProperty("bootstrap.servers",
> "192.168.1.100:9081,192.168.1.100:9082,
> 192.168.1.100:9083,,192.168.1.100:9087,,192.168.1.100:9088");
> AdminClient client = KafkaAdminClient.create(props);
> DescribeConsumerGroupsResult describeConsumerGroups =
> client.describeConsumerGroups(Arrays.asList("test"));
> Map<String, KafkaFuture<ConsumerGroupDescription&gt;&gt;
> describedGroups = describeConsumerGroups.describedGroups();
> Iterator<String&gt; iterator =
> describedGroups.keySet().iterator();
> while (iterator.hasNext()) {
> String key = iterator.next();
> KafkaFuture<ConsumerGroupDescription&gt; value =
> describedGroups.get(key);
> ConsumerGroupDescription consumerGroupDescription
> = value.get();
> Collection<MemberDescription&gt; members =
> consumerGroupDescription.members();
> }
>
>
> }
>
>
>
> but i can't find about any method about bottom column:
> GROUP&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;TOPIC&nbsp; &nbsp; &nbsp;
> &nbsp; &nbsp; &nbsp;PARTITION&nbsp; CURRENT-OFFSET&nbsp;
> LOG-END-OFFSET&nbsp; LAG&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
> &nbsp;CONSUMER-ID&nbsp; &nbsp; &nbsp;HOST&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
> &nbsp; CLIENT-ID
>
>
>
> By its three info :
> PARTITION&nbsp; CURRENT-OFFSET&nbsp; LOG-END-OFFSET&nbsp;
Re: thank you ! which java-client api can has same effect about
kafka-consumer-groups.sh command ?
Posted by Liam Clarke-Hutchinson <li...@adscale.co.nz>.
Yep, looking at the source code of our app we use to track lag, we're using
that method.
On Wed, Apr 22, 2020 at 7:35 PM Liam Clarke-Hutchinson <
liam.clarke@adscale.co.nz> wrote:
> Looking at the source code, try listConsumerGroupOffsets(String
> groupId, ListConsumerGroupOffsetsOptions options) instead?
>
> On Wed, Apr 22, 2020 at 6:40 PM 一直以来 <27...@qq.com> wrote:
>
>> ./kafka-consumer-groups.sh --bootstrap-server localhost:9081 --describe
>> --group test
>>
>>
>> use describeConsumerGroups method ??
>>
>>
>> private static void print() throws InterruptedException,
>> ExecutionException {
>> Properties props = new Properties();
>> props.setProperty("bootstrap.servers",
>> "192.168.1.100:9081,192.168.1.100:9082,
>> 192.168.1.100:9083,,192.168.1.100:9087,,192.168.1.100:9088");
>> AdminClient client = KafkaAdminClient.create(props);
>> DescribeConsumerGroupsResult describeConsumerGroups =
>> client.describeConsumerGroups(Arrays.asList("test"));
>> Map<String, KafkaFuture<ConsumerGroupDescription>>
>> describedGroups = describeConsumerGroups.describedGroups();
>> Iterator<String> iterator =
>> describedGroups.keySet().iterator();
>> while (iterator.hasNext()) {
>> String key = iterator.next();
>> KafkaFuture<ConsumerGroupDescription> value =
>> describedGroups.get(key);
>> ConsumerGroupDescription consumerGroupDescription
>> = value.get();
>> Collection<MemberDescription> members =
>> consumerGroupDescription.members();
>> }
>>
>>
>> }
>>
>>
>>
>> but i can't find about any method about bottom column:
>> GROUP TOPIC
>> PARTITION CURRENT-OFFSET
>> LOG-END-OFFSET LAG
>> CONSUMER-ID HOST
>> CLIENT-ID
>>
>>
>>
>> By its three info :
>> PARTITION CURRENT-OFFSET LOG-END-OFFSET
>
>
Re: thank you ! which java-client api can has same effect about
kafka-consumer-groups.sh command ?
Posted by Liam Clarke-Hutchinson <li...@adscale.co.nz>.
Looking at the source code, try listConsumerGroupOffsets(String
groupId, ListConsumerGroupOffsetsOptions options) instead?
On Wed, Apr 22, 2020 at 6:40 PM 一直以来 <27...@qq.com> wrote:
> ./kafka-consumer-groups.sh --bootstrap-server localhost:9081 --describe
> --group test
>
>
> use describeConsumerGroups method ??
>
>
> private static void print() throws InterruptedException,
> ExecutionException {
> Properties props = new Properties();
> props.setProperty("bootstrap.servers",
> "192.168.1.100:9081,192.168.1.100:9082,
> 192.168.1.100:9083,,192.168.1.100:9087,,192.168.1.100:9088");
> AdminClient client = KafkaAdminClient.create(props);
> DescribeConsumerGroupsResult describeConsumerGroups =
> client.describeConsumerGroups(Arrays.asList("test"));
> Map<String, KafkaFuture<ConsumerGroupDescription>>
> describedGroups = describeConsumerGroups.describedGroups();
> Iterator<String> iterator =
> describedGroups.keySet().iterator();
> while (iterator.hasNext()) {
> String key = iterator.next();
> KafkaFuture<ConsumerGroupDescription> value =
> describedGroups.get(key);
> ConsumerGroupDescription consumerGroupDescription
> = value.get();
> Collection<MemberDescription> members =
> consumerGroupDescription.members();
> }
>
>
> }
>
>
>
> but i can't find about any method about bottom column:
> GROUP TOPIC
> PARTITION CURRENT-OFFSET
> LOG-END-OFFSET LAG
> CONSUMER-ID HOST
> CLIENT-ID
>
>
>
> By its three info :
> PARTITION CURRENT-OFFSET LOG-END-OFFSET