You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Sachin Mittal <sj...@gmail.com> on 2017/08/06 12:27:29 UTC
Why does kafka-consumer-groups show the topics written to too in its describe
Hi,
I am executing following command
bin/kafka-consumer-groups.sh --new-consumer --bootstrap-server
localhost:9092 --describe --group new-part-advice
It gives output like
GROUP TOPIC PARTITION
CURRENT-OFFSET LOG-END-OFFSET LAG OWNER
new-part-advice advice-stream 8
114548551 114548853 302
84-StreamThread-3-consumer_/10.9.56.84
new-part-advice fname-stream 1 584
610 26
84-StreamThread-4-consumer_/10.9.56.84
.........................
My pipeline is:
KStream<String, BeaconNode> input = builder.stream(Serdes.String(),
beaconSerde, "advice-stream");
input.
....
foreach();
input.
....
.through(Serdes.String(), valueSerde, "fname-stream");
So I don't understand why it is showing topic partitions from fname-stream
in describe, as the process is just writing to that topic and not consuming
from it.
Also what does lag in the case mean?
Thanks
Sachin
Re: Why does kafka-consumer-groups show the topics written to too in
its describe
Posted by Kamal Chandraprakash <ka...@gmail.com>.
`through` = `to` + `stream` operation. So, the consumer-groups command
showing the "fname-stream" topic.
Use `to`, if you just want to write the output to the topic.
-- Kamal
On Mon, Aug 21, 2017 at 12:05 PM, Sachin Mittal <sj...@gmail.com> wrote:
> Folks any thoughts on this.
> Basically I want to know on what topics does consumer group command reports
> on.
>
> I always thought it would only be the topics streams application consumes
> from and not write to.
>
> Any inputs or any part of code I can look at to understand this better
> would be helpful.
>
> Thanks
> Sachin
>
>
> On Sun, Aug 6, 2017 at 5:57 PM, Sachin Mittal <sj...@gmail.com> wrote:
>
> > Hi,
> > I am executing following command
> > bin/kafka-consumer-groups.sh --new-consumer --bootstrap-server
> > localhost:9092 --describe --group new-part-advice
> >
> > It gives output like
> >
> > GROUP TOPIC PARTITION
> > CURRENT-OFFSET LOG-END-OFFSET LAG OWNER
> > new-part-advice advice-stream 8
> > 114548551 114548853 302
> > 84-StreamThread-3-consumer_/10.9.56.84
> > new-part-advice fname-stream 1 584
> > 610 26
> > 84-StreamThread-4-consumer_/10.9.56.84
> > .........................
> >
> > My pipeline is:
> > KStream<String, BeaconNode> input = builder.stream(Serdes.String(),
> > beaconSerde, "advice-stream");
> >
> > input.
> > ....
> > foreach();
> >
> >
> > input.
> > ....
> > .through(Serdes.String(), valueSerde, "fname-stream");
> >
> > So I don't understand why it is showing topic partitions from
> fname-stream
> > in describe, as the process is just writing to that topic and not
> consuming
> > from it.
> > Also what does lag in the case mean?
> >
> > Thanks
> > Sachin
> >
> >
>
Re: Why does kafka-consumer-groups show the topics written to too in
its describe
Posted by Sachin Mittal <sj...@gmail.com>.
Folks any thoughts on this.
Basically I want to know on what topics does consumer group command reports
on.
I always thought it would only be the topics streams application consumes
from and not write to.
Any inputs or any part of code I can look at to understand this better
would be helpful.
Thanks
Sachin
On Sun, Aug 6, 2017 at 5:57 PM, Sachin Mittal <sj...@gmail.com> wrote:
> Hi,
> I am executing following command
> bin/kafka-consumer-groups.sh --new-consumer --bootstrap-server
> localhost:9092 --describe --group new-part-advice
>
> It gives output like
>
> GROUP TOPIC PARTITION
> CURRENT-OFFSET LOG-END-OFFSET LAG OWNER
> new-part-advice advice-stream 8
> 114548551 114548853 302
> 84-StreamThread-3-consumer_/10.9.56.84
> new-part-advice fname-stream 1 584
> 610 26
> 84-StreamThread-4-consumer_/10.9.56.84
> .........................
>
> My pipeline is:
> KStream<String, BeaconNode> input = builder.stream(Serdes.String(),
> beaconSerde, "advice-stream");
>
> input.
> ....
> foreach();
>
>
> input.
> ....
> .through(Serdes.String(), valueSerde, "fname-stream");
>
> So I don't understand why it is showing topic partitions from fname-stream
> in describe, as the process is just writing to that topic and not consuming
> from it.
> Also what does lag in the case mean?
>
> Thanks
> Sachin
>
>