You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Allen Wang <aw...@netflix.com.INVALID> on 2014/11/21 20:13:37 UTC
BytesInPerSec greater than BytesOutPerSec
We observed that for a topic, BytesIn is greater than BytesOut. We are
under the impression that BytesOut should include replication. The topic
has two replicas for each partitions and all replicas are in sync. Then
BytesOut should be at least same as BytesIn since it always needs to
replicate to a follower. Is that right?
Thanks.
Re: BytesInPerSec greater than BytesOutPerSec
Posted by Guozhang Wang <wa...@gmail.com>.
Which Kafka version are you using?
On Fri, Nov 21, 2014 at 5:29 PM, Allen Wang <aw...@netflix.com.invalid>
wrote:
> Yes, we have consumers for the topic fetching at the same time and we can
> see the its BytesPerSec metric. So we find it hard to explain why
> BytesInPerSec would be greater than BytesOutPerSec on the broker given that
> we have active consumer and replication.
>
>
> On Fri, Nov 21, 2014 at 4:32 PM, Guozhang Wang <wa...@gmail.com> wrote:
>
> > Are you having consumers fetching from these topics at the same time?
> >
> > BytesInPerSec only counts the bytes appended to the log from the produce
> > request, and BytesOutPerSec counts the bytes fetch from fetch requests;
> > hence both replica fetcher and normal consumer fetcher's requests count
> in
> > BytesOutPerSec.
> >
> > Guozhang
> >
> >
> > On Fri, Nov 21, 2014 at 11:13 AM, Allen Wang <aw...@netflix.com.invalid>
> > wrote:
> >
> > > We observed that for a topic, BytesIn is greater than BytesOut. We are
> > > under the impression that BytesOut should include replication. The
> topic
> > > has two replicas for each partitions and all replicas are in sync. Then
> > > BytesOut should be at least same as BytesIn since it always needs to
> > > replicate to a follower. Is that right?
> > >
> > > Thanks.
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>
--
-- Guozhang
Re: BytesInPerSec greater than BytesOutPerSec
Posted by Allen Wang <aw...@netflix.com.INVALID>.
Yes, we have consumers for the topic fetching at the same time and we can
see the its BytesPerSec metric. So we find it hard to explain why
BytesInPerSec would be greater than BytesOutPerSec on the broker given that
we have active consumer and replication.
On Fri, Nov 21, 2014 at 4:32 PM, Guozhang Wang <wa...@gmail.com> wrote:
> Are you having consumers fetching from these topics at the same time?
>
> BytesInPerSec only counts the bytes appended to the log from the produce
> request, and BytesOutPerSec counts the bytes fetch from fetch requests;
> hence both replica fetcher and normal consumer fetcher's requests count in
> BytesOutPerSec.
>
> Guozhang
>
>
> On Fri, Nov 21, 2014 at 11:13 AM, Allen Wang <aw...@netflix.com.invalid>
> wrote:
>
> > We observed that for a topic, BytesIn is greater than BytesOut. We are
> > under the impression that BytesOut should include replication. The topic
> > has two replicas for each partitions and all replicas are in sync. Then
> > BytesOut should be at least same as BytesIn since it always needs to
> > replicate to a follower. Is that right?
> >
> > Thanks.
> >
>
>
>
> --
> -- Guozhang
>
Re: BytesInPerSec greater than BytesOutPerSec
Posted by Guozhang Wang <wa...@gmail.com>.
Are you having consumers fetching from these topics at the same time?
BytesInPerSec only counts the bytes appended to the log from the produce
request, and BytesOutPerSec counts the bytes fetch from fetch requests;
hence both replica fetcher and normal consumer fetcher's requests count in
BytesOutPerSec.
Guozhang
On Fri, Nov 21, 2014 at 11:13 AM, Allen Wang <aw...@netflix.com.invalid>
wrote:
> We observed that for a topic, BytesIn is greater than BytesOut. We are
> under the impression that BytesOut should include replication. The topic
> has two replicas for each partitions and all replicas are in sync. Then
> BytesOut should be at least same as BytesIn since it always needs to
> replicate to a follower. Is that right?
>
> Thanks.
>
--
-- Guozhang