You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Michael Popov <mp...@microsoft.com> on 2014/02/04 20:57:10 UTC
Message latency
Hi,
I am trying to understand what influences message latency in Kafka.
According to the Kafka Replication Design document (https://linkedin.jira.com/secure/attachment/10050/kafka_replication_highlevel_design.pdf):
1. Writes: Once the leader receives the acknowledgment from all replicas in ISR, the message is committed. The leader advances the HW and sends an acknowledgment to the client (page 4)
2. Reads: Only messages up to the HW are exposed to the reader (page 5)
It means message latency depends on how fast followers pulled data from the leader and reported successful data writes.
Is this correct?
Thank you,
Michael Popov
Re: Message latency
Posted by Guozhang Wang <wa...@gmail.com>.
That is correct if your producer ack > 1.
Guozhang
On Tue, Feb 4, 2014 at 11:57 AM, Michael Popov <mp...@microsoft.com> wrote:
> Hi,
>
> I am trying to understand what influences message latency in Kafka.
>
> According to the Kafka Replication Design document (
> https://linkedin.jira.com/secure/attachment/10050/kafka_replication_highlevel_design.pdf
> ):
>
> 1. Writes: Once the leader receives the acknowledgment from all
> replicas in ISR, the message is committed. The leader advances the HW and
> sends an acknowledgment to the client (page 4)
>
> 2. Reads: Only messages up to the HW are exposed to the reader (page
> 5)
>
> It means message latency depends on how fast followers pulled data from
> the leader and reported successful data writes.
> Is this correct?
>
> Thank you,
> Michael Popov
>
>
--
-- Guozhang
Re: Message latency
Posted by Jay Kreps <ja...@gmail.com>.
There are two definitions of latency:
1. How long before the writer gets an acknowledgement for their write. This
depends on the acks setting the producer has as Guozhang says. If acks = 1
we wait just on the leader, if acks=-1 we wait on all "in sync" brokers
(i.e. alive brokers).
2. How long before the consumer gets the message. This always blocks on
full acknowledgement by "in sync" brokers.
So both can be dependent on the latency of the followers, however if a
followers falls behind it will be removed from the in sync set, so there is
a bound.
-Jay
On Tue, Feb 4, 2014 at 11:57 AM, Michael Popov <mp...@microsoft.com> wrote:
> Hi,
>
> I am trying to understand what influences message latency in Kafka.
>
> According to the Kafka Replication Design document (
> https://linkedin.jira.com/secure/attachment/10050/kafka_replication_highlevel_design.pdf
> ):
>
> 1. Writes: Once the leader receives the acknowledgment from all
> replicas in ISR, the message is committed. The leader advances the HW and
> sends an acknowledgment to the client (page 4)
>
> 2. Reads: Only messages up to the HW are exposed to the reader (page
> 5)
>
> It means message latency depends on how fast followers pulled data from
> the leader and reported successful data writes.
> Is this correct?
>
> Thank you,
> Michael Popov
>
>