You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Libo Yu <yu...@hotmail.com> on 2014/10/27 20:20:38 UTC
question about async publishing for 0.8.1
Hi team,
I am testing async publishing + acknowledgement.
Assume all settings are the default and queue.buffering.max.messages is 10k. I use a simple for loop to publish 100k messages. In the process, I unplugged the net cable.
What should I expect in this case?
I assume the send will be blocked when the queue is full. But surprisingly, the send() is not blocked and all msgs are "sent". Thanks.
RE: question about async publishing for 0.8.1
Posted by Libo Yu <yu...@hotmail.com>.
Thanks, Jay. A large retry number can help in this case.
> Date: Mon, 27 Oct 2014 18:12:12 -0700
> Subject: Re: question about async publishing for 0.8.1
> From: jay.kreps@gmail.com
> To: users@kafka.apache.org
>
> You may also need to set the retries to something high, I think. I think
> the default is something like 1 or 3 so it will try a few times then give
> up.
>
> -Jay
>
> On Mon, Oct 27, 2014 at 6:01 PM, Libo Yu <yu...@hotmail.com> wrote:
>
> > This seems to be a bug of 0.8.1
> > More info:
> > queue.enqueue.timeout.ms is explicitly set to -1.
> > queue.buffering.max.messages is explicitly set o 10000.
> >
> > But still, when the net cable was unplugged, the producer did not block at
> > all and
> > all messages were "sent".
> >
> >
> > > From: yu_libo@hotmail.com
> > > To: users@kafka.apache.org
> > > Subject: question about async publishing for 0.8.1
> > > Date: Mon, 27 Oct 2014 15:20:38 -0400
> > >
> > > Hi team,
> > >
> > > I am testing async publishing + acknowledgement.
> > >
> > > Assume all settings are the default and queue.buffering.max.messages is
> > 10k. I use a simple for loop to publish 100k messages. In the process, I
> > unplugged the net cable.
> > >
> > > What should I expect in this case?
> > > I assume the send will be blocked when the queue is full. But
> > surprisingly, the send() is not blocked and all msgs are "sent". Thanks.
> > >
> >
> >
Re: question about async publishing for 0.8.1
Posted by Jay Kreps <ja...@gmail.com>.
You may also need to set the retries to something high, I think. I think
the default is something like 1 or 3 so it will try a few times then give
up.
-Jay
On Mon, Oct 27, 2014 at 6:01 PM, Libo Yu <yu...@hotmail.com> wrote:
> This seems to be a bug of 0.8.1
> More info:
> queue.enqueue.timeout.ms is explicitly set to -1.
> queue.buffering.max.messages is explicitly set o 10000.
>
> But still, when the net cable was unplugged, the producer did not block at
> all and
> all messages were "sent".
>
>
> > From: yu_libo@hotmail.com
> > To: users@kafka.apache.org
> > Subject: question about async publishing for 0.8.1
> > Date: Mon, 27 Oct 2014 15:20:38 -0400
> >
> > Hi team,
> >
> > I am testing async publishing + acknowledgement.
> >
> > Assume all settings are the default and queue.buffering.max.messages is
> 10k. I use a simple for loop to publish 100k messages. In the process, I
> unplugged the net cable.
> >
> > What should I expect in this case?
> > I assume the send will be blocked when the queue is full. But
> surprisingly, the send() is not blocked and all msgs are "sent". Thanks.
> >
>
>
RE: question about async publishing for 0.8.1
Posted by Libo Yu <yu...@hotmail.com>.
This seems to be a bug of 0.8.1
More info:
queue.enqueue.timeout.ms is explicitly set to -1.
queue.buffering.max.messages is explicitly set o 10000.
But still, when the net cable was unplugged, the producer did not block at all and
all messages were "sent".
> From: yu_libo@hotmail.com
> To: users@kafka.apache.org
> Subject: question about async publishing for 0.8.1
> Date: Mon, 27 Oct 2014 15:20:38 -0400
>
> Hi team,
>
> I am testing async publishing + acknowledgement.
>
> Assume all settings are the default and queue.buffering.max.messages is 10k. I use a simple for loop to publish 100k messages. In the process, I unplugged the net cable.
>
> What should I expect in this case?
> I assume the send will be blocked when the queue is full. But surprisingly, the send() is not blocked and all msgs are "sent". Thanks.
>