Kafka 0.8.2 new producer blocking on metadata

[Paul Pearcy <Pa...@blackboard.com>]

Heya,
  Playing around with the 0.8.2-beta producer client. One of my test cases is to ensure producers can deal with Kafka being down when the producer is created. My tests failed miserably because of the default blocking in the producer with regard to metadata.fetch.timeout.ms. The first line of new producer is waitOnMetadata which is blocking.

I can handle this case by loading topic meta on init and setting the timeout value to very low metadata.fetch.timeout.ms and either throwing away messages or creating my own internal queue to buffer.

I'm surprised the metasync isn't done async. If it fails, return that in the future/callback. This way the API could actually be considered safely async and the producer buffer could try to hold on to things until block.on.buffer.full kicks in. You'd probably need a partition callback since numPartitions wouldn't be available.

The implication is that people's apps will work fine if first messages are sent while kafka server is up, however, if kafka is down and they restart their app, the new producer will block all sends and blow things up if you haven't written your app to be aware of this edge case.

Thanks,
Paul
This email and any attachments may contain confidential and proprietary information of Blackboard that is for the sole use of the intended recipient. If you are not the intended recipient, disclosure, copying, re-distribution or other use of any of this information is strictly prohibited. Please immediately notify the sender and delete this transmission if you received this email in error.

Re: Kafka 0.8.2 new producer blocking on metadata

[Paul Pearcy <pp...@gmail.com>]

I double posted by accident, sorry. Have another thread discussing this.
Thanks!
On Dec 22, 2014 11:21 AM, "Jun Rao" <ju...@confluent.io> wrote:

> Yes, that's a potential issue. Perhaps we just need to have a lower default
> value for metadata.fetch.timeout.ms ?
>
> Thanks,
>
> Jun
>
> On Wed, Dec 17, 2014 at 11:10 PM, Paul Pearcy <Pa...@blackboard.com>
> wrote:
>
> > Heya,
> >   Playing around with the 0.8.2-beta producer client. One of my test
> cases
> > is to ensure producers can deal with Kafka being down when the producer
> is
> > created. My tests failed miserably because of the default blocking in the
> > producer with regard to metadata.fetch.timeout.ms. The first line of new
> > producer is waitOnMetadata which is blocking.
> >
> > I can handle this case by loading topic meta on init and setting the
> > timeout value to very low metadata.fetch.timeout.ms and either throwing
> > away messages or creating my own internal queue to buffer.
> >
> > I'm surprised the metasync isn't done async. If it fails, return that in
> > the future/callback. This way the API could actually be considered safely
> > async and the producer buffer could try to hold on to things until
> > block.on.buffer.full kicks in. You'd probably need a partition callback
> > since numPartitions wouldn't be available.
> >
> > The implication is that people's apps will work fine if first messages
> are
> > sent while kafka server is up, however, if kafka is down and they restart
> > their app, the new producer will block all sends and blow things up if
> you
> > haven't written your app to be aware of this edge case.
> >
> > Thanks,
> > Paul
> > This email and any attachments may contain confidential and proprietary
> > information of Blackboard that is for the sole use of the intended
> > recipient. If you are not the intended recipient, disclosure, copying,
> > re-distribution or other use of any of this information is strictly
> > prohibited. Please immediately notify the sender and delete this
> > transmission if you received this email in error.
> >
>

Re: Kafka 0.8.2 new producer blocking on metadata

[Jun Rao <ju...@confluent.io>]

Yes, that's a potential issue. Perhaps we just need to have a lower default
value for metadata.fetch.timeout.ms ?

Thanks,

Jun

On Wed, Dec 17, 2014 at 11:10 PM, Paul Pearcy <Pa...@blackboard.com>
wrote:

> Heya,
>   Playing around with the 0.8.2-beta producer client. One of my test cases
> is to ensure producers can deal with Kafka being down when the producer is
> created. My tests failed miserably because of the default blocking in the
> producer with regard to metadata.fetch.timeout.ms. The first line of new
> producer is waitOnMetadata which is blocking.
>
> I can handle this case by loading topic meta on init and setting the
> timeout value to very low metadata.fetch.timeout.ms and either throwing
> away messages or creating my own internal queue to buffer.
>
> I'm surprised the metasync isn't done async. If it fails, return that in
> the future/callback. This way the API could actually be considered safely
> async and the producer buffer could try to hold on to things until
> block.on.buffer.full kicks in. You'd probably need a partition callback
> since numPartitions wouldn't be available.
>
> The implication is that people's apps will work fine if first messages are
> sent while kafka server is up, however, if kafka is down and they restart
> their app, the new producer will block all sends and blow things up if you
> haven't written your app to be aware of this edge case.
>
> Thanks,
> Paul
> This email and any attachments may contain confidential and proprietary
> information of Blackboard that is for the sole use of the intended
> recipient. If you are not the intended recipient, disclosure, copying,
> re-distribution or other use of any of this information is strictly
> prohibited. Please immediately notify the sender and delete this
> transmission if you received this email in error.
>