You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Stevo Slavić <ss...@gmail.com> on 2016/04/01 17:52:03 UTC
New async producer NetworkClient endlessly trying to fetch metadata
for deleted topics
Hello Apache Kafka community,
With kafka-clients 0.9.0.1, async producer doesn't seem to handle deleted
topics well.
If I create a topic, publish few messages to it, and delete it,
KafkaProducer instance and underlying NetworkClient which was used to
publish to that topic, will endlessly every 10 seconds log warnings that it
cannot fetch metadata for partitions of deleted topics.
I searched in JIRA but couldn't find related ticket. Is this a known issue?
Kind regards,
Stevo Slavic.
Re: New async producer NetworkClient endlessly trying to fetch
metadata for deleted topics
Posted by Stevo Slavić <ss...@gmail.com>.
Thanks for sharing Ismael!
On Fri, Apr 1, 2016 at 6:24 PM, Ismael Juma <is...@juma.me.uk> wrote:
> See:
>
> https://github.com/apache/kafka/pull/645
> https://github.com/apache/kafka/pull/735
>
> Ismael
>
> On Fri, Apr 1, 2016 at 5:04 PM, Stevo Slavić <ss...@gmail.com> wrote:
>
> > Correction, it's much more often logging these warnings - every 100ms.
> > Current workaround is to regularly deploy fresh instance of app using
> > KafkaProducer, or recreate KafkaProducer instances, so that cache of
> > topics/partitions in it gets invalidated, cleared from deleted topics.
> >
> > On Fri, Apr 1, 2016 at 5:52 PM, Stevo Slavić <ss...@gmail.com> wrote:
> >
> > > Hello Apache Kafka community,
> > >
> > > With kafka-clients 0.9.0.1, async producer doesn't seem to handle
> deleted
> > > topics well.
> > >
> > > If I create a topic, publish few messages to it, and delete it,
> > > KafkaProducer instance and underlying NetworkClient which was used to
> > > publish to that topic, will endlessly every 10 seconds log warnings
> that
> > it
> > > cannot fetch metadata for partitions of deleted topics.
> > >
> > > I searched in JIRA but couldn't find related ticket. Is this a known
> > issue?
> > >
> > > Kind regards,
> > > Stevo Slavic.
> > >
> >
>
Re: New async producer NetworkClient endlessly trying to fetch
metadata for deleted topics
Posted by Ismael Juma <is...@juma.me.uk>.
See:
https://github.com/apache/kafka/pull/645
https://github.com/apache/kafka/pull/735
Ismael
On Fri, Apr 1, 2016 at 5:04 PM, Stevo Slavić <ss...@gmail.com> wrote:
> Correction, it's much more often logging these warnings - every 100ms.
> Current workaround is to regularly deploy fresh instance of app using
> KafkaProducer, or recreate KafkaProducer instances, so that cache of
> topics/partitions in it gets invalidated, cleared from deleted topics.
>
> On Fri, Apr 1, 2016 at 5:52 PM, Stevo Slavić <ss...@gmail.com> wrote:
>
> > Hello Apache Kafka community,
> >
> > With kafka-clients 0.9.0.1, async producer doesn't seem to handle deleted
> > topics well.
> >
> > If I create a topic, publish few messages to it, and delete it,
> > KafkaProducer instance and underlying NetworkClient which was used to
> > publish to that topic, will endlessly every 10 seconds log warnings that
> it
> > cannot fetch metadata for partitions of deleted topics.
> >
> > I searched in JIRA but couldn't find related ticket. Is this a known
> issue?
> >
> > Kind regards,
> > Stevo Slavic.
> >
>
Re: New async producer NetworkClient endlessly trying to fetch
metadata for deleted topics
Posted by Stevo Slavić <ss...@gmail.com>.
Correction, it's much more often logging these warnings - every 100ms.
Current workaround is to regularly deploy fresh instance of app using
KafkaProducer, or recreate KafkaProducer instances, so that cache of
topics/partitions in it gets invalidated, cleared from deleted topics.
On Fri, Apr 1, 2016 at 5:52 PM, Stevo Slavić <ss...@gmail.com> wrote:
> Hello Apache Kafka community,
>
> With kafka-clients 0.9.0.1, async producer doesn't seem to handle deleted
> topics well.
>
> If I create a topic, publish few messages to it, and delete it,
> KafkaProducer instance and underlying NetworkClient which was used to
> publish to that topic, will endlessly every 10 seconds log warnings that it
> cannot fetch metadata for partitions of deleted topics.
>
> I searched in JIRA but couldn't find related ticket. Is this a known issue?
>
> Kind regards,
> Stevo Slavic.
>