When publishing to non existing topic, TimeoutException is thrown instead of UnknownTopicOrPartitionException

[Stevo Slavić <ss...@gmail.com>]

Hello Apache Kafka community,

When using new async KafkaProducer, one can register callback with send
calls.

With auto.create.topics.enable set to false, when I try to publish to non
existing topic, I expect callback to complete with
UnknownTopicOrPartitionException. Instead, I get back
"org.apache.kafka.common.errors.TimeoutException: Failed to update metadata
after..."

When topic doesn't truly exist (so metadata request reached broker and
response included error that topic is unknown) I would like to handle that
case differently, than when there are e.g. networking problems like when
metadata response was not received on time and timeout exception is
appropriate.

I've reproduced this unwanted behavior with both Kafka 0.9.0.1 and 0.10.1.1

Is this a feature or a bug? If feature, would it make sense to improve it
in this case to throw UnknownTopicOrPartitionException instead?

Kind regards,
Stevo Slavic.

Re: When publishing to non existing topic, TimeoutException is thrown instead of UnknownTopicOrPartitionException

[Ewen Cheslack-Postava <ew...@confluent.io>]

Stevo,

Agreed that this seems broken if we're just timing out trying to fetch
metadata if we should be able to tell that the topic will never be created.

Clients can't explicitly tell whether auto topic creation is on. Implicit
indication via the error code seems like a good idea. My only concern is
what happens in clusters where that setting is mixed (whether because it is
being turned on (I hope not) or turned off (I hope so, but I want things to
behave nicely)).

-Ewen

On Fri, Jan 27, 2017 at 5:43 AM, Stevo Slavić <ss...@gmail.com> wrote:

> Found somewhat related ticket https://issues.apache.org/
> jira/browse/KAFKA-4385
>
> On Fri, Jan 27, 2017 at 1:09 PM, Stevo Slavić <ss...@gmail.com> wrote:
>
> > Hello Apache Kafka community,
> >
> > When using new async KafkaProducer, one can register callback with send
> > calls.
> >
> > With auto.create.topics.enable set to false, when I try to publish to non
> > existing topic, I expect callback to complete with
> > UnknownTopicOrPartitionException. Instead, I get back
> > "org.apache.kafka.common.errors.TimeoutException: Failed to update
> > metadata after..."
> >
> > When topic doesn't truly exist (so metadata request reached broker and
> > response included error that topic is unknown) I would like to handle
> that
> > case differently, than when there are e.g. networking problems like when
> > metadata response was not received on time and timeout exception is
> > appropriate.
> >
> > I've reproduced this unwanted behavior with both Kafka 0.9.0.1 and
> 0.10.1.1
> >
> > Is this a feature or a bug? If feature, would it make sense to improve it
> > in this case to throw UnknownTopicOrPartitionException instead?
> >
> > Kind regards,
> > Stevo Slavic.
> >
>

Re: When publishing to non existing topic, TimeoutException is thrown instead of UnknownTopicOrPartitionException

[Stevo Slavić <ss...@gmail.com>]

Found somewhat related ticket https://issues.apache.org/
jira/browse/KAFKA-4385

On Fri, Jan 27, 2017 at 1:09 PM, Stevo Slavić <ss...@gmail.com> wrote:

> Hello Apache Kafka community,
>
> When using new async KafkaProducer, one can register callback with send
> calls.
>
> With auto.create.topics.enable set to false, when I try to publish to non
> existing topic, I expect callback to complete with
> UnknownTopicOrPartitionException. Instead, I get back
> "org.apache.kafka.common.errors.TimeoutException: Failed to update
> metadata after..."
>
> When topic doesn't truly exist (so metadata request reached broker and
> response included error that topic is unknown) I would like to handle that
> case differently, than when there are e.g. networking problems like when
> metadata response was not received on time and timeout exception is
> appropriate.
>
> I've reproduced this unwanted behavior with both Kafka 0.9.0.1 and 0.10.1.1
>
> Is this a feature or a bug? If feature, would it make sense to improve it
> in this case to throw UnknownTopicOrPartitionException instead?
>
> Kind regards,
> Stevo Slavic.
>