You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by "M. Manna" <ma...@gmail.com> on 2017/07/08 15:38:10 UTC

Re[2]: Kafka behavior when consuming a topic which doesn't exist?

Please ensure that you set the desired minimum partition and replication 
settings in the server.properties file. These are the ones which will be 
used for auto creation of topics.

KR,

------ Original Message ------
From: "Ali Akhtar" <al...@gmail.com>
To: users@kafka.apache.org
Sent: 7/8/2017 4:27:55 PM
Subject: Re: Kafka behavior when consuming a topic which doesn't exist?

>Oh gotcha, thanks. So a topic will be created if topic creation is 
>enabled.
>
>On Sat, Jul 8, 2017 at 8:14 PM, M. Manna <ma...@gmail.com> wrote:
>
>>Please check my previous email.
>>
>>On Sat, 8 Jul 2017 at 2:32 am, Ali Akhtar <al...@gmail.com> 
>>wrote:
>>
>> > What happens if auto creation is enabled but the topic doesn't 
>>exist.
>> > Consumers subscribe to that topic which doesn't exist. Then messages 
>>are
>> > posted to that topic.
>> >
>> > Will the consumer receive those messages in this scenario?
>> >
>> >
>> >
>> > On 8 Jul 2017 4:38 a.m., "M. Manna" <ma...@gmail.com> wrote:
>> >
>> > That depends.
>> >
>> > If auto creation of non-existent topic enabled (check docs), then it 
>>will
>> > simple use the minimum partiotion and replication settings defined 
>>in
>> > broker config to create a topic. If auto creation is disabled, your
>> > consumer group won't do anything.
>> >
>> > With auto creation enable - It's the same if you publish to a topic 
>>that
>> > doesn't yet exist. But that means topic will get created first, and 
>>then
>> > the messages are sent on the bus. If you subscribe to that topic 
>>later
>>you
>> > will get messages. Check below:
>> >
>> > http://grokbase.com/t/kafka/users/1648kbr04c/subscribe-on-
>> > a-topic-that-does-not-exist
>> >
>> >
>> >
>> > On 7 Jul 2017 9:46 pm, "Ali Akhtar" <al...@gmail.com> wrote:
>> >
>> > > Sometimes I see warnings in my logs if i create a consumer for a 
>>topic
>> > > which doesn't exist. Such as:
>> > >
>> > > org.apache.kafka.clients.NetworkClient  - Error while fetching
>>metadata
>> > > with correlation id 1 : {example_topic=LEADER_NOT_AVAILABLE}
>> > >
>> > > If later messages are posted to that topic (which will create it), 
>>will
>> > my
>> > > consumer receive those messages?
>> > >
>> >
>>