[jira] [Commented] (KAFKA-4633) Always use regex pattern subscription to avoid auto create topics

["Andres Gomez Ferrer (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-4633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16040795#comment-16040795 ] 

Andres Gomez Ferrer commented on KAFKA-4633:
--------------------------------------------

Hi all,

I have some question about this issue. With this change, the Kafka Streams app doesn't auto-create topics, so you need to create the topics before start running the APP. This is ok if you create the input/output topics, but when you use the *through* method to do some repartition .. Do you need to create the topic before? Maybe this is a little strange, isn't?

I have seen the *through* method, and it works with *to(...)* and *stream(...)*

{code}
    @Override
    public KStream<K, V> through(Serde<K> keySerde, Serde<V> valSerde, StreamPartitioner<? super K, ? super V> partitioner, String topic) {
        to(keySerde, valSerde, partitioner, topic);

        return topology.stream(keySerde, valSerde, topic);
    }
{code}

What do you think? Maybe this is a new issue or Is normal that you need to create the topic that later you use in the * through* method?

Regards,
Andrés

> Always use regex pattern subscription to avoid auto create topics
> -----------------------------------------------------------------
>
>                 Key: KAFKA-4633
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4633
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>            Reporter: Guozhang Wang
>            Assignee: Guozhang Wang
>              Labels: architecture
>             Fix For: 0.10.2.0
>
>
> In {{KafkaConsumer}}, a metadata update is requested whenever {{subscribe(List<String> topics ..)}} is called. And when such a metadata request is sent to the broker upon the first {{poll}} call, it will cause the broker to auto-create any topics that do not exist if the broker-side config {{topic.auto.create}} is turned on.
> In order to work around this issue until the config is default to false and gradually be deprecated, we will let Streams to always use the other {{subscribe}} function with regex pattern, which will send the metadata request with empty topic list and hence won't trigger broker-side auto topic creation.
> The side-effect is that the metadata response will be larger, since it contains all the topic infos; but since we only refresh it infrequently this will add negligible overhead.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)