[jira] [Commented] (KAFKA-4633) Always use regex pattern subscription to avoid auto create topics

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-4633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15822839#comment-15822839 ] 

ASF GitHub Bot commented on KAFKA-4633:
---------------------------------------

GitHub user guozhangwang opened a pull request:

    https://github.com/apache/kafka/pull/2379

    KAFKA-4633: Always using regex pattern subscription in StreamThread

    1. In StreamThread, always use subscribe(Pattern, ..) function in order to avoid sending MetadataRequest with specific topic names and cause brokers to possibly auto-create subscribed topics; the pattern is generated as "topic-1|topic-2..|topic-n".
    
    2. Minor cleanup: changed String[] to List<String> to avoid overloaded functions.
    
    3. Minor cleanup: augmented unit test error message and fixed a potential transient failure in KafkaStreamTest.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/guozhangwang/kafka K4633-regex-pattern

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/2379.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2379
    
----
commit c571fd6fe33155ed355469997c5572b92ccdabab
Author: Guozhang Wang <wa...@gmail.com>
Date:   2017-01-14T08:17:37Z

    enforce always using regex pattern subscription

----


> Always use regex pattern subscription to avoid auto create topics
> -----------------------------------------------------------------
>
>                 Key: KAFKA-4633
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4633
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>            Reporter: Guozhang Wang
>            Assignee: Guozhang Wang
>              Labels: architecture
>
> In {{KafkaConsumer}}, a metadata update is requested whenever {{subscribe(List<String> topics ..)}} is called. And when such a metadata request is sent to the broker upon the first {{poll}} call, it will cause the broker to auto-create any topics that do not exist if the broker-side config {{topic.auto.create}} is turned on.
> In order to work around this issue until the config is default to false and gradually be deprecated, we will let Streams to always use the other {{subscribe}} function with regex pattern, which will send the metadata request with empty topic list and hence won't trigger broker-side auto topic creation.
> The side-effect is that the metadata response will be larger, since it contains all the topic infos; but since we only refresh it infrequently this will add negligible overhead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)