[jira] [Commented] (KAFKA-1893) Allow regex subscriptions in the new consumer

["Ashish K Singh (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14660584#comment-14660584 ] 

Ashish K Singh commented on KAFKA-1893:
---------------------------------------

Should we support topic and regex subscription at same time? There are a few special cases that will need attention if we decide to have regex and topic subscription work at same time. Say one subscribed to test* and then later explicitly unsubscribed from test1. test1 will get subscribed the next time regex check is run. This can be handled by maintaining a blacklist for a particular pattern. The point is we can support that, however it will have complicated logic. Wondering what will we gain from this complicated logic though. One can have a pattern to subscribe to individual topic or a collection of topics or both. Having support for topic and regex subscription does not buy a lot. If nobody suggests otherwise I will go with making topic and regex subscription mutually exclusive.

Also, in my current implementation I am only supporting subscribing to a single pattern and not multiple patterns as multiple patterns can be combined into a single pattern. Hope that is fine as well.

> Allow regex subscriptions in the new consumer
> ---------------------------------------------
>
>                 Key: KAFKA-1893
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1893
>             Project: Kafka
>          Issue Type: Sub-task
>          Components: consumer
>            Reporter: Jay Kreps
>            Assignee: Ashish K Singh
>            Priority: Critical
>             Fix For: 0.8.3
>
>
> The consumer needs to handle subscribing to regular expressions. Presumably this would be done as a new api,
> {code}
>   void subscribe(java.util.regex.Pattern pattern);
> {code}
> Some questions/thoughts to work out:
>  - It should not be possible to mix pattern subscription with partition subscription.
>  - Is it allowable to mix this with normal topic subscriptions? Logically this is okay but a bit complex to implement.
>  - We need to ensure we regularly update the metadata and recheck our regexes against the metadata to update subscriptions for new topics that are created or old topics that are deleted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)