Kafka Streams Dynamic Topic consumer

[David Garcia <da...@spiceworks.com>]

Is there any way to specify a dynamic topic list (e.g. like a regex whitelist filter…like in the consumer API) with kafka streams?  We would like to get the benefit of automatic checkpointing and load balancing if possible.

-David

Re: Kafka Streams Dynamic Topic consumer

[Guozhang Wang <wa...@gmail.com>]

Hello David,

Regex subscription is already added in trunk as per
https://issues.apache.org/jira/browse/KAFKA-3443 and will be available in
the 0.10.1.0 release.

If you want to try it out and experiment you can just build from current
trunk directly.


Regarding automatic checkpointing / load balancing, it is enabled even
without regex subscription. Note that since 0.9 we have revamped the
consumer to not based on ZK but using our own membership protocol that
Kafka Streams also leverages on.


Guozhang


On Mon, Jul 18, 2016 at 4:38 PM, David Garcia <da...@spiceworks.com> wrote:

> Is there any way to specify a dynamic topic list (e.g. like a regex
> whitelist filter…like in the consumer API) with kafka streams?  We would
> like to get the benefit of automatic checkpointing and load balancing if
> possible.
>
> -David
>



-- 
-- Guozhang