hierarchical topics

[Jay Kreps <ja...@gmail.com>]

One idea that has occasionally been thrown around is allowing topics to be
hierarchical. The primary goal was just to allow logical grouping as well
as to make it easier to subscribe via a wildcard (it may be easier to give
/search/* rather than (search_click|search_result|...)).

Another problem we have seen is that it can be a pain for people to
physically locate which cluster has their data since we segregate a few use
cases onto separate clusters. It would be nice to have a federated
namespace that puts all of these on a single directory tree.

Both of these problems kind of arise from having lots and lots of topics
and lots of people using the system so they may not be interesting to
everyone.

Nonetheless I wrote up a quick wiki capturing various thoughts people had
thrown around on previous mailing list discussions:
  https://cwiki.apache.org/confluence/display/KAFKA/Hierarchical+Topics

The linkedin folks are pretty busy with replication so i doubt we would
work on this in the very near future.

-Jay