You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by Michal Haris <mi...@visualdna.com> on 2012/11/03 15:09:26 UTC

Partitioning strategies

Hi,

Would it be reasonable to say that deterministic partotioner should always
be deciding on attributes with high cardinality?

I.e. with low cardinality some partitions would end up with large volumes
of messages which could not be scaled horizontally and so in cases like
that separate topics would be appropriate.

Michal

Re: Partitioning strategies

Posted by Jun Rao <ju...@gmail.com>.

Michal,

When partitioning the data, you can put the data in different partitions in
the same topic or different topics. The difference is that in the latter,
the consumer has to manually balance the load across topics.

Thanks,

Jun

On Sat, Nov 3, 2012 at 7:09 AM, Michal Haris <mi...@visualdna.com>wrote:

> Hi,
>
> Would it be reasonable to say that deterministic partotioner should always
> be deciding on attributes with high cardinality?
>
> I.e. with low cardinality some partitions would end up with large volumes
> of messages which could not be scaled horizontally and so in cases like
> that separate topics would be appropriate.
>
> Michal
>