You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Paul Garner <pa...@rockaboxmedia.com> on 2012/11/22 17:57:00 UTC

changing number of partitions

what happens if I reduce num.partitions in server.properties?

I mean, what happens to the partitions (or messages in them) that already
exist above this number

eg I have 32 partitions, messages have been distributed across them, then I
set num.partitions=16

do the partitions 17-32 stay available to consumers, but new messages are
just distributed across the first 16 from now on?

Re: changing number of partitions

Posted by Neha Narkhede <ne...@gmail.com>.
> I mean, what happens to the partitions (or messages in them) that already
> exist above this number

We do not attempt to re-partition existing data within the partitions.
Discussion on this issue is here -
http://markmail.org/message/iyvtug6hlcykgbva?q=Repartitioning+list:org%2Eapache%2Eincubator%2Ekafka-users

> do the partitions 17-32 stay available to consumers, but new messages are
> just distributed across the first 16 from now on?

In 0.7, the extra partitions will stay available to consumers and the
data within those partitions will eventually be garbage collected by
the server, though
technically the server will never completely delete the directory
associated with the extra partitions.

In 0.8, we don't have the feature that allows you to change the number
of partitions yet, but we do plan to add it soon. JIRA is here -
https://issues.apache.org/jira/browse/KAFKA-347

Thanks,
Neha
On Thu, Nov 22, 2012 at 8:57 AM, Paul Garner
<pa...@rockaboxmedia.com> wrote:
> what happens if I reduce num.partitions in server.properties?
>
> I mean, what happens to the partitions (or messages in them) that already
> exist above this number
>
> eg I have 32 partitions, messages have been distributed across them, then I
> set num.partitions=16
>
> do the partitions 17-32 stay available to consumers, but new messages are
> just distributed across the first 16 from now on?