Resource Buckets

[Vinayak Borkar <vb...@yahoo.com>]

Hi,


The addResource call has a variant that lets users set a bucket size. 
What is a bucket? When would I use this feature? I could not find any 
mention of this other than in the Javadoc of that method that says "... 
where bucket size > 1".



Thanks,
Vinayak

Re: Resource Buckets

[kishore g <g....@gmail.com>]

Hi Vinayak,

It useful when there you have large number of partitions. It is used to
split the znodes in zookeeper. Helix by default uses one znode to represent
all partitions in IdealState,CurrentState, ExternalView. This reduces the
number of watches and also allows us to improve performance. How ever
zookeeper has a size limit of 1mb per znode and when we have large number
of partitions we can split the Idealstate,CurrentState, ExternalView into
multiple znodes.

For example if you have 10k partitions, you can set bucket size to 100 and
we create 100 znodes each with 0-99,100-199 etc

Thanks
Kishore G


On Wed, Apr 3, 2013 at 12:47 AM, Vinayak Borkar <vb...@yahoo.com> wrote:

> Hi,
>
>
> The addResource call has a variant that lets users set a bucket size. What
> is a bucket? When would I use this feature? I could not find any mention of
> this other than in the Javadoc of that method that says "... where bucket
> size > 1".
>
>
>
> Thanks,
> Vinayak
>