You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by Narayanan A R <na...@gmail.com> on 2012/05/17 22:55:35 UTC
Listing the znodes efficiently
Hi,
Does ZK has or have plans to add the following features:
- Query znodes under a path by passing filter on the attributes.
- Ability to paginate the znodes under a given path.
Without this, the app has to pull all the nodes and perform the comparison
at the client end.
Regards,
ARN
Re: Listing the znodes efficiently
Posted by Flavio Junqueira <fp...@yahoo-inc.com>.
One pointer for this discussion is the HierarchicalLedgerManager in BookKeeper, org.apache.bookkeeper.meta.
-Flavio
On May 18, 2012, at 2:26 AM, Patrick Hunt wrote:
> On Thu, May 17, 2012 at 5:04 PM, Narayanan Arunachalam
> <na...@gmail.com> wrote:
>> Another approach to improve the performance:
>> For example creating znode to represent something "/abc001"
>> Compute hash and index of the slot for the resource name using HashMap like hash() and indexFor() methods. So if the slot size is 10 this could result in a number between 0 and 9. Assume we get 4 for this path.
>> Create znode /4/abc001
>> So the nodes gets distributed like in a HashMap.
>> For listing:
>> Do a getChildren() on /0 and /1 and so on to bring back the desired results.
>
> Typically users who need this functionality implement it in client side code.
>
> Patrick
>
>> On May 17, 2012, at 3:31 PM, Patrick Hunt <ph...@apache.org> wrote:
>>
>>> On Thu, May 17, 2012 at 1:55 PM, Narayanan A R
>>> <na...@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> Does ZK has or have plans to add the following features:
>>>>
>>>> - Query znodes under a path by passing filter on the attributes.
>>>
>>> Sounds intriguing, depending on the details. Might be a good addition
>>> to the multi* feature. (not discussed before afaik)
>>>
>>>> - Ability to paginate the znodes under a given path.
>>>
>>> Check jira/mailarchives - this has definitely been discussed in the
>>> past. (Henry iirc... aha, see here
>>> https://issues.apache.org/jira/browse/ZOOKEEPER-423 but search around,
>>> you'll likely find more)
>>>
>>>> Without this, the app has to pull all the nodes and perform the comparison
>>>> at the client end.
>>>
>>> Yes, that's true of what's available today.
>>>
>>> Patrick
flavio
junqueira
senior research scientist
fpj@yahoo-inc.com
direct +34 93-183-8828
avinguda diagonal 177, 8th floor, barcelona, 08018, es
phone (408) 349 3300 fax (408) 349 3301
Re: Listing the znodes efficiently
Posted by Patrick Hunt <ph...@apache.org>.
On Thu, May 17, 2012 at 5:04 PM, Narayanan Arunachalam
<na...@gmail.com> wrote:
> Another approach to improve the performance:
> For example creating znode to represent something "/abc001"
> Compute hash and index of the slot for the resource name using HashMap like hash() and indexFor() methods. So if the slot size is 10 this could result in a number between 0 and 9. Assume we get 4 for this path.
> Create znode /4/abc001
> So the nodes gets distributed like in a HashMap.
> For listing:
> Do a getChildren() on /0 and /1 and so on to bring back the desired results.
Typically users who need this functionality implement it in client side code.
Patrick
> On May 17, 2012, at 3:31 PM, Patrick Hunt <ph...@apache.org> wrote:
>
>> On Thu, May 17, 2012 at 1:55 PM, Narayanan A R
>> <na...@gmail.com> wrote:
>>> Hi,
>>>
>>> Does ZK has or have plans to add the following features:
>>>
>>> - Query znodes under a path by passing filter on the attributes.
>>
>> Sounds intriguing, depending on the details. Might be a good addition
>> to the multi* feature. (not discussed before afaik)
>>
>>> - Ability to paginate the znodes under a given path.
>>
>> Check jira/mailarchives - this has definitely been discussed in the
>> past. (Henry iirc... aha, see here
>> https://issues.apache.org/jira/browse/ZOOKEEPER-423 but search around,
>> you'll likely find more)
>>
>>> Without this, the app has to pull all the nodes and perform the comparison
>>> at the client end.
>>
>> Yes, that's true of what's available today.
>>
>> Patrick
Re: Listing the znodes efficiently
Posted by Narayanan Arunachalam <na...@gmail.com>.
Another approach to improve the performance:
For example creating znode to represent something "/abc001"
Compute hash and index of the slot for the resource name using HashMap like hash() and indexFor() methods. So if the slot size is 10 this could result in a number between 0 and 9. Assume we get 4 for this path.
Create znode /4/abc001
So the nodes gets distributed like in a HashMap.
For listing:
Do a getChildren() on /0 and /1 and so on to bring back the desired results.
On May 17, 2012, at 3:31 PM, Patrick Hunt <ph...@apache.org> wrote:
> On Thu, May 17, 2012 at 1:55 PM, Narayanan A R
> <na...@gmail.com> wrote:
>> Hi,
>>
>> Does ZK has or have plans to add the following features:
>>
>> - Query znodes under a path by passing filter on the attributes.
>
> Sounds intriguing, depending on the details. Might be a good addition
> to the multi* feature. (not discussed before afaik)
>
>> - Ability to paginate the znodes under a given path.
>
> Check jira/mailarchives - this has definitely been discussed in the
> past. (Henry iirc... aha, see here
> https://issues.apache.org/jira/browse/ZOOKEEPER-423 but search around,
> you'll likely find more)
>
>> Without this, the app has to pull all the nodes and perform the comparison
>> at the client end.
>
> Yes, that's true of what's available today.
>
> Patrick
Re: Listing the znodes efficiently
Posted by Patrick Hunt <ph...@apache.org>.
On Thu, May 17, 2012 at 1:55 PM, Narayanan A R
<na...@gmail.com> wrote:
> Hi,
>
> Does ZK has or have plans to add the following features:
>
> - Query znodes under a path by passing filter on the attributes.
Sounds intriguing, depending on the details. Might be a good addition
to the multi* feature. (not discussed before afaik)
> - Ability to paginate the znodes under a given path.
Check jira/mailarchives - this has definitely been discussed in the
past. (Henry iirc... aha, see here
https://issues.apache.org/jira/browse/ZOOKEEPER-423 but search around,
you'll likely find more)
> Without this, the app has to pull all the nodes and perform the comparison
> at the client end.
Yes, that's true of what's available today.
Patrick