You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by yosefarr <yo...@gmail.com> on 2011/08/11 11:23:16 UTC
Zookeeper indexing
Hi,
how does search for a node or search for part of node works on zookeeper,
is there any indexing mechanism for adding nodes?
Thanks,
Yosef
--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/Zookeeper-indexing-tp6675662p6675662.html
Sent from the zookeeper-user mailing list archive at Nabble.com.
Re: Zookeeper indexing
Posted by Ted Dunning <te...@gmail.com>.
You haven't said enough about your application to answer this question. You might try solr
Sent from my iPad
On Aug 13, 2011, at 2:01 AM, yosefarr <yo...@gmail.com> wrote:
> Thanks a lot that's was very helpful,
> Do you know about any open source that make this indexing?
> Thanks again
> Yosef
>
> Sent from my iPhone
>
> On 11 באוג 2011, at 23:33, "Ted Dunning [via zookeeper-user]"<ml...@n2.nabble.com> wrote:
>
>> Zookeeper only provides access/search via the name of the znode.
>>
>> Any indexing on content has to be done outside of zookeeper although for
>> simple applications, you could keep a reverse index pretty easily inside
>> zookeeper. The idea is that you would have an additional znode for each
>> value of an indexed entry. That znode would contain a list of names for
>> znodes that have that indexed element. Updating the document would involve
>> parsing the new value, reading all the old index znodes, updating the index
>> values and then returning all of this in a multi with version checks. If
>> the multi fails, then you need to re-read the changed index files or
>> somebody changed the basic file underneath you. Deletion would require
>> something comparable.
>>
>> If you are only indexing a few simple things, then this technique would work
>> fairly well. You can do it without multi as well, but you won't have any
>> guarantees of consistency and will probably have to have a background
>> process that rebuilds the index over time.
>>
>> On Thu, Aug 11, 2011 at 1:25 PM, yosefarr <[hidden email]> wrote:
>>
>>> Thank you for your replay,
>>> what i meant is when i have a lots of znodes (thousands) and i want to
>>> find specific child (search in all the nodes), what is the effective way to
>>> do this,
>>> does zookeeper keep any indexing on its nodes?
>>>
>>> Thanks for your help,
>>> Yosef
>>>
>>> On Thu, Aug 11, 2011 at 8:15 PM, Ted Dunning [via zookeeper-user] <
>>> [hidden email]> wrote:
>>>
>>>> Yosef,
>>>>
>>>> I really don't understand the question here.
>>>>
>>>> What do you mean by node? In the Zookeeper world, that can mean a
>>> computer
>>>>
>>>> running a copy of Zookeeper or it can mean a data structure whose name is
>>>> normally a znode.
>>>>
>>>> Which kind of node are you talking about?
>>>>
>>>> Also, assuming you mean znode, which is something a little bit like a
>>> file,
>>>>
>>>> there is no search in Zookeeper other than by name. Zookeeper is not
>>> very
>>>> satisfactory as a content store because it requires that all data be kept
>>>> in
>>>> memory at all times.
>>>>
>>>> Can you say more about what you are trying to do? There might be a good
>>>> answer, but I can't say what it is without knowing more about the goals.
>>>>
>>>> On Thu, Aug 11, 2011 at 2:23 AM, yosefarr <[hidden email]<
>>> http://user/SendEmail.jtp?type=node&node=6677210&i=0>>
>>>> wrote:
>>>>
>>>>> how does search for a node or search for part of node works on
>>> zookeeper,
>>>>
>>>>> is there any indexing mechanism for adding nodes?
>>>>>
>>>>
>>>>
>>>> ------------------------------
>>>> If you reply to this email, your message will be added to the discussion
>>>> below:
>>>>
>>>>
>>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-indexing-tp6675662p6677210.html
>>>> To unsubscribe from Zookeeper indexing, click here<
>>>> .
>>>>
>>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-indexing-tp6675662p6677960.html
>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>
>>
>> If you reply to this email, your message will be added to the discussion below:
>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-indexing-tp6675662p6677983.html
>> To unsubscribe from Zookeeper indexing, click here.
>
>
> --
> View this message in context: http://zookeeper-user.578899.n2.nabble.com/Zookeeper-indexing-tp6675662p6682889.html
> Sent from the zookeeper-user mailing list archive at Nabble.com.
Re: Zookeeper indexing
Posted by yosefarr <yo...@gmail.com>.
Thanks a lot that's was very helpful,
Do you know about any open source that make this indexing?
Thanks again
Yosef
Sent from my iPhone
On 11 באוג 2011, at 23:33, "Ted Dunning [via zookeeper-user]"<ml...@n2.nabble.com> wrote:
> Zookeeper only provides access/search via the name of the znode.
>
> Any indexing on content has to be done outside of zookeeper although for
> simple applications, you could keep a reverse index pretty easily inside
> zookeeper. The idea is that you would have an additional znode for each
> value of an indexed entry. That znode would contain a list of names for
> znodes that have that indexed element. Updating the document would involve
> parsing the new value, reading all the old index znodes, updating the index
> values and then returning all of this in a multi with version checks. If
> the multi fails, then you need to re-read the changed index files or
> somebody changed the basic file underneath you. Deletion would require
> something comparable.
>
> If you are only indexing a few simple things, then this technique would work
> fairly well. You can do it without multi as well, but you won't have any
> guarantees of consistency and will probably have to have a background
> process that rebuilds the index over time.
>
> On Thu, Aug 11, 2011 at 1:25 PM, yosefarr <[hidden email]> wrote:
>
> > Thank you for your replay,
> > what i meant is when i have a lots of znodes (thousands) and i want to
> > find specific child (search in all the nodes), what is the effective way to
> > do this,
> > does zookeeper keep any indexing on its nodes?
> >
> > Thanks for your help,
> > Yosef
> >
> > On Thu, Aug 11, 2011 at 8:15 PM, Ted Dunning [via zookeeper-user] <
> > [hidden email]> wrote:
> >
> > > Yosef,
> > >
> > > I really don't understand the question here.
> > >
> > > What do you mean by node? In the Zookeeper world, that can mean a
> > computer
> > >
> > > running a copy of Zookeeper or it can mean a data structure whose name is
> > > normally a znode.
> > >
> > > Which kind of node are you talking about?
> > >
> > > Also, assuming you mean znode, which is something a little bit like a
> > file,
> > >
> > > there is no search in Zookeeper other than by name. Zookeeper is not
> > very
> > > satisfactory as a content store because it requires that all data be kept
> > > in
> > > memory at all times.
> > >
> > > Can you say more about what you are trying to do? There might be a good
> > > answer, but I can't say what it is without knowing more about the goals.
> > >
> > > On Thu, Aug 11, 2011 at 2:23 AM, yosefarr <[hidden email]<
> > http://user/SendEmail.jtp?type=node&node=6677210&i=0>>
> > > wrote:
> > >
> > > > how does search for a node or search for part of node works on
> > zookeeper,
> > >
> > > > is there any indexing mechanism for adding nodes?
> > > >
> > >
> > >
> > > ------------------------------
> > > If you reply to this email, your message will be added to the discussion
> > > below:
> > >
> > >
> > http://zookeeper-user.578899.n2.nabble.com/Zookeeper-indexing-tp6675662p6677210.html
> > > To unsubscribe from Zookeeper indexing, click here<
> > >.
> > >
> > >
> >
> >
> > --
> > View this message in context:
> > http://zookeeper-user.578899.n2.nabble.com/Zookeeper-indexing-tp6675662p6677960.html
> > Sent from the zookeeper-user mailing list archive at Nabble.com.
>
>
> If you reply to this email, your message will be added to the discussion below:
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-indexing-tp6675662p6677983.html
> To unsubscribe from Zookeeper indexing, click here.
--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/Zookeeper-indexing-tp6675662p6682889.html
Sent from the zookeeper-user mailing list archive at Nabble.com.
Re: Zookeeper indexing
Posted by Ted Dunning <te...@gmail.com>.
Zookeeper only provides access/search via the name of the znode.
Any indexing on content has to be done outside of zookeeper although for
simple applications, you could keep a reverse index pretty easily inside
zookeeper. The idea is that you would have an additional znode for each
value of an indexed entry. That znode would contain a list of names for
znodes that have that indexed element. Updating the document would involve
parsing the new value, reading all the old index znodes, updating the index
values and then returning all of this in a multi with version checks. If
the multi fails, then you need to re-read the changed index files or
somebody changed the basic file underneath you. Deletion would require
something comparable.
If you are only indexing a few simple things, then this technique would work
fairly well. You can do it without multi as well, but you won't have any
guarantees of consistency and will probably have to have a background
process that rebuilds the index over time.
On Thu, Aug 11, 2011 at 1:25 PM, yosefarr <yo...@gmail.com> wrote:
> Thank you for your replay,
> what i meant is when i have a lots of znodes (thousands) and i want to
> find specific child (search in all the nodes), what is the effective way to
> do this,
> does zookeeper keep any indexing on its nodes?
>
> Thanks for your help,
> Yosef
>
> On Thu, Aug 11, 2011 at 8:15 PM, Ted Dunning [via zookeeper-user] <
> ml-node+6677210-1757486836-343064@n2.nabble.com> wrote:
>
> > Yosef,
> >
> > I really don't understand the question here.
> >
> > What do you mean by node? In the Zookeeper world, that can mean a
> computer
> >
> > running a copy of Zookeeper or it can mean a data structure whose name is
> > normally a znode.
> >
> > Which kind of node are you talking about?
> >
> > Also, assuming you mean znode, which is something a little bit like a
> file,
> >
> > there is no search in Zookeeper other than by name. Zookeeper is not
> very
> > satisfactory as a content store because it requires that all data be kept
> > in
> > memory at all times.
> >
> > Can you say more about what you are trying to do? There might be a good
> > answer, but I can't say what it is without knowing more about the goals.
> >
> > On Thu, Aug 11, 2011 at 2:23 AM, yosefarr <[hidden email]<
> http://user/SendEmail.jtp?type=node&node=6677210&i=0>>
> > wrote:
> >
> > > how does search for a node or search for part of node works on
> zookeeper,
> >
> > > is there any indexing mechanism for adding nodes?
> > >
> >
> >
> > ------------------------------
> > If you reply to this email, your message will be added to the discussion
> > below:
> >
> >
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-indexing-tp6675662p6677210.html
> > To unsubscribe from Zookeeper indexing, click here<
> http://zookeeper-user.578899.n2.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=6675662&code=eW9zZWYuYXJyYWZAZ21haWwuY29tfDY2NzU2NjJ8MTI4MTA4Mjk2NQ==
> >.
> >
> >
>
>
> --
> View this message in context:
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-indexing-tp6675662p6677960.html
> Sent from the zookeeper-user mailing list archive at Nabble.com.
Re: Zookeeper indexing
Posted by yosefarr <yo...@gmail.com>.
Thank you for your replay,
what i meant is when i have a lots of znodes (thousands) and i want to
find specific child (search in all the nodes), what is the effective way to
do this,
does zookeeper keep any indexing on its nodes?
Thanks for your help,
Yosef
On Thu, Aug 11, 2011 at 8:15 PM, Ted Dunning [via zookeeper-user] <
ml-node+6677210-1757486836-343064@n2.nabble.com> wrote:
> Yosef,
>
> I really don't understand the question here.
>
> What do you mean by node? In the Zookeeper world, that can mean a computer
>
> running a copy of Zookeeper or it can mean a data structure whose name is
> normally a znode.
>
> Which kind of node are you talking about?
>
> Also, assuming you mean znode, which is something a little bit like a file,
>
> there is no search in Zookeeper other than by name. Zookeeper is not very
> satisfactory as a content store because it requires that all data be kept
> in
> memory at all times.
>
> Can you say more about what you are trying to do? There might be a good
> answer, but I can't say what it is without knowing more about the goals.
>
> On Thu, Aug 11, 2011 at 2:23 AM, yosefarr <[hidden email]<http://user/SendEmail.jtp?type=node&node=6677210&i=0>>
> wrote:
>
> > how does search for a node or search for part of node works on zookeeper,
>
> > is there any indexing mechanism for adding nodes?
> >
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-indexing-tp6675662p6677210.html
> To unsubscribe from Zookeeper indexing, click here<http://zookeeper-user.578899.n2.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=6675662&code=eW9zZWYuYXJyYWZAZ21haWwuY29tfDY2NzU2NjJ8MTI4MTA4Mjk2NQ==>.
>
>
--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/Zookeeper-indexing-tp6675662p6677960.html
Sent from the zookeeper-user mailing list archive at Nabble.com.
Re: Zookeeper indexing
Posted by Ted Dunning <te...@gmail.com>.
Yosef,
I really don't understand the question here.
What do you mean by node? In the Zookeeper world, that can mean a computer
running a copy of Zookeeper or it can mean a data structure whose name is
normally a znode.
Which kind of node are you talking about?
Also, assuming you mean znode, which is something a little bit like a file,
there is no search in Zookeeper other than by name. Zookeeper is not very
satisfactory as a content store because it requires that all data be kept in
memory at all times.
Can you say more about what you are trying to do? There might be a good
answer, but I can't say what it is without knowing more about the goals.
On Thu, Aug 11, 2011 at 2:23 AM, yosefarr <yo...@gmail.com> wrote:
> how does search for a node or search for part of node works on zookeeper,
> is there any indexing mechanism for adding nodes?
>