You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Shailendra Mudgal <mu...@gmail.com> on 2008/04/17 19:45:44 UTC
Document ids collected from HitCollector.collect and used in FieldCache..
Hi All,
I have a small confusion regarding the document ids which we collect using
HitCollector.collect() method. Here is the description of the confusion :
First i created a FieldCache of type <docID, <list of companyIDs>> using a
query which collects all the articles which are only a month old. I am
storing them into a map.
So now my cache has same document id 's associated with their field value
ready in the memory.
Now suppose for a search which has duration less then a month, i get a list
of docIds using hit collector. Can i map these document ids to the map,
which i created earlier. And use those field values???
Any help will be appreciated ....
Regards,
Shailendra
Re: Document ids collected from HitCollector.collect and used in FieldCache..
Posted by Shailendra Mudgal <mu...@gmail.com>.
Thanks a lot Erik. I just wanted to confirm this.
Regards,
On Fri, Apr 18, 2008 at 7:34 PM, Erick Erickson <er...@gmail.com>
wrote:
> They'll be in synch forever unless and until you *change* the index. Once
> you do anything with an IndexWriter, you have to be very careful about
> relying on doc IDs.
>
> But remember that opening a searcher takes a snapshot of the index and
> that reader/searcher will NOT see changes. So you could think about
> building your cache and using it *as long as you haven't closed/reopened
> your index reader*. The cache and search will be in synch for that
> interval.
>
> Best
> Erick
>
> On Fri, Apr 18, 2008 at 9:14 AM, Shailendra Mudgal <
> mudgal.shailendra@gmail.com> wrote:
>
> > Hi Erik Thanks for you prompt reply.
> >
> > So if i refresh the searcher in every one hour and that time itself if i
> > refresh this cache also, is this going to work? I mean will the document
> > ids
> > will be in sync for that one hour.
> >
> >
> > On Fri, Apr 18, 2008 at 2:02 AM, Erick Erickson <erickerickson@gmail.com
> >
> > wrote:
> >
> > > Doc IDs are NOT permanent. If you don't change your index at all
> > > (delete especially, but sometimes adding/optimizing can chage IDs)
> > > then you can re-use them. Otherwise not.
> > >
> > >
> > > On Thu, Apr 17, 2008 at 1:45 PM, Shailendra Mudgal <
> > > mudgal.shailendra@gmail.com> wrote:
> > >
> > > > Hi All,
> > > >
> > > > I have a small confusion regarding the document ids which we collect
> > > using
> > > > HitCollector.collect() method. Here is the description of the
> > confusion
> > > :
> > > >
> > > > First i created a FieldCache of type <docID, <list of companyIDs>>
> > using
> > > a
> > > > query which collects all the articles which are only a month old. I
> am
> > > > storing them into a map.
> > > >
> > > > So now my cache has same document id 's associated with their field
> > > value
> > > > ready in the memory.
> > > >
> > > > Now suppose for a search which has duration less then a month, i get
> a
> > > > list
> > > > of docIds using hit collector. Can i map these document ids to the
> > map,
> > > > which i created earlier. And use those field values???
> > > >
> > > >
> > > > Any help will be appreciated ....
> > > >
> > > > Regards,
> > > > Shailendra
> > > >
> > >
> >
>
Re: Document ids collected from HitCollector.collect and used in FieldCache..
Posted by Erick Erickson <er...@gmail.com>.
They'll be in synch forever unless and until you *change* the index. Once
you do anything with an IndexWriter, you have to be very careful about
relying on doc IDs.
But remember that opening a searcher takes a snapshot of the index and
that reader/searcher will NOT see changes. So you could think about
building your cache and using it *as long as you haven't closed/reopened
your index reader*. The cache and search will be in synch for that interval.
Best
Erick
On Fri, Apr 18, 2008 at 9:14 AM, Shailendra Mudgal <
mudgal.shailendra@gmail.com> wrote:
> Hi Erik Thanks for you prompt reply.
>
> So if i refresh the searcher in every one hour and that time itself if i
> refresh this cache also, is this going to work? I mean will the document
> ids
> will be in sync for that one hour.
>
>
> On Fri, Apr 18, 2008 at 2:02 AM, Erick Erickson <er...@gmail.com>
> wrote:
>
> > Doc IDs are NOT permanent. If you don't change your index at all
> > (delete especially, but sometimes adding/optimizing can chage IDs)
> > then you can re-use them. Otherwise not.
> >
> >
> > On Thu, Apr 17, 2008 at 1:45 PM, Shailendra Mudgal <
> > mudgal.shailendra@gmail.com> wrote:
> >
> > > Hi All,
> > >
> > > I have a small confusion regarding the document ids which we collect
> > using
> > > HitCollector.collect() method. Here is the description of the
> confusion
> > :
> > >
> > > First i created a FieldCache of type <docID, <list of companyIDs>>
> using
> > a
> > > query which collects all the articles which are only a month old. I am
> > > storing them into a map.
> > >
> > > So now my cache has same document id 's associated with their field
> > value
> > > ready in the memory.
> > >
> > > Now suppose for a search which has duration less then a month, i get a
> > > list
> > > of docIds using hit collector. Can i map these document ids to the
> map,
> > > which i created earlier. And use those field values???
> > >
> > >
> > > Any help will be appreciated ....
> > >
> > > Regards,
> > > Shailendra
> > >
> >
>
Re: Document ids collected from HitCollector.collect and used in FieldCache..
Posted by Shailendra Mudgal <mu...@gmail.com>.
Hi Erik Thanks for you prompt reply.
So if i refresh the searcher in every one hour and that time itself if i
refresh this cache also, is this going to work? I mean will the document ids
will be in sync for that one hour.
On Fri, Apr 18, 2008 at 2:02 AM, Erick Erickson <er...@gmail.com>
wrote:
> Doc IDs are NOT permanent. If you don't change your index at all
> (delete especially, but sometimes adding/optimizing can chage IDs)
> then you can re-use them. Otherwise not.
>
>
> On Thu, Apr 17, 2008 at 1:45 PM, Shailendra Mudgal <
> mudgal.shailendra@gmail.com> wrote:
>
> > Hi All,
> >
> > I have a small confusion regarding the document ids which we collect
> using
> > HitCollector.collect() method. Here is the description of the confusion
> :
> >
> > First i created a FieldCache of type <docID, <list of companyIDs>> using
> a
> > query which collects all the articles which are only a month old. I am
> > storing them into a map.
> >
> > So now my cache has same document id 's associated with their field
> value
> > ready in the memory.
> >
> > Now suppose for a search which has duration less then a month, i get a
> > list
> > of docIds using hit collector. Can i map these document ids to the map,
> > which i created earlier. And use those field values???
> >
> >
> > Any help will be appreciated ....
> >
> > Regards,
> > Shailendra
> >
>
Re: Document ids collected from HitCollector.collect and used in FieldCache..
Posted by Erick Erickson <er...@gmail.com>.
Doc IDs are NOT permanent. If you don't change your index at all
(delete especially, but sometimes adding/optimizing can chage IDs)
then you can re-use them. Otherwise not.
On Thu, Apr 17, 2008 at 1:45 PM, Shailendra Mudgal <
mudgal.shailendra@gmail.com> wrote:
> Hi All,
>
> I have a small confusion regarding the document ids which we collect using
> HitCollector.collect() method. Here is the description of the confusion :
>
> First i created a FieldCache of type <docID, <list of companyIDs>> using a
> query which collects all the articles which are only a month old. I am
> storing them into a map.
>
> So now my cache has same document id 's associated with their field value
> ready in the memory.
>
> Now suppose for a search which has duration less then a month, i get a
> list
> of docIds using hit collector. Can i map these document ids to the map,
> which i created earlier. And use those field values???
>
>
> Any help will be appreciated ....
>
> Regards,
> Shailendra
>