You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Tate Avery <ta...@nstein.com> on 2003/11/17 18:22:38 UTC
Which operations change document ids?
Hello,
I am considering using the document id in order to implement a fast 'join' during relational search.
My first question is: should I steer clear of this all together? And why? If not, I need to know which Lucene operations can cause document ids to change.
I am assuming that the following can cause potential changes:
1) Add document
- since it might trigger a merge
2) Optimize index
- since it does trigger a merge
3) Update document
- since it is a delete + add
What else could cause a document id to change? Could delete provoke a doc id change?
And, I am assuming that the following DO NOT change the document id:
1) Query the index
Also, am I missing any others that will or will not cause a document id to change?
Thank you,
Tate
P.S. It appears (to me) that the SearchBean (in lucene sandbox) sorting makes use of the Hits.id(int _n) method. How does it cope, if at all, with changes to the underlying document ids?
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Re: Which operations change document ids?
Posted by Doug Cutting <cu...@lucene.com>.
If they're optimized at different times then the document ids could get
out of sync, as the optimized version will have deleted documents
removed, while the un-optimized one won't.
Also, for add/delete to keep document ids in sync you need to also be
sure to use the same mergeFactor.
Doug
Jamie Stallwood wrote:
> If you create two parallel indices (to use different parsing methods for
> instance), and always add and delete documents in parallel, will the
> document ID's always correspond in both indices? And could optimization
> destroy any such invariance?
>
>
>
> -----Original Message-----
> From: Doug Cutting [mailto:cutting@lucene.com]
> Sent: 17 November 2003 19:51
> To: Lucene Users List
> Subject: Re: Which operations change document ids?
>
> Tate Avery wrote:
>
>>My first question is: should I steer clear of this all together?
>
>
> No, I think this is appropriate.
>
>
>>If not, I need to know which Lucene operations can cause document ids to
>
> change.
>
>>I am assuming that the following can cause potential changes:
>> 1) Add document
>> 2) Optimize index
>>
>>What else could cause a document id to change?
>
>
> Nothing. And even these can only cause an id to change if there have
> been deletions.
>
>
>>Could delete provoke a doc id change?
>
>
> Not when you perform the delete. Later, when you add to or optimize the
> index, the ids for deleted documents are reclaimed.
>
>
>>And, I am assuming that the following DO NOT change the document id:
>>
>> 1) Query the index
>
>
> That is correct.
>
> Document ids never change with an instance of IndexReader. When you
> open a new index reader you should usually assume that ids have changed.
>
> Doug
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
RE: Which operations change document ids?
Posted by Jamie Stallwood <ju...@project76.co.uk>.
If you create two parallel indices (to use different parsing methods for
instance), and always add and delete documents in parallel, will the
document ID's always correspond in both indices? And could optimization
destroy any such invariance?
-----Original Message-----
From: Doug Cutting [mailto:cutting@lucene.com]
Sent: 17 November 2003 19:51
To: Lucene Users List
Subject: Re: Which operations change document ids?
Tate Avery wrote:
> My first question is: should I steer clear of this all together?
No, I think this is appropriate.
> If not, I need to know which Lucene operations can cause document ids to
change.
>
> I am assuming that the following can cause potential changes:
> 1) Add document
> 2) Optimize index
>
> What else could cause a document id to change?
Nothing. And even these can only cause an id to change if there have
been deletions.
> Could delete provoke a doc id change?
Not when you perform the delete. Later, when you add to or optimize the
index, the ids for deleted documents are reclaimed.
> And, I am assuming that the following DO NOT change the document id:
>
> 1) Query the index
That is correct.
Document ids never change with an instance of IndexReader. When you
open a new index reader you should usually assume that ids have changed.
Doug
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Re: Which operations change document ids?
Posted by Doug Cutting <cu...@lucene.com>.
Tate Avery wrote:
> My first question is: should I steer clear of this all together?
No, I think this is appropriate.
> If not, I need to know which Lucene operations can cause document ids to change.
>
> I am assuming that the following can cause potential changes:
> 1) Add document
> 2) Optimize index
>
> What else could cause a document id to change?
Nothing. And even these can only cause an id to change if there have
been deletions.
> Could delete provoke a doc id change?
Not when you perform the delete. Later, when you add to or optimize the
index, the ids for deleted documents are reclaimed.
> And, I am assuming that the following DO NOT change the document id:
>
> 1) Query the index
That is correct.
Document ids never change with an instance of IndexReader. When you
open a new index reader you should usually assume that ids have changed.
Doug
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org