You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Yuval Feinstein <yu...@answers.com> on 2010/02/11 08:34:56 UTC
Do deleted documents affect scores?
I want to focus my previous question.
Say we have two Lucene indexes: A and B.
Index A contains documents a and b.
Index B used to contain documents a, b and c,
But c was deleted.
All documents share some vocabulary.
If we search using terms common to documents b and c,
Can we get a different score for document b in index A and index B?
Note that both indexes are identical with regard to the non-deleted documents,
And only differ by the deleted document c.
Thanks,
Yuval
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: Do deleted documents affect scores?
Posted by Yuval Feinstein <yu...@answers.com>.
Thanks Ian and Andrzej.
You solved a mystery for us.
-- Yuval
________________________________________
From: Andrzej Bialecki [ab@getopt.org]
Sent: Thursday, February 11, 2010 6:53 PM
To: java-user@lucene.apache.org
Subject: Re: Do deleted documents affect scores?
On 2010-02-11 17:35, Ian Lea wrote:
> I'm pretty sure that the answer is no and a quick test on a small
> index with/without deleted docs showed no difference in the scores,
> using 3.0. But that was hardly a rigorous test and I don't know
> enough about lucene internals and scoring to give a definitive answer.
>
> Shouldn't be too hard for you to verify or disprove: build an index
> and throw loads of updates and deletes at it, checking scores as you
> go.
Actually, deleted docs do affect scoring for a time - IDF of a term is
not updated until you optimize (or when Lucene decides to merge segments).
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Do deleted documents affect scores?
Posted by Andrzej Bialecki <ab...@getopt.org>.
On 2010-02-11 17:35, Ian Lea wrote:
> I'm pretty sure that the answer is no and a quick test on a small
> index with/without deleted docs showed no difference in the scores,
> using 3.0. But that was hardly a rigorous test and I don't know
> enough about lucene internals and scoring to give a definitive answer.
>
> Shouldn't be too hard for you to verify or disprove: build an index
> and throw loads of updates and deletes at it, checking scores as you
> go.
Actually, deleted docs do affect scoring for a time - IDF of a term is
not updated until you optimize (or when Lucene decides to merge segments).
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Do deleted documents affect scores?
Posted by Ian Lea <ia...@gmail.com>.
I'm pretty sure that the answer is no and a quick test on a small
index with/without deleted docs showed no difference in the scores,
using 3.0. But that was hardly a rigorous test and I don't know
enough about lucene internals and scoring to give a definitive answer.
Shouldn't be too hard for you to verify or disprove: build an index
and throw loads of updates and deletes at it, checking scores as you
go.
--
Ian.
On Thu, Feb 11, 2010 at 7:34 AM, Yuval Feinstein <yu...@answers.com> wrote:
> I want to focus my previous question.
> Say we have two Lucene indexes: A and B.
> Index A contains documents a and b.
> Index B used to contain documents a, b and c,
> But c was deleted.
> All documents share some vocabulary.
> If we search using terms common to documents b and c,
> Can we get a different score for document b in index A and index B?
> Note that both indexes are identical with regard to the non-deleted documents,
> And only differ by the deleted document c.
> Thanks,
> Yuval
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org