Terms Component with documents marked for deletion

[qwamci <ma...@qwamci.com>]

I have been playing around with Terms Component in solr and hit a situation i
do not understand.

When indexing documents and then updating them the termscomponent does not
always have the correct count. In specific when updating a document, the
termscomponent keeps a track of the former version of the document it then
has a extra count.

I understand, and was told that this a corrct behavior , that the terms
component keeps track of doucments marked for deletion but not yet deleted.
However what i fail to understand is hox to force the actual deletion of
these docs of the merge of the indexes. 
I flush and commit my changes  however thos does not suffice to get the
termscomponent to get the correct count.
I have also set the merge factor as low as possible (aka 2) to try and get a
merge a soon as possible however that did not do it either.
I also lowered the maxPendingDeletes which did not do the trick.

If someone has a better understanding of the problem i would love an
explanation, or to see what i am missing.



for example ot the situation
example indexing two documents with a country field as such:

<add>
<doc>
<field name="COUNTRY">US</field>
<field name="ID">L20110121151204207</field>
</doc>
<doc>
<field name="COUNTRY">Canada</field>
<field name="ID">L20110121151204208</field>
</doc>
</add>

Termscomponent returns:
US(1)
Canada(1)

Update the first document:

<add>
<doc>
<field name="COUNTRY">US</field>
<field name="ID">L20110121151204207</field>
</doc>
</add>

Termscomponent returns:
US(2)
Canada(1)

where there is only one US.

--
View this message in context: http://lucene.472066.n3.nabble.com/Terms-Component-with-documents-marked-for-deletion-tp3548271p3548271.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Terms Component with documents marked for deletion

[lboutros <bo...@gmail.com>]

Hi,

you have to use the 'expungeDeletes' additional parameter:

http://wiki.apache.org/solr/UpdateXmlMessages

and depending on the version of Solr you are using, you perhaps have to use
a merge policy like the LogByteSizeMergePolicy.

See : https://issues.apache.org/jira/browse/SOLR-2725

Ludovic. 



-----
Jouve
France.
--
View this message in context: http://lucene.472066.n3.nabble.com/Terms-Component-with-documents-marked-for-deletion-tp3548271p3548569.html
Sent from the Solr - User mailing list archive at Nabble.com.