[jira] [Commented] (LUCENE-7344) Deletion by query of uncommitted docs not working with DV updates

["Shai Erera (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LUCENE-7344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413216#comment-15413216 ] 

Shai Erera commented on LUCENE-7344:
------------------------------------

Hmm ... I had to refresh my memory of the DV updates code and I agree with [~mikemccand] that the fix is hairy (which goes hand-in-hand with the hairy {{BufferedUpdatesStream}}). The problem is that deleteByQuery uses the existing LeafReader, but the DV updates themselves were not yet applied so the reader is unaware of the change.

I changed the test to call {{updateDocument}} instead of updating the NDV and the test passes. This is expected because updating a document deletes the old one and adds a new document. So when DBQ is processed, a LeafReader is opened on the new segment (with the new document; it has to work that way cause the new document isn't yet flushed) and the new segment thus has the new document with the updated NDV.

I agree this is a bug *only* because updating a document followed by DBQ works as expected. The internals of how in-place updates are applied should not concern the user.

I wonder if we need to implement a complex merge-sorting approach as [~mikemccand] proposes, or if we applied the DV updates before processing and DBQ would be enough (ignoring the adversarial affects that Mike describes; they're true, but I ignore them for the moment). I want to try that.

If that works, then perhaps we can detect if a DBQ involves an NDV field (or BDV field for that matter) and refresh the reader only then, or refresh the reader whenever there are DBQ and any DV updates, even if they are unrelated. But first I want to try and make the test pass, before we decide on how to properly fix it.

> Deletion by query of uncommitted docs not working with DV updates
> -----------------------------------------------------------------
>
>                 Key: LUCENE-7344
>                 URL: https://issues.apache.org/jira/browse/LUCENE-7344
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Ishan Chattopadhyaya
>         Attachments: LUCENE-7344.patch
>
>
> When DVs are updated, delete by query doesn't work with the updated DV value.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org