[jira] [Commented] (LUCENE-7049) merge eats CPU much when there are many deleteByQuery

["Shawn Heisey (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LUCENE-7049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15167204#comment-15167204 ] 

Shawn Heisey commented on LUCENE-7049:
--------------------------------------

I have noticed that deleteByQuery in Solr will be held up by a merge.  That roadblock probably also stands in the way of any further indexing requests made after the deleteByQuery.  I would not be overly surprised to learn that it makes the CPU spin, but I have not actually checked this.

Adds and deletes by ID can happen at the same time as a merge since 4.0, but deleteByQuery apparently works differently, so it is blocked by any merge activity.  I recently changed my indexing program so that it turns deleteByQuery into a query with fl=unique_key_field, and then does the ID delete with the results.  I had gotten rid of all the "optimizeUnderway" checking that I had added back in the 3.x days, and didn't want to add that back in, so I "fixed" my delete code.

> merge eats CPU much when there are many deleteByQuery 
> ------------------------------------------------------
>
>                 Key: LUCENE-7049
>                 URL: https://issues.apache.org/jira/browse/LUCENE-7049
>             Project: Lucene - Core
>          Issue Type: Improvement
>          Components: core/index, core/search
>            Reporter: Mikhail Khludnev
>            Priority: Minor
>         Attachments: Selection_114.png, Selection_115.png, Selection_116.png
>
>
> h2. When
> adding very many delete><query> 
> h2. Then
> we got CPU spike in merge thread that blocks indexing process
>  
> h2. Considerations
> Despite adding too many <delete><query> is odd itself, I suppose [the code can more efficient|https://issues.apache.org/jira/browse/LUCENE-5666?focusedCommentId=15129040&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15129040]. See sampling snapshots attached. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org