You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Thomas Wöckinger (Jira)" <ji...@apache.org> on 2020/11/04 06:54:00 UTC
[jira] [Commented] (SOLR-14923) Indexing performance is
unacceptable when child documents are involved
[ https://issues.apache.org/jira/browse/SOLR-14923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17225901#comment-17225901 ]
Thomas Wöckinger commented on SOLR-14923:
-----------------------------------------
[~dsmiley] Last investigations on this issue showed that flagging the UpdateLog to open a new RealTimeSearcher (as you mentionend at https://issues.apache.org/jira/browse/SOLR-12638?focusedCommentId=16872898&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16872898) seems to be the best solution.
So i can start working on this, a question in advance: Would it be sufficient to track the document ids which require a reload and clear them on each openRealTimeSearcher call?
> Indexing performance is unacceptable when child documents are involved
> ----------------------------------------------------------------------
>
> Key: SOLR-14923
> URL: https://issues.apache.org/jira/browse/SOLR-14923
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update, UpdateRequestProcessors
> Affects Versions: master (9.0), 8.3, 8.4, 8.5, 8.6
> Reporter: Thomas Wöckinger
> Priority: Critical
> Labels: performance
>
> Parallel indexing does not make sense at moment when child documents are used.
> The org.apache.solr.update.processor.DistributedUpdateProcessor checks at the end of the method doVersionAdd if Ulog caches should be refreshed.
> This check will return true if any child document is included in the AddUpdateCommand.
> If so ulog.openRealtimeSearcher(); is called, this call is very expensive, and executed in a synchronized block of the UpdateLog instance, therefore all other operations on the UpdateLog are blocked too.
> Because every important UpdateLog method (add, delete, ...) is done using a synchronized block almost each operation is blocked.
> This reduces multi threaded index update to a single thread behavior.
> The described behavior is not depending on any option of the UpdateRequest, so it does not make any difference if 'waitFlush', 'waitSearcher' or 'softCommit' is true or false.
> The described behavior makes the usage of ChildDocuments useless, because the performance is unacceptable.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org