You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Cao Manh Dat (Jira)" <ji...@apache.org> on 2020/04/28 01:41:00 UTC
[jira] [Commented] (SOLR-12642) SolrCmdDistributor should send
updates in batch when use Http2SolrClient?
[ https://issues.apache.org/jira/browse/SOLR-12642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094072#comment-17094072 ]
Cao Manh Dat commented on SOLR-12642:
-------------------------------------
Above numbers are not correct. Basically above benchmark was performed on a buggy Jetty version. During indexing, leader failed to send updates to replicas quite frequently without networking issue, therefore replicas were put into recovery mode. Replicas did not do indexing much so that result the gain in CPU time and garbage generated. On Leader's side the CPU time is 50% more since it need to stream index files back to replicas during recovery.
> SolrCmdDistributor should send updates in batch when use Http2SolrClient?
> -------------------------------------------------------------------------
>
> Key: SOLR-12642
> URL: https://issues.apache.org/jira/browse/SOLR-12642
> Project: Solr
> Issue Type: Improvement
> Reporter: Cao Manh Dat
> Assignee: Cao Manh Dat
> Priority: Major
> Fix For: 8.0
>
> Attachments: http2-branch.log, master-branch.log
>
>
> In the past, batch updates are sent in a single stream from the leader, the replica will create a single thread to parse all the updates. For the simplicity of {{SOLR-12605}}, the leader is now sending individual updates to replicas, therefore they are now parsing updates in different threads which increase the usage of memory and CPU.
> In the past, this is an unacceptable approach, because, for every update, we must create different connections to replicas. But with the support of HTTP/2, all updates will be sent in a single connection from leader to a replica. Therefore the cost is not as high as it used to be.
> On the other hand, sending individual updates will improve the indexing performance and better error-handling for failures of a single update in a batch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org