You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Atri Sharma (JIRA)" <ji...@apache.org> on 2019/04/22 18:03:00 UTC
[jira] [Commented] (LUCENE-8757) Better Segment To Thread Mapping
Algorithm
[ https://issues.apache.org/jira/browse/LUCENE-8757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16823295#comment-16823295 ]
Atri Sharma commented on LUCENE-8757:
-------------------------------------
Attached is a first cut for the patch. The main idea there is that smaller segments are coalesced into single threads with a hard cap on the number of segments per thread to avoid overwhelming a single thread.
This can be enhanced with taking number of cores on the machine and current CPU utilization, but that might lead to a higher IndexSearcher bootstrap time.
Comments and thoughts are welcome.
> Better Segment To Thread Mapping Algorithm
> ------------------------------------------
>
> Key: LUCENE-8757
> URL: https://issues.apache.org/jira/browse/LUCENE-8757
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Atri Sharma
> Priority: Major
> Attachments: LUCENE-8757.patch
>
>
> The current segments to threads allocation algorithm always allocates one thread per segment. This is detrimental to performance in case of skew in segment sizes since small segments also get their dedicated thread. This can lead to performance degradation due to context switching overheads.
>
> A better algorithm which is cognizant of size skew would have better performance for realistic scenarios
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org