You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by "David Smiley (Jira)" <ji...@apache.org> on 2022/10/16 05:23:00 UTC
[jira] [Commented] (SOLR-16348) New SplitShard UpdateRequestProcessor
[ https://issues.apache.org/jira/browse/SOLR-16348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17618201#comment-17618201 ]
David Smiley commented on SOLR-16348:
-------------------------------------
bq. An alternative to an URP is a SolrEventListener implementing postCommit(). But in some bulk loading scenarios, a commit doesn't happen until the end which would be way too late.
In SolrCloud (from which this functionality requires), there is always going to be a hard commit on regular intervals for keeping the updateLog in check. We'll explore this a bit.
Also something internally discussed was the possibility of this thing (be it an URP or commit event listener) issuing an expungeDeletes if the ratio of deletes documents is very high (think after a split with method=link). It's befitting of this thing to do both as they are highly related -- ensuring segments don't get too big.
> New SplitShard UpdateRequestProcessor
> -------------------------------------
>
> Key: SOLR-16348
> URL: https://issues.apache.org/jira/browse/SOLR-16348
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: UpdateRequestProcessors
> Reporter: David Smiley
> Priority: Major
>
> The [SplitShard|https://solr.apache.org/guide/solr/latest/deployment-guide/shard-management.html#splitshard] command is used to split a shard into smaller shards to get better query scalability, especially across multiple machines. The most practical way to use it is to split shards larger than a configured size. Of course shards don't just grow by themselves; they grow when data is added. Here I propose a new UpdateRequestProcessor that splits based on the shard size.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org