You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by "Bruno Roustant (Jira)" <ji...@apache.org> on 2022/09/26 15:59:00 UTC
[jira] [Comment Edited] (SOLR-16348) New SplitShard UpdateRequestProcessor
[ https://issues.apache.org/jira/browse/SOLR-16348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17609560#comment-17609560 ]
Bruno Roustant edited comment on SOLR-16348 at 9/26/22 3:58 PM:
----------------------------------------------------------------
The first implementation with a skew factor seems good to decide if we need more complexity. We would evaluate it with some synthetic or real cases.
With this URP approach, is it possible to trigger multiple times the same split of the same shard?
Actually this is probably not an issue since there is an existing split lock (SplitShardCmd.lockForSplit()/unlockForSplit()) that prevents a split on the same shard.
Should we limit the number of concurrent splits per host on the SplitShardCmd? For example could we have the static semaphore mentioned above inside the lockForSplit()?
The skew scale probably depends on the load of the indexing flow, and maybe on the number of shards? It might be hard to decide. But it's too early to guess, we need some evaluation.
was (Author: broustant):
The first implementation with a skew factor seems good to decide if we need more complexity. We would evaluate it with some synthetic or real cases.
With this URP approach, is it possible to trigger multiple times the same split of the same shard?
Actually this is probably not an issue since there is an existing split lock (SplitShardCmd.lockForSplit()/unlockForSplit()) that prevents a split on the same shard.
Should we limit the number of concurrent splits per host on the SplitShardCmd
The skew scale probably depends on the load of the indexing flow, and maybe on the number of shards? It might be hard to decide. But it's too early to guess, we need some evaluation.
> New SplitShard UpdateRequestProcessor
> -------------------------------------
>
> Key: SOLR-16348
> URL: https://issues.apache.org/jira/browse/SOLR-16348
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: UpdateRequestProcessors
> Reporter: David Smiley
> Priority: Major
>
> The [SplitShard|https://solr.apache.org/guide/solr/latest/deployment-guide/shard-management.html#splitshard] command is used to split a shard into smaller shards to get better query scalability, especially across multiple machines. The most practical way to use it is to split shards larger than a configured size. Of course shards don't just grow by themselves; they grow when data is added. Here I propose a new UpdateRequestProcessor that splits based on the shard size.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org