You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Andy Vuong (Jira)" <ji...@apache.org> on 2019/12/20 22:54:00 UTC
[jira] [Comment Edited] (SOLR-14044) Support shard/collection
deletion in shared storage
[ https://issues.apache.org/jira/browse/SOLR-14044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001263#comment-17001263 ]
Andy Vuong edited comment on SOLR-14044 at 12/20/19 10:53 PM:
--------------------------------------------------------------
After some discussion, the shared core concurrency cache eviction can fall under two strategies
# Distributed clean up which was what the initial comments above were proposing i.e. hooking into the same path of the blob deletion logic
# Time-based eviction and lazy eviction from the cache
The second strategy allows us to design the deletion path independently and simplifies the code. Time-based eviction will support a larger scale if we need to evict for thousands of collections that don't get indexing/querying traffic often. Lazy eviction will support cases such as collection name reuse in which a collection is deleted and soon after recreated. I'll likely separate the eviction into its own Jira -
EDIT: Linked https://issues.apache.org/jira/browse/SOLR-14134 to this since the discussion that initiated that JIRA's creation started here.
was (Author: andy_vuong):
After some discussion, the shared core concurrency cache eviction can fall under two strategies
# Distributed clean up which was what the initial comments above were proposing i.e. hooking into the same path of the blob deletion logic
# Time-based eviction and lazy eviction from the cache
The second strategy allows us to design the deletion path independently and simplifies the code. Time-based eviction will support a larger scale if we need to evict for thousands of collections that don't get indexing/querying traffic often. Lazy eviction will support cases such as collection name reuse in which a collection is deleted and soon after recreated. I'll likely separate the eviction into its own JIRA.
> Support shard/collection deletion in shared storage
> ---------------------------------------------------
>
> Key: SOLR-14044
> URL: https://issues.apache.org/jira/browse/SOLR-14044
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
> Reporter: Andy Vuong
> Priority: Major
>
> The Solr Cloud deletion APIs for collections and shards are not currently supported by shared storage but are an essential functionality required by the shared storage design. Deletion of objects from shared storage currently only happens in the indexing path (on pushes) and after the index file listings between the local solr process and external store have been resolved.
>
> This task is to track supporting the delete shard/collection API commands and its scope does not include cleaning up so called “orphaned” index files from blob (i.e. files that are no longer referenced by any core.metadata file on the external store). This will be designed/covered in another subtask.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org