[GitHub] [lucene-solr] andyvuong commented on a change in pull request #1188: SOLR-14044: Support collection and shard deletion in shared storage

[GitBox <gi...@apache.org>]

andyvuong commented on a change in pull request #1188: SOLR-14044: Support collection and shard deletion in shared storage
URL: https://github.com/apache/lucene-solr/pull/1188#discussion_r375441526
 
 

 ##########
 File path: solr/core/src/java/org/apache/solr/cloud/api/collections/DeleteCollectionCmd.java
 ##########
 @@ -142,6 +148,34 @@ public void call(ClusterState state, ZkNodeProps message, NamedList results) thr
           break;
         }
       }
+      
+      // Delete the collection files from shared store. We want to delete all of the files before we delete
+      // the collection state from ZooKeeper.
+      DocCollection docCollection = zkStateReader.getClusterState().getCollectionOrNull(collection);
+      if (docCollection != null && docCollection.getSharedIndex()) {
+        SharedStoreManager sharedStoreManager = ocmh.overseer.getCoreContainer().getSharedStoreManager();
+        BlobDeleteManager deleteManager = sharedStoreManager.getBlobDeleteManager();
+        BlobDeleteProcessor deleteProcessor = deleteManager.getOverseerDeleteProcessor();
+        // deletes all files belonging to this collection
+        CompletableFuture<BlobDeleterTaskResult> deleteFuture = 
+            deleteProcessor.deleteCollection(collection, false);
+        
+        try {
+          // TODO: Find a reasonable timeout value
+          BlobDeleterTaskResult result = deleteFuture.get(60, TimeUnit.SECONDS);
+          if (!result.isSuccess()) {
+            log.warn("Deleting all files belonging to shared collection " + collection + 
+                " was not successful! Files belonging to this collection may be orphaned.");
+          }
+        } catch (TimeoutException tex) {
+          // We can orphan files here if we don't delete everything in time but what matters for potentially
+          // reusing the collection name is that the zookeeper state of the collection gets deleted which 
+          // will happen in the finally block
+          throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Could not complete deleting collection" + 
 
 Review comment:
   I throw an exception in both cases so the client calling is aware if the command fails and files are orphaned. In DeleteCollection the files are "truly orphaned" because even if we error out here, the collection will always be deleted from zookeeper in the finally block and it's effectively gone from Solr's perspective. The DeleteShard will fail the whole command without doing the same delete from zookeeper action and a subsequent delete shard command can be called to try again which isn't the case in the former (unless the same collection name is created again).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org