[GitHub] [arrow-rs] tustvold commented on a diff in pull request #4060: feat: support bulk deletes in object_store

["tustvold (via GitHub)" <gi...@apache.org>]

tustvold commented on code in PR #4060:
URL: https://github.com/apache/arrow-rs/pull/4060#discussion_r1163643919


##########
object_store/src/lib.rs:
##########
@@ -367,6 +367,31 @@ pub trait ObjectStore: std::fmt::Display + Send + Sync + Debug + 'static {
     /// Delete the object at the specified location.
     async fn delete(&self, location: &Path) -> Result<()>;
 
+    /// Delete all the objects at the specified locations
+    ///
+    /// When supported, this method will use bulk operations that delete more
+    /// than one object at a time. The maximum number of objects deleted per call
+    /// is provided by [`DELETE_BATCH_SIZE`]. You can use that parameter to wrap
+    /// this method to customize the parallelism or provide a progress indicator.
+    ///
+    /// This method may create multiple threads to perform the deletions in parallel.
+    async fn delete_all(&self, locations: Vec<Path>) -> Result<()> {

Review Comment:
   What do you think about making this instead something like
   
   ```suggestion
       async fn delete_all(&self, locations: Vec<Path>) -> Result<BoxStream<'_, Result<()>>>  {
   ```
   
   This would allow for granular error and progress reporting? One could even go so far as to make the input also a `BoxStream` :thinking: 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org