[GitHub] [arrow-rs] alamb commented on a diff in pull request #3791: Add ObjectStore::append

["alamb (via GitHub)" <gi...@apache.org>]

alamb commented on code in PR #3791:
URL: https://github.com/apache/arrow-rs/pull/3791#discussion_r1123428450


##########
object_store/src/lib.rs:
##########
@@ -306,6 +312,33 @@ pub trait ObjectStore: std::fmt::Display + Send + Sync + Debug + 'static {
         multipart_id: &MultipartId,
     ) -> Result<()>;
 
+    /// Returns an [`AsyncWrite`] that can be used to append to the object at `location`
+    ///
+    /// A new object will be created if it doesn't already exist, otherwise it will be
+    /// opened, with subsequent writes appended to the end.
+    ///
+    /// This operation cannot be supported by all stores, most use-cases should prefer
+    /// [`ObjectStore::put`] and [`ObjectStore::put_multipart`] for better portability
+    /// and stronger guarantees
+    ///
+    /// This API is not guaranteed to be atomic, in particular
+    ///
+    /// * On error, `location` may contain partial data
+    /// * Concurrent calls to [`ObjectStore::list`] may return partially written objects
+    /// * Concurrent calls to [`ObjectStore::get`] may return partially written data
+    /// * Concurrent calls to [`ObjectStore::put`] may result in data loss / corruption
+    /// * Concurrent calls to [`ObjectStore::append`] may result in data loss / corruption
+    ///
+    /// Additionally some stores, such as Azure, may only support appending to objects created

Review Comment:
   👍 the docstrings make a lot of sense -- thank you for spelling it out so clearly



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org